Piecewise Linear Optimal Controllers for Hybrid Systems Alberto Bemporad, Francesco Borrelli, Manfred Morari Automatic Control Laboratory ETH Zentrum, ETL I 6 CH-9 Zurich, Switzerland bemporad,borrelli,morari@aut.ee.ethz.ch Abstract In this paper we propose a procedure for synthesizing piecewise linear optimal controllers for discrete-time hybrid systems. A stabilizing controller is obtained by designing a model predictive controller (MPC), which is based on the minimization of a weighted `=-norm of the tracking error and the input trajectories over a finite horizon. The control law is obtained by solving a multiparametric mixed-integer linear program (mp- MILP), which avoids solving mixed-integer programs on-line [6]. As the resulting control law is piecewise affine, on-line computation is drastically reduced to a simple linear function evaluation. Introduction Hybrid systems provide a unified framework for describing processes evolving according to continuous dynamics, discrete dynamics, and logic rules [,,,]. The interest in hybrid systems is mainly motivated by the large variety of practical situations, for instance real-time systems, where physical processes interact with digital controllers. Several modeling formalisms have been developed to describe hybrid systems, and among others, Bemporad and Morari [6] introduced a new class of hybrid systems called Mixed Logical Dynamical (MLD) systems. These model a broad class of systems arising in many applications: linear hybrid dynamical systems, hybrid automata, nonlinear dynamic systems where the nonlinearity can be approximated by a piecewise linear function, some classes of discrete event systems, linear systems with constraints, etc. Examples of real-world applications that can be naturally modeled within the MLD frameworkare reported in [5, 6, ]. In [6], Bemporad and Morari propose Model Predictive Control (MPC) as a general approach to control hybrid systems. MPC has been widely adopted in industry to solve control problems of systems subject to input and output constraints. MPC is based on the so called receding horizon philosophy: a sequence of future control actions is chosen according to a prediction of the future evolution of the system and applied to the plant until new measurements are available. Then, a new sequence is established which replaces the previous one. Each sequence is determined by means of an optimization procedure which takes into account two objectives: optimize the tracking performance, and protect the system from possible constraint violations. When the model of the system is a hybrid MLD model and the performance index is quadratic, the optimization problem is a Mixed-Integer Quadratic Programming (MIQP) problem. Similarly, -norm/-norm performance indexes lead to Mixed-Integer Linear Programming (MILP) problems. By appropriately defining the concepts of equilibrium and stability for MLD systems, and by using Lyapunov arguments, it can be proven [6] that model predictive control laws stabilize MLD systems, as well as provide tracking of reference trajectories. So far, the main drawbackof such a control approach has been its intensive on-line computation requirement. Although efficient branch and bound algorithms exist to solve MIQP/MILP, these are known to be NP-hard problems. In this paper we propose a different approach where all computation is moved off line, by generalizing to hybrid systems the result of [] for linear systems. The idea stems from observing that the linear part of the objective and the rhs of the constraints in the optimization problem depend linearly on the state vector x(t). This can be considered as a vector of parameters, and for -norm/-norm performance indexes, the optimization problem can be treated as a Multiparametric MILP (mp-milp). In this paper we consider an algorithm to efficiently solve mp-milps, and show that the solution is a piecewise linear function of the parameters. In other words, the proposed model predictive controllerforhybrid MLD systems, besides being stabilizing and optimal with respect to an -norm/-norm performance design criterion, is also a piecewise linear controller. Therefore on-line computation reduces to a simple linear function evaluation, instead of an expensive mixed-integer linear program. The basic ideas of
the approach are illustrated with an example of controller synthesis for a piecewise affine system. Model Predictive Control of MLD Systems Consider the mixed logical dynamical (MLD) system described by the relations x(t +)=Ax(t)+B u(t)+b ffi(t)+b z(t) (a) y(t) =Cx(t)+D u(t)+d ffi(t)+d z(t) (b) E ffi(t)+e z(t)» E u(t)+e x(t)+e 5 (c) where x R nc f; g n` is a vector of continuous and binary states, u R mc f; g m` are the inputs, y R pc f; g p` the outputs, ffi f; g r`, z r R c represent auxiliary binary and continuous variables respectively, which are introduced when transforming logic relations into mixed-integer linear inequalities [6], and A, B, B, B, C, D, D, D, E,...,E 5 are matrices of suitable dimensions. It is interesting from both a theoretical and practical point of view to askwhether or not an MLD system can be stabilized to an equilibrium state or can tracka desired reference trajectory, possibly via feedbackcontrol. Despite the fact that the system is neither linear nor even smooth, we show in this section how model predictive control (MPC) provides successful tools to perform this task. As recalled above, the main idea of MPC is to use the model of the plant topredict the future evolution of the system, and based on this prediction to optimize a certain performance index under operating constraints to generate the control action. Only the first sample of the optimal sequence is actually applied to the plant attimet. At +, a new sequence is evaluated to replace the previous one. This on-line re-planning"provides the desired feedbackcontrol feature. Consider an equilibrium pair (x e ;u e ) and let (ffi e ;z e )be a corresponding equilibrium pair of auxiliary variables. Let t be the current time, and x(t) the current state. Consider the following optimal control problem X min J(v T fv T ;x(t)), T kq (v(k) u e)k+ g subj. to k= kq (ffi(kjt) ffi e)k + kq (z(kjt) z e)k+ kq (x(kjt) x e)k + kq 5(y(kjt) y e)k () x(t jt) = x e x(k +jt) = Ax(kjt)+B v(k)+b ffi(kjt)+ B z(kjt) y(kjt) = Cx(kjt)+D v(k)+d ffi(kjt)+ D z(kjt) () E ffi(kjt) + E z(kjt)» E v(k)+e x(kjt)+ E 5 u min» v(t + k)» u max; k =; ;:::;T x min» x(t + kjt)» x max; k =;:::;N c where T and N c» T are the prediction and state constraint horizons, respectively, Q,...,Q 5, are nonsingular weighting matrices, x(kjt) is the state predicted at + k by applying the input u(t + k) = v(k) to () from x(jt) = x(t), u min, u max and x min, x max are hard bounds on the inputs and on the states, respectively. Assume for the moment that the optimal solution fv Λ t (k)g k=;:::;t exists. According to the receding horizon philosophy mentioned above, we set u(t) =v Λ t (); () disregard the subsequent optimal inputs vt Λ ();:::;vt Λ (T ), and repeat the whole optimization procedure at +. The control law ()() will be referred to as the Hybrid MPC law, and represent the counterpart in infinity norm of the Hybrid MPC law proposed in [6]. Note that once x e, u e have been fixed, consistent steady-state vectors ffi e, z e can be obtained by choosing feasible points in the domain described by (c), for instance by solving a MILP. In the next section we will show how to formulate the problem ()-() as a mixed integer linear program (MILP). Several formulations of predictive controllers for MLD systems might be proposed. For instance, the number of control degrees of freedom can be reduced to N u < T,by setting u(k) u(n u ), k = N u ;:::;T. However, while in other contexts this amounts to hugely down-sizing the optimization problem at the price of a reduced performance, here the computational gain is only partial, since all the T ffi(kjt) andz(kjt) variables remain in the optimization. Infinite horizon formulations are inappropriate for both practical and theoretical reasons. In fact, approximating the infinite horizon with a large T is computationally prohibitive, as the number of - variables involved in the MILP, aswill be shown later, depends exponentially on T. Moreover, the quadratic term in ffi might oscillate and hence good" (i.e. asymptotically stabilizing) input sequences might be ruled out by a corresponding infinite value of the performance index; it could even happen that no input sequence has finite cost. The following theorem shows that the control law () () stabilizes system () asymptotically Theorem Let (x e ;u e ) be an equilibrium pair and (ffi e ;z e ) definitely admissible. Assume that the initial state x() is such that a feasible solution ofproblem () exists at =. Then for all non singular Q and
Q, the MIPC law ()() stabilizes the system in that t! = x e t! = u e t! kq (ffi(t) ffi e )k = t! kq (z(t) z e )k = t! kq 5(y(t) y e )k = while fulfilling the dynamic/relational constraints (c) and the input and state constraints u min» u(t)» u max, x min» x(t)» max. Note that if Q is non singular (or Q, Q 5 ), convergence of ffi(t) (orz(t), y(t)) follows as well. Proof: The proof easily follows from standard Lyapunov arguments. Let Ut Λ denote the optimal control sequence fvt Λ ();:::;vt Λ (T )g, let V (t), J(U Λ t ;x(t)) denote the corresponding value attained by the performance index, and let U be the sequence fv Λ t ();:::;v Λ t (T );u e g. Then, U is feasible at time t +, along with the vectors ffi(kjt +) = ffi(k +jt), z(kjt+) = z(k +jt), k =;:::;T, ffi(t jt+) = ffi e, z(t jt+) = z e,beingx(t jt+) = x(t jt) =x e and (ffi e ;z e ) definitely admissible. Hence, V (t +)» J(U ;x(t + )) = V (t) kq (x(t) x e )k + kq (u(t) u e )k kq (ffi(t) ffi e )k + kq (z(t) z e )k kq 5 (y(t) y e )k (5) and V (t) is decreasing. Since V (t) is lower-bounded by, there exists V = t! V (t), which implies V (t +) V (t)!. Therefore, each termofthesum kq (x(t) x e )k + kq (u(t) u e )k + kq (ffi(t) ffi e )k + kq (z(t) z e )k + kq 5 (y(t) y e )k» (6) V (t) V (t +) converges to zero as well, which proves the theorem which proves the theorem as Q ;:::;Q 5 are nonsingular. The end point constraint x(t jt) = x e can be relaxed by weighting the final state. Moreover from a theoretical point of view, it is not clear, in general, how to reformulate an infinite dimensional problem as a finite dimensional one for a MLD system, as can instead be done for linear systems through Lyapunov or Riccati algebraic equations. The MPC formulation ()-() can be equivalently rewritten as a mixed-integer linear program, by using the following standard approach. The sum of the components of any vector f" u ;:::;"u T ;"ffi ;:::;"ffi T ;"z ;:::;"z T ;"x ;:::;"x T ; " y ;:::;"y T g that satisfies m " u k» Q u(kjt) u e k =; ;:::;T m " u k»q u(kjt) u e k =; ;:::;T r`" ffi k» Q ffi(kjt) ffi e k =; ;:::;T r`" ffi k»q ffi(kjt) ffi e k =; ;:::;T rc " z k» Q z(kjt) z e k =; ;:::;T rc " z k»q z(kjt) z e k =; ;:::;T n " x k» Q x(kjt) x e k =; ;:::;T n " x k»q x(kjt) x e k =; ;:::;T p " y k» Q 5y(kjt) y e k =; ;:::;T p " y k»q 5y(kjt) y e k =; ;:::;T () represents an upper bound on J(v T ;x(t)), where k is a column vector of ones of length equal to k, and where X k x kjt = A k x(t)+ A j (B u(k jjt) + j= B ffi(k jjt) +B z(k jjt)) () Similarly to what was shown in [], it is easy to prove that the vector p, f" u ;:::;"u T ;"ffi ;:::;"ffi T ;"z ;:::;"z T ; " x ;:::;"x T ;"y ;:::;"y T g that satisfies equations () and simultaneously minimizes J(p) = " u + :::+ " u T + " ffi + :::+ " ffi T + " z + :::+ " z T + " x o + :::+ " x T + " y + :::+ "y T (9) Piecewise Linear Solution of MPC In the previous section we have defined an optimal receding horizon control law for MLD systems. also solves the original problem, i.e. the same optimum J Λ (v T ;x(t)) is achieved. Therefore, problem ()-()
can be reformulated as the following MILP problem min p J(p) s.t. m" u» ±Q k (v e + v(kjt)) k =; ;:::;T m" ffi» ±Q k (ffi e + ffi(kjt)) k =; ;:::;T m" z» ±Q k (z e + z(kjt)) k =; ;:::;T n" ±Q k (x e + A k x(jt)+ P k j= Aj (B v(k jjt)+ B ffi(k jjt)+b v(k jjt))) k =;:::;T n" y» ±Q k 5 (y e + CA k x(jt)+ k C j= Aj (B v(k jjt)+ B ffi(k jjt)+b v(k jjt))) +D v(k)+d ffi(kjt)+d z(kjt) k =;:::;T x min» A k x(jt)+ P k j= Aj (B v(k jjt)+b ffi(k jjt) +B v(k jjt))» x max; k =;:::;N c u min» v kjt» u max; k =; ;:::;T x(t jt) = x e x(k +jt) = Ax(kjt)+B v(k)+ B ffi(kjt)+b z(kjt); k y(kjt) = Cx(kjt)+D v(k)+ D ffi(kjt)+d z(kjt) E ffi(kjt) + E z(kjt)» E v(k)+ E x(kjt)+e 5 ; k () where the variable x(kjt) appears only in the constraints in () as a vector parameter. Problem () can be rewritten in the more compact form min p subj. to l(p c ;p d ;ο(t)) = f T c p c + f T d p d G c p c + G c p d» S + Fο(t) () Multiparametric-MILP Solvers Two main approaches have been proposed for solving mp-milp problems. In [], the authors develop an algorithm based on branch and bound (B&B) methods. At each node of the B&B tree an mp-lp is solved. The solution at the root node representavalid lower bound, while the solution at a node where all the integer variables have been fixed represents a valid upper bound. As in standard B&B methods, the complete enumeration of combinations of - integer variables is avoided by comparing the multiparametric solutions, and by fathoming the nodes where there is no improvement of the value function. In [] an alternative algorithm was proposed, which instead of solving mp-lp problems with integer variables relaxed in the interval [; ], only solves mp-lps where the integer variables are fixed to the optimal value determined by an MILP. More in detail, problem () is alternatively decomposed into an mp-lp and an MILP subproblem. When the values of the binary variable are fixed, an mp-lp is solved, and its solution provides a parametric upper bound. On the other hand, when the parameters in ο(t) are treated as free variables, an MILP is solved, which provides a new integer vector (see [] for more details). The algorithmic implementation of the mp-milp algorithm adopted in this paper relies on [9] for solving mp-lp problems, and on [] for solving MILP's. where ο(t) =[x (t) ff(t )], the matrices G, S, F can be straightforwardly defined from (), and p c, p d represent continuous and discrete variables, respectively. The MILP problem () depends on the current value of ο(t), and needs to be solved in order to compute the command input. Rather than solving the MILP on line, we follow the ideas of [, ], and propose an approach where all computation is moved off line. In fact, by treating ο(t) as a vector of parameters, the MILP becomes a multiparametric MILP (mp-milp), and its solution for all admissible initial state ο(t) will be the explicit MPC controller law for MLD systems. We will also show that such a control law is piecewise affine with respect to the state vector. As we will describe in the next section, we use the algorithm developed in [] for solving the mp-milp formulated above. Once the multi-parametric problem () has been solved off line, i.e. the solution p Λ t = f(ο(t)) of () has been found, the model predictive controller ()-() is available explicitly, as the optimal input u(t) consists simply of m components of p Λ t u(t) =[ ::: I ::: ]f(ο(t)): () As the solution p Λ of the mp-milp problem is piecewise affine with respect to the state x(t), clearly, because of (), the same property is inherited by the controller. 5 An Example Consider the following system [6]» cos ff(t) sin ff(t) x(t +) = : x(t)+» sin ff(t) cos ff(t) u(t) y(t) = [ ρ ]x(t) ß ff(t) = if [ ]x(t) ß if [ ]x(t) < x(t) [5; 5] [5; 5] u(t) [; ] () By using auxiliary variables z(t) R and ffi(t) f; g such that[ffi(t) =] $ [[ ]x(t) ], Eq. () can be rewritten in the form () as in [6]. In order to optimally transfer the state from x = [ ] to x f = [ ], the performance index () is minimized subject to () and the MLD system dynamics (), along with the weights Q =:, Q =:, Q =I, Q =I, Q 5 =. By solving the mp- MILP associated with this MPC problem we obtain the
In Fig. the state space region are depicted while he resulting optimal trajectories are shown in Fig.. 6 Acknowledgments X 5 6 This researchwas supported by the Swiss National Science Foundation. References x..6.. -. -.6 -. X Figure : Polyhedral partition of the state-space -. x-space - -.5 x.5 u(t).5 -.5 - -..6.. x (t,x,u y ) x (t,x,u y ) d(t,x,u y ) Figure : Closed-loop MPC control of system () explicit form of the controller u = : if [ :9 : ] x if [ :9 : ] x if : if [ :69 : ] x if [ :69 : ] x if : if " : :96 : : : : 9:56 : (Region #) #» :5 : :5 :99 : : : : (Region #)» : : :99 6:9 : : : : (Region #)» :5 : : :65 : : : :6 (Region #) : :9 : :95 : : :9 :6 :9 : (Region #5) : :5 :5 :99 : : : :69 6: : (Region #6) 6 : :66 : :9 : : : :6 : 65: : :69 : : (Region #) " # :6 5:5» :6 : :» :9 : :» :9 :6 665:6 5 : : : : 5 " : : 5 : :5 55:59 5 6 :9 : : 9:55 :5 [] J. Acevedo and E. N. Pistikopoulos. Amultiparametric programming approach for linear process engineering problems under uncertainty. Ind. Eng. Chem. Res., 6:, 99. [] R. Alur, C. Courcoubetis, T.A. Henzinger, and P.-H. Ho. Hybrid automata: An algorithmic approach to the specification and verification of hybrid systems. In A.P. Ravn R.L. Grossman, A. Nerode and H. Rischel, editors, Hybrid Systems, volume 6 of Lecture Notes in Computer Science, pages 99. Springer Verlag, 99. [] A. Asarin, O. Maler, and A. Pnueli. On the analysis of dynamical systems having piecewise-constant derivatives. Theoretical Computer Science, :565, 995. [] A. Bemporad, F. Borrelli, and M. Morari. Explicit solution of constrained /-norm model predictive control. Technical Report AUT-5, Automatic Control Laboratory, ETH Zurich, Switzerland, February. [5] A. Bemporad, D. Mignone, and M. Morari. Moving horizon estimation for hybrid systems and fault detection. In Proc. American Contr. Conf., 999. [6] A. Bemporad and M. Morari. Control of systems integrating logic, dynamics, and constraints. Automatica, 5():, March 999. [] A. Bemporad and M. Morari. Verification of hybrid systems via mathematical programming. In F.W. Vaandrager and J.H. va nschuppen, editors, Hybrid Systems: Computation and Control, volume 569 of Lecture Notes in Computer Science, pages 5. Springer Verlag, 999. [] A. Bemporad, M. Morari, V. Dua, and E. N. Pistikopoulos. The explicit linear quadratic regulator for constrained systems. In Proc. American Contr. Conf.,. [9] F. Borrelli, A. Bemporad, and M. Morari. A geometric algorithm for multi-parametric linear programming. Technical Report AUT-6, Automatic Control Laboratory, ETH Zurich, Switzerland, February. [] P.J. Campo and M. Morari. Robust model predictive control. In Proc. American Contr. Conf., volume, pages # 5 6, 9. [] V. Duaand E. N. Pistikopoulos. An algorithm for the solution of multiparametric mixed integer linear programming problems. Annals ofoperations Research, toappear999. [] ILOG, Inc., Gentilly Cedex, France. CPLEX 6.5 Reference Manual, 999. [] Y. Kesten, A. Pnueli, J. Sifakis, and S. Yovine. Integration graphs: aclass of decidable hybrid systems. In R.L. Grossman, A. Nerode, A.P. Ravn, and H. Rischel, editors, Hybrid Systems, volume 6 of Lecture Notes in Computer Science, pages9. Springer Verlag, 99. [] J. Lygeros, C. Tomlin, and S. Sastry. Controllers for reachability specifications for hybrid systems. 5():9, 999. Automatica,