MPC: Tracking, Soft Constraints, Move-Blocking

MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger Institut für Automatik, ETH Zürich Institute for Dynamic Systems and Control, ETH Zürich UC Berkeley EPFL Spring Semester 2016 MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016

Table of Contents 1. Reference Tracking 1.1 The Steady-State Problem 1.2 Offset Free Reference Tracking 2. Soft Constraints 2.1 Motivation 2.2 Mathematical Formulation 3. Generalizing the Problem PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016

Reference Tracking Table of Contents 1. Reference Tracking 1.1 The Steady-State Problem 1.2 Offset Free Reference Tracking 2. Soft Constraints 2.1 Motivation 2.2 Mathematical Formulation 3. Generalizing the Problem PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016

Reference Tracking Tracking problem Consider the linear system model x k+1 = Ax k + Bu k y k = Cx k Goal: Track given reference r such that y k r as k. Determine the steady state target condition x s, u s : x s = Ax s + Bu s Cx s = r [ ] [ ] [ I A B xs 0 = C 0 u s r] MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 1-2

Reference Tracking The Steady-State Problem Table of Contents 1. Reference Tracking 1.1 The Steady-State Problem 1.2 Offset Free Reference Tracking PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016

Reference Tracking The Steady-State Problem Steady-state target problem In the presence of constraints: (x s, u s ) has to satisfy state and input constraints. In case of multiple feasible u s, compute cheapest steady-state (x s, u s ) corresponding to reference r: min us T R s u s [ ] [ ] I A B xs s.t. = C 0 u s x s X, u s U. [ ] 0 r In general, we assume that the target problem is feasible If no solution exists: compute reachable set point that is closest to r: min (Cx s r) T Q s (Cx s r) s.t. x s = Ax s + Bu s x s X, u s U. MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 1-3

Reference Tracking The Steady-State Problem RHC Reference Tracking We now use control (MPC) to bring the system to a desired steady-state condition (x s, u s ) yielding the desired output y k r. The MPC is designed as follows N 1 min y N Cx s 2 P + y k Cx s 2 Q + u k u s 2 R u 0,...,u N 1 subj. to model constraints x 0 = x(t). Drawback: controller will show offset in case of unknown model error or disturbances. k=0 MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 1-4

Reference Tracking Offset Free Reference Tracking Table of Contents 1. Reference Tracking 1.1 The Steady-State Problem 1.2 Offset Free Reference Tracking PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016

Reference Tracking Offset Free Reference Tracking RHC Reference Tracking without Offset (1/6) Discrete-time, time-invariant system (possibly nonlinear, uncertain) Objective: x m (t + 1) = g(x m (t), u(t)) y m (t) = h(x m (t)) Design an RHC in order to make y(t) track the reference signal r(t), i.e., (y(t) r(t)) 0 for t. In the rest of the section we study step references and focus on zero steady-state tracking error, y(t) r as t. Consider augmented model with constant disturbance d(t) R n d. x(t + 1) = Ax(t) + Bu(t) + B d d(t) d(t + 1) = d(t) y(t) = Cx(t) + C d d(t) MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 1-5

Reference Tracking Offset Free Reference Tracking RHC Reference Tracking without Offset (2/6) State observer for augmented model [ ] [ ˆx(t + 1) A Bd = ˆd(t + 1) 0 I [ Lx + L d ] [ ˆx(t) ˆd(t) ] [ B + 0 ] u(t) ] ( y m (t) + C ˆx(t) + C d ˆd(t)) Lemma Suppose the observer is stable and the number of outputs p equals the dimension of the constant disturbance n d. The observer steady state satisfies: [ A I B C 0 ] [ ˆx u ] [ = B d ˆd y m, C d ˆd where y m, and u are the steady state measured outputs and inputs. The observer output C ˆx + C d ˆd tracks the measured output y m, without offset. MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 1-6 ].

Reference Tracking Offset Free Reference Tracking RHC Reference Tracking without Offset (3/6) For offset-free tracking at steady state we want y m, = r. The observer condition [ ] [ ] [ ] A I B ˆx B = d ˆd C 0 u y m, C d ˆd suggests that at steady state the MPC should satisfy [ A I B C 0 ] [ ] [ xtarget, = u target, B d ˆd r C d ˆd ] MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 1-7

Reference Tracking Offset Free Reference Tracking RHC Reference Tracking without Offset (4/6) Formulate the RHC problem N 1 min x N x t 2 P + x k x t 2 Q + u k ū t 2 R U 0 k=0 subj. to x k+1 = Ax k + Bu k + B d d k, k = 0,..., N x k X, u k U, k = 0,..., N 1 x N X f d k+1 = d k, k = 0,..., N x 0 = ˆx(t) d 0 = ˆd(t), with the targets ū t and x t given by [ A I B ] [ xt C 0 ū t ] [ = B d ˆd(t) r(t) C d ˆd(t) ] MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 1-8

Reference Tracking Offset Free Reference Tracking RHC Reference Tracking without Offset (5/6) Denote by c 0 (ˆx(t), ˆd(t), r(t)) = u 0(ˆx(t), ˆd(t), r(t)) the control law when the estimated state and disturbance are ˆx(t) and ˆd(t), respectively. Theorem Consider the case where the number of constant disturbances equals the number of (tracked) outputs n d = p = r. Assume the RHC is recursively feasible and unconstrained for t j with j N + and the closed-loop system x(t + 1) = f (x(t), c 0 (ˆx(t), ˆd(t), r(t))) ˆx(t + 1) = (A + L x C)ˆx(t) + (B d + L x C d )ˆd(t) +Bc 0 (ˆx(t), ˆd(t), r(t)) L x y m (t) ˆd(t + 1) = L d C ˆx(t) + (I + L d C d )ˆd(t) L d y m (t) converges to ˆx, ˆd, y m,, i.e., ˆx(t) ˆx, ˆd(t) ˆd, y m (t) y m, as t. Then y m (t) r as t. PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 1-9

Reference Tracking Offset Free Reference Tracking RHC Reference Tracking without Offset (6/6) Question: How do we choose the matrices B d and C d in the augmented model? Lemma The augmented system, with the number of outputs p equal to the dimension of the constant disturbance n d, and C d = I is observable if and only if (C, A) is observable and [ ] A I Bd det = det(a I B C I d C) 0. Remark: If the plant has no integrators, then det (A I ) 0 and we can choose B d = 0. If the plant has integrators then B d has to be chosen specifically to make det (A I B d C) 0. PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 1-10

Soft Constraints Table of Contents 1. Reference Tracking 1.1 The Steady-State Problem 1.2 Offset Free Reference Tracking 2. Soft Constraints 2.1 Motivation 2.2 Mathematical Formulation 3. Generalizing the Problem PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016

Soft Constraints Motivation Table of Contents 2. Soft Constraints 2.1 Motivation 2.2 Mathematical Formulation PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016

Soft Constraints Motivation Soft Constraints: Motivation Input constraints are dictated by physical constraints on the actuators and are usually hard State/output constraints arise from practical restrictions on the allowed operating range and are rarely hard Hard state/output constraints always lead to complications in the controller implementation Feasible operating regime is constrained even for stable systems Controller patches must be implemented to generate reasonable control action when measured/estimated states move outside feasible range because of disturbances or noise In industrial implementations, typically, state constraints are softened MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 2-11

Soft Constraints Mathematical Formulation Table of Contents 2. Soft Constraints 2.1 Motivation 2.2 Mathematical Formulation PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016

Soft Constraints Mathematical Formulation Mathematical Formulation Original problem: Assume for now g(z) is scalar valued. Softened problem: min z f (z) subj. to g(z) 0 min z,ɛ subj. to f (z) + l(ɛ) g(z) ɛ ɛ 0 Requirement on l(ɛ) If the original problem has a feasible solution z, then the softened problem should have the same solution z, and ɛ = 0. Note: l(ɛ) = v ɛ 2 does not meet this requirement for any v > 0 as demonstrated next. PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 2-12

Soft Constraints Mathematical Formulation Quadratic Penalty Constraint function g(z) z z 0 induces feasible region (grey) = minimizer of the original problem is z Quadratic penalty l(ɛ) = v ɛ 2 for ɛ 0 = minimizer of f (z) + l(ɛ) is (z + ɛ, ɛ ) instead of (z, 0) MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 2-13

Soft Constraints Mathematical Formulation Quadratic Penalty Constraint function g(z) z z 0 induces feasible region (grey) = minimizer of the original problem is z Quadratic penalty l(ɛ) = v ɛ 2 for ɛ 0 = minimizer of f (z) + l(ɛ) is (z + ɛ, ɛ ) instead of (z, 0) PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 2-13

Soft Constraints Mathematical Formulation Linear Penalty Constraint function g(z) z z 0 induces feasible region (grey) = minimizer of the original problem is z Linear penalty l(ɛ) = u ɛ for ɛ 0 with u chosen large enough so that u + lim z z f (z) > 0 = minimizer of f (z) + l(ɛ) is (z, 0) MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 2-14

Soft Constraints Mathematical Formulation Main Result Theorem (Exact Penalty Function) l(ɛ) = u ɛ satisfies the requirement for any u > u 0, where u is the optimal Lagrange multiplier for the original problem. Disadvantage: l(ɛ) = u ɛ renders the cost non-smooth. Therefore in practice, to get a smooth penalty, we use with u > u and v > 0. l(ɛ) = u ɛ + v ɛ 2 Extension to multiple constraints g j (z) 0, j = 1,..., r: l(ɛ) = r u j ɛ j + v j ɛ 2 j (1) j=1 where u j > uj differently. and v j > 0 can be used to weight violations (if necessary) PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 2-15

Generalizing the Problem Table of Contents 1. Reference Tracking 1.1 The Steady-State Problem 1.2 Offset Free Reference Tracking 2. Soft Constraints 2.1 Motivation 2.2 Mathematical Formulation 3. Generalizing the Problem PC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016

Generalizing the Problem Generalizing the Problem Modify problem as: min U 0 x N y 2 P + N y 1 k=0 [ xk 2 Q + u k 2 R] subj. to y min (k) y k y max (k), k = 1,..., N c u min u k u max, k = 0, 1,..., N u x 0 = x(t) x k+1 = Ax k + Bu k, k 0 y k = Cx k, k 0 u k = Kx k, N u k < N y with N u N y and N c N y 1. Many applications require time-varying constraints, e.g. y min (k), y max (k) Complexity can be reduced by introducing separate horizons N u, N c, N y But, all theoretical feasibility and stability guarantees are lost! MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 3-16

Generalizing the Problem Generalizing the Problem More effective way to reduce the computational effort: Move-blocking Manipulated variables are fixed over time intervals in the future degrees of freedom in optimization problem are reduced By choosing the blocking strategies carefully RHC stability results remain applicable MPC: Tracking, Soft Constraints, Move-Blocking M. Morari, F. Borrelli, C. Jones, M. Zeilinger - Spring Semester 2016 3-17

Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger Institut für Automatik, ETH Zürich Institute for Dynamic Systems and Control, ETH Zürich UC Berkley EPFL Spring Semester 2016 Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016

Table of Contents 1. Explicit Model Predictive Control 1.1 Introduction 1.2 mpqp 1.3 mplp 1.4 Constrained Finite Time Optimal Control 1.5 Online Evaluation: Point Location Problem 1.6 MPT Example 1.7 Summary xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016

Explicit Model Predictive Control Table of Contents 1. Explicit Model Predictive Control 1.1 Introduction 1.2 mpqp 1.3 mplp 1.4 Constrained Finite Time Optimal Control 1.5 Online Evaluation: Point Location Problem 1.6 MPT Example 1.7 Summary xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016

Explicit Model Predictive Control Introduction Table of Contents 1. Explicit Model Predictive Control 1.1 Introduction 1.2 mpqp 1.3 mplp 1.4 Constrained Finite Time Optimal Control 1.5 Online Evaluation: Point Location Problem 1.6 MPT Example 1.7 Summary xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016

Explicit Model Predictive Control Introduction Introduction Requires at each time step on-line solution of an optimization problem Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-2

Explicit Model Predictive Control Introduction Introduction OFFLINE ONLINE N 1 U0 (x(t)) = argmin xt N Px N + x k Qx k + u k Ru k k=0 subj. to x 0 = x(t) x k+1 = Ax k + Bu k, k = 0,..., N 1 x k X, u k U, k = 0,..., N 1 x N X f Optimization problem is parameterized by state Pre-compute control law as function of state x Control law is piecewise affine for linear system/constraints Result: Online computation dramatically reduced and real-time Tool: Parametric programming U * ( x( t)) 0 Plant state xt () Plant U * ( x) 0 Output yt () Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-3

Explicit Model Predictive Control Introduction Example (1/2) Consider the following problem: x R is a parameter. The goals: 1 find z (x) = argmin z J(z, x), J (x) = min z J(z, x) = 1 2 z2 + 2xz + 2x 2 subj. to z 1 + x, 2 find all x for which the problem has a solution 3 compute the value function J (x) Define Lagrangian: L(z, x, λ) = f (z, x) + λ(z x 1) KKT conditions: z + 2x + λ = 0, z x 1 0, λ(z x 1) = 0, λ 0. Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-4

Explicit Model Predictive Control Introduction Example (2/2) Consider the two strictly complementary cases: 1 z x 1 = 0 λ 0 z (x) = x + 1, x 1 3 J (x) = 9 2 x2 + 3x + 1 2 2 z x 1 < 0 λ = 0 z (x) = 2x, x > 1 3 J (x) = 0 { z x + 1, if x 1 (x) = 3 2x, if x 1 3 This problem has a solution for all x., J (x) = { 9 2 x2 + 3x + 1 2, if x 1 3 0, if x 1 3 Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-5

Explicit Model Predictive Control mpqp Table of Contents 1. Explicit Model Predictive Control 1.1 Introduction 1.2 mpqp 1.3 mplp 1.4 Constrained Finite Time Optimal Control 1.5 Online Evaluation: Point Location Problem 1.6 MPT Example 1.7 Summary xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016

Explicit Model Predictive Control mpqp mpqp - Problem formulation J 1 (x) = min z 2 z Hz, subj. to Gz w + Sx where H > 0, z R s, x R n and G R m s. Given a closed and bounded polyhedral set K R n of parameters denote by K K the region of parameters x K such that the problem is feasible Goals: 1 find z (x) = argmin z J(z, x), K {x K : z, Gz w + Sx} 2 find all x for which the problem has a solution 3 compute the value function J (x) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-6

Explicit Model Predictive Control mpqp Active Set and Critical Region Let I {1,..., m} be the set of constraint indices. Definition: Active Set We define the active set at x, A(x), and its complement, NA(x), as A(x) {i I : G i z (x) S i x = w i } NA(x) {i I : G i z (x) S i x < w i }. G i, S i and w i are the i-th row of G, S and w, respectively. Definition: Critical Region CR A is the set of parameters x for which the same set A I of constraints is active at the optimum. For a given x K let (A, NA) (A( x), NA( x)). Then, CR A {x K : A(x) = A}. xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-7

Explicit Model Predictive Control mpqp mpqp - Global properties of the solution The following theorem summarizes the properties of the mpqp solution. Theorem: Solution of mpqp i) The feasible set K is a polyhedron. ii) The optimizer function z (x) : K R m is: continuous polyhedral piecewise affine over K. It is affine in each critical region CR i, every CR i is a polyhedron and CR i = K. iii) The value function J (x) : K R is: continuous convex polyhedral piecewise quadratic over K, it is quadratic in each CR i xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-8

Explicit Model Predictive Control mpqp mpqp - Example (1/4) Consider the example min z(x) 1 2 (z2 1 + z 2 2 ) subj. to z 1 1 + x 1 + x 2 z 1 1 x 1 x 2 z 2 1 + x 1 x 2 z 2 1 x 1 + x 2 z 1 z 2 x 1 + 3x 2 z 1 + z 2 2x 1 x 2 1 x 1 1, 1 x 2 1 Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-9

Explicit Model Predictive Control mpqp mpqp - Example (2/4) The explicit solution is defined over i = 1,..., 7 regions P i = {x R 2 A i x b i } in the parameter space x 1 x 2. Critical regions Piecewise quadratic objective function J (x) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-10

Explicit Model Predictive Control mpqp mpqp - Example (3/4) Primal solution is given as piecewise affine function z(x) = F i + g i x if x P i. ( ) 0.5 1.5 x if x P 1 0.5 1.5 ( ) ( ) 2 2 1 z (x) = x + if x P 2 1 1 1... Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-11

Explicit Model Predictive Control mpqp mpqp - Example (4/4) Primal solution is given as piecewise affine function z(x) = F i + g i x if x P i. Piecewise affine function z 1 (x) Piecewise affine function z 2 (x) xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-12

Explicit Model Predictive Control mplp Table of Contents 1. Explicit Model Predictive Control 1.1 Introduction 1.2 mpqp 1.3 mplp 1.4 Constrained Finite Time Optimal Control 1.5 Online Evaluation: Point Location Problem 1.6 MPT Example 1.7 Summary xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016

Explicit Model Predictive Control mplp mplp - Problem formulation J (x) = min z subj. to c z Gz w + Sx where z R s, x R n and G R m s. Given a closed and bounded polyhedral set K R n of parameters, denote by K K the region of parameters x K such that the problem is feasible Goals: 1 find z (x) = argmin z J(z, x), K {x K : z, Gz w + Sx} 2 find all x for which the problem has a solution 3 compute the value function J (x) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-13

Explicit Model Predictive Control mplp mplp - Global properties of the solution The following theorem summarizes the properties of the mplp solution. Theorem: Solution of mplp i) The feasible set K is a polyhedron. ii) If the optimal solution z is unique x K, the optimizer function z (x) : K R m is: continuous polyhedral piecewise affine over K. It is affine in each critical region CR i, every CR i is a polyhedron and CR i = K. Otherwise, it is always possible to choose such a continuous and PPWA optimizer function z (x). iii) The value function J (x) : K R is: continuous convex polyhedral piecewise affine over K, it is affine in each CR i. xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-14

Explicit Model Predictive Control mplp mplp - Example (1/4) Consider the example min z(x) 3z 1 8z 2 subj. to z 1 + z 2 13 + x 1 5z 1 4z 2 20 8z 1 + 22z 2 121 + x 2 4z 1 z 2 8 z 1 0 z 2 0 1 x 1 1 1 x 2 1 Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-15

Explicit Model Predictive Control mplp mplp - Example (2/4) The explicit solution is defined over i = 1,..., 3 regions P i = {x R 2 A i x b i } in the parameter space x 1 x 2. Critical regions P i Piecewise affine objective function J (x) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-16

Explicit Model Predictive Control mplp mplp - Example (3/4) Primal solution is given as piecewise affine function z (x) = F i x + g i if x P i. ( ) ( ) 0.733 0.033 5.5 x + if x P 1 0.267 0.033 7.5 ( ) ( ) z 0 0.051 11.846 (x) = x + if x P 2 0 0.064 9.808 ( ) ( ) 0.333 0 1.667 x + if x P 3 1.333 0 14.667 Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-17

Explicit Model Predictive Control mplp mplp - Example (4/4) Primal solution is given as piecewise affine function z (x) = F i x + g i if x P i. Piecewise affine function z 1 (x) Piecewise affine function z 2 (x) xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-18

Explicit Model Predictive Control Constrained Finite Time Optimal Control Table of Contents 1. Explicit Model Predictive Control 1.1 Introduction 1.2 mpqp 1.3 mplp 1.4 Constrained Finite Time Optimal Control 1.5 Online Evaluation: Point Location Problem 1.6 MPT Example 1.7 Summary xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016

Explicit Model Predictive Control Constrained Finite Time Optimal Control Problem Formulation - Quadratic Cost Quadratic cost function N 1 J 0 (x(0), U 0 ) = x NPx N + x kqx k + u kru k (1) k=0 with P 0, Q 0, R 0. Constrained Finite Time Optimal Control problem (CFTOC). J 0 (x(0)) = min U 0 J 0 (x(0), U 0 ) subj. to x k+1 = Ax k + Bu k, k = 0,..., N 1 x k X, u k U, k = 0,..., N 1 x N X f x 0 = x(0) (2) N is the time horizon and X, U, X f are polyhedral regions. Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-19

Explicit Model Predictive Control Constrained Finite Time Optimal Control Construction of the QP with substitution Step 1: Rewrite the cost as (see lectures on Day 1 & 2) J 0 (x(0), U 0 ) = U 0H U 0 + 2x(0) FU 0 + x(0) Yx(0) = [U 0 x(0) ] [ ] H F [U0 x(0) ] F Y Note: [ ] H F F Y 0 since J0 (x(0), U 0 ) 0 by assumption. Step 2: Rewrite the constraints compactly as (details provided on the next slide) G 0 U 0 w 0 + E 0 x(0) Step 3: Rewrite the optimal control problem as J0 (x(0)) = min [U 0 x(0) ] [ ] H F [U0 x(0) ] U F Y 0 subj. to G 0 U 0 w 0 + E 0 x(0) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-20

Explicit Model Predictive Control Constrained Finite Time Optimal Control Solution J0 (x(0)) = min [U 0 x(0) ] [ ] H F [U0 x(0) ] U F Y 0 subj. to G 0 U 0 w 0 + E 0 x(0) For a given x(0) U 0 can be found via a QP solver. Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-21

Explicit Model Predictive Control Constrained Finite Time Optimal Control Construction of QP constraints with substitution If X, U and X f are given by: X = {x A x x b x } U = {u A u u b u } X f = {x A f x b f } Then G 0, E 0 and w 0 are defined as follows A u 0... 0 0 b u 0 A u... 0 0 b ụ........ 0 0... A u 0 b u G 0 = 0 0... 0, E 0 = A x, w 0 = b x A x B 0... 0 A x A b x A x AB A x B... 0 A x A 2 b x........ A f A N 1 B A f A N 2 B... A f B A f A N b f Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-22

Explicit Model Predictive Control Constrained Finite Time Optimal Control 2-Norm State Feedback Solution Start from QP with substitution. Step 1: Define z U 0 + H 1 F x(0) and transform the problem into Ĵ (x(0)) = min z subj. to z Hz G 0 z w 0 + S 0 x(0), where S 0 E 0 + G 0 H 1 F, and Ĵ (x(0)) = J 0 (x(0)) x(0) (Y FH 1 F )x(0). The CFTOC problem is now a multiparametric quadratic program (mp-qp). Step 2: Solve the mp-qp to get explicit solution z (x(0)) Step 3: Obtain U 0 (x(0)) from z (x(0)) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-23

Explicit Model Predictive Control Constrained Finite Time Optimal Control 2-Norm State Feedback Solution Main Results 1 The Open loop optimal control function can be obtained by solving the mp-qp problem and calculating U 0 (x(0)), x(0) X 0 as U 0 = z (x(0)) H 1 F x(0). 2 The first component of the multiparametric solution has the form u (0) = f 0 (x(0)), x(0) X 0, f 0 : R n R m, is continuous and piecewise affine on polyhedra f 0 (x) = F i 0x + g i 0 if x CR i 0, i = 1,..., N r 0 3 The polyhedral sets CR i 0 = {x R n H i 0x K i 0}, i = 1,..., N r 0 are a partition of the feasible polyhedron X 0. 4 The value function J 0 (x(0)) is convex and piecewise quadratic on polyhedra. xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-24

Explicit Model Predictive Control Constrained Finite Time Optimal Control Example Consider the double integrator x(t + 1) = subject to constraints [ 1 1 0 1 y(t) = [ 1 0 ] x(t) ] [ 0 x(t) + 1 1 u(k) 1, k = 0,..., 5 [ ] [ ] 10 10 x(k), k = 0,..., 5 10 10 ] u(t) Compute the state feedback optimal controller u (0)(x(0)) solving the CFTOC problem with N = 6, Q = [ 1 0 0 1 ], R = 0.1, P the solution of the ARE, X f = R 2. Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-25

Explicit Model Predictive Control Constrained Finite Time Optimal Control Example 10 5 x 2 (0) 0 5 10 10 5 0 5 10 x 1 (0) Figure : Partition of the state space for the piecewise affine control law u (0) (N r 0 = 13) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-26

Explicit Model Predictive Control Constrained Finite Time Optimal Control Problem Formulation - Piecewise Linear Cost Piecewise linear cost function N 1 J 0 (x(0), U 0 ) := Px N p + Qx k p + Ru k p (3) k=0 with p = 1 or p =, P, Q, R full column rank matrices Constrained Finite Time Optimal Control Problem (CFTOC) J 0 (x(0)) = min U 0 J 0 (x(0), U 0 ) subj. to x k+1 = Ax k + Bu k, k = 0,..., N 1 x k X, u k U, k = 0,..., N 1 x N X f x 0 = x(0) N is the time horizon and X, U, X f are polyhedral regions. (4) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-27

Explicit Model Predictive Control Constrained Finite Time Optimal Control Construction of the LP with substitution Recall that the norm problem can be equivalently formulated as min ε x 0 +... + ε x N + ε u 0 +... + ε u N 1 z 0 [ ] k 1 subj. to 1 n ε x k ±Q A k x 0 + A j Bu k 1 j, 1 r ε x N ±P [ 1 m ε u k ±Ru k, A N x 0 + j=0 N 1 j=0 A j Bu N 1 j ] k 1 A k x 0 + A j Bu k 1 j X, u k U, j=0 N 1 A N x 0 + A j Bu N 1 j X f, j=0 k = 0,..., N 1 x 0 = x(0), Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-28

Explicit Model Predictive Control Constrained Finite Time Optimal Control Construction of the LP with substitution The problem yields the following standard LP min z 0 c 0z 0 subj. to Ḡ 0 z 0 w 0 + S 0 x(0) where z 0 := {ε x 0,..., ε x N, εu 0,..., ε u N 1, u 0,..., u N 1 } Rs, s (m + 1)N + N + 1 and Ḡ 0 = [ ] [ ] [ ] Gε 0 Sε wε, S0 =, w 0 G 0 S 0 = 0 w 0 For a given x(0) U0 similar). can be obtained via an LP solver (the 1 norm case is Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-29

Explicit Model Predictive Control Constrained Finite Time Optimal Control 1- / -Norm State Feedback Solution Main Results 1 The Open loop optimal control function can be obtained by solving the mp-lp problem and calculating z 0 (x(0)) 2 The component u 0 = [0... 0 I m 0... 0]z 0 (x(0)) of the multiparametric solution has the form u (0) = f 0 (x(0)), x(0) X 0, f 0 : R n R m, is continuous and piecewise affine on polyhedra f 0 (x) = F i 0x + g i 0 if x CR i 0, i = 1,..., N r 0 3 The polyhedral sets CR i 0 = {x R n H i 0x K i 0}, i = 1,..., N r 0 are a partition of the feasible polyhedron X 0. 4 In case of multiple optimizers a piecewise affine control law exists. 5 The value function J 0 (x(0)) is convex and piecewise affine on polyhedra. xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-30

Explicit Model Predictive Control Constrained Finite Time Optimal Control Explicit MPC OFFLINE ONLINE N 1 U0 (x(t)) = argmin xt N Px N + x k Qx k + u k Ru k k=0 subj. to x 0 = x(t) x k+1 = Ax k + Bu k, k = 0,..., N 1 x k X, u k U, k = 0,..., N 1 x N X f Optimization problem is parameterized by state Pre-compute control law as function of state x Control law is piecewise affine for linear system/constraints Result: Online computation dramatically reduced and real-time Tool: Parametric programming U * ( x( t)) 0 Plant state xt () Plant U * ( x) 0 Output yt () xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-31

Explicit Model Predictive Control Online Evaluation: Point Location Problem Table of Contents 1. Explicit Model Predictive Control 1.1 Introduction 1.2 mpqp 1.3 mplp 1.4 Constrained Finite Time Optimal Control 1.5 Online Evaluation: Point Location Problem 1.6 MPT Example 1.7 Summary xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016

Explicit Model Predictive Control Online Evaluation: Point Location Problem Online evaluation: Point location Calculation of piecewise affine function: 1 Point location 2 Evaluation of affine function 1 2 Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-32

Explicit Model Predictive Control Online Evaluation: Point Location Problem Sequential search x Sequential search for each i if A i x + b i 0 then x is in region i Very simple Linear in number of regions xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-33

Explicit Model Predictive Control Online Evaluation: Point Location Problem Logarithmic search (1/6) h Offline construction of search tree Find hyperplane that separates regions into two equal sized sets Repeat for left and right sets xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-34

Explicit Model Predictive Control Online Evaluation: Point Location Problem Logarithmic search (2/6) h h x Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-35

Explicit Model Predictive Control Online Evaluation: Point Location Problem Logarithmic search (3/6) h L h h L h x h LR Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-36

Explicit Model Predictive Control Online Evaluation: Point Location Problem Logarithmic search (4/6) h L h LR h h L h x h LR h LLR Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-37

Explicit Model Predictive Control Online Evaluation: Point Location Problem Logarithmic search (5/6) h L h LRR h LR h h L h x h LR h LLR h LLRL Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-38

Explicit Model Predictive Control Online Evaluation: Point Location Problem Logarithmic search (6/6) h L h LRR h LR h h L h x h LRRL h LR h LLR h LLRL Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-39

Explicit Model Predictive Control Online Evaluation: Point Location Problem Point Location - Summary Sequential search Very simple Works for all problems Search tree Potentially logarithmic Significant offline processing (reasonable for < 1 000 regions) Many other options for special cases Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-40

Explicit Model Predictive Control MPT Example Table of Contents 1. Explicit Model Predictive Control 1.1 Introduction 1.2 mpqp 1.3 mplp 1.4 Constrained Finite Time Optimal Control 1.5 Online Evaluation: Point Location Problem 1.6 MPT Example 1.7 Summary xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016

Explicit Model Predictive Control MPT Example Ball and Plate y d g x a a' b' b Linearized model: four states for each axis: plate angle, ball position, plate angular speed, ball speed. Constraints on inputs and states Plate angle Ball position Acceleration MPC objective: path tracking Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-41

Explicit Model Predictive Control MPT Example Ball and Plate - System Matlab + RTW Computer: Pentium 166 Sampling Time: 30 ms TCP/IP Code Data Ball & Plate u 1, u 2 xpc Target α, β x, y Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-42

Explicit Model Predictive Control MPT Example Ball and Plate - MPC Problem 4 states + 1 tracking variable = 5 parameters Move-blocking reduces complexity Horizon of 10 Inputs 2 10 must be equal min 9 100 y i y t 2 2 + 0.1 u i 2 2 i=0 s.t. x 0 = x x i+1 = Ax i + Bu i y i = Cx i u min u i u max y min y i y max x min x i x max u i+1 = u i, i = {1,..., 9} Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-43

Explicit Model Predictive Control MPT Example Multi-Parametric Toolbox Modeling Optimal control Analysis Computational geometry control.ee.ethz.ch/~mpt Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-44

Explicit Model Predictive Control MPT Example Multi-Parametric Toolbox 3.0 Formulation M P T % Linear discrete-time prediction model model=ltisystem( A, A, B, B, C, C); % Input constraints model.u.min = -10; model.u.max = 10; % Output constraints model.y.min = -30; model.y.max = 30; % State constraints model.x.min = [-30; -15; -15*pi/180; -1]; model.x.max = [30; 15; 15*pi/180; 1]; % Penalties in the cost function model.y.penalty = QuadFunction(100); model.u.penalty = QuadFunction(0.1); % Adjustment via input blocking model.u.with( block ); model.u.from = 1; model.u.to = 9; % Time varying reference signal model.y.with( reference ); model.y.reference = free ; % Online MPC object online_ctrl = MPCController( model, 9 ) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-45

Explicit Model Predictive Control MPT Example Explicit Solution M P T % Compute explicit solution explicit_ctrl = online_ctrl.toexplicit() % Plot control law (primal solution) explicit_ctrl.optimizer.fplot( primal ) Control input % plot the objective function explicit_ctrl.optimizer.fplot( obj ) Value function % Plot controller partition explicit_ctrl.optimizer.plot Controller partition Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-50

Explicit Model Predictive Control MPT Example Exporting Explicit Solutions Search tree and export to flat c-file Export to flat c-file, sequential search M P T % generate binary search tree tree = BinTreePolyUnion(explicit_ctrl.optimizer) % export control law to C tree.toc( primal ) % export control law to C explicit_ctrl.optimizer.toc( primal ) #define MPT_NU 1 #define MPT_NX 2 static float MPT_ST[] = { -5.225216e-01, 8.526260e-01, 2.007907e+00, 2.000000e+00, 3.000000e+00, -6.669647e-01, 7.450893e-01, 3.576296e+00, 4.000000e+00, 5.000000e+00, 6.347758e-01, -7.726964e-01, 2.699058e+00, 6.000000e+00, 7.000000e+00, #ifdef tmwtypes_h -2.990218e-01, -9.542463e-01, -2.286671e-01, 8.000000e+00, 9.000000e+00, /* RTW is used, switch to real_t data types to avoid problems with TLC compi -4.546596e-01, -8.906653e-01, 3.576036e-01, 1.000000e+01, 1.100000e+01, lation */ 3.630512e-01, 9.317692e-01, 4.490581e-02, 1.200000e+01, 1.300000e+01, static long mpt_searchtree(const real_t *X, real_t *U) 4.546596e-01, 8.906653e-01, 3.576036e-01, 1.400000e+01, 1.500000e+01, #else -2.528581e-01, -9.675034e-01, -4.781395e-01, -5.000000e+00, -2.600000e+01, static long mpt_searchtree(const float *X, float *U) -6.811331e-01, 7.321596e-01, 4.632449e+00, 1.600000e+01, 1.700000e+01, #endif -3.630512e-01, -9.317692e-01, 4.490581e-02, -5.000000e+00, -6.000000e+00,{ -6.347758e-01, 7.726964e-01, 2.699058e+00, 1.800000e+01, 1.900000e+01, int ix, iu; 6.811331e-01, -7.321596e-01, 4.632449e+00, 2.000000e+01, 2.100000e+01, long node = 1, row; 6.669647e-01, -7.450893e-01, 3.576296e+00, 2.200000e+01, 2.300000e+01, float hx, k; 5.845669e-01, 8.113455e-01, 7.438927e-01, -4.000000e+00, -1.000000e+00, 5.225216e-01, -8.526260e-01, 2.007907e+00, 2.400000e+01, 2.500000e+01, /* initialize U to zero*/ 2.528581e-01, 9.675034e-01, 1.703078e+00, -4.000000e+00, -2.600000e+01, for (iu=0; iu<mpt_nu; iu++) { 2.990218e-01, 9.542463e-01, 1.489549e+00, -4.000000e+00, -1.400000e+01, U[iu] = 0; 3.630512e-01, 9.317692e-01, 1.263985e+00, -4.000000e+00, -6.000000e+00, } 4.546596e-01, 8.906653e-01, 1.018374e+00, -4.000000e+00, -2.000000e+00, 2.528581e-01, 9.675034e-01, -4.781395e-01, -4.000000e+00, -3.100000e+01, /* find region which contains the state x0 */ 2.990218e-01, 9.542463e-01, -2.286671e-01, -4.000000e+00, -1.900000e+01, while (node > 0) { hx = 0; row = (node-1)*(mpt_nx+3); for (ix=0; ix<mpt_nx; ix++) { hx = hx + MPT_ST[row+ix]*X[ix]; } k = MPT_ST[row+MPT_NX]; } if ((hx - k) < 0) { /* x0 on plus-side of the hyperplane */ node = (long)mpt_st[row+mpt_nx+2]; } else { /* x0 on minus-side of the hyperplane */ node = (long)mpt_st[row+mpt_nx+1]; } node = -node; Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-51

Explicit Model Predictive Control MPT Example Ball and Plate - Explicit Controller 4 states + 1 tracking variable = 5 parameters Move-blocking reduces complexity Horizon of 10 Inputs 2-10 must be equal Explicit solution (per dimension) Regions : 529 Storage : 48 000 numbers (192 kb) Computation : 89 000 FLOPS (~1ms) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-52

Explicit Model Predictive Control MPT Example Ball and Plate - Pentagram Simulation Experiment Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-53

Explicit Model Predictive Control MPT Example Real-time MPC Software Toolbox Formal specification YALMIP HYSDEL Linear + Hybrid models Verified controller Control law Explicit MPC Fixed-complexity solutions Software synthesis Real-time workshop Bounded-time solvers Verifiable code generation Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-54

Explicit Model Predictive Control Summary Table of Contents 1. Explicit Model Predictive Control 1.1 Introduction 1.2 mpqp 1.3 mplp 1.4 Constrained Finite Time Optimal Control 1.5 Online Evaluation: Point Location Problem 1.6 MPT Example 1.7 Summary xplicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016

Explicit Model Predictive Control Summary Summary Linear MPC + Quadratic or linear-norm cost Controller is PWA function We can pre-compute this function offline efficiently Online evaluation of a PWA function is very fast (ns - µs) We can only do this for very small systems! (3-6 states) Explicit Model Predictive Control F. Borrelli, C. Jones, M. Morari, M. Zeilinger - Spring Semester 2016 1-55

Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones, M. Zeilinger Institut für Automatik, ETH Zürich Institute for Dynamic Systems and Control, ETH Zürich UC Berkley EPFL Spring Semester 2016 Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016

Table of Contents 1. Modeling of Hybrid Systems 1.1 Introduction 1.2 Examples of Hybrid Systems 1.3 Piecewise Affine (PWA) Systems 1.4 Mixed Logical Dynamical (MLD) Hybrid Model 2. Optimal Control of Hybrid Systems 3. Model Predictive Control of Hybrid Systems 4. Explicit MPC of Hybrid Systems ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016

Modeling of Hybrid Systems Table of Contents 1. Modeling of Hybrid Systems 2. Optimal Control of Hybrid Systems 3. Model Predictive Control of Hybrid Systems 4. Explicit MPC of Hybrid Systems ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016

Modeling of Hybrid Systems Introduction Table of Contents 1. Modeling of Hybrid Systems 1.1 Introduction 1.2 Examples of Hybrid Systems 1.3 Piecewise Affine (PWA) Systems 1.4 Mixed Logical Dynamical (MLD) Hybrid Model ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016

Modeling of Hybrid Systems Introduction Introduction Up to this point: Discrete-time linear systems with linear constraints. We now consider MPC for systems with 1 Continuous dynamics: described by one or more difference (or differential) equations; states are continuous-valued. 2 Discrete events: state variables assume discrete values, e.g. binary digits {0, 1}, N, Z, Q,... finite set of symbols Hybrid systems: Dynamical systems whose state evolution depends on an interaction between continuous dynamics and discrete events. Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-3

Modeling of Hybrid Systems Introduction Introduction Introduction binary inputs discrete dynamics and logic binary outputs events mode switches real-valued outputs continuous dynamics real-valued inputs Figure: Hybrid systems. Logic-based discrete dynamics and continuous dynamics interact through events and mode switches Hybrid systems: Logic-based discrete dynamics and continuous dynamics interact through events and mode switches Francesco Borrelli (UC Berkeley) Hybrid Systems April 25, 2011 3 / 59 Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-4

Modeling of Hybrid Systems Examples of Hybrid Systems Table of Contents 1. Modeling of Hybrid Systems 1.1 Introduction 1.2 Examples of Hybrid Systems 1.3 Piecewise Affine (PWA) Systems 1.4 Mixed Logical Dynamical (MLD) Hybrid Model ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016

Modeling of Hybrid Systems Hybrid Systems: Examples (I) Mechanical System system with with backlash Backlash Examples of Hybrid Systems δ ε x x 1 x2 Continuous dynamics: :statesx x 1, 1, x 2 x, 2, ẋ 1 ẋ, 1, ẋ 2 ẋ. 2. Discrete events: : a) a) contact contact mode mode mechanical mechanical parts parts are are in in contact contact and and the the force force is is transmitted. Condition: transmitted. Condition: [( x = δ) (ẋ 1 > ẋ 2 )] [( x = ε) (ẋ2 ẋ 1 )] [( x = δ) (ẋ 1 > ẋ b) backlash mode mechanical 2)] [( x = ε) (ẋ 2 > ẋ parts are not in contact 1)] b) backlash mode mechanical parts are not in contact Francesco Borrelli (UC Berkeley) Hybrid Systems April 25, 2011 4 / 59 Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-5

Hybrid Systems: Examples (II) DCDC DC2DC Converter Modeling of Hybrid Systems Examples of Hybrid Systems ybrid Systems: Examples (II) C2DC Converter r l r l r l v l v l v s v s v l v s i l i l l S S =0 =0 =0 S S =1 =1 S =1 v c v c v c i i r 0 r 0 c c i r c r 0 r 0 v 0 v 0 r 0 v 0 i c i c i c Continuous dynamics :statesv l, l i, l i, l v, c v, c i, c i, c v, 0 v, 0 i, 0 i 0 Continuous dynamics: states v l, i l, v c, i c, v 0, i 0 Continuous Discrete events dynamics : : S S =0,S :statesv =1 =1 l, i l, v c, i c, v 0, i Discrete events: S = 0, S = 1 0 Discrete events : S =0,S =1 MODE Mode 1 1 (S (S =1) = MODE Mode 22 (S 2 (S = (S =0) MODE 1 (S =1) MODE 2 (S =0) Francesco Borrelli (UC (UCBerkeley) Hybrid Systems April April 25, 25, 2011 2011 5 / 559 / 59 ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-6

Modeling of Hybrid Systems Piecewise Affine (PWA) Systems Table of Contents 1. Modeling of Hybrid Systems 1.1 Introduction 1.2 Examples of Hybrid Systems 1.3 Piecewise Affine (PWA) Systems 1.4 Mixed Logical Dynamical (MLD) Hybrid Model ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016

Modeling of Hybrid Systems Piecewise Affine (PWA) Systems Piecewise Affine (PWA) Systems PWA systems are defined by: affine dynamics and output in each region: { } x(t + 1) = Ai x(t) + B i u(t) + f i y(t) = C i x(t) + D i u(t) + g i polyhedral partition of the (x, u)-space: with x R n, u R m {X i } s i=1 := {x, u H ix + J i u K i } if (x(t), u(t)) X i(t) Physical constraints on x(t) and u(t) are defined by polyhedra X i Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-7

Modeling of Hybrid Systems Piecewise Affine (PWA) Systems Piecewise Affine (PWA) Systems Examples: linearization of a non-linear system at different operating point useful as an approximation tool closed-loop MPC system for linear constrained systems When the mode i is an exogenous variable, the partition disappears and we refer to the system as a Switched Affine System (SAS) Definition: Well-Posedness Let P be a PWA system and let X = s i=1x i R n+m be the polyhedral partition associated with it. System P is called well-posed if for all pairs (x(t), u(t)) X there exists only one index i(t) satisfying the membership condition. ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-8

Modeling of Hybrid Systems Piecewise Affine (PWA) Systems Binary States, Inputs, and Outputs Remark: In the previous example, the PWA system has only continuous states and inputs. We will formulate PWA systems including binary state and inputs by treating 0 1 binary variables as: Numbers, over which arithmetic operations are defined, Boolean variables, over which Boolean functions are defined. We will use the notation x = [ xc x l ] R nc {0, 1} n l, n n c + n l ; y R pc {0, 1} p l, p p c + p l ; u R mc {0, 1} m l, m m c + m l. Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-9

Modeling of Hybrid Systems Piecewise Affine (PWA) Systems Boolean Algebra: Basic Definitions and Notation Boolean variable: A variable δ is a Boolean variable if δ {0, 1}, where δ = 0 means false, δ = 1 means true. A Boolean expression is obtained by combining Boolean variables through the logic operators (not), (or), (and), (implied by), (implies), and (iff). A Boolean function f : {0, 1} n 1 {0, 1} is used to define a Boolean variable δ n as a logic function of other variables δ 1,..., δ n 1 : δ n = f (δ 1, δ 2,..., δ n 1 ). Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-10

Modeling of Hybrid Systems Piecewise Affine (PWA) Systems Example x c (t + 1) = 2x c (t) + u c (t) 3u l (t) x l (t + 1) = x l (t) u l (t) can be represented in the PWA form [ 2xc (t) + u c (t) 0 [ 2xc (t) + u c (t) 3 [ ] xc (t + 1) 0 = x l (t + 1) [ 2xc (t) + u c (t) 0 [ 2xc (t) + u c (t) 3 1 ] ] ] ] if x l 1 2, u l 1 2 if x l 1 2, u l 1 2 + ɛ if x l 1 2 + ɛ, u l 1 2 if x l 1 2 + ɛ, u l 1 2 + ɛ. by associating x l = 0 with x l 1 2 and x l = 1 with x l 1 2 + ɛ for any 0 < ɛ 1 2. Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-11

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model Table of Contents 1. Modeling of Hybrid Systems 1.1 Introduction 1.2 Examples of Hybrid Systems 1.3 Piecewise Affine (PWA) Systems 1.4 Mixed Logical Dynamical (MLD) Hybrid Model ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model Mixed Logical Dynamical Systems Goal: Describe hybrid system in form compatible with optimization software: continuous and boolean variables linear equalities and inequalities Idea: associate to each Boolean variable p i a binary integer variable δ i : p i {δ i = 1}, p i {δ i = 0} and embed them into a set of constraints as linear integer inequalities. Two main steps: 1 Translation of Logic Rules into Linear Integer Inequalities 2 Translation continuous and logical components into Linear Mixed-Integer Relations Final result: a compact model with linear equalities and inequalities involving real and binary variables Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-12

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model Boolean formulas as Linear Integer Inequalities Goal Given a Boolean formula F(p 1, p 2,..., p n ) define a polyhedral set P such that a set of binary values {δ 1, δ 2,..., δ n } satisfies the Boolean formula F in P F(p 1, p 2,..., p n ) TRUE Aδ B, δ {0, 1} n where: {δ i = 1} p i = TRUE. Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-13

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model Analytic Approach 1 Transform F(p 1, p 2,..., p n ) into a Conjuctive Normal Form (CNF): F(p 1, p 2,..., p n ) = [ ] p i j i 2 Translation of a CNF into algebraic inequalities: relation Boolean linear constraints AND δ 1 δ 2 δ 1 1, δ 2 1 also δ 1 + δ 2 2 OR δ 1 δ 2 δ 1 + δ 2 1 NOT δ 1 (1 δ 1 ) 1 also δ 1 = 0 XOR δ 1 δ 2 δ 1 + δ 2 = 1 IMPLY δ 1 δ 2 δ 1 δ 2 0 IFF δ 1 δ 2 δ 1 δ 2 = 0 ASSIGNMENT δ 1 + (1 δ 3 ) 1 δ 3 = δ 1 δ 2 δ 3 δ 1 δ 2 δ 2 + (1 δ 3 ) 1 (1 δ 1 ) + (1 δ 2 ) + δ 3 1 Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-14

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model Analytic Approach. Example Given F(p 1, p 2, p 3, p 4 ) [(p 1 p 2 ) (p 3 p 4 )] find the equivalent set of linear integer inequalities. 1 remove implication: F(p 1, p 2, p 3, p 4 ) = (p 1 p 2 ) (p 3 p 4 ) 2 using DeMorgan s theorem, obtain CNF: F(p 1, p 2, p 3, p 4 ) = ( p 1 p 2 p 3 ) ( p 1 p 2 p 4 ) 3 introduce [δ i = 1] p i = TRUE and write the inequalities: δ 1 + δ 2 δ 3 1 F(p 1, p 2, p 3, p 4 ) = TRUE δ 1 + δ 2 δ 4 1 δ 1,2,3,4 {0, 1} Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-15

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model Linear Inequality As Logic Condition Definition: Event Generator An event generator is defined by function f EG : X c U c N 0 D: δ e (t) = f EG (x c (t), u c (t), t) Consider the Boolean expression consisting of a Boolean variable p and continuous variable x R n : where a R n, b R, x X R n : p a T x b X = {x a T x b [m, M ]} Translated to linear inequalitites: a T x b M (1 δ) a T x b > mδ ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-16

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model Switched Affine Dynamics Rewrite the state update functions of the SAS as { A1 x z 1 (t) = c (t) + B 1 u c (t) + f 1, if (i(t) = 1), 0, otherwise,. { As x z s (t) = c (t) + B s u c (t) + f s, if (i(t) = s), 0, otherwise, s x c (t + 1) = z i (t), i=1 In general, use the IF-THEN-ELSE relations IF δ THEN z = a 1x + b 1u + f 1 ELSE z = a 2x + b 2u + f 2, Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-17

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model IF-THEN-ELSE Relations IF p THEN (m 2 M 1)δ + z t a T 2 x t + b 2 ELSE z t = a T 1 x t + b 1 z t = a T 2 x t + b 2 (m 1 M 2)δ z t a T 2 x t b 2 (m 1 M 2)(1 δ) + z t a T 1 x t + b 1 (m 2 M 1)(1 δ) z t a T 1 x t b 1 where x X, with sup x X a T i x + b i M i, inf x X at i x + b i m i. ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-18

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model Hybrid Modelling - An Example Consider the following system with constraints: x 10, u 10 { 0.8x t + u t if x t 0 x t+1 = 0.8x t + u t if x t < 0 1 associate {δ t = 1} {x t 0} mδ t x t m (M + ɛ)δ t x t ɛ, where: M = m = 10, ɛ > 0. 2 state update equation: x t+1 = 1.6δ t x t 0.8x t + u t Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-19

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model Hybrid Modelling - An Example 3 introduce variable: z t = δ t x t x t+1 = 1.6z t 0.8x t + u t 4 constraints on z: z t M δ t z t mδ t z t x t m(1 δ t ) z t x t M (1 δ t ) Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-20

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model MLD Hybrid Model A DHA can be converted into the following MLD model x t+1 = Ax t + B 1 u t + B 2 δ t + B 3 z t y t = Cx t + D 1 u t + D 2 δ t + D 3 z t E 2 δ t + E 3 z t E 4 x t + E 1 u t + E 5 where x R nc {0, 1} n l, u R mc {0, 1} m l and z R rc. y R pc {0, 1} p l, δ {0, 1} r l Physical constraints on continuous variables: {[ ] } xc C = R nc+mc Fx u c + Gu c H c Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-21

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model MLD Hybrid Model. Well-Posedness Well-Posedness: for a given x = [ xt u t ] x t+1 and y t uniquely determined Complete Well-Posedness: well-posedness + uniquely determined δ t and z t, [ xt u t ] Well-posedness is sufficient for the computation of the state and output prediction Complete well-posedness allows transformation into equivalent hybrid models Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-22

Modeling of Hybrid Systems Mixed Logical Dynamical (MLD) Hybrid Model HYbrid System DEscription Language HYSDEL based on DHA enables description of discrete-time hybrid systems in a compact way: automata and propositional logic continuous dynamics A/D and D/A conversion definition of constraints automatically generates MLD models for MATLAB freely available from: http://control.ee.ethz.ch/ hybrid/hysdel/ Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 1-23

Optimal Control of Hybrid Systems Table of Contents 1. Modeling of Hybrid Systems 2. Optimal Control of Hybrid Systems 3. Model Predictive Control of Hybrid Systems 4. Explicit MPC of Hybrid Systems ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016

Optimal Control of Hybrid Systems Optimal Control for Hybrid Systems: General Formulation Consider the CFTOC problem: N 1 J (x(t)) = min p(x N ) + q(x k, u k, δ k, z k ), U 0 k=0 x k+1 = Ax k + B 1 u k + B 2 δ k + B 3 z k E s.t. 2 δ k + E 3 z k E 4 x k + E 1 u k + E 5 x N X f x 0 = x(t) where x R nc {0, 1} n b, u R mc {0, 1} m b, y R pc {0, 1} p b, δ {0, 1} r b and z R rc and U 0 = {u 0, u 1,..., u N 1 } Mixed Integer Optimization Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 2-24

Optimal Control of Hybrid Systems Mixed Integer Linear Programming Consider the following MILP: inf [zc,z b ] subj. to c cz c + c b z b + d G c z c + G b z b W z c R sc, z b {0, 1} s b where z c R sc, z b {0, 1} s b MILP are nonconvex, in general. For a fixed z b the MILP becomes a linear program: inf [zc,z b ] c cz c + (c b z b + d) subj. to G c z c W G b z b z c R sc Brute force approach to solution: enumerating the 2 s b integer values of the variable z b and solve the corresponding LPs. By comparing the 2 s b optimal costs one can find the optimizer and the optimal cost of the MILP. Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 2-25

Optimal Control of Hybrid Systems Mixed Integer Quadratic Programming Consider the following MIQP: inf [zc,z b ] subj. to where H 0, z c R sc, z b {0, 1} s b. MIQP are nonconvex, in general. 1 2 z Hz + q z + r G c z c + G b z b W z c R sc, z b {0, 1} s b z = [z c, z b ], s = s c + s d For a fixed integer value z b of z b, the MIQP becomes a quadratic program: inf [zc] subj. to 1 2 z ch c z c + q cz + k G c z c W G b z b z c R sc Brute force approach to the solution: enumerating all the 2 s b integer values of the variable z b and solve the corresponding QPs. By comparing the 2 s b optimal costs one can derive the optimizer and the optimal cost of the MIQP. Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 2-26

Optimal Control of Hybrid Systems Branch and Bound (B&B) A common solution method for MIPs, based on relaxations of binaries: {0, 1} [0, 1]. 4.5 Mixed-Integer Optimization Solvers 71 x d,1 =0 relaxed LP LB=10 10 x d,1 =1 x d,2 =0 13 12 x d,2 =1 x d,2 =0 x d,2 =1 19 16 14 infeasible 19 > UB 16 > UB x d,3 =0 x d,3 =1 optimal solution 15 17 UB=15 15 < UB = 17 Figure 4.7 Illustration of Branch and Bound procedure. Optimal cost of a solution to a modified problem where some binaries are relaxed is a lower bound on optimal cost. Example 4.6 Illustrative Example for Branch and Bound Procedure We refer Any feasible to Figure solution 4.7. AssumethatwearesolvinganMILPwiththree0-1integervariables to original problem is an upper bound on optimal cost. xd,1,xd,2,xd,3. Use bounds to rule out parts of the B&B tree systematically - much more efficient than brute force search. 1. In a first step we relax the integer constraints, i.e., we allow the variables xd,1,xd,2,xd,3 to vary continuously between 0 and 1. We find the minimum Hybrid Model Predictive Control of the Relaxed LP to F. beborrelli, 10, which M. Morari, must bec. ajones lower- bound Spring Semester on the solution. 2016 2-27

Model Predictive Control of Hybrid Systems Table of Contents 1. Modeling of Hybrid Systems 2. Optimal Control of Hybrid Systems 3. Model Predictive Control of Hybrid Systems 4. Explicit MPC of Hybrid Systems ybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016

Model Predictive Control of Hybrid Systems Model Predictive Control of Hybrid Systems MPC solution: Optimization in the loop As for linear MPC, at each sample time: Measure / estimate current state x(t) Find the optimal input sequence for the entire planning window N : U t = {u t, u t+1,..., u t+n 1 } Implement only the first control action u t Key difference: Requires online solution of an MILP or MIQP Hybrid Model Predictive Control F. Borrelli, M. Morari, C. Jones - Spring Semester 2016 3-28