On the Inherent Robustness of Suboptimal Model Predictive Control

On the Inherent Robustness of Suboptimal Model Predictive Control James B. Rawlings, Gabriele Pannocchia, Stephen J. Wright, and Cuyler N. Bates Department of Chemical & Biological Engineering Computer Sciences Department Department of Civil and Industrial Engineering, Univ. of Pisa, Italy OMPC 2013 SADCO Summer School and Workshop on Optimal and Model Predictive Control Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 1 / 34

Outline 1 Robustness of stability: overview and literature review Optimal and Suboptimal MPC (nominal) Disturbances and Robustness 2 New Results on Inherent Robustness 3 Analysis of a Troublesome Example 4 Conclusion Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 2 / 34

Overview and objectives Nominal stability of constrained nonlinear systems x + = f (x, u), x X, u U in closed-loop with optimal MPC, u = κ N (x), can be proved by Lyapunov arguments (Mayne, Rawlings, Rao, and Scokaert, 2000). Issues Optimal control problem must be solved exactly at each decision time Can perturbations (e.g. process disturbances or measurement noise) destroy stability? Objectives General and implementable suboptimal MPC Inherent robustness questions (recursive feasibility and stability) Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 3 / 34

Robustness and MPC: literature review (1/2) Robust MPC synthesis Perturbations are considered explicitly by design (e.g. (Bemporad and Morari, 1999), (Rawlings and Mayne, 2009, Ch. 3) and refs. therein). Robust MPC formulations (usually numerically tractable only for linear systems) tend to be conservative to preserve recursive feasibility Inherent robustness of MPC A difficult problem that received less attention (De Nicolao, Magni, and Scattolini, 1996; Scokaert, Rawlings, and Meadows, 1997; Grimm, Messina, Tuna, and Teel, 2004) Grimm et al. (2004) showed examples of nonlinear systems for which arbitrarily small disturbances destroy asymptotic stability Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 4 / 34

Robustness and MPC: literature review (2/2) Recursive feasibility and restricted constraints Grimm, Messina, Tuna, and Teel (2007) presented conditions to ensure recursive feasibility by adopting a constraint tightening approach (Limón Marruedo, Álamo, and Camacho, 2002). ISS and suboptimal MPC Inherent robustness of suboptimal MPC was first addressed in (Lazar and Heemels, 2009) They showed Input-to-State-Stability (ISS) of the equilibrium provided that a sub-optimality margin is guaranteed Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 5 / 34

The basic nonlinear, constrained MPC problem The (nonlinear) system model is x + = f (x, u) (1) Only the input is subject to constraints (state constraints are soft) u(k) U for all k I 0 Given an integer N (referred to as the finite horizon), and an input sequence u of length N, u = {u(0), u(1),..., u(n 1)} Let φ(k; x, u) denote the solution of (1) at time k for a given initial state x(0) = x. Terminal state constraint (and penalty) φ(n; x, u) X f Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 6 / 34

Feasible sets The set of feasible initial states and associated control sequences Z N = {(x, u) u(k) U, and φ(n; x, u) X f } and X f is the feasible terminal set. The set of feasible initial states is X N = {x R n u U N such that (x, u) Z N } (2) For each x X N, the corresponding set of feasible input sequences is U N (x) = {u (x, u) Z N } Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 7 / 34

Cost function and control problem For any state x R n and input sequence u U N, we define V N (x, u) = N 1 k=0 l(φ(k; x, u), u(k)) + V f (φ(n; x, u)) l(x, u) is the stage cost; V f (x(n)) is the terminal cost Consider the finite horizon optimal control problem P N (x) : min V N (x, u) u U N Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 8 / 34

Suboptimal MPC May not be able to solve online P N (x) exactly, so we consider using any suboptimal algorithm having the following properties. Let u U N (x) denote the (suboptimal) control sequence for the initial state x, and let ũ denote a warm start for the successor initial state x + = f (x, u(0; x)), obtained from (x, u) by ũ := {u(1; x), u(2; x),..., u(n 1; x), u + } (3) u + U is any input that satisfies the invariance conditions of Assumption 4 for x = φ(n; x, u) X f, i.e., u + κ f (φ(n; x, u)). Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 9 / 34

Suboptimal MPC The warm start satisfies ũ U N (x + ). The suboptimal input sequence for any given x + X N is defined as any u + U N that satisfies: u + U N (x + ) V N (x +, u + ) V N (x +, ũ) (4a) (4b) V N (x +, u + ) V f (x + ) when x + rb (4c) in which r is a positive scalar sufficiently small that rb X f. Condition (4b) ensures that the computed suboptimal cost is no larger than that of the warm start. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 10 / 34

Extended state Since the suboptimal algorithm requires a measured state and warm start pair, we define the extended state by z = (x, ũ) The procedure to generate the next warm start is ũ = {u(1), u(2),..., u(n 1), κ f (φ(n; x, u))} (5) The extended system evolves as z + H(z) = {(x +, ũ + ) x + = f (x, u(0)), where ζ( ) is the mapping corresponding to (5) ũ + = ζ(x, u), u U r (z)} (6) Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 11 / 34

Stability definitions Definition A function σ : R 0 R 0 belongs to class K if it is continuous, zero at zero, and strictly increasing; A function β : R 0 I 0 R 0 belongs to class KL if it is continuous and if, for each t 0, β(, t) is a class K function and for each s 0, β(s, ) is nonincreasing and satisfies lim t β(s, t) = 0. Definition (Asymptotic stability) We say the origin of the difference inclusion z + H(z) is asymptotically stable on the positive invariant set Z if there exists a function β KL such that for any z Z, all solutions ψ(k; z) satisfy ψ(k; z) β( z, k) k I 0 Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 12 / 34

Lyapunov function Definition (Lyapunov function) V is a Lyapunov function on the positive invariant set Z for the difference inclusion z + H(z) if there exists functions α 1, α 2, α 3 K such that for all z Z α 1 ( z ) V (z) α 2 ( z ) max V z + H(z) (z+ ) V (z) α 3 ( z ) Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 13 / 34

Basic assumptions for MPC Assumption (1 Continuity of system and stage cost) The model f : R n R m R n, stage cost l : R n R m R 0 and terminal cost V f : R n R 0 are continuous. Furthermore, for some steady state (x s, u s ), we have l(x s, u s ) = 0 and V f (x s ) = 0. We assume without loss of generality that (x s, u s ) = (0, 0). Assumption (2 Stage cost bound) There exists a function α l K such that α l ( (x, u) ) l(x, u) (x, u) Z Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 14 / 34

Constraint sets, terminal control law, and terminal region Assumption (3 Properties of constraint set) The set U is compact and contains the origin. The set X f is defined by X f = lev α V f = {x R n V f (x) α}, with α > 0. Assumption (4 Stability assumption) There exists a terminal control law κ f : X f U such that f (x, κ f (x)) X f V f (f (x, κ f (x))) V f (x) l(x, κ f (x)) x X f x X f Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 15 / 34

Nominal stability result Theorem (Nominal Asymptotic Stability of Suboptimal MPC) Under Assumptions 1 4, there exists β( ) KL such that φ(k; z) β( x, k) for any initial extended state z = (x, ũ) Z r. Outline of proof: Establish that V N (x, ũ) is a Lyapunov function for the extended system z + H(z). Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 16 / 34

So far so good; now is the stability robust? Consider disturbances to the process (d) and state measurement (e). x + = f (x, κ N (x)) x + = f (x, κ N (x + e)) + d nominal system nominal controller with disturbances Study of inherent robustness motivated by Teel (2004) who showed examples for which arbitrarily small perturbations can destabilize the nominally stabilizing controller. Kellet and Teel (2004) establish that for x + = f (x) with f locally bounded, a compact invariant set is robustly asymptotically stable if and only if the system admits a continuous global Lyapunov function. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 17 / 34

Effect of disturbances The closed-loop state and measurement evolutions are x + = f (x, κ N (x + e)) + d x + m = f (x m e, κ N (x m )) + d + e + where x m = x + e is the measured state and d is the additive process disturbance. Note that the suboptimal control law is now calculated based on the measured state u = κ N (x m, ũ). The results are simpler to state using the measurement system evolution The perturbed extended system then evolves as z + m H ed (z m ) = {(x + m, ũ + ) x + m = f (x m e, u(0)) + d + e +, where ζ( ) is the mapping corresponding to (5) ũ + = ζ(x m, u), u U r (z m )} (7) Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 18 / 34

Desired robustness property Definition (Robust asymptotic stability Teel (2004)) The origin of the closed-loop system (7) is robustly asymptotically stable (RAS) on C if there exists δ > 0, β KL, and σ d, σ e K such that for each x m C and for all ũ Ũ r (x m ), and for all disturbance sequences d and e satisfying d, e δ, we have that for all k I 0. φ ed (k; x m, ũ) β( x m, k) + σ d ( d k 1 ) + σ e ( e k ) (8) Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 19 / 34

Behavior with and without disturbances x0 Nominal System x + = f (x, u) u = κ N (x) Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 20 / 34

Behavior with and without disturbances Asymptotic robust invariant set Shrinks to zero with (d, e) x0 x0 Nominal System System with Disturbance x + = f (x, u) u = κ N (x) x + = f (x, u) + d u = κ N (x + e) d is the process disturbance e is the measurement disturbance Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 20 / 34

New result Theorem (Robust Asymptotic Stability of Suboptimal MPC) Under Assumptions 1 4 the origin of the perturbed closed-loop system (7) is RAS on any compact subset of X N This result is an improvement on Pannocchia, Rawlings, and Wright (2011). The nominal controller is completely unchanged here. For the optimal case, we have not (explicitly) assumed anything about continuity of VN 0 (x) here. Yu, Reble, Chen, and Allgöwer (2011) first to point out continuity of VN 0 (x) not required. See also Lazar and Heemels (2009) for robustness of suboptimal MPC on hybrid systems. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 21 / 34

A troublesome example [ x1 x 2 x + = f (x, u) ] + [ ] [ ] x1 u = + u 3 Two state, single input example. The origin is the desired steady state: u = 0 at x = 0. Cannot be stabilized with continuous feedback u = κ(x). x 2 Because (u, u 3 ) have the same sign, must use negative u to stabilize first quadrant. Must use positive u to stabilize third quadrant. But u cannot pass through zero or that point is a closed-loop steady state. Therefore discontinuous feedback. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 22 / 34

And its troubled history Introduced by Meadows, Henson, Eaton, and Rawlings (1995) to show MPC control law and optimal cost can be discontinuous. Based on a CT example by Coron (1990). Grimm, Messina, Tuna, and Teel (2005) established robustness for MPC with horizon N 4 with a terminal cost and no terminal region constraint. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 23 / 34

MPC with terminal equality constraint Because we do not know even a local controller, we try a terminal constraint x(n) = 0 in the MPC controller. For what initial x is this constraint feasible? (x 1 (1), x 2 (1)) = (x 1 (0), x 2 (0)) + (u 0, u 3 0) (x 1 (2), x 2 (2)) = (x 1 (1), x 2 (1)) + (u 1, u 3 1) (x 1 (3), x 2 (3)) = (x 1 (2), x 2 (2)) + (u 2, u 3 2) For N = 1, the feasible set X 1 is only the line x 2 = x 3 1. For N = 2, to have real roots u 0, u 1, we require 3x 4 1 + 12x 1x 2 0 which defines X 2 For N = 3, we have X 3 is all of R 2. So the shortest horizon that can globally stabilize the system is N = 3. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 24 / 34

Feasibility sets X 1, X 2, and X 3 0.25 0.2 0.15 0.1 0.05 X 3 = R 2 X 2 X 1 x 2 0-0.05-0.1-0.15-0.2-0.25-1 -0.5 0 0.5 1 x 1 Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 25 / 34

Structure of Feasibility Sets X f X N 2 X N 1 X N The feasibility sets are nested: X N X N 1 X N 2 X f The set X N is forward invariant. Important for recursive feasibility of controller. The set X N 1 is also forward invariant! The sets X N 2, X N 3,..., X f are not necessarily forward invariant. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 26 / 34

Optimal MPC with N = 3 The control constraint set U N (x) and optimal control κ N (x) for x on the unit circle is given by the following (Rawlings and Mayne, 2009, p. 105) 2 1.5 1 0.5 u 0 0-0.5-1 -1.5-2 -1-0.5 0 θ 0.5 1 π Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 27 / 34

Optimal cost function with N = 3 The discontinuity in the optimal cost for x on the unit circle 10 V 0 3 1-1 -0.5 0 θ 0.5 1 π Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 28 / 34

Where is V 0 N discontinuous? From the theory of cubic equations, we know that changes in the number of real roots to the equation au 3 + bu 2 + cu + d = 0 are determined by the sign of the discriminant = 18abcd 4b 3 d + b 2 c 2 4ac 3 27a 2 d 2 For our system, substituting a = 3, b = 3x 1, c = 3x 2 1, and d = x 3 1 + 4x 2 into the expression for the discriminant and factoring gives = 432( x 6 1 + 10x 3 1 x 2 9x 2 2 ) = 432(x 3 1 9x 2 )(x 3 1 x 2 ) Setting = 0 gives two (x 1, x 2 ) lines at which the number of real roots changes from one to three. The line that generates a discontinuity in V3 0 (x) corresponds to setting the first factor to zero giving x 2 = (1/9)x1 3 Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 29 / 34

Where is VN 0 discontinuous? Set D. 0.25 x 2 0.2 X 2D 0.15 0.1 0.05 0-0.05-0.1-0.15-0.2-0.25-1 -0.5 0 0.5 1 Note that invariant set X 2 and discontinuity set D do not intersect (the origin is not an element of D). But they approach each other at the origin. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 30 / 34 x 1

Robustness result for troublesome example RGAS follows because the nominal invariant set and set of discontinuities of VN 0 (x) do not intersect Outline of proof: For x large, X 2 is far from D, and (continuous) Lyapunov function argument applies. For x small, cost can increase due to interaction of discontinuity in VN 0 (x) and nonzero disturbance. But cost increase is small because x and hence VN 0 (x) are small. These two together give an asymptotic robust invariant set that shrinks to zero with the size of disturbances. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 31 / 34

Conclusion Suboptimal MPC with a well chosen warm start has the same inherent robustness properties as optimal MPC. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 32 / 34

Conclusion Suboptimal MPC with a well chosen warm start has the same inherent robustness properties as optimal MPC. Robust stability of nominal MPC extended to compact subset of feasible set (X N ) with no changes to the MPC controller. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 32 / 34

Conclusion Suboptimal MPC with a well chosen warm start has the same inherent robustness properties as optimal MPC. Robust stability of nominal MPC extended to compact subset of feasible set (X N ) with no changes to the MPC controller. Recall: no state constraints. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 32 / 34

Conclusion Suboptimal MPC with a well chosen warm start has the same inherent robustness properties as optimal MPC. Robust stability of nominal MPC extended to compact subset of feasible set (X N ) with no changes to the MPC controller. Recall: no state constraints. The control law and optimal cost may be discontinuous on X N. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 32 / 34

Conclusion Suboptimal MPC with a well chosen warm start has the same inherent robustness properties as optimal MPC. Robust stability of nominal MPC extended to compact subset of feasible set (X N ) with no changes to the MPC controller. Recall: no state constraints. The control law and optimal cost may be discontinuous on X N. Still no general analysis tools for discontinuous optimal cost and terminal equality constraint. But the example shows robustness for even this case. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 32 / 34

Conclusion Suboptimal MPC with a well chosen warm start has the same inherent robustness properties as optimal MPC. Robust stability of nominal MPC extended to compact subset of feasible set (X N ) with no changes to the MPC controller. Recall: no state constraints. The control law and optimal cost may be discontinuous on X N. Still no general analysis tools for discontinuous optimal cost and terminal equality constraint. But the example shows robustness for even this case. Exploits empty intersection of invariant set X N 1 (not X N ) and discontinuous set. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 32 / 34

Conclusion Suboptimal MPC with a well chosen warm start has the same inherent robustness properties as optimal MPC. Robust stability of nominal MPC extended to compact subset of feasible set (X N ) with no changes to the MPC controller. Recall: no state constraints. The control law and optimal cost may be discontinuous on X N. Still no general analysis tools for discontinuous optimal cost and terminal equality constraint. But the example shows robustness for even this case. Exploits empty intersection of invariant set X N 1 (not X N ) and discontinuous set. Future work: extend robustness results to economic MPC. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 32 / 34

Further reading I A. Bemporad and M. Morari. Control of systems integrating logic, dynamics, and constraints. Automatica, 35:407 427, 1999. J.-M. Coron. A necessary condition for feedback stabilization. Sys. Cont. Let., 14:227 232, 1990. G. De Nicolao, L. Magni, and R. Scattolini. Stabilizing nonlinear receding horizon control via a nonquadratic penalty. In Proceedings IMACS Multiconference CESA, volume 1, pages 185 187, Lille, France, 1996. G. Grimm, M. J. Messina, S. E. Tuna, and A. R. Teel. Examples when nonlinear model predictive control is nonrobust. Automatica, 40:1729 1738, 2004. G. Grimm, M. J. Messina, S. E. Tuna, and A. R. Teel. Model predictive control: For want of a local control Lyapunov function, all is not lost. IEEE Trans. Auto. Cont., 50(5):546 558, 2005. G. Grimm, M. J. Messina, S. E. Tuna, and A. R. Teel. Nominally robust model predictive control with state constraints. IEEE Trans. Auto. Cont., 52(10):1856 1870, October 2007. C. M. Kellet and A. R. Teel. Discrete-time asymptotic controllability implies smooth control-lyapunov function. Sys. Cont. Let., 52:349 359, 2004. M. Lazar and W. P. M. H. Heemels. Predictive control of hybrid systems: Input-to-state stability results for sub-optimal solutions. Automatica, 45(1):180 185, 2009. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 33 / 34

Further reading II D. Limón Marruedo, T. Álamo, and E. F. Camacho. Input-to-state stable MPC for constrained discrete-time nonlinear systems with bounded additive disturbances. In Proceedings of the 41st IEEE Conference on Decision and Control, pages 4619 4624, Las Vegas, Nevada, December 2002. D. Q. Mayne, J. B. Rawlings, C. V. Rao, and P. O. M. Scokaert. Constrained model predictive control: Stability and optimality. Automatica, 36(6):789 814, 2000. E. S. Meadows, M. A. Henson, J. W. Eaton, and J. B. Rawlings. Receding horizon control and discontinuous state feedback stabilization. Int. J. Control, 62(5):1217 1229, 1995. G. Pannocchia, J. B. Rawlings, and S. J. Wright. Conditions under which suboptimal nonlinear MPC is inherently robust. Sys. Cont. Let., 60:747 755, 2011. J. B. Rawlings and D. Q. Mayne. Model Predictive Control: Theory and Design. Nob Hill Publishing, Madison, WI, 2009. 576 pages, ISBN 978-0-9759377-0-9. P. O. M. Scokaert, J. B. Rawlings, and E. S. Meadows. Discrete-time stability with perturbations: Application to model predictive control. Automatica, 33(3):463 470, 1997. A. R. Teel. Discrete time receding horizon control: is the stability robust. In Marcia S. de Queiroz, Michael Malisoff, and Peter Wolenski, editors, Optimal control, stabilization and nonsmooth analysis, volume 301 of Lecture notes in control and information sciences, pages 3 28. Springer, 2004. S. Yu, M. Reble, H. Chen, and F. Allgöwer. Inherent robustness properties of quasi-infinite horizon MPC. In 18th IFAC World Congress, Milan, Italy, pages 179 184, Sep. 2011. Rawlings/Pannocchia/Wright/Bates Inherent robustness of suboptimal MPC 34 / 34