Whither discrete time model predictive control?

Transcription

1 TWCCC Texas Wisconsin California Control Consortium Technical report number Whither discrete time model predictive control? Gabriele Pannocchia Dept. of Civil and Industrial Engineering (DICI) Univ. of Pisa (Italy) James B. Rawlings Dept. of Chemical and Biological Engineering Univ. of Wisconsin, Madison (USA) David Q. Mayne Dept. of Electrical & Electronic Engineering Imperial College London (UK) Giulio M. Mancuso United Technologies, Rome (Italy) April 23, 214 This report is an expanded version of the journal paper [2] conditionally accepted as Technical Note in the IEEE Transactions on Automatic Control. Author to whom correspondence should be addressed

2 TWCCC Technical Report Abstract This paper proposes an efficient computational procedure for the continuous time, input constrained, infinite horizon, linear quadratic regulator problem (CLQR). To ensure satisfaction of the constraints, the input is approximated as a piecewise linear function on a finite time discretization. The solution of this approximate problem is a standard quadratic program. A novel lower bound on the infinite dimensional CLQR problem is developed, and the discretization is refined until a user supplied error tolerance on the CLQR cost is achieved. The offline storage of the matrices required for the solution of the model and adjoint equations, integration of the cost function, and computation of the cost gradient at several levels of discretization tailor the method for online use as required in model predictive control (MPC). The performance of the proposed algorithm is then compared with the standard discrete time algorithms used in most industrial model predictive control implementations. The proposed method is shown on two examples to be significantly more efficient than standard discrete time MPC that uses a sample time short enough to generate a cost close to the CLQR solution. 1 INTRODUCTION In this paper we are concerned with the infinite horizon, continuous time optimal control problem for a linear system subject to input bounds. For brevity, we refer to this as the CLQR (constrained linear quadratic regulator) problem. It is perhaps the simplest optimal control problem of significant interest after the classical, unconstrained LQR. One of the compelling features of both LQR and CLQR is the guarantee of nominal, closedloop stability that they provide for unconstrained and input constrained linear systems, respectively. Model predictive control, which is based on implementing solutions to optimal control problems as state measurements (or state estimates) become available, is arguably the most important advanced industrial control design method in use today. Although theoretical research on nonlinear MPC has reached at least a respectable level of completeness, almost all industrial applications remain based on linear models. Research on linear MPC therefore remains useful and important. Besides linearity, however, another feature of almost all industrial MPC methods is the use of discrete time models. It is this use of discrete time that we would like to examine in this paper. Is it necessary? Is it convenient? Is it as good as, or even better than, continuous time? For what reasons and in what ways? To provide a sound basis for comparison, we first develop and present a new algorithm for solving the CLQR problem. The problem is doubly infinite dimensional, first because the input is a continuous time function, and second because the cost function is defined on an infinite horizon. We show that neither feature causes insurmountable computational difficulties, and we can solve this problem reasonably efficiently with a guarantee of proximity to optimality.

3 TWCCC Technical Report The paper is organized as follows. In Section 2, we develop the basic numerical discretization of the continuous time problem using a piecewise linear input parameterization. In Section 3, we present quadrature formulas, based on matrix exponentiation, that can be computed and stored offline for fast, repetitive online calculation, as is required in MPC. These formulas allow us to efficiently solve the model and adjoint differential equations, evaluate the cost function, and its gradient with respect to the input. Because the original CLQR problem is (strictly) convex, we are able to develop a novel lower bound on the optimal cost; this is presented in Section 4. This lower bound enables a stopping criterion that meets a user specified proximity to optimality. In Section 5, we propose an algorithm for refining the discretization to solve the CLQR problem and discuss stopping criteria. Next in Section 6, we show that this algorithm converges. In Section 7, we provide numerical examples that show the algorithm can be solved quickly, and we briefly explore how it scales with state dimension. Notice that the usual discrete time issue of scaling with respect to horizon length is rendered moot because of the infinite horizon. Finally in Section 8 we draw conclusions of the study. At the end of the conclusions, we enumerate a number of issues that should be examined, or perhaps reexamined, in light of these new results when choosing between discrete time and continuous time in formulating MPC algorithms for industrial applications with linear models. An earlier, condensed version of this paper was presented at the 213 European Control Conference [19]. Some of the changes in this paper include the following: (i) improved second order, instead of first order, lower bounding functions, (ii) a revised and improved adaptive interval partitioning scheme, (iii) a convergence proof that does not require bisecting the largest interval, (iv), closed-loop simulations and comparison with discrete time MPC algorithms, and (v) larger scale examples. Notation The symbols R and Z denote the fields of reals and integers, respectively. Given two reals (integers) a, b with a < b, R a:b (Z a:b ) denotes all reals (integers) x such that a x b. The symbol is the transpose operator. All vector inequalities are meant component wise. Given two vectors x, y R n, (x, y) [ x y ] denotes the composite vector, x, y denotes the inner product, x x, x denotes the Euclidean norm, and x 2 Q x, Qx. Given a matrix A and positive integers a, b, c, d, the symbol A a:b,c:d denotes the submatrix with rows a to b and columns c to d, and A a:b,: denotes the submatrix of rows a to b and all columns. I m is the identity matrix of dimension m m (the dimension is omitted if not necessary). Given two symmetric matrices A, B of the same dimensions, the relation A > B (A B) means that A B is positive definite (positive semi-definite); λ min (A) is the smallest eigenvalue of A. Given a set S, int(s) denotes its interior.

4 TWCCC Technical Report PRELIMINARIES 2.1 Continuous time optimal control problem In this paper we address the computation of the optimal solution to the continuous time, infinite horizon, input constrained, linear quadratic regulation problem: { } P (x) : V (x, u( )) l(x(t), u(t))dt, s.t. x() = x, (1a) inf u( ) ẋ = f(x, u) Ax + Bu, for all t [, ), and (1b) u( ) U, in which U is the class of measurable controls defined on [, ) and taking values in U = m i=1 U i, where U i [u min i, u max i ], int(u), which is a compact, convex subset of R m. The state x R n, and the function l( ) is quadratic: l(x, u) 1 2 (x Qx + u Ru). Assumption 1. The pair (A, B) is stabilizable and (Q 1/2, A) is observable. Q and R >. Remark 2. Observability of (Q 1/2, A) can be relaxed to detectability. To clarify this point, assume without loss of generality that matrices (A, B, C), with C = Q 1/2 are in observability canonical form, i.e., the system of problem P (x) evolves as: ] [ ] [ ] [ ] [ẋ1 A11 x1 B1 = + u, y = [ C A 21 A 22 x 2 B 1 ] [ ] x 1, (2) 2 x 2 ẋ 2 in which (A 11, C 1 ) is observable, A 22 is Hurwitz, and x 2 is the unobservable mode. In fact, the evolution of x 1 does not depend on x 2 and the cost function can be expressed as: l(x, u) = 1 2 (y y + u Ru) = 1 2 (x 1C 1C 1 x 1 + u Ru), which shows that the cost function is not affected by substate x 2. Therefore, it can be removed from (2), reducing the system matrices to (A 11, B 1 ) and Q = C 1 C 1 with (Q 1/2, A 11 ) observable, without altering problem P (x). All the results in the paper, such as closed-loop stability, then apply to the (x 1, u) system. But since x 2 satisfies ẋ 2 = A 22 x 2 + [ ] [ ] x A 21 B 1 2 u and A 22 is Hurwitz, stability of the (x 1, u) system implies x 2 converges to zero and, therefore, stability of the (x, u) system as well. We define X as the set of initial states x for which there exists u( ) U such that V (x, u( )) is finite. Thus V : X U R. Existence and uniqueness of a solution to P (x) for each x X is established after (6). (1c)

5 TWCCC Technical Report In order to rewrite the infinite horizon problem P(x) as an equivalent finite horizon problem, we define a suitable ellipsoid invariant set as follows. Let P be the unique symmetric positive definite solution to the (continuous time) algebraic Riccati equation (which exists under Assumption 1): = Q + A P + P A P BR 1 B P. (3) Given a positive scalar α, we consider the following set: X f {x R n x P x α}. (4) Clearly, X f is an invariant (ellipsoidal) set for the unconstrained closed-loop system: ẋ = Ax + Bu, u = Kx, with K = R 1 B P. Because U contains the origin in its interior, if α is sufficiently small, then for any x X f there holds Kx U. Hence, u(t) = Kx(t) remains feasible at all times with respect to the constraint (1c) once x(t) has entered X f. Let U be rewritten as {u R m Du d}. The largest admissible value of α can be found as follows. Algorithm 3 (Invariant set). Require: P, K, D, d. 1: Perform eigenvalue decomposition of P, i.e. P = SΛS. 2: Define M DKSΛ : Evaluate radius of largest sphere, y y r 2, s.t. My d. 4: return α r 2. Given T >, we replace P(x) by the following finite horizon optimal control problem: P T (x) : subject to x() = x and { T } min V T (x, u( )) l(x(t), u(t))dt + V f (x(t )), (5a) u( ) model (1b) and constraint (1c) for all t [, T ], (5b) V f (x) 1 2 x P x with P > computed from (3), U T is the class of measurable controls defined on [, T ] and taking values in the compact, convex set U. Thus, V T : R n U T R. P T (x) has a unique solution for any x R n [16, Thm. 14, Chapter 3]. Let u T ( ) be the (finite time) input trajectory solution to P T (x) and x T ( ) the associated (finite time) state trajectory. Proposition 4. For each x X, there exists T R > such that x T (T ) X f T T, and lim T x T (T ) =. Proof: We prove the result in three parts.

6 TWCCC Technical Report (i) There exists finite T such that x T ( T ) X f. Since (A, Q 1/2 ) is observable, there exists an L such that (A LC) is Hurwitz and the linear system ẋ = Ax + Bu can be rewritten as ẋ = (A LC)x + [ B L ] [ ] u y with y = Q 1/2 x. Observability of (A, Q 1/2 ) also implies that X f is compact, contains the origin in its interior, and there exists a > such that x not in X f implies x a. Since X f is forward invariant for ẋ = (A + BK)x, if x T (T ) is not in X f, then neither is x T (t) for all t [, T ]. So we assume contrary to what we wish to prove that there is no finite T such that x T (t) X f for t [, T ]. Lemma 15 (reported in Appendix) then applies and we know that (u T ( ), y T ( )) 2 as T. But this contradicts the fact that T l(x T (t), u T (t))dt = T ( yt (t) 2 + u T (t) 2 )dt is uniformly bounded for all T, and R the result is established. (ii). Given T with x T ( T ) X f, then x T (T ) X f for every T T. From optimality and the fact that X f is forward invariant for the unconstrained optimal control problem ẋ = (A + BK)x, we have that x T ( T ) = x T ( T ; x) for T T and the result follows. (iii). Convergence: x T (T ) as T. Let T > T. From (i) we know that x T ( T ) X f and from (ii) that x T (T ) X f. From optimality of the unconstrained control law, the evolution after T is given by x T (T ) = e(a+bk)(t T ) x T ( T ). Since (A + BK) is Hurwitz, x T (T ) as T. For x X, if T R is large enough that x T (T ) X f, since V f (x) is the optimal infinite horizon cost for any x X f, by the principle of optimality it follows that the (infinite time) input and state trajectories defined as: ( u ( ), x ( ) ) {( u T (t), x T (t)) if t [, T ], ( Ke (A+BK)(t T ) x T (T ), e(a+bk)(t T ) x T (T )) if t > T, (6) are, respectively, the minimizer of P (x) and its associated state trajectory. Thus, P (x) and P T (x) yield the same solution, i.e. V T (x) V T (x, u T ( )) = V (x) V (x, u ). From the above discussion it follows that P (x) has a unique solution for all x X. In discrete time, earlier but less general results on existence and uniqueness of solutions to P are available. For example, [7] treats only the case Q >, whose proof is only a few lines (see Assumption H1 of their paper). The authors of [27, 28] treat the general case, (A, Q 1/2 ) detectable, but do not include a proof of existence and uniqueness of the infinite horizon problem (see remark 4 in [28]). There is a rich literature on solution methods for finite horizon nonlinear constrained optimal control. In most approaches a piecewise constant input parameterization is considered and numerical discretization is deployed to derive and solve (approximate) optimality conditions (see e.g. [12, 5, 2, 3, 26, 13] and references therein). On the other hand, methods specifically tailored to constrained linear systems are less common, but some interesting results and methods can be found in [6, 14, 24, 1, 17, 25, 1, 8]. We remark that one distinguishing feature of our method is that computes a solution to P T (x) that is accurate to a

7 TWCCC Technical Report user defined tolerance. Furthermore, our method is based on the solution of strictly convex Quadratic Programming problems, for which reliable (off-the-shelf or tailored) algorithms exist, and has no specific restriction on the system (state and input) dimensions. Remark 5. We are assuming that the controlled system evolves as in (1b), i.e. the actuator hardware, if digital, is able to implement the continuous time input solution to P (x) without introducing noticeable discretization effects. 2.2 Input parameterizations For all T R >, let γ be a partition of the interval [, T ], defined as a sequence of J γ Z > intervals {I j [t j, t j+1 ] j Z :Jγ 1} such that = t < t 1 < < t Jγ = T. Let j t j+1 t j denote the length of I j ; we assume that each j satisfies j = 2 q j δ with q j Z and δ >, in which case we say that γ Γ T δ. In order to consider a finite parameterization of the function u : [, T ] R m, it is customary in sampled data control of continuous time systems (see, e.g. [31]) to assume that the input is constant in each interval I j, i.e., u(t) = u j for all t I j. (7) Formally, given a partition γ of [, T ], we define U γ,zoh T as the set of all functions u( ) = U T satisfying the zero-order hold (ZOH) parameterization (7) in which u j U for all j Z :Jγ 1. Since u( ) is piecewise constant it is measurable. Besides the fact that restricting u( ) to the set U γ,zoh T makes P T (x) finite dimensional, it also ensures that u(t) U for all t [, T ]. In [18] we argued that a better choice is to assume the input piecewise linear in each interval: u(t) = (1 η j (t))u j + η j (t)v j, for all t I j, with η j (t) t t j j. (8) Formally, given a partition γ of [, T ], we define U γ,pwlh T as the set of all functions u( ) U T satisfying the piecewise linear hold (PWLH) parameterization (8) in which (u j, v j ) U 2 for all j Z :Jγ 1. Notice that for all j Z :Jγ 1, we have that η j (t j ) = and η j (t j+1 ) = 1. Thus, if (u j, v j ) U 2, then u(t) U for all t I j, all j Z :Jγ 1. A variant to PWLH (8), also discussed in [18] and called forward first order hold (FFOH), enforces continuity of u( ) for all t [, T ] by adding to (8) the restriction: v j = u j+1 for all j Z :Jγ 2. (9) For the sake of brevity, this variant is not discussed in this paper. 2.3 Discretized optimal control problem Given a discretization γ and choosing either ZOH or PWLH, i.e. defining U γ U γ ZOH or U γ U γ PWLH, we can obtain a suboptimal solution to P T (x) by solving the following discretized optimal control problem: P γ T (x) : min V u( ) U γ T (x, u( )) subject to x() = x and model (1b). (1)

8 TWCCC Technical Report As discussed in the next section, we rewrite P γ T (x) as an equivalent discrete time CLQR problem and solve it via Quadratic Programming (QP). Then, under certain conditions we accept the achieved solution or we refine the discretization γ. In most existing approaches to solve CLQR (and general nonlinear optimal control) problems, the time discretization is uniform, but a number of methods exist which take advantage of (offline predetermined) non-uniform discretization schemes [18, 17, 23]. Such schemes can be equivalently interpreted as move blocking constraints in conventional discrete time MPC formulations [11]. 3 ODE SOLVER FREE DISCRETIZATION 3.1 LQR discretization for ZOH via matrix exponential Given an interval I j, assuming to use the ZOH parameterization (7), it is well known [15,31] that we can compute an equivalent discrete time system evolution as: in which x j x(t j ) and: Moreover, there holds: where in which l j (x j, u j ) x j+1 = A j x j + B j u j, (11) A j = e A j, B j = V T (x, u( )) = tj+1 [ Qj M j M j R j J γ 1 j= j e As Bds. (12) l j (x j, u j ) + V f (x(t )), (13) t j l(x, u)dt = 1 2 (x jq j x j + u jr j u j + 2x jm j u j ), (14) ] = j e [ A B ] s [ Q R [ ] ] A B s e ds. (15) The above formulas allow one to compute all matrices (A j, B j, Q j, R j, M j ) by solving a system of ordinary differential equations (ODE). However, Van Loan [29] showed that all of these matrices can be found by means of a single matrix exponentiation as follows. First, define a block upper triangular matrix C and partition its exponential as follow: Then, obtain: C [ A I A Q A B ], e Cτ F 1 (τ) G 1 (τ) H 1 (τ) K 1 (τ) F 2 (τ) G 2 (τ) H 2 (τ) F 3 (τ) G 3 (τ) F 4 (t) A j = F 3 ( j ), B j = G 3 ( j ), Q j = F 3( j )G 2 ( j ), M j = F 3( j )H 2 ( j ),. (16) R j = R j + [ B F 3( j )K 1 ( j ) ] + [ B F 3( j )K 1 ( j ) ]. (17)

9 TWCCC Technical Report LQR discretization for PWLH via matrix exponential Numerical experience shows that computation of (A j, B j, Q j, R j, M j ) for ZOH via matrix exponential formulas (16) (17) is faster and (typically) more accurate than via an ODE solver. We show here how a similar procedure can be implemented for PWLH. To this aim, in each interval I j, we can consider an augmented system with state z (z (1), z (2) ) R 2n, in which z (1) (t) x(t) and z (2) (t) u(t) u j = η j (t)(v j u j ), and constant input w j = (u j, v j ) R 2m. This augmented system evolves in I j as: If we set A [ ] A B, B ż = [ A B [ ] B Im I m j j ] z + [ B Im j ] I m j w j. (18), Q [ ] Q and we define C and its partitioned exponential as in (16) with (A, B, Q) replaced by (A, B, Q ), under PWLH (8) we obtain: z j+1 = A jz j + B j w j, where l j(z j, w j ) tj+1 l(x, u)dt = 1 2 (z jq jz j + w jr j w j + 2z jm j w j ), t j (19) A j = F 3 ( j ), B j = G 3 ( j ), Q j = F 3( j )G 2 ( j ), M j = F 3( j )H 2 ( j ), R j = (1/6) [ 2R R R 2R ] j + [ B F 3( j )K 1 ( j ) ] + [ B F 3( j )K 1 ( j ) ]. (2) Finally, by noticing that z (2) (t j ) =, in the discrete time evolution and cost function we can remove the component z (2) to obtain: V T (x, u( )) = where A j = A j 1:n,1:n, B j = B j 1:n,: and x j+1 = A j x j + B j w j, (21) J γ 1 j= l j (x j, w j ) + V f (x(t )), (22) l j (x j, w j ) = 1 2 (x jq j x j + w jr j w j + 2x jm j w j ), (23) in which, Q j = Q j 1:n,1:n, M j = Q j 1:n,:, and R j is defined in (2). We observe that in (21) the discrete time evolution of the system under PWLH is still described by a linear system with the original state x j and an augmented input w j = (u j, v j ). For the sake of brevity, from now on we focus solely on PWLH, but all derivations and results will apply directly to ZOH, which can be seen as a PWLH in which w j = (u j, u j ). Given the above premises, problem P γ T (x) can be rewritten as a conventional discrete time CLQR problem. Let u (w, w 1,..., w Jγ 1) be an augmented input sequence of

10 TWCCC Technical Report V ˆV (ν) = V (u) + (d/du)v (u)(ν u) + α(ν u) 2, with α < 1 2 H V (ν) = V (u) + (d/du)v (u)(ν u) V (u) V ˆV V VˆV V ū û u u U ν Figure 1: Illustrating the main idea of computing lower bounds on the optimal cost of the scalar function V (ν) = V (u) + (d/du)v (u)(ν u) H(ν u)2 in U, given a feasible point u. ˆV ( ) is the approximate second order lower bounding function, û is its minimizer and ˆV its minimum value. V ( ) is the first order lower bounding function, ū is its minimizer and V its minimum value. length J γ. Then, the discretized problem P γ T (x) can be equivalently written as: J P γ T (x) : V γ 1 γ T (x, u) l j (x j, w j ) + V f (x Jγ ), min u U 2Jγ j= subject to x = x and model (21). (24) 4 LOWER BOUNDS ON OPTIMAL COST OF P T (x) AND P γ T (x) Next we show how, exploiting the convexity of both P T (x) and P γ T (x), we can obtain a lower bound of the optimal cost of each problem, given any feasible input u( ). The main idea is graphically explained in Figure Lower bound of the continuous time optimal cost V T : R n U T R is defined by: V T (x, u( )) T l(xu (t; x)), u(t)dt + V f (x u (T ; x)), in which x u (t; x) is the solution of (1b) at time t given that the initial state is x at time

11 TWCCC Technical Report and the control is u( ) U T. Similarly, the cost due to another control ν( ) U T is V T (x, ν( )) T l(xν (t; x)), ν(t)dt + V f (x ν (T ; x)). Let u( ) ν( ) u( ) and let x( ) x ν ( ; x) x u ( ; x) for all t [, T ]. Because l(x, u) = 1 2 (x Qx + u Ru) and V f (x) = 1 2 x P x, we can write: V T (x, ν( )) V T (x, u( )) = T T ( x(t), Qx u (t; x) + u(t), Ru(t) )dt+ x(t ), P x u (T ) ( x(t), Q x(t) + u(t), R u(t) )dt x(t ), P x(t ). (25) The first order terms may be computed in the usual way [4, pp ]: T x(t), Qx u (t; x) + u(t), Ru(t) dt + x(t ), P x u (T ; x) = T u H(x u (t; x), u(t), λ u (t; x)), u(t) dt, in which the Hamiltonian H : R n R m R n R is defined by H(x, u, λ) l(x, u) + λ (Ax + Bu), and λ u (t; x) is the solution at time t of the adjoint system: λ(t) = A λ(t) + Qx u (t; x), λ(t ) = P x u (T ; x). Since Q and P >, for any R R {S R R}, we have: V T (x, ν( )) V T (x, u( )) T g(x, u( ))(t), (ν(t) u(t)) ν(t) u(t) 2 R dt, for all ν( ) U T, in which g( ) is defined by: g(x, u( ))(t) u H(x u (t; x), u(t), λ u (t; x)). We now define the optimality function [21] θ : R n U T R for problem P T (x) as: θ(x, u( )) = inf ν( ) U T T T g(x, u( ))(t), ν(t) u(t) ν(t) u(t) 2 R dt min v U { uh(x u (t; x), u(t), λ u (t; x)), v u(t) v u(t) 2 R }dt, where the last equality is established in Proposition 16 in the Appendix if the function L : [, T ] R m R in Proposition 16 is defined by L(t, v) u H(x u (t; x), u(t), λ u (t; x), v u(t) v u(t) 2 R Thus, for any ν( ) U T, we have: V T (x, ν( )) V T (x, u( )) θ(x, u( )). Hence we have proved: Proposition 6. For any (x, u( ), T ) R n U T R >, the following inequality holds: V T (x) V T (x, u( )) + θ(x, u( )). (26)

12 TWCCC Technical Report Lower bound of the optimal cost for a given partition In various stages of the algorithm described in 5, it is useful to have a lower bound on the optimal cost of the discretized problem P γ T (x). Given an input u( ) U γ T defined on a partition γ and its associated parameterization vector u = (w, w 1,..., w Jγ 1) U 2Jγ, then: J γ 1 V T (x, u( )) = V γ T (x, u) j= ( 1 2 x j, Q j x j w j, R j w j + x j, M j w j ) x J γ, P x Jγ. For another ν( ) U γ T, with associated parameterization vector ν = (ν, ν 1,..., ν Jγ 1) then: J γ 1 V γ T (x, ν) V γ T (x, u) = J γ j= j= ( x j, Q j x j + w j, R j w j + x j, M j w j )+ x Jγ, P x Jγ + ( x j, Q j x j + w j, R j w j + 2 x j, M j w j ) x J γ, P x Jγ. (27) The first order terms may be computed in the usual way [4, pp ]: J γ 1 j= ( x j, Q j x j + w j, R j w j + x j, M j w j ) + x Jγ, P x Jγ = g γ (x, u), ν u, in which g γ (x, u) (g γ (x, u), gγ 1 (x, u),..., gγ J γ 1 (x, u)) with: g γ j (x, u) w j H j (x j, w j, λ j+1 ) = M jx j + R j w j + B jλ j+1, j Z :Jγ 1, (28) where H j : R n R 2m R n R is the Hamiltonian defined by H j (x, w, λ) l j (x, w) + λ (A j x + B j w) and {λ, λ 1,..., λ Jγ } is the solution of the discrete time adjoint system: λ j = A jλ j+1 + M jw j + Q j x j, λ Jγ = P x Jγ. The second row of (27) consists of the second order terms. Forming the Schur complement, we note that: ( x j, Q j x j + w j, R j w j + 2 x j, M j w j ) w j, (R j M j Q 1 j M j ) w j. Since P >, it follows from (27) and (28) that, for all ν( ) and u( ) in U γ T : V γ T (x, ν) V γ T (x, u) gγ (x, u), ν u ν u, R (ν u), with R a block diagonal matrix formed by matrices Rj R j {S S R j M j Q 1 j M j }, j Z :Jγ 1. We now define the optimality function θ γ : R n U γ T R for (x) as: P γ T θ γ (x, u( )) J min g γ (x, u), ν u + 1 ν U 2Jγ 2 ν u, γ 1 R (ν u) = θ γ j (x, u( )), (29) θ γ j (x, u( )) min w U 2 gγ j (x, u), w w j w w j, Rj (w w j ). (3) Thus, for any ν U 2Jγ, we have: V γ T (x, ν) V γ T (x, u) θγ (x, u( )). Hence we have proved: j=

13 TWCCC Technical Report Proposition 7. For any (x, u( ), T ) R n U γ T R >, the following inequality holds: V γ, T (x) V T (x, u( )) + θ γ (x, u( )). (31) Finally, let θ δ (x, u( )) denote θ γ (x, u( )) for the special uniform partition γ δ Γ T δ in which each constituent interval has length δ. Recalling that for any partition in Γ T δ, all intervals I j have a length that is a multiple of δ (or at most equal to δ), we will refer to γ δ as the finest partition. 5 ALGORITHM: CONCEPTUAL DESIGN AND PRAC- TICAL IMPLEMENTATION 5.1 Conceptual algorithm As anticipated, we solve P γ T (x) repeatedly, refining γ at each iteration, until we obtain a satisfactory solution of P (x). We refer to γ Γ T δ as a refinement of γ Γ T δ if some of the intervals {Ĩj} defining γ are obtained by bisecting one or more intervals in the set {I j } that defines γ and if the remaining intervals in γ are the same as the corresponding ones in γ. If VT (x) are, respectively, the optimal value functions of P T (x) and P γ γ, T (x) then, clearly VT (x) V T (x), for all x Rn, all γ Γ T δ, all permissible δ (, T ). Moreover, if γ is a refinement of γ, it follows that V γ, γ, T (x) VT (x). We now state the (conceptual) optimization algorithm to solve P (x). (x) and V γ, T Algorithm 8 (Conceptual algorithm). Require: δ, ɛ >, γ Γ T δ, c (, 1), T, T >. 1: Solve P γ T (x) yielding control u( ) U γ T and state trajectory x( ). Compute θδ (x, u( )). 2: Refine γ (repeatedly if necessary) until θ γ (x, u( )) cθ δ (x, u( )). 3: If θ δ (x, u( )) ɛ, go to Step 1. Else, go to Step 4. 4: If x(t ) X f, define I Jγ = [T, T + T ], and γ {γ, I Jγ }, T T + T, J γ J γ : Replace ɛ ɛ/2, δ δ/2. Go to Step 1. A procedure for refining γ (repeatedly if necessary) is given in Section 5.2. In Step 5, ɛ ɛ/2 and δ δ/2 may be replaced, respectively, by ɛ c 1 ɛ and δ c 2 δ, with c 1 (, 1) and c 2 = (1/2) q (with q Z > ). Remark 9. The control u( ) U γ T obtained in Step 1 satisfies θγ (x, u( )) = ; if γ is the refined partition obtained in Step 2, and u( ) is not optimal for P γ T (x), then θ γ (x, u( )) <. Remark 1. T is increased in Step 4 if the (implicit) terminal constraint x(t ) X f is not satisfied. As shown later by Theorem 13, this step occurs only a finite number of iterations. 5.2 Refinement strategy Since the length of each interval in the current partition γ is an even multiple of the current δ and since the length of all intervals in the refined partition should also be a multiple of δ, the refinement strategy of Step 2 consists of bisecting each interval with length greater than or equal to 2δ and selecting a subset whose bisection satisfies the condition in Step 2.

14 TWCCC Technical Report Suppose the current partition γ consists of the intervals {I, I 1,..., I Jγ 1}. Because the current u( ) is optimal for P γ T (x), then θγ j (x, u( )) = for all j J γ Z :Jγ 1. If I j is bisected, yielding I j1 = [t j, t j1 ] and I j2 = [t j1, t j+1 ] with t j1 = t j+t j+1 2, let w j (u j, v j ) be replaced by w j1 = (u j, u j+v j 2 ) in I j1 and w j2 = ( u j+v j 2, v j ) in I j2, and let x j1 and λ j1 denote the value of x( ) (the current state trajectory) and λ( ) at time t j1. Then the gradients g γ j1 (x, u( )) and gγ j2 (x, u( )) of the cost with respect to w j1 and w j2 may be computed from (28) yielding: θ γ j (x, u( )) θ γ j1 (x, u( )) + θ γ j2 (x, u( )), (32) where θ γ j1 (x, u( )) and θ γ j2 (x, u( )) are defined as in (3), respectively, for I j1 and I j2. Notice that θ γ j (x, u( )) is a lower bound on the cost reduction obtainable by bisecting I j. Given a candidate set of intervals to be bisected, J J γ, we obtain: θ γ (x, u( )) = θ γ j J j (x, u( )). By ordering θ γ j (x, u( )) in ascending manner, i.e., from the most negative to the least negative, J is chosen as the subset of J γ with smallest cardinality such that the condition in Step 2 is satisfied by θ γ (x, u( )). If no such J can be found even if all intervals I j are bisected, i.e., if J = J γ, the procedure is repeated with γ replaced by the partition with every I j bisected. 5.3 Practical considerations and algorithm with stopping conditions The discrete time matrices appearing in the various steps of Algorithm 8 can be computed and stored offline for a (finite) number of possible interval sizes, in geometric sequence of ratio 2, using the formulas of 3. The minimization in (3) is analytic if Rj are chosen diagonal, due to the fact that U (and hence U 2 also) is a box constraint set. The choice of Rj diagonal is always possible, e.g. R j λ min(r j M j Q 1 j M j )I 2m is a valid choice because < Rj R j M j Q 1 j M j. For a general polytopic set U, the minimization in (3) is a small dimensional convex QP, namely in 2m decision variables. For a given δ, the loop in Steps 1-3 is always exited in a finite number of iterations because, otherwise, the refinement of γ would reach γ δ and then we would have θ γ (x, u( )) = θ δ (x, u( )) =, which makes the condition to proceed to Steps 4-5 true. However, as written, Algorithm 8 never terminates because it would keep entering Step 5, reducing δ (and ɛ) and then going to Step 1. A practical variant could terminate after Step 1 and return the computed solution u( ) when θ(x, u( )) ρ, for a given ρ >. By doing so, there is a guarantee that the achieved cost V T (x, u( )) satisfies: V T (x, u( )) VT (x) ρ. However, evaluation of θ(x, u( )) from (26) requires computing a numerical integral of the scalar function ψ : [, T ] R whose value at any time t [, T ] is ψ(t) min v U { uh(x u (t; x), u(t), λ u (t; x)), v u(t) v u(t) 2 R, i.e., θ(x, u( )) T ψ(t)dt. Notice that if R R is chosen as a diagonal matrix, the previous minimization can be performed analytically. Thus, evaluating θ(x, u( )) and ensuring an exact bound on the termination error is achievable, but for fast closed-loop implementations a simpler alternative is to terminate

15 TWCCC Technical Report after Step 1 when θ δ (x, u( )) ρ. (33) In this way the computed solution is guaranteed to satisfy V T (x, u( )) V δ, T (x) ρ, i.e., the solution to P γ T (x) is a ρ-close approximation to the problem Pγδ T (x) at the current finest partition γ δ. Notice that ρ should be chosen (significantly) smaller than the initial value of ɛ. 6 PROPERTIES OF THE ALGORITHM In this section we discuss the main properties of Algorithm The space of control and state trajectories In order to analyze the convergence properties, we notice that each u( ) U T lies in L p (T ) L p ([, T ], R m ) for all p {1, 2,..., }, in which L p ([, T ], R m ) = {u : [, T ] [ R m T 1/p u( ) measurable, u( ) p < }, with u( ) p dt] u(t) p and u( ) ess sup [,T ] u(t). The space L p (T ) is a Banach space. Moreover, the spaces L p (T ), p = 1, 2,..., are nested; i.e. p < q implies L q (T ) L p (T ). In fact, since [, T ] has finite measure T, for all u( ) U T u( ) p T (1/p 1/q) u( ) q, (34) so that u( ) q implies u( ) p for all p, q I 1, p < q and all u U T. It is also possible to show that, for all u( ) U T and all p, q I 1, u( ) p, u( ) U T, implies u( ) q. The next result follows from Theorem 3.1 in [9]. Theorem 11. For all (x, T ) R n R >, u T : [, T ] U is Lipschitz continuous. Since u( ) is bounded and T is finite, it follows that the solution x( ) = φ( ; x, u( )) of (1b) is absolutely continuous for any (x, u( )) R n U T. 6.2 Convergence of Algorithm 8 Let u i ( ), x i ( ), ɛ i, γ i, δ i and T i denote, respectively, the values of u( ), x( ), ɛ, γ, δ and T at iteration i of Algorithm 8, where i Z 1 increases each time Step 1 is executed. Proposition 12. The loop in Steps 1 3 is always exited in a finite number of iterations. We can now state the main result. Theorem 13. For each x X, there exists an i Z 1 and a T R > such that T i = T and x i (T i ) X f for all i i. Also V Ti (x, u i ( )) VT (x) = V (x) and u i ( ) u T ( ) in L p (T ) for all p Z 1 as i (with u T ( ) = u ( ) restricted to [, T ].)

16 TWCCC Technical Report Proof: Since the loop Steps 1 3 is always exited in a finite number of iterations, both ɛ i and δ i tend to as i. For each i Z 1, let ũ i ( ) U Ti denote the sample-hold version of u T i ( ) in a PWLH sense obtained by sampling u T i ( ) at times jδ i, j Z :(Ti /δ i 1); thus ũ i ( ) is defined as in (8) with u j u T i (jδ i ) and v j u T i (j + 1)δ i ). Let T denote the smallest time T such that x T (T ) X f. By Theorem 11, u ( ) is Lipschitz continuous in T [, T ] with Lipschitz constant κ 1 and is equal to u ( ) (the optimal control for P (x)) restricted to [, T ]. In [T, ), u ( ) is as defined in (6) with T = T ; since x ( ) is bounded in [T, ) (because x (t) X f, t [T, )), so is u ( ) in [T, ). Hence u ( ) is Lipschitz continuous with constant κ 2 in this interval. It follows that u T ( ) is Lipschitz continuous for all T [, ) with Lipschitz constant κ = max{κ 1, κ 2 }. Since δ i as i, {ũ i ( ) U δ i T i i Z 1 } is a sequence of controls satisfying ũ i ( ) u T i ( ) as i so that VT i (x) V Ti (x, ũ i ( )) as i. Let V δ i, T i ( ) be the optimal value function for P γδ i T i (x). Then VT i (x) V δ i, T i (x) V Ti (x, ũ i ( ))) for all i so that V δ i, T i (x) VT i (x) as i. The algorithm ensures V Ti (x, u i ( )) = V γ i, (x) V δ i, T i (x) for all i. Hence V T i (x) V δ i, T i (x) V Ti (x, u i ( )) V δ i, T i (x) θ δ i (x, u i ( )) T i where the last inequality follows from (31) with γ chosen as γ δ i. convergence of V δ i, T i (x) VT i (x) and of θ δ i (x, u i ( )) to zero, that It follows from the V Ti (x, u i ( )) V T i (x) as i. In (25) let ν( ) = u i ( ), u( ) = u T i ( ) and T = T i noting that VT i (x) = V Ti (x, u i ( )). The first order component (first line) of V Ti (x, ν( )) V Ti (x, u( )) is non-negative since both ν( ) = u i ( ) and u( ) = u T i ( ) lie in U Ti and u T i ( ) is optimal. Since V Ti (x, u i ( )) VT i (x), each term on the right hand side of (25), in particular, (1/2) x i (T i ) x T i (T i ) P, tends to zero as i. Hence x i (T i ) x T i (T i ) as i. From Proposition 4, it follows that x T i (T i ) as T i. Thus x i (T i ) as i if Step 4 is entered at every iteration of the algorithm. However, since T i increases by a finite amount each time Step 4 is entered, there exists a finite integer i such that x i (T i ) X f and T i = T for all i i (T i stops increasing only after x i (T i ) enters X f ). This also implies that x T i (T i ) X f for all i i. Since x T i (T i ) X f implies VT i (x) = V (x), it follows that V Ti (x, u i ( )) VT (x) = V (x) as i. It follows from (25), with ν( ) = u i ( ) and u( ) = u T ( ), and the non-optimality of ν( ) = u i ( ) that T (1/2) u i (t) u T (t) 2 Rdt V T (x, u i ( )) VT (x) for all i i so that u i ( ) u T ( ) in L 2(T ) (hence, in L p (T ) for any p Z > ) as i, where u T ( ) = u ( ) restricted to [, T ].

17 TWCCC Technical Report Remark 14. To prove this theorem, we do not have to assume that the largest interval in the partition goes to zero as i. 7 APPLICATION EXAMPLES 7.1 Introduction and performance indicators We present in this section some illustrative simulation results. Computations are performed in Matlab (R212b) on a MacBook Air (1.8 GHz Intel Core i7, 4 GB of RAM). The discretized CLQR problems P γ T (x) are solved using the function quadprog.m 1, in which both (augmented) input and state sequences ({w j } and {x j }) are the QP decision variables (see, e.g., [22]). Timing is measured with the functions tic and toc. We remark that with this formulation and solver, the solution time approximately scales linearly with the number of intervals J γ. All required discretized matrices are precomputed offline and stored in order to speed up the online computations. The following performance indicators are considered during the execution of Algorithm 8: number of intervals, J γ, at a given iteration; continuous time cost error bound, θ(x, u( )), at a given iteration; cumulative solution time (in ms) of all past and current iterations. In the computation of θ δ (x, u( )) given in (29) (3) we use R j λ min(r j M j Q 1 j M j ), and in the computation of θ(x, u( )) given in (26) we use R R (because in all examples we use continuous time R diagonal). 7.2 Example # 1: SISO open-loop stable system The first example is a three-state one-input system defined by the (continuous time) matrices: ] [.25 ] [ 1 ] A =, B = 2., Q = 1, R =.1. 1 [ We remark that the system matrix A has three stable eigenvalues (.1, 1 ± 4.899i). Given the initial state x() = [ ], we consider the first five iterations of Algorithm 8 in three different variants: using PWLH parameterization (8) and adaptive refinement as in 5.2; using PWLH parameterization (8) and fixed refinement in which all intervals are bisected; using ZOH parameterization (7) and fixed refinement in which all intervals are bisected. 1 With options: interior-point-convex algorithm, function tolerance of 1 12 and variable tolerance of 1 1.

18 TWCCC Technical Report Number of intervals, Jγ Solution time (ms) PWLH adapt. refin. PWLH fixed refin. ZOH fixed refin. PWLH adapt. refin. PWLH fixed refin. ZOH fixed refin Cost error bound, θ( ) Figure 2: Example 1: Performance indicators during the first five iterations of Algorithm 8 using PWLH with adaptive discretization refinement (red), PWLH with fixed discretization refinement (blue), ZOH with fixed discretization refinement (green). In all cases we use the following fixed parameters T = 1, c =.8, and initial values of δ =.125 and ɛ =.1. The storage of the required discretized matrices, for 11 different interval sizes, takes 7 kb. Comparative results are depicted in Figure 2 in which number of intervals and solution time are plotted against the continuous time cost error bound. By comparing adaptive and fixed refinement strategies (using the same PWLH parametrization) we immediately observe that the adaptive refinement allows Algorithm 8 to achieve smaller errors with far fewer intervals (top plot), and in turns this reduces the required computation time (bottom plot). By comparing PWLH and ZOH (using the same fixed refinement procedure, i.e. the same discretization) we observe that PWLH grants a much lower error than ZOH for the same number of intervals (top plot). In order to further examine the efficiency of the adaptive refinement procedure we depict in Figure 3 the input u i ( ) achieved during each iteration of Algorithm 8 using PWLH. In this picture the discretization intervals at each iteration are also reported. From this picture we notice that the devised adaptive procedure is able to detect, at each iteration, which intervals require (possibly repeated) bisection. Hence, the proposed algorithm appears parsimonious in the usage of intervals and hence of decision variables in the discretized CLQR problems P γ T (x).

19 TWCCC Technical Report u(t) u(t) u(t) u(t) u(t) Iteration 1 Iteration 2 Iteration 3 Iteration 4 Iteration Time (s) Figure 3: Example 1: Input u γ ( ) computed during the first five iterations of Algorithm 8 using PWLH, and associated (adaptive) discretization intervals. Next, we discuss the closed-loop performance of the proposed continuous time CLQR and compare it with that of standard discrete time MPC, formulated as in [22] and solved using the function quadprog.m with same options and tolerances used in solving problems P γ T (x). Every sampling time T s, given the current state x, we solve P T (x) and inject the first portion, t [, T s ), of the computed input u( ). Three controllers, which use a sampling time of T s = 1 s, are compared: CTCLQR 1 and CTCLQR 2 use Algorithm 8 with PWLH parameterization and stopping condition (33) for ρ = and ρ = 5 1 3, respectively; DTMPC 1 is a discrete time MPC with horizon N = 1 (hence same finite time as CTCLQR 1 and CTCLQR 2 of T = T s N = 1s). DTMPC 2 instead uses a horizon of N = 2 and hence a sampling time of T s = T/N =.5 s. In Table 1 we report: the ratio between computation time T C and sampling time T s, the closed-loop suboptimality with respect to CTCLQR 1 defined as (V CL VCL )/V CL in which V CL = T f l(x, u)dt, T f = 2 s, is the closed-loop cost achieved with a given controller and VCL is that achieved with CTCLQR 1. We can observe that CTCLQR 2 has a small relative suboptimality of (up to.53%), whereas DTMPC 1 has a suboptimality (up to 28.2%). By decreasing the sample time, DTMPC 2 achieves a small suboptimality (up to.12%). However, CTCLQR 1, CTCLQR 2 and DTMPC 1 have computation times significantly smaller than their sampling time. DTMPC 2, instead, have computation times similar and even larger than its sampling time.

20 TWCCC Technical Report Table 1: Example 1: closed-loop performance comparison of continuous time CLQR and discrete time MPC. Results are averaged over 5 closed-loop simulations of 2 s, each starting from a different random initial state. Controller (T C /T s ) (V CL V CL )/V CL Mean Max Mean Max CTCLQR 1 (ρ = 5 1 4, T s = 1 s) CTCLQR 2 (ρ = 5 1 3, T s = 1 s) DTMPC 1 (N = 1, T s = 1 s) DTMPC 2 (N = 2, T s =.5 s) Example #2: MIMO open-loop unstable system The second example is an open-loop unstable three-input three-output system defined by the transfer function matrix: G(s) = 5s+1 36s 2 +6s+1.1( 1s+1) (8s+1)s 2s+1 12s 2 +3s+1.5 8s+1.1 (64s 2 +6s+1)s 2( 5s+1) 16s 2 +2s+1 A minimal realization of the system has n = 1 states. The input constraint set is U = [ 1, 1] 3, and we consider continuous time LQR penalties of Q = I 1 and R =.25 I 3. As in the first example, we first focus on the solution of P T (x) for a given (random) initial state, chosen in a way that inputs constraints become active for some time in [, T ]. We consider the same three variants of Algorithm 8 discussed in the first example (namely, PWLH with adaptive refinement, PWLH with fixed refinement and ZOH with fixed refinement), and in all cases we use the following parameters T = 6, c =.85, and initial values of δ =.75 and ɛ =.1. The storage of the required discretized matrices, for 11 different interval sizes, takes 82 kb. Comparative results are depicted in Figure 4 in which number of intervals and solution time are plotted against the continuous time cost error bound during the algorithm s iterations. Also in this multivariable example we observe that the adaptive refining strategy is much more effective than a brute force bisection approach. For instance, at the second iteration the adaptive refining strategy (red curves) uses about 2 intervals and the cost error bound is less than 1 3, value that is achieved by the fixed refinement strategy (blue curves) at the fourth iteration using 4 intervals. Consequently, the cumulative computation time is less than 1 ms using adaptive bisection and is about 22 ms using fixed bisection. Similar considerations apply to other iterations. In Table 2 we report the performance indicators, as discussed for the first example, using four alternative controllers in closed-loop implementation (sampling time T s = 5 s, for all controllers except DTMPC 2). In particular, CTCLQR 1 and CTCLQR 2 use Algorithm 8 with PWLH parameterization and stopping condition (33) with ρ = 1 4 and ρ = 1 3, respectively; DTMPC 1 is a discrete time MPC with horizon N = 12 (hence same finite time as CTCLQR 1 and CTCLQR 2 of T = T s N = 6 s). DTMPC 2 instead.

21 TWCCC Technical Report Number of intervals, Jγ Solution time (ms) Cost error bound, θ( ) PWLH adapt. refin. PWLH fixed refin. ZOH fixed refin. PWLH adapt. refin. PWLH fixed refin. ZOH fixed refin. Figure 4: Example 2: Performance indicators during the first five iterations of Algorithm 8 using PWLH with adaptive discretization refinement (red), PWLH with fixed discretization refinement (blue), ZOH with fixed discretization refinement (green). uses a horizon of N = 24 and hence a sampling time of T s = T/N =.25 s. We notice, as in the first example, that CTCLQR 1, CTCLQR 2 and DTMPC 1 have (average and maximum) computation times much smaller than the sampling time. However, DTMPC 1 is rather suboptimal (3.67% on average, up to 11.7%). By decreasing the sampling time, DTMPC 2 is much less suboptimal than DTMPC 1 (.76% on average, up to 3.4%), but DTMPC 2 has an average computation time more than twice its sampling time (almost five times in worst case). From these results it is clear than CTCLQR 1 and CTCLQR 2 provide much better results than standard discrete-time MPC algorithms. 8 CONCLUSIONS The method presented in this paper solves the input constrained, infinite horizon, continuous time linear quadratic regulator problem to a user specified accuracy. The doubly infinite dimensional nature of the problem is addressed without undue online computation. The algorithm is efficient due to the storage of matrix exponentials for exact solution of the model, cost, and gradients at several levels of discretization. The storage requirement for these matrices is minor. The issue of implementation of this continuous time input in a standard industrial control system has been separated from the numerical solution of

22 TWCCC Technical Report Table 2: Example 2: closed-loop performance comparison of continuous time CLQR and discrete time MPC. Results are averaged over 5 closed-loop simulations of 6 s, each starting from a different random initial state. Controller (T C /T s ) (V CL V CL )/V CL Mean Max Mean Max CTCLQR 1 (ρ = 1 1 4, T s = 5 s) CTCLQR 2 (ρ = 1 1 3, T s = 5 s) DTMPC 1 (N = 12, T s = 5 s) DTMPC 2 (N = 24, T s =.25 s) the CLQR problem. The small interface code required to deliver a piecewise linear input to the actuator hardware, when it is digital, can evolve as the capabilities of the actuator hardware improve; discretization effects are negligible given the relatively small sampling time of the digital actuator. With this approach, the MPC problem and solution method are independent of sample time and actuator hardware. The advantages of discrete time linear MPC, and the reasons for its dominant position in industrial implementations, are mainly computational. All that is required for implementation is the solution of a strictly convex, finite dimensional quadratic program, and standard software exists for solving the QP to near machine precision in a finite number of iterations. But if one wants to implement an algorithm with a guarantee of even nominal recursive feasibility and closed-loop stability, more is required. Current theory requires a terminal penalty and an implicit or explicit terminal constraint. The terminal constraint restricts the set of feasible initial states that can be handled. One method to offset this reduction in the feasible set is to increase the horizon length. But computational cost increases at least linearly with horizon length, so this creates an unpleasant tradeoff between the size of the feasible set and computational efficiency. The same difficult tradeoff applies to the choice of sample time. If chosen too large, closed-loop performance and robustness to disturbances degrade; if chosen too small, the horizon and computation time are excessively large or the feasible set of initial states is excessively small. For online solutions as in MPC, it is often intractable to solve the infinite horizon discrete time problem because the input parameterization as a zero-order hold with a fixed and short sample time is inefficient. (See Figure 3, for example.) The use of the infinite horizon, continuous time CLQR in MPC removes the need for terminal sets and terminal constraints. Of course we made liberal use of both in designing the CLQR algorithm, but those details can be hidden in the algorithm itself. The higher level controller design problem is free from these considerations. The simplicity of the resulting CLQR theory may be its most significant advantage. Nominal recursive feasibility and closed-loop stability follow directly from the optimal control problem s design. The CLQR feasible set is the largest set possible, i.e., the set of states for which there exists an input trajectory with a finite infinite horizon open-loop cost. It remains to be seen whether this kind of approach can handle the largest industrial