ESC794: Special Topics: Model Predictive Control Discrete-Time Systems Hanz Richter, Professor Mechanical Engineering Department Cleveland State University
Discrete-Time vs. Sampled-Data Systems A continuous-time (C-T) system operates on input functions. It evolves according to a set of differential equations where time t is the independent variable, t R. A discrete-time (D-T) system operates on input sequences. It evolves according to a set of difference equations where sequences are indexed with an integer k N 0. A sampled-data system is a D-T system obtained from a C-T system by a sampling process. Inputs, however, are functions defined on a continuous sampling interval. See remarks on G&P, p.21. Sampled-data analysis will not be included in this course. We will describe the dynamics of D-T systems using the difference equation x + = f(x,u) where the transition map f : X U X returns the next state x + given a current state and control value. By repeatedly applying a sequence u(k), we obtain a state sequence x(k). Exercise: find the transition map of a savings account with an interest rate of i and initial balance x 0. Positive u represent deposits and negative u represent withdrawals. 2 / 23
Zero-Order Hold Sampling: LTI Systems Clock u(k) u(t) Continuous y(t) y(k) D/A System A/D Sample and hold (ZOH) Sample and hold (ZOH) We may assume that there is no delay between the application of u k and the effect on y k (do not confuse lag with delay). The piecewise-constant signal u(t) is treated as input and y(t) is obtained from the general state-space solution of the continuous-time system Then y(t) is sampled (in sync with u(k)) to give y(k). u(k) maps into y(k) through a discrete-time dynamic system that can be obtained knowing the continuous-time model and the sampling period. 3 / 23
Zero-Order Hold Equivalent Calculations Let the LTI continuous-time system be represented by ẋ = Ax+Bu; y = Cx+Du The general solution given initial condition x 0 is well-known: x(t) = e A(t t 0) x(t 0 )+ t t 0 e A(t τ) Bu(τ)dτ A state-space discrete equivalent based on the ZOH is found by substituting a piecewise-constant function u(t k ) and integrating by segments: x(k +1) = Φx(k)+Γu(k); y(k) = Cx(k)+Du(k) where Φ = e AT s Γ = Ts 0 e As Bds 4 / 23
Some ways to compute Φ 1. Use series expansion e AT s = I +AT s +A 2 T 2 s/2+... and hope that A n becomes zero for some n (nilpotent matrix). 2. Use the Laplace identity (only adequate for low orders) e At = L 1 (si A) 1 3. Use the Cayley-Hamilton theorem (convenient for analytical evaluations) 4. Numerically in Matlab or symbolically in Maple/Mathematica/Mathcad. 5 / 23
Using the Cayley-Hamilton Theorem A square matrix satisfies its own characteristic equation. This can be clearly seen from the eigenvalue equation det(λi A) = 0. If we define a matrix function f(a) (for example e At ), the Cayley-Hamilton theorem can be used to show that f(a) = α 0 A n 1 +α 1 A n 2 +...+α n 1 I It can be shown that the eigenvalues of A must satisfy the above. If we have n distinct eigenvalues, we can determine the α i, and consequently the matrix exponential. If an eigenvalue is repeated m times, we take the first m 1 derivatives of the above to obtain additional conditions. 6 / 23
Refresher Exercise : Linear Systems Theory Obtain the ZOH equivalent of the double-integrator G(s) = 1/s 2 for T s = 1 using the four methods. 7 / 23
Difference equation representations The state-space discrete-time system presented above can be considered to be the realization of a linear n-th order input-output difference equation: y(k)+a 1 y(k 1)+...a na y(k n a ) = b 0 u(k d)+...+b nb u(k d n b ) Conversely, the difference equation can be obtained from the state-space system by algebraic elimination of states (entirely analogous to continuous-time case). To obtain a state-space realization from a difference equation, proceed as in the continuous-time case. For instance, states may be chosen as y(k) and its delayed versions. To recall state-space realization methods from I/O equations see Dorf, Modern Control Systems. Exercise: for the CT transfer function G(s) = s+1 s 2 +s+1 obtain a ZOH discretization in state-space and find the corresponding difference equation. From it, find a state-space realization that matches the one returned by Matlab s c2d applied to G(s). 8 / 23
Exercise - Simulink Use a Simulink model to simulate the C-T double integrator under ZOH and the D-T equivalent in state-space and difference equation forms. Apply some input and check that the sampled-data system and the D-T systems all give the same output sequences. 9 / 23
Discretization of nonlinear C-T systems How do we obtain x + = f(x,u) from ẋ = h(x,u) when h is a nonlinear function? We will use two methods in this course: 1. Euler discretization (forward differences) 2. Numerical integration of the C-T system (using ode45 or similar solvers) There are other options, including implicit forms that use a backward difference, and in some cases, exact analytical discretization can be attempted. The numerical integration method is useful to generate online MPC predictions when dealing with C-T system models. We simply apply a constant input over a short period of time (corresponding to the sampling period δ) and use a numerical solver to find the corresponding state trajectories. For this, we write a Matlab function x=dynamics(x0,delta,u) which calls ode45 using an initial point and a constant input to determine the state x + at the end of the simulation. 10 / 23
Forward Euler Discretization Given a C-T system in the form we approximate the time derivative with ẋ = h(x,u) ẋ x(t+δ) x(t) δ Substitution gives and using t = kδ gives: x(t+δ) = x(t)+δh(x(t),u(t)) x(k +1) = x(k)+δh(x(k),u(k)) or simply with f(x,u) = x+δh(x,u). x + = f(x,u) The above gives an explicit discretization. If we used the backward difference, we wouldn t be able to solve for x + in the general nonlinear case. In the linear case this is possible. 11 / 23
Discretization of LTI and Stability Preservation Suppose ẋ = Ax and a forward Euler discretization is used. Then the discrete equivalent is x + = (I +δa)x The C-T is asymptotically stable if and only if all eigenvalues λ C of A lie in the region defined by Re(λ) < 0. The D-T is asymptotically stable if and only if all eigenvalues λ D of I +δa lie in the unit circle, defined by λ D < 1. What is the relationship between these two regions for a given δ? Does forward Euler preserve stability? Now suppose that a backward Euler discretization is used. What is the discrete equivalent? What can we say about stability preservation for LTI systems? Fact: A+I is invertible is none of the eigenvalues of A is equal to -1. 12 / 23
Basic State-Space D-T LTI Design The pole placement and LQR methods follow the same principles as in the C-T case, using the open unit circle as stability region. For low-order designs, pole placement can be useful: 1. Use the 2nd-order C-T response as a prototype: select a damping ratio ζ and natural frequency w n for the dominant poles. 2. Use a pole mapping from C-T to D-T to determine the approximate locations of the desired poles in D-T. Keep in mind that zeroes can appear in the D-T system even when the C-T did not have them. For details see Franklin, Powell & Workman or Åstrom & Wittenmark. 3. The ZOH pole mapping is z = e st s. 13 / 23
Pole placement... 1. Since the closed-loop system matrix is still A BK, we may use the same formulas and tools of C-T pole placement. 2. For state feedback with setpoint following, check that the (SISO) control law is u = Pr Kx, where the reference gain is P = 1 (C DK)(I (A BK)) 1 B +D 3. Deadbeat response: If we can place all closed-loop poles at zero, the setpoint will be reached in a finite number of steps, which is at most equal to the system order. See FPW. Exercise: Design K and P for the double-integrator plant discretized with ZOH, Ts = 0.1. The settling time should be 1 second and the overshoot 5% or less. Redesign for deadbeat response. Check in Simulink against the D-T and sampled-data systems. 14 / 23
D-T LQR As in C-T, an optimal design can be attempted using an infinite-horizon quadratic performance index: J = x T (k)qx(k)+u T (k)ru(k) Assume the following: k=0 1. (A, B) is stabilizable (uncontrollable modes are asymptotically stable) 2. R > 0 and Q 0 3. (Q,A) has no unobservable modes on the unit circle Then the solution is a state feedback controller u = Kx where K = (B T PB +R) 1 B T PA and P is the unique positive-definite solution of the discrete algebraic Riccati equation (DARE): A T PA P A T PB(R+B T PB) 1 B T PA+Q = 0 The Matlab function dlqr calculates K and P. 15 / 23
Quadratic Lyapunov Functions and Invariance MPC was developed to seek the best performance for the given constraints. Constraints are usually given as a set of polyhedral regions in the state and control spaces, for example 10 x 1 10, 5 x 2 5, or more generally Fx 1, where F has as many rows as constraint boundaries. When a LTI system is controlled with linear state feedback to give an asymptotically stable closed-loop, ellipsoidal invariant sets appear, which are easy to compute. Invariant ellipsoids inscribed in a polyhedral constraint set are conservative safe operating regions. More importantly, ellipsoids are used as terminal regional constraints to establish stability of MPC, including nonlinear cases. An ellipsoidal set centered at the origin of R n is defined as for some P > 0. E = {x R n x T Px 1} 16 / 23
Finding invariant ellipsoids: DLTI Suppose the closed-loop system is x + = A cl x where A cl is asymptotically stable. Consider the quadratic Lyapunov function with arbitrary P = P T > 0 V(x) = x T Px The difference V = V(x + ) V(x) along closed-loop trajectories is V = x T (A T clpa cl P)x Intuitively at this point, we know that V(k) must be monotonically decreasing with k, since x(k) is exponentially converging to zero. This means that V must be negative for any value of x (because any such value could be used as initial condition). Then A T cl PA cl P must be negative definite. We can generate P by selecting Q > 0 and solving the discrete Lyapunov equation A T clpa cl P = Q where Q sets how fast V decreases. In Matlab dlyap can be used, adjusting for a transposed A cl. 17 / 23
Finding invariant ellipsoids... The level sets of V define ellipsoidal sets. Therefore, the monotonic decrease of V indicates that E must be positively-invariant, that is: x(0) E = x(k) E for k > 0 We will use the Ellipsoidal Toolbox (Alex Kurzhanskiy, UC Berkeley) for various operations with ellipsoids and hyperplanes. Example: We find a conservative set (E) of initial conditions such that the states of the controlled D-T double-integrator (for r = 0) satisfy unit box constraints. The largest E should be found. Then we plot some trajectories. We minimize the trace of P to obtain the largest volume for E We require P = P T > 0 The Lyapunov inequality must hold We must keep E inside the unit box This can be efficiently programmed using LMIs, but we will use the general-purpose optimizer fmincon and Lagrange multiplier techniques. 18 / 23
Example... We first solve the useful sub-problem: given a family of hyperplanes defined by γ = Γx, minimize γ subject to x T Px = 1, where P = P T > 0. Check that the solution is γ = ΓP 1 Γ T and interpret graphically in 2D using the Ellipsoidal Toolbox. The box constraints are defined with Γ 1 = [1 0], Γ 2 = [ 1 0], Γ 3 = [0 1] and Γ 4 = [0 1]. To simplify the optimization problem, we search over Q = Q T > 0, since any such choice will automatically guarantee P = P T > 0 and the Lyapunov inequality. 19 / 23
Example... The optimization problem is then formulated as minimize Q trace P subject to Q = Q T > 0 A T clp +PA cl Q = 0 γ 1 1 < 0 γ 3 1 < 0 This is readily solved using fmincon s equality constraints and nonlinear constraints capabilities. Note that the Lyapunov equation is handled by substitution (not programmed as an equality constraint). The algorithm tries a Q, then finds P and checks for constraint satisfaction. The requirement Q = Q T is programmed as an equality constraint based on a vectorized form of Q (using reshape). 20 / 23
Example... 1.5 Invariant Ellipsoid and Trajectories: Double Integrator 1 0.5 x 2 0-0.5-1 -1.5-1.5-1 -0.5 0 0.5 1 1.5 x 1 The ellipsoidal region is conservative. Initial conditions outside E may still be feasible, we don t know until we do a simulation. 21 / 23
Output Admissible Sets The initial state of an unforced linear system is output admissible with respect to a constraint set Y if the resulting output function satisfies the pointwise-in-time condition y(t) Y, t 0. The set of all possible such initial conditions is the maximal output admissible set O. Gilbert E.G. and Tan, K.T., Linear Systems with State and Control Constraints: The Theory and Application of Maximal Output Admissible Sets, IEEE Trans. Automatic Control, 36(9) pp.1008-1020, 1991. Gilbert and Tan showed that O preserved the closure, symmetry and convexity of Y and proposed an algorithm to approximate O with a finite intersection of regions (several ellipsoids in the figure below). 22 / 23
Semi-Ellipsoidal Sets and Invariant Cylinders For linear systems under state feedback, O Dell proposed that a good tradeoff between the complexity of polyhedrals and the conservatism of ellipsoids could be achieved by intersecting an invariant ellipsoid with the constraints themselves to define a semi-ellipsoidal set. This set was designed by maximizing the ellipsoid volume under applicable constraints. O Dell, B.D. and Misawa, E.A, Semi-Ellipsoidal Controlled Invariant Sets for Constrained Linear Systems, ASME J. Dynamic Systems, Measurements and Control, 124(1) pp.98-103, 2002. For systems under sliding mode control, Richter followed a similar approach, using infinite cylinders as primary invariant sets. Richter, H., O Dell, B.D. and Misawa, E.A, Robust Positively Invariant Cylinders in Constrained Variable Structure Control, IEEE Trans. Automatic Control, 52 (11) pp.2058-2069, 2007. 23 / 23