A new approach to explicit MPC using self-optimizing control

Transcription

1 28 American Control Conference Westin Seattle Hotel, Seattle, Washington, USA June -3, 28 WeA3.2 A new approach to explicit MPC using self-optimizing control Henrik Manum, Sriharakumar Narasimhan an Sigur Skogesta Abstract Moel preictive control (MPC) is a favore metho for hanling constraine linear control problems. Normally, the MPC optimization problem is solve on-line, but in explicit MPC an explicit piecewise affine feeback law is compute an implemente ]. This approach is similar to self-optimizing control, where the iea is to fin simple pre-compute policies for implementing optimal operation, for example, by keeping selecte controlle variable combinations c constant. The nullspace metho 2] generates optimal variable combinations, which turn out to be equivalent to the explicit MPC feeback laws, that is, c = u Kx, where K is the optimal state feeback matrix in a given region. More importantly, this link gives new insights an also some new results. One is that regions changes may be ientifie by tracking the variables c for neighboring regions. I. INTRODUCTION Consier the general static optimization problem 2]: min J (x,u,) u,x s.t. f i (x,u,) =, i E h i (x,u,), i I, (P) where x R nx are the states, u R nu are the inputs, an D R n are isturbances. By iscretization an reformulation this may also represent some ynamic optimization problems. Usually f is a moel of the physical system, whilst h is a set of inequality constraints that limits the operation (e.g., physical limits on temperature measurements or flow constraints). In aition to (P) we have measurements on the form y = f y (x,u,). () In this work the emphasis is on implementation of the solution to (P). This means that the optimization problem (P) is solve off-line to generate a control policy which is suitable for on-line implementation, with particular emphasis on remaining close to optimal solution when there are unknown isturbances. That is, we search for control policies such that the cost J remains optimal or close to optimal when isturbances occur without the nee to reoptimize. A. Self-optimizing control In our previous work on self-optimizing control we have looke for simple control policies to implement optimal operation, an in particular what shoul we control (choice of controlle variables (CV s)). Using off-line optimization Department of Chemical Engineering, Norwegian University of Science an Technology, N-749 Tronheim, Norway. Author to whom corresponence shoul be aresse. skoge@ntnu.no we may etermine regions where ifferent sets of active constraints are active, an implementation of optimal operation is then in each region to: ) Control the active constraints. 2) For the remaining unconstraine egrees of freeom: Control self-optimizing variables c = Hy which have the property that keeping them constant (c = c s ) inirectly achieves close-to optimal operation (with a small loss), in spite of isturbances. We here allow for linear measurement combinations, c = Hy. There are here two factors that shoul be consiere: a) Disturbances. Ieally, we want the optimal value of c (c opt ) to be inepenent of. b) Measurements errors n y. The loss shoul be insensitive to these. B. Relationship to explicit MPC Consier a simple static optimization problem min u J(u,), where u are the unconstraine egrees of freeom an the states x an the active constraints have been eliminate by substitution. For the quaratic case J(u,) = u ] T Su ] ] Juu J where S = u. J u T J In aition we have available measurements y = G y u+ G. A key result, which is the basis for this paper, is For a quaratic optimization problem there exists (infinitely many) linear measurement combinations c = Hy that are optimally invariant to isturbances. One sees immeiately that there may be some link to explicit MPC, because the iscrete form MPC problem can be written as a static quaratic problem. The link is: If we let y contain the inputs u an the states x, then the selfoptimizing variable combination c = Hy is the same as the explicit MPC feeback law (control policy), i.e. c = u Kx. (This is shown in section III.) Base on this, we provie in this contribution some new ieas on explicit MPC: ) We propose that tracking the variables c (eviation from optimal feeback law) for all regions, may be use as a local metho to etect when to switch between regions. 2) We may use our results to inclue measurement error in y (e.g. in x an u) when eriving the optimal explicit MPC. 3) We may exten the results to output feeback (c = u Ky) by incluing in y present an past outputs (an not present states x). (2) /8/$ AACC. 435

2 Off-line Optimizer c s Feeback Controller u c + n y m n c Measurement combination (H) Process y (G y,g y ) ny Fig.. Block iagram of a feeback control structure incluing an optimization layer 4]. 4) We can also exten the results to the case where only a subset of the states are measure (but in this case there will be a loss, which we can quantify). This may be of interest even in the unconstraine LQ case. In this paper the basic framework an issue () are iscusse. In 3] it is shown how the results can be extene to hanle items (2)-(4), both with theorems an examples. II. RESULTS FROM SELF-OPTIMIZING CONTROL A. Steay state conitions Once the set of active constraints in (P) is known we can form the reuce problem an the unconstraine egrees of freeom u can be etermine. The unconstraine measurements are y = G y u + G y, (3) an y contain information about the present state an isturbances (y may inclue u an, but not the active constraints.) The (measure) value of y m available for implementation is y m = y + n y, (4) where n y represents uncertainty in the measurement of y incluing uncertainty of implementation in u. The following theorem escribes a metho to fin linear invariants that yiels zero loss from optimality when the invariants are controlle at constant setpoint. The theorem is base on the nullspace metho presente in 2]. Figure illustrates how the H matrix is use to linearly combine measurements (an square own the plant). Theorem : (Linear invariants for quaratic optimization problem 4]) Consier an unconstraine quaratic optimization problem in the variables u (input vector of length n u ) an (isturbance vector of length n ) min u J(u,) = u ] Juu J u J u T J ] u ] (5) In aition, there are measurement variables y = G y u + G y. If there exists n y n u + n inepenent measurements (where inepenent means that the matrix G y = G y G y ] has full rank), then the optimal solution to (5) has the property that there exists n c = n u linear variable combinations (constraints) c = Hy that are invariant to the isturbances. The optimal measurement combination matrix H is foun by either: (): Let F = yopt T be the optimal sensitivity matrix evaluate with constant active constraints. Uner the assumptions state above possible to select the matrix H in the left nullspace of F, H N(F T ), such that (2): If n y = n u + n : HF = (6) H = Mn J( G y ), (7) where J ] = Juu /2 Juu /2 J u an G y = G y G y ] is the augmente plant. Mn may be seen as a free parameter. (Note that M n = J cc is the Hessian of the cost with respect to the c-variables; in most cases we select M n = I for convenience.) Remark : The sensitivity F matrix can be obtaine from F = ( G y Juu J u G y ). (8) Remark 2: An equivalent formulation is: Assume that there exists a set of inepenent measurements y an that the (operational) constraint c Hy = c s (where c s is a constant) is ae to the problem. Then there exists an H that oes not change the solution to (5). In terms of operation, this means that zero loss (optimal operation) is obtaine by controlling n c = n u variables c = Hy with a constant set-point policy c = c s, where H is selecte accoring to theorem. Theorem may be extene: Lemma : (Linear invariants for constraine quaratic optimization methos) Consier an optimization problem of the form min J = x u ] S x u u,x s.t. Ax + Bu + C = Ãx + Bu + C, with et(a) an à B] full row rank. Assume that the isturbance space has been partitione into n a critical regions. In each region there are n i u = n u n i A unconstraine egrees of freeom, where ni A n m is the number of optimally active constraints in region i. If there exists a set of inepenent unconstraine measurements y i = (G y ) i u i +(G y )i in each region i, such that n y i n u i + n, the optimal solution to (9) has the property that there exists variable combinations c i = H i y i that are invariant to the isturbances in the critical region i. The corresponing optimal H i may be obtaine from Theorem. (9) 436

3 Within each region, optimality requires that c i c i s = (where c i s is a constant). From continuity of the solution, we have that c i is continuous across the bounary of region i. This implies that the elements in the variable vector c i c i s will change sign or remain zero when crossing into or from a neighboring region. Proof: See internal report 5]. B. Implementation of optimal solution For the case of no measurement error, n y =, Theorem shows that for the solution to quaratic optimization problems, variable combinations c = Hy that are invariant to the isturbances can be foun. In section III this insight will be use as a new approach to the explicit MPC problem. III. APPLICATION TO EXPLICIT MPC We will now look at the moel preictive control problem (MPC) with constraints on inputs an outputs. For a iscussion on MPC in a unifie theoretical framework see 6]. The following iscrete MPC formulation is base on 7]. Consier the state-space representation of a given process moel: x(t + ) = Ax(t) + Bu(t) () subject to the following constraints: y (t) = Cx(t), () y min y (t) y max (2) u min u(t) u max, (3) where x(t) R n, u(t) R m, an y(t) R p are the state, input an output vectors, respectively, subscripts min an max enote the lower an upper bouns, respectively, an (A,B) is stabilizable. MPC problems for regulating to the origin can then be pose as the following optimization problem: min J(U,x(t)) = x t+n T y y Px U t+ny t+ + N y k= xt+k t T Qx t+k t + u t+k T Ru t+k ] s.t. y min y t+k t y max, u min u t+k u max, x t t = x(t) k =,...,N c k =,,...,N c x t+k+ t = Ax t+k t + Bu t+k, k y t+k t = Cx t+k t, k u t+k = Kx t+k t, N u k N y where U {u t,...,u t+nu }, Q = Q T, R = R T >, P, N y N u, an K is some feeback gain. 7] show that by substitution of the moel equations, the problem can be rewritten on the form min U 2 U T HU + x(t) T FU + 2 x(t)t Y x(t) s.t. GU W + Ex(t) (4) The MPC control law is base on the following iea: At time t, compute the optimal solution U (t) = {u t,...,u t+n u } an apply u(t) = u t ]. Remark 3: The trae-off between robustness an performance is inclue in the weights in the MPC cost function an in the constraints. If we let the initial state x(t) be treate as a isturbance, (4) can be written as: min U 2 U T T] H F F Y s.t. GU W + E, ] ] U (5) an we observe that (5) is on the same form as (9), where the moel equations f(x,u,) = have alreay been substitute into the objective function. A property of the solution to (5) is that the isturbance space (initial state space) is ivie into critical regions. In the i th critical region there are n i u = n U n i A unconstraine egrees of freeom, where n i A is the number of active constraints in region i. As we iscuss in section III-A, a possible set of measurements y is the current state an the inputs, y T = x T u T]. We further note that causality is not an issue here, as we have the information at the current time. A. Exact measurements of all states (state feeback) The following theorem is well known, but we will here prove the theorem using the nullspace metho. Theorem 2: (Optimal state feeback ]) The control law u(t) = f(x(t)), f : R n R m, efine by the MPC problem, is continuous an piecewise affine f(x) = K i x + g i if H i x k i, i =,...,N mpc (6) { where the polyheral sets H i x k i}, i =,...,N mpc N r are a partition of the given set of states X. In this case causality is not a problem an from Theorem the optimal solution is simply u = Kx + g (i.e. c = u (Kx g)). Note that n = n x in this case. Proof: We consier the explicit MPC formulation as in (5). First we consier the unconstraine case. Let y = (U,x) be the set of caniate measurements. With this choice of measurements an isturbances on the present state, we form the process moel: y = G y U + G y (7) ] G y nx (n = un u) R (nx+nunu) (nunu) (8) I (nun u) (n un ] u) G y = Inx n x R (nx+nunu) nx. (9) (nun u) n x 437

4 We then get the optimal sensitivity as F = yopt T = ( G y Juu J u G y ) = (2) ( ] ]) nx (n un u) Inx n (Juu x (2) J u ) (nun u) n x (nun ] u) n x Inx n = x Juu (22) J u We now search for a matrix H that gives a non-trivial solution to HF = : ] ] I (H ) (nun u) n x (H 2 ) nx n x (nun u) (n un u) Juu = (23) J u ( ) = H H 2 J uu J u = (24) To ensure a non-trivial solution we can for example choose H 2 = I (nun u) n un u. Then we must have H = Juu J u, an hence the optimal combination c of x an U becomes c = Hy = J uu J u x + U = R (nunu) (25) In the internal report by 5] it is shown how the affine term in (6) enters as a function of the active constraints. Remark 4: (Comparison with previous results on unconstraine MPC) In (25) the state feeback gain matrix is given as Juu J u. This is gives the same result as conventional MPC, see equation (3) in 8]. Remark 5: These are not new results but the alternative proof leas to some new insights. The most important is probably that the self-optimizing variables c i = u (K i x+ g i ) which are optimally zero in region i, may be use for ientifying when to switch between regions (Theorem 3) rather than using a centralize approach, for example base on a state tree structure search. This seems to be new. Another insight is to unerstan why a simple feeback solution must exist in the first place. A thir is to allow for new extensions. Theorem 3: (Optimal region for explicit MPC etection using feeback law) The variables c = u k (Kx k + g) can be use to ientify region changes. Proof: See report by 5]. Remark 6: Neighboring regions with the same feeback law (incluing regions where the feeback law is to keep the input saturate) can be merge (provie that the regions remain convex or if the crossings insie a non-convex region ue to the optimal irection of the process in close loop only occurs in the convex part of the region). This may greatly reuce the number of regions compare to presently use enumeration schemes. Note that the number of c-variables that nee to be tracke to etect region changes is only equal to the number of inputs n u times the number of istinct merge regions. Because of the merging of regions, this may be a small number even with a large input or control horizon an with output (state) constraints. We present a simple example from ] that confirms that our switching policy base on tracking the sign of the c- variables works in practice. Algorithm Detect current region an calculate u k Require: CR k, i.e. the region of the last sample time, an x k : u k = K(CR k ) + g(cr k ) 2: Regions,α] = Neighbors(CR k ) 3: for i = to length(regions) o 4: c k (i) = α i (u k (K(Regions(i)) + g(regions(i)))) 5: en for 6: if sign(c k (i) ) then 7: CR k = Regions(i) 8: else 9: CR k = CR k : en if : return u k = K(CR k )x k + g(cr k ), CR k Example 3. (Optimal switching): This example is taken from ] (with correction), an is inclue here to emonstrate optimal switching using the sign change of c = u Kx as the criterion. The system is: y(t) = 2 s 2 + 3s + 2 u(t). With a sampling time T =. secons the following statespace representation is obtaine: ] ] x(t + ) = x(t) + u(t) y(t) =.442 ] x(t) One observes that only the last state is measure, but it will be assume that both states are known (measure) in the remainer of this example. The task is to regulate the system to the origin while fulfilling the input constraint 2 u(t) 2. (26) The objective function to be minimize is minx t+2 t T Px t+2 t + k= T xt+k t x t+k t +.u 2 ] t+k (27) subject to the constraints an x t t = x(t). P solves the Lyapunov equation P = A T PA + Q, where Q = I in this case. The optimal control problem can be solve for example using the MPT toolbox 9]. The P -matrix is numerically: ] P = To illustrate ieas a simulation from x = (,) was one. State space trajectories an inputs are shown in figures 2 an 3. As long as the state is in the input-constraine region where u opt = 2, the linear combination c = u k Kx k remains positive. One chooses to leave the input-constraine region when c k becomes zero. As one observes, this happens at time instant 8, where the process inee is on the bounary between the input-saturate region an the center region. 438

5 x u = 2 u = Kx u = x Fig. 2. Partition of state space for first input. (Example 3..) States x(t) Input u(t) c k = u k (Kx k ) Fig. 3. Close loop MPC with region etection using u k (Kx k ). (Example 3..) After the switching the controller for the center region is implemente. The state trajectory is the same as in ]. The reason for why c never becomes negative is because both states are assume measure at the present time an hence optimal switching is achieve. This can be unerstoo from the algorithm, where we show how the current critical region (CR k ) is tracke an how the current input u k is calculate. Example 3.2 (Double integrator): Consier the ouble integrator isusse by ], y(t) = /s 2 u(t), an its equivalent iscrete-time state-space representation, ] ] x k+ = x k + u k, y k = ] x k, which is obtaine by setting ÿ(t) = (ẏ(t + T s ) ẏ(t))/t s, x u = x 7 u = 8 9 Fig. 4. Regions for ouble integrator example (Example 3.2). ẏ(t) = (y(t + T s ) y(t))/t s, T s =. The control objective is to regulate the system to the origin while minimizing the quaratic cost funcion J = t= y(t)t y(t) + u2 subject to the input constraint u(t). The infinite horizion control problem can be converte to a finite horizion problem by solving ], ]: K LQ = (R + B T PB) B T PA, P = (A + BK LQ ) T P(A + BK LQ ) + K LQ T RK LQ + Q to obtain the unconstraine feeback gain K LQ an the final state weight matrix P (see example 3.). In this case we get K LQ = ] ] an P = For emonstration purposes we choose N u = 6, an by solving the paramteric program we get 73 regions initially. In this case there are regions of unsaturate control actions, which agrees with the general result of (2N u ) regions given in ]. Merging all regions where the first optimal input is the same, leaves us with the unsaturate regions, an two regions for which the optimal input is either at the high or low constraint. The final partitioning with 3 regions is shown in figure 4. We note that ] fin 57 regions after their merging scheme. Consiering figure 4 one observes that the input-saturate regions are non-convex. However, optimally, this process moves clockwise in the state space, an we observe that the non-convex crossings will not occur in practise. The remaing bounaries then form convex regions (inicate by the ashe lines in the figure.) Figure 5 shows the evolution of the invariants c i in each region when we start the simulation at x = (, 3) an close the loop by using the optimal inputs. We start in the input-saturate region u =, an nee to track the invariants for regions,2,3,4,5, an 6 to etermine optimal switching. We shoul switch to unsaturate control when one the variables c to c 6 becomes zero or changes sign. As one sees, this happens for c 3 at t = 6, so we change to this region. After using the feeback law for region 3 for one 439

6 c c 2 c 7 c 3 c c 4 c c 5 c 5 5 c 6 c Fig. 5. Invariants for ouble integrator example. sample time, we reach the other input constraint u = at t = 8. Now, to ecie when to leave this constraine region we track the invariants for regions,7,8,9,,, an we observe that at t = 9 the invariant for the center region becomes zero, hence we switch control to this region. Note that in this example, where we have only input constraine regions, the challenge is to ecie when to leave the input constraints. Note that the converse crossing can also be tracke using their invariants on the form c k = u k g. The iea of using irectionality (clockwise movement in this case) to reuce the number of reigons in explicit MPC can be generalize by using the irectional erivative of the process uner optimal control, (A BK i ), together with the normal vectors to the bounaries of the regions, an by some normalization scheme we remove all bounaries for which crossings uner optimal control will not occur. Here K i is the optimal feeback gain for region i. u regions, but we note that for some processes irectionality of the process in close loops implies that the non-convex crossings may be ignore. In a forthcoming contribution 3] we show how the results can be extene to output feeback an how to fin invariants that give minimal loss when controlle at constant set points also when we have noisy measurements. We further show how we one choose the orer of the controller an we show by examples that the resulting controller will have performance in the orer of magnitue of LQG controllers. The most important problem of using results from steay state self-optimizing control is causality, in steay state optimization all measurements are available at the current time (i.e. t ), but in ynamic optimization we may nee to fin invariants between measurements at current an future times an then switch the invariants back to get a casual controller, but this controller will be non-optimal by construction. Also this is iscusse in more etail in 3]. REFERENCES ] A. Bempora, M. Morari, V. Dua, an E. N. Pistikopoulos, The explicit linear quaratic regulator for constraine systems, Automatica, vol. 38, pp. 3 2, 22, see also corrigenum 39(23), pages ] V. Alsta an S. Skogesta, Null space metho for selecting optimal measurement combinations as controlle variables, In. Eng. Chem. Res, vol. 46, pp , 27. 3] H. Manum, S. Narasimhan, an S. Skogesta, Explicit MPC with output feeback using self-optimizing control, in Proceeings of IFAC Worl Conference, Seoul, Korea, 28. 4] V. Alsta, S. Skogesta, an E. Hori, Optimal measurement combinations as controlle variables, 28, in Press: Journal of Process Control, oi:.6/j.jprocont ] H. Manum, S. Narasimhan, an S. Skogesta, A new approach to explicit MPC using self-optimizing control, 27, avaliable at: 6] K. R. Muske an J. B. Rawlings, Moel preictive control with linear moels, AIChE Journal, vol. 39, pp , February ] E. N. Pistikopoulos, V. Dua, N. A. Bozinis, A. Bempora, an M. Morari, On-line optimization via off-line parametric optimization tools, Computers an Chemical Engineering, vol. 26, pp , 22. 8] J. B. Rawlings an K. R. Muske, The stability of constraine receing-horizon control, in IEEE Transactions on Automatic Control, vol. 38, 993, pp ] M. Kvasnica, P. Grieer, an M. Baotić, Multi-Parametric Toolbox (MPT), 24. Online]. Available: mpt/ ] D. Chmielewski an V. Manousiouthakis, On constraine infinitetime linear quaratic optimal control, Systems an Control Letters, vol. 29, pp. 2 29, 996. IV. DISCUSSION In this paper we have escribe the link between selfoptimizing control an explicit MPC. This link has been use to propose a new metho for etecting region changes. This new metho lets us reuce the number of regions by merging all regions for which the first input is the same. In its simple form presente in this paper, it oes not hanle non-convex 44