Performance Bounds for Constrained Linear Min-Max Control

Transcription

1 01 European Control Conference (ECC) July 1-19, 01, Zürich, Switzerland. Performance Bound for Contrained Linear Min-Max Control Tyler H. Summer and Paul J. Goulart Abtract Thi paper propoe a method to compute lower performance bound for dicrete-time infinite-horizon min-max control problem with input contraint and bounded diturbance. Such bound can be ued a a performance metric for control policie yntheized via uboptimal deign technique. Our approach i motivated by recent work on performance bound for tochatic contrained optimal control problem uing relaxation of the Bellman equation. The central idea of the paper i to find an uncontrained min-max control problem, with negatively weighted diturbance a in H 1 control, that provide the tightet poible lower performance bound on the original problem of interet and whoe value function i eaily computed. The new method i demontrated via a numerical example for a ytem with box contrained input. I. INTRODUCTION Conider the dicrete-time linear time-invariant ytem x t+1 = Ax t + Bu t + Gw t z t = Cx t + Du t (1) with tate x t R n, input u t R m, unknown exogenou diturbance w R l, and coted/controlled output z t R p at each time t = {0, 1,...}. We aume throughout that the initial tate x 0 i known and that the tate i directly obervable at each time tep. We further aume that the input are ubject to a compact contraint u t U, that the diturbance are contrained to a compact et w t W, that the pair (A, B) and (A, C) are controllable and obervable, repectively, and that D i full rank. We denote by W the et of all infinite-horizon diturbance equence generated by W. We define a tationary control policy : R n! U for the ytem (1) a a deciion rule mapping oberved tate to control input, i.e. u t = (x t ). We denote the et of all uch tationary policie atifying the ytem contraint for all infinite-horizon diturbance equence by. The linear min-max control problem i to deign a control policy that minimize the wore-cae value of ome objective function over all admiible diturbance equence, tarting from ome known initial tate. Given an initial tate x := x 0, a control policy and a diturbance equence w W, we conider a cot function in the form J(x 0,,w) = t`(z t,w t ) () T. H. Summer i upported by an ETH Zurich Potdoctoral Fellowhip. Automatic Control Laboratory, Department of Information Technology and Electrical Engineering, ETH Zurich, 809 Zurich, Switzerland. tummer pgoulart@control.ee.ethz.ch where ` : R n UW! R i a cot function and (0, 1) i a dicount factor. The optimal value function and policy are given by V (x 0 )=min max ww t`(z t,w t ) (x) = arg min max ww t`(z t,w t ) It i well known that, given the preceding aumption and certain mild condition on the tage cot `, a tationary optimal control policy exit [1, Ch. 5]. However, it i extremely difficult to compute the optimal cot and policy in general, becaue the optimization problem to be olved i over an infinite dimenional function pace and over the wort cae diturbance equence. It wa hown in [] that for contrained finite-horizon problem with a cot convex and quadratic in z and concave and quadratic in w, the optimal value function i piecewie quadratic and the optimal control policy piecewie affine over a polytopic partition of the tate pace, though computing and toring the policy i poible only for very mall ytem. Recent reearch ha focued on developing tractable but uboptimal policie for ytem in which the approach of [] i intractable. Example include variou formulation of minmax model predictive control [], [4], [5], []. Thi immediately raie quetion about the degree of uboptimality of uch policie. To the bet of our knowledge there i currently no general method to anwer thee quetion for min-max problem. We therefore propoe a method for computing a lower bound on the optimal cot for input contrained problem, which provide a way to bound the degree of uboptimality of any control policy contructed by any given deign method. Our approach i motivated by recent work from Wang and Boyd [], who propoe a imilar approach to computing performance bound for tochatic optimal control problem (i.e. thoe with an expected value, rather than a min-max, cot). The central idea i to find an uncontrained linear min-max control problem for which the optimal cot can be efficiently computed that bet approximate from below the optimal cot of the original contrained problem. In particular, we contruct an auxiliary convex-concave quadratic cot function for the ytem (1) without contraint, and exploit thi auxiliary function to compute a lower bound on the value function V of the contrained ytem. The particular choice of a convex-concave auxiliary cot function i motivated by the fact that uch cot function appear () / 01 EUCA 185

2 naturally in H 1 control deign problem, for which exact performance value function can be computed. Our reulting bound are not generic, but are intead computed individually for each problem intance. Such bound can be ueful in evaluating the performance of uboptimal policie in cae where the optimal policy cannot be efficiently computed: when the gap between the lower bound and the performance of a uboptimal policy i mall, one can conclude that the uboptimal policy i nearly optimal. The paper i organized a follow. Section II briefly review olution method for uncontrained min-max problem, which are cloely related to the H 1 control problem. Section III decribe the method for computing the propoed bound. Section IV provide a numerical example for which the bound i computed. Finally, Section V give concluding remark. II. UNCONSTRAINED PROBLEMS WITH QUADRATIC COST Thi ection review olution method for uncontrained minmax problem whoe objective function are convex-concave in (z,w), which will be ued to compute lower bound for input contrained problem in Section III. If the tage cot for ome uncontrained problem i le than or equal to the tage cot for the contrained problem, then we will how that the uncontrained optimal cot i a lower bound on the contrained optimal cot for a certain et of initial tate. One can then optimize the parameter in the tage cot of the uncontrained problem to find tighter lower bound for the original contrained problem. We begin by recalling that a pecial cae, cloely related to the H 1 optimal control problem, in which the min-max optimal policy can be efficiently computed i when the tage cot i convex and quadratic in (x, u) and concave and quadratic in w, with dicount factor =1, i.e. J = kcx t + Du t k kw t k, > 0 (4) and there are no contraint, i.e. when U = R m and W = R l. In thi cae, the optimal value function i quadratic in the initial tate, the optimal tate feedback policy i linear, and the cloed-loop ytem ha ` gain bounded by, whenever i larger than the H 1 optimal gain. Furthermore, the optimal cot and feedback gain can be computed efficiently from the problem data uing a variety of well-known technique, ee e.g. [8], [9], [10]. One tandard olution method for thi problem i baed on dynamic programming recurion uing the Iaac equation, which i the dicrete-time dynamic game counterpart to the Bellman equation from dynamic programming [10]. Suppoe that (P, P ) R nn are ymmetric matrice that atify the generalized algebraic Riccati equation P = Q + A T PA A T PB(R + B T PB) 1 B T PA (5a) P := P + PG( I G T PG) 1 G T P, (5b) where we have aumed for implicity that Q = C T C>0, R = D T D>0, C T D =0and D T C =0. If P 0 i a olution to (5), then V (x 0 )=x T 0 Px 0 i the optimal value function, and the optimal controller and diturbance policie are u = Kx and w = K w x, repectively, where K = K w =( I (R + B T PB) 1 B T PA G T PG) 1 G T P (A + BK). For (0, 1) the optimal cot and policie have the ame tructure but with ome problem data caled by the dicount factor: A! p A, B! p B and! / p (ee the Appendix). In other word, the optimal cot and controller for the dicounted problem can be computed by olving an aociated undicounted problem. Alternative olution method involve relaxing the Riccati equation (5a) to an inequality and olving a emidefinite programming problem. If (5a) i relaxed to the right (i.e by replacing = with ), it can be hown that the reulting inequality i equivalent to the linear matrix inequality obtained by applying the Bounded Real Lemma to the ytem (1) in cloe-loop with the optimal controller u t = Kx t and w t = K w x t [9], [8] (ee Appendix). The optimal cot and tate feedback gain matrice are given by the olution to the following optimization problem in variable P and K: minimize Tr(P ) ubject to P 0 P 1 A + BK G 0 (A + BK) T P 0 (C + DK) T 4 G T 0 I 0 0 C + DK 0 I () 5 0. () The uual approach to olving () i to tranform thi problem into a emidefinite program in variable P 1 and KP 1. However, in the next ection we will allow D to vary in order to obtain the tightet poible lower bound for the problem (), o an alternative to () i preferred. A hown in [9], the optimal cot can alo be computed eparately from the optimal tate feedback via the emidefinite program (SDP) in variable P and X. The matrice N and M in (SDP) are bae for the null pace of B T and it orthogonal complement, repectively. Note that in contrat to (), the problem (SDP) i affine in R 1. Thi property will be ueful when computing lower bound for the original problem (). Remark 1: The problem (SDP) i in contrat with the computation of performance bound for contrained linear tochatic control problem in [], in which bound were obtained by relaxing the tandard linear quadratic regulator Riccati equation in the other direction and maximizing over the reulting linear matrix inequality. If (5a) i relaxed to the left (i.e. by replacing = with ), one obtain the following 18

3 minimize Tr(P ) ubject to P 0, X 0 4 N T (AXA T X) N T AX N T (AXA T X)M N T G XA T N Q 1 XA T M 0 M T (AXA T X)N M T AX M T (AXA T X BR 1 B T )M M T G G T N 0 G T M I I + G T PG 0, X I I P (SDP) optimization problem in variable P and P maximize Tr(P ) R + B T PB B T PA ubject to A T PB Q+ A T 0 PA P (8) P = P + PG( I G T PG) 1 G T P P 0. Unfortunately, thi problem i not convex. For tochatic linear quadratic regulator problem conidered in [] (recovered with!1and thu P! P ), the ituation i eaier ince the problem become a emidefinite program in P and i alo concave in (Q, R). III. PERFORMANCE BOUND FOR PROBLEMS WITH INPUT CONSTRAINTS Thi ection decribe a method to obtain lower bound on performance for min-max problem with input contraint. The central idea i to find an uncontrained problem whoe optimal cot, which a hown in the previou ection can be efficiently computed, i a lower bound on the optimal cot of the contrained problem. To thi end, we aociate Q 0, R 0, >0 with the tage cot of ome uncontrained problem and denote by P, K and K w the correponding optimal cot and gain matrice given by (5) (). The cloedloop ytem for thi uncontrained problem i x + = A cl x where A cl = A + BK + GK w. We alo define the et X ={x R n K w (A + BK + GK w ) k x W, 8k N}, (9) which correpond to the et of tate for which the optimal uncontrained diturbance equence alway remain in W. We have the following reult, which give a baic lower performance bound for initial tate in X: Lemma 1: Suppoe Q, R, x T Qx + u T Ru Then we have for (0, 1) x T 0 Px 0 + and R atify w T w + `(z,w), 8x R n, 8u U, 8w W. (10) 1 V (x 0), 8x 0 X (11) where P i the optimal cot matrix for the uncontrained min-max problem aociated with Q, R, and. Proof: Oberve that (10) implie min max ww t [x T t Qx t + u T t Ru t wt T w t + ] min max ww t l(z,w). (1) The inequality (1) remain valid if the input contraint in the et of admiible policie on the outer left-hand-ide minimization i dropped. If a given x 0 i in the et X then the contraint in the inner left-hand-ide maximization over diturbance equence can be dropped without changing the value becaue the contraint i never active by the definition of X. The term involving in the left-hand-ide of (1) can be taken out of the optimization and evaluated to yield the econd term on the left-hand-ize of (11). The ret of the left-hand ide i then the uncontrained optimal min-max cot and the right-hand-ide i the optimal cot of the contrained problem, and the reult follow immediately. A. Optimizing the bound Lemma 1 give a baic but efficiently computable lower bound on the value function V for our contrained minimax problem. We next conider the problem of maximizing thi lower bound by allowing the parameter R 1 in (SDP) to vary 1. Let J (R 1 ) denote the optimal value of (SDP). To optimize the lower bound, we can olve the following optimization problem over the parameter R 1 and maximize J (R 1 )+ 1 ubject to (10), R 1 0. (1) Thi problem preent two difficultie. Firt, ince (SDP) i a minimization problem and (1) i a maximization problem, the optimization cannot be done jointly over P, X, R 1 and. We can however form the dual of (SDP) and maximize jointly over R 1, and the dual variable. Thi reult in the following bilinear problem with variable R 1, 1 One could alo contemplate allowing Q 1 to vary. Here, ince we do not conider tate contraint, (10) mut hold on all of R n, and one doe not gain anything by allowing Q 1 to vary. Therefore, a in [], we fix Q to the cot for the contrained problem. 18

4 and dual variable Z, and : max Tr(Z)+Tr(F (R 1 ) ) + Tr( 1 )+ 1 ubject to + I + G T ZG =0, (10), R A T N 11 N T A N 11 N T +A T N 1 +A T N 1 M T A N 1 M T + + M T A + A T M M T A M M T =0 Z 0, = = 4 where F (R 1 ):= T T 1 4 T 1 T 4 T 14 T 4 T N T G 0 Q M T BR 1 B T M M T G G T N 0 G T M I (14) 5. (15) Oberve that there i a product term involving and R 1 in the dual objective, o thi problem i bilinear, and conequently non-convex. However, local and global method for olving uch bilinear optimization problem (e.g. PENBMI [11]) are available. In the next ection we adopt the implet approach, of alternately olving emidefinite program by fixing one variable in the product and optimizing over the other. We how via numerical example that thi method can be ued to produce improved lower bound. The econd difficulty i that (10) i a emi-infinite contraint in general ince it mut hold for all point in R n, U and W. A hown in [], we recall in the following ubection that when `(z,w) i quadratic the contraint (10) can be enforced exactly when U i a finite et, and that it can be replaced with a conervative approximation via the S-procedure in other cae. Remark : The reaon why the dual of (SDP) i preferred to the dual of () i that the latter ha primal variable multiplying D which we allow to vary in the dual problem. A a conequence, the aociated dual problem ha bilinear equality contraint. Since R 1 enter a a contant term in (SDP), the dual problem ha the bilinearity in the objective, which we find more convenient to work with. B. Verifying valid initial tate The bound (11) i valid only for initial tate x 0 X, which i characterized by an infinite et of contraint in (9). We now decribe a procedure to verify that a given x 0 i indeed contained in X. Suppoe we have a ymmetric matrix S and calar R uch that W = {w R l w T Sw } W. Suppoe alo that x T Hx i a diipated quantity for the optimal uncontrained cloed-loop ytem x + = A cl x, where A cl = A + BK + GK w (i.e. H atifie A T cl HA cl H 0). To how x 0 X, it i ufficient to how x T Hx x 0 Hx 0 ) x T K T wsk w x. By the S-procedure (ee e.g. [1]) thi i equivalent to the exitence of 0 atifying K T wsk w H, x T 0 Hx 0. Thu, to verify x 0 X it i ufficient to how that there exit H and 1/ atifying the LMI condition (1/ )K T wsk w H, / x T 0 Hx 0, A T clha cl H 0, 1/ > 0, H 0. (1) If a feaible point i found, the bound i valid for all initial tate in the et {x R n x T Hx x T 0 Hx 0 }. For a particular initial tate, thi procedure may be omewhat conervative. The conervatim can be reduced by evaluating a finite number of the contraint in X and then olving the feaibility problem from the lat tate evaluated. C. Finite input et Suppoe that `(z,w) i a quadratic function in the form `(z,w) =x T Q 0 x + u T R 0 u 0w T w (with 0 =0poible) and the input contraint et i finite (U = {u 1,...,u k }). If we fix Q = Q 0 and 0, then (10) reduce to u T i Ru i + 1 ut i R 0 u i, i =1,...,k, (1) which can be expreed uing Schur complement a LMI in R 1 and : " # u T i R 0u i 1 u T i u i R 1 0, i =1,...,k. (18) Replacing (10) in (14) with (18) and olving the bilinear problem can yield an improved lower bound. D. S-Procedure relaxation Suppoe again the tage cot are quadratic and now that we have R 1,...,R M and 1,..., M for which U Ū = {u ut R i u + i 0, i =1,...,M}. (19) Setting Q = Q 0 and 0 again, a ufficient condition via the S-procedure (ee [1] and []) for (10) to hold i the exitence of nonnegative 1,..., M atifying R R 0 M X i=1 ir i, M 1 X i=1 i i. (0) The firt inequality can be written uing Schur complement a an LMI in R 1 R0 + P M i=1 ir i 0 0 R 1 0. (1) Again, replacing (10) in (14) with (1) and olving the bilinear problem can yield an improved lower bound. 188

5 IV. NUMERICAL EXAMPLE In thi ection we compute a performance bound for an example ytem in which the input contraint et U i a box. The diturbance contraint et i taken to be a unit ball, and the cot i taken to be quadratic: `(z,w) = x T Q 0 x + u T R 0 u 0w T w. Sytem and cot matrice with dimenion n = l =4, m =, and p =were randomly generated, with entrie from A, B, G, and Choleky factor of Q 0 and R 0 drawn from a tandard normal ditribution. The diturbance weight 0 wa et to ten percent larger than the uncontrained H 1 optimal value, and the dicount factor wa et to =0.95. For the aociated uncontrained problem, we et Q = Q 0 and = 0. The problem (SDP) i firt olved with R = R 0 to obtain a baic lower bound. Then the bound i optimized by olving (14) by alternately fixing R 1 and. The correponding emidefinite program were olved uing the modeling language YALMIP [1] and the cone olver SeDuMi [14]. A box contraint i defined by U = {u R m kuk 1 U max }. Thi contraint can be repreented by a et of quadratic inequalitie a follow u T e i e T i u U max, i =1,..., m, where e i i the ith unit vector. In relation to (19), we have R i = e i e T i and i = U max. We now decribe a pecific example. The ytem matrice are: A = , B = G = Q 0 = 4 5,R 0 = Solving (SDP) with Q = Q 0, R = R 0 and = 0 we obtain an optimal value of.5. We then run an alternating emidefinite programming procedure on (14) and obtain the value.4, which i 8% improvement on the baic bound obtained from (SDP). Set of initial tate for which thee bound are valid can be computed via (1). V. CONCLUSIONS Thi paper developed a method to compute performance bound for infinite-horizon linear min-max control problem with input contraint. The method require a bilinear matrix inequality to be olved. We howed that improved bound can be computed by olving emidefinite program in an alternating fahion. Further reearch direction include computing bound for problem with tate contraint and comparion of the bound with uboptimal policie. APPENDIX A. Infinite-horizon dicounted H 1 problem Thi ection derive the olution of the infinite-horizon uncontrained linear min-max optimal control problem with a dicounting factor. The quadratic form z T Qz i denoted by kzk Q. Given the ytem x + = Ax + Bu + Gw, conider the problem of minimizing the wort-cae dicounted cot function with dicount factor (0, 1) J(x 0,,w) := k (kxk Q + kuk R kwk ). The Iaac equation i where k=0 V =minmax [`(x, u, w)+ V (Ax + Bu + Gw)], u w =min kxk Q + kuk R + J(x, u) u J(x, u) :=max w ( )kwk + V (Ax + Bu + Gw). Aume that V (x) =x T Px. The inner maximization problem can be olved explicitly by rewriting it firt a J(x, u) = max kwk + kax + Bu + Gwk P w h = max kwk w ( I + kax + Buk P The optimizer w (x, u) for the above i w (x, u) = I o that J(x, u) =(Ax + Bu) P (Ax + Bu) i G T ( P )G) + wt G T ( P )(Ax + Bu) G T ( P )G 1 G T ( P )(Ax + Bu), P := ( P )+( P )G I G T ( P )G 1 G T ( P ). The value function can then be written a V (x) =min kxk Q + kuk Q + J(x, u) u =min kxk Q + kuk Q +(Ax + Bu) T P (Ax + Bu). u The ret of the argument then follow along the line of thoe in [10], yielding P = Q + A T PA A T PB(R + B T PB) 1 B T PA () P := ( P )+( P )G I G T ( P )G 1 G T ( P ). () The optimal input and diturbance policie are K u = (R + B T PB) 1 B T P A, K w =( I G T ( P )G) 1 G T ( P )(A + BK u ). 189

6 We can now try to equate a olution of the above to ome other undicounted problem. Rewrite () a P =( P )+ 1 1 ( P )G I GT PG G T ( P ) h i = P + PG I G T PG 1 G T P = P where := / p and P := P + PG I G T PG 1 G T P. Then ubtitute P = P into () to obtain P =Q+A T ( P)A A T ( P)B(R+B T ( P)B) 1 B T ( P)A = Q + ÃT Pà ÃT P B(R + BT P B) 1 BT Pà where à := p A and B := p B. The concluion i that the olution to the dicounted H 1 problem can be obtained by olving the equation P = P + PG I G T PG 1 G T P P = Q + ÃT Pà ÃT P B(R + BT P B) 1 BT Pà i.e. by olving an undicounted problem uing the problem data à p A, B p B and / p. The value function for the dicounted problem i then V (x) =x T Px, where P come from the olution to the undicounted H 1 problem uing the modified problem data. Remark : We note that the control that minimize the infinite-horizon dicounted cot i not necearily tabilizing for the ytem (1), even if the optimal cot i finite. However, only the cot i ued when determining a lower bound. B. Equivalence of Riccati Inequality and Bounded-Real LMI Thi ection demontrate the equivalence between the Riccati inequality obtained by replacing = with in (5) and the bounded-real LMI in (). We firt recall a reult from [8]. Theorem 1 (Thm.., [8]): For the ytem x + = Ax + Gw z = Cx, the following tatement are equivalent: (4) 1) A i a table matrix and kc(zi A) 1 Gk 1. ) There exit a tabilizing olution P = P T 0 to the Riccati equation P =A T PA+ 1 AT PG(I 1 GT PG) 1 G T PA+ C T C. (5) Claim 1: If one ubtitute A! (A + BK) and C! (C + DK) in (5) with K a in (), then (5), (5). Proof: We have from (5) P = A T PA+ A T PG G T PG 1 G T PA+ C T C = A T PA+ A T ( P P )A + C T C = A T PA+ C T C, where the firt tep ue the definition of P from (5). Next replace A! (A + BK) and C! (C + DK) to get P =(A + BK) T P (A + BK)+(C + DK) T (C + DK) =(A + BK) T P (A + BK)+Q + K T RK = A T PA+ Q + K T (R + B T PB)K + A T PBK + K T B T P A, () where the firt tep come from the aumption D T D = R, C T C = Q, and C T D =0. Recalling the definition of K in (), the final three term in () can be rewritten a K T (R + B T PB)K + A T PBK + K T B T PA () = A T PB(R + B T PB) 1 B T P A. (8) Subtituting thi into () reult in P = A T PA+ Q A T PB(R + B T PB) 1 B T P A, which i identical to (5). Finally, if we now relax (5) by replacing = with, apply the Schur complement lemma three time and ubtitute A! (A + BK) and C! (C + DK), we arrive at the boundedreal LMI in (). REFERENCES [1] D. Berteka and S. Shreve. Stochatic optimal control: The dicrete time cae, volume 19. Athena Scientific, 198. [] D.Q. Mayne, SV Raković, RB Vinter, and EC Kerrigan. Characterization of the olution to a contrained H 1 optimal control problem. Automatica, 4():1 8, 00. [] S. Lall and K. Glover. A game theoretic approach to moving horizon control. Advance in model-baed predictive control, page , [4] M.V. Kothare, V. Balakrihnan, and M. Morari. Robut contrained model predictive control uing linear matrix inequalitie. Automatica, (10):11 19, 199. [5] J. Lofberg. Approximation of cloed-loop minimax MPC. In Proceeding of the 4nd IEEE Conference on Deciion and Control, page , 00. [] P.J. Goulart, E.C. Kerrigan, and T. Alamo. Control of contrained dicrete-time ytem with bounded ` gain. IEEE Tranaction on Automatic Control, 54(5): , 009. [] Y. Wang and S. Boyd. Performance bound for linear tochatic control. Sytem & Control Letter, 58():18 18, 009. [8] C.E. de Souza and L. Xie. On the dicrete-time bounded real lemma with application in the characterization of tatic tate feedback H 1 controller. Sytem & Control Letter, 18(1):1 1, 199. [9] P. Gahinet and P. Apkarian. A linear matrix inequality approach to h control. International Journal of Robut and Nonlinear Control, 4(4):41 448, [10] T. Baar and P. Bernhard. H 1 optimal control and related minimax deign problem. Birkhauer Boton, 004. [11] M. Kocvara and M. Stingl. PENBMI uer guide. Available from [1] S. Boyd, L. El Ghaoui, E. Feron, and V. Balakrihnan. Linear matrix inequalitie in ytem and control theory, volume 15. Society for Indutrial and Applied Mathematic, [1] J. Lofberg. Yalmip: A toolbox for modeling and optimization in matlab. In IEEE International Sympoium on Computer Aided Control Sytem Deign, page IEEE, 004. [14] J.F. Sturm. Uing edumi 1.0, a matlab toolbox for optimization over ymmetric cone. Optimization method and oftware, 11(1-4):5 5,