Connections Between Duality in Control Theory and

Transcription

1 Connections Between Duality in Control heory an Convex Optimization V. Balakrishnan 1 an L. Vanenberghe 2 Abstract Several important problems in control theory can be reformulate as convex optimization problems. From uality theory in convex optimization, ual problems can be erive for these convex optimization problems. hese ual problems can in turn be reinterprete in control or system theoretic terms, often yieling new results or new proofs for existing results from control theory. Moreover, the most ecient algorithms for convex optimization solve the primal an ual problems simultaneously. Insight into the system-theoretic meaning of the ual problem can therefore be very helpful in eveloping ecient algorithms. We emonstrate these observations with some examples. 1. Introuction Over the past few years, convex optimization has come to be recognize as a valuable tool for control system analysis an esign via numerical methos. Convex optimization problems enjoy a number of avantages over more general optimization problems: Every stationary point is also a global minimizer; they can be solve in polynomial-time; we can immeiately write own necessary an sucient optimality conitions; an there is a well-evelope uality theory. From a practical stanpoint, there are eective an powerful algorithms for the solution of convex optimization problems, that is, algorithms that rapily compute the global optimum, with non-heuristic stopping criteria. hese algorithms range from simple escent-type or quasi-newton methos for smooth problems to sophisticate cutting-plane or interiorpoint methos for non-smooth problems. A comprehensive literature is available on algorithms for convex programming; see for example, [1] an [2]; see also [3]. 1 School of Electrical Engineering, Purue University, West Lafayette, IN , ragu@ecn.purue.eu. 2 K. U. Leuven, Electrical Engineering Department, K. Mercierlaan 94, 31 Leuven, Belgium, Lieven.Vanenberghe@esat.kuleuven.ac.be. Research supporte by the NFWO, an IUAP projects 17 an 5. A typical application of convex optimization to a problem from control theory procees as follows: he control problem is reformulate (or in many cases, approximately reformulate) as a convex optimization problem, which is then solve using convex programming methos. Once the control problem is reformulate into a convex optimization problem, then it is straightforwar to write own the ual convex optimization problem. his ual problem can be often be reinterprete in control-theoretic terms, yieling new insight. he control-theoretic interpretation of the ual problem in turn helps in the ecient (numerical) implementation of primal-ual algorithms, which are among the most ecient techniques known for solving convex optimization problems. In this paper, we illustrate each of these points. First, we examine the stanar LQR problem from control theory, an show how convex uality provies insight into its solution. We then iscuss the implementation of primal-ual algorithms for another important control problem, namely the Linear Quaratic Regulator problem for linear time-varying systems with state-space parameters that lie in a polytope. 2. Convex programming uality In the sequel, we will be concerne with convex optimization problems involving linear matrix inequalities or LMIs. hese optimization problems have the form where minimize c x subject to F (x) > F (x) = F + x 1 F x m F m : (1) he problem ata are the vector c = R m an m + 1 symmetric matrices F, F 1,..., F m 2 R nn. he inequality F (x) > means that F (x) is positive efinite. We call problem (1) a semienite program. 3 Semienite programs can be solve eciently using recently evelope interior-point methos (see [2, 4]). he book [5] lists a large number of problems in 3 Strictly speaking, the term \semienite program" refers to problem (1) with the constraint F (x) instea of F (x) >.

2 control an system theory that can be reuce to a semienite program. As a consequence of convexity, we have a complete uality theory for semienite programs. For the special form of problem (1) uality reuces to the following. With every problem (1) we associate a ual problem maximize r F Z subject to Z > r F i Z = c i ; i = 1; : : :; m: (2) Here r X enotes the trace of a matrix X. he variable in (2) is the matrix Z = Z 2 R nn. We have the following properties. If a matrix Z is ual feasible, i.e., Z > an r F i Z = c i, i = 1; : : :; m, then the ual objective r F Z is a lower boun for the optimal value of (1): r F Z inf fc xjf (x) > g: If an x 2 R m is primal feasible, i.e., F (x) >, then the primal objective c x is an upper boun for the optimal value of (2): 8 < c x sup : r F Z Z = Z > ; r F i Z = c i ; i = 1; : : :; m 9 = ; : Uner mil conitions, the optimal values of the primal problem (1) an its ual (2) are equal. 3. Primal-ual algorithms Primal-ual algorithms are a class of iterative numerical algorithms for solving semienite programs. hese algorithms solve problems (1) an (2) simultaneously; as they procee, they generate a sequence of primal an ual feasible points x (k) an Z (k) (k = ; 1; : : : enotes iteration number). his means that for every k, we have an upper boun c x (k) an a lower boun r F Z (k) on the optimal value of problem (1). General primal-ual interior-point methos that solve semienite programs are often more ecient than methos that work on the primal problem only. heir worst-case complexity is typically lower, an they are often faster in practice as well. An important class of interior-point methos is base on the primal-ual potential function (x; Z)=(n+ p n) log(c x+r F Z)log et F (x)z: ( 1 is xe.) If a metho ecreases this function by at least a xe amount, inepenent of the problem size, in every iteration, then it can be shown that the number of iterations grows at most as O( p n) with the problem size. In practice the number of iterations appears to grow slower with n. Moreover the amount of work per iteration can be reuce consierably by taking avantage of the structure in the equations (see [6]). An outline of a potential-reuction metho ue to Nesterov an o [7] this is the algorithm use for solving the semienite programs that occur in this paper is as follows. he metho starts at strictly feasible x an Z. Each iteration consists of the following steps. 1. Compute a matrix R that simultaneously iagonalizes F (x) 1 an Z: R F (x) 1 R = 1=2 ; R ZR = 1=2 : he matrix is iagonal, with as iagonal elements the eigenvalues of F (x)z. 2. Compute x 2 R m an Z = Z 2 R nn from RR ZRR + P m x if i = F (x) + Z 1 r F j Z = ; j = 1; : : :; m with = (n + p n)=(c x + r F Z). 3. Fin p; q 2 Rthat minimize (x+px; Z +qz) an upate x := x + px an Z := Z + qz. For etails, we refer the reaer to [7]; see also [4]. 4. Convex uality an control theory We rst consier the stanar Linear Quaratic Regulator problem, an show how convex uality escribe in x2 can be use to reinterpret the stanar LQR solution. We then consier a multi-moel (or \robust") version of the LQR problem, an escribe an application of the primal-ual algorithm of x3 for computing bouns for this problelm he Linear Quaratic regulator Consier the following optimal control problem: For the system n u that minimizes J = _x = Ax + Bu; x() = x ; (3) x(t) Qx(t) + u(t) Ru(t) t; (4) with Q an R >, subject to lim t!1 x(t) =. We assume the pair (A; B) is controllable. Let J opt enote the minimum value.

3 Lower boun via quaratic functions We can write own a lower boun for J opt using quaratic functions; the following is essentially from [8, heorem 2]. Suppose the quaratic function P with P > satises t x(t) P x(t) > x(t) Qx(t) + u(t) Ru(t) ; (5) for all t, an for all x an u satisfying _x = Ax + Bu, x( ) =. hen, integrating both sies from to, we get x P x < Z x(t) Qx(t) + u(t) Ru(t) t; or we have a lower boun for J opt. Conition (5) hols for all x an u (not necessarily those that steer state to zero) if the Linear Matrix Inequality A P + P A + Q P B B > (6) P R is satise. hus, the problem of computing the best lower boun using quaratic functions is maximize: x P x subject to: P > ; (6) (7) he optimization variable in problem (7) is the symmetric matrix P. Upper boun with state-feeback Consier system (3) with a constant, linear statefeeback u = Kx that stabilizes the system: _x = (A + BK)x; x() = x ; (8) with A + BK stable. hen the LQR objective J reuces to J K = x(t) Q + K RK x(t) t: Clearly, for every K, J K yiels an upper boun on the optimum LQR objective J opt. From stanar results in control theory, J K can be evaluate as where Z satises r Z(Q + K RK); (A + BK)Z + Z(A + BK) + x x = ; with A + BK stable. It will be useful for us to rewrite this expression for J K as inf Z> r Z(Q + K RK); where Z satises (A + BK)Z + Z(A + BK) + x x < : (9) hus, the best upper boun on J opt, achievable using state-feeback control, is given by the optimization problem with the optimization variables Z an K: minimize: r Z(Q + K RK) subject to: Z > ; (9) Duality We observe the following: Problems (7) an (1) are uals of each other. (1) Proof: he proof is by irect verication. For problem (7), the ual problem is given by Q minimize r Z; over Z; R subject to Z = Z Z11 Z = 12 Z12 > Z 22 I A [A B] Z + [I ] Z B + x x < : With the change of variables K = Z12Z 1 11, an some stanar arguments, we get the equivalent problem with variables 1 an K: minimize: r(q + K RK)1 ; subject to: 1 > ; (A + BK)1 + 1 (A + BK) +x x < ; (11) which is the same as problem (1). Note that this shows that the optimal solution to the LQR problem is a linear state-feeback he multi-moel LQR problem Let us now consier a mutlti-moel version of the LQR problem. We consier the multi-moel or polytopic system (see [5]) t x(t) = A(t)x(t) + B(t)u(t); x() = x ; (12) where for every time t, [A(t) B(t)] 2 = Co f[a 1 B 1 ] ; : : :; [A L B L ]g : (13) Our objective now is to n u that minimizes sup A();B()2 x(t) Qx(t) + u(t) Ru(t) t; with Q an R >, subject to lim t!1 x(t) =. Let J opt enote the minimum value. 2

4 Lower boun via quaratic functions We now repeat the steps of the previous section to write own a lower boun for J opt using quaratic functions. Suppose the quaratic function P with P > satises t x(t) P x(t) > x(t) Qx(t) + u(t) Ru(t) ; (14) for all t, an for all x an u satisfying (12) with x( ) =. hen, integrating both sies from to, we get x P x < Z x(t) Qx(t) + u(t) Ru(t) t; or we have a lower boun for J opt. Conition (14) hols for all x an u if the inequality A(t) P + P A(t) + Q P B(t) B(t) > P R hols for all t, which in turn is equivalent to A i P + P A i + Q P B i Bi P R > ; i = 1; : : :; L: (15) hus, the problem of computing the best lower boun via quaratic functions is maximize: x P x subject to: P > ; (15) (16) he optimization variable in problem (16) is the symmetric matrix P. he ual of problem (16) is minimize LX r over Z i, i = 1; : : :; L, subject to Q R Z i ; Z i = Zi Zi;11 Z = i;12 Z > i;12 Z i;22 [A i B i ] Z i I + [I ] Z i A i B i + x x < : With the change of variables K i = Z i;12 Z1, an i;11 some stanar arguments, we get the equivalent problem with variables Z i;11 an K i : minimize: r (Q + K i RK i )Z i;11 ; subject to: Z i;11 > ; ((A i + B i K i )Z i;11 +Z i;11 (A i + B i K i ) + x x < : (17) We are not aware of a nice control-theoretic interpretation of the ual problem at the time of writing of this paper. It is easy to calculate feasible points for (17) by solving an LQR problem (recall that this is important for the application of the primal-ual algorithm of x3). Select any system [ A; B] from the convex hull (13), i.e., choose A = LX i A i ; B = L X i B i for some i, i = 1; : : :; L, i = 1. By solving the LQR problem with A, B, we obtain matrices 1 > an K that satisfy ( A + B K) ( A + B K) + x x < : (18) From this it is clear that Z i;11 = i Z11, K i = K, are feasible solutions in (17). hose ual solutions can be use as starting points for a primal-ual algorithm. Upper boun with state-feeback Restricting u to be a constant, linear state-feeback yiels an upper boun on J opt. With u = Kx, the equations governing system (12) are t x(t) = (A(t) + B(t)K) x(t); x() = x ; (19) with the matrices A an B satisfying (13). hen the LQR objective J reuces to J K = sup A();B() x(t) Q + K RK x(t) t: Once again, for every K, J K yiels an upper boun on the optimum LQR objective J opt. Unlike with the LQR problem however, J K is not easy to compute. We therefore present a simple upper boun for J K using quaratic functions. Suppose the quaratic function P with P > satises t x(t) P x(t) < x(t) Q + K RK x(t); (2) for all t, an for all x an u satisfying (12) with x( ) =. hen, integrating both sies from to, we get x P x > Z x(t) Q + K RK x(t) t; or we have an upper boun for J opt. Conition (2) hols for all x an u (not necessarily those that steer state to zero) if the inequality (A(t) + B(t)K) P + P (A(t) + B(t)K)+ Q + K RK <

5 hols for all t, which in turn is equivalent to P 1 (A i + B i K) + (A i + B i K) P 1 + P 1 Q + K RK P 1 < ; i = 1; : : :; L: With the change of variables W = P 1 an Y = KP 1, we get the matrix inequality (which can be written as an LMI using Schur complements) W A i + A iw + B i Y + Y Bi W QW + Y RY < ; i = 1; : : :; L: (21) hus the best upper boun on J opt using constant state-feeback an quaratic functions can be obtaine by solving the semienite program with variables W = W an Y : he ual problem is maximize subject to : Z i =Z i = minimize r x W 1 x subject to W > ; (21) 2 (r Z i;22 + r Z i;33 ) 2z x ; 4 Z i;11 Z i;12 Z i;13 Zi;12 Z i;22 Z i;23 Z i;13 Z i;23 Z i;33 (22) 3 5 > Zi;11 A i + A i Z i;11 +Z i;12 Q 1=2 + Q 1=2 Zi;12 > zz B i Z i;11 + R 1=2 Z i;13 = (23) where the variables are the L matrices Z i an the vector z. As with the lower boun, it is possible to obtain a ual feasible solution by solving an LQR problem. We omit the etails here. A Numerical Example Figure 1 shows the results of a numerical example. he ata are ve matrices A i 2 R 55 an ve matrices B i 2 R 53. he gures shows the objective values of the four semienite programs that we iscusse above. 5. Conclusion We have consiere an optimal control problem with a quaratic objective, an have shown how we may obtain useful bouns for the optimal value using LMIbase convex optimization. Convex uality can be use to reerive the well-known LQR solution; control theory uality can be use to evise ecient primalual convex optimization algorithms. he results presente herein are preliminary; it woul be interesting to erive control-theoretic interpretations of the many primal-ual convex optimization (c) (b) (a) 2 () iteration Figure 1: Upper an lower bouns versus iteration number. Curve (a) shows the lower boun (16) uring execution of the primal-ual algorithm. Curve (b) shows the value of the associate ual problem (17). Curve (c) shows the upper boun (22) uring execution of the primal-ual algorithm. Curve () shows the value of the associate ual problem (23). problems presente here. Also of interest woul be a careful stuy of the numerical avantages gaine by using primal-ual algorithms over primal-only solvers. he primal-ual metho outline in this paper can be use to eciently solve a large class of semienite programs from system an control theory, e.g., most of the ones presente in the book [5]. References [1] J.-B. Hiriart-Urruty an C. Lemarechal. Convex Analysis an Minimization Algorithms II: Avance heory an Bunle Methos, volume 36 of Grunlehren er mathematischen Wissenschaften. Springer-Verlag, New York, [2] Yu. Nesterov an A. Nemirovsky. Interior-point polynomial methos in convex programming, volume 13 of Stuies in Applie Mathematics. SIAM, Philaelphia, PA, [3] S. Boy an C. Barratt. Linear Controller Design: Limits of Performance. Prentice-Hall, [4] L. Vanenberghe an S. Boy. Semienite programming. Submitte to SIAM Review, July [5] S. Boy, L. El Ghaoui, E. Feron, an V. Balakrishnan. Linear Matrix Inequalities in System an Control heory, volume 15 of Stuies in Applie Mathematics. SIAM, Philaelphia, PA, June [6] L. Vanenberghe an S. Boy. Primal-ual potential reuction metho for problems involving matrix inequalities. o be publishe in Math. Programming, [7] Yu. E. Nesterov an M. J. o. Self-scale cones an interior-point methos in nonlinear programming. echnical Report 191, Cornell University, April [8] J. C. Willems. Least squares stationary optimal control an the algebraic Riccati equation. IEEE rans. Aut. Control, AC-16(6):621{634, December 1971.