Notes on duality in second order and -order cone otimization E. D. Andersen Λ, C. Roos y, and T. Terlaky z Aril 6, 000 Abstract Recently, the so-calle

Transcription

1 McMaster University Advanced Otimization Laboratory Title: Notes on duality in second order and -order cone otimization Author: Erling D. Andersen, Cornelis Roos and Tamás Terlaky AdvOl-Reort No. 000/8 May 000, Hamilton, Ontario, Canada

2 Notes on duality in second order and -order cone otimization E. D. Andersen Λ, C. Roos y, and T. Terlaky z Aril 6, 000 Abstract Recently, the so-called second order cone otimization roblem has received much attention, because the roblem has many alications and the roblem can at least in theory be solved efficiently by interior-oint methods. In this note we treat duality for second order cone otimization roblems and in articular whether a nonzero duality ga can be introduced when casting a convex quadratically constrained otimization roblem as a second order cone otimization roblem. Furthermore, we also discuss the -order cone otimization roblem which isanatural generalization of the second order case. Secifically, we suggest a new self-concordant barrier for the -order cone otimization roblem. Introdution The second order cone otimization roblem can be stated as (SOC) minimize f T x jja i x b i jj» c i: x d i ; i =;:::;k; Hx = h: where A i R (mi ) n and H R l n and all the other quantities have conforming dimensions. c i: denotes the ith row of C R k n. k k denotes the Euclidean norm. Clearly, the roblem (SOC) is a convex but non-smooth roblem because the norm is not differentiable at zero. We will not survey the numerous alications for this otimization model here, but rather refer the reader to [4]. However, it can be observed that linear, convex quadratic, and convex quadratically constrained otimization roblems all can be stated as second order cone otimization roblems. For examle if m i = for all i's, then (SOC) reduces to an ordinary linear otimization roblem. The outline of the aer is as follows. We first discuss duality for second-order cone otimization roblems and resent two examles which demonstrate that nonzero duality ga can occur. Next we show that if a convex quadratically constrained otimization roblem is formulated as a second order cone otimization roblem then a ositive duality ga cannot occur. Finally, we discuss the -order generalization of the second order cone case. Λ TU Delft, Mekelweg 4, 68 CD Delft, The Netherlands, e.d.andersen@mosek.com. y TU Delft, Mekelweg 4, 68 CD Delft, The Netherlands, c.roos@its.tudelft.nl. z McMaster University, Deartment of Comuting and Software, Hamilton, Ontario, Canada. L8S 4L7. terlaky@cas.mcmaster.ca. art of this research was done while the author was emloyed at TU Delft.

3 Duality The dual roblem corresonding to (SOC) is (SOCD) maximize b T z + d T w + h T v A T z + C T w + H T v = f; jjz i jj» w i ; i =;:::;k; where z i R m i and w R k and we use the notation that A := 6 4 A. A k The following duality theorem is well-known. and z := Theorem. Let ν SOC denote the otimal objective value of (SOC) where minimize is relaced by inf. Similarly, let ν SOCD denote the otimal objective value of (SOCD) where maximize is relaced by su. Then the following holds:. Weak duality: If both (SOC) and (SOCD) are feasible, then ν SOC ν SOCD.. Strong duality: If either (SOC) or (SOCD) is Slater regular, then ν SOC = ν SOCD. Moreover, If both the rimal and dual roblem are Slater regular, then both roblems have an otimal solution and ν SOC = ν SOCD. (We use the convention ν SOC = if (SOC) is infeasible. Similarly, ν SOCD = if (SOCD) is infeasible.) roof: [] Subsequently, we will resent some examles that demonstrates that strong duality in general requires the Slater regularity condition, because otherwise there might be a ositive duality ga between the rimal and dual roblem.. Infinite duality ga The first examle is 6 4 z. z k : minimize x q x + x x» 0; which has f(x ;x ): x 0; x =0g as the set of feasible solutions. From here it is obvious that the otimal objective value ν SOC is zero and the roblem is not Slater regular. The dual roblem corresonding to () is maximize 0 z + w = 0; q z = ; z + z» w : () () The constraints of () can be reduced to q w +» w which imlies () is infeasible thus ν SOCD =. Hence, in this case the duality ga is infinity.

4 . Finite but nonzero duality ga Next consider the examle minimize x q x +(x )» x ; q ( x + x )» x : (3) The first constraint clearly imlies that x follows from the second constraint that = in any feasible solution. Given that fact it» x : Hence, the feasible set consists of f(x ;x ): x ; x =g and the otimal objective value is. The dual roblem corresonding to (3) is maximize z z + w z 3 + w = 0; q z + z 3 = ; z + z» w ; z3» w : (4) Since, it follows from the two last constraints that then Using the first constraint this imlies Now using the second constraint we have that Therefore, (4) is equivalent to w jz j and w jz 3 j; w + z 0andw z 3 0: w = z and w = z 3 : z = z 3 = w : maximize q w w +( w )» w ; (5) qw» w which has the feasible set f(w ;w ): w 0; w =g and the otimal objective value is zero. Hence, we have constructed an examle where both the rimal roblem () and dual roblem (4) has an otimal solution, but nevertheless the duality ga is nonzero. The reader can verify that if (x ) is relaced by (x ff) in the roblem definition (3), then for any ff>0therewill be a ositive duality ga of size ff. 3

5 .3 Non-attainment The examle minimize x " x x #» x (6) shows that the otimal objectivevalue is not always attained. The reason is that the constraint imlies +x» x x : (7) This shows that the otimal objective value is zero but cannot be attained..4 From quadratically constrained otimization to second order cone otimization According to [4] then one of the advantages of second order cone otimization is that convex quadratically constrained otimization can be cast as a second order cone otimization. However, an issue not addressed in [4] is whether recasting a quadratically constrained otimization roblem as a second order cone otimization roblem can introduce ositive duality ga. As demonstrated in the revious section a ositive dualtity ga might occur. This would be very bad because it has already been demonstrated in [8, 9, 0] that there exist a dual roblem corresonding to any convex quadratically constrained otimization roblem which has zero duality ga. We will therefore address the issue whether a convex quadratically constrained otimization roblem recast as a second order cone otimization roblem has worse duality roerties than the originally roblem. Any convex quadratically constrained otimization roblem can be stated on the form minimize c T x jj(qi ) T xjj a i: x + b i» 0; i =;:::;m; where Q i R n l i and A Rm n. The remaining quantities are assumed to have conforming dimensions. The ordinary Lagrange dual corresonding to (8) is identical to maximize b T y m y ijj(q i ) T xjj c + m y i Q i (Q i ) T x A T y = 0; (9) y 0: (8) Now using the definition we obtain the alternative dual roblem maximize z i := y i (Q i ) T x b T y m jjz i jj y i c + m Q i z i A T y = 0; y 0; (0) which aears in [0]. The reader can easily verify that (9) and (0) are equivalent. Observe that the dual roblem does not contain any rimal variables and has linear constraints only. 4

6 Using the duality theory develoed for ` rogramming resented in [8, 9, 0] we obtain the roosition: roosition. Given that both (8) and (0) has a feasible solution, then the duality ga between those roblems are zero and (8) attains its otimum value. Moreover, the duality ga between (8) and (9) is zero as well. roof: Given (8) and (0) both has a feasible solution then it follows from [0] that the duality gabetween (8) and (0) is zero and (8) attains its otimum value. Hence, we have that (0) has a feasible solution (^y; ^z i )having bounded objective value i.e. Obviously if the system b T ^y mx mx jj^z i jj > : () ^y i ^y i Q i (Q i ) T x = mx Q i^z i () has a solution, then (9) also has a solution. Assume the contrary is the case i.e. () does not have solution. This imlies there exists a u such that u T m X ^y i Q i (Q i ) T =0 (3) and u T m X Q i^z i 6=0: (4) Now multily both sides of (3) from the left by u and is obtained. Therefore, u T m X ^y i Q i (Q i ) T u = (Q ^y i i ) T u mx (Q ^y i i ) T u =0; 8i =0 holds and hence (Q i ) T u =0if ^y i > 0. On the other hand if ^y i =0,then () imlies ^z i =0. The combination of these two facts imlies u T Q i^z i =0; 8i which is a contradiction to (4). Therefore, we conclude if (0) is feasible, then (9) is feasible. Now let ^x be any solution to (), then 0» m = m = m = m k^y i (Q i ) T ^x ^z i k ^y i k^y i (Q i ) T ^xk +k^z i k ^y i (^z i ) T (Q i ) T ^x ψ i k^z k ^y i ψ k^z i k ^y i ^y i (Qi! +^y i ) T ^x (Qi! ^y i ) T ^x m (^y i Q i (Q i ) T ^x) T ^x (5) 5

7 and it follows mx (Q ^y i i ) T ^x» mx k^z i k ^y i : (6) Clearly, (^x; ^y) is a feasible solution to (9) and (6) shows that this solution has the same or a better objective value than the feasible solution (^y; ^z) to the roblem (0). In summary we have roved for any feasible solution to (0) with a bounded objective value, then there exists a feasible solution to (9) having the same or a better objective value. This imlies that the duality ga between (8) and (9) is zero. It has been roved in [3] that roblem (8) and roblem (0) satisfies the self-concordant condition and thus both roblems can be solved efficiently. Alternatively roblem (8) can be recast as a second order cone otimization roblem and solved as such. One way to erform the reformulation of (8) is as follows. First introduce two additional variables t and u to obtain minimize c T x jj(q i ) T xjj» u i t i ; i =;:::;m; u t = Ax b; u + t = e: (7) e is the vector of all ones. Note that this formulation imlicitly contains the constraints u t =(u i + t i )(u i t i ) 0 and u i + t i 0 which imlies that u i 0 in all feasible solutions. roblem (7) is equivalent to minimize c T x (Qi ) T x t i» u i ; i =;:::;m; u = e + (Ax b) ; t = e (Ax b) which after elimination of the variables u and t leads to the second order cone otimization roblem minimize c T x (Q i ) T x (a i:x b i )» + (a i:x b i ) (9) ; i =;:::;m: The dual roblem corresonding to (9) is (8) maximize (b +e)t μz + (b e)t w k (Q i )z i AT w A T μz = c; zi» w i ; ;:::;m: μz i (0) Now our question can be stated as if there is a ositive duality ga between the rimal roblem (9) and the dual roblem (0)? The answer is given in the subsequent roosition. roosition. Given both (8) and (9) have a feasible solution, then both roblem (9) and roblem (0) has a feasible solution. Moreover, the duality ga is zero. 6

8 roof: It should be obvious that any feasible solution to (8) essentially defines a feasible solution to (9) as well having the same objective value for both roblems. Moreover, given the assumtions then (9) has a feasible solution (^x; ^y). Next let z i = ^y i (Q i ) T ^x; i =;:::;m; w = ^y i ψ k(q i ) T ^xk μz = w ^y: 4 +! ; i =;:::;m; which we claim is a feasible solution of (0). Clearly w 0 and if these values are substituted into (0) and the resulting roblem is simlified, then we obtain the roblem minimize b T ^y m ^y i (Q i ) T ^x c + m ^y i (Q i )(Q i ) T x A T ^y = 0; ^y 0: Obviously, (9) and () are identical. Hence, we have demonstrated that any feasible solution to (9) can easily be converted to a feasible solution to (). Moreover, this leaves the objective value of the solution unchanged. In conclusion given the duality ga between (8) and (9) is zero, then the duality ga between the roblems (9) and (0) is zero as well. Hence, nothing is lost (duality wise) by reformulating a quadratically constrained otimization roblem as a second order cone otimization roblem. Finally, observe that the dual roblem (0) is equivalent to the following second order cone otimization roblem maximize b T y m t i c + m Q i z i A T y = 0; jjz i jj» y i t i ; y 0 involving the so-called rotated quadratic cone. One obvious question is which of the many formulations of the convex quadratically constrained otimization can be solved most efficiently. The answer is that an f-otimal solution can be obtained in O( m ) Newton stes for the roblems (8), (9), and (0) using an f interior-oint algorithm. This is roved in [5], [7], and [3] Therefore, from a comlexity oint of view all three formulations of the quadratically constrained otimization roblem is equally difficult to solve. However, only the second order cone otimization formulation has the secial roerty that the roblem is self-dual (see next section) and the cone is homogeneous [7]. This imlies that efficient rimal-dual algorithms exist for this class of roblems and not for the other formulations [7]. Hence, casting a convex quadratically otimization roblem as second order cone otimization roblem and solving it using a rimal-dual algorithm might be the most efficient way. However, this still has to be verified in ractice. () () 7

9 3 -order cone otimization A generalization of the second order otimization model is the -order cone otimization roblem which can be exressed as follows where A R m n. Moreover, let x i R n i In this case we use the definition Given i > and (OC) minimize c T x Ax = b; x i K i ; i =;:::;k; K i := then the dual cone corresonding to K i is K Λ i := and x := 6 4 x x. x k : 8 0 >< X >: xi R n i : n i i + q i = j= 8 0 >< X >: si R n i : n i j= jx i jj i js i jj q i 9 i > = A 9 q i > = A >; : >; : (3) The second order case is when = and in that case the rimal and dual cone is identical i.e. self-dual. The dual roblem corresonding to (OC)is (DOC) maximize b T y A T y + s = c; s i K Λ i ; i =;:::;k; where s i and s is constructed as x i and x. This has been shown in []. Solution of the -order cone roblem has already been studied by Xue and Ye []. Indeed they develo several olynomial time algorithm using different self-concordant barriers for the cone. Subsequently, we resent a new self-concordant barrier for the -order cone which has a better barrier arameter than the one suggested by Xue and Ye []. Moreover, we will demonstrate that both (OC) and (DOC) can be reformulated as ordinary smooth convex rograms. This has the advantage that the roblems can be solved using existing otimization software. Although it might not be as efficient as using secial urose algorithms. By definition the constraint (r;x) K where K has the form (3) is equivalent to kxk» r (4) 8

10 where we assume that x R l and r R. Although this constraint exresses a convex set then it is nonsmooth, because the norm is not differentiable at zero. However, it is easy to verify that constraint (4) can be relaced by the constraints which in turn is equivalent to the constraints 0» r; kxk r» 0; (5) 0» r; kxk r» 0: (6) r Note in articular for the second order case i.e. =,thenthe function kxk is a smooth convex function on its domain f(r;x):r>0g. Moreover, it can be roven that r ln(r r kxk ) ln(r) =ln(r kxk ) is a -self-concordant barrier for the constraint (6). This imlies in the second order case, that the cone constraints can be relaced by ordinary" convex constraints and the resulting rogram can be solved in olynomial time using a standard interior-oint algorithm. On the other hand if is odd and larger than, then the constraint (6) is not a smooth convex constraint. However, the constraints (6) be relaced by which is identical to l t i» r; jx i j r» t i ; i =;:::;l; r;t i 0; i =;:::;l; l t i» r; jx i j» t i r ; i =;:::;l; r;t i 0; i =;:::;l: Using the usual trick by introducing some additional constraints, then we can get rid of the absolute sign as follows l r i» t; x i t i r» 0; i =;:::;l; x i t i r» 0; i =;:::;l; r;t i 0; i =;:::;l: Clearly, l new constraints and variables has been introduced into the roblem. Moreover, we have the following roosition (7) roosition 3. The set 8 >< (x i ;r i ;t): >: x i t i r» 0; x i t i r» 0; t i ;r >= >;

11 is a convex set and the function ln(t i r x i ) ln(t i r + x i ) ln(t) ln(r i ) is a 4-self-concordant barrier for this set. roof: See Lemma 6 in [3]. We are now ready to state the main theorem Theorem 3. The function ln(r lx t i ) lx ( ln(t i r x i ) ln(t i r + x i ) ln(t) ln(r i )) is a (+4l)-self-concordant barrier for the set given by (7). roof: It follows directly from roosition 3. and the barrier calculus outlined in [6,. 9]. The best barrier resented in Xue and Ye [] for the -order cone where is arbirary large has the arameter 00l. Xue and Ye also resents another barrier, but it deends on. Hence, the new barrier function is better. Although indeendent of the barrier then a short-ste interior-oint algorithm will solve the roblem in O( n ln(=f)) Newton stes. 4 Conclusion In this aer we have shown that when a convex quadratically constrained otimization roblem is cast as a second order cone otimization roblem using the method outlined in Section.4 then the resulting rimal and dual second order cone otimization roblem has zero duality ga. Finally,we discuss the -order cone otimization roblem and suggest a new self-concordant barrier function for the roblem which has a better arameter than the one suggested in []. References [] I. Adler and F. Alizadeh. rimal-dual interior oint algorithms for convex quadratically constrained and semidefinite otimization roblems. Technical Reort RRR--95, RUTCOR, Rutgers Center for Oerations Research,.O. Box 506, New Brunswick, New Jersey, 995. [] A. Ben-Tal and A Nemirovski. Convex otimization in engineering: Modeling, analysis, algorithms [3] D. den Hertog, F. Jarre, C. Roos, and T. Terlaky. A sufficient conditions for selfconcordance, with alication to some classes of structured convex rogramming roblems. Math. rogramming, 69():75 88,

12 [4] M.S. Lobo, L. Vandenberghe, S. Boyd, and H. Lebret. Second-order cone rogramming. Technical reort, ISL, Electrical Engineering Deartment, Stanford University, Stanford, CA, 997. Submitted to Linear Algebra and Alications. [5] Y. Nesterov and A. Nemirovskii. Interior-oint olynomial Algorithms in Convex rogramming. SIAM, hiladelhia, A, edition, 994. [6] Yu. Nesterov. Interior-oint methods: An old and new aroach to nonlinear rogramming. Math. rogramming, 79:85 97, 998. [7] Yu. Nesterov and J.-h. Vial. Homogeneous analytic center cutting lane methods for convex roblems and variational inequalities. Technical Reort 997.4, Logilab, HEC Geneva, Section of Management Studies, University of Geneva, jul 997. [8] E. L. eterson and J. G. Ecker. Geometric rogramming: Duality in quadratic rogramming and ` aroximation ii. J. on Al. Math., 3:37 340, 967. [9] E. L. eterson and J. G. Ecker. Geometric rogramming: Duality in quadratic rogramming and ` aroximation iii. J. Math. Anal. Al., 9: , 970. [0] T. Terlaky. On ` rogramming. Euroean J. Oer. Res., :70 00, 985. [] G. Xue and Y. Ye. An Efficient Algorithm for Minimizing a Sum of -Norms. Technical reort, Deartment of Comuter Science and Electrical Engineering, The University of Vermont, Setember 997. [] G. Xue and Y. Ye. An efficient algorithm for minimizing a sum of euclidean norms with allications. SIAM J. on Otim., 7(4):07 036, 997.