Convergent LMI relaxations for nonconvex quadratic programs

Transcription

1 Convergent LMI relaations for nonconve quadratic progras Jean B. Lasserre LAAS-CNRS, Avenue du Colonel Roche 10 Toulouse Céde, France Abstract We consider the general nonconve quadratic prograg proble provide a series of conve positive seidefinite progras (or LMI relaations) whose sequence of optial values is onotone converges to the optial value of the original proble. It iproves includes as a special case the well-known Shor s LMI forulation. Often, the optial value is obtained at soe particular earl relaation as shown on soe nontrivial test probles fro Floudas Pardalos [9]. 1 Introduction In this paper we consider the general nonconve quadratic prograg proble P! p Λ := R fg 0()jg i () 0; i = 1;;:::;g: (1) n where the g i () are all quadratic polnoials, i = 0;:::, where neither the criterion nor the constraint set are assued to be conve. We also allow equalit constraints since the can be written with two opposite inequalities. In fact, an optiization proble with polnoials can be put in this for in an augented space (see Shor [1], [1] Ferrier []). This proble is ver difficult in general, for several NP-hard probles can be put in this for. However, there is a well-known relaation of this proble, known as Shor s relaation [1], which replaces P b a conve positive seidefinite (psd) progra (or LMI relaation). This relaation has been proved ver useful for coputing estiates of global ia, even in cobinatorial optiization. For instance, for the well-known MAX-CUT proble (a special case of (1)), ver good approiations of a global iu have been obtained in Goeans Williason [] using Shor s relaation, followed b a roized rounding procedure. In this paper, we propose a fail fq i g of conve LMI relaations (or psd progras) that contains Shor s relaation as its first eber with an associated increasing sequence of lower bounds finfq i g. When the constraint set is copact, the nondecreasing sequence of these lower bounds converges to the global optial value p Λ in (1). In fact, in an cases, the global optial value is reached eactl in a few steps. The approach is based on the theor of oents recent results on the representation of polnoials that are strictl positive on a copact sei algebraic set. For results on the theor of oents the representation of positive polnoials, the reader is referred to Curto Fialkow [], Berg [1], Schüdgen [11], Putinar [10], Jacobi [], Jacobi Prestel []. It turns out that this theor is a natural appropriate tool for global optiization since g 0 () p Λ is precisel a (non strictl) positive polnoial, a feature that distinguishes p Λ fro the other (local) ia. Moreover, the LMI relaations are well-suited since both prial dual psd progras perfectl atch both sides of the sae theor (oents positive polnoials). Indeed, when the optial value p Λ is obtained at soe relaation, sa Q i,the prial psd progra provides a global iizer whereas the dual psd progra Qi Λ provides the coefficients of polnoials in the decoposition of g 0 () p Λ into a weighted su of squares. Of course, there is a price to pa for these refined relaations. The nuber of variables in the LMI relaation Q i is the diension of the vector space of real-valued polnoials in n variables of degree i,thatis,o(n i ). However, the Q relaation is alread interesting. On a saple of 0 rol generated MAX-CUT probles in R 10, the Q relaation alwas provided the optial solution, in contrast to onl a few cases for the relaation. The results obtained on a saple of nontrivial test probles taken fro Floudas Pardalos [9] are also proising. Notation, Definitions Preliaries For all i = 0;1;:::;, leta i be a real-valued setric (n;n)-atri, c i R n, consider the general nonconve quadratic prograg proble (1) with g i () = 0 A i +.Let K be the c 0 i + b i, i = 1;::: g 0 () = 0 A 0 + c 0 0 feasible set of P,thatis,K = fg i () 0; i = 1;:::g. Given an two real-valued setric atrices A;B let ha;bi denote the usual scalar product trace(ab) let A ν B (resp. A χ B)stforA B psd (resp. A B positive definite). Let 1; 1 ; ;::: n ; 1; 1 :::; r 1;:::; r n; ()

2 beabasisforther-degree real-valued polnoials let s(r) be its diension. Therefore, a r-degree polnoial p() : R n! R is written where p() =»r p ; R n ; = 1 1 ::: n n ; with n i=1 i = k; is a onoial of degree k with coefficient p. Denote b p = fp gr s(r) the coefficients of the polnoial p() in the basis (). Hence, the respective vectors of coefficients of the polnoials g i (), i = 0;1;:::, are denoted f(g i ) g =: g i R s(), i = 0;1;:::. Given a s(r)-sequence (1; 1 ;:::;), letm r () be the oent atri of diension s(r), with rows coluns labelled b (). For instance, for illustration purposes, for clarit of eposition, consider the -diensional case. The oent atri M r () is the block atri fm i; j()g 0»i; j»r defined b M i; j() = i+ j;0 i+ j 1;1 ::: i; j i+ j 1;1 i+ j ; ::: i 1; j+1 ::: ::: ::: ::: j;i i+ j 1;1 ::: 0;i+ j To fi ideas, when n = r = one obtains M () = : () 1 j 10 0;1 j ; 10 j 0 11 j j 11 0 j j 0 1 j j 1 1 j ;1 1 0 j 1 0 j 1 0 For the -diensional case, M r () is defined via blocks fm i; j;k()g,0» i; j;l» r in a siilar fashion, so on. If the entr (i; j) of the atri M () is β,letβ(ij) denotes the subscript β of β. Net, for a polnoial θ() : R n! R, let M r (θ) be the atri defined b For instance, with M 1 () = M 1 (θ) reads M r (θ)(i; j) = θ fβ(i; j)+g: () ! θ() =a 1 ; a 0 0 ; a ; a a ; a 0 0 ; a a ; a ; a 0 0 : M r () define a bilinear for h:;:i on the space A r of r- degree real-valued polnoials b hq();v()i := hq;m r ()vi; q();v() A r ; if is a sequence of oents of soe easure µ,then hq;m r ()qi = so that M r () ν 0, siilarl, hq;m r (θ)qi = Z Z q() µ (d) 0; () θ()q() µ (d); () so that M r (θ) ν 0 whenever µ has its support contained in fθ() 0g. The theor of oents identifies those sequences with M r () ν 0, which are oent-sequences. For details recent results, the interested reader is referred to Berg [1], Curto Fialkow [], Sion [1], Schüdgen [11] the an references therein. A fail of conve LMI relaations Consider the proble P with feasible set K = fg i () 0; i = 1;:::g criterion g 0 (). When needed below, the vectors g i R s() are etended to vectors of R s(i) b copleting with zeros, i.e., the quadratic polnoials polnoials g i (), i = 0;:::, are considered as i-degree polnoials with null coefficients for ters of degree larger than. As we iize g 0, we assue that its constant ter is zero, that is, g 0 (0) =0. For i = 1;;:::, consider the following fail of conve LMI probles Q i > < with respective dual probles Q Λ i (g 0 ) M i () ν 0 M i 1 (g k ) ν 0; k = 1;:::: a X X ;Z k ν0 (1;1) g k (0)Z k (1;1) hx;b i + where we have written hz k ;C k i =(g 0) ; = 0; M i () = B ; M i 1 (g k )= C k ; k = 1;:::; (with 0 = 1) for appropriate real-valued setric atrices B ;C k, k = 1;:::. At this stage, it is worth to write down the LMI progra,thatis,wheni = 1. > < (g 0 ) M 1 () ν 0 (g k ) 0 k = 1;:::; () () (9)

3 with dual Q Λ 1 a X (1;1) λ 0;Xν0 λ k g k (0) hx;b i + λ k (g k ) =(g 0 ) ; = 0: Observe that, if we write M 1 () =» 1 0 Y ; (10) reebering that g k () = 0 A k + c 0 k + b k, k = 0;1;:::, then has the equivalent for ha 0;Y i + c 0 0 Y;» 1 0 ha k ;Y i + c 0 k b k; k = 1;::: Y ν 0 (11) which is the dual of the well-known Shor s relaation for quadratic progras (see e.g. [1]). Therefore, Shor s relaation is the first LMI progra in the hierarch of relaations fq i g. We need the following interediate result: Proposition.1 The fail of LMI probles fq i g satisfies inf Q i» infq i+1» inf P; i = 1;;:::: (1) Proof: That infq i» infp is iediate because to ever adissible point of P, corresponds a solution :=( 1 ; ;::: i 1 ;::: i n ); adissible for Q i, thus infq i» inf P for ever i = 1;:::. Net, consider an LMI proble Q i, with i > 1, let = f g a feasible sequence for Q i,thatis,(1;) is a vector of diension s(i), the diension of the vector space of realvalued polnoials p() : R n! R, ofdegreei. If we write =( 1 ; ) with (1; 1 ) of diension s((i 1)),wehave» Mi 1 ( 1 ) M M i () = M 0» Mi (g M i 1 (g k 1 ) V k )= V 0 NS N ; for appropriate atrices M;N;V;S. Therefore, M i () ν 0 ) M i 1 ( 1 ) ν 0; ; k = 1;::: M i 1 (g k ) ν 0 ) M i (g k 1 ) ν 0; k = 1;:::; so that 1 is adissible for Q i 1. Moreover, the value of in Q i is the sae as the value of 1 in Q i 1, the result follows. Proposition.1 ensures that better better lower bounds on P can be obtained b solving the relaations Q i, i = 1;:::. The net result shows that in fact, whenever K is copact, one a approach as closel as desired, the optial value p Λ = inf P. We will use the fact that under a certain condition on the feasible set K, everpolnoial p(), strictl positive on K, has the following representation: p() = q j () + g k ()" q kj () (1) for a finite fail of polnoials fq j ()g, j = 1;:::, fq kj ()g, j = 1;:::r k, k = 1;:::. In fact, a necessar sufficient condition for the representation (1) to eist is that there eists a polnoial u() of the for (1) such that fu() 0g is copact (see Putinar [10] Jacobi []). For instance, the representation (1) holds whenever fg k () 0g is copact for soe k, or when all the g k () are linear K is copact. In particular, it holds for ever 0-1 progra. Indeed, write the integral constraints as i i 0 i i 0, for all i = 1;:::n. Then, consider the polnoial u() := n i=1 ( i i ). Its level set fu() 0g is copact. Moreover, if one knows that a global iizer is contained in soe ball kk» M, forsoem large enough, then one a add the redundant constraint g +1 () := M n i=1 i 0, the set K (defined as previousl with the latter additional constraint g +1 () 0) has the required propert. And, we have: Theore. Assue that there is soe polnoial u() : R n! R of the for (1) with fu() 0g copact. Then, as i!, inf Q i " P: (1) If K has a nonept interior, then as i!, a Q Λ i = inf Q i " P: (1) The equalit a Q Λ i = P occurs for all i i 0 (for soe inde i 0 ) if onl if g 0 () p Λ is of the for (1). Proof: Let ε > 0 be fied arbitrar. Then, the polnoial! G 0 () := g 0 () p Λ + ε is strictl positive on K. Fro the assuption on K, it follows that G 0 () has the representation G 0 () = q j () + g k ()" q kj () (1) for soe polnoials q j (), j = 1;:::,q kj (), j = 1;:::r k, k = 1;::: (see e.g. Putinar [10], Jacobi []). Now, let i 1 (ε) (that for notational convenience we sipl write i 1 ) be the aiu degree of q j (), j = 1;:::,

4 i (ε) (noted i ) be the aiu degree of the polnoials fq kj ()g, so that the polnoials fg k ()q kj () g have aiu degree i + (astheg k () s are quadratic polnoials). Let i := a[i 1 ;i + 1]. Letq j R s(i) q kj R s(i 1) be the respective vectors of coefficients of the polnoials fq j ()g fq kj ()g, write X := Observe that with then, q j q 0 j; Z k := r k q kj q 0 kj; k = 1;:::: (1) =( 1 ;:::; i 1 ::: i n ); g k ()q kj () = hq kj ;M i 1 (g k )q kj i = hq kj q 0 kj;m i 1 (g k )i; so that siilarl, r k g k () with B ;C k as in Q Λ i. q kj () = hz k ;M i 1 (g k )i (1) = hz k ;C k i; j () q = hx;b i; (19) Therefore, fro the representation (1), it follows that "hx;b i + hz k ;C k i (g 0 ) p Λ +ε: (0) = Identifing ters of sae power ields hx;b i +, for the constant ter, X (1;1)+ hz k ;C k i =(g 0) = 0; Z k (1;1)g k (0) = p Λ + ε; which proves that (X;Z 1 :::Z ) is adissible for Qi Λ with value p Λ ε. Therefore, p Λ ε» inf Q i» p Λ, fro the onotonicit the fact that ε was arbitrar, infq i " p Λ = P. The stateent (1) follows fro a stard result in conveit. Indeed, we have just proved that Qi Λ has a feasible solution (X;Z 1 ;:::Z ). Moreover, if K has a nonept interior, let µ be a probabilit easure with support K with a unifor distribution. Denote b µ the vector of all the (well-defined) oents of µ. Fro ()-() the fact that µ has a unifor distribution on K, it follows easil that M i ( µ ) χ 0M i 1 (g k µ ) χ 0forallk = 1;:::, sothat µ is a strictl adissible solution. Therefore, there is no dualit gap between Q Λ i Q i, (1) follows. Finall, consider the if part of the final stateent. Fro the representation (1) of g 0 () p Λ using the sae above arguents, the corresponding atrices X;Z 1 ;:::Z for an adissible solution for Q Λ i proving aq Λ i = P. with value p Λ = P, hence Conversel, the onl if part of the final stateent also follows easil fro the above proof. Let (X;Z 1 ;:::Z ) be an optial solution of Qi Λ with aq Λ i = p Λ = P. As X;Z 1 ;:::Z ν 0, write the as in (1). The adissiblit of (X;Z 1 ;:::Z ) iplies (0) with p Λ in lieu of p Λ ε, which in turn, using (1)-(19) ields the desired result. The representation (1) has a nice interpretation as a global optialit condition à la Karush-Kuhn-Tucker. Indeed, Proposition. Assue that g 0 () p Λ has the representation (1). Then, at a global iizer Λ, we have g 0 ( Λ )= g k ( )" Λ q kj ( Λ ) (1) g k ( )" Λ q kj ( Λ ) = 0; k = 1;:::: () Proof: The proof is iediate. As g 0 ( Λ ) p Λ = 0, the representation (1) leads q j ( Λ ) = 0; g k ( )" Λ q kj ( Λ ) = 0; so that () follows. Differentiating in (1) using the above relationship also ields (1). Hence, the theor of representation of polnoials, positive on the feasible set K, can be viewed as a global optialit condition. The polnoials (sus of squares) j q kj () in the representation (1) of g 0 () p Λ (when it holds) are nothing less than Lagrange Karush-Kuhn-Tucker ultipliers. In contrast to the usual Karush-Kuhn-Tucker necessar optialit condition with scalar ultipliers λ k, a nonactive constraint g k () at a global iizer Λ a have an nontrivial associated polnoial ultiplier (when this constraint plas a role to eliate soe better (non feasible) solutions. However, this polnoial ultiplier vanishes at Λ, as in the usual Karush-Kuhn-Tucker conditions where λ k = 0 at a non-active constraint g k ( Λ ) > 0. Eaples For illustration purposes we will consider the MAX-CUT proble soe nontrivial nonconve quadratic test probles fro Floudas Pardalos [9].

5 .1 The a-cut proble Roughl speaking, the MAX-CUT proble is a special case of (1) with ffl g 0 () = 0 A 0 diag(a 0 )=0. ffl g i () = i 1 the constraint is an equalit constraint. As fo-1 progras, the feasible set K satisfies the condition required in Theore.. Indeed, write the integral constraint i = 1as i 1 01 i 0foralli = 1;:::n, consider the polnoial u() := n i=1 (1 i ). Its level set fu() 0g is copact. Shor s relaation (equivalentl, ), is the conve LMI proble: hy;a 0 i 1 j 0 j Y ν 0; diag(y )=e () where e is a vector of ones. To visualize the two relaations Q, take an eaple in R. In that case, reads q q q whereas Q reads with Q B = C = ν 0; q q q M 1() j B B 0 j C ν 0; : where we have used the fact that the equalit 100 = 1translates into 00 = 1 therefore, a ter like 00 is replaced b 1, a ter like 10 is replaced b 010, 11 b 011,etc. so that onl the variables with i» 1 i i» are present. Hence, the Q relaation of a MAX-CUT proble of diension n is an LMI proble with n+1 + n+1 variables one LMI constraint of diension (n + 1)(n + )= in n+1 coparison with variables one LMI constraint of diension n + 1forthe relaation. Of course, as i increases, then so does the coputational burden, a price to pa to obtain better better lower bounds. For a saple of 0 rol generated probles in R 10 with positive negative weights, the Q relaation provided the optial solution in all cases whereas the relaation in less than 0% of the cases onl. In both relaations Q, Slater s interior point condition fails, we have to ention that in using the MAT- LAB LMI toolbo, the running tie for Q was surprisingl ver high copared to, despite the fact that Q contains relativel few variables (1 for R 0 for R, fro R 10 ).. Nonconve quadratic probles The nonconve quadratic test probles below are fro Floudas Pardalos [9]...1 : Proble. in Floudas Pardalos [9]. ; f (;) := c T 0: T Q + d T » : » 0 0» i» 1; i = 1;:::; 0» with Q := I c =[ 10:; :; :; :; 1:]. The optial value 1 is obtained at the Q relaation... : Proble. in Floudas Pardalos [9]. with A being the atri f () := c T 0: T Q A» b 0» i» 1; i = 1;:::;10» c =[;;;;;1;;;;], Q = 100I b = [ ;; ; ; 1]. The optial value 9 is obtained at the Q relaation... : Proble.9 in Floudas Pardalos [9]. a 9 f () := i=1 i i+1 + i=1 i i i=1 i = 1 i 0; i = 1;:::10:

6 The optial value 0: is obtained at the Q relaation... : Proble. in Floudas Pardalos [9]. f () := ( 1 ) ( ) ( 1) ( ) ( 1) ( ) ( ) + ; ( ) + 1» ; 1 +» 1 +» ; 1 + 1»» ; 0»» 1»» ; 0»» 10 1 ; ; 0 The optial value 10 is obtained at the Q relaation... : Proble. in Floudas Pardalos [9]. f () := » 1» ;» ; +» i 0; i = 1;; T B T B r T B + k 0:kb vk 0 with r =[1:; 0:; ] B = ; b = 0 ;v = 0 1 : The optial value is obtained at the Q relaation whereas infq = :0. Conclusion We have proposed a sequence of LMI relaations fq i g an associated sequence of nondecreasing lower bounds that converges to the global iu p Λ, in an cases, p Λ is obtained at a particular relaation. It has been shown that the prial dual psd progras perfectl atch both sides of the sae theor (oents positive polnoials). Moreover, the representation of polnoials, positive on the feasible set K, is interpreted as a Karush-Kuhn- Tucker global optialit condition with polnoial Lagrange ultipliers instead of scalar ultipliers. However, although efficient LMI software packages are now available, high order relaations require a lot of variables. It is hoped that for a large class of probles, low order relaations like Q, Q or Q will provide the optial value, or at least, a good lower bound that could be eploited in other optiization ethods. [] R.E. CURTO AND L.A. FIALKOW, Recursiveness, positivit, truncated oent probles, Houston J. Math. 1 (1991), 0. [] C. FERRIER, Hilbert s 1th proble best dual bounds in quadratic iization, Cbern. Sst. Anal. (199), pp [] M. X. GOEMANS AND D.P. WILLIAMSON, Iproved approiation algoriths for aiu cut satisfiabilit probles using seidefinite prograg, J. ACM (199), [] T. JACOBI, A representation theore for certain partiall ordered coutative rings, Matheatische Zeitschrift, to appear. [] T. JACOBI AND A. PRESTEL, On special representations of strictl positive polnoials, Technical-report, Konstanz Universit, Janua00. [] J.B. LASSERRE, The global iization of a polnoial is an eas conve proble, LAAS Technical report 99, Toulouse, Deceber [] J.B. LASSERRE, Global iization with polnoials the proble of oents, LAAS Technical report 000, Toulouse, Janua00, subitted. [9] C. FLOUDAS AND P.M. PARDALOS, A Collection of Test Probles for Constrained Global Optiization Algoriths, Springer-Verlag, Berlin, [10] M. PUTINAR, Positive polnoials on copact seialgebraic sets, Ind. Univ. Math. J. (199), pp [11] K. SCHMÜDGEN, The K-oent proble for copact sei-algebraic sets, Math. Ann. 9 (1991), pp [1] B. SIMON, The classical oent proble as a selfadjoint finite difference operator, Adv. Math. 1 (199), 0. [1] N.Z. SHOR, Quadratic optiization probles, Tekhnicheskaa Kibernetika 1 (19), pp [1] N.Z. SHOR, Nondifferentiable Optiization Polnoial Probles, Kluwer Acadeic Publishers, Dordrecht, 199. [1] L. VANDENBERGHE AND S. BOYD, Seidefinite prograg,siamreview (199), pp References [1] C. BERG, The ultidiensional oent proble sei-groups, in: Moent in Matheatics (190), pp , ed. H.J. Lau, Aer. Math. Soc.