MATHEMATICAL ENGINEERING TECHNICAL REPORTS Successve Lagrangan Relaxaton Algorthm for Nonconvex Quadratc Optmzaton Shnj YAMADA and Akko TAKEDA METR 2017 08 March 2017 DEPARTMENT OF MATHEMATICAL INFORMATICS GRADUATE SCHOOL OF INFORMATION SCIENCE AND TECHNOLOGY THE UNIVERSITY OF TOKYO BUNKYO-KU, TOKYO 113-8656, JAPAN WWW page: http://www.kesu.t.u-tokyo.ac.jp/research/techrep/ndex.html
The METR techncal reports are publshed as a means to ensure tmely dssemnaton of scholarly and techncal work on a non-commercal bass. Copyrght and all rghts theren are mantaned by the authors or by other copyrght holders, notwthstandng that they have offered ther works here electroncally. It s understood that all persons copyng ths nformaton wll adhere to the terms and constrants nvoked by each author s copyrght. These works may not be reposted wthout the explct permsson of the copyrght holder.
Successve Lagrangan Relaxaton Algorthm for Nonconvex Quadratc Optmzaton Shnj Yamada Akko Takeda March 31th, 2017 Abstract Optmzaton problems whose objectve functon and constrants are quadratc polynomals are called quadratcally constraned quadratc programs (QCQPs). QCQPs are NP-hard n general and are mportant n optmzaton theory and practce. There have been many studes on solvng QCQPs approxmately. Among them, sem-defnte program (SDP) relaxaton s a well-known convex relaxaton method. In recent years, many researchers have tred to fnd better relaxed solutons by addng lnear constrants as vald nequaltes. On the other hand, SDP relaxaton requres a long computaton tme, and t has hgh space complexty for large-scale problems n practce; therefore, SDP relaxaton may not be useful for such problems. In ths paper, we propose a new convex relaxaton method that s weaker but faster than SDP relaxaton methods. The proposed method transforms a QCQP nto a Lagrangan dual optmzaton problem and successvely solves subproblems whle updatng the Lagrange multplers. The subproblem n our method s a QCQP wth only one constrant for whch we propose an effcent algorthm. Numercal experments confrm that our method can quckly fnd a relaxed soluton wth an approprate termnaton condton. 1 Introducton We consder the followng quadratcally constraned quadratc program (QCQP): mnmze x R n x Q 0 x + 2q 0 x + γ 0 subject to x Q x + 2q x + γ 0, = 1,, m, (1) S. Yamada s wth the Department of Mathematcal Informatcs, Graduate School of Informaton Scence and Technology, The Unversty of Tokyo, Tokyo, Japan (emal: shnj yamada@mst..u-tokyo.ac.jp) A. Takeda s wth the Department of Mathematcal Analyss and Statstcal Inference, The Insttute of Statstcal Mathematcs, Tokyo, Japan (emal: atakeda@sm.ac.jp) 1
where each Q s an n n symmetrc matrx. Q = O means a lnear functon. We call a QCQP wth m constrants m-qcqp. In the case Q O for every = 0,, m, (1) s a convex program. However, n general, postve semdefnteness s not assumed, and (1) s NP-hard [23]. QCQPs are mportant n optmzaton theory and n practce. QCQPs are fundamental nonlnear programmng problems that appear n many applcatons such as max-cut problems [24] and bnary quadratc optmzatons. Some relaxaton methods exst for fndng a global soluton of (1). A standard approach s the branch-and-bound (or cut) method, where a smple relaxaton, e.g., a lnear programmng (LP) relaxaton problem [2], s solved n each teraton. Audet, Hansen, Jaumard and Savard [5] proposed to ntroduce addtonal constrants, constructed usng the reformulaton lnearzaton technque (RLT), to the relaxaton problem. Lagrangan bounds,.e., bounds computed by Lagrangan relaxaton, have also been used n branch-and-bound methods n order to reduce the dualty gap [22, 27, 29]. The branch-and-bound (or cut) algorthm yelds a global soluton by solvng many relaxaton subproblems, whch restrcts the sze of QCQPs. Another avenue of research has nvestgated tght relaxaton problems for QCQPs. Among the many convex relaxaton methods, semdefnte program (SDP) relaxaton s well known and have been extensvely studed [11, 19, 28]. It s known that an SDP relaxaton can be vewed as a Lagrangan dual problem of the orgnal QCQP. SDP relaxaton has been appled to varous QCQPs that appear n combnatoral optmzaton problems [12, 13] as well as n sgnal processng and communcatons [19]. SDP relaxaton s powerful, and t gves the exact optmal value partcularly when there s only one constrant on the trust regon subproblem (TRS). Furthermore, recent studes such as [3] have proposed to add new vald nequaltes (e.g., lnear constrants for matrx varables usng the orgnal upper and lower bound constrants) to SDP relaxaton problems. In partcular, Zheng, Sun, and L [30, 31] proposed a decomposton-approxmaton scheme that generates an SDP relaxaton at least as tght as the ordnary one. Jang and L [16] proposed second order cone constrants as vald nequaltes for the ordnary SDP relaxaton. Such methods am at obtanng a better relaxed soluton even f they take more tme to solve than the orgnal SDP relaxaton. However, SDP relaxaton ncludng addtonal vald nequaltes ncreases the problem sze, whch leads to a longer computaton tme and often memory shortage errors for large-scale problems. Here, Km and Kojma [17] proposed a second-order cone programmng relaxaton (SOCP relaxaton), where vald second-order cone constrants derved from the postve semdefnte nequalty are added to the LP relaxaton. Burer, Km, and Kojma [9] proposed a weaker but faster method than SDP relaxaton that uses a block matrx decomposton. Such faster relaxaton methods are useful for large-scale problems, and they can be repeatedly solved n, e.g., a branchand-bound method. 2
In ths paper, we propose a faster convex relaxaton method that s not stronger than SDP relaxaton f vald constrants are not consdered. Our method solves the Lagrangan dual problem of the orgnal QCQP by usng a subgradent method, though the dual problem can be reformulated as an SDP and s solvable wth the nteror pont method. Indeed, there are varous studes that propose to solve the Lagrangan dual problems for nonconvex problems, but most of them transform the dual problem nto an SDP problem [6] or a more general cone problem [18]. Here, to resort to more easly solved problems, we dvde the mnmzaton of the objectve functon n the Lagrangan dual problem nto two stages and teratvely solve the nner problem as a 1-QCQP, whch can be solved exactly and quckly. There are manly two approaches to solvng a 1-QCQP: one s based on egenvalue computaton, the other on SDP relaxaton. In partcular, Moré and Sorensen [20] proposed to teratvely solve a symmetrc postve-defnte lnear system for TRS, whle Adach, Iwata, Nakatsukasa, and Takeda [1] proposed an accurate and effcent method that solves only one generalzed egenvalue problem. In ths paper, we propose a new relaxaton method that can solve a 1-QCQP exactly and quckly as a convex quadratc optmzaton problem. Furthermore, we prove that the convex quadratc problem constructs the convex hull of the feasble regon of a 1-QCQP. Numercal experments confrm that our convex quadratc relaxaton method for solvng 1-QCQPs s faster than SDP relaxaton and egenvalue methods. They also show that our method can quckly fnd a relaxed soluton of an m- QCQP by teratvely solvng a 1-QCQP wth updated Lagrange multplers. By addng vald constrants to our formulaton, our method can sometmes fnd a better relaxed soluton n a shorter computaton tme compared wth the ordnary SDP relaxaton. The relaxaton technque can be embedded wthn a branch-and-bound framework to determne a global optmum to the orgnal m-qcqp. The remander of ths paper s organzed as follows. We ntroduce SDP relaxaton and other related studes n Secton 2. We descrbe our method and ts some propertes n Secton 3 and 4. We present computatonal results n Secton 5. We conclude the paper n Secton 6. The Appendx contans our proofs of the presented theorems. Throughout the paper, we denote matrces by usng uppercase letters such as Q, vectors by usng bold lowercase such as q and scalars by usng normal lower case such as γ. The notaton A B or A B mples that the matrx A B s postve defnte or semdefnte. e means the all-one vector. 3
2 Exstng SDP relaxaton methods for m-qcqp 2.1 SDP relaxaton An SDP relaxaton can be expressed as a Lagrangan dual problem of the orgnal problem (1) as follows: max ξ 0 ϕ(ξ). (2) Here, ϕ(ξ) s an optmal value functon defned by ) ( ) m m ϕ(ξ) := mn x (Q x 0 + ξ Q x + 2 q 0 + ξ q x + γ 0 + = { q(ξ) Q(ξ) q(ξ) + γ(ξ),, (f Q(ξ) O), (otherwse), m ξ γ, where Q(ξ) := Q 0 + m ξ Q, q(ξ) := q 0 + m ξ q, γ(ξ) := γ 0 + m ξ γ and means the pseudo-nverse. Note that from (4), (2) s equvalent to max ξ 0 ϕ(ξ) s.t. Q 0 + (3) (4) m ξ Q O. (5) ( ) Q(ξ) q(ξ) By consderng q(ξ) Q(ξ) q(ξ)+γ(ξ) as a Schur complement of q(ξ), γ(ξ) we can equvalently rewrte the dual problem (5) as a semdefnte program (SDP) max ξ 0,τ s.t. τ ( ) Q(ξ) q(ξ) q(ξ) O, (6) γ(ξ) τ whch can be solved by usng an nteror pont method. It should be noted that the dual of (6) s mn x,x Q 0 X + 2q 0 x + γ 0 s.t. Q X + 2q x + γ 0, = 1,, m, (7) X xx and (6) and (7) are equvalent under the Prmal/Dual Slater condton. 4
SDP relaxaton s a popular approach to dealng wth (1). Sturm and Zhang [26] proved that when there s one constrant (.e. a 1-QCQP), SDP relaxaton can always obtan the exact optmal value. Goemans and Wllamson showed an approxmaton bound of SDP relaxaton for max-cut problems [13], and Goemans [12] appled SDP relaxaton to varous combnatoral problems. Ther numercal experments show that SDP relaxaton can fnd a very tght relaxed soluton for many knds of problems. However, SDP relaxaton has dsadvantages n both computaton tme and space complexty because of the matrx varable; t cannot deal wth large-scale problems because of shortage of memory. Although polynomal tme algorthms, such as an nteror pont method, have been establshed, they often take a long tme to solve an SDP relaxaton problem n practce. 2.2 Stronger SDP relaxaton usng RLT For further strengthenng the SDP relaxaton, Anstrecher [3] proposed the reformulaton lnearzaton technque (RLT). Moreover, [3] added new constrants and restrcted the range of the new varables X j,, j. Here, one assumes the orgnal problem (1) has box constrants,.e., lower and upper bounds on each varable x j (l j and u j, respectvely). Note as well that even f there are no box constrants, we may be able to compute l j and u j by usng the orgnal constrants f the feasble regon s bounded. The nequalty l j x j u j (as a vector expresson, l x u) leads to (x u )(x j u j ) 0 x x j u x j u j x + u u j 0, (8) (x u )(x j l j ) 0 x x j u x j l j x + u l j 0, (9) (x l )(x j l j ) 0 x x j l x j l j x + l l j 0, (10) for, j = 1,, n. By replacng x x j wth X j, we get X j u x j u j x + u u j 0, (11) X j u x j l j x + u l j 0, (12) X j l x j l j x + l l j 0. (13) (11) (13) are lnear nequaltes that nclude matrx varables X j. Therefore, by addng these constrants, we can get a stronger relaxaton. The dsadvantage of RLT s that t ncreases computaton tme because of the ncreased varables X j and addtonal constrants (11) (13). Many studes have amed at strengthenng the relaxaton by addng vald nequaltes other than (11) (13) [16, 25, 30, 31]. Ther methods gve very tght bounds, but they ental large amounts of computaton tme. 2.3 Weaker SDP relaxaton method by block decomposton Burer, Km, and Kojma [9] ams to solve a relaxed problem faster than SDP relaxaton can, although t s a weaker relaxaton; as such, t shares 5
a smlar motvaton as ours. Frst, [9] assumes that the orgnal problem has [0,1] box constrants (.e. ; 0 x 1) n order to avod a stuaton n whch the optmal value dverges. Then, [9] proves that we can compute a block dagonal matrx D whch satsfes Q + D O for the matrx Q appearng n the objectve functon or the constrants. By usng D, we can transform a quadratc polynomal, x Q x + 2q x + γ = x D x + x (Q + D )x + 2q x + γ and relax x D x to D X and X O as n SDP relaxaton. As a whole, a relaxaton problem s as follows. mn x,x D 0 X + x (Q 0 + D 0 )x + 2q 0 x + γ 0 s.t. D X + x (Q + D )x + 2q x + γ 0, = 1,, m, X k x k x k, k = 1,, r, where r denotes the number of blocks of D and X k or x k denotes a partal matrx or vector n X or x correspondng to each block of D. Note that n a smlar way as (12), we get new constrants X x, = 1,, m for the matrx varable X from the box constrants. Snce we relax only the quadratc form for each block part, the matrx X only has block part components. Therefore, we can consder the postve semdefnte constrant only for the block parts: X k x k x k. The number of varables related to the postve semdefnte constrant s reduced, and that s why we can obtan the optmal value so quckly. We call ths method Block-SDP and use t n the numercal experments n Secton 5. In [9], t s proposed to dvde D as evenly as possble, that s, by makng the dfference between the largest block sze and the smallest block sze at most one for a gven r. 3 Proposed Method 3.1 Assumptons Before we explan our method, we wll mpose the followng three assumptons. Assumpton 1. (a) The feasble regon of (1) has some nteror ponts. (b) There exsts at least one matrx Q ( = 0,, m) such that Q O. (c) When Q 0 O, any optmal soluton x := Q 0 q 0 of the followng unconstraned optmzaton problem mn x s not feasble for the orgnal QCQP (1). x Q 0 x + 2q 0 x + γ 0. (14) 6
Assumpton 1 (a) s the prmal Slater condton, and (b) s a suffcent condton of the Dual Slater condton of the orgnal QCQP. Assumpton 1 (c) s not a strong one because f x s feasble for (1), t s a global optmal soluton and we can check t easly. 3.2 The Whole Algorthm We further transform the Lagrangan dual problem (2) nto max λ Λ s max µ 0 ϕ(µλ), (15) where Λ s := {λ 0 e λ = 1} s a smplex. Now we defne ψ(λ) as the optmal value of the nner optmzaton problem of (15) for a gven λ Λ s : ψ(λ) := max µ 0 ϕ(µλ). (16) Note that (16) s the Lagrangan dual problem for the followng 1-QCQP: ψ(λ) = mn x s.t. x Q 0 x + 2q 0 x + γ 0 ( m ) ( m ) m x λ Q x + 2 λ q x + λ γ 0. (17) There s no dualty gap between (16) and ts Lagrangan dual (17), snce [26] proves that the SDP formulaton of (16) has the same optmal value as the 1-QCQP (17). We wll show how to solve the 1-QCQP (17) exactly and quckly n Secton 4.1. The SDP relaxaton problem (2) can be wrtten as max ψ(λ). (18) λ Λ s Here, we propose an algorthm whch teratvely solves (17) wth updated λ Λ s for fndng an optmal soluton of the SDP relaxaton problem. Λ s s a convex set, and ψ(λ) s a quas-concave functon, as shown n Secton 3.3. Therefore, we wll apply the standard gradent descent method to (18) for updatng λ. The speed of convergence of gradent methods s slow n general especally near optmal solutons, and therefore, we wll obtan a relaxed soluton by usng an approprate termnaton crteron. Algorthm 1 summarzes the proposed method. We dvde the max for the Lagrange functon nto two parts and teratvely solve the 1-QCQPs. Note that our method solves the 1-QCQPs ψ(λ) successvely, so t s not stronger than SDP relaxaton. We explan our method (especally, the relatonshp between (P k ) and (27) or (28)) n Secton 4. 7
Algorthm 1 Successve Lagrangan Relaxaton (SLR) Gven Q 0,, Q m ( ; Q O), q 0,, q m, γ 0,, γ m tolerance ϵ and suffcently small value ψ(λ 1 ), Step 1: Set k = 0, and defne an ntal pont λ (0). Step 2: Fnd an optmal soluton x (k) and the optmal value ψ(λ (k) ) of (P k ): ψ(λ (k) ) = mn x s.t. x Q 0 x + 2q 0 x + γ 0 ( m ) ( m x λ (k) Q x + 2 λ (k) q ) x + m λ (k) γ 0 (P k ) ψ(λ k 1 ) by solvng the convex problem (27) or (28). Step 3: If ψ(λk ) ψ(λ k 1 ) < ϵ, then stop the algorthm. Otherwse, update λ (k) by Algorthm 2 shown n Secton 4.2 and k k + 1. Go to Step 2. 3.3 Quas-Concavty of ψ(λ) Objectve functons of Lagrangan dual problems are concave for Lagrange multplers (e.g. [7]). The functon ϕ(µλ) for fxed λ s hence concave for µ, but ψ(λ) s not necessarly concave for λ. However, we can prove that ψ(λ) s a quas-concave functon and has some of the desrable propertes that concave functons have. Before we prove the quas-concavty of ψ(λ), we have to defne the set Λ +, Λ + = {λ Λ s ψ(λ) > ϕ(0)}, (19) n order to explan the propertes of ψ(λ). Note that ϕ(0) s the optmal value of the unconstraned problem (14) and ψ(λ) ϕ(0) holds for all λ. We can also see that Λ + s nonempty f and only f the SDP relaxaton value s larger than ϕ(0),.e., OPT SDP > ϕ(0) holds. The above statement s obvous from OPT SDP = max{ψ(λ) λ Λ s } (see (18)) and (19). In other words, OPT SDP = ϕ(0) means that for all λ Λ s, there exsts an optmal soluton of (14) whch s feasble for (17). From the defnton of Λ +, we can see that for λ Λ +, an optmal soluton of (16), µ λ, s a postve value. When λ / Λ + (.e. ψ(λ) = ϕ(0)), we can set µ λ to zero wthout changng the optmal value and soluton. By 8
usng such µ λ, we wll dentfy (19) and Λ + = {λ Λ s µ λ > 0}. (20) Now let us prove the quas-concavty of ψ(λ) and some other propertes.. Theorem 1. Let ψ(λ) be the optmal value of (16) and ( x λ, µ λ ) be ts optmal soluton. Then, the followng () (v) hold. () The vector ( g λ ) = µ λ ( x λ Q x λ + 2q x λ + γ ) (21) s a subgradent, whch s defned as a vector n the quas-subdfferental (see e.g., [14, 15]): of ψ at λ. ψ(λ) := {s s (ν λ) 0, ν; ψ(ν) > ψ(λ)} (22) () ψ(λ) s a quas-concave functon for λ Λ s. () Λ + s a convex set. (v) If ψ(λ) has statonary ponts n Λ +, all of them are global optmal solutons n Λ s. The proof s n the Appendx. Note that the set of global solutons of ψ s convex because of the quasconcavty of ψ(λ). (v) s smlar to the property that concave functons have. Therefore, a smple subgradent method such as SLR, whch searches for statonary ponts, works well. SLR s an algorthm for fndng a statonary pont n Λ +, whch Theorem 1 (v) proves to be a global optmal soluton n Λ s. Fgures 1 and 2 are mages of ψ(λ) for m = 2, where λ Λ s s expressed by one varable α [0, 1] as λ = (α, 1 α). The vertcal axs shows ψ(λ), and the horzontal one shows α for a randomly generated 2-QCQP. We can make sure that ψ(λ) s a quas-concave functon from these fgures. There are subgradent methods for maxmzng a quas-concave functon. If the objectve functon satsfes some assumptons, the convergence of the algorthms s guaranteed. However, ths may not be the case for the problem settng of (18). For example, n [15], ψ(λ) must satsfy the Hölder condton of order p > 0 wth modulus µ > 0, that s, ψ(λ) ψ µ(dst(λ, Λ )) p, λ R m, where ψ s the optmal value, Λ s the set of optmal solutons and dst(y, Y ) denotes the Eucldean dstance from a vector y to a set Y. It s hard to check whether ψ(λ) satsfes the Hölder condton. Ensurng the convergence of SLR seems dffcult, but numercal experments mply that SLR works well and often obtans the optmal value same as SDP relaxaton can. 9
-260 10 7 0-270 ψ(λ) -0.5 ψ(λ) -280-1 -1.5-290 -2-300 -310 Λ + µ λ > 0 µ λ = 0-320 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-2.5 Λ + -3 φ(0) -3.5 µ λ > 0 µ λ = 0 ( ψ(λ) = - ) 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 Λ Λ Fgure 1: ψ(λ) and Λ + when Q 0 O Fgure 2: ψ(λ) and Λ + when Q 0 O 4 Detals of the Algorthm 4.1 1-QCQP as a Subproblem SLR needs to solve 1-QCQP (P k ). Here, we descrbe a fast and exact soluton method for a general 1-QCQP. Frst, we transform the orgnal 1-QCQP by usng a new varable t nto the form: where Q λ = mn x,t t s.t. x Q 0 x + 2q 0 x + γ 0 t, (23) m x Q λ x + 2q λ x + γ λ 0, λ (k) Q, q λ = m λ (k) q, γ λ = m λ (k) γ, n the SLR algorthm. Here, we assume that (23) satsfes the followng Prmal and Dual Slater condtons: (Prmal Slater condton) x R n s.t. x Q λ x + 2q λ x + γ λ < 0 (Dual Slater condton) σ 0 s.t. Q 0 + σq λ O Note that (P k ) satsfes the Prmal Slater condton because of Assumpton 1 (a), and t also satsfes the Dual Slater condton because ether Q 0 or Q λ s postve defnte by the updatng rule of λ (k) explaned n the next subsecton. Here, we defne S : = {σ 0 Q 0 + σq λ O}, 10
whch s a convex set of one dmenson, that s, an nterval. The Dual Slater condton mples that S s not a pont, and therefore, σ < σ holds for σ := sup σ, (24) σ S σ := nf σ. (25) σ S We set σ = + when Q λ O and σ = 0 when Q 0 O. For (23), we make the followng relaxaton problem usng σ and σ: mn x,t t s.t. x (Q 0 + σq λ )x + 2(q 0 + σq λ ) x + γ 0 + σγ λ t, (26) x (Q 0 + σq λ )x + 2(q 0 + σq λ ) x + γ 0 + σγ λ t. Note that n the SLR algorthm, we keep ether Q 0 or Q λ postve semdefnte. When σ = 0, (26) s equvalent to the followng relaxed problem: mn x,t t s.t. x Q 0 x + 2q 0 x + γ 0 t, (27) x (ˆσQ 0 + Q λ )x + 2(ˆσq 0 + q λ ) x + (ˆσγ 0 + γ λ ) ˆσt, where ˆσ = 1/ σ, and ˆσ can be easly calculated. On the other hand, when σ = +, (26) s equvalent to mn x,t t s.t. x (Q 0 + σq λ )x + 2(q 0 + σq λ ) x + γ 0 + σγ λ t, (28) x Q λ x + 2q λ x + γ λ 0. (28) can be vewed as dvdng the second constrant of (26) by σ and σ. The followng theorem shows the equvalence of the proposed relaxaton problem (26) and the orgnal problem (23). Theorem 2. Under the Prmal and Dual Slater condtons, the feasble regon rel of the proposed relaxaton problem (26) s the convex hull of the feasble regon of the orgnal problem (23),.e., rel = conv( ). The proof s n the Appendx. Theorem 2 mples that (26) gves an exact optmal soluton of 1-QCQP (23) snce the objectve functon s lnear. The outlne of the proof s as follows (see Fgure 3). We choose an arbtrary pont (x, t ) n rel and show that there exsts two ponts P and Q n whch express (x, t ) as a convex combnaton of P and Q. We show n the Appendx how to obtan P and Q for an arbtrary pont n rel. Usng ths technque, we can fnd an 11
P (x t ) rel Q Fgure 3: Image of and rel optmal soluton of 1-QCQP (23). By comparson, SDP relaxaton can not always fnd a feasble soluton for the 1-QCQP (though t can obtan the optmal value). (26) s a convex quadratc problem equvalent to 1-QCQP, whch we wll call CQ1. Note that CQ1 has only two constrants, and we can solve t very quckly. CQ1 can be constructed for general 1-QCQPs, ncludng the Trust Regon Subproblem (TRS). The numercal experments n Secton 5 mply that CQ1 can be solved by a convex quadratc optmzaton solver faster than by an effcent method for solvng a TRS and hence that CQ1 can speed up SLR. Now let us explan how to calculate ˆσ or σ especally when ether Q 0 or Q λ s postve defnte. We explan how to convexfy a matrx whch has some negatve egenvalues by usng a postve defnte matrx,.e., for both cases when Q 0 O n (27) and Q λ O n (28). Frst, we calculate ˆσ n (27) when Q 0 O. Let Q 1 2 0 be the square root of the matrx Q 0 and Q 1 2 0 be ts nverse. Then, σq 0 + Q λ O Q 1 2 0 (σq 0 + Q λ )Q 1 2 0 O holds. Therefore, ˆσ can be calculated as σi + Q 1 2 0 Q λ Q 1 2 0 O ˆσ = mn{σ mn (Q 1 2 0 Q λ Q 1 2 0 ), 0}, where σ mn (X) s the mnmum egenvalue of X. Smlarly, we can calculate σ n (28) as σ = mn{σ mn (Q 1 2 λ Q 0 Q 1 2 λ ), 0}, when Q λ O. It s true that (26) gves us the exact optmal value of (23). When both Q 0 and Q λ have zero (or even negatve) egenvalues, ˆσ and σ can not be 12
computed, but such cases can be gnored because an 1-QCQP (23) wth such Q 0 and Q λ does not gve an optmal soluton of (18). Therefore, n the Algorthm 1, we keep ether Q 0 or m σ Q postve defnte. 4.2 Update Rule of λ Now let us explan the update rule of λ, whch s shown n Step 2 of Algorthm 1. The update rule s constructed n a smlar way to the gradent projecton method for solvng (18). Step 4 s needed only when Q 0 O and m λ(k+1) Q O hold. We update the Lagrange multplers correspondng to convex constrants, whose ndex set s defned as C := { 1 m, Q O}. When Q 0 O, Assumpton 1 (b) assures that C s non-empty. Algorthm 2 Update rule of λ Gven a suffcently small postve scalar δ, Step 1: Calculate the gradent vector g (k) 2q x(k) + γ. as g (k) = x (k) Q x (k) + Step 2: Normalze g (k) as g (k) λ (k+1) as g(k) e λ and set the step sze h. Update λ (k+1) = proj Λs (λ (k) + hg (k) ), (29) where proj Λs (a) := arg mn b Λ s a b 2 s the projecton onto Λ s. Step 3: If Q 0 O or m λ(k+1) Q O, termnate and return λ (k+1). Step 4: Otherwse, fnd a mnmum postve scalar α such that α C Q + m λ(k+1) Q O and update λ (k+1) λ (k+1) + α + δ for C. After computng λ (k+1) 1 m λ(k+1) λ (k+1), termnate and return λ (k+1). Theorem 1 () shows that a subgradent vector of ψ(λ) at λ (k) s g λ (k) = µ λ (k)g (k), where g (k) = x (k) Q x (k) + 2q x(k) + γ,. To fnd a larger functon value of ψ(λ) at the kth teraton, we use g (k) as the subgradent vector 13
of ψ(λ) rather than g λ (k) for the followng reasons. When µ λ (k) > 0, we can use g (k) as a subgradent vector of ψ(λ) at λ (k). When µ λ (k) = 0 (.e. λ (k) / Λ + ), Q 0 should be postve semdefnte because of the constrant Q 0 + µ m λ Q O, and the optmal value of (17) equals that of (14) (= ϕ(0)). In ths case, ϕ(0) s the smallest possble value, but t s not the optmal one of the orgnal problem (1) because an optmal soluton of the unconstraned problem (14), x, s not n the feasble regon of (1), from Assumpton 1 (c). Therefore, when µ λ (k) = 0, the algorthm needs to move λ (k) toward Λ + ; precsely, λ (k) s moved n the drecton of g (k), although g λ (k) s the zero vector. It can be easly confrmed that by movng λ (k) suffcently far n ths drecton, the left sde of the constrant of (P k ) becomes postve and x moves out of the feasble regon of (P k ). The whole algorthm updates λ (k) and x (k), k = 1, 2,.... In order for (P k ) to have an optmal soluton x (k), λ (k) needs to be set approprately so as to satsfy Q 0 + µ m λ(k) Q O for some µ 0. If the nput Q 0 of the gven problem satsfes Q 0 O, then Q 0 + µ m λ(k) Q O holds wth µ = 0, whch makes (P k ) bounded. On the other hand, n the case of Q 0 O, the optmal value of (P k ), ψ(λ (k) ), possbly becomes, and we can not fnd an optmal soluton. In such case, we can not calculate g (k) and the algorthm stops. To prevent ths from happenng, we defne a subset of Λ s so that the optmal value does not become. When Q 0 O, Λ + can be rewrtten as Λ + = {λ 0 e λ = 1, µ 0; Q 0 + µ m λ Q O}. Λ + s the set of λ for whch ψ(λ) >. However, the above descrpton of Λ + s complcated because of the postve semdefnte constrant. Furthermore, CQ1 requres that ether Q 0 or m λ Q be postve defnte. Therefore, when Q 0 O, we approxmate Λ + as Λ + := {λ 0 e λ = 1, m λ Q O}, and keep λ (k) n Λ + by Step 4. It can be easly confrmed that Λ + s a convex set. By replacng the feasble regon Λ s of (18) by Λ + ( Λ + ), the relaxaton can be weaker and the optmal value s not necessarly equal to the SDP relaxaton value. Thus, when Q 0 O, SLR may be worse than t s when Q 0 O. Now let us explan how to choose the step sze h. Gradent methods have varous rules to determne an approprate step sze. Smple ones nclude a constant step sze h = c or a dmnshng step sze (e.g. h = c/ k), where k s the number of teratons (e.g., see [8]). A more complcated one s the backtrackng lne search (e.g., see [4, 21]). Although the backtrackng lne 14
search has been shown to perform well n many cases, we use a dmnshng step sze h = c/ k to save the computaton tme of SLR. The pont of SLR s to obtan a relaxed soluton quckly, so we should choose the smpler way. In Step 2, we compute λ (k+1) by usng g (k) and h by usng (29). We can easly compute the projecton onto Λ s by usng the method proposed by Chen and Ye [10]. Here, the condton: µ 0; Q 0 + µ m λ Q O n Λ + s gnored n the projecton operaton, but when Q 0 O, the resultng projected pont λ (k+1) s n Λ +. On the other hand, when Q 0 O, the vector λ (k+1) s not necessarly n Λ + or Λ +. In such case, λ (k+1) s modfed n Step 4 so as to belong to Λ +. Step 4 s a heurstc step; t s needed to keep λ Λ + when Q 0 O. 4.3 Settng the Intal Pont The number of teratons of SLR depends on how we choose the ntal pont. In ths secton, we propose two strateges for choosng t. Note that at an optmal soluton λ, all elements λ correspondng to convex constrants wth Q O, C, are expected to have postve weghts. Hence, we wll gve postve weghts only for λ, C (f t exsts). Here, we assume that (Q, q, γ ) n each constrant s approprately scaled by a postve scalar as follows. When the matrx Q has postve egenvalues, (Q, q, γ ) s scaled so that the mnmum postve egenvalue of Q s equal to one. If Q has no postve egenvalues, t s scaled such that the maxmum negatve egenvalue s equal to 1. The frst approach s equal weghts. It gves equal weghts to λ (0) s.t. Q O or f there are no Q O (whch mples that Q 0 O), t gves equal weghts to all λ (0) as follows: Equal weghts rule we defne λ (0) as If the ndex set of convex constrants C s nonempty, λ (0) = { 1 C, f Q O, 0, otherwse. (30) If C =, we defne λ (0) as λ (0) = 1, = 1,..., m. (31) m The second approach uses the dea of the Schur complement. Note that ths rule only apples when there are some ( 1) such that Q O. For the constrant wth Q O, we have x Q x + 2q x + γ 0 (x + Q 1 q) Q (x + Q 1 q ) q Q 1 q γ. 15
The rght-hand sde η := q Q 1 q γ can be consdered the volume of the ellpsod. From Assumpton 1 (a), the ellpsod has postve volume and we have η > 0. A numercal experment shows that constrants havng small postve η tend to become actve n SDP relaxaton. Therefore, t seems reasonable to gve large weghts to constrants whose η (> 0) s small. On the other hand, snce we treat the constrant as ( ) ( 1 γ q ) ( ) 1 0, x q Q x ( γ q the value η can be vewed as the Schur complement of ). It q Q ( γ q s known that when Q O, ) O s equvalent to η q Q 0. ( γ q However, n ths case, ) O does not hold snce η > 0. But we q Q consder ths value to be an ndcator of convexty. We gve large weghts for the constrants whose Schur complement η s large. Then, snce η > 0, we gve large weghts for the constrants whose η (< 0) are close to zero; that s, 1 η are large. Here, we consder the followng rule: Schur complement rule For C, calculate s := 1/ η. We defne λ (0) as λ (0) = { s m s f Q O, 0, otherwse. (32) Although the Schur complement rule also has no theoretcal guarantee, numercal experments show ther usefulness especally when Q 0 O. 4.4 RQT Constrants We may be able to fnd a better optmal value of the SDP relaxaton problem (2) by addng a redundant convex quadratc constrant constructed smlarly to RLT (ths s dscussed n Secton 2.2) to (1) when there are box constrants and by applyng SLR to the resultng QCQP. Snce (9) holds for 1 = j n, we have x 2 (u + l )x + u l 0, = 1,, n. (33) The summaton of (33) for = 1,, n leads to x x (u + l) x + u l 0. (34) We call ths method the reformulaton quadratczaton technque (RQT). Snce (34) s a convex quadratc constrant, t may be effectve for the SLR 16
relaxaton tghter. The numercal experments n Secton 5 show that by addng (34), we could get a tghter optmal value n some cases than SDP relaxatons. There are other ways of makng new convex quadratc constrants. Furthermore, even nonconvex constrants (lke (8) or (10)) are possbly effectve for tghtenng SLR. However, n ths study, we only consdered (34) to save computaton tme. 5 Numercal Experments We mplemented SLR, SDP relaxaton, and Block-SDP (n Secton 2.3) and compared ther results. In [9], there are no rules to decde the number of blocks r of Block-SDP. In our experments, we tred several values of r and chose r := 0.05 n, whch seemed to work well. We used MATLAB Ver. 8.4.0 (R2014b) for all the numercal experments. We solved the SDP relaxaton and Block-SDP by usng SeDuM 1.3 [32]. To solve the convex quadratc optmzaton problems, 1-QCQP (27) and (28) n the SLR algorthm, we used CPLEX Ver. 12.5. We used a computer wth a 2.4 GHz CPU and 16GB RAM. 5.1 Random m-qcqp Frst, we checked the performance of SLR for random m-qcqp generated n the way that Zheng Sun and L [31] dd. In Sectons 5.1.1 5.1.4, we consder problems wthout box constrants; we compare SLR (or CQ1) and SDP relaxaton. In Secton 5.1.5, we consder problems ncludng box constrants; we compare SLR, SDP relaxaton, and Block-SDP. 5.1.1 Tolerance ϵ and Computaton Tme We now nvestgate the computaton tmes of SLR for gven tolerance values ϵ. We randomly generated 30 nstances of a 10-QCQP, whose problem szes were n = 30 and m = 10. Among the m = 10 constrants, there were fve convex ones. The objectve functons of all nstances were strctly convex,.e., Q 0 O. The relatonshp between the tolerance ϵ and the computaton tme s shown n Fgure 4. The smaller ϵ becomes, the longer the computaton takes. In ths settng, SLR can solve the 10-QCQP faster than SDP relaxaton can when ϵ > 10 4. Hence, we set ϵ = 10 4 n what follows. 5.1.2 Effect of Intal Ponts We compared the two strateges for choosng the ntal ponts (30) (or (31)) and (32) and checked the results for Q 0 O and Q 0 O. We only show 17
10 0 SLR SDP relaxaton Tme(s) 10-1 10-10 10-8 10-6 10-4 10-2 Tolerance ǫ Fgure 4: Average Computaton Tme versus Tolerance ϵ results for Q 0 O because both ntal pont rules gave almost the same results when Q 0 O. We randomly generated 30 nstances for each settng, where n = 100 and m = 10, and vared the number of convex constrants from C = 1 to 9. Note that SLR s not stronger than SDP relaxaton and we do not know the exact optmal value of each random m-qcqp. Therefore, we checked the performance of SLR by comparng ts value wth the optmal value of SDP relaxaton. Here, we used the error rato defned as Rato := OPT SLR. OPT SDPrelax Ths ndcator was used n all of the experments descrbed below. It s greater than or equal to one snce SLR s not stronger than SDP relaxaton. The performance of SLR s sad to be good when the rato s close to one. Fgures 5 and 6 plot the number of teratons and the error rato versus the number of convex constrants. When there s only one convex constrant, an optmal soluton λ for ψ(λ) usually has only one postve element correspondng to the convex constrant and all the other elements are zero. In ths case, SLR needs only few teratons. When Q 0 O, the Schur complement rule works well n terms of computaton tme and the error rato as the number of convex constrants ncreases. Ths may be because an optmal soluton of SDP relaxaton has many non-zero components and the equal weghts rule can not represent each weght approprately. On the bass of the above consderatons, we decded to use the Schur complement rule n the remanng experments. 18
Iteraton 10 3 Schur Equal 10 2 10 1 Rato 1.5 1.45 1.4 1.35 1.3 1.25 1.2 1.15 1.1 Schur Equal 1.05 10 0 1 2 3 4 5 6 7 8 9 Convex constrants 1 1 2 3 4 5 6 7 8 9 Convex constrants Fgure 5: Average Number of Iteratons (Q 0 O) Fgure 6: Average Error Rato (Q 0 O) 5.1.3 Effect of the Number of Varables n on the Computaton Tme and Error We checked the computaton tme of SLR by changng the number of varables n. Note that SLR works better when Q 0 O than when Q 0 O because we have to approxmate the feasble regon Λ + when Q 0 O. In ths experment, n was vared from 25 to 5000, and we set m = 15, of whch 8 constrants were convex. We generated 30 nstances when n 250, ten nstances when 250 < n 1000, and one nstance when n 2500. In ths experment, we set ϵ = 1.0 3 because large problems take a very long tme to solve. Case 1. Q 0 O. SLR performed well when Q 0 O (Fgures 7 and 8). The computaton tme was almost one order of magntude smaller than that of SDP relaxaton, and the error rato was less than 1.06. There were many nstances whch SLR can obtan the optmal value same as SDP relaxaton can. Furthermore, SLR was able to solve problems that SDP relaxaton could not because t ran out of memory. Case 2. Q 0 O. We replaced the objectve functon of each nstance used n Case 1 by a nonconvex quadratc functon and conducted the same experments n each case. Fgures 9 and 10 show the results for Q 0 O. The performance of SLR deterorated, but t was stll faster than SDP relaxaton and the error rato was about 1.1. Note that we conducted only one experment on n = 2500, 5000 to shorten the tme of the experment. 19
10 5 SLR SDP relaxaton 10 4 1.06 1.05 10 3 1.04 Tme(s) 10 2 Rato 1.03 10 1 1.02 10 0 1.01 10-1 10 2 10 3 Varables n 1 10 2 10 3 Varables n Fgure 7: Average Computaton Tme for n (Q 0 O) Fgure 8: Average Error Rato for n (Q 0 O) Tme(s) 10 4 SLR SDP relaxaton 10 3 10 2 10 1 10 0 Rato 1.1 1.09 1.08 1.07 1.06 1.05 1.04 1.03 1.02 1.01 10 2 10 3 Varables n 1 10 2 10 3 Varables n Fgure 9: Average Computaton Tme for n (Q 0 O) Fgure 10: Average Error Rato for n (Q 0 O) 20
10 1 SLR SDP relaxaton 1.0015 1.001 Tme(s) 10 0 Rato 1.0005 10-1 10 1 Constrants m 1 10 1 Constrants m Fgure 11: Average Computaton Tme for m (Q 0 O) Fgure 12: Average Error Rato for m (Q 0 O) 5.1.4 Effect of the Number of Constrants m on the Computaton Tme and Error We checked the computaton tme of SLR by varyng the number of constrants m. For n = 100, the number of constrants m was vared from 2 to 50. Half of the constrants (.e. cel(m/2)) were convex. We generated 30 nstances for each settng. Case 1. Q 0 O. Fgures 11 and 12 show the results. As a whole, the error ratos were less than 1.0015, and the computaton tme was about one order of magntude smaller than that of SDP relaxaton. Case 2. Q 0 O. Fgures 13 and 14 show the results. SLR took longer than n Case 1, and the error rato was about 1.06. SLR performed worse when Q 0 O because we approxmated the feasble regon. 5.1.5 RQT Constrants We randomly generated problems wth box constrants and added the RQT constrant proposed n Secton 4.4 to the problems. n was vared from 30 to 500, and we set m = 0.3n, ncludng cel(m/2) convex constrants. We generated 30 nstances when n 100 and 10 nstances when n > 100. We added box constrants ; 1 x 1 to all the nstances. The followng results are only for the case n whch the objectve functon s nonconvex. When Q 0 O, the RQT constrant dd not affect the performance of our method by much. The results are shown n Fgures 15 and 16. In Fgure 16, the rato s less than one. Ths mples that SLR can get better optmal values than 21
10 1 SLR SDP relaxaton 1.06 1.05 1.04 Tme(s) 10 0 Rato 1.03 1.02 10-1 1.01 10 1 Constrants m 1 10 1 Constrants m Fgure 13: Average Computaton Tme for m (Q 0 O) Fgure 14: Average Error Rato for m (Q 0 O) SDP relaxaton can by addng RQT constrants. SLR s thus stronger and faster than SDP relaxaton. In ths sense, Block-SDP s smlar to SLR. The performance of Block-SDP depends on the number of blocks r, but n ths settng, SLR s faster than Block-SDP, although ts error rato s worse than that of Block-SDP. 5.2 1-QCQP We checked the performance of CQ1 n solvng the 1-QCQP of (P k ). We compared CQ1, SDP relaxaton, and an egen-computaton-based method for random 1-QCQPs. For a 1-QCQP wth Q 1 O, Adach, Iwata, Nakatsukasa and Takeda [1] proposed an accurate and effcent method that solves a generalzed egenvalue problem only once. They called ths method GEP. We ran ther MATLAB code for solvng a 1-QCQP wth Q 1 O. Note that all of the methods obtaned the exact optmal value of 1-QCQP. The computaton tme was plotted versus n. As descrbed n Secton 5.1, we generated the 1-QCQP n the way [31] dd. Fgures 17 and 18 are double logarthmc charts of n and the average computaton tme of 30 nstances. Fgure 17 shows that CQ1 s about one order of magntude faster than SDP relaxaton for all n. Fgure 18 shows that CQ1 s faster than GEP when n s large. CQ1 or SLR s ntended to be a weaker, but faster method than SDP relaxaton, and such methods are useful for large-scale problems. 5.3 Max-Cut Problems Max-cut problems [24] can be vewed as an applcaton of 1-QCQP. A graph Laplacan matrx L can be obtaned from a gven undrected and weghted graph G. For the max-cut value for G, we solve the followng nonconvex 22
10 4 SLR SDP relaxaton 10 3 Block-SDP 10 0 SLR Block-SDP 10 2 Tme(s) 10 1 10 0 Rato 10-1 10-1 10-2 10 2 Varables n 10-2 10 2 Varables n Fgure 15: Average Computaton Fgure 16: Average Error Rato for n Tme for n (Addtonal RQT constrant) (Addtonal RQT constrant) 10 2 10 3 CQ1 SDP relaxaton 10 2 10 3 CQ1 SDP relaxaton GEP 10 1 10 1 Tme(s) 10 0 Tme(s) 10 0 10-1 10-1 10-2 10-2 10-3 10 0 10 1 10 2 10 3 Varables n 10-3 10 0 10 1 10 2 10 3 Varables n Fgure 17: Average Computaton Fgure 18: Average Computaton Tme for 1-QCQP n the case of Q 0 Tme for 1-QCQP n the case of Q 1 O O 23
{1, 1} nteger program: We relax (35) nto mn x x Lx s.t. x 2 = 1, = 1,, n. (35) mn x x Lx s.t. 1 x 1 = 1,, n, and then apply SDP relaxaton and Block-SDP. For CQ1, we further relax the box constrants as follows: ; 1 x 1 = x x n, because CQ1 needs at least one convex constrant. The resultng 1-QCQP s mn x x Lx s.t. x x n. (36) Note that (36) can be regarded as a smple mnmum egenvalue problem. An optmal soluton s an egenvector correspondng to the mnmum egenvalue. However, our purpose s to check the computatonal result, and we use CQ1 for (36). We solved max-cut nstances from [24]. Many randomly generated nstances are shown n [24], and the optmal values are known. The results are n Table 1. In ths table, the error s defned as Error := OPT method OPT OPT where OPT method s the optmal value of each method and OPT s the exact optmal value. In [24], the names of the nstances ndcate how they were generated as well as the number of varables. For example, g05 80, 80 means the number of varables, and g05 means the densty of edges and whether the weghts of graph are all postve or nclude negatve values. The detals are gven n [24] and there are ten nstances for each knd of problem. In Table 1, Tme(s) means the average tme for ten nstances, and the best methods among SDP relaxaton, Block-SDP, and CQ1 n terms of ether average computaton tme or average error are lsted n bold. Table 1 shows that CQ1 s weaker but faster than SDP relaxaton. Block-SDP s weaker and even slower than SDP relaxaton n these problem settngs. CQ1 s much faster than SDP relaxaton, so we can solve CQ1, 24
Table 1: Tme and Error for Max-cut Method SDP relaxaton Block-SDP CQ1 Multple CQ1 Instance Error Tme(s) Error Tme(s) Error Tme(s) Error Tme(s) g05 80 0.02 0.363 0.15 0.472 0.15 0.022 0.02 0.075 g05 100 0.02 0.504 0.13 0.545 0.14 0.014 0.01 0.093 pm1d 80 0.17 0.341 0.53 0.419 1.10 0.012 0.10 0.093 pm1d 100 0.17 0.487 0.52 0.504 1.01 0.020 0.09 0.114 pm1s 80 0.15 0.324 0.54 0.392 1.14 0.013 0.10 0.086 pm1s 100 0.14 0.490 0.50 0.491 1.08 0.016 0.12 0.107 pw01 100 0.05 0.478 0.19 0.550 0.62 0.015 0.04 0.099 pw05 100 0.03 0.509 0.13 0.579 0.17 0.015 0.02 0.096 pw09 100 0.02 0.498 0.14 0.611 0.09 0.017 0.01 0.103 w01 100 0.13 0.494 0.53 0.544 1.28 0.013 0.10 0.113 w05 100 0.17 0.482 0.51 0.556 0.92 0.020 0.10 0.114 w09 100 0.17 0.485 0.51 0.562 0.94 0.015 0.12 0.123 many tmes n the same perod of tme t takes to solve the SDP relaxaton once. Accordngly, we tred to strengthen CQ1 by teratng t wth a certan roundng rule as follows. An optmal soluton of CQ1, x, satsfes x x = n because the objectve functon s nonconvex. Consequently, there exsts a component of x whose absolute value s more than one (otherwse, all the components are ±1, and x s an exact optmal soluton for (35)). Then, we fx such a component as ±1 and solve a small problem recursvely. Note that f the objectve functon becomes postve (sem)defnte by fxng some of the components and there exsts no x whose absolute value s more than one, we set the component whch has the maxmum absolute value of all the components to 1 or 1. We perform ths roundng untl all the components are ±1. Therefore, we have a feasble soluton of the orgnal problem (35) and obtan an upper bound of the orgnal optmal value, whle SDP relaxaton, Block-SDP, and CQ1 fnd lower bounds. We call ths roundng Multple CQ1 and show the results n the rght-most column of Table 1. The results ndcate that Multple CQ1 s stll faster than SDP relaxaton. Such a faster method s useful when we want to solve a problem repeatedly. 6 Conclusons In ths paper, we proposed SLR, a new, fast convex relaxaton for QCQP. SLR s a method for solvng the Lagrangan dual problem of a gven QCQP. There have been many studes on constructng Lagrangan dual problems 25
for nonconvex problems and reformulatng them as semdefnte problems (SDPs). Instead of solvng an SDP, our method dvdes the objectve functon of the Lagrangan dual problem nto two parts and teratvely solves a 1-QCQP. We furthermore transform the 1-QCQP nto a convex quadratc 1- QCQP called CQ1 whose feasble regon forms the convex hull of the feasble regon of the orgnal 1-QCQP. Hence, we can obtan the exact optmal value of the 1-QCQP by solvng CQ1. SDP relaxaton can also solve the 1-QCQP exactly, but CQ1 s much faster. Numercal experments confrmed ths advantage of CQ1. CQ1 performed well for randomly generated 1-QCQP and max-cut problems. In SLR, we successvely solve a 1-QCQP wth the Lagrange multpler λ updated usng a gradent method. We proved that the objectve functon ψ(λ) s quas-concave and has the good property that all the statonary ponts n Λ + are global optmal solutons, and thus, smple gradent methods work well. SLR s a faster relaxaton compared wth the nteror pont method for SDP relaxaton for large n and m. Furthermore, by addng a new vald RQT constrant, we could obtan even a better optmal value than SDP relaxaton could for some m-qcqp nstances. Our method can be regarded as a subgradent method that s appled to a quas-concave problem nduced from the Lagrangan dual of an m-qcqp. To ensure convergence, the quas-concave problem must satsfy certan condtons, (e.g., n [15]) but unfortunately, t s not easy to check whether our quas-concave problem satsfes the Hölder condton. In the future, we would lke to nvestgate the global convergence of our algorthm. When the objectve functon s nonconvex, we need to approxmate the feasble regon Λ + of the Lagrangan dual problem, and as a result, the SLR become worse n performance than that of solvng m-qcqp wth the convex objectve functon. We would lke to mprove the performance of SLR for nstances havng nonconvex objectve functons. References [1] S. Adach, S. Iwata, Y. Nakatsukasa and A. Takeda: Solvng the trust regon subproblem by a generalzed egenvalue problem. SIAM Journal on Optmzaton, 27:269 291, 2017. [2] F. A. Al-Khayyal, C. Larsen and T. V. Voorhs: A relaxaton method for nonconvex quadratcally constraned quadratc programs. Journal of Global Optmzaton, 6: 215 230, 1995. [3] K. M. Anstrecher: Semdefnte programmng versus the reformulatonlnearzaton technque for nonconvex quadratcally constraned quadratc programmng. Journal of Global Optmzaton, 43: 471 484, 2009. 26
[4] L. Armjo: Mnmzaton of functons havng Lpschtz contnuous frst partal dervatves. Pacfc Journal of Mathematcs, 16: 1 3, 1966. [5] C. Audet, P. Hansen, B. Jaumard and G. Savard: A branch and cut algorthm for nonconvex quadratcally constraned quadratc programmng. Mathematcal Programmng, 87: 131 152, 2000. [6] L. Ba, J. E. Mtchell and J. S. Pang: Usng quadratc convex reformulaton to tghten the convex relaxaton of a quadratc program wth complementarty constrants. Optmzaton Letters, 8: 811 822, 2014. [7] S. Boyd and L. Vandenberghe: Convex Optmzaton. Cambrdge Unversty Press, UK, 2010. [8] S. Boyd, L. Xao, A. Mutapcc: Subgradent Methods. lecture notes of EE392o, Stanford Unversty, Autumn Quarter, 2003. [9] S. Burer, S. Km and M. Kojma: Faster, but weaker, relaxatons for quadratcally constraned quadratc programs. Computatonal Optmzaton and Applcatons, 59: 27 45, 2014. [10] Y. Chen and X. Ye: Projecton onto s smplex. ArXv:1101.6081v2, 2011. [11] T. Fuje and M. Kojma: Semdefnte programmng relaxaton for nonconvex quadratc programs. Journal of Global Optmzaton, 10: 367 380, 1997. [12] M. X. Goemans: Semdefnte programmng n combnatoral optmzaton. Mathematcal Programmng, 79: 143 161, 1997. [13] M. X. Goemans and D. P. Wllamson. Improved approxmaton algorthms for maxmum cut and satsfablty problems usng semdefnte programmng. Journal of the Assocaton for Computng Machnery, 42: 1115 1145, 1995. [14] Y. Hu, C. Yu and C. L: Stochastc Subgradent method for quasconvex optmzaton problems. Journal of Nonlnear and Convex Analyss, 17: 711-724, 2016. [15] Y. Hu, X. Yang and C. Sm: Inexact subgradent methods for quasconvex optmzaton problems. European Journal of Operatonal Research, 240: 315 327, 2015. [16] R. Jang and D. L: Convex relaxatons wth second order cone constrants for nonconvex quadratcally constraned quadratc programmng. ArXv:1608.02096v1, 2016. 27
[17] S. Km and M. Kojma: Second order cone programmng relaxaton of nonconvex quadratc optmzaton problems. Optmzaton Methods and Software, 15: 201 224, 2001. [18] C. Lu, S. Fang, Q. Jn, Z. Wang and W. Xng: KKT soluton and conc relaxaton for solvng quadratcally constraned quadratc programmng problems. SIAM Journal on Optmzaton, 21:1475 1490, 2011. [19] Z. Q. Luo, W. K. Ma, A. M. C. So, Y. Ye and S. Zhang: Semdefnte relaxaton of quadratc optmzaton problems. IEEE Sgnal Processng Magazne, 27: 20 34, 2010. [20] J. J. Moré and D. C. Sorensen: Computng a trust regon step. SIAM Journal on Scentfc and Statstcal Computng, 4: 553 572, 1983. [21] J. Nocedal and S. J. Wrght: Numercal Optmzaton. Sprnger-Verlag, Berln, 2006. [22] I. Novak: Dual bounds and optmalty cuts for all-quadratc programs wth convex constrants. Journal of Global Optmzaton, 18: 337 356, 2000. [23] P. M. Pardalos and S. A. Vavass: Quadratc programmng wth one negatve egenvalue s NP-hard. Journal of Global Optmzaton, 1: 15 22, 1991. [24] F. Rendl, G. Rnald and A. Wegele: Solvng max-cut to optmalty by ntersectng semdefnte and polyhedral relaxatons. Mathematcal Programmng Seres A, 121: 307 335, 2010. [25] H. D. Sheral and B. M. P. Fratcell: Enhancng RLT relaxaton va a new class of semdefnte cuts. Journal of Global Optmzaton, 22: 233 261, 2002. [26] J. F. Sturm and S. Zhang: On cones of nonnegatve quadratc functons. Mathematcs of Operatons Research, 28: 246 267, 2003. [27] H. Tuy: On solvng nonconvex optmzaton problems by reducng the dualty gap. Journal of Global Optmzaton, 32: 349 365, 2005. [28] L. Vandenberghe and S. Boyd: Semdefnte programmng. SIAM Revew, 38: 49 95, 1996. [29] T. V. Voorhs: A global optmzaton algorthm usng Lagrangan underestmates and the nterval newton method. Journal of Global Optmzaton, 24: 349 370, 2002. 28
[30] X. J. Zheng, X. L. Sun, D. L: Convex relaxatons for nonconvex quadratcally constraned quadratc programmng: matrx cone decomposton and polyhedral approxmaton. Mathematcal Programmng, 129: 301 329, 2011. [31] X. J. Zheng, X. L. Sun, D. L: Nonconvex quadratcally constraned quadratc programmng: best D.C.decompostons and ther SDP representatons. Journal of Global Optmzaton, 50: 695 712, 2011. [32] SeDuM optmzaton over symmetrc cones. http://sedum.e. lehgh.edu/. A Proofs of Theorems Proof of Theorem 1 Proof. The vector g λ of (21) can be found from ψ(λ) = ϕ( µ λ λ) ( = x λ Q 0 + µ λ m λ Q ) x λ + 2 ( q 0 + µ λ m λ q ) x λ + γ 0 + µ λ We prove that the vector g λ s n the quas-subdfferental ψ defned by (22). Note that n [14, 15], the quas-subdfferental s defned for a quasconvex functon, but ψ s quas-concave. Therefore (22) s modfed from the orgnal defnton of ψ for a quas-convex functon. We further consder (22) as ψ(λ) = {s ψ(ν) ψ(λ), ν; s (ν λ) < 0} = {s ψ(ν) ψ(λ), ν; s ν < s λ} (37) Now we show that g λ s n (37). When µ λ = 0, g λ = 0 satsfes (22) and g λ ψ(λ) holds. When µ λ > 0, t s suffcent to consder the vector x λ Q 1 x λ + 2q 1 x λ + γ 1 g λ :=. x λ Q m x λ + 2q m x λ + γ m nstead of g λ because ψ(λ) forms a cone. Then, snce x λ s feasble for λ, we have m λ γ. x λ ( m ) ( m ) m λ Q x λ + 2 λ q x λ + λ γ 0. (38) 29