Complex Quadratic Optimization and Semidefinite Programming

Transcription

1 Coplex Quadratic Optiization and Seidefinite Prograing Shuzhong Zhang Yongwei Huang August 4 Abstract In this paper we study the approxiation algoriths for a class of discrete quadratic optiization probles in the Heritian coplex for. A special case of the proble that we study corresponds to the ax-3-cut odel used in a recent paper of Goeans and Williason. We first develop a closed-for forula to copute the probability of a coplex-valued norally distributed bivariate rando vector to be in a given angular region. This forula allows us to copute the expected value of a randoized with a specific rounding rule solution based on the optial solution of the coplex SDP relaxation proble. In particular, we study the liit of that odel, in which the proble reains NP-hard. We show that if the objective is to axiize a positive seidefinite Heritian for, then the randoization-rounding procedure guarantees a worst-case perforance ratio of π/4.7854, which is better than the ratio of /π.6366 for its counter-part in the real case due to Nesterov. Furtherore, if the objective atrix is real-valued positive seidefinite with non-positive off-diagonal eleents, then the perforance ratio iproves to Keywords: Heritian quadratic functions, approxiation ratio, randoized algoriths, coplex SDP relaxation. MSC subject classification: 9C, 9C. Departent of Systes Engineering and Engineering Manageent, The Chinese University of Hong Kong, Shatin, Hong Kong zhang@se.cuhk.edu.hk. Research supported by Hong Kong RGC Eararked Grants CUHK433/E and CUHK474/3E. Departent of Systes Engineering and Engineering Manageent, The Chinese University of Hong Kong, Shatin, Hong Kong ywhuang@se.cuhk.edu.hk.

2 Introduction The pioneering work of Goeans and Williason [5] has caused a great deal of exciteent in the field of optiization, as it used a new tool SDP in continuous optiization, through randoization and probabilistic analysis, to yield an excellent approxiation ratio for a classical cobinatorial optiization proble, known as the ax-cut proble. This ground-breaking work has been extended in various ways since its first appearance. Aong others, Frieze and Jerru [4] extended the ethod to solve the general ax-k-cut proble. Bertsias and Ye [3] introduced another randoization schee using noral distributions, to achieve the sae approxiation result as in Goeans and Williason s original paper [5]. The Bertsias-Ye analysis akes use of an iportant result in statistics, which states that the probability of a bivariate -diensional norally distributed rando vector to be in the first orthant can be expressed analytically using eleentary functions. This is ipossible however, for any diension higher than three; see []. Recently, Goeans and Williason [6] proposed another novel approach to solve the ax-3-cut proble, using the unit circle in the coplex plane as a key odelling ingredient. In this paper we show that it is possible to copute the probability of the bivariate coplex-valued norally distributed rando vector to be in a specific angular region in C. Using this result, we are able to copute the expected quality of a particular randoized solution for solving a general quadratic optiization odel, where the variables take values fro the roots of z = is an integer paraeter of the odel. The odel of Goeans and Williason for ax-3-cut = 3 is a special case of this general odel. It is interesting to study the liit of this odel; that is, the case where. It turns out that the proble reains NP-hard. However, the corresponding coplex SDP relaxation yields an approxiation ratio of π/4.7854, whereas for its counter-part in the real case, the ratio is /π.6366 as shown by Nesterov [8]. If the off-diagonal eleents of the objective atrix are real-valued and non-positive, then the approxiation ratio is actually This paper is organized as follows. In Section we discuss the coputation of the probability for the coplex-valued noral distributions. In Section 3 we apply the results developed in Section to solve coplex-valued quadratic optiization probles. In particular, Subsection 3. discusses the Heritian quadratic function iniization proble, where the coplex decision variables take discrete values. Subsection 3. considers the continuous version of the proble. Subsection 3.3 considers a special case where a sign restriction on the objective atrix is observed. Notation. Throughout, we denote by ā the conjugate of a coplex nuber a, by C n the space of n-diensional coplex vectors. For a given vector z C n, z H denotes the conjugate transpose of z. The space of n n real syetric and coplex Heritian atrices are denoted by S n and H n, respectively. For a atrix Z H n, we write Re Z and I Z for the real and iaginary part of Z, respectively. Matrix Z being Heritian iplies that Re Z is syetric and I Z is skew-syetric.

3 We denote by S+ n S++ n and H+ n H++ n the cones of real syetric positive seidefinite positive definite and coplex Heritian positive seidefinite positive definite atrices, respectively. The notation Z eans that Z is positive seidefinite positive definite. For two coplex atrices Y and Z, their inner product Y Z = Re tr Y H Z = tr [ Re Y T Re Z + I Y T I Z ], where tr denotes the trace of a atrix and T denotes the transpose of a atrix. Coplex Bivariate Noral Distribution It is well known that the density function of an n-diensional real-valued ultivariate noral distribution is given as follows fx = π n/ det Ω exp x µt Ω x µ, where µ R n is the ean and Ω S n ++ is the covariance atrix. Let us consider a coplex-valued norally distributed rando variable in C, with the ean value z C and variance σ R +. For ore inforation on the coplex-valued noral distributions, we refer the reader to []. Siilar as in the real-valued case, its density function can be written as fz = πσ exp z z /σ, z C. Denote by N c z, σ the coplex-valued noral distribution with ean z and variance σ. Using Euler s forula, i.e., letting z z = ρe iθ, we have fρ, θ = ρ πσ exp ρ σ, with ρ, θ [, + [, π, where the variable transforation is { Re z z = ρ cos θ I z z = ρ sin θ. As a atter of terinology, ρ is usually called the odulus of z z, also denoted as z z ; θ is called the arguent of z z, denoted as Arg z z. The density of the joint coplex-valued noral distribution z = z, z,..., z n, with z k C, k =,..., n, has the following for fz = π n det Ω exp z µh Ω z µ, where z, µ C n, and Ω H n ++; µ is the ean vector, and Ω is the covariance atrix. 3

4 Let us denote the above coplex-valued noral distribution as N c µ, Ω. The bivariate case is of particular interest to us. Consider a coplex-valued, bivariate noral rando vector. Suppose that it has zero-ean. Furtherore, suppose that its covariance atrix is [ ] λ Ω = λ where λ C denotes the conjugate of λ C. In particular, let λ = γe iα, and so λ = γe iα. Since Ω, it follows that γ >. Moreover, Ω = γ [ γe iα γe iα ]. Then, by letting z = ρ e iθ and z = ρ e iθ, we ay rewrite the density function as [ ] H [ ] [ fρ, ρ, θ, θ = 4π γ exp ρ e iθ γe iα γ ρ e iθ γe iα ρ ρ = 4π γ exp ρ + ρ ρ ρ γ cos α + θ θ γ, where the doain of the variables is given as ρ, ρ, θ, θ [, + [, π. ρ e iθ ρ e iθ ] Now let us further introduce a variable transforation { ρ = ρ cos ξ ρ = ρ sin ξ with the doain ρ, ξ [, + [, π/]. The density function can be further written as fρ, ξ, θ, θ = ρ3 cos ξ sin ξ 4π γ exp ρ γρ cos ξ sin ξ cos α + θ θ γ ρ 3 sin ξ = 8π γ exp ρ ρ γ sin ξ cos α + θ θ γ, and the doain is ρ, ξ, θ, θ [, + [, π/] [, π. Let us note the following two siple facts. Lea. Suppose that a > is a given real nuber. Then, it holds that ρ 3 exp aρ dρ = a. 4

5 Lea. Suppose that < b < is a given real nuber. Then, with respect to the variable θ, it holds that sin θ b sin θ dθ = cos θ b b sin θ + b tanθ/ b b arctan + C. 3/ b Consider θ b < θe π and θb < θe θ, θ [θ b, θe ] [θb, θe ]. π. Below we shall copute the probability that Let us denote P : = Prob {θ b θ θ; e θ b θ θ} e θ e θ e π/ [ ρ 3 sin ξ = θ b θ b 8π γ exp ρ ρ ] γ sin ξ cos α + θ θ γ dρ dξdθ dθ θ e [ θ e π/ γ = 6π γ sin ξ dξ] dθ dθ θ b θ b γ sin ξ cos α + θ θ = θ γ e [ θ e ] π/ sin ξ 4π γ cos α + θ θ sin ξ dξ dθ dθ = γ 4π θ b θ b θ e θ e θ b θ b [ γ cos α + θ θ + + γ cos α + θ θ arccos γ cos α + θ θ γ cos α + θ θ 3/ ] dθ dθ, where in the third equality we used Lea. and in the last equality we used Lea.. To further copute the above integration, we note the following two ore facts: Lea.3 With respect to the variable θ, it holds that [ γ cos θ + γ cos θ arccos γ cos θ γ cos θ 3/ ] dθ = γ θ + γ sin θ arccos γ cos θ + C. γ cos θ Lea.4 With respect to the variable θ, it holds that [ ] γ sinβ θ arccos γ cosθ β dθ = γ cos θ β arccos γ cosθ β + C. Using Lea.3 we obtain [ P = 4π θ e θθ b e θ b + θ e θ b θ e θ b γ sinθ b α θ arccos γ cosθ b α θ γ cos θ b α θ γ sinθ e α θ arccos γ cosθ e α θ γ cos θ e α θ dθ 5 dθ,

6 and further using Lea.4, we have P = θe θb θe θb 4π + 8π + arccos γ cosθ b θ b + α [ arccos γ cosθ e θ e + α arccos γ cosθ e θ b + α arccos γ cosθ b θ e + α ]. Suarizing, we have proven the following result by a liiting arguent. [ Theore.5 For the coplex-value bivariate noral rando vector [ ] [ µ = and Ω = it holds that ] γe iα γe iα H+, z z ] N c µ, Ω with Prob {θ b Arg z θ e ; θ b Arg z θ e } = θe θb θe θb 4π + 8π + arccos γ cosθ b θ b + α [ arccos γ cosθ e θ e + α arccos γ cosθ e θ b + α arccos γ cosθ b θ e + α ]. 3 Quadratic Progras and Coplex SDP Forulations 3. Discrete Coplex Quadratic Optiization Suppose that Q is a Heritian atrix. Consider the following quadratic prograing proble with discrete decision variables, P ax z H Qz s.t. z k {, ω,..., ω }, k =,..., n, where and ω = e i π = cos π + i sin π. Denote the optial value of P to be vp. Consider the following coplex-valued apping F F z := ω ω 8π j= ω j arccos Re ω j z. For a Heritian atrix Z with Z kl for all k, l, define the coponentwise atrix function F Z := F Z kl n n H n. It is easy to verify that F z = F z. Therefore, if Z is Heritian, then so is F Z. 6

7 Lea 3. We have = ω ω 8π j= ω j arccos Re ω j. Moreover, for any z {, ω,..., ω } it follows that F z = z. Proof. We observe that Moreover, we have j= ω ω 8π = ω ω 8π = ω ω 8 4 j= j= j= j ω j = ω ω ω and ω j arccos cos j π ω j π j j ω j 4 j= j= jω j = Substituting the above equations into yields the intended result. jω j. ω. Suppose z = ω j for soe j {,..., n}. Then, ω ω 8π = ω ω 8π = ω ω 8π j= j= j= = ω j ω ω 8π = ω j = z. ω j arccos Re ω j z ω j arccos cos j j π ω j arccos cos j j π j j= j ω j arccos cos j π This copletes the proof for Lea 3.. Hence we can rewrite P as ax Q F zz H s.t. z k {, ω,..., ω }, k =,..., n. 7

8 Consider the following nonlinear coplex seidefinite prograing proble SP ax Q F Z s.t. Z kk =, k =,..., n, Z. Let vsp denote the optial value of SP. Theore 3. It holds that vp = vsp. Proof. Let ẑ is optial to P, then, by Lea 3., Ẑ = ẑẑ H is a feasible solution to SP and F Ẑ = Ẑ. Therefore, vsp Q FẐ = Q Ẑ = vp. On the other hand, for every feasible solution Z of SP, we randoly generate a coplex vector ξ such that ξ N c, Z, and assign, if Arg ξ k [, π ω,. if Arg ξ k [ π, π y k = σξ k = ω j, if Arg ξ k [ j j+ π, π. ω, if Arg ξ k [ π, π and finally let z k = ȳ k, k =,..., n. Suppose that Z kl = γe iα. Then by Theore.5, we have Prob {y k = y l ω j, y l = ω r } = Prob {y k = ω j+r, y l = ω r } = Prob {Arg ξ k [ j + r π, j + r + π, Arg ξ l [ r π, r + π} = + 8π arccos γ cos j π + α arccos γ cos j π + α arccos γ cos j + π + α for any j, r {,,..., }. Therefore, for any given k and l we have = Prob {y k ȳ l = ω j } r= Prob {y k = y l ω j, y l = ω r } = + 8π arccos γ cos j π + α arccos γ cos j π + α arccos γ cos j + π + α. 3 8

9 It follows that = Consequently, E[y k ȳ l ] j= ω j Prob {y k ȳ l = ω j } = 8π ω j arccos γ cos j π + α arccos γ cos j π + α j= arccos γ cos j + π + α = 8π ω j ω j ω j+ arccos γ cos j π + α j= = ω ω 8π = ω ω 8π j= j= E[z k z l ] = E[y k ȳ l ] = ω ω 8π = ω ω 8π = ω ω 8π = ω ω 8π ω j arccos γ cos j π + α ω j arccos Re ω j Z kl. 4 j= j= j= j= By the linearity of atheatical expectation, we get ω j arccos γ cos j π + α ω j arccos γ cos j π + α ω j arccos γ cos j π + α ω j arccos Re ω j Z kl. E[z H Qz] = Q F Z. Since the solution z so generated is feasible to P, we have vp E[z H Qz] = Q Z, for every feasible solution Z of SP. This cobining with vsp vp yields the desired result. 9

10 In particular, if = then one can verify that proble P reduces to and proble SP reduces to ax x T Qx s.t. x k {±}, k =,..., n, ax π Q arcsinx s.t. X kk =, k =,..., n, X, where arcsinx := [arcsinx kl ] n n. In that case, Theore 3. specializes to Theore.9 in [5] or Theore in []. If = 3, then P is ax z H Qz s.t. z k {, ω, ω }, k =,..., n, with ω = e i π 3. In fact, Goeans and Williason [6] odel the ax-3-cut proble as M3C ax k<l n w klz k z l H z k z l s.t. z k = {, ω, ω }, k =,..., n, and they consider the following coplex SDP relaxation ax k<l n w kl Re Z kl s.t. Z kk =, k =,..., n Re Z kl /, Re ωz kl /, Re ω Z kl /, k < l n Z. Let the optial solution of the SDP relaxation be Z. Then, Theore 3. asserts that the expected value of the randoized solution based on Z is k<l n w kl Re F 3 Z kl where F 3 z = 9 8π [ arccos Re z + ωarccos Re ω z + ω arccos Re ωz ]. Since arccosx is a convex function, it follows that Re F 3 Zkl = 9 [ 8π arccos Re Zkl arccos Re ω Zkl + arccos Re ωzkl ] [ 9 8π arccos Re Zkl arccos Re ωzkl + ω Zkl ] = 9 8π [arccos Re Z kl arccos Re Z kl ].

11 Further noticing that [ 9 arccos ] x + 4π in arccos x = x< x the approxiation ratio of Goeans and Williason [6] thus follows fro the fact that k<l n w kl Re F 3 Z kl { w kl 9 [arccos Re 8π Z kl arccos ]} Re Z kl k<l n.836 w kl Re Zkl k<l n.836 v M3C. 3. Continuous Coplex Quadratic Optiization By taking the liit, i.e., the quadratic optiization odel P becoes CP ax z H Qz s.t. z k =, k =,..., n, where Q H n +. In that case, the proble is equivalent to with SCP ax Q F Z F z := li F z = 4π where γ = z and α = Arg z. π s.t. Z kk =, k =,..., n, Z e iθ arccos γ cosθ α dθ The applications of Heritian quadratic optiization odels such as CP can be found, e.g. in [7], although in [7] the iniization version of the proble was considered. Proposition 3.3 Proble CP is strongly NP-hard in general.

12 Proof. The optiization proble in the for of ax s.t. z T Az z k C, z k, k =,..., n is called coplex prograing, and was shown in [9] to be NP-hard in general. We thank André Tits for drawing our attention to coplex prograing. Proble CP is related to coplex prograing, but they are not the sae: the objective in CP takes the Heritian for, and is assued to be positive seidefinite. The proof for Proposition 3.3 to be presented below is due to To Luo of Minnesota University, who sketched this proof to us in a private counication. As a first step we shall prove that the following proble in z H Qz s.t. z k =, k =,..., n, is NP-hard in general, where Q H+. n To this end, we consider a reduction fro the following strongly NP-coplete atrix partition proble; i.e., given a atrix G = [G,..., G N ] R M N, decide whether or not a subset of {,..., N} exists, say I, such that G k = N G k. k I k= Let the decision vector be z = z, z,, z N, z N+,, z N T C N+. Let n = N +, and e N I N I N A := Ge N G T R M+N n, N where e N R N is the vector of all ones. Let Q := A T A. Next we show that a atrix partition exists is equivalent to the fact that there is z C n with z k = for all k, such that z H Qz =. Clearly, z H Qz = is equivalent to Az = ; that is, = z + z k + z N+k, k =,..., N 5 = N N G k z + G k z k. 6 k= k= Let z k /z = e iθ k for k =,..., N. Using 5 we have cos θ k + cos θ N+k = 7 sin θ k + sin θ N+k = 8

13 where k =,..., N. Equations 7 and 8 iply that θ k { π/3, π/3}. This in particular eans that cos θ k = cos θ N+k = / for k =,..., N. Since Re N N G k + G k z k /z = N N G k + G k cos θ k = k= k= is always satisfied, 6 is true if and only if I N G k + k= k= k= k= N N G k z k /z = G k sin θ k =, which aounts to the existence of a atrix partition. Let λ ax be the axiu eigenvalue of Q. By observing that in z H Qz s.t. z k =, k =,..., n, k= is equivalent to ax z H λ ax I Qz s.t. z k =, k =,..., n, where λ ax I Q H+, n the desired result follows. For a given z C with z = γe iα and z = γ, we have F z = π e iθ arccos γ cosθ α dθ 4π π = 4π eiα e iθ arccos γ cos θ dθ = [ π 4π eiα e iθ arccos γ cos θ dθ = eiα π e iθ π arccosγ cos θ dθ π ] e iθ arccosγ cos θ dθ = π eiα e iθ arcsinγ cos θdθ = π eiα e γ iθ k! cos θ + 4 k k! γ cos θk+ dθ k + k= = π 4 γeiα + π k! 4k+ k! 4 k + γk+ e iα k= = π 4 z + π k! 4k+ k! 4 k + z k z, 9 k= 3

14 where the second last step follows fro the fact that π sin θcos θ k+ dθ = and π cos θ k+ dθ = k + k π, k =,,... k + k Clearly, if Z H n + then Z T H n +. Furtherore, observe that the Hadaard product of any two positive seidefinite Heritian atrices reains Heritian positive seidefinite. Denote A B to k {}}{ be the Hadaard product of A and B, and denote A k to be A A A. It thus follows fro 9 that F Z = π 4 Z + π k= k! 4k+ k! 4 k + ZT Z k Z π 4 Z. Therefore, if Q, then we have Q F Z π 4 Q Z. Consider the following coplex SDP relaxation for CP CSDP ax Q Z s.t. Z kk =, k =,..., n, Z. Let the optial value of CP be v CP, and the optial value of CSDP be v CSDP. Let the expected value of the randoized solutions based on the optial solution of CSDP be vhc. Then vhc π 4 v CSDP π 4 v CP.7854 v CP. It is interesting to copare this ratio with that of its real counterpart: RP ax x T Qx s.t. x k =, k =,..., n. Nesterov [8] showed that the randoization solution based on the SDP relaxation RSDP ax Q X s.t. X kk =, k =,..., n, X, has the following approxiation ratio vhr π v RSDP π v RP.6366 v RP. 4

15 Therefore, the coplex SDP relaxation for the coplex quadratic optiization proble is ore effective than the real SDP relaxation for its real counter-part, in the sense that the forer has a slightly better approxiation ratio. Reark that siilar as the analysis in Nesterov [8], Ye [], and Zhang [] for the real case, we can extend all the approxiation results to the following ore general setting ax z H Qz s.t. z, z,, z n T F, where F is a closed convex set in R n. The corresponding coplex and convex SDP relaxation is ax s.t. Q Z diag Z F Z. It is also interesting to reark that if we regard CP as an equivalent real quadratic proble ax u T, v T Re Q I Q u I Q Re Q v s.t. u k + v k =, k =,..., n, then the approxiation ratio obtained that way would be /π, instead of π/4. This shows that the coplex SDP relaxation does have an advantage in this particular case. 3.3 Structured Continuous Coplex Quadratic Optiization In this subsection, we study a special case of CP with a sign structure on the object atrix, which is parallel to the original real ax-cut odel studied in [5]: CPS ax z H Qz s.t. z k =, k =,..., n, where we assue that Q = [q jl ] n n S n + and q jl for all j < l n. Using 9 we know that the expected value of the randoized solution based on the coplex SDP relaxation is vhc = j<l = j<l q jl Re F Z jl + n q jl π 4 + π k= j= q jj k! n 4k+ k! 4 k + Z jl k Re Zjl + q jj j= 5

16 where Z is the optial solution of the coplex SDP relaxation. Define the following real function gy := π 4 + π k= k! 4k+ k! 4 k + yk on y [, ]. We have gy for all y [, ]. Suppose that x is real, and x y. Then, gyx in = in gy + gy = + gyy. x y x x y x + y One coputes that Therefore, + gyy in y + y.9349 =: β. gyx β βx, for all y [, ] and x y, or equivalently, k= gyx β + βx for all y [, ] and x y. Using, we have π 4 + π k! 4k+ k! 4 k + Z jl k Re Zjl β + βre Z jl. Now we apply in a coponentwise fashion to, and obtain, thanks to the sign restriction, the following inequalities vhc = j<l j<l q jl π 4 + π k= q jl β + βre Z jl + n = βe T Qe + βq Z βv CSDP k! n 4k+ k! 4 k + Z jl k Re Zjl + j= q jj βv CP S. 3 j= q jj This yields an approxiation ratio of.9349 for CPS. Acknowledgeent: We would like to thank To Luo, Anthony So, Yinyu Ye, and Jiawei Zhang for stiulating discussions on the subject. 6

17 References [] I.G. Abrahason. Orthant probability for the quadrivariate noral distribution. The Annals of Matheatical Statistics 35: , 964. [] H.H. Andersen. Linear and graphical odels for the ultivariate coplex noral distribution. Springer-Verlag, 995. [3] D. Bertsias and Y. Ye. Seidefinite relaxations, ultivarite noral distribution, and order statistics. In D.Z. Du and P.M. Pardalos, editors, Handbook of Cobinatorial Optiization, volue 3, pages 9. Kluwer Acadeic Publishers, 998. [4] A. Frieze and M. Jerru. Iproved approxiation algoriths for MAX-k-CUT and Max BI- SECTION. Algorithica 8: 67 8, 997. [5] M.X. Goeans and D.P. Williason. Iproved approxiation algoriths for axiu cut and satisfiability probles using seidefinite prograing. Journal of the ACM 4: 5 45, 995. [6] M.X. Goeans and D.P. Williason. Approxiation algoriths for MAX-3-CUT and other probles via coplex seidefinite prograing. Journal of Coputer and Syste Sciences 68: 44 47, 4. [7] Z.Q. Luo, X.D. Luo, M. Kisialiou. An efficient quasi-axiu likelihood decoder for PSK signals. Proceedings IEEE International Conference on Acoustics, Speech, and Signal Processing ICASSP 3, Pages: VI vol. 6, 3. [8] Yu. Nesterov. Seidefinite relaxation and nonconvex quadratic optiization. Optiization Methods and Softwares 9: 4 6, 998. [9] O. Toker and H. Özbay. On the coplexity of purely coplex µ coputation and related probles in ultidiensional systes. IEEE Transactions on Autoatic Control 43: 49 44, 998. [] Y. Ye. Approxiating quadratic prograing with bound and quadratic constraints. Matheatical Prograing 84: 9 6, 999. [] S. Zhang. Quadratic axiization and seidefinite relaxation. Matheatical Prograing 87: ,. 7