Combining SOS and Moment Relaxations with Branch and Bound to Extract Solutions to Global Polynomial Optimization Problems

Transcription

1 Combining SOS and Moment Relaxation with Branch and Bound to Extract Solution to Global Polynomial Optimization Problem Heameddin Mohammadi, Matthew M. Peet Abtract In thi paper, we preent a branch and bound algorithm for extracting approximate olution to Global Polynomial Optimization (GPO) problem with bounded feaible et. The algorithm i baed on a combination of SOS/Moment relaxation and ucceively biecting a hyper-rectangle containing the feaible et of the GPO problem. At each iteration, the algorithm make a comparion between the volume of the hyper-rectangle and their aociated lower bound to the GPO problem obtained by SOS/Moment relaxation to chooe and ubdivide an exiting hyper-rectangle. For any deired accuracy, if we ue ufficiently large order of SOS/Moment relaxation, then the algorithm i guaranteed to return a uboptimal point in a certain ene. For a fixed order of SOS/Moment relaxation, the complexity of the algorithm i linear in the number of iteration and polynomial in the number of contrain. We illutrate the effectivene of the algorithm for a 6-variable, 5-contraint GPO problem for which the ideal generated by the equality contraint i not zero-dimenional - a cae where the exiting Moment-baed approach for extracting the global minimizer might fail. I. INTRODUCTION Global Polynomial Optimization (GPO) i defined a optimization of the form f := min. f (x) (1) x R n ubject to g i (x) 0 for i = 1,, h j (x) = 0 for j = 1,,t, where f, g i, and h i are real-valued polynomial in deciion variable x. A defined in Eq. 1, the GPO problem encompae many well-tudied ub-clae including Linear Programming (LP), Quadratic Programming (QP), Integer Programming (IP), Semidefinite Programming (SDP) and Mixed-Integer Nonlinear Programming (MINLP) [1]. Becaue of it generalized form, almot any optimization problem can be cat or approximately cat a a GPO, including certain NP-hard problem from economic dipatch [2], optimal power flow [3] and optimal decentralized control [4]. A applied to control theory, GPO can be ued for tability analyi of polynomial dynamical ytem by, e.g., verifying polytopic invariant a in [5]. Although GPO i NP-hard, there exit a number of heuritic and algorithm that can efficiently olve pecial cae of GPO. For example, LP [6], [7], QP [8], and SDP all have aociated polynomial-time algorithm. More broadly, if the feaible et i convex and the objective function i convex, then barrier function and decent method will typically yield a computationally tractable algorithm. When the problem i not convex, there alo exit pecial cae in which the GPO problem i olvable. For example, in [9] the uncontrained problem wa olved by parameterizing the critical point of the objective function via Groebner bae. In the pecial cae of x R 1, the problem wa olved in [10], [11]. In addition, there exit everal widely ued heuritic which often yield reaonably uboptimal and approximately or exactly feaible olution to the GPO problem (e.g. [12], [13]), but which we will not dicu here in depth. If we expand our definition of algorithm to include thoe with combinatorial complexity but finite termination time, then if the feaible et of the GPO problem i compact and the ideal generated by the equality contraint i radical and zero dimenional (typical for integer programming [14]), then one may ue the moment approach to olving equential Greatet Lower Bound (GLB) problem to obtain an algorithm with finite termination time. Unfortunately, however, it ha been hown that the cla of problem for which thee method terminate i a trict ubet of the general cla of GPO problem [15] and furthermore, there are no tractable condition verifying if the algorithm will terminate or bound on computational complexity in the cae of finite termination. In thi paper, however, we take the idea of generating Greatet Lower Bound and propoe an alternative method for extracting approximate olution that doe not require finite convergence and, hence, ha polynomial-time complexity. Let S := {x R n : g i (x) 0, h j (x) = 0} (2) be the feaible et and x be an optimal olution of GPO Problem (1). The Greatet Lower Bound (GLB) problem aociated to GPO Problem (1) i defined a λ :=max. λ (3) ubject to f (x) λ > 0, x S. The GLB and GPO problem are cloely related, but are not equivalent. For example, it i clear that λ = f = f (x ), where f i a defined in Eq. (1). Furthermore, a dicued in Section V, an algorithm which olve the GLB problem with complexity O(k) can be combined with Branch and Bound to approximately olve the GPO for any deired level of accuracy ε with complexity O(log(1/ε)k), where we define an approximate olution to the GPO a a point x R n uch that x x ε and f (x) f (x ) < ε. Many convex approache have been applied to olving the GLB problem, all of which are baed on parameterizing the cone of polynomial which are poitive over the feaible et of the correponding GPO problem. The mot well-known of thee approache are Sum of Square (SOS) programming [16], and it dual Moment-relaxation problem [17]. Both thee approache are well-tudied and

2 have implementation a Matlab toolboxe, including SOS- TOOLS [18] and Gloptipoly [19]. Thee approache both yield a hierarchy of primal/dual emidefinite program with an increaing aociated equence of optimal value {p k } k N (SOS) and {dk } k N (Moment) uch that if we denote the OBV of the correponding GPO problem by f, then under mild condition, both equence atify p k d k f and lim k p k = lim k d k = f [20]. Moreover, if the feaible et of GPO Problem (1), S, a defined in (2), i nonempty and compact, then there exit bound on the error of SOS/Moment relaxation which cale a p k f = c 2 c 1 for contant log(k) c 1 and c 2 that are function of polynomial f, g i and h j [21]. The goal of thi paper, then, i to combine the SOS and Moment approache with a branch and bound methodology to create an algorithm for olving the GPO problem and extracting a olution. Specifically, in Sec. VI, we propoe a equence of branch and bound algorithm, denoted by E k, uch that for any k N and for a given GPO problem of Form (1) with a bounded feaible et, Algorithm E k in polynomial-time return a point x k R n that i ub-optimal to the GPO problem in the following ene. If x k i the equence of propoed olution produced by the equence of algorithm E k, we how that if the feaible et, S, of Problem (1) i bounded, then there exit a equence of feaible point y k S uch that lim x k y k = 0 k and lim f (y k) = lim f (x k) = f, where f i the OBV of the k k GPO problem. The equence of algorithm can be briefly ummarized a follow. Algorithm E k initialize with hyper-rectangle C 1 := {x (x i c i,1 )( c i,1 x i ) 0, i = 1,,n} uch that S C 1. Then, at iteration m, the algorithm form two new hyperrectangle by biecting C m by it longet edge ( c i c i ) and interecting each of the new hyper-rectangle with S to create two new GPO problem. Then, the algorithm compute a GLB etimate of the OBV of the new GPO problem by olving the correponding k th-order SOS/Moment relaxation. If λ m denote the bet lower bound to f, obtained up to iteration m, then C m+1 i determined to be the exiting hyperrectangle with a mallet volume ubject to the contraint that the correponding GLB i le that mη l + λm, where η > 0 and l N are the deign parameter. Finally, after certain number of iteration, the algorithm terminate by returning the centroid of the lat hyper-rectangle. In Sec. VII we dicu the complexity of the propoed algorithm by firt howing that for any k 2, the feaible et of the k th order SOS relaxation aociated to a branch i completely contained in that of each of it ubdivided branche - implying that the lower bound obtained by both the SOS and Moment relaxation (due to the duality between SOS and Moment method) are increaing. Of coure, at thoe branche with S C m = /0, thee bound will approach +. Next, we how that if the error of the k th order SOS/Moment relaxation aociated to the hyper-rectangle c 2 c 1 log(k) η/(l + 1), then each of the biected hyper-rectangle i obtained through Algorithm E k i bounded by guaranteed to contain a feaible point that i η-uboptimal. Therefore, the feaible et will be reduced in volume a 2 1 m, up to iteration m. In other word, the number of iteration neceary to achieve a hyper-rectangle with a longet edge of the length ε i logarithmic in 1/ε. Finally, ince at each iteration the number of contraint i fixed and the complexity i linear in the number of iteration, we conclude that algorithm E k i polynomial-time. In Sec. VIII, we illutrate the effectivene of the propoed algorithm by applying it to an example problem wherein the exiting Moment baed approach fail to extract a olution. Finally we conclude in Sec. IX. II. NOTATION Let N n be the et of n-tuple of natural number. We ue S n and S n+ to denote the ymmetric matrice and cone of poitive emidefinite matrice of ize n n, repectively. For any a, b R n, we denote by C(a,b) the hyper-rectangle {x R n a x b}, where y 0 i defined by the poitive orthant. We ue l to denote the et of bounded infinite equence. For any k,n N, let N n (k) := {b Nn : b l1 k}, where b l1 := n b i. Finally, we denote the ring of multivariate polynomial with real coefficient a R[x]. III. PROBLEM STATEMENT In thi paper, we conider implified GPO problem of the form: f := min. f (x) (4) x R n ubject to g i (x) 0 for i = 0,, where f,g i R[x]. The cla of problem in (4) i equivalent to that in (1), where we have imply replaced every h i (x) = 0 contraint with ome g 1 (x) = h(x) 0 and g 2 (x) = h(x) 0. For every problem of Form (4), we define the aociated feaible et S := {x R n : g i (x) 0}. In thi paper, we aume S /0. Note that given g i, one may ue SOS optimization combined with Poitivtellenatz reult [22] to determine feaibility of S. Propoed Algorithm In thi paper, we propoe a GLB and Branch and Bound-baed algorithm which, for any given ε > 0, will return ome x R n for which there exit a point y S uch that: f (y) f ε, and y x < ε. (5) Furthermore x itelf i ε-uboptimal in the ene that f (x) f ε and g i (x) ε. Before defining thi algorithm, however, in the following ection, we decribe ome background on the dual SOS and Moment algorithm for generating approximate olution of the GLB Problem. IV. BACKGROUND ON SEMIDEFINITE REPRESENTATIONS OF SOS/MOMENT RELAXATIONS In thi ection, we decribe two well-known aymptotic algorithm which are known to generate equence of increaingly accurate uboptimal olution to the GLB problem - namely the SOS and Moment approache. Both thee method ue Poitivtellenatz reult which parameterize

3 the et of polynomial which are poitive over a given emialgebraic et. A. Sum-of-Square Polynomial In thi ubection, we briefly define and denote et of um of quare of polynomial. We denote monomial in variable x R n a x α := n xα i i where α N n. Monomial can be ordered uing variou ordering on N n. In thi paper, we ue the graded lexicographical ordering. Thi ordering i defined inductively a follow. For a,b N n, a b if n a i < n b i, or a 1 = b 1 and [a 2,,a n ] [b 2,,b n ]. Denote by Z(x) the infinite ordered vector of all monomial, where x α < x β if α < β. Becaue we have ued the graded lexicographical ordering, if we retrict ourelve to the firt ( d+n) d element of Z, then thi i the vector of all monomial of degree d or le. We denote thi truncated vector a Z d (x) and the length of Z d a Λ(d) := ( d+n) d. Uing thi definition, it i clear that any polynomial can be repreented a p(x) = c T Z d (x) for ome c R Λ(d), where d i the degree of p. The vector of monomial can alo be combined with poitive matrice to completely parameterize the cone of um-of-quare polynomial. Formally, we can denote the ubet of polynomial which are the um of quare of polynomial a l Σ S := { R[x] : (x) = p 2 i (x), p i R[x], l N}. (6) Clearly any element of Σ S i a nonnegative polynomial. Furthermore, if p Σ S and i of degree 2d, then there exit a poitive emidefinite matrix Ω S Λ(d)+ uch that p(x) = Z d (x) T ΩZ d (x). Converely, any polynomial of thi form, with Ω 0, i SOS. Thi parametrization of SOS polynomial uing poitive matrice will allow u to convert the GLB problem to an LMI. However, before defining thi LMI approach, we mut examine the quetion of poitivity on emialgebraic ubet of R n. B. Putinar Poitivtellenatz, Quadratic Module and the Archimedean Property Sum-of-Square polynomial are globally non-negative. In thi ection, we briefly review Putinar poitivtellenatz which give neceary condition for a polynomial to be poitive on the emiaglebraic et S := {x R n : g i (x) 0, i = 1,...,}, where S /0 and i compact. Putinar Poitivtellenatz ue the g i which define S to deduce a cone of polynomial which are non-negative on S. Thi cone i the quadratic module which we define a follow. Definition 1: Given a finite collection of polynomial g i R[x], we define the quadratic module a M :={p p = σ 0 + σ i g i σ i Σ S }, and the degree-k bounded quadratic module a M (k) :={p p = σ 0 + σ i g i σ i Σ S deg(σ i g i ) k}. Clearly, any polynomial in M i non-negative on S. Furthermore, ince Σ S parameterize M and poitive matrice parameterize Σ S, the contraint p M k can be repreented a an LMI. Furthermore, if the module atifie the Archimedean property, then Putinar Poitivtellenatz tate that any polynomial which i poitive on S i an element of M. That i, M parameterize the cone of polynomial poitive on S. A quadratic module M i aid to be Archimedean if there exit ome p M and R 0 uch that p(x) = R 2 n x2 i. We ay that {g i } i an Archimedean repreentation of S if the aociated quadratic module i Archimedean. Note that the Archimedean property i a property of the function g i which then define the quadratic module and not a property of S. Specifically, if S i compact, then there alway exit an Archimedean repreentation of S. Specifically, in thi cae, there exit an R > 0 uch that x R for all x S. Now define g +1 (x) = R 2 n x2 i. C. SOS approach to olving the GLB problem In thi ubection, we briefly decribe the ue of SOS programming to define a hierarchy of GLB problem. Conider the GPO Problem (4) where {g i } i an Archimedean repreentation of the feaible et, S, with aociated quadratic module M. We now define the degreeunbounded verion of the SOS GLB problem. f = λ := max. λ (7) ubject to f (x) λ M. Since M i Archimedian, it follow that λ = f (where f i a defined in (4)). Although Problem (7) i convex, for practical implementation we mut retrict the degree of the SOS polynomial which parameterize M - meaning, we mut retrict ourelve to optimization on M (k). Thi define a new equence of GLB problem a p k := max. λ (8) ubject to f (x) λ M (k). Clearly, p i p j λ for any i < j. Additionally, it wa hown in [16] that lim p k = p. Furthermore, it wa hown k in [20], [21] that bound on the convergence rate of p k λ exit a a function of g i, f and k. Finally, the computational complexity of p k i equivalent to that of a emidefinite program with order ( + 1)Λ( 2 k )2 calar variable. D. Moment approach to olving the GLB problem In thi ubection, we briefly decribe the Moment approach to olving the GLB problem. Let K denote the et of Borel ubet of R n and let M (K) be the et of finite and igned Borel meaure on K. The following lemma ue the et of probability meaure with upport on S to provide a neceary and ufficient condition for a polynomial to be poitive over S. Lemma 1: Given S K and f : R n R integrable over S, a polynomial f (x) i nonnegative on S if and only if f (x) dµ 0 for all µ M (K) uch that µ(s) = 1 and S µ(r n /S) = 0.

4 Again, conider GPO Problem (4) with the feaible et S. Now, if we define the moment optimization problem λ := max. µ M (K), λ (9) ubject to ( f λ) dµ 0, S µ(s) = 1 and µ(r n /S) = 0, then Lemma 1 implie, uing a duality argument, that λ = f where f i a defined in Problem (4) [17]. Unfortunately, Problem (9) require u to optimize over the pace of meaure µ M (K). However, a yet we have no way of parameterizing thee meaure or impoing the contraint µ(s) = 1 and µ(r n /S) = 0. Fortunately, we find that a meaure can be parameterized effectively uing the moment generated by the meaure. That i, any meaure µ M (K) ha an aociated ordered vector of moment, indexed by α N n uing monomial x α a ψ(µ) α := x α dµ. Furthermore, if µ(s) = 1 and µ(r n /S) = 0, then ψ(µ) T c 0 for all c C g, where we define C g := {c : c T Z(x) M} l, where M i the module defined by the g i in S := {x : g i (x) 0}. More ignificantly, the convere i alo true. Thi mean that if we add the contraint ψ 1 = 1 to enure µ(s) = 1, then we may replace the meaure variable µ by the moment variable ψ, a decribed. However, thi require u to enforce the contraint ψ T c 0 for all c uch that c T Z(x) M. Thi contraint, however, can be repreented uing emidefinite programming [20], [17]. Finally, if f (x) = d T Z(x), we can enforce the integral contraint of optimization problem (1) a f (x) λ dµ = d T ψ λ 0. S Thi allow u to formulate the equivalent GLB problem a S d = max. λ (10) y l, ubject to y T d λ 0, y T c 0, c C g, y 0 = 1. Then d = λ and a for the SOS GLB problem, we define a equence of truncation of the moment GLB problem dk := max. λ (11) y R q, ubject to y T d λ 0, y T c 0, c C k g, y 0 = 1, where q = Λ(k) and Cg k := {c : c T Z k (x) M (k) }. It ha been hown that p k d k for all k and furthermore lim k p k = lim k d k = λ. Moreover, if the interior of S i not empty, then p k = d k [20]. Note that yt c 0, c Cg k i an SDP contraint and the number of deciion variable i Λ(k). Thi implie that the computational complexity of d k and p k are imilar. In the following ection, we combine the GLB problem defined by the SOS/Moment approach with a Branch and Bound equence to approximately olve the GPO problem. V. SOLVING THE GPO PROBLEM USING SOS, MOMENTS AND BRANCH AND BOUND In thi ection, we how that the following algorithm can be ued to olve the GPO problem given a olution to the GLB problem. The Ideal Branch and Bound Algorithm At every iteration, we have a hyper-rectangle A i = [a i,b i ]; 1) Initialize the algorithm; 2) Biect A = [a i,b i ] = [a,b ] [a,b ] = A 1 A 2 ; 3) Compute the Greatet Lower Bound of λ i :=max. λ (12) ubject to f (x) λ > 0, x S A i ; 4) If λ 1 > λ 2, et A = A 1, otherwie A = A 2 ; 5) Goto 2 ; At termination, we chooe any x A, which will be accurate within x x r2 k/n. Let u examine thee tep in more detail. Initialize the algorithm Since the et S i compact, there exit ome r > 0 uch that S B r (0). We may then initialize A = [ r1,r1], where 1 i the vector of all 1. Biect Biection of the hypercube occur along the longet edge. Thu, after n iteration, we are guaranteed a two-fold increae in accuracy. A a reult, the larget edge of the hypercube diminihe a 2 k/n. Compute the Greatet Lower bound We aume that our olution to the GLB problem i exact. In thi cae, we are guaranteed that an optimizing x will alway lie in A i. A. Complexity of the Ideal Branch and Bound Algorithm In thi ubection, we how that any exact olution to the GLB can be ued to olve the GPO problem with arbitrary accuracy in a logarithmic number of tep uing the Ideal Branch and Bound algorithm. Suppoe G i a GLB Problem of the Form (3) with olution λ. Define the algorithm H : G λ a λ = H(G). Further uppoe H ha time-complexity O(k), where k i a meaure for the ize of G. In the Ideal Branch and Bound algorithm, if we ue H to perform Step (3), it i traightforward to how that for any ε > 0, after m = 2n(c 1 + log 1 ε ) iteration, if A m = [a,b], then max i b i a i ε and GPO Problem (4) ha a minimizer x C(a,b), where c 1 depend on the ize of S and n i the number of variable. Now ince all the GLB problem defined in Step (3) of the algorithm are of equal ize, k, the complexity of the Ideal Branch and Bound algorithm for a given ε, i O(2nk(c 1 + log 1 ε )). In thi ection, we conidered the ideal cae when the GLB can be olved exactly. In the following ection we adapt thi algorithm to the cae when equential algorithm uch a SOS/Moment problem are ued to olve the GLB problem. In thi cae, our approach will alo be defined a a equence of approximation algorithm.

5 VI. MODIFIED BRANCH AND BOUND ALGORITHM In thi ection, we preent a lightly modified branch and bound algorithm that combined with SOS/Moment relaxation, can approximate the olution to the GPO problem to any deired accuracy, in a certain ene. The Modified Branch and Bound Algorithm At every iteration, we have an active hyper-rectangle A = [a,b] and a et of feaible rectangle Z = {[a i,b i ]} i each with aociated GLB λ i. 1) Initialize the algorithm 2) Biect A = [a,b] = [a,b ] [a,b ] = A 1 A 2 3) Compute the Greatet Lower Bound of λ i :=max. λ (13) ubject to f (x) λ > 0, x S A i. 4) If λ i λ + ε, add A i to Z. 5) Set A = Z i where Z i i the mallet element of Z. 6) Goto 2 At termination, we chooe any x A, which will be accurate within x x r2 k/n. A. Problem Definition and SOS/Moment Subroutine Conider GPO Problem (4) and uppoe the correponding feaible et S := {x R n : g i (x) 0}, i nonempty and compact with S C(a,b), for ome a,b R n with aociated Archimedean quadratic module M. Before defining the main equential algorithm E k, we will define the k t h-order SOS/Moment GLB ubroutine, denoted B k, which calculate the GLB in Step (3) of the Modified Branch and Bound Algorithm. SOS/Moment Subroutine λ k = B k[a,b] Given A = [a,b], define the polynomial w i (x) := (b i x i )(x i a i ). Thee polynomial are then ued to define the modified feaible et S A a S ab := {x R n :g i (x) 0, i : 1 i, (14) w j (x) 0, j : 1 j n}, and the correponding modified degree-k bounded quadratic module a M (k) ab := {p : p = i=0 σ i g i + +n i=+1 σ i w i, σ i Σ S, (15) } deg(σ i g i ) k, deg(σ i w i ) k, where g 0 (x) = 1. Thi allow u to formulate and olve the modified k-th order SOS GLB problem p k := max. ubject to λ (16) f (x) λ M (k) ab and the correponding dual GLB moment problem a decribed in the preceeding ection. The ubroutine return the value λ k = p k. B. Formal Definition of the Modified Branch and Bound Algorithm, E k We now define a equence of Algorithm E k uch that for any k N, E k take GPO Problem (4) and return an etimated feaible point x. The Sequence of Algorithm E k : In the following, we ue the notation a b to indicate that the algorithm take value b and aign it to a. That i, a = b. In addition, 0 < η < 1 and l N are deign parameter and may be choen arbitrarily. Some guidance on the choice of η and l i given in the ection on complexity analyi. The input to the following algorithm E k are the function {g i } and f, the initial hyper-rectangle uch that S [a,b], and the deign parameter. The output i the etimated feaible point, x. Algorithm E k : input: η > 0, l N, a,b R n, f,g 1,...,g R[x]. output: x R n (a an approximate olution to GPO Problem (4)). Initialize: a(0) a; b(0) b; m 0; λ(0) B k (a(0),b(0)); While (m < l) : { j argmin λ( j); j {0,...,m} i argmin j {0,...,m} n (b( j) i a( j) i ) ubject to λ( j) λ( j ) + mη 1 + l ; (17) a a(i ); b b(i ); r argmax(b j a j); ã a ; ˆb b ; j {1,...,n} For r from 1 to n : { { b { r +a r b r 2 if r = r b r +a b r otherwie ; â r r 2 if r = r a r otherwie, ;} λ B k (ã, b); ˆλ Bk (â, ˆb); m m + 1; a(i ) ã; b(i ) b; λ(i ) λ; a(m) â; b(m) ˆb; λ(m) ˆλ;} Return x := a(l)+b(l) 2 ; In the following ection we will dicu the complexity and accuracy of the equence of Algorithm E k. VII. CONVERGENCE AND COMPLEXITY OF E k In thi ection, we firt how that for any k N, the lower bound obtained through Algorithm E k are increaing. Next we how that for a ufficiently large k, Algorithm E k can be deigned to return an accurate uboptimal point, in a certain ene. In the following lemma, we ue Lemma 3 from the Appendix to how that for any a 1, a 2, b 1, b 2 R n uch that a 1 a 2 and b 2 b 1 R n, the feaible et of the SOS

6 problem olved in Subroutine B k (a 1,b 1 ) i contained in that of Subroutine B k (a 2,b 2 ). Lemma 2: For any k N and a b R n, let M (k) ab be the modified degree-k bounded quadratic module aociated to polynomial g 1,...,g, a defined in (15). If γ α < β δ R n, then M (k) γ δ M(k) α β, for all k 2. Proof: For any j = 1,...,n, let w j,1 (x) := (β j x j )(x j α j ), and w j,2 (x) := (δ j x j )(x j γ j ). Since γ j α j < β j δ j, then it i followed from Lemma 3 in the Appendix that there exit p j,q j,r j R uch that w j,2 (x) = p 2 j(x) w j,1 (x) + q 2 j(x) (x j + r j ) 2. Now, Let h M (k) γ δ, we will how that h M(k) α β. Clearly, there exit σ i,ω j,2 Σ S uch that h = i=0 σ ig i + n j=1 ω jw j,2, where g 0 (x) = 1. Hence, we can plug in the expreion for w j,2 to get h = =(σ 0 + i=0 n σ i g i + j=1 n j=1 ω j (p 2 j w j,1 + q 2 j (x j + r j ) 2 ) q 2 j ω j (x j + r j ) 2 ) + } {{ } σ 0 new σ i g i + n p 2 j ω j w j,1. j=1}{{} ω j new Then, ince k 2, it can be hown that there exit σ 0 new,ω j new Σ S where deg(σ 0 new ) k, and deg(ω j new w j,1 ) k which implie that h M (k) α β. If d i the maximum degree of polynomial g i, then for any k d +2 and for any hyper-rectangle C(c,d) C(a,b), if λ (a,b) and λ (c,d) be the olution obtained by Subroutine B k (a,b) and B k (c,d) applied to GPO Problem (4), then Lemma 2 how that λ (a,b) λ (c,d). Now, for a fixed k N, let η and l be the deign parameter of Algorithm E k applied to GPO Problem (1). For m = 0,...,l, let (λ ) m := λ( j ), where j i defined a in iteration m of Algorithm E k. Uing Lemma 2, it i traightforward to how that (λ ) m (λ ) m+1 for m l 1. In the next theorem, we will how that for any given ε > 0, there exit k N uch that Algorithm E k applied to GPO Problem (4) can be deigned to provide a point x R n atifying (5). Theorem 1: Suppoe GPO Problem (4) ha a nonempty and compact feaible et S. Chooe a,b R n uch that S C(a,b). For any deired accuracy, ε > 0, there exit k,l N, η R uch that if x = E k (η,l,a,b, f,g i ) then there exit a feaible point y S uch that if f i the OBV of Problem (4), then f (y) f ε and y x < ε, where f i the OBV of GPO Problem (4). Furthermore, there exit a bound c > 0, depending only on f and g i uch that we may retrict the number of branche, l, in E k (η,l,a,b, f,g i ) a l < clog(1/ε). Proof: Chooe η uch that 0 < η < min{ε,1}. For any l N, define P l a the et of all hyper-rectangle generated through Algorithm E k, k N with the two firt input η and l. The vertice of all element of P l, clearly, lie on a grid with pacing a i,b i for i = 1,...,n. Therefore, P 2 l l i finite. Let L be a longet edge of C(a,b). Define l η a the mallet natural number l ubject to L n 2 l/n η. It can be hown that for any C α := C(e, f ) P lη, there exit a k α N uch that for any k k α, the olution of Subroutine η B k (e, f ) i accurate with the error tolerance 1+l η. Define k η := max{k α : C α P lη }. We now claim that for the deign parameter l := l η and η, k k η, Algorithm E k return the deired point. Let k k η. Firt, we how that Algorithm E k with deign parameter l η and η generate exactly l η neted hyper-rectangle. Proof by induction on m. For m = 0,...,l η 1, let (a) m := a(i ), (b) m := b(i ), (C) m := C((a) m,(b) m ), (λ) m := B k ((a) m,(b) m ), (ã) m := ã, ( b) m := b, (â) m := â, (ˆb) m := ˆb, ( C) m := C((ã) m,( b) m ), (Ĉ) m := C((â) m,(ˆb) m ) and (λ ) m := λ( j ) where i, j, ã, b, â and ˆb are defined a in iteration m of Algorithm E k. We ue induction on m to how that for all m l η : (C) m (C) m 1. The bae cae m = 0 i trivial. For the inductive tep, firt note that (λ ) m f for all m l η and (λ ) 1 (λ ) lη. The latter i obtained from Lemma 2 and the former i becaue at each iteration, S m i=0 C(a(i),b(i)). Contraint (17) at iteration m, implie that B k ((a) m,(b) m ) (λ ) m + mη l η + 1. (18) Now we will how that again, Contraint (17) at iteration m+1 i atified at leat by one of ( C) m and (Ĉ) m. Suppoe thi i not true. Then we can write (λ (m + 1)η ) m+1 <B k ((ã) m,( b) m ), (19) 1 + l η (λ ) m+1 <B k ((â) m,(ˆb) m ) (m + 1)η 1 + l η. Now, ince (λ ) m+1 (λ ) m, Eq. (19) implie (λ (m + 1)η ) m <B k ((ã) m,( b) m ), (20) 1 + l η (λ ) m <B k ((â) m,(ˆb) m ) Uing Eq. (20) and Eq. (18) one can write (m + 1)η 1 + l η. B k ((a) m,(b) m ) <B k (( b) m,( b) m ) η/l η, (21) B k ((a) m,(b) m ) <B k ((â) m,(ˆb) m ) η/l η. Thi contradict with the fact that all B k ((ã) m,( b) m ), B k ((â) m,(ˆb) m ) and B k ((a) m,(b) m ) have accuracy higher than η/(1+l η ). Therefore, it i clear that both ( C) m and (Ĉ) m can be poible choice to be biected at iteration m + 1. Thi fact, together with the induction hypothei which certifie that (C) m poee the mallet volume between all the hyper-rectangle obtained up to that iteration, guaranteeing that either ( C) m or (Ĉ) m, will be branched at the next iteration. Therefore, the algorithm will generate l η neted hyper-rectangle. Now, (λ ) 0 [ f η/l η, f ] implie that (λ ) m [ f η/l η, f ], for all m = 1,...,l η. (22)

7 Eq. (18) and Eq.(22) together with the fact that (λ) m (λ ) m imply (λ) m [ f η/l η, f + mη l η + 1 ], m = 1,...,l η. (23) Finally, a a pecial cae m = l η, one can write: f η B k ((a) l,(b) l ) f + l η η 1 + l η l η + 1. Now, note that the η/(l η + 1)-accuracy of B k ((a) lη,(b) lη ) implie that (C) lη i feaible. It alo can be implied that (C) lη S contain y uch that f (y) B k ((a) lη,(b) lη f (y) η/(l η + 1). Therefore, f (y) f η ε. Finally, if x be the point return by Algorithm E k, then baed on the definition of l η, it i implied that after the lat iteration m = l η 1, the larget diagonal of the branched hyper-rectangle i le than η ε, hence y x 2 ε, a deired. Theorem 1 enure that for any accuracy ε > 0 there exit a k N uch that Algorithm E k, in logarithmic number of iteration, achieve approximate olution to the GPO problem in a certain ene. The following corollary how that thee approximate olution can approximately atify the contraint a follow. Corollary 1: Let GPO Problem (4) have nonempty and compact feaible et that i contained in C(a, b) for ome a,b R n. For any given δ > 0, there exit ε > 0 uch that if ε and x = E(η,l,a,b, f,g i ) atify the condition in Theorem 1, then f (x) f δ and g i (x) δ, i = 1,...,. (24) Proof: Let L be uch that any polynomial h { f,g 1,...,g } atifie h(c) h(d) L c d 2, c,d C(a,b). ( Exitence of L follow from the Lipchitz continuity of polynomial on compact et.) Chooe ε uch that ε δ/l. Let ε and x atify the condition in Thoerem 1. It i traightforward to how that x atifie Eq. (24). Unfortunately, however, Theorem 1 doe not provide any bound on the ize of thi k. Now, ince all the SOS problem that are olved in Subroutine B k, differ at only 2n parameter, (which i baically hown in the proof of Lemma 2) one may wih to ue SOS bound p k f = c 1 introduced in log(k) [21] to obtain a bound on uch k a a function of polynomial g i. However, depite the effort that led to Lemma 2, the author were not able to find uch a unique bound for all the B k problem, ued in Algorithm E k. Thi i remained unolved and will be a topic for the future reearch. Finally, ince obtaining a bound on k for Algorithm E k i computationally difficult, for practical application, given ε > 0, we firt chooe a k and run the algorithm until it aturate. By aturation we mean it doe not generate neted hyperrectangle effectively and a a reult approximation do not get improved effectively. For mall k N, Algorithm E k may yet yield good approximation of an optimal point. Indeed, it may not generate only neted hyper-rectangle, however, it can be hown that for a fixed k, by increaing the number of iteration we can eventually obtain any number of neted hyper-rectangle at a cot of higher complexity. VIII. NUMERICAL RESULTS Conider the following GPO problem. min. f (x) = 7x 1 x 3 x R x 1x5 2 x 6 + 9x 2 x x 2x 4 x 5 + 3x 2 x 5 x 6 + x 3 x 4 x 5 ubject to g 1 (x) = 100 (x x x x x x2 6 ) 0 g 2 (x) = x x2 2x 4 + x 3 x g 3 (x) = x 2 2x 1 + x x 4x 1 x 2 0 h 1 (x) = x 1 + x 2 2 x x 4 x 5 = 0 h 2 (x) = x 5 x 1 x 2 4 = 0 In thi example we have 6 variable, an objective function of degree 4 and everal equality and inequality contraint of degree 4 or le. The ideal generated by equality contraint i not zero dimenional, hence the exiting Moment method of extracting olution might fail. We applied Algorithm E 5 with deign parameter η = and l = 200 to thi example uing Sedumi to olve the SDP related to both the SOS and Moment problem. A een in Figure 1, the branch and bound algorithm converge relatively quickly to a certain level of error and then aturate. Iteration pa thi point do not ignificantly improve accuracy of the feaible point. A predicted, thi aturation and reidual error (blue haded region) i due the ue of a fixed degree bound k = 5. A k i decreaed, the reidual error increae and a k i increaed the reidual error decreae. For thi problem the final iteration return the point ˆx = [5.1416, , , , , ] for which all inequalitie are feaible and the equality contraint h 1 and h 2 have error of and , repectively. The objective value i f ( ˆx) = IX. CONCLUSION We propoed a hierarchy of Algorithm E k, k N to extract olution to the GPO problem baed on a combination of Branch and Bound and SOS/Moment relaxation. The computational-complexity of Algorithm E k i polynomial in k, polynomial in the number of contraint and linear in the number of branche l. Additionally, for any caler ε > 0, there exit k N uch that Algorithm E k, in O(log(1/ε)) number of iteration, return a point that i within the ε- ditance of a feaible and ε-uboptimal point. For a fixed degree of emidefinite relaxation, our numerical cae tudy demontrate convergence to a level of reidual error which can then be decreaed by increaing the degree. In ongoing work, we eek to bound thi reidual error a a function of degree uing available bound on the error of SOS/Moment relaxation. Thi, however require a more deep undertanding of the relationhip between thee bound and tructure of the emialgebraic et.

8 Sum of Error in Inequality Contraint (a) Sum of error in inequality contraint at center, gi (x m) 0 g i (x m ) v. number of iteration Objective Value at Centroid Minu Lower Bound (c) Gap between lower bound and objective value at center point, f (x m ) l v. number of iteration Volume of the Branched Hypercube (e) Lower bound v. number of iteration Objective Value at the centroid Sum of Error in Equality Contraint (b) Sum of error in equality contraint at center, j h j (x m ) v. number of iteration The Lowet Lower Bound (d) Lower bound v. number of iteration Longet Edge of the Branched Hypercube (f) Longet edge of the branched hypercube v. number of iteration (g) Objective function value at center point v. number of iteration Fig. 1. Numerical reult of the example APPENDIX The following lemma give an algebraic property of the polynomial of the form w(x) = (x a i )(b i x) which are ued to define the augmented feaible et S ab. Lemma 3: Let a c < d b R, g := (x a)(b x) and h := (x c)(d x). Then, there exit α,β and γ R, uch that g(x) = αh(x) + β(x + γ) 2, α,β 0 Proof: Without lo of generality, one can aume that a = 0 (conider the change of variable z := x a). Now let p 2 := c, q 2 := d c, and r 2 := b d. Firt, we conider the cae where p 2,r 2 0. Thi lead to two ub-cae: Cae 1 : r 2 p 2. Let γ = p4 + p 2 q 2 p 2 r 2 (p 2 + q 2 )(q 2 + r 2 ) r 2 p 2, β = p4 + p 2 q 2 γ 2 p 4 p 2 q 2, and α = β +1. Verifying the equality g(x) = αh(x)+β(x+ γ) 2 i traightforward. To how that β,α 0, we ue the following. β 0 γ 2 > p 4 + p 2 q 2 ( ) 2 p 4 + p 2 q 2 p 2 r 2 (p 2 + q 2 )(q 2 + r 2 ) > (p 4 + p 2 q 2 )(r 2 p 2 ) 2 (p 4 + p 2 q 2 ) 2 + p 2 r 2 (p 2 + q 2 )(q 2 + r 2 ) (p 4 + p 2 q 2 )(r 2 p 2 ) 2 > }{{} L 2 ( { p 4 + p 2 q 2) p 2 r 2 (p 2 + q 2 )(q 2 + r 2 L > 0 ) }{{} L 2 > U 2 U After implification we have: L 2 U 2 = p 4 q 4 (p 2 + q 2 ) 2 (p 2 r 2 ) 2 > 0, and L = p 2 ( p 2 + q 2) ( p 2 q p 2 r 2 + q 2 r 2) > 0 which complete the proof for Cae 1. Cae 2 : r 2 = p 2. In thi cae, let γ = 2p2 + q 2 2, β = 4p2 (p 2 + q 2 ) q 4, α = β + 1 Equality and poitivity for thi cae can then be eaily verified. Now, uppoe r 2 = p 2 = 0. In thi cae, imply et β = 0, α = 1. If p 2 = 0,r 2 0, et β = b d 1, α = b d, γ = 0. The cae p 2 0, r 2 = 0 i imilar to p 2 = 0, r 2 0,through the change of variable z = b x. REFERENCES [1] M. Laurent, Emerging Application of Algebraic Geometry, ch. Sum of Square, Moment Matrice and Optimization Over Polynomial, pp New York, NY: Springer New York, [2] J. B. Park, Y. W. Jeong, J. R. Shin, and K. Y. Lee, An improved particle warm optimization for nonconvex economic dipatch problem, IEEE Tranaction on Power Sytem, vol. 25, no. 1, pp , [3] B. Ghaddar, J. Marecek, and M. Mevien, Optimal power flow a a polynomial optimization problem, IEEE Tranaction on Power Sytem, vol. 31, no. 1, pp , [4] J. Lavaei, Optimal decentralized control problem a a rankcontrained optimization, in Communication, Control, and Computing (Allerton), t Annual Allerton Conference on, pp , IEEE, [5] M. A. B. Sai and A. Girard, Computation of polytopic invariant for polynomial dynamical ytem uing linear programming, Automatica, vol. 48, no. 12, pp , [6] L. Khachiyan, Polynomial algorithm in linear programming, USSR Computational Mathematic and Mathematical Phyic, vol. 20, no. 1, pp , [7] N. Karmarkar, A new polynomial-time algorithm for linear programming, Combinatorica, vol. 4, no. 4, pp [8] Z.-Q. Luo and S. Zhang, A emidefinite relaxation cheme for multivariate quartic polynomial optimization with quadratic contraint, SIAM Journal on Optimization, vol. 20, no. 4, pp , [9] B. Sturmfel, Solving ytem of polynomial equation. No. 97, American Mathematical Soc., [10] H. D. Sherali and C. H. Tuncbilek, New reformulation linearization/convexification relaxation for univariate and multivariate polynomial programming problem, Operation Reearch Letter, vol. 21, no. 1, pp. 1 9, [11] N. Z. Shor, Quadratic optimization problem, Soviet Journal of Computer and Sytem Science, vol. 25, no. 6, pp. 1 11, 1987.

9 [12] W. Cook, T. Koch, D. E. Steffy, and K. Wolter, A hybrid branchand-bound approach for exact rational mixed-integer programming, Mathematical Programming Computation, vol. 5, no. 3, pp , [13] D. E. Steffy and K. Wolter, Valid linear programming bound for exact mixed-integer programming, INFORMS Journal on Computing, vol. 25, no. 2, pp , [14] J. Laerre, Polynomial nonnegative on a grid and dicrete optimization, Tranaction of the American Mathematical Society, vol. 354, no. 2, pp , [15] J. Nie, Optimality condition and finite convergence of laerre hierarchy, Mathematical Programming, vol. 146, no. 1, pp , [16] P. A. Parrilo, Structured emidefinite program and emialgebraic geometry method in robutne and optimization. PhD thei, Citeeer, [17] J. B. Laerre, Moment, poitive polynomial and their application, vol. 1. London: World Scientific, [18] S. Prajna, A. Papachritodoulou, and P. A. Parrilo, Introducing otool: A general purpoe um of quare programming olver, in Deciion and Control, 2002, Proceeding of the 41t IEEE Conference on, vol. 1, pp , IEEE, [19] D. Henrion and J.-B. Laerre, Poitive Polynomial in Control, ch. Detecting Global Optimality and Extracting Solution in GloptiPoly, pp Berlin, Heidelberg: Springer Berlin Heidelberg, [20] M. Schweighofer, Optimization of polynomial on compact emialgebraic et, SIAM Journal on Optimization, vol. 15, no. 3, pp , [21] J. Nie and M. Schweighofer, On the complexity of putinar poitivtellenatz, Journal of Complexity, vol. 23, no. 1, pp , [22] G. Stengle, A nulltellenatz and a poitivtellenatz in emialgebraic geometry, Mathematiche Annalen, vol. 207, no. 2, pp , 1974.