Global Optimization of Polynomials

Transcription

1 Semidefinite Programming Lecture 9 OR 637 Spring 2008 April 9, 2008 Scribe: Dennis Leventhal Global Optimization of Polynomials Recall we were considering the problem min z R n p(z) where p(z) is a degree 2m polynomial such that p(z) = α 2m p α z α, where α = (α 1,..., α n ) is a vector of exponents, using the notation that z α = z α zn αn. One equivalent way of expressing this problem is to notice this is the same as minimizing p(z)µ(dz) over the set of all probability distributions µ and note that it s equivalent to consider only the first 2m moments of µ. We came up with a new problem by noting that even moments are non-negative, leading to the semi-definite relaxation: (D) min y { p α y α : y 0 = 1, M(y) 0}, α 2m where the matrix M(y) satisfies M(y) β,γ = y β+γ where β, γ m. We can express this problem in dual form with the constraint A y C 0 with C = e 0 e T 0 and take the dual to get: (P ) max X { x 0,0 : β, γ m,β+γ=α Is there a natural interpretation for (P)? Recall, for any polynomial q(z) and probability distribution µ, x βγ = p α, for all α 2m, α 0, X 0}. q 2 (z)µ(dz) = α q β q γ y α β, γ m,β+γ=α where the inner term is the (β, γ) entry of qq T, a rank-1 positive semi-definite matrix. Since X 0, we can express X = i λ i q i qi T using the eigenvalue decomposition. Hence, we can express the constraints as 1

2 β, γ m,β+γ=α i λ i q i βq i γ = p α, for all α 2m, α 0, or, equivalently, λ i qβq i γ i = p α, for all α 2m, α 0, i β, γ m,β+γ=α i.e., p(z) is a non-negative (since eigenvalues of X are non-negative) combination of squares of polynomials (almost, i.e., its constant coefficient may not be correct). Since we assumed without loss of generality that p 0 = 0, we ve actually shown that the constraints say that p(z) p, where p = x 0,0, is a sum of squares (SOS) polynomial and we want to maximize p. Since being a sum of squares is a sufficient condition for being non-negative everywhere, the p term in a feasible solution gives a lower bound on min z p(z). This approach was championed by Parrilo (and, in fact, goes back to Shor). This leads to the natural question of when the set of everywhere-non-negative polynomials of degree 2m in n variables coincides with the set of SOS polynomials. This question was answered by Hilbert, who showed the non-negative everywhere polynomials are the SOS polynomials exactly when one of the following conditions hold: 1. n = 1 (univariate polynomials); 2. m = 1 (quadratic polynomials in any number of variables); and 3. n = m = 2 (quartic polynomials in two variables). Hence, there are many examples of non-negative polynomials that are not SOS. However, it can be shown that if p(z) > 0 everywhere, then for some finite r, the polynomial p(z)( i z 2 i ) r is in fact SOS. Hence, there exists a hierarchy of SDP problems (for r = 0, 1,...) whose optimal values approach min z p(z). 2

3 SDP Duality SDP and Lagrangian Duality We will show that SDP Duality is in fact a special case of Lagrangian Duality. In general, consider an optimization problem of the form min{f(x) : g(x) = 0, x S}. Then its Lagrangian Dual, corresponding to dualizing the constraints g(x) = 0, is given by: (LD) maxmin {f(x) y x S yt g(x)}. Note that since g(x) = 0 for any feasible solution to the original problem, the inner minimization is a lower bound on the original problem for any y. Hence, (LD) seeks the best possible lower-bound of this form. For example, consider an SDP of the form: (P ) min{c X : AX = b, X 0}. By dualizing the constraints AX b = 0, we obtain the new problem (LD) max y min{c X y T (AX b) : X 0} = maxb T y + min (C x y X 0 A y) X. Note that if (C A y) 0, we can find a direction such that the inner minimization goes toward. Otherwise, the optimal value is 0, giving us the better-known form of the dual problem max{b T y : C A y 0} by appealing to the self-duality of the semi-definite cone. Exercise: Show that (P) is the Lagrangian Dual of (D) by dualizing the constraints A y + S = C. 3

4 Weak Duality Recall that for any X feasible for P and any (y, S) feasible for D, we have C X b T y = S X 0. This leads to a natural corollary: If X and (y, S) are (P) and (D) feasible, respectively, and X S = 0 (or equivalently XS = 0), then the solutions are optimal for their respective problems. Note that this is the strongest form of strong duality; optimal solutions to both problems exist and they have equal objective value. This isn t necessarily the case. For example, it can fail even for Second Order Conic Programming an example is left to the homework. We will consider some SDP examples. Example 1: Maximize y 1 subject to: y y y1 1 These constraints hold iff 0, iff (y 1 y 1 0 and y 1 y 2 1). Note, however, that 2 the optimal value is 0 but it is not attained; although y T = ɛ, 1 is feasible for every ɛ > 0, ɛ y 1 = 0 is not feasible. The dual problem for this example is:. 0 1 min{ X : x 11 = 1, x 22 = 0, X 0}. The constraints force x 12 = x 21 = 0, so the only feasible solution is X = with objective value 0. Hence, the objective values are equal but the primal problem has no solution that attains the optimal objective value. Example 2: minimize 4 X

5 subject to: 1 X = 1, X = 0, X 0. The second constraint forces the second row and column of X to be 0. The first constraint then forces x 33 = 1. Therefore, X = e 3 e T 3 is optimal with objective value 1. The Dual Problem, however, becomes to maximize y 1 subject to: 1 y 1 + y 2 or, equivalently, 0 y y 1 y 2 0 y 1 0. This implies that y 1 = 0, y 2 0, leading to optimal objective value 0. Hence, the duality gap the difference in optimal objective values is 1. Exercise: check that the duality gap goes away if we either change b 2 to ɛ > 0 (the primal optimal value jumps to 0) or if we change c 11 to ɛ > 0 (the dual optimal value jumps to 1). Think of what happens if we make both of these changes simultaneously. 5