Semidefinite programming lifts and sparse sums-of-squares

Transcription

1 1/15 Semidefinite programming lifts and sparse sums-of-squares Hamza Fawzi (MIT, LIDS) Joint work with James Saunderson (UW) and Pablo Parrilo (MIT) Cornell ORIE, October 2015

2 Linear optimization 2/15 Central question in optimization is to optimize a linear function l on a convex set C: min l(x). x C Need good description of C to solve optimization problem efficiently. Interested in linear programming and semidefinite programming descriptions.

3 3/15 Lifts in optimization Lifts (a.k.a. extended formulations): introducing additional variables can simplify an optimization problem dramatically

4 Lifts in optimization 3/15 Lifts (a.k.a. extended formulations): introducing additional variables can simplify an optimization problem dramatically Example l 1 ball P = {x R n : x 1 1} = {x R n : a T x 1 a { 1, 1} n } P has 2 n facets, yet we have an efficient description using 2n linear inequalities: { P = x R n : y R n s.t. y i x i y i and n i=1 } y i = 1.

5 Lifts in optimization: Permutahedron 4/15 Permutahedron } P = conv {(σ(1),..., σ(n)) : σ S n R n. P has n! vertices and Ω(2 n ) facets. A simple formulation using n 2 inequalities Source: Wikipedia Permutahedron Then where DS n = conv(permutation matrices) = doubly-stochastic matrices. and u = (1, 2,..., n). P = π(ds n ) π : R n n R n M Mu.

6 5/15 Formal definition of lift Formal definition of LP lift Polytope P has a LP lift of size d if P = π(r d + L) L R d + where π : R d R n linear map L affine subspace of R d π LP extension complexity of P is smallest d such that P has a LP lift of size d. P

7 5/15 Formal definition of lift Formal definition of LP lift Polytope P has a LP lift of size d if P = π(r d + L) L R d + where π : R d R n linear map L affine subspace of R d π LP extension complexity of P is smallest d such that P has a LP lift of size d. P Examples of LP lifts Regular N-gon in R 2 has LP lift of size O(log N) (Ben-Tal and Nemirovski 2001). Permutahedron has LP lift of size O(n log n) (Goemans 2009).

8 Semidefinite programming lifts 6/15 Semidefinite programming min L(Y ) subject to Y S d +, Y L where S d + = cone of d d positive semidefinite matrices, L is a linear function and L affine subspace of S d. SDP forms a superset of LP.

9 6/15 Semidefinite programming lifts Semidefinite programming min L(Y ) subject to Y S d +, Y L where S d + = cone of d d positive semidefinite matrices, L is a linear function and L affine subspace of S d. SDP forms a superset of LP. Positive semidefinite lifts P has SDP lift of size d if P = π(s d + L) L S d + where π linear map L affine subspace of S d P π SDP extension complexity of P is smallest d such that P has a SDP lift of size d.

10 LP lifts vs. SDP lifts 7/15 Example The square P = [ 1, 1] 2 : SDP lifts: P has an SDP lift of size 3: [ 1, 1] 2 = (x 1, x 2 ) R 2 : u R 1 x 1 x 2 x 1 1 u 0 x 2 u 1 SDP extension complexity of [ 1, 1] 2 is 3. LP lifts: Can show that LP extension complexity of [ 1, 1] 2 is 4.

11 LP lifts vs. SDP lifts 8/15 Question: How powerful are SDP lifts compared to LP lifts? Theorem (Fawzi-Saunderson-Parrilo 2015) There is a family of polytopes P d R 2d such that ( ) LP extension complexity of P d d Ω +. SDP extension complexity of P d log d Only example known so far of gap between SDP and LP extension complexity. Polytopes P d are highly symmetric and well-studied (trigonometric cyclic polytopes). Proof idea relies on finding sparse sum-of-squares certificates for facet inequalities.

12 Constructing lifts using sum-of-squares 9/15 P R n polytope, X = extreme points of P Facet of P is an affine function l such that l(x) 0 x X. l(x) = b a T x Sum-of-squares certificate for l(x): l(x) = j h j (x) 2 P for some functions h i : X R SDP lifts sum-of-squares: If we can find small sum-of-squares certificates for each facet l of P then we get a small SDP lift.

13 10/15 Constructing lifts using sum-of-squares P R n polytope, X = extreme points of P. F(X, R) = space of real-valued functions on X. Theorem (Lasserre 2010, Gouveia et al. 2011) Assume there is a subspace V of F(X, R) such that any facet-defining inequality l(x) 0 for P can be certified using sum-of-squares in V, i.e., there exist h 1,..., h J V : l(x) = J h j (x) 2 x X. j=1 Then conv(x) has an (explicit) SDP lift of size dim V.

14 10/15 Constructing lifts using sum-of-squares P R n polytope, X = extreme points of P. F(X, R) = space of real-valued functions on X. Theorem (Lasserre 2010, Gouveia et al. 2011) Assume there is a subspace V of F(X, R) such that any facet-defining inequality l(x) 0 for P can be certified using sum-of-squares in V, i.e., there exist h 1,..., h J V : l(x) = J h j (x) 2 x X. j=1 Then conv(x) has an (explicit) SDP lift of size dim V. This theorem reduces the problem of constructing SDP lifts to studying sum-of-squares certificates of facets of P.

15 Lasserre SDP lifts 11/15 Lasserre SDP lift works by degree: certificates of facets l of the form l(x) = j h j (x) 2 where deg h j k. This often results in large SDP lifts (e.g., regular polygons).

16 Lasserre SDP lifts 11/15 Lasserre SDP lift works by degree: certificates of facets l of the form l(x) = j h j (x) 2 where deg h j k. This often results in large SDP lifts (e.g., regular polygons). Idea to construct smaller lifts: Look instead for sparse sum-of-squares certificates, i.e., l(x) = h j (x) 2 j where h j are sparse polynomials.

17 Lasserre SDP lifts 11/15 Lasserre SDP lift works by degree: certificates of facets l of the form l(x) = j h j (x) 2 where deg h j k. This often results in large SDP lifts (e.g., regular polygons). Idea to construct smaller lifts: Look instead for sparse sum-of-squares certificates, i.e., l(x) = h j (x) 2 j where h j are sparse polynomials. Key question: Given a nonnegative function that is sparse, can we write it as a sum-of-squares of sparse functions?

18 Sparse sum-of-squares on finite abelian group 12/15 Setting: Functions on a finite abelian group G Natural basis to measure sparsity of functions, namely Fourier basis of G. G = Z N usual Fourier basis (complex exponentials) G = { 1, 1} n Fourier analysis on the hypercube (square-free monomials).

19 Sparse sum-of-squares on finite abelian group 12/15 Setting: Functions on a finite abelian group G Natural basis to measure sparsity of functions, namely Fourier basis of G. G = Z N usual Fourier basis (complex exponentials) G = { 1, 1} n Fourier analysis on the hypercube (square-free monomials). Main result (informal) Given G finite abelian group and S Ĝ, we give a method to construct T Ĝ such that the following holds: any nonnegative function f : G R + supported on S has a sum-of-squares certificate supported on T, i.e., f (x) = j h j(x) 2 where support(h j ) T. Method involves constructing nice chordal covers of the Cayley graph Cay(Ĝ, S).

20 Sparse sum-of-squares on finite abelian group 12/15 Setting: Functions on a finite abelian group G Natural basis to measure sparsity of functions, namely Fourier basis of G. G = Z N usual Fourier basis (complex exponentials) G = { 1, 1} n Fourier analysis on the hypercube (square-free monomials). Main result (informal) Given G finite abelian group and S Ĝ, we give a method to construct T Ĝ such that the following holds: any nonnegative function f : G R + supported on S has a sum-of-squares certificate supported on T, i.e., f (x) = j h j(x) 2 where support(h j ) T. Method involves constructing nice chordal covers of the Cayley graph Cay(Ĝ, S). Consequence for lifts: Allows us to construct SDP lifts of moment polytope M(G, S) of size T.

21 13/15 Application 1: Degree d polynomials on Z N TC N,2d = conv{ (cos ( 2πx N ), sin ( 2πx N ),..., cos ( 2πdx N Trigonometric cyclic polytope (corresponds to G = Z N and S = { d,..., d}). ) (, sin 2πdx )) N : } x {0, 1, 2..., N 1} R 2d

22 13/15 Application 1: Degree d polynomials on Z N TC N,2d = conv{ (cos ( 2πx N ), sin ( 2πx N ),..., cos ( 2πdx N Trigonometric cyclic polytope (corresponds to G = Z N and S = { d,..., d}). ) (, sin 2πdx )) N : } x {0, 1, 2..., N 1} R 2d Using main result (with good choice of chordal cover): If d divides N then TC N,2d has a PSD lift of size 3d log 2 (N/d). Case N = d 2 gives gap between SDP and LP lifts SDP lift of size O(d log(d)) LP lift must have size Ω(d 2 ) (lower bound due to Fiorini et al. for d-neighborly polytopes)

23 14/15 Consequence 2: Quadratic polynomials on { 1, 1} n Conjecture (Laurent 2003): If f (x) = a 0 + i<j a ij x i x j non-negative x { 1, 1} n then f is a sum of squares of polynomials of degree at most n/2. Laurent (2003): degree at least n/2 necessary Blekherman, Gouveia, Pfeiffer (2014): true if allow multipliers

24 14/15 Consequence 2: Quadratic polynomials on { 1, 1} n Conjecture (Laurent 2003): If f (x) = a 0 + i<j a ij x i x j non-negative x { 1, 1} n then f is a sum of squares of polynomials of degree at most n/2. Laurent (2003): degree at least n/2 necessary Blekherman, Gouveia, Pfeiffer (2014): true if allow multipliers In our language: Group: G = { 1, 1} n = Z n 2 Characters: χ S (x) = i S x i (square-free polynomials) non-negative functions with support S = {S : S {0, 2}} Good choices in main result prove Laurent s conjecture

25 15/15 Conclusion Summary Used sparse sums-of-squares to construct lifts that are smaller than the those from the Lasserre construction. Allowed us to give the first example of a polytope with a gap between SDP lifts and LP lifts. Allowed us to prove conjecture of Laurent. Questions: All the lifts we produce respect the symmetry of the polytope P (they are equivariant). Does breaking symmetry help in reducing the size of lifts? (for LP lifts it does). Lower bounds? For more information: preprint arxiv:

26 15/15 Conclusion Summary Used sparse sums-of-squares to construct lifts that are smaller than the those from the Lasserre construction. Allowed us to give the first example of a polytope with a gap between SDP lifts and LP lifts. Allowed us to prove conjecture of Laurent. Questions: All the lifts we produce respect the symmetry of the polytope P (they are equivariant). Does breaking symmetry help in reducing the size of lifts? (for LP lifts it does). Lower bounds? For more information: preprint arxiv: Thank you!