Lower bounds on the size of semidefinite relaxations. David Steurer Cornell

Transcription

1 Lower bounds on the size of semidefinite relaxations David Steurer Cornell James R. Lee Washington Prasad Raghavendra Berkeley Institute for Advanced Study, November 2015

2 overview of results unconditional computational lower bounds for classical combinatorial optimization problems o examples: MAX CUT, TRAVELING SALESMAN in restricted but powerful model of computation o generalizes best known algorithms o all possible linear and semidefinite relaxations o first super-polynomial lower bound in this model general program goes back to [Yannakakis 88] for refuting flawed P=NP proofs connection to optimization / convex geometry settle open question about semidefinite lifts of polytopes and positive semidefinite rank [see survey Fazwi, Gouveia, Parrilo, Robinson, Thomas]

3 overview of results proof strategy for lower bounds optimal approximation algorithm in this model achieves best possible approximation guarantees among all poly-time algorithms in this model wide-range of problems: concrete algorithm: every constraint satisfaction problem sum-of-squares (aka Lasserre) hierarchy [Shor 87, Parrilo 00, Lasserre 00] derive lower bounds for general model from known counterexamples (integrality gaps) for sum-of-squares algorithm [Grigoriev, Schoenebeck, Tulsiani, Barak-Chan-Kothari]

4 mathematical programming relaxations: powerful general approach for approximating NP-hard optimization problems three flavors: linear (LP) (A)TSP spectral edge expanders semidefinite (SDP) MAX CUT, CSP, SPARSEST CUT intriguing connection to hardness reductions (e.g., Unique Games Conjecture) plausibly optimal polynomial-time algorithms

5 mathematical programming relaxations: powerful general approach for approximating NP-hard optimization problems Yannakakis s model motivated by flawed P=NP proofs [Yannakakis 88] formalizes intuitive notion of LP relaxations for problem enough structure for unconditional lower bounds (indep. of P vs. NP) Fiorini-Massar-Pokutta-Tiwary-de Wolf 12, Braun-Pokutta-S., Braverman-Moitra, Chan-Lee-Raghavendra-S., Rothvoß extends to SDP relaxations (but LP lower bound techniques break down) [Fiorini-Massar-Pokutta-Tiwary-de Wolf, Gouveia-Parrilo-Thomas] testing computational complexity conjectures approximation / PCP: UNIQUE GAMES, sliding scale conjecture average-case: RANDOM 3 SAT, PLANTED CLIQUE

6 LP formulations of MAX CUT find bipartition in given n-vertex graph G to cut as many edges as possible maximize f G x = σ ij E G x i = 1 x i x j 2 /4 over x 1,1 n (hypercube) x i = 1 equivalently: maximize σ ij E G (1 X ij )/2 over cut polytope CUT n = convex hull of xx x 1,1 n #facets is exponential no small direct LP formulation

7 LP formulations of MAX CUT find bipartition in given n-vertex graph G to cut as many edges as possible maximize f G x = σ ij E G x i = 1 x i x j 2 /4 over x 1,1 n (hypercube) x i = 1 equivalently: maximize σ ij E G (1 X ij )/2 over cut polytope CUT n = convex hull of xx x 1,1 n #facets is exponential no small direct LP formulation general size-n d LP formulation of MAX CUT [Yannakakis 88] polytope P R nd defined by n d linear inequalities that projects to CUT n often exponential savings: l 1 -norm unit ball, Held-Karp TSP relaxation, LP/SDP hierarchies size lower bounds for LP formulations of MAX CUT? (implied by NP P/poly)

8 example: poly-size LP formulation for l 1 -norm ball l 1 -unit ball σ i x i 1 x R n project on x variables y x y σ i y i 1 x, y R n 2 n linear inequalities 2n + 1 linear inequalities exponential savings general size-n d LP formulation of MAX CUT [Yannakakis 88] polytope P R nd defined by n d linear inequalities that projects to CUT n often exponential savings: l 1 -norm unit ball, Held-Karp TSP relaxation, LP/SDP hierarchies size lower bounds for LP formulations of MAX CUT? (implied by NP P/poly)

9 general size-n d LP formulation of MAX CUT polytope P R nd defined by n d linear inequalities that projects to CUT n size lower bounds for LP formulations of MAX CUT exponential lower bound: d Ω(n) [Fiorini-Massar-Pokutta-Tiwary-de Wolf 12] approx. ratio > ½ requires superpolynomial size [Chan-Lee-Raghavendra-S. 13] but: best known MAX CUT algorithms based on semidefinite programming

10 SDP general size-n d LP formulation of MAX CUT polytope P R nd defined by n d linear inequalities that projects to CUT n spectrahedron P R nd n d defined by intersecting some affine linear subspace with psd cone SDP size lower bounds for LP formulations of MAX CUT? (artistic freedom) [Lee-Raghavendra-S. 15] exponential lower bound: d Ω n 0.1 approx. ratio > 0.99 requires super polynomial size (match NP-hardness for CSPs) best approx. ratio by n d -size SDP no better than O(d)-deg. sum-of-squares sum-of-squares is optimal SDP approximation algorithm for CSPs

11 recall MAX CUT: maximize f G x = σ ij E G x i x j 2 /4 over x 1,1 n upper bound certificates algorithm with approx. guarantee must certify upper bounds on objective function f G approx. ratio α algorithm certifies f G c for some c OPT G /α can characterize LP/SDP algorithms by their certificates certificates of deg-d sum-of-squares algorithm (n d -size SDP example) certify f 0 for function f: ±1 n R iff f = σ i g 2 i with i. deg g i d captures best known algorithms for wide range of problems equal as f ns on hypercube deg-1 sum-of-squares captures Goemans-Williamson MAX CUT approx. for every graph G, OPT G f G = σ i g 2 i with deg g i 1

12 recall MAX CUT: maximize f G x = σ ij E G x i x j 2 /4 over x 1,1 n certificates of deg-d sum-of-squares algorithm (n d -size SDP example) certify f 0 for function f: ±1 n R iff f = σ i g i 2 with i. deg g i d equal as f ns on hypercube captures best known algorithms for wide range of problems deg-1 sum-of-squares captures Goemans-Williamson MAX CUT approx. for every graph G, OPT G f G = σ i g i 2 with deg g i 1

13 recall MAX CUT: maximize f G x = σ ij E G x i x j 2 /4 over x 1,1 n connection to Unique Games Conjecture best candidate algorithm to refute UGC: deg- O 1 sum of squares enough to show: d. G. OPT G f G = σ i g i 2 with deg g i d does larger degree help? yes: if f 0, then f = g 2 for some function g with deg g n (but 2 n -size SDP) (tight: ½ σ i x i 2 ¼ 0 has no deg-o(n) s.o.s. certificate) certificates of deg-d sum-of-squares algorithm (n d -size SDP example) certify f 0 for function f: ±1 n R iff f = σ i g i 2 with i. deg g i d equal as f ns on hypercube captures best known algorithms for wide range of problems deg-1 sum-of-squares captures Goemans-Williamson MAX CUT approx. for every graph G, OPT G f G = σ i g i 2 with deg g i 1

14 where are the vectors? suppose: no deg-d sos certificate for f G c separating hyperplane D: ±1 n R σ x D x g x 2 0 whenever deg g d σ x D(x) 1 = 1 σ x D x f G x > c f G c D σ i g i 2 deg g i d M = σ x D x xx is usual SDP solution (in particular M 0 and M ii = 1) R ±1 n vectors v i with M ij = v i, v j certificates of deg-d sum-of-squares algorithm (n d -size SDP example) certify f 0 for function f: ±1 n R iff f = σ i g i 2 with i. deg g i d equal as f ns on hypercube captures best known algorithms for wide range of problems deg-1 sum-of-squares captures Goemans-Williamson MAX CUT approx. for every graph G, OPT G f G = σ i g i 2 with deg g i 1

15 where are the vectors? suppose: no deg-d sos certificate for f G c separating hyperplane D: ±1 n R σ x D x g x 2 0 whenever deg g d σ x D(x) 1 = 1 σ x D x f G x > c M = σ x D x xx is usual SDP solution (in particular M 0 and M ii = 1) vectors v i with M ij = v i, v j f G c certificates of deg-d sum-of-squares algorithm (n d -size SDP example) certify f 0 for function f: ±1 n R iff f = σ i g i 2 with i. deg g i d equal as f ns on hypercube captures best known algorithms for wide range of problems deg-1 sum-of-squares captures Goemans-Williamson MAX CUT approx. for every graph G, OPT G f G = σ i g i 2 with deg g i 1 D σ i g i 2 deg g i d D behaves like probability distribution over MAX CUT solutions with expected value > c R ±1 n pseudo-distribution: useful way to think about LP/SDP relaxations in general

16 certificates of deg-d sum-of-squares SDP algorithm certify f 0 for function f: ±1 n R iff f = σ i g i 2 with i. deg g i d certificates of general n d -size SDP algorithm characterized by psd-matrix valued function Q: ±1 n R nd n d certify f 0 iff P 0. x ±1 n. f x = Tr PQ(x) = P Q x example: deg-d sum-of-squares SDP algorithm, Q x = x d x d 2 F general SDP Q captured by deg-d sum-of-squares if deg Q x d

17 certificates of deg-d sum-of-squares SDP algorithm certify f 0 for function f: ±1 n R iff f = σ i g i 2 with i. deg g i d certificates of general n d -size SDP algorithm characterized by psd-matrix valued function Q: ±1 n R nd n d certify f 0 iff P 0. x ±1 n. f x = Tr PQ(x) = P Q x example: deg-d sum-of-squares SDP algorithm, Q x = x d x d 2 F general SDP Q captured by deg-d sum-of-squares if deg Q x d where does Q come from? general spectrahedron: S = z R nd σ i z i A i B SDP relax. for MAX CUT: cost functions y G and feasible solutions z x S with y G, z x = f G x (obj. value of cut x in graph G) choose Q with Q x = B σ i z x,i A i (slack of constraint at z x S) duality: max z S y G, z c iff P 0. c y G, z = Tr P B σ i z i A i c f G x = Tr P Q x for all x

18 certificates of deg-d sum-of-squares SDP algorithm certify f 0 for function f: ±1 n R iff f = σ i g i 2 with i. deg g i d certificates of general n d -size SDP algorithm characterized by psd-matrix valued function Q: ±1 n R nd n d certify f 0 iff P 0. x ±1 n. f x = Tr PQ(x) = P Q x example: deg-d sum-of-squares SDP algorithm, Q x = x d x d 2 F general SDP Q captured by deg-d sum-of-squares if deg Q x d low-degree can simulate general n d -size SDP algorithm by deg-o d sum-of-squares n d -size SDP algorithm Q. low-deg matrix-valued function F. F, Q F, Q low-deg SDP algorithm Q. deg Q x log n d and Q Q [Lee-Raghavendra-S. 15]

19 certificates general phenomenon of deg-d sum-of-squares SDP algorithm in order certify to f approximate 0 for function an object f: ±1 with n respect R iff f to = a σ family i g 2 i with of tests, i. deg g i d the approximator need not be more complex than the tests certificates of general n d -size SDP algorithm technical challenge characterized by psd-matrix valued function Q: ±1 n R nd n d naïve application allows us to bound deg Q x but need to bound deg Q x certify f 0 iff P 0. x ±1 n 2 in general: deg Q deg Q (at the heart. f of x sum-of-squares = Tr PQ(x) = counterexamples) P Q x example: deg-d sum-of-squares SDP algorithm, Q x = x d x d F general SDP Q captured by deg-d sum-of-squares if deg Q x d low-degree can simulate general n d -size SDP algorithm by deg-o d sum-of-squares n d -size SDP algorithm Q. low-deg matrix-valued function F. F, Q F, Q low-deg SDP algorithm Q. deg Q x log n d and Q Q [Lee-Raghavendra-S. 15]

20 low-degree can simulate general n d -size SDP algorithm by deg-o d sum-of-squares n d -size SDP algorithm Q. low-deg matrix-valued function F. F, Q F, Q low-deg SDP algorithm Q. deg Q x log n d and Q Q approach: learn simplest SDP algorithm Q that satisfies F, Q F, Q measure of simplicity: quantum entropy (classical entropy of eigenvalues of Q x ) closed-form solution: Q x = et F x where t = entropy-defect Q log n d matrix multiplicative weights method! simple square root: Q x = e t F(x)/2 σt 1 k=0 k! t F x Τ 2 k degree deg F t

21 summary can simulate general small SDP alg. by low-degree SDP alg. interpret poly-size SDP algorithm as quantum state with high entropy learn simplest SDP / quantum state via matrix multiplicative weights (maximum entropy) open questions approximation beyond CSP and relatives rule out approximation for TRAVELING SALEMAN by poly-size LP/SDP strong quantitative lower bounds for approximation rule out approximation for MAX CUT by 2 nω 1 -size LP/SDP stronger quantitative lower bounds for SDP rule out 2 n size SDP for (exact) MAX CUT latest news: solved by Raghavendra-Meka-Kothari! Thank you!

22