Approximation & Complexity

Transcription

1 Summer school on semidefinite optimization Approximation & Complexity David Steurer Cornell University Part 1 September 6, 2012

2 Overview Part 1 Unique Games Conjecture & Basic SDP Part 2 SDP Hierarchies: Algorithms Part 3 SDP Hierarchies: Limits

3 Constraint Satisfaction Problems variables,, over finite alphabet Σ list of predicates/constraints... local: only depend on variables only predicates of certain types Goal: satisfy as many predicates as possible

4 Constraint Satisfaction Problems MAX 3SAT variables,, over finite alphabet Σ = {true, false} list of predicates/constraints... = = 9 7 Goal: satisfy as many predicates as possible

5 Constraint Satisfaction Problems MAX CUT variables,, over finite alphabet Σ = list of predicates/constraints... = + = = + = Goal: satisfy as many predicates as possible

6 Constraint Satisfaction Problems UNIQUE GAMES(k) variables,, over finite alphabet Σ = list of predicates/constraints... = + = 4 = + = 9 value of one variable uniquely determines value of other variable Goal: satisfy as many predicates as possible

7 Optimization & Complexity inherent difficulty, required computational resources Goal: understand complexity of optimization problems upper bounds lower bounds What are good algorithms? What are hard instances?

8 Optimization & Complexity Goal: understand complexity of optimization problems require prohibitive resources (assuming P NP) 1970s Most discrete optimization problems are NP-hard [Cook, Karp, Levin] (including MAX 3SAT, MAX CUT, and UNIQUE GAMES) So we can t hope to prove anything and have to resort to heuristics? No! Do not (blindly) trust impossibility results!

9 Optimization is not all or nothing! What about approximate solutions? (Many classical algorithms for convex optimization are fundamentally approximation algorithms) Goal understand trade-off between complexity and approximation

10 Approximation Goal understand trade-off between complexity and approximation How to measure approximation? α-approximating easy to state, but sometimes too coarse ALG OPT (c,s)-approximating finest measure if OPT, then ALG

11 Approximation Goal understand trade-off between complexity and approximation poly-time approximation algorithms: non-trivial approximations for many problems, e.g., approx for MAX CUT [Goemans-Williamson] NP-hardness of approximation as hard as solving it exactly! for many problems, some approximation is NP-hard e.g., approx for MAX CUT [PCP Theorem] For very few problems, upper and lower bounds match!

12 Complexity vs Approximation Trade-off MAX 3SAT complexity Ω [,(astad, Moshkovitz-Raz 8] no guarantee 7/8 exact approximation guarantee

13 Complexity vs Approximation Trade-off Most other problems complexity Ω? no guarantee What algorithms are we missing? exact approximation guarantee What hard instances do we not know of?

14 Unique Games Conjecture (UGC) For every >, there exists k, [Khot ] constraints: = mod k, -approximation for UNIQUE GAMES(k) is NP-hard Implications of UGC [Khot-Regev, Khot-Kindler-Mossel-O Donnell, Mossel-O Donnell-Oleszkiewicz, Raghavendra 8] For every CSP, the Basic SDP relaxation has optimal integrality gap ( higher-degree sum-of-squares relaxation have same gap) Is the conjecture true?

15 Is the conjecture true? subexponential-time algorithm [Arora-Barak-S., Barak-Raghavendra-S. ], -approximation for UG in time exp 1/3 contrast: all known hardness results for CSPs imply Ω -hardness part of framework for rounding SDP hierarchies lower bounds for certain SDP hierarchies subexp.-time essentially optimal within the rounding framework hard instances based on new kind graphs (with extremal spectral properties) sum-of-squares relaxations [Barak-Gopalan-Håstad- Meka-Raghavendra-S. ] [Barak-Brandão-Harrow- Kelner-S.-Zhou ] all known instances of UG are solved in -degree sos relaxation (including instances that are hard for other SDP hierarchies)

16 Generic Approximation Algorithm for CSPs [Raghavendra-S. ] For any CSP X, OPT vs SDP approximation for X = integrality gap of Basic SDP for X ALG vs OPT based on rounding optimal solutions to Basic SDP relaxation new perspective on previous rounding algorithms, like GW no explicit approximation guarantee polynomial-time but huge constants (depending on desired accuracy)

17 Basic SDP Relaxation for Constraint Satisfaction Problems variables,, over finite alphabet Σ list of predicates/constraints first two moments are consistent and positive semidefinite local distributions... = = Goal: maximize expected number of satisfied predicates

18 Approximating CSPs using Folding CSP Instance I approximation algorithm approximate solution for I Folding identification of variables Unfolding of the assignment preserves value of assignment CSP Instance I Brute Force enumerate all assignments optimal solution for I Efficient whenever folding leaves only O distinct variables Challenge: ensure I has a good solution

19 Approximating CSPs using Folding CSP Instance I approximation algorithm approximate solution for I Folding identification of variables Unfolding of the assignment preserves value of assignment CSP Instance I Brute Force enumerate all assignments optimal solution for I Theorem can fold every CSP instance efficiently to y / variables sdp I sdp I optimal rounding scheme

20 How to fold using SDP solutions CSP Instance I Folding guided by SDP solution CSP Instance I optimal solution for Basic-SDP(I) solve SDP Dimension Reduction Discretize Folding identify variables with same vectors solution for SDP(I) average violation < R Project on random -dimensional subspace solution for SDP(I) average violation < R Move vectors to closest point on -net (size < / )

21 How to fold using SDP solutions CSP Instance I Folding guided by SDP solution CSP Instance I found solution for SDP(I ) with value sdp I But: some constraints violated, on average by Robustness property of Basic SDP relaxation can repair violations at proportional cost for objective value sdp I sdp I 4

22 Summer school on semidefinite optimization Approximation & Complexity David Steurer Cornell University Part 2 September 7, 2012

23 Overview Part 1 Unique Games Conjecture & Basic SDP Part 2 SDP Hierarchies: Algorithms Part 3 SDP Hierarchies: Limits

24 Subexponential Algorithm for Unique Games UG in time exp 1Τ 3 via level- 1Τ 3 SDP relaxation General framework for rounding SDP hierarchies (not restricted to Unique Games) [Barak-Raghavendra-S., Guruswami-Sinop ] Potentially applies to wide range of graph problems Examples: MAX CUT, SPARSEST CUT, COLORING, MAX 2-CSP Some more successes (polynomial time algorithms) Approximation scheme for general MAX 2-CSP on constraint graphs with significant eigenvalues [Barak-Raghavendra-S. ] Better 3-COLORING approximation for some graph families [Arora-Ge ] Better approximation for MAX BISECTION (general graphs) [Raghavendra-Tan ] [Austrin-Benabbas-Georgiou ]

25 Subexponential Algorithm for Unique Games UG in time exp 1Τ 3 via level- 1Τ 3 SDP relaxation General framework for rounding SDP hierarchies (not restricted to Unique Games) [Barak-Raghavendra-S., Guruswami-Sinop ] Potentially applies to wide range of graph problems Examples: MAX CUT, SPARSEST CUT, COLORING, MAX 2-CSP Key concept: global correlation

26 Interlude: Pairwise Correlation Two jointly distributed random variables and Correlation measures dependence between and Does the distribution of change if we condition? Examples: (Statistical) distance between {, } and { }{ } Covariance (if and are real-valued) Mutual Information I, = entropy lost due to conditioning

27 Sampling Rounding problem Given random variables,, over Z Pr = for typical constraint = degree-l moments of a distribution over assignments with expected value Sample UG instance + level-l SDP solution with value (l = 1/3 ) distribution over assignments with expected value similar (?) More convenient to think about actual distributions instead of SDP solutions But: proof should only use linear equalities satisfied by these moments and certain linear inequalities, namely non-negativity of squares (Can formalize this restriction as proof system)

28 Sampling by conditioning Pick an index Sample assignment for index from its marginal distribution Condition distribution on this assignment, = If we condition times, we correctly sample the underlying distribution Issue: after conditioning step, know only degree l moments (instead of degree l) Hope: need to condition only a small number of times; then do something else How can conditioning help?

29 How can conditioning help? Allows us to assume: distribution has low global correlation, I, l typical pair of variables almost independent Claim: general cases reduces to case of low global correlation Proof: Idea: significant global correlation conditioning decreases entropy Potential function Φ = Can always find index such that for Φ Φ =,,, Potential can decrease l/ times by more than /l

30 How can conditioning help? Allows us to assume: distribution has low global correlation, I, l typical pair of variables almost pairwise independent How can low global correlation help?

31 How can low global correlation help?, I, l For some problems, this condition alone gives improvement over BASIC SDP Example: MAX BISECTION [Raghavendra-Tan, Austrin-Benabbas-Georgiou ] hyperplane rounding gives near-bisection if global correlation is low

32 How can low global correlation help?, I, l For Unique Games random variables,, over Z Pr = for typical constraint = Extreme cases with low global correlation l equal-sized components 1) no entropy: all variables are fixed 2) many small independent components: all variables have uniform marginal distribution & partition: I, =... inter-component constraint cannot be typical fraction of constraints are inter-component

33 How can low global correlation help?, I, l For Unique Games random variables,, over Z Pr = for typical constraint = Only Extreme cases with low global correlation l equal-sized components 1) no entropy: all variables are fixed 2) many small independent components: all variables have uniform marginal distribution & partition: I, =... Show: no other cases are possible! (informal) inter-component constraint cannot be typical fraction of constraints are inter-component

34 How Idea: can round low components global correlation independently help? & recurse, Ion them, l For How Unique many Games edges ignored in total? (between different components) random We chose variables l = for,, over Z Pr each level = of recursion for typical decrease constraint component = size by factor at most / levels of recursion Only total fraction of ignored edges / Extreme cases with low global correlation -time algorithm for UG 1) no entropy: all variables are fixed l equal-sized components 2) many small independent components: all variables have uniform marginal distribution & partition:... easy to sample

35 How can low global correlation help?, I, l For Unique Games random variables,, over Z Pr = for typical constraint = Only Extreme cases with low global correlation l equal-sized components 1) no entropy: all variables are fixed 2) many small independent components: all variables have uniform marginal distribution & partition:...

36 Suppose: random variables,, over Z with uniform marginals Pr = for typical constraint = global correlation / Then:. & all constraints touching stay inside of except for an / fraction (in constraint graph, S has low expansion) Proof: Define Corr, = max Pr = Correlation Propagation For random walk of length in constraint graph Corr, Corr, Pr 1 = Pr = proof uses non-negativity of squares (sum-of-squares proof) works also for SDP hierarchy

37 Suppose: random variables,, over Z with uniform marginals Pr = for typical constraint = global correlation / Then:. & all constraints touching stay inside of except for an / fraction (in constraint graph, S has low expansion) Proof: Define Corr, = max Pr = = log For random walk of length in constraint graph Correlation Propagation Corr, On the other hand, Corr, / / for typical j random walk from doesn t mix in -steps (actually far from mixing) exist small set around with low expansion low global correlation

38 Suppose: random variables,, over Z with uniform marginals Pr = for typical constraint = global correlation / /l Then: constraint graph has l eigenvalues Proof: a graph has l eigenvalues vectors,, v n (local: typical edge), (global: typical pair),, /l = For graphs with < l such eigenvalues, algorithm runs in time n l Thanks!

39 Summer school on semidefinite optimization Approximation & Complexity David Steurer Cornell University Part 3 September 7, 2012

40 Overview Part 1 Unique Games Conjecture & Basic SDP Part 2 SDP Hierarchies: Algorithms Part 3 SDP Hierarchies: Limits

41 Approximation limits of s.o.s. methods random assignment has value ½ in expectation predicates ഥ [Grigoriev, Schoenebeck 8] For a random instance I of MAX 3XOR, with high probability (1) value of I is at most / +. (2) value of degree-. s.o.s. relaxation for I is at least.99 Corresponding NP-hardness result is known! Why is this result interesting? # predicates # variables independent of P vs NP question suggests random instances are hard evidence that NP-hard problem take exp. time

42 Approximation limits of s.o.s. methods In terms of polynomials: # edges # vertices random -uniform hypergraph, random sign vector ± degree- polynomial = σ Then, w.h.p., Chernoff bound over (1). over ± (2) all s.o.s. certificate for.99 over ± have degree Ω no degree- s.o.s. refutation of the system = =

43 Interlude: Bounded-width Gaussian Elimination system of polynomials over ± system of affine linear forms over = = + + = = width- Gaussian refutation derivation of = by adding equations of width # variables in equation

44 Approximation limits of s.o.s. methods Part 1 random 3-uniform signed hypergraph, corresponding system has elimination width Ω Part 2 For systems we consider, width- Gaussian refutation degree- Nullstellensatz refutation degree- Positivstellensatz refutation

45 Want to show: Random hypergraph system no width-ω Gaussian refutation bipartite graph vertex sets with S < / Ω unique neighbors is product of edge terms has width Γ unique every refutation contains term product of / edges terms (edge) terms variables

46 Want to show: No width- Gaussian refutation no degree- Positivstellensatz refutation How would degree-d s.o.s. refutation look like? degree- multipliers + S. O. S. = To rule out refutation: over ± How to construct M? Gaussian elimination exhibit linear form on polynomials over ± = S. O. S = S. O. S, degree- Q

47 Want to show: No width- Gaussian refutation no degree- Positivstellensatz refutation Let E be set of such that = derived by width- elimination Relation: ~ if = over ± for some E Claim: equivalence relation on degree- terms symmetry uses = transitivity uses width > Define: = 1 if ~ if ~ 0 otherwise

48 Want to show: No width- Gaussian refutation no degree- Positivstellensatz refutation Let E be set of such that = derived by width- elimination Relation: ~ if = over ± for some E = We wanted: 1 if ~ if ~ 0 otherwise = S. O. S S. O. S? =, degree- Q

49 Want to show: No width- Gaussian refutation no degree- Positivstellensatz refutation Let E be set of such that = derived by width- elimination Relation: ~ if = over ± for some E 1 if ~ = if ~ 0 otherwise S. O. S pair up equivalence classes orthogonal unit vectors,, Check: =,