Introduction to LP and SDP Hierarchies

Size: px

Start display at page:

Download "Introduction to LP and SDP Hierarchies"

Adelia Owens
5 years ago
Views:

1 Introduction to LP and SDP Hierarchies Madhur Tulsiani Princeton University

2 Local Constraints in Approximation Algorithms Linear Programming (LP) or Semidefinite Programming (SDP) based approximation algorithms impose constraints on few variables at a time.

3 Local Constraints in Approximation Algorithms Linear Programming (LP) or Semidefinite Programming (SDP) based approximation algorithms impose constraints on few variables at a time. When can local constraints help in approximating a global property (eg. Vertex Cover, Chromatic Number)?

4 Local Constraints in Approximation Algorithms Linear Programming (LP) or Semidefinite Programming (SDP) based approximation algorithms impose constraints on few variables at a time. When can local constraints help in approximating a global property (eg. Vertex Cover, Chromatic Number)? How does one reason about increasingly larger local constraints?

5 Local Constraints in Approximation Algorithms Linear Programming (LP) or Semidefinite Programming (SDP) based approximation algorithms impose constraints on few variables at a time. When can local constraints help in approximating a global property (eg. Vertex Cover, Chromatic Number)? How does one reason about increasingly larger local constraints? Does approximation get better as constraints get larger?

6 LP/SDP Hierarchies Various hierarchies give increasingly powerful programs at different levels (rounds). Lovász-Schrijver (LS, LS +) Sherali-Adams Lasserre Mixed

7 LP/SDP Hierarchies Various hierarchies give increasingly powerful programs at different levels (rounds). Lovász-Schrijver (LS, LS +) Sherali-Adams Lasserre Mixed. SA (2) SA (1). Mix (2) Mix (1). LS (2) LS (1). Las (2) Las (1). LS (2) + LS (1) +

8 LP/SDP Hierarchies Various hierarchies give increasingly powerful programs at different levels (rounds). Lovász-Schrijver (LS, LS +) Sherali-Adams Lasserre Mixed. SA (2) SA (1). Mix (2) Mix (1). LS (2) LS (1). Las (2) Las (1). LS (2) + LS (1) + Can optimize over r th level in time n O(r). n th level is tight.

9 LP/SDP Hierarchies Powerful computational model capturing most known LP/SDP algorithms within constant number of levels.

10 LP/SDP Hierarchies Powerful computational model capturing most known LP/SDP algorithms within constant number of levels. Lower bounds rule out large and natural class of algorithms.

11 LP/SDP Hierarchies Powerful computational model capturing most known LP/SDP algorithms within constant number of levels. Lower bounds rule out large and natural class of algorithms. Performance measured by considering integrality gap at various levels. Integrality Gap = Optimum of Relaxation Integer Optimum (for maximization)

12 Why bother? - Conditional - All polytime algorithms - Unconditional - Restricted class of algorithms NP-Hardness LP/SDP Hierarchies UG-Hardness

13 What Hierarchies want Example: Maximum Independent Set for graph G = (V, E) minimize u x u subject to x u + x v 1 (u, v) E x u [0, 1] Hope: x 1,..., x n is convex combination of 0/1 solutions.

14 What Hierarchies want Example: Maximum Independent Set for graph G = (V, E) minimize u x u subject to x u + x v 1 (u, v) E x u [0, 1] Hope: x 1,..., x n is convex combination of 0/1 solutions. 1/ = /3 1/3

15 What Hierarchies want Example: Maximum Independent Set for graph G = (V, E) minimize u x u subject to x u + x v 1 (u, v) E x u [0, 1] Hope: x 1,..., x n is marginal of distribution over 0/1 solutions. 1/ = /3 1/3

16 What Hierarchies want Example: Maximum Independent Set for graph G = (V, E) minimize u x u subject to x u + x v 1 (u, v) E x u [0, 1] Hope: x 1,..., x n is marginal of distribution over 0/1 solutions. 1/ = /3 1/3 Hierarchies add variables for conditional/joint probabilities.

17 The Sherali-Adams Hierarchy

18 The Sherali-Adams Hierarchy Start with a 0/1 integer linear program. Hope: Fractional (x 1,..., x n ) = E [(z 1,..., z n )] for integral (z 1,..., z n )

19 The Sherali-Adams Hierarchy Start with a 0/1 integer linear program. Hope: Fractional (x 1,..., x n ) = E [(z 1,..., z n )] for integral (z 1,..., z n ) Add big variables X S for S r (think X S = E [ i S z i] = P [All vars in S are 1])

20 The Sherali-Adams Hierarchy Start with a 0/1 integer linear program. Hope: Fractional (x 1,..., x n ) = E [(z 1,..., z n )] for integral (z 1,..., z n ) Add big variables X S for S r (think X S = E [ i S z i] = P [All vars in S are 1]) Constraints:

21 The Sherali-Adams Hierarchy Start with a 0/1 integer linear program. Hope: Fractional (x 1,..., x n ) = E [(z 1,..., z n )] for integral (z 1,..., z n ) Add big variables X S for S r (think X S = E [ i S z i] = P [All vars in S are 1]) Constraints: a i z i b [( ) ] E a i z i z 5 z 7 (1 z 9 ) i i E [b z 5 z 7 (1 z 9 )]

22 The Sherali-Adams Hierarchy Start with a 0/1 integer linear program. Hope: Fractional (x 1,..., x n ) = E [(z 1,..., z n )] for integral (z 1,..., z n ) Add big variables X S for S r (think X S = E [ i S z i] = P [All vars in S are 1]) Constraints: a i z i b [( ) ] E a i z i z 5 z 7 (1 z 9 ) i i E [b z 5 z 7 (1 z 9 )] a i (X {i,5,7} X {i,5,7,9} ) b (X {5,7} X {5,7,9} ) i LP on n r variables.

23 Sherali-Adams Locally Consistent Distributions

24 Sherali-Adams Locally Consistent Distributions Using 0 z 1 1, 0 z X {1,2} 1 0 X {1} X {1,2} 1 0 X {2} X {1,2} X {1} X {2} + X {1,2} 1

25 Sherali-Adams Locally Consistent Distributions Using 0 z 1 1, 0 z X {1,2} 1 0 X {1} X {1,2} 1 0 X {2} X {1,2} X {1} X {2} + X {1,2} 1 X {1}, X {2}, X {1,2} define a distribution D({1, 2}) over {0, 1} 2.

26 Sherali-Adams Locally Consistent Distributions Using 0 z 1 1, 0 z X {1,2} 1 0 X {1} X {1,2} 1 0 X {2} X {1,2} X {1} X {2} + X {1,2} 1 X {1}, X {2}, X {1,2} define a distribution D({1, 2}) over {0, 1} 2. D({1, 2, 3}) and D({1, 2, 4}) must agree with D({1, 2}).

27 Sherali-Adams Locally Consistent Distributions Using 0 z 1 1, 0 z X {1,2} 1 0 X {1} X {1,2} 1 0 X {2} X {1,2} X {1} X {2} + X {1,2} 1 X {1}, X {2}, X {1,2} define a distribution D({1, 2}) over {0, 1} 2. D({1, 2, 3}) and D({1, 2, 4}) must agree with D({1, 2}). SA (r) LCD (r+k) = LCD (r). If each constraint has at most k vars, = SA (r)

28 The Lasserre Hierarchy

29 The Lasserre Hierarchy Start with a 0/1 integer quadratic program.

30 The Lasserre Hierarchy Start with a 0/1 integer quadratic program. Think big" variables Z S = i S z i.

31 The Lasserre Hierarchy Start with a 0/1 integer quadratic program. Think big" variables Z S = i S z i. Associated psd matrix Y (moment matrix) 2 3 Y S1,S 2 = E ˆZ S1 Z S2 = E 4 Y 5 i S 1 S 2 z i

32 The Lasserre Hierarchy Start with a 0/1 integer quadratic program. Think big" variables Z S = i S z i. Associated psd matrix Y (moment matrix) 2 3 Y S1,S 2 = E ˆZ S1 Z S2 = E 4 Y 5 i S 1 S 2 z i

33 The Lasserre Hierarchy Start with a 0/1 integer quadratic program. Think big" variables Z S = i S z i. Associated psd matrix Y (moment matrix) 2 3 Y S1,S 2 = E ˆZ S1 Z S2 = E 4 Y 5 = P[All vars in S 1 S 2 are 1] i S 1 S 2 z i

34 The Lasserre Hierarchy Start with a 0/1 integer quadratic program. Think big" variables Z S = i S z i. Associated psd matrix Y (moment matrix) 2 3 Y S1,S 2 = E ˆZ S1 Z S2 = E 4 Y 5 = P[All vars in S 1 S 2 are 1] i S 1 S 2 z i (Y 0) + original constraints + consistency constraints.

35 The Lasserre hierarchy (constraints) Y is psd. (i.e. find vectors U S satisfying Y S1,S 2 = U S1, U S2 )

36 The Lasserre hierarchy (constraints) Y is psd. (i.e. find vectors U S satisfying Y S1,S 2 = U S1, U S2 ) Y S1,S 2 only depends on S 1 S 2. (Y S1,S 2 = P[All vars in S 1 S 2 are 1])

37 The Lasserre hierarchy (constraints) Y is psd. (i.e. find vectors U S satisfying Y S1,S 2 = U S1, U S2 ) Y S1,S 2 only depends on S 1 S 2. (Y S1,S 2 = P[All vars in S 1 S 2 are 1]) Original quadratic constraints as inner products. SDP for Independent Set maximize subject to X i V U {i} 2 U{i}, U {j} = 0 US1, U S2 = US3, U S4 (i, j) E S 1 S 2 = S 3 S 4 US1, U S2 [0, 1] S1, S 2

38 The Mixed hierarchy Motivated by [Raghavendra 08]. Used by [CS 08] for Hypergraph Independent Set. Captures what we actually know how to use about Lasserre solutions.

39 The Mixed hierarchy Motivated by [Raghavendra 08]. Used by [CS 08] for Hypergraph Independent Set. Captures what we actually know how to use about Lasserre solutions. Level r has Variables X S for S r and all Sherali-Adams constraints. Vectors U 0, U 1,..., U n satisfying U i, U j = X {i,j}, U 0, U i = X {i} and U 0 = 1.

40 Hands-on: Deriving some constraints

41 The triangle inequality U i U j 2 + U j U k 2 U i U k 2 is equivalent to U i U j, U k U j 0

42 The triangle inequality U i U j 2 + U j U k 2 U i U k 2 is equivalent to U i U j, U k U j 0 Mix (3) = distribution on z i, z j, z k such that E[z i z j ] = U i, U j (and so on).

43 The triangle inequality U i U j 2 + U j U k 2 U i U k 2 is equivalent to U i U j, U k U j 0 Mix (3) = distribution on z i, z j, z k such that E[z i z j ] = U i, U j (and so on). For all integer solutions (z i z j ) (z k z j ) 0.

44 The triangle inequality U i U j 2 + U j U k 2 U i U k 2 is equivalent to U i U j, U k U j 0 Mix (3) = distribution on z i, z j, z k such that E[z i z j ] = U i, U j (and so on). For all integer solutions (z i z j ) (z k z j ) 0. U i U j, U k U j = E [(z i z j ) (z k z j )] 0

45 Clique constraints for Independent Set For every clique B in a graph, adding the constraint X x i 1 makes the independent set LP tight for perfect graphs. i B

46 Clique constraints for Independent Set For every clique B in a graph, adding the constraint X x i 1 i B makes the independent set LP tight for perfect graphs. Too many constraints, but all implied by one level of the mixed hierarchy.

47 Clique constraints for Independent Set For every clique B in a graph, adding the constraint X x i 1 i B makes the independent set LP tight for perfect graphs. Too many constraints, but all implied by one level of the mixed hierarchy. For i, j B, U i, U j = 0. By Pythagoras, X i B fi U 0, fl 2 U i U 0 2 = 1 = X U i i B x 2 i x i 1. Derived by Lovász using the ϑ-function.

48 The Lovász-Schrijver Hierarchy

49 The Lovász-Schrijver Hierarchy Start with a 0/1 integer program and a relaxation P. Define tigher relaxation LS(P).

50 The Lovász-Schrijver Hierarchy Start with a 0/1 integer program and a relaxation P. Define tigher relaxation LS(P). Hope: Fractional (x 1,..., x n ) = E [(z 1,..., z n )] for integral (z 1,..., z n )

51 The Lovász-Schrijver Hierarchy Start with a 0/1 integer program and a relaxation P. Define tigher relaxation LS(P). Hope: Fractional (x 1,..., x n ) = E [(z 1,..., z n )] for integral (z 1,..., z n ) Restriction: x = (x 1,..., x n ) LS(P) if Y satisfying (think Y ij = E [z i z j ] = P [z i z j ]) Y = Y T Y ii = x i i Y i x i P, Y 0 x Y i 1 x i P i

52 The Lovász-Schrijver Hierarchy Start with a 0/1 integer program and a relaxation P. Define tigher relaxation LS(P). Hope: Fractional (x 1,..., x n ) = E [(z 1,..., z n )] for integral (z 1,..., z n ) Restriction: x = (x 1,..., x n ) LS(P) if Y satisfying (think Y ij = E [z i z j ] = P [z i z j ]) Y = Y T Y ii = x i i Y i x i P, Y 0 x Y i 1 x i P i Above is an LP (SDP) in n 2 + n variables.

53 Lovász-Schrijver in action r th level optimizes over distributions conditioned on r variables. 1/3 1/3 1/3 1/3 1/3 1/3

54 Lovász-Schrijver in action r th level optimizes over distributions conditioned on r variables. 1/3 1/3 1/3 1/3 1/3 1/3

55 Lovász-Schrijver in action r th level optimizes over distributions conditioned on r variables. 1/3 1/3 1/3 2/3 1/3 1/2 1/3 1/3 0 1/2 1/2 1/ /2 1/

56 Lovász-Schrijver in action r th level optimizes over distributions conditioned on r variables. 1/3 1/3 1/3 2/3 1/3 1/2 1/3 1/3 0 1/2 1/2 1/ /2 1/

57 Lovász-Schrijver in action r th level optimizes over distributions conditioned on r variables. 1/3 1/3 1/3 2/3 1/3 1/2 1/3 1/3 0 1/2 1/2 1/ /2 1/ /

58 Lovász-Schrijver in action r th level optimizes over distributions conditioned on r variables. 1/3 1/3 1/3 2/3 1/3 1/2 1/3 1/3 0 1/2 1/2 1/ /2 1/ /2? 0 1/2 0 1?? 0 0??

59 And if you just woke up...

60 And if you just woke up.... SA (2) SA (1). Mix (2) Mix (1). LS (2) LS (1). Las (2) Las (1). LS (2) + LS (1) +

61 And if you just woke up... SA + U i. Mix (2) Mix (1).. X SA (2) S LS (2) + SA (1) LS. (1) + LS (2) LS (1). Las (2) Las (1) U S Y 0 Y i x i, x Y i 1 x i

62 Integrality Gaps for Expanding CSPs

63 CSP Expansion MAX k-csp: m constraints on k-tuples of (n) boolean variables. Satisfy maximum. e.g. MAX 3-XOR (linear equations mod 2) z 1 + z 2 + z 3 = 0 z 3 + z 4 + z 5 = 1

64 CSP Expansion MAX k-csp: m constraints on k-tuples of (n) boolean variables. Satisfy maximum. e.g. MAX 3-XOR (linear equations mod 2) z 1 + z 2 + z 3 = 0 z 3 + z 4 + z 5 = 1 Expansion: Every set S of constraints involves at least β S variables (for S < αm).

65 CSP Expansion MAX k-csp: m constraints on k-tuples of (n) boolean variables. Satisfy maximum. e.g. MAX 3-XOR (linear equations mod 2) z 1 + z 2 + z 3 = 0 z 3 + z 4 + z 5 = 1 Expansion: Every set S of constraints involves at least β S variables (for S < αm). C 1 z 1.. C m z n

66 CSP Expansion MAX k-csp: m constraints on k-tuples of (n) boolean variables. Satisfy maximum. e.g. MAX 3-XOR (linear equations mod 2) z 1 + z 2 + z 3 = 0 z 3 + z 4 + z 5 = 1 Expansion: Every set S of constraints involves at least β S variables (for S < αm). C 1 z 1.. C m z n

67 CSP Expansion MAX k-csp: m constraints on k-tuples of (n) boolean variables. Satisfy maximum. e.g. MAX 3-XOR (linear equations mod 2) z 1 + z 2 + z 3 = 0 z 3 + z 4 + z 5 = 1 Expansion: Every set S of constraints involves at least β S variables (for S < αm). C 1 z 1.. C m In fact, γ S variables appearing in only one constraint in S. z n

68 CSP Expansion MAX k-csp: m constraints on k-tuples of (n) boolean variables. Satisfy maximum. e.g. MAX 3-XOR (linear equations mod 2) z 1 + z 2 + z 3 = 0 z 3 + z 4 + z 5 = 1 Expansion: Every set S of constraints involves at least β S variables (for S < αm). C 1 z 1.. C m In fact, γ S variables appearing in only one constraint in S. Used extensively in proof complexity e.g. [BW01], [BGHMP03]. For LS + by [AAT04]. z n

69 Local Satisfiability C 1 C 2 C 3 z 1 z 2 z 3 z 4 z 5 z 6 Take γ = 0.9 Can show any three 3-XOR constraints are simultaneously satisfiable.

70 Local Satisfiability C 1 C 2 C 3 z 1 z 2 z 3 z 4 z 5 z 6 Take γ = 0.9 Can show any three 3-XOR constraints are simultaneously satisfiable. E z1...z 6 [C 1 (z 1, z 2, z 3 ) C 2 (z 3, z 4, z 5) C 3 (z 4, z 5, z 6 )]

71 Local Satisfiability C 1 C 2 C 3 z 1 z 2 z 3 z 4 z 5 z 6 Take γ = 0.9 Can show any three 3-XOR constraints are simultaneously satisfiable. E z1...z 6 [C 1 (z 1, z 2, z 3 ) C 2 (z 3, z 4, z 5) C 3 (z 4, z 5, z 6 )] = E z2...z 6 [C 2 (z 3, z 4, z 5) C 3 (z 4, z 5, z 6 ) E z1 [C 1 (z 1, z 2, z 3 )]]

72 Local Satisfiability C 1 C 2 C 3 z 1 z 2 z 3 z 4 z 5 z 6 Take γ = 0.9 Can show any three 3-XOR constraints are simultaneously satisfiable. E z1...z 6 [C 1 (z 1, z 2, z 3 ) C 2 (z 3, z 4, z 5) C 3 (z 4, z 5, z 6 )] = E z2...z 6 [C 2 (z 3, z 4, z 5) C 3 (z 4, z 5, z 6 ) E z1 [C 1 (z 1, z 2, z 3 )]] = E z4,z 5,z 6 [C 3 (z 4, z 5, z 6 ) E z3 [C 2 (z 3, z 4, z 5)] (1/2)]

73 Local Satisfiability C 1 C 2 C 3 z 1 z 2 z 3 z 4 z 5 z 6 Take γ = 0.9 Can show any three 3-XOR constraints are simultaneously satisfiable. E z1...z 6 [C 1 (z 1, z 2, z 3 ) C 2 (z 3, z 4, z 5) C 3 (z 4, z 5, z 6 )] = E z2...z 6 [C 2 (z 3, z 4, z 5) C 3 (z 4, z 5, z 6 ) E z1 [C 1 (z 1, z 2, z 3 )]] = E z4,z 5,z 6 [C 3 (z 4, z 5, z 6 ) E z3 [C 2 (z 3, z 4, z 5)] (1/2)] = 1/8

74 Local Satisfiability C 1 C 2 C 3 z 1 z 2 z 3 z 4 z 5 z 6 Take γ = 0.9 Can show any three 3-XOR constraints are simultaneously satisfiable. Can take γ (k 2) and any αn constraints. Just require E[C(z 1,..., z k )] over any k 2 vars to be constant. E z1...z 6 [C 1 (z 1, z 2, z 3 ) C 2 (z 3, z 4, z 5) C 3 (z 4, z 5, z 6 )] = E z2...z 6 [C 2 (z 3, z 4, z 5) C 3 (z 4, z 5, z 6 ) E z1 [C 1 (z 1, z 2, z 3 )]] = E z4,z 5,z 6 [C 3 (z 4, z 5, z 6 ) E z3 [C 2 (z 3, z 4, z 5)] (1/2)] = 1/8

75 Sherali-Adams LP for CSPs Variables: X (S,α) for S t, partial assignments α {0, 1} S mx X maximize C i (α) X (Ti,α) i=1 α {0,1} T i subject to X (S {i},α 0) + X (S {i},α 1) = X (S,α) i / S X (S,α) 0 X (, ) = 1

76 Sherali-Adams LP for CSPs Variables: X (S,α) for S t, partial assignments α {0, 1} S mx X maximize C i (α) X (Ti,α) i=1 α {0,1} T i subject to X (S {i},α 0) + X (S {i},α 1) = X (S,α) i / S X (S,α) 0 X (, ) = 1 X (S,α) P[Vars in S assigned according to α]

77 Sherali-Adams LP for CSPs Variables: X (S,α) for S t, partial assignments α {0, 1} S mx X maximize C i (α) X (Ti,α) i=1 α {0,1} T i subject to X (S {i},α 0) + X (S {i},α 1) = X (S,α) i / S X (S,α) 0 X (, ) = 1 X (S,α) P[Vars in S assigned according to α] Need distributions D(S) such that D(S 1 ), D(S 2 ) agree on S 1 S 2.

78 Sherali-Adams LP for CSPs Variables: X (S,α) for S t, partial assignments α {0, 1} S mx X maximize C i (α) X (Ti,α) i=1 α {0,1} T i subject to X (S {i},α 0) + X (S {i},α 1) = X (S,α) i / S X (S,α) 0 X (, ) = 1 X (S,α) P[Vars in S assigned according to α] Need distributions D(S) such that D(S 1 ), D(S 2 ) agree on S 1 S 2. Distributions should locally look like" supported on satisfying assignments.

79 Obtaining integrality gaps for CSPs [BGMT 09] C 1 z 1.. C m z n Want to define distribution D(S) for set S of variables.

80 Obtaining integrality gaps for CSPs [BGMT 09] C 1 z 1.. C m z n Want to define distribution D(S) for set S of variables.

81 Obtaining integrality gaps for CSPs [BGMT 09] C 1 z 1.. C m z n Want to define distribution D(S) for set S of variables.

82 Obtaining integrality gaps for CSPs [BGMT 09] C 1 z 1.. C m z n Want to define distribution D(S) for set S of variables. Find set of constraints C such that G C S remains expanding. D(S) = uniform over assignments satisfying C

83 Obtaining integrality gaps for CSPs [BGMT 09] C 1 z 1.. C m z n Want to define distribution D(S) for set S of variables. Find set of constraints C such that G C S remains expanding. D(S) = uniform over assignments satisfying C Remaining constraints independent" of this assignment. Gives optimal integrality gaps for Ω(n) levels in the mixed hierarchy.

84 Vectors for Linear CSPs

85 A new look Lasserre Start with a { 1, 1} quadratic integer program. (z 1,..., z n ) (( 1) z 1,..., ( 1) z n )

86 A new look Lasserre Start with a { 1, 1} quadratic integer program. (z 1,..., z n ) (( 1) z 1,..., ( 1) z n ) Define big variables Z S = i S ( 1)z i.

87 A new look Lasserre Start with a { 1, 1} quadratic integer program. (z 1,..., z n ) (( 1) z 1,..., ( 1) z n ) Define big variables Z S = i S ( 1)z i. Consider the psd matrix Ỹ Ỹ S1,S 2 = E [ ZS1 Z ] S2 = E ( 1) zi i S 1 S 2

88 A new look Lasserre Start with a { 1, 1} quadratic integer program. (z 1,..., z n ) (( 1) z 1,..., ( 1) z n ) Define big variables Z S = i S ( 1)z i. Consider the psd matrix Ỹ Ỹ S1,S 2 = E [ ZS1 Z ] S2 = E ( 1) zi i S 1 S 2 Write program for inner products of vectors W S s.t. Ỹ S1,S 2 = W S1, W S2

89 Gaps for 3-XOR SDP for MAX 3-XOR maximize subject to X 1 + ( 1) b i W{i1,i 2,i 3 }, W C i (z i1 +z i2 +z i3 =b i ) 2 WS1, W S2 = WS3, W S4 S 1 S 2 = S 3 S 4 W S = 1 S, S r

90 Gaps for 3-XOR SDP for MAX 3-XOR maximize subject to X 1 + ( 1) b i W{i1,i 2,i 3 }, W C i (z i1 +z i2 +z i3 =b i ) 2 WS1, W S2 = WS3, W S4 S 1 S 2 = S 3 S 4 W S = 1 S, S r [Schoenebeck 08]: If width 2r resolution does not derive contradiction, then SDP value =1 after r levels of Lasserre.

91 Gaps for 3-XOR SDP for MAX 3-XOR maximize subject to X 1 + ( 1) b i W{i1,i 2,i 3 }, W C i (z i1 +z i2 +z i3 =b i ) 2 WS1, W S2 = WS3, W S4 S 1 S 2 = S 3 S 4 W S = 1 S, S r [Schoenebeck 08]: If width 2r resolution does not derive contradiction, then SDP value =1 after r levels of Lasserre. Expansion guarantees there are no width 2r contradictions.

92 Gaps for 3-XOR SDP for MAX 3-XOR maximize subject to X 1 + ( 1) b i W{i1,i 2,i 3 }, W C i (z i1 +z i2 +z i3 =b i ) 2 WS1, W S2 = WS3, W S4 S 1 S 2 = S 3 S 4 W S = 1 S, S r [Schoenebeck 08]: If width 2r resolution does not derive contradiction, then SDP value =1 after r levels of Lasserre. Expansion guarantees there are no width 2r contradictions. Used by [FO 06], [STT 07] for LS + hierarchy.

93 Schonebeck s construction

94 Schonebeck s construction z 1 + z 2 + z 3 = 1 mod 2 = ( 1) z 1+z 2 = ( 1) z 3 = W {1,2} = W {3}

95 Schonebeck s construction z 1 + z 2 + z 3 = 1 mod 2 = ( 1) z 1+z 2 = ( 1) z 3 = W {1,2} = W {3} Equations of width 2r divide S r into equivalence classes. Choose orthogonal e C for each class C.

96 Schonebeck s construction z 1 + z 2 + z 3 = 1 mod 2 = ( 1) z 1+z 2 = ( 1) z 3 = W {1,2} = W {3} Equations of width 2r divide S r into equivalence classes. Choose orthogonal e C for each class C. No contradictions ensure each S C can be uniquely assigned ±e C. W S2 = e C2 W S3 = e C2 W S1 = e C1

97 Schonebeck s construction z 1 + z 2 + z 3 = 1 mod 2 = ( 1) z 1+z 2 = ( 1) z 3 = W {1,2} = W {3} Equations of width 2r divide S r into equivalence classes. Choose orthogonal e C for each class C. No contradictions ensure each S C can be uniquely assigned ±e C. Relies heavily on constraints being linear equations. W S2 = e C2 W S3 = e C2 W S1 = e C1

98 Reductions

99 Spreading the hardness around (Reductions) [T] If problem A reduces to B, can we say Integrality Gap for A = Integrality Gap for B?

100 Spreading the hardness around (Reductions) [T] If problem A reduces to B, can we say Integrality Gap for A = Integrality Gap for B? Reductions are (often) local algorithms.

101 Spreading the hardness around (Reductions) [T] If problem A reduces to B, can we say Integrality Gap for A = Integrality Gap for B? Reductions are (often) local algorithms. Reduction from integer program A to integer program B. Each variable z i of B is a boolean function of few (say 5) variables z i1,..., z i5 of A.

102 Spreading the hardness around (Reductions) [T] If problem A reduces to B, can we say Integrality Gap for A = Integrality Gap for B? Reductions are (often) local algorithms. Reduction from integer program A to integer program B. Each variable z i of B is a boolean function of few (say 5) variables z i1,..., z i5 of A. To show: If A has good vector solution, so does B.

103 Spreading the hardness around (Reductions) [T] If problem A reduces to B, can we say Integrality Gap for A = Integrality Gap for B? Reductions are (often) local algorithms. Reduction from integer program A to integer program B. Each variable z i of B is a boolean function of few (say 5) variables z i1,..., z i5 of A. To show: If A has good vector solution, so does B. Question posed in [AAT 04]. First done by [KV 05] from Unique Games to Sparsest Cut.

104 What can be proved MAX k-csp Independent Set Approximate Graph Coloring Chromatic Number NP-hard UG-hard Gap Levels 2 k 2 2 k 2 k 2k k+o(k) 2k Ω(n) n n 2 2 c log n log log n 2 (log n)3/4+ɛ 2 c 1 log n log log n l vs log2 l l vs. 2 l/2 4l 2 Ω(n) n n 2 2 c log n log log n 2 (log n)3/4+ɛ 2 c 1 log n log log n Vertex Cover ɛ 1.36 Ω(n δ ) All the above results are for the Lasserre hierarchy.

105 The FGLSS Construction Reduces MAX k-csp to Independent Set in graph G Φ. z 1 + z 2 + z 3 = 1 z 3 + z 4 + z 5 =

106 The FGLSS Construction Reduces MAX k-csp to Independent Set in graph G Φ. z 1 + z 2 + z 3 = 1 z 3 + z 4 + z 5 = Need vectors for subsets of vertices in the G Φ.

107 The FGLSS Construction Reduces MAX k-csp to Independent Set in graph G Φ. z 1 + z 2 + z 3 = 1 z 3 + z 4 + z 5 = Need vectors for subsets of vertices in the G Φ. Every vertex (or set of vertices) in G Φ is an indicator function!

108 The FGLSS Construction Reduces MAX k-csp to Independent Set in graph G Φ. z 1 + z 2 + z 3 = 1 z 3 + z 4 + z 5 = Need vectors for subsets of vertices in the G Φ. Every vertex (or set of vertices) in G Φ is an indicator function! U {(z1,z 2,z 3 )=(0,0,1)} = 1 8 (W + W {1} + W {2} W {3} + W {1,2} W {2,3} W {1,3} W {1,2,3} )

109 Graph Products v 1 w 1 v 2 w 2 v 3 w 3

110 Graph Products v 1 w 1 v 2 w 2 v 3 w 3

111 Graph Products v 1 w 1 v 2 w 2 v 3 w 3 U {(v1,v 2,v 3 )} =?

112 Graph Products v 1 w 1 v 2 w 2 v 3 w 3 U {(v1,v 2,v 3 )} = U {v1 } U {v2 } U {v3 }

113 Graph Products v 1 w 1 v 2 w 2 v 3 w 3 U {(v1,v 2,v 3 )} = U {v1 } U {v2 } U {v3 } Similar transformation for sets (project to each copy of G).

114 Graph Products v 1 w 1 v 2 w 2 v 3 w 3 U {(v1,v 2,v 3 )} = U {v1 } U {v2 } U {v3 } Similar transformation for sets (project to each copy of G). Intuition: Independent set in product graph is product of independent sets in G.

115 Graph Products v 1 w 1 v 2 w 2 v 3 w 3 U {(v1,v 2,v 3 )} = U {v1 } U {v2 } U {v3 } Similar transformation for sets (project to each copy of G). Intuition: Independent set in product graph is product of independent sets in G. Together give a gap of n 2 O( log n log log n).

116 A few problems

117 Problem 1: Lasserre Gaps Show an integrality gap of 2 ɛ for Vertex Cover, even for O(1) levels of the Lasserre hierarchy. Obtain integrality gaps Unique Games (and Small-Set Expansion) Gaps for O((log log n) 1/4 ) levels of mixed hierarchy were obtained by [RS 09] and [KS 09]. Extension to Lasserre?

118 Problem 2: Generalize Schoenebeck s technique

119 Problem 2: Generalize Schoenebeck s technique Technique seems specialized for linear equations. Breaks down even if there are few local contradictions (which doesn t rule out a gap).

120 Problem 2: Generalize Schoenebeck s technique Technique seems specialized for linear equations. Breaks down even if there are few local contradictions (which doesn t rule out a gap). We have distributions, but not vectors for other type of CSPs.

121 Problem 2: Generalize Schoenebeck s technique Technique seems specialized for linear equations. Breaks down even if there are few local contradictions (which doesn t rule out a gap). We have distributions, but not vectors for other type of CSPs. What extra constraints do vectors capture?

122 Thank You Questions?

Hierarchies. 1. Lovasz-Schrijver (LS), LS+ 2. Sherali Adams 3. Lasserre 4. Mixed Hierarchy (recently used) Idea: P = conv(subset S of 0,1 n )

Hierarchies. 1. Lovasz-Schrijver (LS), LS+ 2. Sherali Adams 3. Lasserre 4. Mixed Hierarchy (recently used) Idea: P = conv(subset S of 0,1 n ) Hierarchies Today 1. Some more familiarity with Hierarchies 2. Examples of some basic upper and lower bounds 3. Survey of recent results (possible material for future talks) Hierarchies 1. Lovasz-Schrijver