Linear Level Lasserre Lower Bounds for Certain k-csps

Transcription

1 Linear Level Lasserre Lower Bounds or Certain k-csps Grant Schoenebeck Abstract We show that or k 3 even the Ωn) level o the Lasserre hierarchy cannot disprove a random k-csp instance over any predicate type implied by k-xor constraints, or example k-sat or k-xor. One constant is said to imply another i the latter is true whenever the ormer is. For example k-xor constraints imply k-cnf constraints.) As a result the Ωn) level Lasserre relaxation ails to approximate such CSPs better than the trivial, random algorithm. As corollaries, we obtain Ωn) level integrality gaps or the Lasserre hierarchy o 7 6 or VertexCover, 2 or k-uniormhypergraphvertexcover, and any constant or k-uniormhypergraphndependentset. This is the irst construction o a Lasserre integrality gap. Our construction is notable or its simplicity. t simpliies, strengthens, and helps to explain several previous results. 1 ntroduction The Lasserre hierarchy [Las01] is a sequence o semideinite relaxations or certain 0-1 polynomial programs, each one more constrained than the last. The kth level o the Lasserre hierarchy requires that any set o k original vectors be sel-consistent in a very strong way. an integer program has n variables, the nth level o the Lasserre hierarchy is suicient to obtain a tight relaxation where the only easible solutions are convex combinations o integral solutions. This is because the nth level requires that the entire set o n vectors are consistent. one starts rom a k-csp with polyn) constraints, then it is possible to optimize over the set o solutions deined by the kth level o Lasserre in time On Ok) ), which is sub-exponential or k = on/ log n). The Lasserre hierarchy is similar to the Lovasz-Schrijver hierarchies [LS91], denoted LS and LS+ or the linear and semideinite versions respectively, and the Sherali-Adams [SA90] hierarchy, denoted SA; however, the Lasserre hierarchy is stronger [Lau03]. The region o easible solutions in lth level o the Lasserre hierarchy is always contained in the region o easible solutions in lth level o LS, LS+, and SA 1. A more complete comparison can be ound in [Lau03]. While there have been a growing number o integrality gap lower bounds or the LS[ABL02, ABLT06, Tou06, STT07b], the LS+[BOGH + 03, AAT05, STT07a, GMPT06], and the SA[dlVKM07, CMM07] hierarchies, similar bounds or the Lasserre hierarchy have remained elusive. The study o these hierarchies is motivated by the success o semideinite programs in approximation algorithms. n many interesting cases, or small constant l, the lth level o the Lasserre hierarchy grant@cs.berkeley.edu, Computer Science Division, UC Berkeley. This material is based upon work supported under a National Science Foundation Graduate Research Fellowship. Work or this paper was done while author was visiting Princeton University and Microsot Research-Silicon Valley 1 n our deinition, or ease o presentation, the lth level o Lasserre or a k-csp is only meaningul i l k, but this can be modiied. 1

2 provides the best known polynomial-time computable approximation. For example, the irst level o the Lasserre hierarchy or the ndependentset problem implies the Lovasz θ-unction and or the MaxCut problem gives the Goemans-Williamson relaxation. The ARV relaxation o the SparsestCut problem is no stronger than the relaxation given in the third level o Lasserre. n addition, recent work by Eden Chlamtac [Chl07] has shown improved approximation algorithms or coloring and independent set in 3-uniorm hypergraphs. n [Chl07] the Lasserre hierarchy was used to ind and/or analyze the constraints which led to improved approximations. This work is unlike the aorementioned work, where it was only later realized that the approximation results could be viewed as an application o semideinite program hierarchies. ntegrality gap results or Lasserre are thus very strong unconditional negative results, as they apply to a model o computation that includes the best known algorithms or several problems. 1.1 Previous Lower-Bounds Work While this is the only work known to us on Lasserre integrality gaps, results are already known about the weaker hierarchical models or several problems, including many problems we study here. Buresh-Oppenheim, Galesy, Hoory, Magen and Pitassi [BOGH + 03], and Alekhnovich, Arora, Tourlakis [AAT05] prove Ωn) LS+ round lower bounds or proving the unsatisiability o random instances o 3-SAT and, in general, k-sat with k 3) and Ωn) 2 round lower bounds or achieving approximation actors better than 7/8 or Max 3-SAT, better than 1 ) ln n or Set Cover, and better than k 1 or HypergraphVertexCover in k-uniorm hypergraphs. They leave open the question o proving LS+ round lower bounds or approximating the Vertex Cover problem. Much work has been done on Vertex Cover. Schoenebeck, Tulsiani, and Trevisan[STT07b] show an integrality gap o 2 remains ater Ωn) rounds o LS, which is optimal. This build on the previous work o Arora, Bollobas, Lovasz, and Tourlakis [ABL02, ABLT06, Tou06] who prove that even ater Ωlog n) rounds the integrality gap o LS is at least 2, and that even ater Ωlog n) 2 ) rounds the integrality gap o LS is at least 1.5. Somewhat weaker results are known or LS+. The best known results are incomparable and were show by shown by Georgiou, Magen, Pitassi, and Tourlakis[GMPT06] and Schoenebeck, Tulsiani, and Trevisan [STT07a]. The ormer result [GMPT06] builds on the previous ideas o Goemans and Kleinberg [KG98] and Charikar [Cha02], and shows that an integrality gap o 2 survives log n Ω log log n ) rounds o LS+. The later result shows an integrality gap o 7 6 survives Ωn) rounds. This result builds on past research which we review here as it is relevant or understanding the results o this paper. The result o Feige and Oek [FO06] immediately implies a 17/16 integrality gap or one round o LS+, and the way in which they prove their result implies also the stronger 7/6 bound. The standard reduction rom Max 3-SAT to VertexCover shows that i one is able to approximate VertexCover within a actor better than 17/16 then one can approximate Max 3-SAT within a actor better than 7/8. This act, and the 7/8 integrality gap or Max 3-SAT o [AAT05], however do not suice to derive an LS+ integrality gap result or VertexCover. The reason is that reducing an instance o Max 3SAT to a graph, and then applying a VertexCover relaxation to the graph, deines a semideinite program that is possibly tighter than the one obtained by a direct relaxation o the Max 3-SAT problem. Feige and Oek [FO06] are able to analyze the value o the Lovasz θ-unction o the graph obtained by taking a random 3-SAT instance and then 2 n all integrality gap containing an, the constant in the Ω depends on. 2

3 reducing it to an instance o ndependentset or, equivalently, o VertexCover). For the Sherali-Adams hierarchy, Charikar, Makarychev, and Makarychev [CMM07] show that, or some, ater n rounds an integrality gap o 2 o1) remains. Other results by Charikar [Cha02] and Hatami, Magen, and Markakis [HMM06] prove a 2 o1) integrality gap result or semideinite programming relaxations o Vertex Cover that include additional inequalities. Charikar s relaxation is implied by the relaxation obtained ater two rounds o Lasserre. The semideinite lower bound o Hatami et al is implied ater ive rounds o Lasserre. t was compatible with previous results that ater a constant number o rounds o Lasserre the integrality gap or Vertex Cover could become 1 + o1). Our Result The main result o this paper, is a proo that, or k 3, the Ωn)th level o Lasserre cannot prove that a random k-csp over any predicate implied by k-xor is unsatisiable. From this main results it quickly ollows that the Ωn)th level o Lasserre: cannot prove a random k-xor ormula unsatisiable. cannot prove a random k-sat ormula unsatisiable. contains integrality gaps o 1/2 + or Max-k-XOR contains integrality gaps o k + or Max k-sat. contains integrality gaps o 7 6 or VertexCover. contains integrality gaps o any constant or k-uniormhypergraphvertexcover. contains integrality gaps o Ω1) or k-uniormhypergraphndependentset. n addition to the power o our result, it is also very short and simple. t extends and simpliies results in [STT07a] and [AAT05]. To a large extent it also explains the proos o [FO06] and [STT07a], and can be seen as being inspired by these results. Road Map n Section 2 we will deine notation and provide background to our results. n Section 3 we will prove the main result. n Section 4 we will state and prove the remaining results, which are corollaries o the main result. 2 Background and Notation We denote the set o Boolean variables [n] = {1,..., n}. Let the range o variables be denoted x = {x i } i [n] = {0, 1} n. For {1,..., n}, let x = {x i } i be the projection o x to the coordinates o. We will consider programs where each constraint is local, captured by the ollowing deinition: Deinition 1 A k-constraint C is a unction : x {0, 1} where k. 3

4 Note that given a constraint C where : x {0, 1}, we can naturally extend it to a constraint C J where J and : x J {0, 1} by irst projecting to the variables o and then applying. Sometimes we abuse notation and denote by C the set {x x : x ) = 1}. Deinition 2 A k-constraint C implies another k-constraint C g i C C g. We say that a predicate is XOR-implied i it is implied by either parity or its negation. For notational convenience, we will denote by C x the constraint C where x ) = 1 i x = x and 0 otherwise. Also, we denote by C 1 the constraint C x, where x is one in each coordinate; by C the constraint that is always satisied; and by C the constraint that is never satisied. We will look at relaxations or two types o integer programs. n the irst, we have a set o constraints, and would like to know i there is any easible solution. n the second, we have a set o constraints and would like to maximize some objective unction subject to satisying the constraints. We ormalize the notions here: Deinition 3 A k-constraint satisiability problem x, C is a set o n Boolean variables x = {0, 1} n, and a set o k-constraints {C }. Deinition 4 A k-constraint maximization or minimization) problem x, C, M is a set o n Boolean variables x = {0, 1} n, a set o k-constraints {C }, and a polynomial objective unction M o total degree at most k such that M : x Z is to be maximized or minimized). Lasserre Let x, C, M be a constraint maximization or minimization) problem. deally, we would like to say that a solution y 1,..., y n ) in the easible region o any level o the Lasserre hierarchy must be the convex combination o integer solutions; however, enorcing this directly is diicult. nstead we note that i y 1,..., y n ) = m j=1 p jz j 1,..., zj n) is rom a probability distribution o integral solutions, that is y 1,..., y n ) = m j=1 p jz j 1,..., zj n) where z j i {0, 1}, zj = z j 1,..., zj n) are a easible integral solutions, and m j=1 p j = 1 then, or each possible k-constraint C we can produce a vector 3 { pj C v C j) = zj ) = 1 0 otherwise 1) we deine scalar variables x C so that x C distribution over integer solutions, then x C satisies the constraint C = v C 2, and think o y 1,..., y n ) as a probability is the probability that a randomly drawn solution. These vectors will satisy all the constraints o the Lasserre hierarchy at any level. the reader is unamiliar with the deinition o Lasserre, then it is a straightorward and useul exercise to veriy this act. Deinition 5 The rth round o Lasserre on the k-constraint maximization problem x, C, M is the semideinite program with a variable v C or every k-constraint C not just those in C ). Let M = m i=1 w i j i x j be the objective unction expressed as the weighted sum o monomials. We 3 That is or any unction : x {0, 1} where k, not simply the constraints that appear in the constraint maximization problem. 4

5 will denote by v 0 the vector or the constraint C, denote by v x the vector or the constraint C x, denote by v i the vector or the constraint C 1 {i}, and denote by v 1 x the vector or the constraint C 1. where m max w i v 1 i 2 i=1 [n] v C 2 = 1 2) C C, Cg J, C, C g J where J = J C Cg J ) C C g J ) C v C 2 = 1 3) v C, v C g = v, v J C C g J = v C x C 4) v x 5) We compare the equivalence o C Cg J ) and C C g J ) as unctions on the domain x J. The semideinite program or the rth Lasserre round o a satisiability problem is the same, but we only check or the existence o easibility, we do not try to maximize over any objective unction. 4 While the equations are conusing, the intuition is that the vectors deine a probability distribution on any set o up to r coordinates Equations 2, 4, and 5); that the probability distributions always satisy the constraints Equation 5); and that the probability distributions properly patch together Equation 4). t is easy to check the suggested vectors satisy all these constraints. Claim 6 Fix [n] such that r. Then we can get probability distribution over the elements o x x by deining the probability o x to be v x 2. Actually, these vectors are all orthogonal, and i you sum over them, you get v 0. Proo: x, x x, then v x and v x are orthogonal because C x Cx = C v x, v x = v C 2 and by Equation 5 v C 2 = 0 and so by Equation 4 Thus, by Equation 2 then Equation 5: 1 = v C 2 = x x v x 2 = x x v x 2. So indeed we have a probability distribution. By Equations 2 and 4 [n], v C = v 0 because v C v 0, v C = v C 2 v C, v 0 = 1 1 = 0. n applications, it is usually important that we have vectors and not simply local distributions that patch together. The act that we have vectors gives some global orientation. The Goemans- Williamson MaxCut algorithm generates a global cut with a hyperplane. t is not clear how to do this with a local distributions alone. 4 This deinition is slightly dierent, but equivalent to other deinitions o the kth level o the Lasserre hierarchy. The way that it is stated, it would require double exponential time to solve the rth level. This is easily remedied by only deining vectors or the constraints C 1 and using linear combinations o these vectors to deine the remaining vectors. We present it like this or ease o notation. 5

6 Claim 7 Equations 2, 4 and 5 are satisied, then Equation 3 is equivalent to requiring that v x 2 = 0 or all x where x C or some C C. Proo: We only used Equations 2, 4 and 5 to show Claim 6. So we know that the v x orthogonal and by Equation 5 additionally know that 0 = 1 1 = v = C 2 1 = v x 2 v x 2 x x x C = v x 2 v x 2 = v x 2 x x x C x C are all Problems Studied Let P be a set o boolean predicates on k-variables. n a k-csp-p we are given a set o predicates or clauses) which are each taken rom P. Each clause becomes a constraint, and we are asked i all the constraints can be simultaneously satisied. n Max- k-csp-p we want to ind the maximum number o clauses that can be satisied. We can deine a distribution D over predicates in some P. To sample a random k-csp-p ormula with n clauses, we uniormly and draw n clauses rom the set o 2 k n k) P possible clauses by irst uniormly and independently sample each set o k variables and the sign applied to each variable, and then draw a predicate rom D. n k-xor we are given a set o clauses which are each o the orm i x i = 0/1 where k. We will denote the clause i x i = b by C =b. Each clause becomes a constraint, and we are asked i all the constraints can be simultaneously satisied. To sample a random k-xor ormula with n clauses, we uniormly and independently draw n clauses rom the set o 2 n k) possible clauses. n k-sat we are given a set o clauses which are each o the orm i x i where k. Each clause becomes a constraint, and we are asked i all the constraints can be simultaneously satisied. n Max k-sat we want to ind the maximum number o clauses that can be satisied. To sample a random k-sat ormula with n clauses, we uniormly and independently draw n clauses rom the set o 2 k n k) possible clauses. Deinition 8 Give a distribution D over predicates in some P we deine rp) to be the probability that a random assignment satisies a predicate drawn rom D. For example, in k-xor, D is uniormly distributed between k-parity and its negation, and rk-xor) = 1/2. n VertexCover we are given a graph G = V, E). There is a Boolean variable x i or each vertex i V. For each edge i, j) E we have a constraint which says that both x i and x j cannot be zero. We are asked to minimize i V x i. n k-uniormhypergraphndependentset we are given a k-uniorm hypergraph G = V, E). There is a variable x i or each vertex v V. For each edge i 1,..., i k ) E we have a constraint which says that not all x i1,..., x ik can be one. We are asked to maximize i V x i. 6

7 k-uniormhypergraphvertexcover is the same as k-uniormhypergraphndependentset except that or each edge i 1,..., i k ) E we have a constraint which says that at least one o x i1,..., x ik must be one. We are asked to minimize. i V x i. Background Results Suiciently dense random k-csp ormulae are ar rom being satisiable as the next proposition states. Proposition 9 For any δ > 0, with probability 1 o1), i ϕ is a random k-csp chosen rom the distribution D with n clauses where ln 2 + 1, at most a rd) + δ raction o the clauses o ϕ 2δ 2 can be simultaneously satisied. Proposition 9 is well known in the literature, we provide a proo in the appendix or completion. Deinition 10 Width-w resolution on an XOR ormula ϕ, successively builds up new clauses by deriving a new clause i J x i = b b whenever the symmetric dierence J w and the clauses i x i = b and i x i = b had either already been derived or belong to ϕ. Width-w resolution proves a ormula ϕ unsatisiable i it derives the clause 0 = 1. The ollowing theorem shows that or random 3-XOR ormula, even or quite large w, width-w resolution ails to produce a contradiction. Theorem 11 For k 3, d > 0, γ > 0, and 0 < k/2 1, i ϕ is a random k-xor ormula with density dn, then with probability 1 o1) ϕ cannot be disproved by width αn 1 resolution nor can any variable be resolved to true or alse. Furthermore, this is true even i the parity sign whether the predicate is parity or its negation) o each clause is adversatively chosen. Wigderson and Ben-Sasson [BSW01] show that a variant o Theorem 11 holds or k-sat ormula. The proo o [BSW01] extends to show Theorem 11 using standard techniques. We include a proo in the appendix or completeness. 3 k-csps over XOR-mplied Predicates We now present the main theorem o the paper. Theorem 12 Let D be a distribution over a set o XOR-implied k-csp predicates P. Then or every δ, γ, d > 0 and 0 < k/2 1 such that i = 0, then d ln 2 + 1) there exists some 2δ 2 constants α 0, such that with probability 1 o1), i ϕ is a random k-csp ormula drawn according to D with n clauses where = dn both the ollowing are true: 1. at most a rd) + δ raction o the clauses o ϕ can be simultaneously satisied. 2. The αn 1 k/2 1 γ level o the Lasserre hierarchy permits a easible solution. This theorem implies integrality gaps or XOR-implied k-csps because the Lasserre relaxation cannot reute that all clauses can be simultaneously satisied, but, in act, at most rd) + δ clauses can be simultaneously satisied. Notice that an algorithm that simply guesses a random assignment 7

8 would expect to satisy an rd) raction o clauses in expectation. n particular this theorem shows that with high probability a random k-xor ormula cannot be reuted by Ωn) rounds o Lasserre which gives an integrality gap o 1/2 + or Ωn) rounds o Lasserre or Max k-xor. Also, this theorem shows that with high probability a random 3-CNF ormula cannot be reuted by Ωn) rounds o Lasserre which gives an integrality gap o 7/8 + or Ωn) rounds o Lasserre or Max k-sat. Theorem 12 ollows almost immediately rom Theorem 11, Proposition 9, and the ollowing Lemma. Lemma 13 Main Lemma) a k-xor ormula ϕ cannot be disproved by width-w resolution, then the w 4 th round o the Lasserre hierarchy permits a easible solution. Proo:[o Theorem 12] Fix δ, γ, d,, P, D as allowed in theorem statement, and let ϕ be a random k-csp P ormula with n clauses where = dn. By Proposition 9, 1) holds with probability 1 o1) because or suiciently large n, = dn > ln δ 2 We can write ϕ as a k-xor ormula ϕ XOR so that ϕ XOR ϕ. Now the Lasserre relaxation or ϕ XOR is strictly tighter than that or ϕ. Let α be as guaranteed in Theorem 11 using k, d, γ, and as inputs so that by Theorem 11 we know that with probability 1 o1) it is the case that ϕ XOR cannot be disproved by width-α n 1 resolution. Let α = α 4. By Lemma 13, ϕ XOR cannot be proven unsatisiable by α 4 n1 = αn 1 rounds o Lasserre. Because the Lasserre relaxation or ϕ XOR is tighter than that or ϕ XOR it must be the case that ϕ cannot be proven unsatisiable by Lasserre either. Lemma 13 is the main original technical contribution o this work. n the rest o this section we irst provide some intuition or the proo o Lemma 13 and then provide its proo. For a irst attempt to prove the lemma we can observe that or any particular set o at most w/4 variables, we can construct vectors or all C as ollows: 1) Run bounded width resolution to derive a set o constraints that any satisying assignment must satisy. 2) Consider the set SAT where SAT = {x x : x satisies all the constaints derived by the resolution whose support is contained in }. Randomize over SAT and construct the vectors as we saw in Equation 1. These vectors will satisy the Lasserre Equations 2, 3, and 5; however, these vectors will ail miserably to satisy Equation 4 o the Lasserre constraints. We have set up valid local distributions; however, these distribution do not patch together consistently. The problem is that when the take the dot product o v x and v xj, the values in each coordinate mean something completely dierent. To remedy this misalignment we design a space o equivalence classes o XORs o at most w/2 variables which we will use to index the coordinates o each vector. We will say that i x i j J x j i or all assignments that satisy the derived resolution clauses, i x i determines i J x i and vice versa. For example, i ϕ contained the clause x 1 x 2 x 3 = 0 then x 1 x 2 x 3 because whatever x 1 x 2 is, x 3 must be the opposite. With some equivalent clauses, ixing one clause automatically ixes the equivalent clause to the opposite value as above). With other equivalent clauses, ixing one clause automatically ixes the equivalent clause to the same value. Using this act, we can split each equivalence class o equivalent clauses into two parts, so that the equivalent clauses in each part always ix each other to the same value, and equivalent clauses in opposite parts always ix each other to the opposite value. We can arbitrarily label one part + and the other. There is a bijection between the set SAT and the equivalence classes o i J x i where J, because, intuitively, each time resolution derives a new relation, the dimension o each o these sets is reduced by 1. 8

9 Finally, or each x SAT we construct a vector where each coordinate is indexed by the equivalence classes we just produced. This vector is ± 1 SAT in each equivalence class that contains an set J, and is 0 elsewhere. The sign is + i x agrees with the + side o the equivalence class and otherwise. we project onto only the non-zero coordinates, then the mapping o our previously constructed vectors that ailed to satisy Equation 4) to these new vectors is simply a rotation. This implies that all the Lasserre equations that were previously satisied will still be satisied. This rotation is similar to taking a Fourier transorm. any two distinct vectors v x and v xj disagree in any non-zero location, they disagree in exactly 1/2 o the places and are orthogonal. Otherwise the signs agree in every non-zero coordinate. These acts allow us to show that these vectors do satisy Equation 4 o the Lasserre constraints. We will now prove Lemma 13. Proo:[Lemma 13] Construction o Vectors We irst deine a set C which later will be used to index the coordinates o the vectors. Let ϕ be a k-xor ormula that has no width-w resolution. Let res-c be the collection o clauses generated by width-w resolution running on ϕ. By the hypothesis o the Lemma we are guaranteed that we cannot derive a contradiction. Let w 2 -C be the collection o all possible XOR clauses over at most w 2 set w 2 -C/res-C. That is we partition w 2 -C into equivalence classes where C =b res-c. C J)=b b ) J variables. Now consider the res-c C J=b J Let S w/2 = { i x i : [n] : w/2}. Let F S w/2 be the XOR unctions ixed by res-c, that is F b {0, 1} where C =b res-c. Consider the set C = S w/2 /F that is F J J F. For each equivalence class [] C, we arbitrarily choose some 0 [] or notational convenience, we always choose [ ]). We deine a unction π : S w/2 {+1, 1} such that { +1 C =0 π) = res-c C 0=0 0 1 C =0 res-c C 0=1 0 Claim 14 F and res-c are equivalence relations and π is well deined. Fix some V [n] where V w/4. Let S w/2 V = { i x i : V } denote the collection o all possible XOR unctions over V. Similarly deine F V and res-c) V. We will denote S w/2 V simply in the uture. S V we interpret S V as a group under the action o symmetric dierence, then F V is a subgroup o S V. We can then consider the group S V /F that is F J J F. Note also that S v /F = S v /F v. Let SAT V x V where x V SAT V i it satisies all the clauses in res-c) V. SAT = 1 by convention. Claim 15 SAT V = S V /F V Proo: Consider the natural bijection between x V and S V where x V x V is considered as a unction that XORs the non-zero bits o x V. Then we can move reely between those two spaces, so it makes sense to write x V /F V = SV /F V. Let [ ] be the equivalence class o in S V /F V, and let 9

10 x V x V be a satisying assignment to all the clauses in res-c) V. Then or x V x V, x V SAT V i and only i x V x V + [ ]. t ollows that SAT V = x V /F V = S V /F V. We now deine the vectors. Let w/4. x SAT, v x = 0. Otherwise: v x [J]) = { 1) k K x k πk) SAT where the range o the coordinates is [J] C K [J] s.t. K 0 K [J] K Recall that v x is the constraint in x that is only satisied on input x. ntuitively, each vector v x is non-zero exactly in the coordinates corresponding to [J] S /F. The sign o each non-zero coordinate [J] S /F corresponds to the evaluation o j J x j and the πk) terms makes the vectors well deined). We obtain the vectors or other constraints by taking linear combinations o these vectors. v C = x :C x )=1 Remark 1 the width-bounded resolution not only does not reute ϕ, but also does not ix any variable x i to either true or alse, then or every i [n], SAT {i} = 2 and so v i 2 = 1/2. Proo that constructed vectors satisy Lasserre constraints To show that Equations 2 and 3 are satisied, we use the ollowing claim. Claim 16 v x, v xj = { 1 SAT J v x [K] S /F S J /F ), signv x [K])) = signv xj [K])) 0 otherwise For any x x and x J x J, only coordinates where both v x and v xj are non-zero will contribute to the value o v x, v xj. Claim 16 states that either there is no cancelation amongst non-zero coordinates all the coordinates where both v x and v xj are non-zero have the same sign), or there is complete cancelation v x, v xj = 0). Proo: Note that v x and v xj are both non-zero only in the coordinates [K] S /F S J /F. Let s say that or some [K] S /F S J /F ), signv x [K])) signv xj [K])). We can use this [K] to show that the signs o v x and v xj dier on exactly hal o the coordinates o S /F S J /F and thus v x, v xj = 0. Let [K 1 ], [K 2 ],..., [K l ] be a complete list o the elements in S /F S J /F ). Now consider the list [K 1 K], [K 2 K],..., [K l K] which is a permutation o the irst list. Because sign[k i K]) = sign[k i ]) sign[k]), i signv x [K i ])) = signv xj [K i ])) then signv x [K i K])) signv xj [K i K])), and i signv x [K i ])) signv xj [K i ])) then signv x [K i K])) = signv xj [K i K])). But because we simply permuted the coordinates, we know that the number o agreements and respectively disagreements) in the irst list are the same as the number o agreements and respectively disagreements) in the second list. And so, the number o agreeing and disagreeing coordinates are equal. 10

11 Now assume that [K] S /F S /F ), signv x [K])) = signv xj [K])). we view S /F and S J /F as subgroups o S J /F we see that S /F S J /F S /F S J /F = S J /F. Combining this with the act that S J /F = S J /F J = SAT J rom Claim 15 we get that v x, v xj = S /F S J /F SAT SAT J = S /F S J /F = S /F S J /F Equation 2, that v C 2 = 1, is satisied, because 1 S J /F = 1 SAT J v C 2 = v x 2 = v x 2 = v x 2 1 = SAT SAT x x x x x SAT Also recall, by our construction v 0 = 1, 0,..., 0) where the 1 is in the location indexed by [ ]. Equation 3 states that C v C 2 = 1. By construction v x 2 1 = SAT i x SAT because each non-zero coordinate o v x is ± 1 SAT and there are SAT such coordinates) and v x 2 = 0 i x SAT by construction. Also, by Claim 16 v x and v x are orthogonal or x x. To show that Equation 3 is satisied we must show that or any clause C =b that appears in ϕ, v C =b 2 = 1. Fix some such clause C =b ϕ. Then v C 2 = x SAT v x 2 = 1. Equation 4 states that or all C, Cg J, C, C g J we have that v C, v C g = v, v J C C g whenever J J = J and C Cg J ) C C g J ). To show that Equation 4 is satisied we show that where SAT C v C, v C g SAT J SAT = C J SAT J SAT C g J is the subset o x that satisies the clause C. t ollows that Equation 4 is satisied because or any C, Cg J, C, C g J SAT C SAT C g J. where J = J and g) = g ), SAT C SAT C g J = v C, v C g = J x C v x, x J C g J v xj = x C x J C g J v x, v xj = SAT J SAT C SAT C SAT J The last equality ollows rom Claim 16. Finally, Equation 5, which states that C v C = x C v x is satisied by construction. 4 Extensions We now mention the corollaries o Theorem 12 and its proo. Corollary 17 For every, there exists some constants α 0, such that the αn level o Lasserre, an integrality gap o 7 6 or VertexCover persists. 11

12 The idea o the proo is to rewrite a 3-XOR ormula ϕ as a vertex cover problem on a graph G ϕ using the standard FGLSS reduction. We will do it in such a way that any vectors that satisy the Lasserre relaxation or the 3-XOR instance ϕ will also satisy the vertex cover Lasserre relaxation or G ϕ. To prove this corollary, we use the ollowing lemma which states that or a certain type o transormations most o the Lasserre constraints continue to be satisied: Lemma 18 Let x, C, M and x, C Ī, M be two constraint maximization or minimization problems. Let g be a map rom constraints on x = {0, 1} n to constraints on x such that 1) For each i [n], gi) = C Ī, where C Ī is some constraint over x. 2) For x x, gx ) = i,xi =1gi) i,xi =0 gi). 3) For each C, gc ) = x C gx ). Let k = max Ī : gi) = C Ī. Then i a collection o vectors { v C } Ī C Ī ater r rounds or x, C Ī, M, then the collection o vectors {v C } C Equations 2, 4, and 5 or r/k rounds o Lasserre. satisy the Lasserre constraints where v C v gc ) satisy Proo: That we only run or r/k rounds o Lasserre makes all the vectors well-deined. Each constraint or which we deine a vector depends on at most r/k, and so the corresponding vector depends on at most r variables. We irst show Equation 2, that [n], v C 2 = 1 is satisied. v C 2 = v gc ) 2 = v x x gx ) 2. Let x be the domain o gx Ī ). Then x x gx ) = C Ī because gx ) = i,xi =1gi) i,xi =0 gi) and so every x x satisies exactly one o these gx ). t ollows that v x x gx ) 2 = v C Ī 2 = 1 because the vectors { v C } Ī C satisy the Lasserre constraints. Ī Equation 5 is satisied by construction. Finally, Equation 4 is satisied. We show that C, Cg J, C, C g J where J = J and C Cg J ) C C g J ) we have that v C, v C g = v gx J ) 2 J x J C Cg J This is suicient because the right hand side will be the same or v C, v C g J. v C, v C g = J x C = x J C g J x C x J C g J v x, v xj = x C x J C g J v gx ), v gxj ) v i,xi =1gi) i,xi =0 gi), v j J,xj =1gj) j J,xj =0 gj) Now i x and x J do not agree on the overlap, then the term is 0. For i they are inconsistent, then one vector will require that gi) holds, and the other that it does not hold, and thus the dot product will be 0. So we can assume that they do agree on the overlap. But then we are simply summing over assignments to x J that satisy C Cg J. And thus the above can be rewritten as v i J,xi =1gi) i J,xi =0gi) 2 = v gx J ) 2 x J C Cg J x J C Cg J 12

13 We now prove Corollary 17 Proo: [Corollary 17] Given a 3XOR instance ϕ with n = m equation, we deine the FGLSS graph G ϕ o ϕ as ollows: G ϕ has 4m vertices, one or each equation o ϕ and or each assignment to the three variables that satisies the equation. We think o each vertex i as being labeled by a partial assignment to three variables Li). Two vertices i and j are connected i and only i Li) and Lj) are inconsistent. For example, or each equation, the our vertices corresponding to that equation orm a clique. t is easy to see that optϕ) is precisely the size o the largest independent set o G ϕ because there is a bijection between maximal independent sets and assignment to the n variables. Note that, in particular, the independent set size o G ϕ is at most N/4, where N = 4m is the number o vertices. Thus the smallest vertex cover o G ϕ is 3N/4 because the complement o any independent set is a vertex cover). Now the constraints or the Lasserre hierarchy or VertexCover on this graph G ϕ, are deined over the vertices o this graph. Formally, let x = {0, 1} V Gϕ). Let C contain the constraint Cx i x j {i,j} or each edge i, j) EG ϕ ), so that the constraint is satisied i and only i at least one o the vertices incident to the edge is in the cover. Let M = i V G ϕ ) x i. Then VertexCover is the 2-constraint minimization problem x, C, M. We can convert it into a Lasserre instance using Deinition 5. However, each vertex is really just an assignment to 3 variables o the 3XOR instance. ntuitively, there is a natural map rom constraints on the instance o G ϕ to constraints on ϕ. Using this intuition, we will solve the instance o ϕ using the vectors o Theorem 12, deine each vector in the instance o G ϕ to be identical to the vector assigned to the negation o the analogous constraint o ϕ, and then show the all the Lasserre constraints are satisied. For vertex i deine gi) = Li) to be the negation o the constraint implied by the label o vertex i. Now extend g as in Lemma 18. By Lemma 18 Equations 2, 4, and 5 or Ωn) rounds o Lasserre. We still must show that Equation 3 is satisied. We must show that or each edge i, j) EG ϕ ) that v i,j 2 + v i, j 2 + v i,j 2 = 1. By Claim 7 we can simply show that v i, j 2 = 0. Let i, j) EG ϕ ), then v i, j 2 = v gi), gj) 2 = v Li),Lj) 2 = 0 The irst equality ollows rom Equation 4. The last equality is true because Li) and Lj) contradict each other. We know this because i and j are joined by an edge. Knowing that the Lasserre constraints are satisied, we compute the objective unction i V G ϕ) v i 2. Four distinct vertices were created or each o the N clauses. We show that the sum o the v i 2 over the our vertices in any clause is always 3. Let C ϕ be such a clause, let i j : 1 j 4 be the our vertices corresponding to C, and let Li j) be the label corresponding to vertex i j. Then 4 j=1 v Li) = v 0 by Claim 6 and the act that the vector corresponding to an unsatisying assignment is 0. And so 4 v ij 2 = j=1 4 v Lij ) 2 = 3 j=1 4 v Lij ) 2 = 3 However, at most 1/2 + )n o the clauses o ϕ can be satisied, and so G ϕ has an independent set o at most )N, and by taking the complement a vertex cover o size at most 7 8. We get the integrality gap o )N/3N/4) = 7 6 j=1 13

14 Corollary 19 There exists some constants α 0, such at the αn level o Lasserre, an integrality gap o any constant persists or 3-UniormHypergraphndependentSet. We will use the ollowing well known proposition which is proved in the appendix or completeness: Proposition 20 For every k 3, > 0, there exists δ > 0, such that i H is a random k-uniorm hypergraph with n edges, where δ, then with probability 1 o1), H has no independent set o size n, and, equivalently, H has no vertex cover o size 1 )n. Proo: Let = 1 2c and let be such that 1 ) e. Let H be a random uniorm hypergraph with n edges. By proposition 20 we know that with high probability H has no independent set o size n. We now must show that there exists a good solution to the Lasserre relaxation. We note that the CSP instance is x, C, M where x = {0, 1}V H), M = i V x i, and or each edge v 1,..., v k ) EH) we add the constraint k i=1 x i to C which we can transorm into a Lasserre relaxation according to Deinition 5. Note that any constraint o the orm k i=1 x i is implied by either k i=1 x i = 1 i k is even or k i=1 x i = 0 i k is odd. Consider then the k-xor ormula ϕ H with n clauses which implies C. We see that by Theorem 11 that ϕ cannot be disproved by width Ωn) resolution and no single variable can be ixed. Even though the k-xor clauses are not truly random because the constants are all the same, the theorem still applies. By Theorem 12 ϕ cannot be disproved by Ωn) levels o Lasserre. Moreover by Remark 1 we have that v i 2 = 1/2 or all i. Thus M = v i 2 = n/2. So the ratio o the Lasserre optimum to the actual optimum is n/2 n = c. Corollary 21 There exists some constants α 0, such at the αn level o Lasserre, an integrality gap o 2 persists or 3-UniormHypergraphVertexCover. The proo o Corollary 19 is very similar to that o Corollary 21 Proo: Let = 1 2c and let be such that 1 ) e. Let H be a random uniorm hypergraph with n edges. By proposition 20 we know that with high probability H has no vertex cover o size 1 )n. We now must show that there exists a good solution to the Lasserre relaxation. We note that the CSP instance is x, C, M where x = {0, 1}V H), M = i V x i, and or each edge v 1,..., v k ) EH) we add the constraint k i=1 x i to C which we can transorm into a Lasserre relaxation according to Deinition 5. Note that any constraint o the orm k i=1 x i is implied by k i=1 x i = 1. Consider then the k-xor ormula ϕ H with n clauses which implies C. We see that by Theorem 11 that ϕ cannot be disproved by width Ωn) resolution and no single variable can be ixed. Even though the k-xor clauses are not truly random because the constants are all the same, the theorem still applies. By Theorem 12 ϕ cannot be disproved by Ωn) levels o Lasserre. Moreover by Remark 1 we have that v i 2 = 1/2 or all i. Thus M = v i 2 = n/2. So the ratio o the actual optimum to the Lasserre optimum is 1 )n n/2 = Acknowledgements would like to both acknowledge and thank Alexandra Kolla and Madhur Tulsiani or many ruitul discussions without which this work would not have been completed. Also, would like to thank Moses Charikar, Eden Chlamtac, Uriel Feige, and Avi Wigderson or helpul thoughts and pointers. Finally, would like to thank the anonymous FOCS reviewers or helpul comments. 14

15 6 Conclusion We have shown the irst known integrality gaps or Lasserre. On the one hand you can see the main theorem Theorem 12) as showing gaps or problems that are already known or thought to be NP-hard. We say that a predicate A is approximation resistant i, given a constraint satisaction problem over A predicates, it is NP-hard to approximate the raction o such predicates which can be simultaneously satisied better than the trivial algorithm which randomly guesses an assignment and returns the raction o predicates it satisies. n [Zwi98], Zwick shows that the only 3-CPSs which are approximation resistant are exactly those which are implied by parity or its negation. So, or k = 3, the main theorem applies exactly to those problems which we already know are NP-hard. On the other hand, the main theorem applies to results that are known to be in P. Deciding i a k-xor ormula is satisiable is equivalent to solving a set o linear equations over F 2, which can be done with Gaussian elimination. Reerences [AAT05] [ABL02] [ABLT06] Michael Alekhnovich, Sanjeev Arora, and annis Tourlakis. Towards strong nonapproximability results in the Lovasz-Schrijver hierarchy. n Proceedings o the 37th ACM Symposium on Theory o Computing, pages , , 2, 3 Sanjeev Arora, Béla Bollobás, and László Lovász. Proving integrality gaps without knowing the linear program. n Proceedings o the 43rd EEE Symposium on Foundations o Computer Science, pages , , 2 Sanjeev Arora, Béla Bollobás, László Lovász, and annis Tourlakis. Proving integrality gaps without knowing the linear program. Theory o Computing, 22):19 51, , 2 [BOGH + 03] Josh Buresh-Oppenheim, Nicola Galesi, Shlomo Hoory, Avner Magen, and Toniann Pitassi. Rank bounds and integrality gaps or cutting planes procedures. n Proceedings o the 44th EEE Symposium on Foundations o Computer Science, pages , , 2 [BSW01] [Cha02] [Chl07] [CMM07] Eli Ben-Sasson and Avi Wigderson. Short proos are narrow-resolution made simple. J. ACM, 482): , Moses Charikar. On semideinite programming relaxations or graph coloring and Vertex Cover. n Proceedings o the 13th ACM-SAM Symposium on Discrete Algorithms, pages , , 3 Eden Chlamtac. Approximation algorithms using hierarchies o semideinite programming relaxations. n FOCS 07: Proceedings o the 48th Annual EEE Symposium on Foundations o Computer Science, pages , Washington, DC, USA, EEE Computer Society. 2 Moses Charikar, Konstantin Makarychev, and Yury Makarychev. ntegrality gaps or Sherali-Adams relaxations, manuscript. 1, 2 15

16 [dlvkm07] [FO06] [GMPT06] [HMM06] [KG98] [Las01] Wenceslas Fernandez de la Vega and Claire Kenyon-Mathieu. Linear programming relaxations o maxcut. n SODA 07: Proceedings o the eighteenth annual ACM- SAM symposium on Discrete algorithms, pages 53 61, Philadelphia, PA, USA, Society or ndustrial and Applied Mathematics. 1 Uriel Feige and Eran Oek. Random 3CNF ormulas elude the Lovász theta unction. Manuscript, , 3 Konstantinos Georgiou, Avner Magen, Toniann Pitassi, and annis Tourlakis. Tight integrality gaps or Vertex Cover SDPs in the Lovasz-Schrijver hierarchy. Technical Report TR06-152, Electronic Colloquium on Computational Complexity, , 2 Hamed Hatami, Avner Magen, and Vangelis Markakis. ntegrality gaps o semideinite programs or Vertex Cover and relations to l 1 embeddability o negative type metrics. Manuscript, Jon M. Kleinberg and Michel X. Goemans. The Lovász Theta unction and a semideinite programming relaxation o Vertex Cover. SAM Journal on Discrete Mathematics, 11: , Jean B. Lasserre. An explicit exact sdp relaxation or nonlinear 0-1 programs. n Proceedings o the 8th nternational PCO Conerence on nteger Programming and Combinatorial Optimization, pages , London, UK, Springer-Verlag. 1 [Lau03] Monique Laurent. A comparison o the sherali-adams, lovász-schrijver, and lasserre relaxations or 0 1 programming. Math. Oper. Res., 283): , [LS91] [SA90] [STT07a] [STT07b] [Tou06] [Zwi98] L. Lovasz and A. Schrijver. Cones o matrices and set-unctions and 0-1 optimization. SAM J. on Optimization, 112): , Hani D. Sherali and Warren P. Adams. A hierarchy o relaxation between the continuous and convex hull representations. SAM J. Discret. Math., 33): , Grant Schoenebeck, Luca Trevisan, and Madhur Tulsiani. A linear round lower bound or Lovasz-Schrijver SDP relaxations o Vertex Cover. n Proceedings o the 39th ACM Symposium on Theory o Computing, Earlier version appeared as Technical Report TR on Electronic Colloquium on Computational Complexity. 1, 2, 3 Grant Schoenebeck, Luca Trevisan, and Madhur Tulsiani. Tight integrality gaps or Lovasz-Schrijver LP relaxations o Vertex Cover and Max Cut. n Proceedings o the 39th ACM Symposium on Theory o Computing, Earlier version appeared as Technical Report TR on Electronic Colloquium on Computational Complexity. 1, 2 annis Tourlakis. New lower bounds or Vertex Cover in the Lovasz-Schrijver hierarchy. n Proceedings o the 21st EEE Conerence on Computational Complexity, , 2 Uri Zwick. Approximation algorithms or constraint satisaction problems involving at most three variables per constraint. n SODA 98: Proceedings o the ninth annual ACM-SAM symposium on Discrete algorithms, pages , Philadelphia, PA, USA, Society or ndustrial and Applied Mathematics

17 7 Appendix Proposition 22 For any δ > 0, with probability 1 o1), i ϕ is a random k-csp chosen rom the distribution D with n clauses where ln 2 + 1, at most a rd) + δ raction o the clauses o ϕ 2δ 2 can be simultaneously satisied. Proo: Fix an assignment to n variables. Now i we choose, m = n clauses at random, the probability that more than a rd) + δ raction o them are satisied is at most exp 2δ 2 m) = exp 2δ 2 n). To get this, we use the Cherno Bound that says Pr[X E[X] + λ] exp 2λ 2 /m) where X is the number o satisied clauses, E[X] = rd)m, λ = δm. Picking a random ormula and random assignment, the probability that more than a rd) + δ raction o the clauses are satisied is exp 2δ 2 n). Taking a union bound over all assignments, we get Pr[any assignment satisies 1/2 + δ)m clauses] exp 2δ 2 n) 2 n because ln 2 2δ = expnln 2 2δ 2 ) = exp 2δ 2 n) Theorem 23 For k 3, d > 0, γ > 0, and 0 < k/2 1, i ϕ is a random k-xor ormula with density dn, then with probability 1 o1) ϕ cannot be disproved by width αn 1 resolution nor can any variable be resolved to true or alse. Furthermore, this is true even i the parity sign whether the predicate is parity or its negation) o each clause is adversatively chosen. We use the ollowing Proposition: Proposition 24 For any k 3, d > 0, γ > 0, and 0 < k/2 1, there exists β > 0 such that i ϕ is a random k-xor ormula with density dn then with probability 1 o1): 1. Every subormula ϕ ϕ where ϕ βn 1 k/ For every subormula ϕ ϕ where ϕ [ 1 3 βn1 k ϕ 2γ ϕ where V ϕ ) is the number o variables in ϕ. is satisiable even ater ixing one variable., 2 3 βn1 ], we have that 2V ϕ ) Proo:[Theorem 23] Let ϕ be a random XOR ormula as in the theorem statement and let C be any clause over the variables o ϕ. We deine µc) to be the smallest size o a subormula ϕ ϕ such that we can start rom ϕ and imply C using resolution. We note that in any resolution tree, i C 1 and C 2 together imply C 3, then µc 1 ) + µc 2 ) µc 3 ). From the irst part o Proposition 24 we know that with high probability µ0 = 1) βn 1 k/2 1. Now consider a resolution tree that derives 0 = 1, that is, a contradiction. We will show that this tree must contain a clause C with many variables. By the subadditivity o µ as we move up the resolution tree, this tree must contain some clause C such that µc) [ 1 3 βn1, 2 3 βn1 ]. We will now show that with high probability C contains γβ 6 n1 variables and thus that the width o the resolution is at least as large. Let ϕ be a subormula o size µc) which implies C. By 17

18 the second part o Proposition 24 we know that 2V ϕ ) k ϕ γ ϕ. Each variable o ϕ must appear either in two o the clauses o ϕ or in C itsel. a variable appears in one clause, but not in C; then no matter what the value o the other variables o that clause, the clause could still be satisied by lipping this one variable. Thereore this clause can always be satisied independently o the rest o ϕ and is not required to imply C. This violates minimality o ϕ. So C + k 2 ϕ V ϕ ) C 1 2 2V ϕ ) k ϕ ) γ ϕ γβ 3 n1 so let α = γβ 3. To show that you cannot ix one variable to true or alse the proo is almost exactly the same. nstead o showing that µ0 = 1) is large, we show that or any x i, µx i = 0) and µx i = 1) are large. This also ollows rom the irst part o Proposition 24. We note that we never used the parity o individual clauses in the proo, only the variables contained in each clause. Thereore the theorem still applies even i the parity o each clause is adversarially chosen. Proo:[Proposition 24] First we bound the probability that or a random ormula ϕ, there exists a set o l clauses containing a total o ewer than cl variables by O1) lk c 1 ) l ; n k c 1 We can upper bound the probability that there is a set o l clauses containing a total o ewer than cl variables by ) n cl )) ) ) k m n l l! cl l l k where ) n cl is the choice o the variables, cl ) k) l is the choice o the l clauses constructed out o such variables, l! m) l is a choice o where to put such clauses in our ordered sequence o m clauses, and n ) l k is the probability that such clauses were generated as prescribed. Using )) N K) < en/k) K, k! < k k, and m = n 1+ we simpliy to obtain the upper bound O l k c 1 l. n k c 1 We irst show that the irst part o the proposition is true i we do not ix any variables. ϕ ϕ is a minimal unsatisiable subormula o ϕ, then each variable that appears in ϕ must occur twice in ϕ. Otherwise the clause in which that variable appears is always satisiable and ϕ is not a minimal unsatisiable subormula. Thus it is suicient to show that no set o l clauses contains ewer than k 2 βn 1 l variables. We will show that i we set c = k/2 in the above ormula, the sum over l rom 1 to k 2 1, can be made o1) with a suiciently small β. 1 k βn 2 1 l=1 O l k 2 1 n k 2 1 Let δ be a suiciently small constant, and let ωn) be some unction that grows in an unbounded ashion. We break up the above sum into: 1 δn k 2 1 ωn) 1 l=1 O l k 2 1 n k 2 1 )) l + )) l 1 k βn l=δn k 2 1 ωn) 1 +1 O l k 2 1 n k 2 1 )) l 18

19 We then bound each o these terms: 1 k δn 2 1 ) ωn) 1 l=1 O l k 2 1 n k 2 1 )) l or suiciently small δ and suiciently large n. l=1 O1)δωn) 1 ) k c 1) l = o1) 1 k βn l=δn k 2 1 ωn) 1 +1 O l k 2 1 n k 2 1 )) l 1 l=δn k 2 1 ωn) 1 +1 β δn 1 k 2 1 ωn) 1 or suiciently small β and suiciently slowly growing ωn). l=1 O1)β k c 1) l O1)β k c 1) l = o1) Now we note that small subormulas are satisiable even i we ix one variable. We can use all the above machinery, but now require that every set o l clauses contains k variables. However, )) this change is absorbed into the O constant in O l k c 1 l n because in the above analysis when k c 1 changing to ) n cl 1 cl 1 k )) l l! m ) l n ) l k k we only get an addition actor o cl 1 ne cl 1) the irst actor helps and the second is bounded by 2 k which is a constant. Now we show the second part o the Proposition. We saw above that we can bound the probability that there exists a subormula o size l that )) l ails to satisy 2V ϕ ) k ϕ 2γ ϕ l by O k 2 γ 1. We will ix β later, and now use n k 2 γ 1 a union bound to upper bound the probability that there exists a clause ϕ such that ϕ, 2 3 βn1 ] and V ϕ ) k 2 + γ) ϕ. [ 1 3 βn1 1 3 βn1 l= 1 3 βn βn1 3 βn1 3 βn1 O ) O l k 2 γ 1 n k 2 γ 1 )) l ) k 2 3 βn1 n k 2 γ 1 2 γ 1 ) ) O 23 ) 1 3 βn1 β) ) ) ) 3 βn1 = o1) ) 1 3 βn1 or a suiciently small choice o β 19