PCP Soundness amplification Meeting 2 ITCS, Tsinghua Univesity, Spring April 2009

Transcription

1 PCP Soundness amplification Meeting 2 ITCS, Tsinghua Univesity, Spring April 2009 Speaker: Andrej Bogdanov Notes by: Andrej Bogdanov Recall the definition of probabilistically checkable proofs (PCP) from last time. We say L admits a PCP of completeness c, soundness s, alphabet Σ, randomness complexity r and query complexity q if there is a randomized polynomial-time verifier V which, on input x of length n, and oracle access to a proof π Σ, uses at most r(n) random coins to select q(n) nonadaptive queries into π and x L π : Pr[V π (x) = 1] c x L π : Pr[V π (x) = 1] s. We denote the class of all such problems L by PCP Σ c,s(r(n), q(n)). The PCP theorem states that Theorem 1 (PCP theorem). SAT PCP {0,1} 1,1/2 (O(log n), O(1)). The most interesting applications of the PCP theorem concern hardness of approximation. Last time we explained this connection by showing how a PCP can be equivalently viewed as a constraint satisfaction problem Ψ with 2 r(n) constraints, each of which contains q(n) variables taking values in Σ. The PCP theorem then states that for r(n) = O(log n) and q(n) = O(1), it is NP-hard to distinguish between the case when all constraints can be simultaneously satisfied and no more than half of them can be satisfied. Here we will always be interested PCPs with randomness complexity O(log n) and mainly consider PCPs with query complexity 2. This is the setting which is most useful for proving various hardness of approximation results and in particular the setting of the unique games conjecture. 1 PCP transformations All theorems in the PCP literature are proved by way of reduction. Proving a PCP theorem amounts to controlling various parameters in the reduction. For example, let s take Håstad celebrated hardness result on systems of linear equations modulo 2 with three variables per equation (3LIN- 2). Håstad proves that it is NP-hard to distinguish between the case when 1 ε of the equations can be satisfied and when not even 1/2 + ε of them can be satisfied for any constant ε > 0. How does Håstad prove his theorem? We cannot go into much detail here but let s at least outline the proof. In PCP language, he is looking to design a PCP with alphabet Σ = {0, 1}, query complexity 3, completeness 1 ε, and soundness 1/2 + ε, and the verifier works in a particular way always XORs the 3 queried bits and compares them against some precomputed answer. The PCP theorem gives some PCP for NP, but far from one with all these required properties. The idea behind Håstad proof (and most PCP proofs) is to start from the PCP theorem and make various transformations (reductions) which will improve it in various ways that are useful for the 1

2 2 application. But it is not always obvious how to proceed. For example Håstad s problem requires improving the query complexity from O(1) to 3, but only affecting completeness and soundness by a little ε! One easy transformation for improving the query complexity is one we discussed last time: Augment the proof with a part π which contains the values of q-tuples of variables appearing in each constraint of Ψ. Then replace V by a new verifier V that queries a random entry ψ inside π together with a random variable x v that appears in ψ inside π and checks for consistency. Call the new constraint satisfaction problem Ψ. By this transformation it is not difficult to see that Theorem 2 (Query reduction). PCP Σ 1,1 δ (r(n), q) PCPΣq 1,1 δ/q (r(n) + log q, 2) showing that the query complexity can be brought down to 3 (even 2), but the soundness and alphabet size go up. So to win in one of the parameters we have to lose in the other ones. Despite this, it turns out that this is a useful transformation to do in Håstad s proof. It turns out that even though the soundness deteriorated significantly by query reduction, it can be improved significantly by a transformation called parallel repetition which preserves query complexity and completeness, but deteriorates alphabet size: Theorem 3 (Soundness amplification via parallel repetition). For every k > 0, PCP Σ 1,s(r(n), 2) PCP Σk 1,s (kr(n), 2), where s = 1 δ and s = 2 Ω Σ(δ 3 k). After performing parallel repetition, we have a PCP with very low soundness, two queries, but enormous alphabet size. The alphabet size can then be reduced by another transformation called the long code. The philosophy behind many PCP proofs is similar. It is often impossible to do everything in one step, so one goes through a sequence of small steps, each of which makes some improvement in one of the parameters while ruining the other ones. 2 The unique games conjecture Håstad s scheme for proving inapproximability results has shown successful in many different applications. In short, this kind of reduction consists of the following steps: NP-hardness of SAT (1) PCP theorem (2) query reduction (3) soundness amplification (4) long code

3 3 approximation hardness of 3LIN-2 After his work, it turned out that the same framework can also be used to prove hardness of approximation of many other problems. One remarkable thing about all these proofs is that steps (1) to (3) are the same in all of them; all they differ in is in the implementation of step (4). So from the perspective of proving hardness of approximation results, it turned out useful to abstract steps (1)-(3) into the following PCP theorem. Theorem 4. For every δ > 0 there exists an alphabet Σ such that SAT PCP Σ 1,δ (O(log n), 2). Moreover, the PCP verifier has the following projection property: Once the answer to the first query is revealed, there is exactly one possible answer to the second query that makes the verifier accept. It turns out that this projection property is crucial in several proofs (including Håstad s proof). Later, it became apparent that if the projection property can be strengthened (at a small cost in completeness), many new and improved hardness of approximation results can be obtained. This strengthening was suggested by Khot and has not been proved yet. Conjecture 1 (Khot s unique games conjecture). For every ε, δ > 0 there exists an alphabet Σ such that SAT PCP Σ 1 ε,δ (O(log n), 2). Moreover, the PCP verifier has the following uniqueness property: For every possible answer to the first query, there is exactly one possible answer to the second query that makes the verifier accept, and vice versa. The corresponding CSP is called a unique game because it consists of binary constraints which are all permutations: The value of one variable completely determines the value of the other, and vice versa. The unique games conjecture implies many optimal hardness of approximation results for natural NP problems. But why should we believe that the conjecture is true? One reason to believe is the abnormal success people have had proving all sorts of PCP theorems by various transformations like the ones we mentioned. Perhaps such a transformation can be applied to one of the many PCP theorems we know to prove the unique games conjecture. Discussion Since the key step in proving Theorem 4 (PCPs with projection property) was a soundness amplification step, I think it is reasonable to believe that soundness amplification should play a key role in the proof of the unique games conjecture. Here is one proof attempt of the unique games conjecture by dream soundness amplification. It follows from Håstad s result that there exist two constants s < c < 1 (I think s 0.91, c 0.94) such that it is NP-hard to distinguish graphs whose max-cut has size c from those whose max-cut has size s. Max-cut is a unique game. If we could run some transformation that keeps c close to 1 and makes s close to 0, while preserving query complexity and uniqueness, we would have proved the conjecture.

4 4 There is also the possibility that the unique games conjecture is false. However, the best known algorithms for constraint satisfaction problems fail to disprove the conjecture. While it is certainly a reasonable possibility that the conjecture is false, it seems that refuting it might require some significant advances in algorithms. There is a third alternative that I believe has not been seriously investigated yet: The unique games conjecture may not be provable using current techniques. Complexity theory has been generally very successful at making arguments of this type (e.g. oracle separations, natural proofs,...) but these tools have not been applied to the investigation of unique games. I think this is a worthwhile direction. Before we get into anything, I think it will be useful to build up our arsenal of soundness amplification techniques. We already saw one sequential amplification but that had the unfortunate effect of blowing up the query complexity. Instead we want to focus on soundness amplification that preserve low query complexity (ideally 2 queries). We ll start with Dinur s gap amplification technique. 3 Dinur s proof of the PCP theorem As we discussed above, PCP transformations that improve one parameter at the cost of others are important tools in going from the PCP theorem to hardness of approximation results. Irit Dinur realized that such transformations can yield a proof of the PCP theorem itself. Dinur s starting point is the fact that the NP-hardness of 3-coloring (3COL) can be viewed as a very weak CSP harndess of approximation result. If there are m edges in the graph, it says that it is hard to distinguish satisfiable instances from those where no more than 1 δ fraction of constraints can be satisfied, where δ = 1/m. Then she designs an efficient transformation that looks roughly as follows: Ψ (3COL instance) Ψ (3COL instance) number of edges m Cm (C is some constant) completeness 1 1 soundness 1 δ 1 2δ (for δ < 1/C) Repeating this transformation O(log m) times yields the PCP theorem. To implement this transformation, it turns out that it helps to apply a similar strategy as in the proof of Theorem 4. The transformation consists of several stages (smaller transformations), where at each stage one of the parameters is improved, but at the expense of the others. At the end, we achieve an improvement in soundness while paying a small price in the size of the instance (randomness complexity of the PCP). To explain the various parameters, we need a bit of notation. The constraint graph of a CSP Ψ with 2 variables per constraint (equivalently a PCP with query complexity 2) is the graph whose vertices v correspond to variables x v in the CSP and where for each constraint ψ(x u, x v ) in Ψ there

5 5 is an edge (u, v) in the graph. Multiple edges are allowed, and we will keep the graph undirected (as the CSP constraints will be symmetric). The size of a CSP is the number of constraints in it. Abusing notation, we will say Ψ has degree d if its constraint graph is a regular graph of degree d, and the expansion of Ψ is the spectral gap 1 λ(g) of its underlying constraint graph G. Dinur s proof of the PCP theorem goes through the following sequence of transformations: transform size Σ degree arity soundness gap expansion reduce arity m 2 k = C δ reduce degree C 2 k 2 C small expanderize C 2 k d = C 2 C 1/2 amplify gap d t 2 kdt d t+1 2 t/poly( Σ ) reduce alphabet 2 2kdt 2 k = C C While all of these are interesting, the ultimate goal here is to improve the soundness, so we will focus on the gap amplification step. In fact, all the other transformations can be viewed as auxiliary steps whose objective is to put the constraint graph in the right form so that gap amplification can be applied. 4 Gap amplification Fix a constant t. The gap amplification step is a transformation from a 2CSP Ψ with degree d and expansion 1/2 to a 2CSP Ψ with the following parameters: Ψ Ψ size m ( Σ d) 5t m alphabet size Σ Σ 1+d+d2 + +d t completeness 1 1 soundness 1 δ 1 Ω(tδ/ Σ 4 ) We will describe a slightly modified version of Dinur s transformation due to Jaikumar Radhakrishnan. Let G be the constraint graph of Ψ. We define the instance Ψ as follows: Variables of Ψ : For each variable x v of Ψ, there is a corresponding variable x v of Ψ. Values of x v: The value of x v is a collection (tuple) of values in Σ, one corresponding to every vertex u at distance t from v in G. We write x v(u) for the component of x v corresponding to u. Distribution over constraints of Ψ : The constraints ψ p of Ψ correspond to paths p of length at most 5t ln Σ in G. (Abusing notation we will identify constraints and the paths they represent.) The paths are generated from the following distribution:

6 6 1. Choose a starting vertex v 0 of p. Set i = 0 2. Repeat for at most 5t ln Σ times: (1) Set v i+1 to be a random neighbor of v i and increment i (2) With probability 1/t, stop the repetition. 3. Output the path v 0, v 1,..., v i. Constraints of Ψ : Let (u, v ) be the endpoints of a path p. The constraint ψ p(x u, x v ) is satisfied if all of the following hold: 1. For every edge (u, v) in G such that u and v are both within distance t of u, the constraint ψ (u,v) is satisfied. 2. For every edge (u, v) in G such that u and v are both within distance t of v, the constraint ψ (u,v) is satisfied. 3. For every vertex v that is within distance t from both u and v, x u (v) = x v (v). The size and alphabet size of Ψ are easy to check. We need to argue completeness and soundness. By design, the transformation has perfect completeness. Suppose x is a satisfying assignment of Ψ. Now consider the assignment x of Ψ given by x u(v) = x v. This satisfies all the constraints of Ψ. The (relatively) difficult part is to argue soundness. To do this, we must show that for every x that satisfies 1 Ω Σ (tδ) constraints of Ψ, there is an x that satisfies 1 δ constraints of Ψ. The assignment x is constructed from x via the following procedure. For every vertex v, 1. Define the following distribution D v on vertices. Initially, set v v. Now repeat the following experiment: (1) With probability 1/t stop, and with the remaining probability, set v a random neighbor of v. 2. Set x v to equal the plurality value of x v (v), when v is chosen from D v, among those v that are within distance t of v. We now need to argue that if x satisfies 1 Ω(tδ/ Σ 4 ) constraints of Ψ, then x satisfies 1 δ constraints of Ψ. In fact, we will argue the contrapositive: Claim 5. Assume tδ < 1. If x violates δ constraints of Ψ, then x violates Ω(tδ/ Σ 4 ) constraints of Ψ. Before we prove the claim, let us make one simplification. We will modify the distribution over constraints of Ψ so that the path p is not truncated after 5t ln Σ steps (see step 2), but can be of any length. Intuitively, this simplification should not make a difference because long paths are unlikely. Formally, we will analyze the effect of this simplification later. Now let s explain the intuition behind this claim. Let F be the set of constraints of Ψ (which we also think of as edges of G) that are violated by x (so F = δm). Now take a random constraint ψ of Ψ. What are the chances that this constraint is violated by x? We have that Pr[x violates ψ ] = Pr[ψ intersects F ] Pr[x violates ψ ψ intersects F ].

7 7 Let s try to estimate both of these quantities. We expect ψ to have about 1/t edges; since F = δm, we expect ψ to contain about δt edges of F. Since δ is fairly small, we might expect that most ψ which intersect F intersect only a single edge of F. If this is the case, then Pr[ψ intersects F ] should be about δt. Roughly, this is where the soundness amplification happens: While bad edges occur in Ψ only with probability δ, they occur in Ψ with probability about tδ. What about the other probability? Let s now fix an edge (u, v) F that is contained in ψ. Now consider the distribution of the endpoints u and v of the path ψ. Since the endpoints of the path are determined by a Poisson process, it follows that conditioned on (u, v) being in ψ, the endpoint v is determined by the following distribution: Start from v and at each step (1) with probability 1/t stop and (2) with the remaining probability move to a random neighbor of v and continue. But this is exactly the distribution D v! Ignoring for now the fact that ψ could be too long, we reason as follows. Since the value x v was defined as the plurality value x v (v), the two should match with probability 1/ Σ. For the same reason, x u (u) and x u should match with probability 1/ Σ. But since the constraint ψ(x u, x v ) is violated, this implies ψ (x u, x v ) is also violated. So roughly, we expect that the probability that a random constraint of Ψ is violated is about δt/ Σ 2. However, our analysis ignored several crucial points, namely: Why can we assume that few ψ intersect multiple edges of F? What happens when ψ contains more than 1/t edges? In this case, it may happen that ψ contains a bad edge, but this edge cannot be seen from its endpoints. Roughly, the answer is: (1) The fact that ψ is unlikely to intersect many edges of F follows from the expansion of G and (2) Long paths will contribute little to the analysis as they only happen with small probability. 5 Analysis of gap amplification Now let us do the actual analysis. Call an edge (u, v) faulty (with respect to ψ, x, x) if (1) (u, v) F, (2) d(u, u), d(v, v ) < t, and (3) x u (u) = x u and x v (v) = x v, where u, v are the endpoints of ψ. If some edge in ψ is faulty, then ψ is violated as the inconsistency between x u and x v can be seen either by x u or by x v. Let N denote the number of faulty edges of ψ, where ψ is chosen at random. We have that Pr[ψ is violated] Pr[N > 0] E[N] 2 / E[N 2 ]. (1) The first moment. We first estimate E[N]. For f F, let N f denote the number of occurrences of f in ψ if f is faulty, and 0 otherwise. Then E[N] = f F E[N f ] f F Pr[N f > 0]. The probability that f = (u, v) occurs in ψ is exactly t/m. Conditioned on this, we analyze the probability that (u, v) is faulty. As discussed above, u follows the distribution D u, and v

8 8 independently follows the distribution D v. In this distribution, the probability that u is at distance more than t from u is (1 1/t) t < 1/2. Conditioned on this distance being at most t, the distribution on u is exactly the one used to define the plurality assignment x u, so the probability that x u (u) = x u is at least 1/ Σ. As the same is true for v and v independently, we have that and Pr[N f > 0] (t/m) ( 1 2 E[N] tδ 1 4 Σ 2. 1 ) 2 Σ The second moment. We now upper bound E[N 2 ]. To do so, let N be the number of edges in F that intersect ψ. Obviously N N (since N counts the number of such edges that are also faulty). So we will bound E[N 2 ] instead. To do so, let Z i be a random variable that indicates if the ith edge of ψ is in F (if ψ has fewer than i edges, then Z i = 0). Then E[N 2 ] = E[Z i ] + 2 E[Z i Z j ]. (2) i=1 1 i<j It is easily seen that E[Z i ] = δ (1 1/t) i, so the first summation is at most tδ. For the second summation, notice that E[Z i Z j ] is the probability that both edges i and j are present in the path and faulty. The probability they are both present is (1 1/t) j. Conditioned on them being both present, the probability they are both faulty is bounded using the following lemma. Lemma 6. Let G be a d-regular graph with spectral gap 1 λ and F be a subset of edges of G of density δ. Consider the following process: First, choose a random edge e of G, then take a random edge connecting to it, and so on for l steps. Call the final edge e. The probability that both e and e are in F is at most δ 2 + δλ l. It follows that E[Z i Z j ] (1 1/t) j δ (δ + λ j i ). Plugging this in (2) we have E[N 2 ] δt + 2δ E[Z i Z j ] δt + 2δ δt + 2δ 1 i<j (1 1/t) i (1 1/t) j i (δ + λ j i ) i=1 j=i+1 (1 1/t) i (δt + 1/(1 λ)) i=1 δt + 2δt(δt + 2) = 5δt + 2(δt) 2. Second moment calculation. Finally, from (1) we have: Pr[N > 0] E[N]2 E[N 2 ] (tδ/4 Σ )2 5δt + 2(δt) 2 = Ω(tδ/ Σ 4 ).

9 9 On the truncation of paths. This calculation was done in the idealized setting where ψ can be arbitrarily long, while it is actually restricted to have length at most 5t ln Σ. It is not hard to see that these long paths contribute little to N. In particular, the contribution from the long paths can be bounded by l=5t ln Σ E[N ψ has length l] Pr[ψ has length l] l=5t ln Σ (δl) (1 1/t) l < E[N]/2 For the calculation of E[N 2 ], the truncation of long paths only improves this quantity, so the lower bound on the probability that N > 0 is only affected by a constant. Proof of Lemma 6. Consider the graph H whose vertices correspond to the edges of G and where (e, e ) are adjacent in H every time they share an endpoint as edges of G. Then the process described above is simply a random walk of length l in H, or equivalently a random edge in H l. By the expander mixing lemma applied to H, the probability that both edges e and e fall inside F is upper bounded by δ 2 + λ(h l )δ = δ 2 + λ(h) l δ. It remains to see that λ(h) λ(g). In fact, we will show that all the eigenvalues of H are either 0 or eigenvalues of G. To see this, let λ be any eigenvalue of H, so that for some eigenvector v va H = λv. Here A H is the adjacency matrix of H. Let E be the incidence matrix of G: E is an m by n matrix where E e,v is 1 if vertex v is incident to edge e and 0 otherwise. Then A H = EE T, and Multiplying both sides by E we obtain vee T = λv. (ve)(e T E) = λ(ve) it follows that either ve = 0, so that λ = 0, or λ is an eigenvalue of E T E. But E T E = A G, the adjacency matrix of G.