Tool: Linearity Testing and the Walsh-Hadamard Code

Transcription

1 11.5 NP PCP(poly(n), 1): PCPfromtheWalsh-Hadamardcode 215 compute this mapping. We call reductions with these properties Levin reductions (see the proof of Theorem 2.18). It is worthwhile to observe that the PCP reductions of this chapter also satisfy this property. For example, the proof of Theorem actually yields awaynotjusttomap,say,acnfformulaϕ into a graph G such that ϕ is satisfiable iff G has a large independent set, but actually shows how to efficiently map a satisfying assignment for ϕ into a large independent set in G and a not-too-small independent set in G into a satisfying assignment for ϕ. This will become clear from our proof of the PCP Theorem in Chapter NP PCP(poly(n), 1): PCPfromtheWalsh-Hadamardcode We now prove a weaker version of the PCP Theorem, showing that every NP statement has an exponentially-long proof that can be locally tested by only looking at a constant number of bits. In addition to giving a taste of how one proves PCP Theorems, techniques from this section will be used in the proof of the full-fledged PCP theorem in Chapter 22. Theorem (Exponential-sized PCP system for NP [ALM + 92]) NP PCP(poly(n), 1) We prove this theorem by designing an appropriate verifier for an NP-complete language. The verifier expects the proof to contain an encoded version of the usual certificate. The verifier checks such an encoded certificate by simple probabilistic tests Tool: Linearity Testing and the Walsh-Hadamard Code We use the Walsh-Hadamard code (see also Section , though the treatment here is selfcontained). It is a way to encode bit strings of length n by linear functions in n variables over GF(2). The encoding function WH : {0, 1} {0, 1} maps a string u {0, 1} n to the truth table of the function x u x, whereforx, y {0, 1} n we define x y = n i=1 x iy i (mod 2). Note that this is a very inefficient encoding method: an n-bit string u {0, 1} n is encoded using WH(u) =2 n bits. If f {0, 1} 2n is equal to WH(u) forsomeu then we say that f is a Walsh-Hadamard codeword. Such a string f {0, 1} 2n can also be viewed as a function from {0, 1} n to {0, 1}. Below, we repeatedly use the following fact (see Claim A.31): random subsum principle: If u v then for 1 /2 the choices of x, u x v x. The random subsum principle implies that the Walsh-Hadamard codeisanerrorcorrecting code with minimum distance 1 /2, bywhichwemeanthatforeveryu v {0, 1} n, the encodings WH(u) and WH(v) differ in at least half the bits. Now we talk about local tests for the Walsh-Hadamard code (i.e., tests making only O(1) queries). Local testing of Walsh-Hadamard code. Suppose we are given access to a function f : {0, 1} n {0, 1} and want to test whether or not f is actually a codeword of Walsh- Hadamard. Since the Walsh-Hadamard codewords are precisely the set of all linear functions from {0, 1} n to {0, 1}, wecantestf by checking that f(x + y) =f(x)+f(y) (3)

2 PCP Theorem and Hardness of Approximation: An introduction for all the 2 2n pairs x, y {0, 1} n (where + on the left side of (3) denotes vector addition over GF(2) n and on the right side denotes addition over GF(2)). This test works by definition, but it involves reading all 2 n values of f. Can we test f by reading only a constant number of its values? The natural test is to choose x, y at random and verify (3). Clearly, even such a local test accepts a linear function with probability 1. However, now we can no longer guarantee that every function that is not linear is rejected with high probability! For example, if f is very close to being a linear function, meaning that f is obtained by modifying a linear function on a very small fraction of its inputs, then such a local test will encounter the nonlinear part with very low probability, and thus not be able to distinguish f from a linear function. So we set our goal less ambitiously: a test that on one hand accepts every linear function,and on the other hand rejects with high probability every function that is far from linear. The natural test above suffices for this job. Definition Let ρ [0, 1]. We say that f,g : {0, 1} n {0, 1} are ρ-close if Pr x R {0,1} n[f(x) = g(x)] ρ. We say that f is ρ-close to a linear function if there exists a linear function g such that f and g are ρ-close. Theorem (Linearity Testing [BLR90]) Let f : {0, 1} n {0, 1} be such that Pr x,y R {0,1} n[f(x + y) =f(x)+f(y)] ρ for some ρ > 1 /2. Thenf is ρ-close to a linear function. We defer the proof of Theorem to Section 22.5 of Chapter 22. For every δ (0, 1 /2), we can obtain a linearity test that rejects with probability at least 1 /2 every function that is not (1 δ)-close to a linear function, by testing Condition (3) repeatedly O(1/δ) times with independent randomness. We call such a test a (1 δ)-linearity test. Local decoding of Walsh-Hadamard code. Suppose that for δ < 1 4 the function f : {0, 1} n {0, 1} is (1 δ)-close to some linear function f. Becauseeverytwolinearfunctions differ on half of their inputs, the function f is uniquely determined by f. Suppose we are given x {0, 1} n and random access to f. Can we obtain the value f(x) usingonlya constant number of queries? The naive answer is that since most x s satisfy f(x) = f(x), we should be able to learn f(x) with good probability by making only the single query x to f. The problem is that x could very well be one of the places where f and f differ. Fortunately, there is still a simple way to learn f(x) whilemakingonlytwoqueriestof: 1. Choose x R {0, 1} n. 2. Set x = x + x. 3. Let y = f(x )andy = f(x ). 4. Output y + y. Since both x and x are individually uniformly distributed (even though they are dependent), by the union bound with probability at least 1 2δ we have y = f(x )and y = f(x ). Yet by the linearity of f, f(x) = f(x + x )= f(x )+ f(x ), and hence with at least 1 2δ probability f(x) =y + y. (We use here the fact that over GF(2), a + b = a b.) This technique is called local decoding of the Walsh-Hadamard code since it allows to recover any bit of the correct codeword (the linear function f) fromacorrupted version (the function f) whilemakingonlyaconstantnumberofqueries. Itisalsoknown as self correction of the Walsh-Hadamard code.

3 11.5 NP PCP(poly(n), 1): PCPfromtheWalsh-Hadamardcode Proof of Theorem We will show a (poly(n), 1)-verifier proof system for a particular NP-complete language L. The result that NP PCP(poly(n), 1) follows since every NP language is reducible to L. TheNP-complete language L we use is QUADEQ, thelanguageofsystemsofquadratic equations over GF(2) = {0, 1} that are satisfiable. Example The following is an instance of QUADEQ over the variables u 1,...,u 5 : u 1 u 2 + u 3 u 4 + u 1 u 5 =1 u 2 u 3 + u 1 u 4 =0 u 1 u 4 + u 3 u 5 + u 3 u 4 =1 This instance is satisfiable since the all-1 assignment satisfies all the equations. QUADEQ is NP-complete, as can be checked by reducing from the NP-complete language CKT-SAT of satisfiable Boolean circuits (see Section 6.1.2). The idea istohavea variable represent the value of each wire in the circuit (including the input wires) and to express AND and OR using the equivalent quadratic polynomial: x y =1iff (1 x)(1 y) =0, and so on. Details are left as Exercise Since u i =(u i ) 2 in GF(2), we can assume the equations do not contain terms of the form u i (i.e., all terms are of degree exactly two). Hence m quadratic equations over the variables u 1,...,u n can be described by an m n 2 matrix A and an m-dimensional vector b (both over GF(2)). Indeed, the problem QUADEQ can be phrased as the task, given such A, b, offindingann 2 -dimensional vector U satisfying (1) AU = b and (2) U is the tensor product u u of some n-dimensional vector u. 4 WH(u) WH(uOu) x Figure 11.2 The PCP proof that a set of quadratic equations is satisfiable consists of WH(u) andwh(u u) forsomevectoru. Theverifierfirstchecksthattheproofiscloseto having this form, and then uses the local decoder of the Walsh-Hadamard code to ensure that u is a solution for the quadratic equation instance. In the figure above the dotted areas represent corrupted coordinates. The PCP verifier. We now describe the PCP system for QUADEQ. Let A, b be an instance of QUADEQ and suppose that A, b is satisfiable by an assignment u {0, 1} n. The verifier V gets access to a proof π {0, 1} 2n +2 n2, which we interpret as a pair of functions f : {0, 1} n {0, 1} and g : {0, 1} n2 {0, 1}. In the correct PCP proof π for A, b, thefunctionf will be the Walsh-Hadamard encoding for u and the function g will be the Walsh-Hadamard encoding for u u. Thatis, wewilldesignthepcp verifier V in a way ensuring that it accepts proofs of this form with probability one, hence satisfying the completeness condition. The analysis repeatedly uses the random subsum principle. Step 1: Check that f, g are linear functions. As already noted, this isn t something that the verifier can check per se using local tests. Instead, the verifier performs a linearity test on both f,g, andrejectstheproofatonceifeithertestfails. Thus, if either of f,g is not close to a linear function, then V rejects with high probability. Therefore for the rest of the procedure we can assume that there exist two 4 If x, y are two n-dimensional vectors then their tensor product, denotedx y, isthen 2 -dimensional vector (or n n matrix) whose i, j th entry is x i y j (identifying [n 2 ]with[n] [n] insomecanonicalway). See also Section

4 PCP Theorem and Hardness of Approximation: An introduction linear functions f : {0, 1} n {0, 1} and g : {0, 1} n2 {0, 1} such that f is close to f, and g is close to g. (Note:inacorrectproof,thetestssucceedwithprobability 1 and f = f and g = g.) In fact, we will assume that for Steps 2 and 3, the verifier can query f, g at any desired point. The reason is that local decoding allows the verifier to recoveranydesiredvalueof f, g with good probability, and Steps 2 and 3 will only use a small (less than 20) number of queries to f, g. Thuswithhighprobability(say> 0.9) local decoding will succeed on all these queries. notation: To simplify notation and in the rest of the procedure we use f,g for f, g respectively. (This is OK since as argued above, V can query f, g at will.) In particular we assume both f and g are linear, and thus they must encode some strings u {0, 1} n and w {0, 1} n2.inotherwords,f,g are the functions given by f(r) =u r and g(z) =w z. Step 2: Verify that g encodes u u, where u {0, 1} n is the string encoded by f. The verifier does the following test ten times using independent random bits: Choose r, r independently at random from {0, 1} n,andiff(r)f(r ) g(r r )thenhaltandreject. In a correct proof, w = u u, so f(r)f(r )= u i r i u j r j = u i u j r i r j =(u u) (r r ), i [n] j [n] i,j [n] which in the correct proof is equal to g(r r ). Thus Step 2 never rejects a correct proof. Suppose now that, unlike the case of the correct proof, w u u. Weclaimthatineach of the ten trials V will halt and reject with probability at least 1 4.(Thustheprobabilityof rejecting in at least one trial is at least 1 (3/4) 10 > 0.9.) Indeed, let W be an n n matrix with the same entries as w, letu be the n n matrix such that U i,j = u i u j and think of r as a row vector and r as a column vector. In this notation, g(r r )=w (r r )= w i,j r i r j = rw r i,j [n] n n f(r)f(r )=(u r)(u r )=( u i r i )( u j r j)= u i u j r i r j = rur i=1 j=1 i,j [n] And V rejects if rw r rur. The random subsum principle implies that if W U then at least 1 /2 of all r satisfy rw ru. Applyingtherandomsubsumprincipleforeach such r, weconcludethatatleast 1 /2 the r satisfy rw r rur.weconcludethatthetrial rejects for at least 1/4 ofallpairsr, r. Step 3: Verify that f encodes a satisfying assignment. Using all that has been verified about f,g in the previous two steps, it is easy to check that any particular equation, say the kth equation of the input, is satisfied by u, namely, A k,(i,j) u i u j = b k. (4) i,j Denoting by z the n 2 dimensional vector (A k,(i,j) ) (where i, j vary over [1..n]), we see that the left hand side is nothing but g(z). Since the verifier knows A k,(i,j) and b k,itsimply queries g at z and checks that g(z) =b k. The drawback of the above idea is that in order to check that u satisfies the entire system, the verifier needs to make a query to g for each k =1, 2,...,m,whereasthenumber of queries is required to be independent of m. Luckily, we can use the random subsum principle again! The verifier takes a random subset of the equations and computes their sum mod 2. (In other words, for k =1, 2,...,m multiply the equation in (4) by a random bit and take the sum.) This sum is a new quadratic equation, and therandomsubsum principle implies that if u does not satisfy even one equation in the original system, then with probability at least 1/2 itwillnotsatisfythisnewequation. Theverifierchecksthat u satisfies this new equation.

5 Chapter notes and history 219 Overall, we get a verifier V such that (1) if A, b is satisfiable then V accepts the correct proof with probability 1 and (2) if A, b is not satisfiable then V accepts every proof with probability at most 0.8. The probability of accepting a proof for a false statement can be reduced to 1 /2 by simple repetition, completing the proof of Theorem Chapter notes and history The notion of approximation algorithms predates the discovery of NP-completeness; for instance Graham s 1966 paper [Gra66] alreadygivesanapproximationalgorithmforaschedulingprob- lem that was later proven NP-complete. Shortly after the discovery of NP-completeness, Johnson [Joh74] formalized the issue of computing approximate solutions, gave some easy approximation algorithms (such as the 1/2-approximation for MAX-SAT) for a variety of problems and posed the question of whether better algorithms exist. Over the next 20 years,althoughafewresultswere proven regarding the hardness of computing approximate solutions (such as for general TSP in Sahni and Gonzalez [SG76]) and a few approximation algorithms were designed, it became increasingly obvious that we were lacking serious techniques for proving the hardness of approximation. One of the problems seemed to be that there were no obvious interreducibilies among problems that preserved approximability. The paper by Papadimitriou andyannakakis[py88] showedsuch interreducibilities among a large set of problems they called MAX-SNP, and showed further that MAX-3SAT is complete for this class. This made MAX-3SAT an attractive problem to study both from point of view of algorithm design and for proving hardness results. Soon after this work, seemingly unrelated developments occurred in study of interactive proofs, some of which we studied in Chapter 8. The most relevant for the topicofthischapterwasthe result of Babai, Fortnow and Lund [BFL90] thatmip = NEXP, whichwassoonmadetoapplyto NP in the paper of Babai, Fortnow, Levin, and Szegedy [BFLS91]. After this there were a sequence of swift developments. In 1991 came a stunning result by Feige, Goldwasser, Lovasz, Safra and Szegedy [FGL + 91] whichshowedthatifsat does not have subexponential time algorithms then the INDSET problem cannot be approximated within a factor 2 log1 ϵ n for any ϵ > 0. This was the first paper to connect hardness of approximation with PCP-like theorems, though at the time many researchers felt (especially because the result did not prove NP-completeness per se) that this was the wrong idea and the result would ultimately be reproven with no mention of interactive proofs. (Intriguingly, Dinur s gap amplification lemma brings us closer to that dream.) However, a year later Arora and Safra [AS92] furtherrefinedtheideasof[bfl90] (togetherwiththeideaofverifier composition) to prove that approximating INDSET is actually NP-complete. They also proved a surprising new characterization of NP in terms of PCP, namely,np = PCP(log n, log n). At that point it became clear that if the query parameter could be sublogarithmic, it might well be made a constant! The subsequent paper of Arora, Lund, Motwani, Sudan, and Szegedy [ALM + 92] took this next step (in the process introducing the constant-bit verifier of Section 11.5, as well as other ideas) to prove NP = PCP(log n, 1), which they showed also implied the NP-hardness of approximating MAX-3SAT. SincethenmanyotherPCP theorems have been proven, as surveyed in Chapter 22. (Note that in this chapter we derived the hardness result for INDSET from the result for MAX-3SAT, eventhoughhistoricallytheformerhappenedfirst.) The overall idea in the AS-ALMSS proof of the PCP Theorem (as indeed the one in the proof of MIP = NEXP) issimilartotheproofoftheorem InfactTheorem11.19 is the only part of the original proof that still survives in our writeup; therestoftheproofinchapter22isa more recent proof due to Dinur. However, in addition to using encodings based upon the Walsh- Hadamard code the AS-ALMSS proof also used encodings based upon low degree multivariate polynomials. These have associated procedures analogous to thelinearitytestandlocaldecoding, though the proofs of correctness are a fair bit harder. The proof also drew intuition from the topic of self-testing and self-correcting programs [BLR90, RS92]. The PCP Theorem led to a flurry of results about hardness of approximation. See Trevisan [Tre05]forarecentsurveyandArora-Lund[AL95] foranolderone. The PCP Theorem, as well as its cousin, MIP = NEXP, doesnotrelativize[frs88]. In this chapter we only talked about some very trivial approximation algorithms, which are unfortunately not very representative of the state of the art. See Hochbaum [Hoc97] andvazirani [Vaz01] for a good survey of the many ingenious approximation algorithms that have been developed.

6 Proofs of PCP Theorems and the Fourier Transform Technique 22.5 Tool: the Fourier transform technique Theorem is proved using Fourier analysis. The continuous Fourier transform is extremely useful in mathematics and engineering. Likewise, the discrete Fourier transform has found many uses in algorithms and complexity, in particular for constructing and analyzing PCP s. The Fourier transform technique for PCP s involves calculating the maximum acceptance probability of the verifier using Fourier analysis of the functions presented in the proof string. (See Note for a broader perspective of uses of discrete Fourier transforms in combinatorial and probabilistic arguments.) It is delicate enough to give tight inapproximability results for MAX-INDSET, MAX-3SAT, and many other problems. To introduce the technique we start with a simple example: analysis of the linearity test over GF(2) (i.e., proof of Theorem 11.21). We then introduce the Long Code and show how to test for membership in it. These ideas are then used to prove Håstad s 3-bit PCP Theorem Fourier transform over GF(2) n The Fourier transform over GF(2) n is a tool to study functions on the Boolean hypercube. In this chapter, it will be useful to use the set {+1, 1} = {±1} instead of {0, 1}. To transform {0, 1} to {±1}, weusethemappingb ( 1) b (i.e., 0 +1, 1 1). Thus we write the hypercube as {±1} n instead of the more usual {0, 1} n.notethismapsthexor operation (i.e., addition in GF(2)) into the multiplication operationoverr. The set of functions from {±1} n to R defines a 2 n -dimensional Hilbert space (i.e., a vector space with an associated inner product) as follows. Addition and multiplication by ascalararedefinedinthenaturalway:(f + g)(x) =f(x)+g(x) and(αf)(x) =αf(x) for every f,g : {±1} n R, α R. Wedefinetheinnerproductoftwofunctionsf,g, denoted f,g, tobee x {±1} n[f(x)g(x)]. (This is the expectation inner product.) The standard basis for this space is the set {e x } x {±1} n,wheree x (y) isequalto1ify = x, andequalto0otherwise.thisisanorthogonalbasis,andevery function f : {±1} n R can be represented in this basis as f = x a xe x.foreveryx {±1} n,thecoefficient a x is equal to f(x). The Fourier basis is an alternative orthonormal basis that contains, for every subset α [n], a function χ α where χ α (x) = i α x i. (We define χ to be the function that is 1 everywhere). This basis is actually the Walsh-Hadamard code (see Section ) in disguise: the basis vectors correspond to the linear functions over GF(2). To see this, note that every linear function of the form b a b (with a, b {0, 1} n )ismappedbyour transformation to the function taking x {±1} n to i s.t. a x i=1 i.tocheckthatthefourier basis is indeed an orthonormal basis for R 2n,notethattherandomsubsumprincipleimplies that for every α, β [n], χ α, χ β = δ α,β where δ α,β is equal to 1 iff α = β and equal to 0 otherwise. Remark Note that in the { 1, 1} view, the basis functions can be viewed as multilinear polynomials (i.e., multivariate polynomials whose degree in each variable is 1). Thus the fact that every real-valued function f :{ 1, 1} n has a Fourier expansion can also be phrased as Every such function can be represented by a multilinear polynomial. This is very much in the same spirit as the polynomial representations used in Chapters 8 and 11. Since the Fourier basis is an orthonormal basis, every function f : {±1} n R can be represented as f = α [n] ˆf α χ α.wecallˆf α the α th Fourier coefficient of f. Wewilloften use the following simple lemma: Lemma Every two functions f,g:{±1} n R satisfy 1. f,g = α ˆf α ĝ α. 2. (Parseval s Identity) f,f = α ˆf 2 α.

7 22.5 Tool: the Fourier transform technique 411 Proof: The second property follows from the first. To prove the first we expand f,g = α ˆf α χ α, β ĝ β χ β = α,β ˆf α ĝ β χ α, χ β = α,β ˆf α ĝ β δ α,β = α ˆf α ĝ α Example Some examples for the Fourier transform of particular functions: 1. The majority function on 3 variables (i.e., the function MAJ(u 1,u 2,u 3 ) that outputs +1 if and only if at least two of its inputs are +1, and 1 otherwise) can be expressed as 1 /2u /2u /2u 3 1 /2u 1 u 2 u 3. Thus, it has four Fourier coefficients equal to 1 /2 and the rest are equal to zero. 2. If f(u 1,u 2,...,u n )=u i (i.e., f is a coordinate function, aconceptwewill see again in context of long codes) then f = χ {i} and so ˆf {i} =1and ˆf α =0 for α {i}. 3. If f is a random Boolean function on n bits, then each ˆf α is a random variable that is a sum of 2 n binomial variables (equally likely to be 1, 1) and hence looks like a normally distributed variable with standard deviation 2 n/2 and mean 0. Thus with high probability, all 2 n Fourier coefficients have values in [ poly(n), poly(n) ]. 2 n/2 2 n/ The connection to PCPs: High level view In the PCP context we are interested in Boolean-valued functions, i.e., those from GF (2) n to GF (2). Under our transformation they turn into functions from {±1} n to {±1}. Thus, we say that f : {±1} n R is Boolean if f(x) {±1} for every x {±1} n. Note that if f is Boolean then f,f = E x [f(x) 2 ]=1. On a high level, we use the Fourier transform in the soundness proofs for PCP s to show that if the verifier accepts a proof π with high probability then π is close to being wellformed (where the precise meaning of close-to and well-formed is context dependent). Usually we relate the acceptance probability of the verifier to an expectation of the form f,g = E x [f(x)g(x)], where f and g are Boolean functions arising from the proof. We then use techniques similar to those used to prove Lemma to relate this acceptance probability to the Fourier coefficients of f,g, allowing us to argue that if the verifier s test accepts with high probability, then f and g have few relatively large Fourier coefficients. This will provide us with some nontrivial useful information aboutf and g, since in a generic or random function, all the Fourier coefficient are small and roughly equal Analysis of the linearity test over GF (2) We will now prove Theorem 11.21, thus completing the proof of the PCP Theorem. Recall that the linearity test is provided a function f :GF(2) n GF(2) and has to determine whether f has significant agreement with a linear function. To do this itpicksx, y GF(2) n randomly and accepts iff f(x + y) =f(x)+f(y). Now we rephrase this test using {±1} instead of GF(2), so linear functions turn into Fourier basis functions. For every two vectors x, y {±1} n,wedenotebyxy their componentwise multiplication. That is, xy =(x 1 y 1,...,x n y n ). Note that for every basis function χ α (xy) =χ α (x)χ α (y). For two Boolean functions f,g, theirinnerproduct f,g is equal to the fraction of inputs on which they agree minus the fraction of inputs on which they disagree. It follows