PCP Theorem And Hardness Of Approximation For MAX-SATISFY Over Finite Fields

Transcription

1 MM Research Preprints KLMM, AMSS, Academia Sinica Vol. 7, 1 15, July, PCP Theorem And Hardness Of Approximation For MAX-SATISFY Over Finite Fields Shangwei Zhao Key Laboratory of Mathematics Mechanization Institute of Systems Science, AMSS, Academia Sinica, Beijing, , China Abstract. This paper will survey the PCP theorem and its application in solving the MAX-SATISFY. For the problem MAX-SATISFY over F, a random assignment satisfies half the equations and thus yields a -approximation algorithm. It is the best possible approximation algorithm in polynomial time based on P NP, even if each equation only contains exactly three variables, which can be proved by the 3-bit PCP theorem due to Johan Håstad. Similar results can be extended to MAX-SATISFY over finite fields. 1. Introduction Consider the problem MAX-SATISFY over F, that is, given a system of m linear equations with n variables over F, find a solution satisfying the maximal number of equations. This is an optimization problem and is NP complete, which shows that it is hard to find its exact solutions unless P = NP. Hence, it is meaningful to consider the approximation algorithms of this problem. Since for each equation, the probability of a random assignment satisfying the equation is 1/, a random assignment for the system of equations will satisfies half of the equations. Therefore, we can get an approximation algorithm with approximation ratio. The PCP theorem is one of the most important results in complexity in recent years and it provides a tool to prove the inapproximability result of optimization problems [, 1]. In this paper, we will introduce the basic theory of PCP and its application to the inapproximability results. Based on the basic PCP theorem, Johan Håstad gives a 3-bit PCP theorem, which can be used to prove that the -approximation algorithm is the best possible polynomial time approximation algorithm for MAX-SATISFY over F unless P = NP [3]. Precisely, he proves that it is NP-hard to approximate MAX-SATISFY over F. He also shows that it is NP-hard to approximate MAX-SATISFY over F p with approximation ratio p ɛ for arbitrary ɛ > 0. We will give a brief introduction to these results.. Basic definitions First, we will give some definitions about approximation algorithms of optimization problems. Definition.1 Let O be a maximization problem and let r 1 be a real number. For an instance x of O let OPT(x) be the optimal value. An r-approximation algorithm is an

2 S. Zhao algorithm that on each input x outputs a number cost(x,s) such that OP T (x)/r cost(x, s) OP T (x), where s is a feasible solution. Definition. Let O be a minimization problem and let r 1 be a real number. For an instance x of O let OPT(x)be the optimal value. An r-approximation algorithm is an algorithm that on each input x outputs a number cost(x,s) such that OP T (x) cost(x, s) r OP T (x), where s is a feasible solution. Definition.3 An efficient r-approximation algorithm is an r-approximation algorithm that runs in worst-case polynomial time. We can also use the formulation having performance ratio r or approximation ratio r instead of saying being an r-approximation algorithm. Definition.4 An optimization problem O is said to have a polynomial-time approximation scheme, if there exists a variety of approximation algorithms {A k } such that for every fixed k 1, A k is a (1 + 1 k )-approximation polynomial-time algorithm of O. Further, if there is a common polynomial p(n, k) such that for each x, the time of A k (x) is bounded by p(n, k),where n = x, the problem is called to have fully polynomial approximation scheme. Definition.5 An optimization problem O is said to be hard to approximate within a factor of r if the existence of an r-approximation algorithm for O implies P=NP. Usually, we can divide the approximation results into three classes: 1. For any constant r > 1, it is NP-hard to approximate O with approximation ratio r.. There is a constant r 0 > 1 such that for r > r 0, O has a polynomial-time r- approximation algorithm while for 1 < r < r 0, it is NP-hard to approximate O with approximation ratio r. 3. For any constant r > 1, O has a polynomial-time r-approximation algorithm. The PCP theorem in the next section is widely used in the research of the second class. 3. PCP Theorem 3.1. PCP Theorem PCP is short for Probabilistically Checkable Proofs, and the PCP Theorem provides an interesting new characterization for NP. Let a verifier be a polynomial-time probabilistic Turing machine containing an input tape, a work tape, a tape that contains a random string, and a tape called the proof string denoted by π. The proof string should be thought of as an array of bits, out of which the verifier will examine a few. The verifier works as follows. First, it reads the input and the random string, and writes down on its work tape some addresses of locations in the proof string π. Next, it examines the bits in those locations in π. The process of reading a bit from π is called a query. Finally, the verifier decides to accept or reject, based on what the input, the random string, and the queried bits of π were.

3 PCP Theorem and hardness of approximation of MAX-SATISFY over finite fields 3 Definition 3.1 A verifier is (r(n),q(n))-restricted if on each input of size n it uses at most O(r(n)) random bits for its computation, and queries at most O(q(n)) bits of the proof. In other words, an (r(n),q(n))-restricted verifier has two associated integers c, k. The random string has length c r(n). The verifier operates as follows on an input of size n. It reads the random string R, computes a sequence of k q(n) locations i 1 (R), i (R),..., i k q(n) (R), and queries those locations in π. Depending on what these bits were, it accepts or rejects. Define M π (x, R) to be 1 if M accepts input x, with access to the proof π, using a string of random bits R, and 0 otherwise. Definition 3. A verifier M probabilistically checks membership proofs for language L if 1.(completeness) For every input x L, there is a proof π x that causes M to accepts for every random string, that is, πp r R [M π (x, R) = 1] = 1..(soundness) For any input x / L, every proof π is rejected with probability at least 1, that is, π, P r R [M π (x, R) = 1] 1 Definition 3.3 A language L is in PCP(r(n),q(n)) if there is an (r(n),q(n))-restricted verifier M that probabilistically checks membership proofs for L. According to the usual definition, the complexity class NP is the class of languages that can be verified by a deterministic polynomial-time Turing machine M (called verifier ). That is, x L πs.t.m π (x, π) = 1; x / L πs.t.m π (x, π) = 0. Theorem 3.4 (PCP) NP=PCP(logn,1) The theorem gives a novel characterization of NP[]. Its proof is complicated, and we will not give it here. The soundness of the PCP verifier in definition. is 1/, which can be reduced to an arbitrary small positive constant by increasing the number of queries. If the original soundness is 1 δ with q queries, then we can obtain soundness (1 δ) l by making lq queries. However, we need small soundness with small number of queries sometimes. For example, in the following Theorem 4., we need only three queries to the proof string. The above verifier called sequential verifier does not work. Raz has shown that it can be achieved by parallel repetition of PCPs at the expanse of increasing the size of the alphabet of proof string. Definition 3.5 Let S be a finite set. Let V be a PCP verifier using alphabet S and l N. The l-times parallel repeated V is the verifier M as follows: 1. M uses the alphabet S l and we denote the input proof string of M by ˆπ.. Let q denote the number of queries V makes. On input x, M chooses l independent random strings r 1,..., r l for V, and runs V on the input and the random strings to obtain l

4 4 S. Zhao sets of q queries as follows: i 1 1, i 1,..., i 1 q i 1, i,..., i q... i l 1, i l,..., i l q. 3. M makes q queries i 1,..., i q to the proof string ˆπ where i j is (i 1 j,..., il j ); 4. For j [q], denote (a 1 j,..., al j ) = ˆπ(i j). The verifier M accepts if and only if for every k [l], the verifier V on random string r k accepts when given the responses a k 1,..., ak q. In the following text, we denote the above verifier V RAZ. From the definition we can see that the parallel verifier accepts with probability one when the original always accepts. Raz gives the following theorem about the soundness parameter. Theorem 3.6 There exist constants a and b such that the soundness parameter of M is at most (1 δ a ) bl. 3.. The equivalent formulation of PCP Theorem Definition 3.7 Let q, W be natural numbers. A qcsp W instance ϕ is a collection of functions ϕ 1,... ϕ m (called constraints) from {0,..., W 1} n to {0, 1} such that each function ϕ i depends on at most q of its input locations. That is, for every i [m] there exist j 1,..., j q [n] and f: {0,..., W 1} q {0, 1} such that ϕ i (u) = f(u j1,..., u jq ) for every u {0,..., W 1} n, where [m] = {1,,..., m} and [n] = {1,,..., n}. We say that an assignment u {0,..., W 1} n satisfies constraint ϕ i if ϕ i (u) = 1. The fraction of constraints satisfied by u is Σm i=1 ϕ i(u) m, and we let opt(ϕ) denote the maximum of this value over all u {0,..., W 1} n. We say that ϕ is satisfiable if opt(ϕ)=1. We call q the arity of ϕ and W the alphabet size. If W = we call ϕ a qcsp -instance. It is easy to see that 3SAT is a special case of constraint satisfaction problems. Definition 3.8 Let ρ (0, 1), the ρ-gap qcsp W problem is to determine, given an instance ϕ of qcsp W whether: 1. opt(ϕ) = 1, in which case we say ϕ is a yes instance of ρ-gap qcsp W.. opt(ϕ) ρ, in which case we say ϕ is a no instance of ρ-gap qcsp W. Also, we can define the problem ρ-gap 3SAT. It is a special case of ρ-gap 3CSP. Definition 3.9 Let ρ (0, 1) and L be a language. If there is a polynomial-time computable function f such that x L f(x) is a yes instance of ρ-gap qcsp W ; x / L f(x) is a no instance of ρ-gap qcsp W. We say that L is polynomial-time reducible to ρ-gap qcsp W, denoted by L p ρ- GAP qcsp W ; If for every language L NP, L p ρ-gap qcsp W, We say that ρ-gap qcsp W is NP-hard.

5 PCP Theorem and hardness of approximation of MAX-SATISFY over finite fields 5 In fact, the PCP Theorem is equivalent to the NP-hardness of a certain gap version of qcsp. Theorem 3.10 There exist constant q N and ρ (0, 1) such that ρ-gap qcsp is NPhard. Proof: Assume that NP P CP (logn, 1). We will show that 1/-GAPqCSP is NP-hard for some constant q. Since 3SAT NP, 3SAT has a PCP system in which the verifier M makes a constant number of queries, which we denote by q, and uses c logn random bits for some constant c. Given every input x and r {0, 1} c logn, define M x,r to be the function that on input a proof π outputs 1 if the verifier will accept the proof π on input x and random bits r. Note that M x,r depends on at most q locations. Thus for every x {0, 1} n, the collection ϕ = {M x,r } r {0,1} c logn is a polynomial-sized qcsp instance. And it is easy to see the transformation can be carried in polynomial-time. By the completeness and soundness of the PCP system, if x 3SAT then ϕ will satisfy opt(ϕ)=1, while x / 3SAT then opt(ϕ) 1/. Theorem 3.11 If there exist ρ (0, 1) and q > 0 such that ρ-gap qcsp is NP-hard, then NP = P CP (logn, 1). Proof: Assume there exist ρ (0, 1) and q > 0 such that ρ-gap qcsp is NP-hard, then for every language L NP, there is a polynomial-time computable function f such that x L opt(ϕ) = 1; x / L opt(ϕ) ρ. Given an input x, the verifier will run the reduction f(x) to obtain a qcsp instance ϕ = {ϕ i } m i=1, where ϕ i = ϕ i (x i1,..., x iq ), and i 1,..., i q [n], m = n a. A PCP verifier M 0 of ρ-gap qcsp is as follows: 1. Input an instance x and proof string π 0 ;. Transform x to f(x)=ϕ; 3. Choose i [m] randomly; 4. Query q bits from π 0 to obtain a i1,..., a iq ; 5. Output ϕ i (a i1,..., a iq ). Since ρ (0, 1), there exists k 1 such that ρ k 1/. Then we can construct another verifier M based on M 0 : 1. Input an instance x and proof string π;. Transform x to f(x)=ϕ; 3. Repeat the following operation for k times: (1).Choose i [m] randomly; ().Query q bits from π 0 to obtain a i1,..., a iq ; (3).Output 0 if ϕ i (a i1,..., a iq ) = 0 and halt; 4. Output 1 and halt. M simulates M 0 for k times in step 3, and it outputs 1 if and only if the operations in step 3 all output 1. Thus, x L πs.t.p r R [M π (x, R) = 1] = 1; x / L πs.t.p r R [M π (x, R) = 1] ρ k 1/.

6 6 S. Zhao M uses ka logn random bits and queries at most kq locations of the proof string. Therefore, L P CP (logn, 1), and then N P P CP (logn, 1). The inverse inclusion is obvious, so the PCP Theorem holds. Since 3SAT is a special case of 3CSP instances, that ρ-gap 3SAT is NP-hard implies that ρ-gap qcsp is NP-hard. The following theorem shows that the former can also be reduced from the later. Theorem 3.1 There exists constant ρ (0, 1) such that ρ-gap 3SAT is NP-hard. That is, there is a polynomial-time computable function fsuch that L NP, Where f(x) is an instance of ρ-gap 3SAT. x L opt(f(x)) = 1; x / L opt(f(x)) ρ. Proof: 1/-GAP qcsp is NP-hard by theorem 3.10 and it suffices to show a reduction from 1/-GAP qcsp to ρ-gap 3SAT where ρ (0, 1). That is, we will show a polynomial-time function mapping yes instances of 1/-GAP qcsp to yes instances of ρ-gap 3SAT and no instances of 1/-GAP qcsp to no instances of ρ-gap 3SAT. Let ϕ = {ϕ 1,..., ϕ m } with variables x 1,..., x n be an instance of 1/-GAP qcsp, then ϕ qcsp opt(ϕ) = 1 and ϕ / qcsp opt(ϕ) 1/. For every i [m], ϕ i can be expressed as at most q clauses, where each clause is CNF with at most q variables. So, ϕ i = j [ q ]C i,j. Assume q > 3, we can transform each clause to q clauses, each of which contains at most 3 variables, by introducing q 3 additional auxiliary variables. For example, for C = L 1 L q, introduce new variables y 1,, y q 3 to create a collection of clauses: (L 1 L y 1 ), ( y 1 L 3 y ),..., ( y q 4 L q y q 3 ), ( y q 3 L q 1 L q ). So, ϕ = i [m] j [ q ] k [q ] Ci,j k, and there are m(q )q clauses with n + m(q 3) q variables in ϕ. It is easy to check that opt(ϕ )=1 when opt(ϕ)=1 and opt(ϕ ) 1 1 (q ) q+1 opt(ϕ) 1/. Let ρ = 1 1 (q ) q+1, then the conclusion follows. when As stated above, the three theorems (Theorem 3.4, Theorem 3.10 and Theorem 3.1) are equivalent. Theorem 3.1 means that it is NP-hard to tell whether opt(ϕ) = 1 or opt(ϕ) ρ where ϕ is an instance of MAX3SAT. Hence it is NP-hard to approximate MAX3SAT with approximation factor 1 ρ. From this, using expander graphs it can be shown that it is NP-hard to tell whether opt(ψ) = 1 or opt(ψ) c where ψ is an instance of MAX3SAT with restriction that each variable is in exactly 5 clauses. We say that c GAP 3SAT (5) is NP-hard. Next we will show instances of MAX3SAT(5) can be reduced to instances of Label Cover. An instance of Label Cover is denoted by L(G(V, W, E), [M], [N], {π vw } (v,w) E ) where 1.G(V,W,E) is a regular bipartite graph.. [M],[N] are sets of labels, and V,W get labels from [N],[M] respectively. 3.{π vw } (v,w) E denote the constraints on each edge. For every edge (v, w) in E, we have a map π v,w : [M] [N].

7 PCP Theorem and hardness of approximation of MAX-SATISFY over finite fields 7 A labeling l : V [N], W [M] satisfies an edge (v, w) if π vw (l(w)) = l(v). Given an instance L, the goal is to find a labeling which satisfies the maximum fraction of edges, denoted by opt(l). Theorem 3.13 It is NP-hard to tell whether L(G(V, W, E), [7], [], {π vw } (v,w) E ) satisfies opt(l) = 1 or opt(l) c for some constant c < 1 where L is a label cover instance. Proof: Given an instance ψ of MAX3SAT(5) we construct an instance L ψ of Label Cover as follows. Let ψ have variables {x 1,..., x n } and clauses {C 1,..., C m }. Then the graph G ψ (V, W, E) has V := {x 1,..., x n } and W := {C 1,..., C m }. Also (x i, C j ) E if and only if x i C j. Note that the degree of a vertex in V is 3 while that in W is 3. Hence it is a regular bipartite graph. Since V gets labels from [] we can assume it to be from the set {0, 1}. Let [7] denote the seven satisfying assignments that a clause C j W can have in some order and π xi,c j : [7] [] be the assignment of variable x i be the satisfying assignment. For instance, if the clause C j were x 1 x x 3 and the third satisfying assignment were x 1 = 1, x = 0, x 3 = 1,, then π x1,c j (3) = 1 while π x,c j (3) = 0. We will see that if opt(ψ) = 1, then opt(l ψ ) = 1, and if opt(ψ) c then opt(l ψ ) 1 1 c 3. If opt(ψ) = 1, then there is an assignment which satisfies all the clauses. Thus labeling x i as per the satisfying assignment, and giving C j the label corresponding to the satisfying assignment, we see that we can satisfying all the edges. Hence opt(l ψ ) = 1. When opt(ψ) c, consider any labeling l of L. Since the labeling of V can be thought of as an assignment, at least 1 c fraction of clauses C j are unsatisfied by this assignment. Pick one such clause C j = x 1 x x 3. Since C j is unsatisfied, whatever label it might have, one of its three neighbors must be violating the constraints, otherwise the clause would be satisfied. Thus at least 1 c 3 of the edges are violated and hence opt(l ψ ) 1 1 c 3. Let c = 1 1 c 3 and we complete the proof. 4. MAX-SATISFY over F Given a system of m linear equations with n variables over F, find a solution satisfying the maximal number of equations. We call it MAX-SATISFY over F, denoted by Max-LIN-. If each equation contains exactly k variables, we denote it by Max-Ek-LIN-. Theorem 4.1 Max-LIN- is NP complete. Proof: MAXSAT is NP complete and it suffices to show that there is a reduction from MAXSAT to MAX-LIN- in polynomial time. Let C = C 1 C C l, where C i = α i β i, i = 1,..., l. For each clause C i = α i β i, we construct three equations C i : α i + β i = 1, α i = 1, β i = 1, It is easy to see that C i = 1 if and only if two equations of C i hold. Hence k equations of C are satisfied if and only if k equations of C i are satisfied. This reduction can be done in polynomial time and so the conclusion follows. (1)

8 8 S. Zhao The theorem shows that it is hard to find exact solutions of MAX-LIN- unless P = NP, and so it is natural and important to consider the approximation algorithm of the problem. The PCP Theorem can be used to prove that many NP optimization problems can not be approximated in polynomial time within some precision with the assumption P NP. But it does not imply how well can we approximate the problems. From theorem 3.1, we can see that it is NP-hard to approximate MAX3SAT with approximation ratio 1 1, which is close to one. And we know that there is a random polynomial-time 8/7-1 (q ) q+1 approximation algorithm for MAX3SAT. So, a natural question is: is there a ρ-approximation polynomial-time algorithm, where ρ lies between the two values? Therefore, we need some stronger PCP Theorems. The following 3-bit PCP Theorem due to Johan Håstad gives some tight inapproximability results about the problems MAX-SATISFY over finite fields and MAX3SAT. Theorem 4. For every ɛ > 0 and every language L NP there is a PCP-verifier for L making three queries having completeness at lest 1 ɛ and soundness at most 1/ + ɛ. Moreover, given a proof π, V chooses a triple (i 1, i, i 3 ) and b {0, 1} according to some distribution and accepts if and only if π i1 + π i + π i3 = b (mod ). Corollary 4.3 It is NP-hard to approximate Max-E3-LIN- with approximation ratio ε for every ε > 0. The result is tight since a random assignment is expected to satisfy half of the equations. Moreover, we have the following corollarys. Corollary 4.4 It is NP-hard to approximate Max-Ek-LIN- with approximation ratio ε for every ε > 0 and k 3. Proof: We will reduce instances of Max-E3-LIN- to instances of Max-Ek-LIN-. Given a system of equations with variables (x i ) n i=1 and 3 variables in each equation. Add the same k 3 new variables (y i ) i=1 k 3 in every equation to make them all have k variables. If k 3 i=1 y i = 1, then the assignment to the larger system satisfies the same equations in the smaller system while if k 3 i=1 y i = 1 then it satisfies exactly the equations not satisfied in the larger system. Changing every x i to its negation, now the assignment satisfies the equations that are satisfied in the larger system. Therefore, the maximal number of equations is preserved and it is easy to translate a solution of the larger system to a solution of the smaller system. And it is done in polynomial time. Corollary 4.5 (8/7 η)-approximation for MAX3SAT is NP-hard for every η > 0. Proof: we reduce instances of Max-E3-LIN- to instances of MAX3SAT. The linear equation over F a + b + c = 0 is equivalent to four clauses (ā b c), (a b c), (a b c) and (ā b c). It is similar for the equation a + b + c = 1. We can see that the linear equation is satisfied if and only if all of the clauses are satisfied, and the equation is unsatisfied then three of the clauses are unsatisfied. Therefore, if the fraction of the satisfied equations is at least 1 ɛ so are the clauses and if the fraction of satisfied equations is at most 1/ + ɛ, the fraction for the clauses is at most 1 (1/ ɛ) 1/4 = 7/8 ɛ/4. 1 ɛ 7/8 ɛ/4 = 8/7 η, where

9 PCP Theorem and hardness of approximation of MAX-SATISFY over finite fields 9 η > 0 and is arbitrary. Then the conclusion holds. Now it suffices to prove Theorem 4.. We can construct a PCP verifier starting from an instance of Label Cover L(G(V, W, E), [M], [N], {π vw } (v,w) E ). Let the labeling to instances L as the proof string and the verifier works as follows: pick one edge (v, w) E at random and read the labels l(v), l(w) from the proof. It accepts if and only if π(l(w)) = l(v). This PCP is called Outer PCP. We can see from theorem 3.13 that the completeness is 1 and soundness is c. In fact, we can make a parallel verifier which make the soundness parameter arbitrary small. We call it V RAZ. Suppose the new alphabet for V is [n] and for W is [m], obviously m > n, and m, n are both polynomials of the input. Hence it is NP-hard to tell whether opt(l) = 1 or opt(l) ɛ for arbitrary ɛ > 0. Clearly, we use O(logmn) random bits and two queries. One query is from the alphabet [n] and one from [m], and hence the total number of bits queried is logm + logn. This is far from the desired goal of three query bits. We need to construct a new PCP verifier based on the Outer PCP V RAZ. The new proof string contains the encodings of the labels of the vertices instead of the labels of the vertices. It is convenient for us to to work over {1, 1} instead of the standard set {0, 1}. We use the mapping b ( 1) b (that is 0 1 and 1 1). It maps the addition operation in {0, 1} to multiplication operation in {1, 1}. The operation over the set of functions from {1, 1} n to {1, 1} is defined as follows: (f + g)(x) = f(x) + g(x), (αf)(x) = αf(x) and the inner product f, g = E x {1, 1} n[f(x)g(x)]. And the Fourier basis of the function space is the set {χ α } α [n] where χ α (x) = Π i α x i (χ = 1). These correspond to the linear equations over F and the basis is an orthonormal basis since χ α, χ β = δ α,β. Every function in the above function set can be represented as f = Σ α [n] ˆfα χ α. We call ˆfα Fourier coefficient of f. It is easy to see that f, g = Σ α ˆfα ĝ α. Expand the left of the formula f, g = Σ α ˆfα χ α, Σ β ĝ β χ β = Σ α,β ˆfα ĝ β δ α,β = Σ α ˆfα ĝ α. Next, we give the definition of long code which is used in the new verifier. Definition 4.6 We say that f : {±1} m {±1} is a coordinate function if there is some i [m] such that f(x 1, x,..., x m ) = x i, that is, f = χ {i}. Definition 4.7 The long code for [m] encodes each i [m] by the table of all values of the function χ {i} : {±1} m {±1}. Hence the long code for l(v) [n] is the truth table of χ {l(v)} : {±1} n {±1}, and the long code for l(w) [m] is the truth table of χ {l(w)} : {±1} m {±1}. Given a function f : {±1} m {±1}, we want to know whether it is long code, that is whether f has good agreement with χ {i} for some i. By the definition of the inner product, f, g = ɛ if and only if f has agreement 1/ + ɛ/ with g. Since f, χ α = ˆf α, f has better agreement with the linear equation if it has larger Fourier coefficient. As to the proof of Theorem 4., we need add some noise to the test.

10 10 S. Zhao Long code test: Input: a string of length m as a function f : {±1} m {±1}. Desired property: The function f should be a long code, that is, there exists i m such that f = χ {i}. Verifier: 1. Pick two random vectors x, y {±1} m with uniform probability;. Pick a vector z {±1} m according to the following distribution: for every i [m] choose z i = 1 with probability 1 ρ and choose z i = 1 with probability ρ; 3. It accepts iff f(x)f(y) = f(xyz). Note: For every two vectors x, y {±1} m, denote by xy their componentwise multiplication. Suppose f = χ {i}, then f(x)f(y)f(xyz) = x i y i (x i y i z i ) = z i Hence the test accepts iff z i = 1 which has probability 1 ρ. Lemma 4.8 If the test accepts with probability at least 1/ + ɛ then Σ α ˆf 3 α (1 ρ) α ɛ. Proof: Since the acceptation probability is 1/ + ɛ then E[f(x)f(y)f(xyz)] (1/ + ɛ) (1/ ɛ) = ɛ. Replacing f by its Fourier expansion, we obtain E x,y,z [(Σ α ˆfα χ α (x)) (Σ β ˆfβ χ β (y)) (Σ γ ˆfγ χ γ (xyz))] = E x,y,z [Σ α,β,γ ˆfα ˆfβ ˆfγ χ α (x)χ β (y)χ γ (x)χ γ (y)χ γ (z)] = Σ α,β,γ ˆfα ˆfβ ˆfγ [E x,y,z χ α (x)χ β (y)χ γ (x)χ γ (y)χ γ (z)]. It equals 0 unless α = β = γ, So the above formula equals Σ α ˆf 3 α E z [χ α (z)], where E z [χ α (z)] = E z [Π i α z i ] = Π i α E[z i ] = (1 ρ) α. Hence we get Σ α ˆf 3 α (1 ρ) α ɛ. The new verifier due to Håstad is based on this and denoted by V H. Instead of using alphabet {1,..., m} or {1,..., n}, V H expects each entry in the proof string to be encoded by the long code. Assume the encodings f is bifolded, that is, for all X {±1} m, f( X) = f(x). Lemma 4.9 If f : {±1} m {±1} is bifolded and ˆf α 0 then α must be an odd number. Proof: ˆf α = f, χ α = 1 m f(x) X i i α If α is even then i α X i = i α ( X i). So if f is bifolded, the sum is 0. New verifier V H : Input: a proof string π encoded by long code. 1. Choose (v, w) E at random, l(w) = i, l(v) = j is the labels in the proof string π of V RAZ ;. π contains in the locations i and j two functions respectively (which may or may not the long code of each entry of π), denoted by f and g, where f and g are bifolded; X

11 PCP Theorem and hardness of approximation of MAX-SATISFY over finite fields 11 3.choose X {±1} n, Y {±1} m at random; 4. Choose a vector µ {±1} m according to the following distribution: for every i [m] choose µ i = 1 with probability 1 ρ and choose µ i = 1 with probability ρ; 5. It accepts iff f(y )g(x)f((x π)y µ) = 1, where (X π) i = X π(i), i = 1,..., m. Our goal is to prove the new verifier has completeness 1 and soundness 1+δ. If opt(l) = 1, then there exists a correct labeling l(w) = i, l(v) = j for v V, w W such that for (v, w) E, π v,w (l(w)) = l(v). That is, π(i) = j. Hence there exists proof string π which contains f = χ i corresponding to i and g = χ j corresponding to j. Therefore, f(y )g(x)f((x π)y µ) = Y i X j ((X π)y µ) i = Y i X j X π(i) Y i µ i = µ i. Hence V H accepts iff µ i = 1, which happens with probability 1 ρ. If V H accepts with probability at least 1+δ, then we can construct a verifier V RAZ such that its soundness is at least 4ρδ, which contradicts with the arbitrary small soundness of V RAZ. The details are as follows: Consider the expectation of the acceptation condition. First, E r,x,y,µ (f(y )g(x)f((x π)y µ)) 1 + δ 1 δ = δ. E r,x,y,µ (f(y )g(x)f((x π)y µ)) = E r,x,y,µ [(Σ α ˆfα χ α (Y )) (Σ β ĝ β χ β (X)) (Σ γ ˆfγ χ γ ((X π)y µ)))], = E r,x,y,µ [Σ α,β,γ ˆfα ĝ β ˆfγ χ α (Y )χ β (X)χ γ (X π)χ γ (Y )χ γ (µ)]. Since χ γ (X π) = i γ (X π) i = i γ X π(i) = j π (γ) X j = χ π (γ)(x), where π (γ) = {j [n] : π 1 (j) γ is odd}, the above formula can be written as: E r,x,y,µ [Σ α,β,γ ˆfα ĝ β ˆfγ χ α γ (Y )χ β π (γ)(x)χ γ (µ)] = E r,x,y,µ [Σ α,β,γ ˆfα ĝ β ˆfγ (1 ρ) γ χ α γ (Y )χ β π (γ)(x)] Since E(χ α γ (Y )) = 0 unless α = γ and E(χ β π (γ)(x)) = 0 unless β = π (γ), the above formula equals E r [Σ α ˆf α ĝ π (α)(1 ρ) α ] δ Next, we will We will use π to construct a proof π for V Raz : 1.Select α [m] with probability ˆf α and then select i at random from α;.select β [n] with probability ĝβ and then select j at random from β; 3.V Raz accepts if and only if π(i) = j. The selection is reasonable because f, f = Σ α ˆf α = 1 and g, g = Σ β ĝβ = 1. If β = π (α), then for all j β there exists at least one i α such that π(i) = j. Hence, the verifier V Raz accepts with probability at least Σ α ˆf α ĝπ (α) α 1. Using Cauchy equality, we have Σ α ˆf α ĝ π (α) α 1/ (Σ α ˆf α ĝ π (α) α 1 ) 1/ (Σ α ˆf α ) 1/ = (Σ α ˆf α ĝ π (α) α 1 ) 1/

12 1 S. Zhao Hence, E r [Σ α ˆf α ĝ π (α) α 1 ] E r [(Σ α ˆf α ĝ π (α) α 1/ ) ] Since (4ρ α ) 1/ exp( ρ α ) (1 ρ) α, [E r (Σ α ˆf α ĝ π (α) α 1/ )] E r [Σ α ˆf α ĝ π (α) α 1 ] 4ρ(E r [Σ α ˆf α ĝ π (α)(1 ρ) α ]) 4ρδ Therefore, the soundness of the verifier V Raz is 4ρδ. Choose ρ = δ, and choose δ such that 4δ 3 ɛ, then the soundness of V Raz we construct is at least ɛ, which is a contradiction with our assumption. The conclusion is that the verifier V H has completeness 1 δ and soundness 1+δ. Transforming {±1} back to {0, 1}, the acceptance condition becomes linear equation over F. Let the bits in the proof to be unknown boolean variables and tests of the verifier (linear equation) as constraints, the task of finding a proof that maximizes acceptance probability is the same as the task of finding an assignment that satisfies the maximum fraction of constraints. It is easy to see that theorem 4. holds. 5. MAX-SATISFY over F p We can extend the results in section 4 to F p, where p is a prime. We also start from the verifier V RAZ of Label Cover, the difference is the definition of the long code. Assume U p is the set of pth roots of unity. Denote U p = {η 0,..., η p 1 } and Z p = {0, 1,..., p 1}. Consider the map:z p U p which maps a Z p to η a U p. It maps the addition operation in Z p to multiplication operation in U p. We work over U p and then the result can be transformed back to Z p. Definition 5.1 The long code for [m] encodes each i [m] by the table of all values of the function χ {i} : U m p U p. Hence the long code for l(v) [n] is the truth table of χ {l(v)} : Up n code for l(w) [m] is the truth table of χ {l(w)} : Up m U p. For f, g : Up m C, their inner product is defined by f, g = 1 p m X U m p f(x)g(x) U p, and the long Where g(x) denotes complex conjugation. The Fourier basis function becomes χ α, χ α (X) = i [m] Xα i i, where α = (α 1,..., α m ), α i Z p. We can see that χ α, χ β = δ α,β. The basis functions form a complete orthonormal system and we define the coefficients by ˆf α = f, χ α and f(x) = ˆf α α χ α (X). f, f = α ˆf α χ α, β ˆf β χ β = α ˆf α

13 PCP Theorem and hardness of approximation of MAX-SATISFY over finite fields 13 and f, f = 1 p m f(x) = 1 Hence, α ˆf α = 1. When working with long code, we need to fold over U p. X Definition 5. Given a function f : Up m U p. The function f Up, folding over U p, is defined by for each set of (γx) γ Up selecting one. If γ 0 X is selected, then f Up (γx) = γγ0 1 f(γ 0X) for all γ U p. It is easy to see that f Up (γx) = γf Up (X), and we all denote f Up by f. Now we will describe the verifier V H : Input: a proof string π encoded by long code. 1. Choose (v, w) E at random, l(w) = i, l(v) = j is the labels in the proof string π of V RAZ ;. π contains in the locations i and j two functions respectively (which may or may not the long code of each entry of π), denoted by f and g, where f : Up m U p, g : Up n U p and they are both folded over U p. 3.Choose X Up n, Y Up m at random; 4. Choose a vector µ Up m according to the following distribution: for every i [m] choose µ i = 1 with probability 1 ρ and otherwise µ i = γ where γ is chosen randomly and uniformly in U p ; 5. It accepts iff f(y )g(x)f(((x π)y µ) 1 ) = 1, where (X π) i = X π(i), i = 1,..., m. If opt(l) = 1, then there exists a correct labeling l(w) = i, l(v) = j for v V, w W such that for (v, w) E, π v,w (l(w)) = l(v). That is, π(i) = j. Hence there exists proof string π which contains f = χ i corresponding to i and g = χ j corresponding to j. Therefore, f(y )g(x)f(((x π)y µ) 1 ) = Y i X j ((X π) 1 Y 1 µ 1 ) i = Y i X j X 1 π(i) Y i 1 µ 1 i = µ 1 i. Hence V H accepts iff µ 1 i = 1, which happens with probability at least 1 ρ. 1+δ accepts with probability at least p, then we can construct a verifier V RAZ such If V H that its soundness is at least ρδ /p, which contradicts with the arbitrary small soundness of V RAZ. The details are as follows: Lemma 5.3 Suppose γ U p, γ Z p, γ Z p,γ 0 γ γ = { p 1 γ = 1 1 Proof: If γ = 1, γ Z p,γ 0 γγ = (p 1) 1 = p 1. otherwise If γ 1, as we vary γ over Z p, γ γ varies over a complete set of roots of unity. Hence, γ Z p γ γ = 0, and then γ Z p,γ 0 γγ = γ 0 = 1.

14 14 S. Zhao Consider the following formula: E r,x,y,µ [ γ 0(f(Y )g(x)f(((x π)y µ) 1 )) γ ] (p 1) 1 + δ p = δ + ( 1) (1 1 + δ p ) Denote G = g γ, F = f γ, E X,Y,µ [F (Y )G(X)F (((X π)y µ) 1 )] = E X,Y,µ [Σ α ˆFα χ α (Y )) Σ β Ĝ β χ β (X)) (Σ γ ˆFγ χ γ (((X π)y µ) 1 )] = E X,Y,µ [Σ α,β,γ ˆFα Ĝ β ˆFγ χ α (Y )χ β (X)χ γ ((X π) 1 )χ γ (Y 1 )χ γ (µ 1 )] = E X,Y,µ [Σ α,β,γ ˆFα Ĝ β ˆFγ χ α γ (Y )χ β πp(γ)(x)χ γ (µ 1 )] E µ (χ γ (µ 1 )) = E µ ( i [m] µ γ i i ) = (1 ρ) s(γ) Where s(γ) = {i : γ i 0} and π p (γ) = {θ : θ j = i:π(i)=j γ i}. Hence, E X,Y,µ [F (Y )G(X)F (((X π)y µ) 1 )] = α ˆF αĝ π (α)(1 ρ)s(α) So, there exists some γ 0 such that E r[ α ˆF αĝ π p(α) (1 ρ)s(α) ] δ p. Lemma 5.4 Given f as above, and γ Z p, F = f γ. Then for all ˆFα 0, we have i α i = γ. In particular if γ is nonzero, there exists some i such that α i 0. Proof: Assume i α i γ, and take some γ U p with γ γ i αi 1. We have ˆF α = 1 p m F (X)χ α (X) = 1 p m = 1 p m = 1 p m We conclude that ˆF α = 0. X U m p X U m p X U m p X U m p = γ γ i [m] α i ˆFα F (γx)χ α (γx) γ γ F (x) (γx i ) α i i [m] γ γ i [m] α i F (X)χ α (X)

15 PCP Theorem and hardness of approximation of MAX-SATISFY over finite fields 15 Next, we will use π to construct a proof π for V Raz : 1.Select α Up m with probability ˆF α and then select i [m] at random such that α i 0 ;.Select β Up n with probability Ĝβ and then select j [n] at random such that β j 0 ; 3.V Raz accepts if and only if π(i) = j. The selection is reasonable because α ˆF α = 1 and β Ĝβ = 1, and Lemma 5.4. If β = π p (α), then for all j [n], β j 0 there exists at least one i [m], α i 0 such that π(i) = j. Hence, the verifier V Raz accepts with probability at least Σ α ˆF α Ĝπ (α) s(α) 1. Using Cauchy equality, we have Hence, Σ α ˆF α Ĝπ (α) s(α) 1/ (Σ α ˆF α Ĝπ (α) s(α) 1 ) 1/ (Σ α ˆF α ) 1/ = (Σ α ˆF α Ĝπ (α) s(α) 1 ) 1/ E r [Σ α ˆF α Ĝπ (α) s(α) 1 ] E r [(Σ α ˆF α Ĝπ (α) s(α) 1/ ) ] Since (ρs(α)) 1/ exp( ρs(α)) (1 ρ) s(α), [E r (Σ α ˆF α Ĝπ (α) s(α) 1/ )] E r [Σ α ˆF α Ĝπ p(α) s(α) 1 ] ρ[e r [Σ α ˆF α Ĝπ (α) (1 ρ) s(α) ]] ρδ /p Therefore, the soundness of the verifier V Raz is ρδ /p. Choose ρ = δ, and choose δ such that δ 3 /p ɛ, then the soundness of V Raz we construct is at least ɛ, which is a contradiction with our assumption. The conclusion is that the verifier V H has completeness 1 δ and soundness 1+δ p. Similar to the case of F, we can get the following theorem. Theorem 5.5 For any ɛ > 0, it is NP-hard to approximate Max-E3-LIN-p with a factor of p ɛ. References [1] Arora S., Barak B., Computational complexity: a modern approach. [] Arora S., Lund C., Motawani R., Sudan M., Szegedy M., Proof verification and the hardness of approximation problems, Journal of the ACM, 45(3): , [3] Håstad J., Some optimal inapproximability results. Journal of the ACM, 48(4): , 001. [4] Hochbaum D. S., Approximation algorithms for NP-hard Problems. International Thomson Publishing Company, [5] Knot S., Probabilistically checkable proofs and hardness of approximation PCPs and hardness of approximiation. khot/pcp-course.html [6] Xu D., PCP theorem and its applications to research on non-approximatable problems. Journal of Frontiers of Computer Science and Technology, 008, (1): 0-31.