[PCP Theorem is] the most important result in complexity theory since Cook s Theorem. Ingo Wegener, 2005

Transcription

1 PCP Theorem

2 [PCP Theorem is] the most important result in complexity theory since Cook s Theorem. Ingo Wegener, 2005 Computational Complexity, by Fu Yuxi PCP Theorem 1 / 88

3 S. Arora, C. Lund, R. Motwani, M. Sudan, and M. Szegedy. Proof Verification and the Hardness of Approximation Problems. J. ACM, Preliminary version FOCS Irit Dinur. The PCP Theorem by Gap Amplification. J. ACM, Preliminary version STOC The 1 st proof is algebraic, the 2 nd combinatorial & non-elementary. Computational Complexity, by Fu Yuxi PCP Theorem 2 / 88

4 Two ways to view the PCP Theorem: It is a result about locally testable proof systems. It is a result about hardness of approximation. PCP = Probabilistically Checkable Proof Computational Complexity, by Fu Yuxi PCP Theorem 3 / 88

5 Synopsis 1. Approximation Algorithm 2. Two Views of PCP Theorem 3. Equivalence of the Two Views 4. Inapproximability 5. Efficient Conversion of NP Certificate to PCP Proof 6. Proof of PCP Theorem 7. Historical Remark Computational Complexity, by Fu Yuxi PCP Theorem 4 / 88

6 Approximation Algorithm Computational Complexity, by Fu Yuxi PCP Theorem 5 / 88

7 Since the discovery of NP-completeness in 1972, researchers had been looking for approximate solutions to NP-hard optimization problems, with little success. The discovery of PCP Theorem in 1992 explains the difficulty. Computational Complexity, by Fu Yuxi PCP Theorem 6 / 88

8 Suppose ρ : N (0, 1). A ρ-approximation algorithm A for a maximum, respectively minimum optimization problem satisfies A(x) Max(x) ρ( x ), respectively for all x. Min(x) A(x) ρ( x ) Computational Complexity, by Fu Yuxi PCP Theorem 7 / 88

9 SubSet-Sum Given n items of sizes s 1, s 2,..., s n, and a positive integer C, find a subset of the items that maximizes the total sum of their sizes without exceeding the capacity C. There is a well-known dynamic programming algorithm. Using the algorithm and a parameter ɛ a (1 ɛ)-approximation algorithm can be designed that runs in O ( ( 1 ɛ 1) n2) time. We say that SubsetSum has an FPTAS. Computational Complexity, by Fu Yuxi PCP Theorem 8 / 88

10 KnapSack Let U = {u 1, u 2,..., u n } be the set of items to be packed in a knapsack of size C. For 1 j n, let s j and v j be the size and value of the j-th item, respectively. The objective is to fill the knapsack with items in U whose total size is at most C and such that their total value is maximum. There is a similar dynamic programming algorithm. Using the algorithm and a parameter( ɛ one can design ) a (1 ɛ)-approximation algorithm of O ( 1 ɛ 1) n 1 ɛ time. We say that KnapSack has a PTAS. Computational Complexity, by Fu Yuxi PCP Theorem 9 / 88

11 Max-3SAT For each 3CNF ϕ, the value of ϕ, denoted by val(ϕ), is the maximum fraction of clauses that can be satisfied by an assignment to the variables of ϕ. ϕ is satisfied if and only if val(ϕ) = 1. Max-3SAT is the problem of finding the maximum val(ϕ). A simple greedy algorithm for Max-3SAT is 1 2 -approximate. We say that Max-3SAT is in APX. By definition, FPTAS PTAS APX OPT. Computational Complexity, by Fu Yuxi PCP Theorem 10 / 88

12 Max-IS Min-VC + Max-IS = n. A 1 2-approximation algorithm for Min-VC. 1. Pick up a remaining edge and collect the two end nodes. 2. Remove all edges adjacent to the two nodes. 3. Goto Step 1 if there is at least one remaining edge. Is Min-VC in PTAS? Is Max-IS in APX? Computational Complexity, by Fu Yuxi PCP Theorem 11 / 88

13 A breakthrough in the study of approximation algorithm was achieved in the beginning of 1990 s. U. Feige, S. Goldwasser, L. Lovász, S. Safra, and M. Szegedy. Interactive Proofs and the Hardness of Approximating Cliques. FOCS J. ACM, S. Arora and S. Safra. Probabilistic Checking of Proofs: A New Characterization of NP. FOCS J. ACM, [1991]. There is no 2 log1 ɛ (n) -approximation algorithm for Max-IS unless SAT SUBEXP. [1992]. Max-IS is not in APX if P NP. Computational Complexity, by Fu Yuxi PCP Theorem 12 / 88

14 Two Views of PCP Theorem Computational Complexity, by Fu Yuxi PCP Theorem 13 / 88

15 Surprisingly, IP = PSPACE. More surprisingly, MIP = NEXP. The latter can be interpreted as saying that nondeterminism can be traded off for randomness + interaction. Computational Complexity, by Fu Yuxi PCP Theorem 14 / 88

16 Interactive Proof Viewpoint Suppose L is an NP problem and x is an input string. 1. Prover provides a proof π of polynomial length. 2. Verifier uses at most logarithmic many random bits, and makes a constant number of queries on π. A query i is a location, necessarily of logarithmic length. The answer to query i is π(i). Queries are written on a special address/oracle tape. We assume that verifier is nonadaptive in that its selection of queries is based only on input and random string. Computational Complexity, by Fu Yuxi PCP Theorem 15 / 88

17 Probabilistically Checkable Proofs Suppose L is a language and q, r : N N. L has an (r(n), q(n))-pcp verifier if there is a P-time verifier V that renders true the following. Efficiency. On input x and given access to any location of a proof π of length q(n)2 r(n), the verifier V uses at most r(n) random bits in its computation and makes at most q(n) nonadaptive queries to the proof π before it outputs 1 or 0. Completeness. If x L, then π.pr[v π (x) = 1] = 1. Soundness. If x / L, then π.pr[v π (x) = 1] 1/2. V π (x) denotes the random variable with fixed x and π. Computational Complexity, by Fu Yuxi PCP Theorem 16 / 88

18 Probabilistically Checkable Proofs Remark. 1. Proof length q(n)2 r(n). At most q(n)2 r(n) locations can be queried by verifier. 2. L NTIME(q(n)2 O(r(n)) ). An algorithm guesses a proof of length q(n)2 r(n). It executes deterministically 2 r(n) times the verifier s algorithm. The total running time is bounded by q(n)2 O(r(n)). 3. An (r(n), q(n))-pcp verifier has randomness complexity r(n) and query complexity q(n). Computational Complexity, by Fu Yuxi PCP Theorem 17 / 88

19 PCP Hierarchy A language is in PCP(r(n), q(n)) if it has a (cr(n), dq(n))-pcp verifier for some c, d. PCP(r(n), q(n)) NTIME(q(n)2 O(r(n)) ). PCP(0, log) = P. PCP(0, poly) = NP. PCP(log, poly) NP. Computational Complexity, by Fu Yuxi PCP Theorem 18 / 88

20 PCP Hierarchy 1. PCP(poly, poly) NEXP. 2. PCP(log, log) NP. 3. PCP(log, 1) NP. In three influential papers in the history of PCP, it is proved that the above can be strengthened to =. Computational Complexity, by Fu Yuxi PCP Theorem 19 / 88

21 The PCP Theorem PCP Theorem. NP = PCP(log, 1). Every NP-problem has specifically chosen certificates whose correctness can be probabilistically verified by checking only 3 bits. Computational Complexity, by Fu Yuxi PCP Theorem 20 / 88

22 Example GNI PCP(poly, 1). Suppose both G 0 and G 1 have n vertices. Proofs are indexed by adjacent matrix representations. If the location, a string of length n 2, represents a graph isomorphic to G i, it has the value i. The verifier picks up b {0, 1} at random, produces a random permutation of G b, and queries the bit of the proof at the corresponding location. Computational Complexity, by Fu Yuxi PCP Theorem 21 / 88

23 Can we scale down PCP Theorem further? Fact. If NP PCP(o(log), o(log)), then P = NP. Computational Complexity, by Fu Yuxi PCP Theorem 22 / 88

24 Scale-Up PCP Theorem Theorem. PCP(poly, 1) = NEXP. Computational Complexity, by Fu Yuxi PCP Theorem 23 / 88

25 Hardness of Approximation Viewpoint For many NP-hard optimization problems, computing approximate solutions is no easier than computing the exact solutions. Computational Complexity, by Fu Yuxi PCP Theorem 24 / 88

26 The PCP Theorem, Hardness of Approximation PCP Theorem. There exists ρ < 1 such that for every L NP there is a P-time computable function f : L 3SAT such that x L val(f (x)) = 1, x / L val(f (x)) < ρ. Figure out the significance of the theorem by letting L = 3SAT. Computational Complexity, by Fu Yuxi PCP Theorem 25 / 88

27 The PCP Theorem, Hardness of Approximation PCP Theorem cannot be proved using Cook-Levin reduction. val(f (x)) tends to 1 even if x / L. The intuitive reason is that computation is an inherently unstable, non-robust mathematical object, in the sense that it can be turned from non-accepting to accepting by changes that would be insignificant in any reasonable metric. Papadimitriou and Yannakakis, 1988 Computational Complexity, by Fu Yuxi PCP Theorem 26 / 88

28 Corollary. There exists some ρ < 1 such that if there is a P-time ρ-approximation algorithm for Max-3SAT then P = NP. The ρ-approximation algorithm for Max-3SAT is NP-hard. Computational Complexity, by Fu Yuxi PCP Theorem 27 / 88

29 Equivalence of the Two Views Computational Complexity, by Fu Yuxi PCP Theorem 28 / 88

30 CSP, Constraint Satisfaction Problem If q is a natural number, then a qcsp instance ϕ with n variables is a collection of constraints ϕ 1,..., ϕ m : {0, 1} n {0, 1} such that for each i [m] the function ϕ i depends on q of its input locations. We call q the arity of ϕ, and m the size of ϕ. An assignment u {0, 1} n satisfies a constraint ϕ i if ϕ i (u) = 1. Let { n i=1 val(ϕ) = max ϕ } i(u). u {0,1} n m We say that ϕ is satisfiable if val(ϕ) = 1. qcsp is a generalization of 3SAT. Computational Complexity, by Fu Yuxi PCP Theorem 29 / 88

31 1. We assume that n qm. 2. Since every ϕ i can be described by a formula of size q2 q, and every variable can be coded up by log n bits, a qcsp instance can be described by O(mq2 q log n) bits. 3. The greedy algorithm for MAX-3SAT can be applied to MAXqCSP to produce an assignment satisfying val(ϕ) 2 q m constraints. Computational Complexity, by Fu Yuxi PCP Theorem 30 / 88

32 Gap CSP Suppose q N and ρ 1. Let ρ-gapqcsp be the promise problem of determining if a qcsp instance ϕ satisfies either (1) val(ϕ) = 1 or (2) val(ϕ) < ρ. We say that ρ-gapqcsp is NP-hard if for every NP-problem L some P-time computable function f : L ρ-gapqcsp exists such that x L val(f (x)) = 1, x / L val(f (x)) < ρ. PCP Theorem: There exists ρ (0, 1) st ρ-gap3sat is NP-hard. Computational Complexity, by Fu Yuxi PCP Theorem 31 / 88

33 PCP Theorem. There exist q N and ρ (0, 1) such that ρ-gapqcsp is NP-hard. Computational Complexity, by Fu Yuxi PCP Theorem 32 / 88

34 Equivalence Proof PCP Theorem PCP Theorem. This is essentially the Cook-Levin reduction. 1. Suppose NP PCP(log, 1). Then 3SAT has a PCP verifier V that makes q queries using c log n random bits. 2. Given input x with x = n and random string r {0, 1} c log n, V(x, r) is a Boolean function of type {0, 1} q {0, 1}. 3. ϕ = {V(x, r)} r {0,1} c log n is a P-size qcsp instance. By completeness, x 3SAT val(ϕ) = 1. By soundness, x / 3SAT val(ϕ) 1/2. 4. The map from 3SAT to 1 2-GAPqCSP is P-time computable. V runs in P-time. Computational Complexity, by Fu Yuxi PCP Theorem 33 / 88

35 Equivalence Proof PCP Theorem PCP Theorem. Suppose L NP and ρ-gapqcsp is NP-hard for some q N, ρ < 1. By assumption there is some P-time reduction f : L ρ-gapqcsp. 1. The verifier for L works as follows: On input x, compute the qcsp instance f (x) = {ϕ i } i [m]. Given a proof π, which is an assignment to the variables, it randomly chooses i [m] and checks if ϕ i is satisfied by reading q bits of the proof. 2. If x L, the verifier always accepts; otherwise it accepts with probability < ρ. Computational Complexity, by Fu Yuxi PCP Theorem 34 / 88

36 Equivalence Proof PCP Theorem PCP Theorem. This is essentially the equivalence between SAT and 3SAT. 1. Let ɛ > 0 and q N be such that (1 ɛ)-gapqcsp is NP-hard. 2. Let ϕ = {ϕ i } m i=1 be a qcsp instance with n variables. 3. Each ϕ i is the conjunction of at most 2 q clauses, each clause being the disjunction of q literals. Altogether there are at most m2 q such clauses. 4. These clauses can be turned into mq2 q 3SAT clauses. 5. If all assignments fail at least an ɛ fragment of the constraints of ϕ, then all assignments fail at least a ɛ q2 fragment of the q clauses of the 3SAT instance. Computational Complexity, by Fu Yuxi PCP Theorem 35 / 88

37 Proof View PCP verifier PCP proof proof length number of queries number of random bits soundness parameter Inapproximability View CSP instance assignment to variables number of variables arity of constraints logarithm of number of constraints maximum fraction of no instances The equivalence of the proof view and the inapproximability view is essentially due to the Cook-Levin Theorem for PTM. Computational Complexity, by Fu Yuxi PCP Theorem 36 / 88

38 Inapproximability Computational Complexity, by Fu Yuxi PCP Theorem 37 / 88

39 Min-VC and Max-IS are inherently different from the perspective of hardness of approximation. Min-VC + Max-IS = n. A ρ-approximation algorithm for Max-IS can be used to design a n IS n ρis -approximation algorithm for Min-VC. Computational Complexity, by Fu Yuxi PCP Theorem 38 / 88

40 Lemma. There is a P-time computable function f from 3CNF formulas to graphs that maps a formula ϕ onto an n-vertex graph f (ϕ) whose independent set is of size val(ϕ) n 7. The standard Karp reduction from 3SAT to Max-IS is as follows: Each clause is translated to a clique of 7 nodes, each node represents a (partial) assignment that validates the clause. Two nodes from two different cliques are connected if and only if they are conflicting. A 3CNF formula ϕ consisting of m clauses is translated to a graph with 7m nodes, and an assignment satisfying l clauses of ϕ if and only if the graph has an independent set of size l. Computational Complexity, by Fu Yuxi PCP Theorem 39 / 88

41 Theorem. The following statements are valid. 1. ρ < 1. ρ -approximation to Min-VC is NP-hard, and 2. ρ < 1. ρ-approximation to Max-IS is NP-hard. There is some ρ < 1 such that no ρ-approximation to Max-3SAT exists unless P = NP. By the previous lemma a ρ-approximation to Max-IS would produce a ρ-approximation to Max-3SAT. The minimum vertex cover has size n val(ϕ) n 7. Let ρ = 6 7 ρ. If Min-VC had a ρ -approximation algorithm, it would produce a vertex cover of size n ρ n 7 in the case val(ϕ) = 1. If val(ϕ) < ρ, the minimum vertex cover has size > n ρ n 7. The algorithm must return a vertex cover of size > n ρ n 7. Continue on the next slide. Computational Complexity, by Fu Yuxi PCP Theorem 40 / 88

42 Suppose Max-IS was ρ 0 -approximate. Let ρ be given in the proof. ( Let k be such that ρ IS ) ( 0 k ρis ) k. Define an algorithm as follows: 1. Input a graph G. Construct G k as follows: The vertices are k-size subsets of VG ; Two vertices S1, S 2 are disconnected if S 1 S 2 is an independent set of G. 2. Apply the ρ 0 -approximation algorithm to G k, and derive an independent set of G from the output of the algorithm. The largest independent set of G k is of size ( ) IS k, where IS is the maximum independent set of G. ( The output is an independent set of size ρ IS ) ( 0 k ρis ) k. An independent set of size ρis can be derived. Contradiction. Computational Complexity, by Fu Yuxi PCP Theorem 41 / 88

43 Efficient Conversion of NP Certificate to PCP Proof Computational Complexity, by Fu Yuxi PCP Theorem 42 / 88

44 Proofs of PCP Theorems involve some interesting ways of encoding NP-certificates and the associated methods of checking if a string is a valid encoding. One idea is to amplify any error that appears in an NP-certificate. We shall showcase how it works by looking at a problem to which the amplification power of Walsh-Hadamard Code can be exploited. Theorem. NP PCP(poly(n), 1). Computational Complexity, by Fu Yuxi PCP Theorem 43 / 88

45 Walsh-Hadamard Code The Walsh-Hadamard function WH : {0, 1} n {0, 1} 2n encodes a string of length n by a function in n variables over GF(2): WH(u) : x u x, where u x = n i=1 u iy i (mod 2). We say that f is a Walsh-Hadamard codeword if f = WH(u) for some u {0, 1} n. Random Subsum Principle. If u v then for exactly half the choices of x, u x v x. Computational Complexity, by Fu Yuxi PCP Theorem 44 / 88

46 Walsh-Hadamard Code A function f of type {0, 1} n {0, 1} 2n is a linear function if f (x + y) = f (x) + f (y). Walsh-Hadamard codewords are precisely the linear functions. This is so because a linear function f corresponds to (f (e 1 ),..., f (e n )). Computational Complexity, by Fu Yuxi PCP Theorem 45 / 88

47 Let ρ [0, 1]. The functions f, g : {0, 1} n {0, 1} are ρ-close if Pr x R{0,1} n[f (x) = g(x)] ρ. Theorem. Let f : {0, 1} n {0, 1} be such that for some ρ > 1 2, Pr x,y R{0,1} n[f (x + y) = f (x) + f (y)] ρ. Then f is ρ-close to a linear function. Computational Complexity, by Fu Yuxi PCP Theorem 46 / 88

48 Local Testing of Walsh-Hadamard Codeword A local test of f makes a constant number of queries to check if it is a Walsh-Hadamard codeword. It accepts every linear function, and with high probability it rejects every function that is far from linear. For δ (0, 1/2) a (1 δ)-linearity test rejects with probability 1 2 any function not (1 δ)-close to a linear function by testing randomly for O( 1 δ ) times. f (x + y) = f (x) + f (y) Computational Complexity, by Fu Yuxi PCP Theorem 47 / 88

49 Local Decoding Suppose δ < 1 4 and f is (1 δ)-close to some linear function f. Given x one can learn f (x) by making only two queries to f. 1. Choose x R {0, 1} n ; 2. Set x = x + x ; 3. Output f (x ) + f (x ). By union bound f (x) = f (x ) + f (x ) = f (x ) + f (x ) holds with a probability at least 1 2δ. Computational Complexity, by Fu Yuxi PCP Theorem 48 / 88

50 Quadratic Equation in GF(2) Suppose A is an m n 2 matrix and b is an m-dimensional vector. Let (A, b) QUADEQ if there is an n-dimensional vector u such that A(u u) = b, where u u is the tensor product of u. u u = (u 1 u 1,..., u 1 u n,..., u n u 1,..., u n u n ). Computational Complexity, by Fu Yuxi PCP Theorem 49 / 88

51 Quadratic Equation in GF(2) An instance of QUADEQ over u 1, u 2, u 3, u 4, u 5 : u 1 u 2 + u 3 u 4 + u 1 u 5 = 1 u 1 u 1 + u 2 u 3 + u 1 u 4 = 0 A satisfying assignment is (0, 0, 1, 1, 0). Computational Complexity, by Fu Yuxi PCP Theorem 50 / 88

52 QUADEQ is NP-Complete CKT-SAT K QUADEQ. Inputs and the outputs are turned into variables. Boolean equality x y = z relating the inputs to the output is turned into algebraic equality (1 x)(1 y) = 1 z in GF(2), which is equivalent to xx + yy + xy + zz = 0. Computational Complexity, by Fu Yuxi PCP Theorem 51 / 88

53 From NP Certificate to PCP Proof Suppose A is an m n 2 matrix and b is an n-dimensional vector. A certificate for (A, b) is an n-dimensional vector u. We convert an NP-certificate u to the PCP-proof WH(u)WH(u u). The proof, say fg, is a string of length 2 n + 2 n2. Using the proof it is easy to verify if (A, b) QUADEQ. Computational Complexity, by Fu Yuxi PCP Theorem 52 / 88

54 Verifier for QUADEQ Step 1. Verify that f, g are linear functions. 1. Perform a linearity test on f, g. If successful we may assume that f (r) = u r and g(z) = w z. Computational Complexity, by Fu Yuxi PCP Theorem 53 / 88

55 Verifier for QUADEQ Step 2. Verify that g encodes (u u). 1. Get independent random r, r. 2. Reject if f (r)f (r ) g(r r ). 3. Repeat the test 10 times. In a correct proof f (r)f (r ) = ( i u ir i )( j u jr j ) = g(r r ). Assume w u u. Let W and U be the n n matrices of w and respectively u u. One has g(r r ) = w (r r ) = i,j w ijr i r j = rw r, and f (r)f (r ) = (u r)(u r ) = ( i u ir i )( j u jr j ) = rur. rw and ru differ for half of the r s; and rw r and rur differ for a quarter of (r, r ) s. The overall probability of rejection 1 ( 3 4 )10 > 0.9. Computational Complexity, by Fu Yuxi PCP Theorem 54 / 88

56 Verifier for QUADEQ Step 3. Verify that g encodes a solution. 1. Take a random subset S of [m]. 2. Reject if g( k S A k,(i,j)) k S b k. There is enough time to check A(u u) = b by checking, for all k [m], the equality g(a k,(i,j) ) = b k. However since m is part of the input, the number of queries, which must be a constant, should not depend on m. If {k [m] g(a k,(i,j) ) b k }, then Pr S R[m][ S {k [m] g(a k,(i,j) ) b k } is odd] = 1 2. Note that g( k S A k,(i,j)) = k S g(a k,(i,j)) by linearity. Computational Complexity, by Fu Yuxi PCP Theorem 55 / 88

57 Proof of PCP Theorem Computational Complexity, by Fu Yuxi PCP Theorem 56 / 88

58 CSP with Nonbinary Alphabet For each number W, qcsp W is analogous to qcsp except that the alphabet is [W ] instead of {0, 1}. The constraints are functions of type [W ] q {0, 1}. For ρ (0, 1), we define ρ-gapqcsp W analogous to ρ-gapqcsp. Computational Complexity, by Fu Yuxi PCP Theorem 57 / 88

59 3COL is a case of 2CSP 3. Computational Complexity, by Fu Yuxi PCP Theorem 58 / 88

60 PCP Theorem states that ρ-gapqcsp is NP-hard for some q, ρ. The proof we shall describe is based on the following: 1. If ϕ of m constraints is unsatisfied, then val(ϕ) 1 1 m. 2. A construction that increases the gap. The idea is to start with an NP-problem, then apply 2 repeatedly. Computational Complexity, by Fu Yuxi PCP Theorem 59 / 88

61 Let f be a function mapping CSP instance to CSP instance. It is a CL-reduction (complete linear blowup reduction) if it is P-time computable and the following are valid for every CSP instance ϕ. Completeness. If ϕ is satisfiable then f (ϕ) is satisfiable. Linear Blowup. If ϕ has m constraints, then f (ϕ) has Cm constraints over a new alphabet W, where C and W can depend on the arity and the alphabet size of ϕ. Computational Complexity, by Fu Yuxi PCP Theorem 60 / 88

62 Main Lemma Lemma. There exist constants q 0 3, ɛ 0 > 0 and CL-reduction f such that for every q 0 CSP instance ϕ and every ɛ < ɛ 0 the instance f (ϕ) is a q 0 CSP satisfying val(ϕ) 1 ɛ val(f (ϕ)) 1 2ɛ. Arity Alphabet Constraint Value ϕ q 0 binary m 1 ɛ f (ϕ) q 0 binary Cm 1 2ɛ Computational Complexity, by Fu Yuxi PCP Theorem 61 / 88

63 PCP Theorem from Main Lemma Let q 0 3 and ɛ 0 > 0 be given by Main Lemma. 1. q 0 CSP is NP-hard. 2. For a q 0 CSP instance with m constraints, apply Main Lemma for log m times to amplify possible gap. We get an instance ψ. 3. If ϕ is satisfiable, then ψ is satisfiable. Otherwise according to Main Lemma val(ψ) 1 min{2ɛ 0, 1 2 log m /m} = 1 2ɛ ψ C log m m = poly(m). We have obtained a reduction from q 0 CSP to (1 2ɛ 0 )-GAPq 0 CSP. C depends on two constants, q 0 and 2 (the size of alphabet). Computational Complexity, by Fu Yuxi PCP Theorem 62 / 88

64 Main Lemma is proved in three steps. 1. Prove that every qcsp instance can be turned into a qcsp W instance with nice property. 2. Construct a CL-reduction f that increase both the gap and the alphabet size of a nice qcsp instance. [Dinur s proof] 3. Construct a CL-reduction g that decreases the alphabet size to 2 with a modest reduction in the gap. [Proof of Arora et al.] Arity Alphabet Constraint Value ϕ q 0 binary m 1 ɛ f (ϕ) 2 W Cm 1 6ɛ g(f (ϕ)) q 0 binary C Cm 1 2ɛ Computational Complexity, by Fu Yuxi PCP Theorem 63 / 88

65 Dinur makes use of expander graphs to construct new constraints. Let ϕ be a qcsp W instance with n variables. The constraint graph G of ϕ is defined as follows: (i) The vertex set is [n], and (ii) (i, j) is an edge if there is a constraint on the variables u i, u j. Parallel edges and self-loops are allowed. Computational Complexity, by Fu Yuxi PCP Theorem 64 / 88

66 A qcsp W instance ϕ is nice if the following are valid: 1. q = There is a constant d such that the constraint graph of ϕ is a (d, 0.9)-expander and at every vertex at least half of the adjacent edges are self loops. Computational Complexity, by Fu Yuxi PCP Theorem 65 / 88

67 Lemma. Let G be an (n, d, λ)-graph, S is a vertex subset and T is the complement of S. Then E(S, T ) (1 λ) d S T S + T. (1) The vector x defined below satisfies x 2 2 = S T ( S + T ) and x 1. x i = { + T, i S, S, i T. Let Z = i,j A i,j(x i x j ) 2. By definition Z = 2 d E(S, T ) ( S + T )2. On the other hand Z = i,j A i,j x 2 i 2 i,j A i,j x i x j + i,j A i,j x 2 j = 2 x x, Ax. Since x 1, x, Ax λ x 2 (cf. Rayleigh quotient). Computational Complexity, by Fu Yuxi PCP Theorem 66 / 88

68 For every c (0, 1) there is a constant d and an algorithm that, given input n, runs in poly(n) time and outputs an n-vertex d-regular expander G with λ G c. Let G = (V, E) be an expander and S V with S V /2. The following holds. Pr (u,v) E [u S, v S] S V ( λ ) G. (2) 2 Proof. Observe that S / V = Pr (u,v) E [u S, v S] + Pr (u,v) E [u S, v S]. And Pr (u,v) E [u S, v S] = E(S, S)/d V S (1 λ)/(2 V ) by (1). We are done by rewriting the first equality. Pr (u,v) E l[u S, v S] S ( ) 1 V 2 + λl G. (3) 2 Computational Complexity, by Fu Yuxi PCP Theorem 67 / 88

69 Step 1: Reduction to Nice Instance Lemma. There is a CL-reduction mapping every qcsp instance ϕ to a 2CSP 2 q instance ψ such that val(ϕ) 1 ɛ val(ψ) 1 ɛ/q. Suppose ϕ has variables u 1,..., u n and m constraints. The new instance ψ has variables u 1,..., u n, y 1,..., y m, where y i {0, 1} q tries to code up an assignment to the variables in ϕ i. For each variable u j in ϕ i, construct the constraint ψ i,j stating that y i produces a satisfying assignment to ϕ i and y i is consistent to u j. Computational Complexity, by Fu Yuxi PCP Theorem 68 / 88

70 Step 1: Reduction to Nice Instance Lemma. There exist an absolute constant d and a CL-reduction mapping every 2CSP W instance ϕ to a 2CSP W instance ψ such that val(ϕ) 1 ɛ val(ψ) 1 ɛ/(100wd) and the constraint graph of ψ is d-regular. [d is independent of ϕ.] Let {G k } k be a (d 1)-regular expander. We get ψ by replacing each k-degree node of the constraint graph of ϕ by G k and adding the identity constraint (of the form x = y) to each edge of G k. If ϕ has m constraints, ψ has dm constraints. Suppose val(ϕ) 1 ɛ and y is an assignment to ψ. Let u be the assignment to ϕ that is defined by the plurality of y. Let t i be the number of y j i s, where j [k], that disagree with u i. t i k(1 1 W ). It suffices to prove that y violates at least ɛm 100W constraints of ψ. Computational Complexity, by Fu Yuxi PCP Theorem 69 / 88

71 Step 1: Reduction to Nice Instance Lemma. There exist an absolute constant d and a CL-reduction mapping every 2CSP W instance ϕ to a 2CSP W instance ψ such that val(ϕ) 1 ɛ val(ψ) 1 ɛ/(100wd) and the constraint graph of ψ is d-regular. 1. n i=1 t i 1 4 ɛm. 2. n i=1 t i < 1 4ɛm. Since val(ϕ) 1 ɛ, there is at least ɛm constraints violated in ϕ by u. These constraints are also in ψ. Less than 1 4 ɛm + 1 4ɛm of these constraints have valuations in ψ different from valuations in ϕ. Therefore at least 1 2ɛm of these constraints are violated. Computational Complexity, by Fu Yuxi PCP Theorem 70 / 88

72 Step 1: Reduction to Nice Instance Lemma. There exist an absolute constant d and a CL-reduction mapping every 2CSP W instance ϕ with d -regular constraint graph for d d into a 2CSP W instance ψ such that val(ϕ) 1 ɛ val(ψ) 1 ɛ/(10d) and the constraint graph of ψ is a 4d-regular expander, with half the edges coming out of each vertex being self-loops. There is a constant d and an explicit (d, 0.1)-expander {G n } n N. We may assume that ϕ is d-regular (add self-loops if d < d). We get ψ from ϕ by adding a tautological constraint for every edge of G n and 2d tautological constraints forming self-loops for each vertex. λ G λ G < 0.9, where G is the constraint graph of ψ. Computational Complexity, by Fu Yuxi PCP Theorem 71 / 88

73 Step 2: Powering Lemma. There is an algorithm that given t > 1 and a nice 2CSP W instance ψ with n variables, m edges and a constraint graph of d degree, produces a 2CSP W instance ψ t satisfying the following. 1. W = W d5t and ψ t has d t+ t+1 constraints. 2. If ψ is satisfiable then ψ t is satisfiable. 3. For ɛ < 1 d t, if val(ψ) 1 ɛ, then val(ψt ) 1 ɛ for ɛ = t 10 5 dw 4 ɛ. 4. The formula ψ t is produced in P-time in m and W dt. Computational Complexity, by Fu Yuxi PCP Theorem 72 / 88

74 Step 2: Powering Construction of ψ t : 1. Let u 1,..., u n denote the variables of ψ. 2. Introduce n new variables y 1,..., y n W = W d5t. 3. A value of y i is an assignment to all the variables appearing in the paths within t + t steps from u i. 4. For each 2t+1 step path p = (i 1,..., i 2t+2 ) in G ψ, introduce a constraint C p = j [2t+2] C j p for ψ t such that 4.1 the length of (i 1,..., i j ) is t + t, 4.2 the length of (i j+1,..., i 2t+2 ) is t + t, and 4.3 C j p is obtained from the constraint of u ij and u ij+1 by replacing u ij, u ij+1 by y i1 s claim for u ij and y i2t+2 s claim for u ij+1 resp.. The construction can be done in P-time in m and W dt. It is easy to lift a true assignment to the old variables to a true assignment to the new variables. Computational Complexity, by Fu Yuxi PCP Theorem 73 / 88

75 Step 2: Powering Plurality assignment. 1. Fix an assignment to y 1,..., y n. 2. Let Z i [W ] be a random variable defined by the following. Starting from vertex i, take a t step random walk in Gψ to reach vertex k. Output y k s claim for u i. Let z i denote the most likely value of Z i. 3. We call z 1,..., z n the plurality assignment. Pr[Z i =z i ] 1 W. Suppose val(ψ) 1 ɛ. There is a set F of ɛm = ɛ dn 2 in ψ violated by the assignment z 1,..., z n constraints Computational Complexity, by Fu Yuxi PCP Theorem 74 / 88

76 Step 2: Powering Denote by p a random 2t+1 step path (i 1,..., u 2t+t ). For j [2t+1] the j-th edge (i j, i j+1 ) is truthful if y i1 claims the plurality value for i j and similarly y i2t+2 for i j+1. If p has an edge that is truthful and in F, then the constraint C p is unsatisfied. Claim. Let E = G ψ s edge set. Pr e RE [e = (i j, i j+1 )] = 1 E = 2 nd. Proof. In a j-step random walk from a random vertex of a regular graph, the last edge in the walk is a random edge. Computational Complexity, by Fu Yuxi PCP Theorem 75 / 88

77 Step 2: Powering Claim. Let δ < 1 100W. For each edge e and each t j t + δ t, Pr[jth edge of p is truthful jth edge of p is e] 1 2W 2. By the claim, a (2t+1) step random walk with (i j, i j+1 ) = e can be generated by a j-step random walk from i j and a (2t j) step random walk from i j+1. We need to prove Pr[y i1 claims plurality for i j, y i2t+2 claims plurality for i j+1 ] 1 2W 2. If j = t then the left hand side is at east 1/W 2. A k-step random walk can be generated by tossing coin for k times and taking S k -step non-self-loop random walk, where S k = head s. The statistical distance between S t and S t+δ t is bounded by 10δ. Thus the left hand side is ( 1 W 10δ) ( 1 W 10δ) 1 2W 2. Computational Complexity, by Fu Yuxi PCP Theorem 76 / 88

78 Step 2: Powering Computational Complexity, by Fu Yuxi PCP Theorem 77 / 88

79 Step 2: Powering Computational Complexity, by Fu Yuxi PCP Theorem 78 / 88

80 Step 3 Computational Complexity, by Fu Yuxi PCP Theorem 79 / 88

81 Step 3 Computational Complexity, by Fu Yuxi PCP Theorem 80 / 88

82 Step 3 Computational Complexity, by Fu Yuxi PCP Theorem 81 / 88

83 Historical Remark Computational Complexity, by Fu Yuxi PCP Theorem 82 / 88

84 It all started with the introduction of interactive proof systems. Computational Complexity, by Fu Yuxi PCP Theorem 83 / 88

85 Shafi Goldwasser, Silvio Micali, Charles Rackoff. The Knowledge Complexity of Interactive Proofs. SIAM László Babai and Shlomo Moran. Arthur-Merlin Games: A Randomized Proof System, and a Hierarchy of Complexity Classes. JCSS, The authors of the papers shared the first Gödel Prize (1993). Computational Complexity, by Fu Yuxi PCP Theorem 84 / 88

86 1989 was an extraordinary year. László Babai, N. Nisan. Co-SAT has multi-prover interactive proofs, announcement, Nov. 27, C. Lund, L. Fortnow, H. Karloff, N. Nisan. The polynomial time hierarchy has interactive proofs, announcement, Dec. 13, A. Shamir. IP=PSPACE, announcement, Dec. 26, Computational Complexity, by Fu Yuxi PCP Theorem 85 / 88

87 Three weeks later (Jan. 17, 1990) another was sent out by L. Babai, L. Fortnow, and L. Lund. Non-Deterministic Exponential Time has Twoprover Interactive Protocols. FOCS CC The main theorem of the paper, MIP = NEXP, inspired almost all future development of PCP theory and a lot of future development of derandomization theory. The theorem can be interpreted as NEXP = PCP(poly, poly). L. Babai, L. Fortnow, L. Levin, and M. Szegedy scaled down the result by showing NP PCP(polylog, polylog). Checking Computation in Polylogarithmic Time. STOC Computational Complexity, by Fu Yuxi PCP Theorem 86 / 88

88 Instead of scaling down the time or space complexity of verifier, scale down the randomness and query complexity. Computational Complexity, by Fu Yuxi PCP Theorem 87 / 88

89 NP PCP(log log log, log log log). U. Feige, S. Goldwasser, L. Lovász, S. Safra, and M. Szegedy. Interactive Proofs and the Hardness of Approximating Cliques. FOCS J. ACM, NP = PCP(log, log). S. Arora and S. Safra. Probabilistic Checking of Proofs: A New Characterization of NP. FOCS J. ACM, NP = PCP(log, 1). S. Arora, C. Lund, R. Motwani, M. Sudan, and M. Szegedy. Proof Verification and the Hardness of Approximation Problems. FOCS J. ACM, Computational Complexity, by Fu Yuxi PCP Theorem 88 / 88