(67611) Advanced Topics in Complexity: PCP Theory November 24, 2004 Lecturer: Irit Dinur Lecture 6 Scribe: Sharon Peri Abstract In this lecture we continue our discussion of locally testable and locally decodable codes. We introduce the family of Reed-Müller Codes, which takes us diving into the geometry of m and the notion of Low Degree Testing. We further introduce the lines-table representation, and a mild yet powerful modification of local decodability, with reject as an optional output answer under cetain limitations 1 Locally Testable Codes In previous lectures we have seen Hadamard Code, Quadratic Functions Code and Reed- Solomon Code. We have proven the Hadamard Code and Quadratic Functions Code to be both locally Testable and locally decodable. However, it was also noted that the Hadamard Code suffers from a major disadvantage in efficiency, being exponential in the input size (i.e. an input word of size n bits is encoded by 2 n bits). For the Quadratic Functions Code things are even a little worse, as it is exp(poly(n)). The Reed-Solomon Code on the other hand is considerably efficient, being polynomial in the input size. But is it locally testable/decodable as well? 1.1 Local Testability of Reed-Solomon Code Recall the definition of Reed-Solomon Code: Definition 1 The Reed-Solomon code RS : k+1 n, wherein n =, is defined for a given vector ā = (a 0,..., a k ) k+1 as the vector (Pā(t)) t n, where Pā(x) = a 0 + a 1x + a 2x 2 +... + a k x k. An illustration of RS code word is presented in Figure 1. Note that the distance of the RS code is n k + 1. How can we locally test the Reed-Solomon Code? We can no longer perform linearity testing, since it no longer applies for n > 2 and k > 1. One idea is to query k+1 entries in the code word (Pā(t)) t, reconstruct the polynomial Pā by interpolation, evaluate it in k+2-th point i and compare with the i-th entry of the code word. Disadvantage: the number of queries is as the size of the input! Moreover, this lower bound of k+2 queries is tight; note that there are k+1 degrees of freedom for a polynomial of degree k. This excludes the Reed-Solomon Code from being locally testable, unless provided with an additional information on the input word. 2 Reed-Müller Code The Reed-Müller Code can be thought of as a generalization of the Hadamard Code and the Reed-Solomon Code, combining the advantages of each (efficiency of RS, local testability of Had) yet overcoming the exact same faults. m d [x 1, x 2... x m] a multivariate polynomial over m variables {x 1, x 2... x m} Denote by P d : of degree d. The degree of a multivariate polynomial i1,i 2,...i m a i1 i 2...i m x i 1 1 x i 2 2... x im m is defined to be the largest d for which there is a non-zero a i1 i 2...i m with i 1 + i 2 +...i m = d, i.e. the maximal degree of the monomials. For example, deg(x 2 1x 3 2) = 5. Question: How many monomials are there of degree d over m variables? Given the definition above, one can easily see that there are d+m d monomials in d[x 1, x 2... x m] which form a basis for this linear space. 6-1
Figure 1: RS Code Definition 2 The Reed-Müller Code RM : (d+m d ) m is defined for a multivariate polynomial over m variables p d [x 1, x 2... x m] (given by its d+m d coefficients) as RM(p) = p(x) : x m, the evaluation of p on each point of m. 2.1 The distance of the Reed-Müller Code Proposition 3 f, g d [x 1, x 2... x m], if f g, then d Prob x m(f(x) = g(x)) Proof Idea By induction on m. Note that for m = 1 the induction basis this is a direct consequence of the basic theorem of Algebra. 1 Corollary 4 The distance of Reed-Müller Code is m d m 1 since there are at most d m = d m 1 points in m which two different polynomials f and g agree on, their corresponding code words are different in at least m d m 1 coordinates. 2.2 Special cases of Reed-Müller Code Consider RM with parameters d = 1 and = 2. The input size n in this case is m+1 1 = m + 1 and the code word length is m = 2 m. Thus in this case we get the Hadamard Code - polynomials in d [x 1, x 2... x m] are affine functions (the monomials basis consists of {x 1, x 2... x m} {1}). If we set m = 1 and = n, d = k then we get the Reed-Solomon Code. 2.3 Establishing the parameters for Reed-Müller Code For an input size of length n and a degree d = log n, what is the magnitude of m? n = m+d d = m+d m mm m = O( log n ). log log n Also, for the code to have a large relative distance, we set = c d for a large enough c. The code word length is m = n O(1) which is reasonably efficient. The relative distance of the code is 1 1 and thus is constant dependent only in c. Can this code be tested in O(d = log n) c queries? 3 Geometry of m and Low Degree Testing Motivation: given a points table A representing a supposedly legitimate code word according to RM encoding, we wish to (locally) test whether P : m, with deg(p) = d, such that x m, A(x) = P(x). Particularly, we would like to determine if the given function A is close to such low degree polynomial P. The closeness of A to P is measured as usual in the Hamming metric. Formally, 1 Obviously, it must hold that > d, otherwise the proposition is trivial. 6-2
Figure 2: Axis Test Definition 5 Let A and P be two functions mapping a discrete domain D to a range R. Then dist(a,p) = Prob[A(x) P(x)] x D In other words, dist(a,p) is the portion of indices with different entries in A and P. Thus a low degree test, for some degree d, given a function/point-table A and a user-defined parameter ɛ, tests whether it holds for some P of degree d that dist(a,p) < ɛ. 3.1 Locally testing the Reed-Müller Code Our goal is to perform a low degree testing of RM in O(d) queries to a given point-table A. For intuition purposes, we first outline a suggestive course of the way to solution and formalize later. The principal tool we have in hand when dealing with polynomials is interpolation. Generally, we require O(d m ) points evaluations in order to reconstruct a degree d polynomial over m variables. We wish to use only O(d) such points; so where do we start? The first and probably most intuitive thing to do, is to consider an arbitrary axis i of m, algebraicly described as {x m : x i = t, x j = 0 j i}. Thus the first attempt we have is 3.1.1 The Axis Test (See Figure 2) fix d + 2 random points on some arbitrary axis i reconstruct the (restriction to the axis i of the) polynomial by interpolation using d + 1 of said points evaluate on the remaining point and compare with the corresponding entry in the points table Problem: there are only m such axes; what if most or all of their corresponding entries in the given points table are corrupted? Suppose that instead we fix some arbitrary point a = (a 1,..., a m) F m and consider the i-th axis parallel line that goes through this point {(a 1, a 2,..., t,... a m) m : t }. This brings to the following most natural extension and our second attempt: 6-3
Figure 3: Axis-Parallel Line Test 3.1.2 The Axis-parallel Line Test: 2 (See Figure 3) pick some arbitrary axis i pick some arbitrary point a = (a 1,...,, a m) fix d + 2 random points on this axis-parallel line that contains a, as defined earlier reconstruct the (restriction to the line of the) polynomial by interpolation evaluate on the remaining point and compare with the corresponding entry in the points table Although this suggestion seems fairly better than the previous, it still appears as if we are not yet exhausting all possibilities; indeed, why restricting ourselves to axis-parallel lines, when we can choose any arbitrary line in m? Let us then introduce the parametric representation of a line in the following Definition 6 A line in m is defined l x,h = {x + t h} t wherein x m, h m \ {0}. A line l x,h can also be considered as a univariate function l x,h : m, defined l x,h (t) = x+t h. Consider the restriction of P to the line l denoted by P l. This is a univariate function P(l(t)) = P(x + t h) of degree d in t. Example m = 3 d = 4 P(x,y,z) = xyz 2 + 3z 4 + 2x l = (0,0, 0) + t (1,1, 1) = (t, t, t) P(l(t)) = t t t 2 + 3t 4 + 2t = 4t 4 + 2t m Remark Since the geometry of m might be very different from the more intuitive would not be a bad idea to perform some cautious checks: m it 2 This test underwent a great deal of analysis in the works of [BFL90, BFLS91, FGL + 96], culminating in a proof that this test works for ɛ = Θ(1/md) wherein ɛ is the upper bound on dist(a, P). New analysis by [AS98] showed this test works for ɛ = Ω(1/m), removing entirely the dependence on the degree. Note that the dependence on the number of variables is unavoidable in this test: consider a function consisting different polynomials p x on (m 1) variables, and where f(x 1,..., x m) = p x1 (x 2,..., x m). Such f can be very far from any degree d polynomial over m variables, yet the test will fail only if i is chosen to be 1. [PS94] improve somewhat the result of [AS98] but of course cannot remove the dependence on m. 6-4
Figure 4: Point and Line Test Is there really exists a line between each two points of Are there really d + 2 different points on such line? The answer to both questions is positive and easily verified: For each two different points x,y m set h = x y and scale by the desired factor t \ {0}. To see that a line contains different points, note that l = {x+th} t. Assume that for some t 1, t 2 it holds that x+t 1h = x+t 2h. It follows that h(t 1 t 2) = 0, and since h 0 it must holds that (t 1 t 2) = 0, and thus t 1 = t 2. We might also like to exploit for our needs the following m? Lemma 7 deg(p) = d iff l m deg(p l ) = d Proof Idea One direction is immediate; if P is of degree d then so is its restriction to any line l m (which yields a univariate polynimal of degree d). The opposite direction is shown by induction, and using interpolation. We can now formalize 3.1.3 The Point and Line Test 3 (See Figure 4) randomly select a line in m (i.e. independently sample x R m and h R m \ {0}). randomly select d+2 points in l x,h and query their evaluation, i.e. pick t 1, t 2,... t d+2 R and query the points table in {x + t ih} d+1 i=1 reconstruct P l by interpolation using d + 1 of said points. evaluate P l on the remaining point and compare with the corresponding entry in the points table 3 This test was implicit in [GLR + 91] and further analyzed by [RS92, ALM + 92, AS97]. Note that it overcomes the dependence of ɛ on m we had in the axis-parallel line test. The analysis by [GLR + 91, RS92] yielded ɛ = Ω(1/d). [ALM + 92] combined the analysis of [RS92, AS98] to obtain ɛ = Ω(1). Using more powerful techniques, [AS97] showed the test achieves ɛ = 1 o(1). [RS97] proposed an even more general test, where a random line is replaced by a random plane, and achieves ɛ = 1 o(1). We will discuss this plane test in detail in the following lectures. 6-5
3.2 Local Decoding of Reed-Müller Code Given a code word (table) A which only a certain portion of it is spoiled, we wish to decode a point x. Thus, our promise is that there exists a low degree polynomial P such that dist(a,p) ɛ. For which settings of ɛ we can guarantee correct local decoding with high probabilty? What is the correlation/dependence of ɛ in the other parameters? Lemma 8 Given a table A : m and provided that P d [x 1,... x m] such that dist(a, P) < 1 3(d + 1) then x m there exists a probabilistic procedure that queries d + 1 entries in A and answers P(x) with probability > 2/3. Proof The decoding procedure works as follows: randomly select a line l x,h through x, specifically, l(0) = x. query the entries l(1),..., l(d+1) in the points table and reconstruct P l by interpolation. evaluate P l on x, i.e. P l (0) Observe that 1 t d + 1 it holds that l(t) is uniformly distributed - this is because h is randomly selected. In other words, y x: Prob h m(l x,h(t) = y) = 1/ m Denote by BAD the set of spoiled entries in A. Then, BAD Prob (l(t) BAD) = l:l(0)=x < 1 m 3(d + 1) and by applying the union bound, the probability that any of the d + 1 points are in BAD, and thus causing a wrong interpolation result, is < 1/3. It follows that as asserted. Prob [P l (0) = P(x)] > 2/3 l:l(0)=x In summary, we have a procedure for local decoding using d+1 = Θ(d = log n) queries, with dist(a,p) = Θ( 1 ), and encoding size of poly(n). For constant dist(a,p) and constant number d of queries there is not known a code whose length is less than exp(n). 3.3 Pairwise independence of points on a line Proposition 9 Proof x,y Prob(l(0) = x, l(1) = y) = Prob(x 1 = x, x 2 = y) l x 1,x 2 is left as an (easy) exercise to the reader. 4 The lines table We have already seen that the degree d gives us a tight lower bound on the number of queries (since there are d degrees of freedom). The idea now is to extend the RM code. Instead of evaluation on each point, we add information about the restriction of the polynomial P to each of the lines l x,h m. The code word is thus a table with an entry per line, altogether m ( m 1) 2m 2 entries (there ( 1) are 2 points pairs which define the same line; we choose one canonical representation to obtain a more compact encoding - this is called folding, since it is as if we collapsed all these redundancy representations into a single entry; See Figure 5). Each entry l consists of the d coefficients of P(l(t)) = d i=0 al ix i. (See Figure 6) 6-6
Figure 5: Folding the lines table Figure 6: The Lines Table 6-7
5 Local-decoding with reject We have seen a local-decoding procedure that makes O(d) queries, and can handle words of 1 distance <. We already mentioned that it is not known how to locally decode words 3(d+1) whose relative distance is some constant distance. Even if we employ the extra information given by the lines-table, an adversary is actually able to spoil all lines that pass through a fixed x, without violating the dist(a, P) < ɛ constraint (because this set of lines is a tiny fraction of all lines). We therefore introduce the concept of a local-decoding with reject procedure. We allow the decoder to output reject when it has detected inconsistency in the input (that guarantees the input is far from a legal input). We shall formalize the notion of local decodability with reject in the following theorem: Theorem 10 Given an input of a point x and a lines-table A, there exists a probabilistic procedure M such that 1. if A is a legitimate table of a polynomial P, M answers P(x) with probability 1. 2. if there exists a polynomial P such that dist(a,p) ɛ (dist(a,p) is the portion of spoiled entries, i.e. lines, in A) then P rob(m outputs reject or P(x)) > 2/3 3. if dist(a, P) > ɛ then M answers reject with high probability. Proof The procedure M is as follows: 1. randomly select a line l x,h through x for a random h 2. randomly select t \ {0} and set y = x + th 3. randomly select t F and a line l for which l (t ) = y 4. query f l = A(l x,h ) and f l = A(l ) 5. if f l (t) = f l (t ) output f l (0), otherwise output reject Analysis: Completeness: if A is indeed a legitimate table then M decodes correctly always. Soundness: first we note that if x was random, then Prob x,h [A(l x,h) = P l ] > 1 ɛ By applying the same analysis to the randomly selected y and l y,h (which works perfectly due to the pairwise independence property of points on line stated in Proposition 9), we have that: Prob[A(l ) = P l ] > 1 ɛ y,l y Specifically for the line l = l y,h y which interests us. If A(l ) however is inconsistent with the restriction of the global polynomial P to l, then by Proposition 3 the number of coordinates where A(l (t)) t P(l(t)) t is at least d. Observe that we access A(l ) at a random point t, so: Prob t [A(l (t )) P(l(t ))] > 1 d By applying the union bound on the probability to err (either A(l ) or A(l) are inconsistent with the global polynomial P yet the test A(l (t )) =? A(l(t)) passes) we have in total that Prob[M succeeds] 1 d ɛ > 2 as asserted (wherein the event of success of M is either 3 responding reject or decoding P(x) correctly). Part 3 of the theorem is harder to prove and requires somewhat heavier tools we are yet to develop in the following lectures. Remark Note that the distance ɛ of A from a legitimate low degree polynomial P that we have in this case is constant which does not depend in d, as opposed to the requirement in Lemma 8. The tradeoff occurs in the alphabet of the proof - the amount of information (e.g. the number of bits in the binary case) read in each query by the procedure M is significantly larger than in the previous case. 6-8
References [ALM + 92] Sanjeev Arora, Carsten Lund, Rajeev Motwani, Madhu Sudan, and Mario Szegedy. Proof verification and hardness of approximation problems. In IEEE Symposium on Foundations of Computer Science, pages 14 23, 1992. [AS97] [AS98] [BFL90] Sanjeev Arora and Madhu Sudan. Improved low-degree testing and its applications. pages 485 495, 1997. Sanjeev Arora and Shmuel Safra. Probabilistic checking of proofs: a new characterization of NP. Journal of the ACM, 45(1):70 122, 1998. L Babai, Lance Fortnow, and Carsten Lund. Non-deterministic exponential time has two-prover interactive protocols. In IEEE Symposium on Foundations of Computer Science, pages 16 25, 1990. [BFLS91] László Babai, Lance Fortnow, Leonid A. Levin, and Mario Szegedy. Checking computations in polylogarithmic time. pages 21 32, 1991. [FGL + 96] Uriel Feige, Shafi Goldwasser, Laszlo Lovász, Shmuel Safra, and Mario Szegedy. Interactive proofs and the hardness of approximating cliques. Journal of the ACM, 43(2):268 292, 1996. [GLR + 91] Peter Gemmell, Richard J. Lipton, Ronitt Rubinfeld, Madhu Sudan, and Avi Wigderson. Self-testing/correcting for polynomials and for approximate functions. In ACM Symposium on Theory of Computing, pages 32 42, 1991. [PS94] [RS92] [RS97] Alexander Polishchuk and Daniel A. Spielman. Nearly-linear size holographic proofs. In STOC 94: Proceedings of the twenty-sixth annual ACM symposium on Theory of computing, pages 194 203. ACM Press, 1994. Rubinfeld and Sudan. Self-testing polynomial functions efficiently and over rational domains. In SODA: ACM-SIAM Symposium on Discrete Algorithms (A Conference on Theoretical and Experimental Analysis of Discrete Algorithms), pages 23 32, 1992. Ran Raz and Shmuel Safra. A sub-constant error-probability low-degree test, and a sub-constant error-probability PCP characterization of NP. pages 475 484, 1997. 6-9