N P-Hardness of Approximately Solving Linear Equations Over Reals

Transcription

1 lectronic Colloquium on Computational Complexity, Report No. 11 (010) N P-Hardness of Approximately Solving Linear quations Over Reals Subhash Khot Dana Moshkovitz July 15, 010 Abstract In this paper, we consider the problem of approximately solving a system of homogeneous linear equations over reals, where each equation contains at most three variables. Since the all-zero assignment always satisfies all the equations exactly, we restrict the assignments to be non-trivial. Here is an informal statement of our result: it is N P-hard to distinguish whether there is a non-trivial assignment that satisfies 1 δ fraction of the equations or every non-trivial assignment fails to satisfy a constant fraction of the equations with a margin of Ω( δ). We develop linearity and dictatorship testing procedures for functions f : R n R over a Gaussian space, which could be of independent interest. Our research is motivated by a possible approach to proving the Unique Games Conjecture. 1 Introduction In this paper, we study the following natural question: given a homogeneous system of linear equations over reals, each equation containing at most three variables (call it 3Lin(R)), we seek a non-trivial approximate solution to the system. In the authors opinion, the question is poorly understood whereas the corresponding question over a finite field, say GF (), is fairly well understood Hås01, HK04. Over a finite field, an equation is either satisfied or not satisfied, whereas over reals, an equation may be approximately satisfied up to a certain margin and we may be interested in the margin. The main motivation for this research is a possible approach to proving the Unique Games Conjecture. More details appear in Section 1.5. We first describe our result and techniques and compare it with known results. 1.1 Our Result Fix a parameter b 0 1. Call a 3Lin(R) system b 0 -regular if every variable appears in the same number of equations, and the absolute values of the coefficients in all the equations are in the This is a new and improved version of our paper KM10 that established the same result, but under the Unique Games Conjecture. khot@cims.nyu.edu. Computer Science Department, Courant Institute of Mathematical Sciences, New York University. Research supported by NSF CARR grant CCF-08338, NSF xpeditions grant CCF , and BSF grant dana.moshkovitz@gmail.com. School of Mathematics, The Institute for Advanced Study. Research supported by NSF Grant CCF ISSN

2 range 1 b 0, b 0. Let X denote the set of variables so that an assignment is a map A : X R. For an equation eq : r 1 x 1 + r x + r 3 x 3 = 0, and an assignment A, the margin of the equation (w.r.t. A) is Margin(A, eq). = r 1 A(x 1 ) + r A(x ) + r 3 A(x 3 ). The all-zeroes assignment, x X, A(x) = 0, satisfies all the equations exactly, i.e. with a zero margin. Therefore, we will be interested only in the non-trivial assignments. For now, think of a non-trivial assignment as one where the distribution of its values {A(x) x X} is well-spread. Specifically, we may consider the Gaussian distributed assignments, for which the set of values {A(x) x X} is distributed (essentially) according to a standard Gaussian. Here is an informal statement of our result: Theorem 1. (Informal) There exist universal constants b 0, c (b 0 = works) such that for every δ > 0, given a b 0 -regular 3Lin(R) system, it is N P-hard to distinguish between: (YS Case): There is a Gaussian distributed assignment that satisfies 1 δ fraction of the equations. (NO Case): For every Gaussian distributed assignment, for at least a fraction c of the equations, the margin is at least c δ. A few remarks are in order. Since the 3Lin(R) instance is finite, we cannot expect the set of values {A(x) x X} to be exactly Gaussian distributed. The proof of our result proceeds by constructing a probabilistically checkable proof (PCP) over a continuous high-dimensional Gaussian space and then this idealized instance is discretized to obtain a finite instance. Theorem 1 holds in the idealized setting. The discretization step introduces, in the YS Case, a margin of at most γ in each equation, but γ can be made arbitrarily small relative to δ and hence this issue may be safely ignored. The distribution of values is still close to a standard Gaussian. We also set all variables with values larger than O(log(1/δ)) to zero. This applies to only poly(δ) fraction of the variables and hence does not have any significant effect on the result. Thus our assignment, in the YS Case, satisfies in particular: x X, A(x) b = O(log(1/δ)), A(x) = 1. (1) x X In the NO Case, our analysis extends to every assignment that satisfies (1), and the conclusion is appropriately modified (which is necessary since an assignment that satisfies (1) could still have a very skewed distribution of its values). A formal statement of the result appears as Theorem 6 in Section. 1. Optimality of Our Result, Squared-l versus l 1 rror, and Homogeneity Optimality: The result of Theorem 1 is qualitatively almost optimal as can be seen from a natural semi-definite programming relaxation and a rounding algorithm. Suppose there are N variables X = {x 1,..., x N }, m equations and j th equation in the system is r j1 x j1 + r j x j + r j3 x j3 = 0. Consider the following SDP relaxation where for every variable x i, we have a vector v i and b = O(log(1/δ)):

3 Minimize j m rj1 v j1 + r j v j + r j3 v j3, Such that x i X, v i b, vi = 1. x i X Suppose that in the YS Case, there is an assignment A that satisfies (1) and satisfies 1 δ fraction of the equations exactly. Then letting v i = A(x i )v 0 for some fixed unit vector v 0 gives a feasible solution to the SDP with the objective O(δ log (1/δ)). Hence the SDP finds a feasible vector solution with the same upper bound on the objective. Suppose the SDP vectors lie in d-dimensional uclidean space. Consider a rounding that picks a standard d-dimensional Gaussian vector r and defines an assignment A(x i ) = v i, r. It is easily seen that after a suitable scaling, with constant probability over the rounding scheme, we have: A(xi ) = 1, rj1 A(x j1 ) + r j A(x j ) + r j3 A(x j3 ) O(δ log (1/δ)). x i X j m Thus the margin r j1 A(x j1 ) + r j A(x j ) + r j3 A(x j3 ) is at most O( δ log(1/δ)) for almost all, say 99%, of the equations. Moreover, since x i X, v i b, after rounding all but poly(δ) fraction of the variables get values bounded by O(log (1/δ)), and these variables can be set to zero without affecting the solution significantly. Optimality of Semidefinite Programming Based Algorithms: As shown by Raghavendra Rag08, the Unique Games Conjecture, if true, implies that for every constraint satisfaction problem 1, a certain semi-definite programming based algorithm gives the best efficient approximation for the problem (as long as P = N P). Similar results hold for many other types of problems. In light of this, a natural question is whether one can point to even a single problem for which an SDP-based algorithm is provably optimal assuming P = N P (and not relying on the Unique Games Conjecture). One could argue that such a result was known for 3Sat: Håstad showed that (7/8 + ɛ)- approximation on satisfiable 3SAT formulas is N P-hard for any ɛ > 0 Hås01, while Karloff and Zwick gave a matching SDP-based algorithm KZ97. In our opinion, this example does not truly demonstrate the phenomenon that SDP-based algorithms are optimal. The reason is that the hardness result produces formulas where each clause contains exactly three literals. For formulas of this kind, a random assignment satisfies, in expectation, 7/8 fraction of the clauses. So, there is a simple, non SDP-based, algorithm achieving 7/8 approximation. To the best of our knowledge, 3Lin(R) is the first problem where a non-trivial SDP algorithm is shown to be optimal assuming P N P. The Squared-l versus l 1 rror: The SDP algorithm described above finds an assignment that minimizes the expected squared margin, i.e. j m Margin(A, j). Thus the problem of minimizing the squared-l error is a computationally easy problem. However, Theorem 1 implies that minimizing the l 1 error (i.e. j m Margin(A, j)), even approximately, is computationally hard (assuming P = N P). In the YS Case therein, all but δ fraction of the equations are exactly satisfied, and the variables are bounded by O(log(1/δ)). Hence the l 1 1 where variables range over a constant sized alphabet, and each constraint depends on a constant number of variables. 3

4 error is O(δ log(1/δ)). In the NO Case, for any Gaussian distributed assignment, for at least a constant fraction of the equations, the margin is at least Ω( δ), and hence the l 1 error is Ω( δ). Thus approximating the l 1 error within a quadratic factor is computationally hard; this is optimal since the squared-l minimization implies an l 1 approximation within a quadratic factor. Homogeneity: Theorem 1 holds for a system of linear equations that is homogeneous and it is necessary therein (in the NO Case) to restrict the distribution of values of an assignment. When the system of equations is non-homogeneous, one might hope to drop the restriction on the distribution of values. However, then a simple LP can directly minimize the l 1 error and hence one cannot hope for a theorem analogous to Theorem Techniques Dictatorship Test Over Reals Similar to most hardness results, our result proceeds by developing an appropriate dictatorship test. However, unlike most previous applications that use a dictatorship test over an n-dimensional boolean hypercube (or k-ary hypercube in some cases), we develop a dictatorship test over R n with the standard Gaussian measure. The test is quite natural, but its analysis turns out to be rather delicate. We think that the test itself is of independent interest and provide its high level overview here. Let N n denote the n-dimensional Gaussian distribution with n independent mean 0 and variance 1 coordinates. Let L (R n, N n ) be the space of all measurable real functions f : R n R with f = x N n f(x) <. This is an inner product space with the inner product f, g =. x N n f(x)g(x). A dictatorship is a function f(x) = x i0 for some fixed coordinate i 0 n. Given oracle access to a function f L (R n, N n ), we desire a probabilistic homogeneous linear test that accesses at most three values of f. The tests, over all choices of randomness, can be written down as a system of homogeneous linear equations over the values of f. We assume that the function f is non-trivial, i.e. f = 1, and anti-symmetric, i.e. f( x) = f(x) x Rn. In particular, f = 0. We desire a test such that a dictatorship function is a good solution to the system of linear equations, whereas a function that is far from a dictatorship, is a bad solution to the system. The test we propose is a combination of a linearity test and a coordinate-wise perturbation test. A dictatorship function satisfies all the equations of the linearity test and 1 δ fraction of the equations of the coordinate-wise perturbation test. A function that is far from a dictatorship, either fails miserably on the linearity test, or a constant fraction of the equations have a margin Ω( δ) on the coordinate-wise perturbation test. One starts out by observing that a dictatorship function is linear. Thus, for any λ, µ R such that λ + µ = 1, say λ = µ = 1, one can test whether f(λx + µy) = λx + µy, where x, y N n are picked independently. Clearly, a dictatorship function satisfies each such equation exactly. The condition λ + µ = 1 ensures that the query point λx + µy is also distributed according to N n. Note that we assume f = 1 and f = 0. Functions in A closer examination of the proof of Theorem 1 shows that the upper bound is actually O(δ); for the equations that are not satisfied, the margin itself is distributed according to a standard Gaussian. 4

5 L (R n, N n ) have the Hermite representation; in particular, f can be decomposed into the linear and non-linear parts: f = f =1 + e, f =1 = n a i x i, i=1 f =1, e = 0. Note that 1 = f = f =1 + e. A simple Fourier analytic argument shows that unless e 0.01, the linearity test fails with large average squared margin (and the analysis of the test is over). Therefore we may assume that e Assume for now, that e 0 and hence the function is linear: f = f =1 = n i=1 a ix i and n i=1 a i = 1. We introduce the coordinate-wise perturbation test to ensure that the coefficients {a i } n i=1 are concentrated on a bounded set. This makes sense because for a dictatorship function, there is exactly one non-zero coefficient. The test picks a random point x N n and for a randomly chosen δ fraction of the coordinates, each chosen coordinate is re-sampled independently from a standard Gaussian. If x is the new point, then one tests whether f( x) f(x) = 0. Note that for a dictatorship function, the above equation is satisfied with probability 1 δ, whereas with probability δ, the margin is distributed as a mean-0 variance- Gaussian. On the other hand, if f = n i=1 a ix i is far from a dictatorship, then coefficients {a i } n i=1 are spreadout, and with a constant probability, the margin is Ω( δ). This is intuitively the idea behind the test; however the presence of the non-linear part e complicates matters considerably. ven though e 0.01, we are dealing with margins of the order of δ, and the non-linear part e could potentially interfere with the above simplistic argument. We therefore need a more refined argument. We observe that since f = f =1 + e, f( x) f(x) = (f =1 ( x) f =1 (x)) + (e( x) e(x)). When f =1 = n i=1 a ix i is spread-out, the first term in the above equation, namely f =1 ( x) f =1 (x), is Ω( δ) with a constant probability as we observed above. The same can be concluded about the left hand side of the equation, namely f( x) f(x), unless the second term e( x) e(x) interferes in a very correlated manner. If this happens, then the function e must be sensitive to noise along a random set of δn coordinates. We add a test ensuring that e is insensitive to noise of comparable magnitude in a random direction. We then show that the two behaviors are contradictory, using a Fourier analytic argument that relies, in addition, on the cut-decomposition of line/l 1 metrics. 1.4 The Reduction The NP-hardness proof proceeds by using the dictatorship test discussed in the previous section as a gadget in a reduction. One might expect the reduction to go along the lines of Håstad s reduction for the Boolean 3Lin, however the real case confronts us with serious challenges. A key component in Håstad s reduction addresses the following problem (in the Boolean case): The Restriction Problem. Given oracle access to a function f : F n F that is approximately a dictatorship function (for the sake of exposition, assume that for some i 0 n, on 5

6 most points y F n we have f(y) = y i0 ), and to a function g : F m F, m l = n, test whether g is the following restriction of f: g(x 1,..., x m ) = f(x 1,..., x 1,......, x m,..., x m ), where each x i repeats l times. The test should check a linear equation on three values of f and g (altogether). The restriction problem can be solved in the Boolean case F = {0, 1} and for any finite field F via self-correction. The tester is as follows: 1. Pick x F m, y F n uniformly at random.. Set z = (x 1,..., x 1,......, x m,..., x m ) F n. 3. Accept if and only if g(x 1,..., x m ) = f(y) + f(z y). Note that when f is a dictatorship function and g is the appropriate restriction of it, the test always accepts (in fact, linearity of f suffices). Also note that the test is linear in three values of f and g. The test works also when f is close to a dictatorship function f, because the points y and z y are uniformly distributed in F n, and with high probability, f evaluates to the correct dictatorship function f at both the points. Note that z itself is not uniformly distributed in F n, but still f(y) + f(z y) yields, with high probability, the correct value f(z). Now consider the analogous problem for functions in Gaussian space. In this case, we can at most gurantee that with high probability over y N n it holds that f(y) y i0. The tester we showed for the finite field case no longer works: even when x R m and y R n are Gaussian distributed, the point z y may not be distributed as a Gaussian in R n. We instead proceed as follows. Define a subspace S of R n as: S := {(x 1,..., x 1,......, x m,..., x m ) x 1,..., x m R}, where each x i repeats l times. Let π : S R m denote the projection that for 1 i m, picks the common coordinate from the i th block. The tester is as follows: 1. Pick y N n.. Write y = y + y, where y S and y S. Set y = y y and let y R m be the vector y := l π(y ). It is easily seen that y is distributed as N m. 3. Check that g(y ) = l f(y) + f(y ). It can be easily checked that if f is a dictatorship and g its appropriate restriction, then the test equation holds. Note that y, y are both Gaussian distributed, and thus if f is close to a dictatorship f, then with high probability f(y) f(y) and f(y ) f(y ) and l f(y)+f(y ) l f(y)+ f(y ) = g(y ) if g is the appropriate restriction of f. One caveat however is that the error involved in the approximating f by f gets multiplied by l in this calculation and if l is too large, the equation becomes rather meaningless. 6

7 How large is l in hardness applications? This parameter corresponds to the Outer PCP (aka Label Cover) being l to 1. In standard hardness results, such as Håstad s, one uses the Parallel Repetition Theorem Raz98 and l = (1/ε) O(1), where ε is the soundness error of the Outer PCP. Moreover, the soundness error ε usually needs to be tiny, which in turn requires l to be large, and this is prohibitive in our application. To avoid having large l, we do not use parallel repetition, and work instead with the basic PCP Theorem AS98, ALM This PCP has high soundness error (say 0.99), but is adequate for the purpose of proving Theorem 1. The reason is that Theorem 1 is also a hige error hardness result we only guarantee in the NO case that a constant fraction of the equations (say 1%) fail with a good margin. Still, working with a high error PCP seems impossible at first sight. The dictatorship test gives rise to a list decoding of possible dictatorship functions, rather than identifying a single dictatorship function, and this seems to call for an Outer PCP with low error. Indeed, virtually all existing hardness results rely on PCP with low error for the same reason (where one of the dictatorship coordinates in the decoded list is picked at random as a candidate label/answer for the Outer PCP). To circumvent the need for a low error PCP, we build a new Outer PCP. Suppose that the basic PCP corresponds to a set of variables Z, a set of tests/constraints C, and each test depends on d variables. The new Outer PCP is as follows: 1. The verifier picks independently at random k possible tests c 1,..., c k C, an index i k, and a variable z in the test c i.. The verifier sends the tuple u = (c 1,..., c k ) to the first prover and the tuple (c 1,..., c i 1, z, c i+1,..., c k ) to the second prover. 3. Both provers are supposed to answer with the values of all the variables in the tuple they were given. 4. The verifier checks that provers answers are consistent and satisfy the tests. Note that this outer PCP is as sound as the basic PCP. Moreover, it is l = d to 1 where each constraint depends on d variables for a fixed constant d. The crux of the analysis is that a short list of each prover s answers in this PCP translate (with high probability) into just one answer for a random coordinate i k on which the basic PCP test is actually performed. Thus, via this Outer PCP, we convert the list decoding setting into a unique decoding setting, and allow the reduction to go through. We make the argument formal by using the technique of correlated sampling KT0, Hol09 to choose a consistent element from two lists, one for each prover. Due to the specific Outer PCP construction, our reduction maps instances of Sat of size N to instances of 3Lin(R) of size N O(k), k = (1/δ) O(1), where δ is the parameter of Theorem 1. Hence, the reduction incurs a blow-up of N (1/δ)O(1) in the size. This blow-up matches the blow-up predicted by the recent work of Arora, Barak and Steurer ABS10 for unique games. We remark that the actual analysis is much more complex than hinted here. The reason is that the 3Lin(R) instance constructed by the reduction consists of several functions f : R n R that could have widely varying norms, whereas list decoding via dictatorship testing can be extracted only from functions with non-negligible norms, and the eventual prover strategies have to be weighted delicately according to these norms. 7

8 1.5 Comparison with Known Results and Motivation for Studying 3LIN(R) MinUncut: Given a graph G(V = N, ), the MinUncut problem seeks a cut in the graph that minimizes the number of edges not cut. It can be thought of as an instance of Lin(R) where one has variables {x 1,..., x N }, and for every edge (i, j), a homogeneous equation: x i + x j = 0, and the goal is to find a boolean, i.e. { 1, 1}-valued assignment that minimizes the number of unsatisfied equations. Khot et al KKMO07 show that assuming the UGC, for sufficiently small δ > 0, given an instance that has an assignment that satisfies all but δ fraction of the equations, it is N P-hard to find an assignment that satisfies all but π δ fraction of the equations. This result is qualitatively similar to Theorem 1, but note that the variables are restricted to be boolean. Balanced Partitioning: Given a graph G(V = N, ), the Balanced Partitioning problem seeks a roughly balanced cut (i.e. each side has Ω(N) vertices) in the graph that minimizes the number of edges cut. It can again be thought of as an instance of Lin(R) where one has variables {x 1,..., x N }, and for every edge (i, j), a homogeneous equation: x i x j = 0, () and the goal is to find a { 1, 1}-valued and roughly balanced assignment that minimizes the number of unsatisfied equations. Arora et al AKK + 08 show that assuming a certain variant of the UGC, given an instance of Balanced Partitioning that has a balanced assignment that satisfies all but δ fraction of the equations, it is N P-hard to find a roughly balanced assignment that satisfies all but δ c fraction of the equations. Here 1 < c < 1 is an arbitrary constant and for every such c, the result holds for all sufficiently small δ > 0. The result is again qualitatively similar to Theorem 1. In fact, the result holds even when the variables are allowed to be real valued, say in the range 1, 1, as long as the set of values is well-separated. Imagine picking a random λ 1, 1 and partitioning the variables (i.e. vertices of the graph) into two sets depending on whether their value is less or greater than λ. The cut is roughly balanced if the set of values is well-separated, and the probability that an edge (i, j) is cut is x i x j. Thus solving the Lin(R) instance w.r.t. l 1 error is equivalent to solving the Balanced Partitioning problem. Motivation for Studying 3LIN(R): The hardness results for the MinUncut and the Balanced Partitioning problem cited above are known only assuming the UGC. It would be a huge progress to prove these results without relying on the UGC and could possibly lead to a proof of the UGC itself. Due to the close connection of both the problems to the Lin(R) problem, it is natural to seek a hardness result for the Lin(R) problem w.r.t. the l 1 error. This is the main motivation behind the work in this paper. We propose that understanding the complexity of the 3Lin(R) problem might help us make progress on the UGC: the plan would be to (1) prove Theorem 1 (which we do) and then () give a gap-preserving reduction from the 3Lin(R) to Lin(R). Regarding the second step, the authors currently have a candidate reduction from 3Lin(R) to Lin(R) along with counterexamples showing that the reduction, as is, does not work. The authors believe that there might be a way to fix the reduction. Guruswami and Raghavendra s Result: Our result is incomparable to that in GR09. Their result shows that given a system of non-homogeneous linear equations over integers (as 8

9 well as over reals), with three variables in each equation, it is N P-hard to distinguish 1 δ satisfiable instances from δ satisfiable instances. The instance produced by their reduction is non-homogeneous, a good solution in the YS Case consists of large (unbounded) integer values, the result is very much about exactly satisfying equations, and in particular does not give, if any, a strong gap in terms of margins, especially relative to the magnitude of integers in a good solution. Comparison with Results over GF (): We argue that, in order to make progress on Min- Uncut, Balanced Partitioning and UGC, studying equations over reals may be the right thing to do, as opposed to equations over GF (). As we discussed before, the Balanced Partitioning problem can be thought of as an instance of Lin(R) (as in quation ()) where one seeks to minimize l 1 error and the set of values is a well-separated set in 1, 1. Assuming a UGC variant, we know that (δ, δ c )-gap is N P-hard for c > 1, whereas Theorem 1 yields a similar gap for 3Lin(R), with a stronger conclusion that a constant fraction of equations have a margin at least Ω( δ). We pointed out that such a gap is also the best one may hope for. Thus the 3-variable case seems qualitatively similar to the -variable case in terms of hardness gap that may be expected. For equations over GF (), the two cases are qualitatively very different. Suppose one thinks of the Balanced Partitioning problem as an instance of Lin(GF ()) where a cut is a GF () valued balanced assignment, and one introduces an equation x i x j = 0 for each edge (i, j). Its generalization to homogeneous equations with three variables, namely 3Lin(GF ()), turns out to be qualitatively very different. Holmerin and Khot HK04 show a hardness gap (in terms of fraction of equations left unsatisfied by a balanced assignment) of (δ, 1 ) which is qualitatively very different from the (δ, δ c ) gap that may be expected for Lin(GF ()). 1.6 Overview of the Paper In Section, we formally state our main result (Theorem 6) and provide preliminaries on Hermite representation of functions in L (R n, N n ). In Section 3, we propose and analyze the linearity test that is used as a sub-routine in the dictatorship test proposed and analyzed in Section 4. The reduction, proving our main result, is presented in Section 5. The soundness analysis is first presented in a simplified setting and then in the general setting. The entire reduction is presented in a continuous setting and then discretized in Section 5.7. Problem Definition, Our Result, and Preliminaries We consider the problem of approximately solving a system of homogeneous linear equations over the reals. ach equation depends on (at most) three variables. The system of equations is given by a distribution over equations, meaning different equations receive different weights. Definition (Robust-3Lin(R) instance). Let b 0 1 be a parameter. A Robust-3Lin(R) instance is given by a set of real variables X and a distribution over equations on the variables. ach equation is of the form: r 1 x 1 + r x + r 3 x 3 = 0, where the coefficients satisfy r 1, r, r 3 1 b 0, b 0 and x 1, x, x 3 X. Definition 3 (Assignment to Robust-3Lin(R) instance). An assignment to the variables of a Robust-3Lin(R) instance (X, ) is a function A : X R. An equation r 1 x 1 + r x + r 3 x 3 = 0 9

10 is exactly satisfied by A if r 1 A(x 1 ) + r A(x ) + r 3 A(x 3 ) = 0. The equation is β-approximately satisfied for an approximation parameter β, if r 1 A(x 1 ) + r A(x ) + r 3 A(x 3 ) β. Notation. The set of variables appearing in an equation eq : r 1 x 1 +r x +r 3 x 3 = 0 is denoted as X eq = {x 1, x, x 3 }. The assignment A will usually be clear from the context. We use the shorthand eq to denote the margin r 1 A(x 1 ) + r A(x ) + r 3 A(x 3 ). An assignment that assigns 0 to all variables trivially exactly satisfies all equations. Hence, we use a measure for how different the assignment is from the all-zero assignment, locally (per equation) and globally (on average over all equations): Definition 4 (Assignment norm). Let (X, ) be a Robust-3Lin(R) instance. Let A : X R be an assignment. Define the squared norm of A at equation eq to be: Define the squared norm of A to be: A eq = A = A(x). x X eq Aeq. eq Remark.1. We will sometimes refer to a distribution on the set of variables X induced by first picking an equation from the distribution and then picking a variable at random from that equation. If D denotes this distribution on variables, then clearly A = x D A(x). Legitimate assignments A are required to be normalized A b, b for some parameter b. We seek to maximize: = 1 and bounded A : X val β (X,) (A). = χ eq β A eq, (3) eq where χ eq β is indicator function of the event that eq β. In words, we seek to maximize 3 the total squared norm of equations that are satisfied with margin of at most β. Definition 5 (Robust-3Lin(R) problem). Let b 0 1, b 0 and 0 < β < 1 be parameters. Given a Robust-3Lin(R) instance where the coefficients are in 1 b 0, b 0 in magnitude, the problem is to find an assignment A : X b, b of norm A = 1 that maximizes valβ (X,) (A). We are now ready to formally state our result: Theorem 6 (Hardness of Robust-3Lin(R)). There exist universal constants b 0 = and c, s > 0, such that for any γ, δ > 0, there is b = O(log(1/δ)), such that given an instance (X, ) of Robust-3Lin(R) with the magnitude of the coefficients in 1 b 0, b 0, it is N P-hard to distinguish between the following two cases: 3 We recommend that the reader takes a pause and convinces himself/herself that this is a reasonable measure of how good an assignment is. Since an assignment may be very skewed, assigning large values to a tiny subset of variables and zero to the rest of the variables, simply maximizing the fraction of equations satisfied does not make much sense. 10

11 Completeness: There is an assignment A : X b, b with A = 1, such that val γ (X,)(A) 1 δ. Soundness: For any assignment A : X b, b with A = 1, it holds that val c δ (X,)(A) 1 s. We note three points: (1) The parameter γ is to be thought of as negligible compared to δ and essentially equal to 0. Our reduction is best thought of as a continuous construction on a Gaussian space, and the parameter γ arises as a negligible error involved in discretization of the construction. () In the YS Case, we can say more about how the good assignment looks like. Consider the distribution D induced on variables by first picking an equation eq and then picking one of the variables in the equation. The values taken by the good assignment, w.r.t. D, are distributed (essentially) as a standard Gaussian, and can be truncated to b = O(log(1/δ)) in magnitude without affecting the result. (3) In the NO Case, if an assignment has either values bounded in 1, 1 or values distributed, w.r.t. D, (essentially) as a standard Gaussian, it is indeed the case that a constant fraction of the equations fail with a margin of at least c δ, proving informal Theorem 1..1 Fourier Analysis Over Gaussian Space Gaussian Space. Let N n denote the n-dimensional Gaussian distribution with n independent mean-0 and variance-1 coordinates. L (R n, N n ) is the space of all real functions f : R n R with x N n f(x) <. This is an inner product space with inner product f, g. = f(x)g(x). x N n Hermite Polynomials. For a natural number j, the j th Hermite polynomial H j : R R is H j (x) = 1 j! ( 1) j e x / dj dx j e x /. The first few Hermite polynomials are H 0 1, H 1 (x) = x, H (x) = 1 (x 1), H 3 = 1 6 (x 3 3x), H 4 (x) = 1 6 (x4 6x + 3). The Hermite polynomials satisfy: Claim.1 (Orthonormality). For every j, H j, H j = 1. For every i j, H i, H j = 0. In particular, for every j 1, x N H j (x) = 0. Claim. (Addition formula). ( ) x + y H j = 1 j/ j (j ) H k (x)h j k (y). k k=0 11

12 Fourier Analysis. The multi-dimensional Hermite polynomials defined as: H j1,...,j n (x 1,..., x n ) = n H ji (x i ), form an orthonormal basis for the space L (R n, N n ). very function f L (R n, N n ) can be written as f(x) = ˆf(S) HS (x), S N n where S is multi-index, i.e. an n-tuple of natural numbers, and the ˆf(S) R are the Fourier coefficients of f. The size of a multi-index S = (S 1,..., S n ) is defined as S = n i=1 S i. The Fourier expansion of degree d is f d = ˆf(S)H S d S (x), and it holds that i=1 lim f f d = 0. d The linear part of f is f =1 = f 1 f 0. When f is anti-symmetric, i.e. x R n, f( x) = f(x), we have f( 0) = f = 0 and f 0 0. Influence. We denote the restriction of a Gaussian variable x N n to a set of coordinates D n by x D. The influence of a set of coordinates D n on a function f L (R n, N n ) is I D (f). = x D Var f(x). x D The influence can also be expressed in terms of Fourier spectrum of f: Proposition.3. I D (f) = S D φ ˆf(S), where S D φ denotes that there exists i D such that S i 0. Note that S N n is a multi-index and D n is a subset of coordinates. Perturbation Operator. The perturbation operator (more commonly known as the Ornstein- Uhlenbeck operator) T ρ takes a function f L (R n, N n ) and produces a function T ρ f L (R n, N n ) that averages the value of f over local neighborhoods: T ρ f(x) = f(ρx + 1 ρ y). y N n The Fourier spectrum of T ρ f can be obtained from the Fourier spectrum of f as follows: Proposition.4. T ρ f = S ρ S ˆf(S) HS. 1

13 . Distributions: ntropy and Distance The entropy of a probability distribution D over a discrete probability space Ω is ntropy satisfies the following properties: H(D). = a Ω D(a) log 1 D(a). Proposition.5. CT91 For distributions D, D 1,..., D k over Ω, is Range: 0 H(D) log Ω ; the lower bound is attained by constant distributions; the upper bound is attained by the uniform distribution. Concavity: H( 1 k k i=1 D i) 1 k k i=1 H(D i). Sub-additivity: H(D 1... D k ) k i=1 H(D i). The statistical distance between distributions D 1 and D over a discrete probability space Ω (D 1, D ) =. 1 D 1 (a) D (a). a Ω A distribution with nearly maximal entropy is close to uniform: Proposition.6. CT91 log Ω H(D) 1 ln D Uniform 1. The squared Hellinger distance between D 1 and D is H(D 1, D ). = 1 ( D1 (a) D (a)) = 1 D1 (a)d (a). a Ω We have the following connection between the Hellinger distance and the statistical distance: Proposition.7. Pol0 a Ω H(D 1, D ) (D 1, D ) H (D 1, D ). 3 Linearity Testing We show how to perform linearity testing for functions in L (R n, N n ) using linear equations on three variables each. Linear functions always exactly satisfy the linear equations. Functions with a large non-linear part give rise to heavy margins in the equations. The linearity test we show resembles linearity testing in finite fields (see, e.g., BLR93, BCH + 96). We change it slightly so as to guarantee that all the queries to the function are distributed according to the Gaussian distribution. Linearity Test: 13

14 Given oracle access to a function f L (R n, N n ), f anti-symmetric, i.e., f( x) = f(x) for every x R n. Pick x, y N n and test: f(x) + f(y) + ( f x + y ) = 0. Note that a linear function always exactly satisfies the test s equation. The following lemma shows that if the test s equations are approximately satisfied, then the weight of f s non-linear part is small: Lemma 3.1 (Linearity testing). Let f L (R n, N n ), f anti-symmetric, i.e., f( x) = f(x) for every x R n. Then f f =1 f(x) + f(y) + ( f x + y ). x,y N n Proof. Since x and y are independent, the variables x, y and x+y to N n. Also f is anti-symmetric. Hence, x,y N n f(x) + f(y) + f Writing in terms of the Fourier representation: ( ) x + y f(x)f = x,y x,y By Claim., Hence, x,y ( ) x + y f(x)f are all distributed according ( x + y ) = 4 f 4 ( x + y f(x)f ). (4) x,y = S,T = S,T ( ) xi + y i H Ti = 1 T i/ = S,T ˆf(S) ˆf(T ) S,T N n ˆf(S) ˆf(T )HS (x)h T ˆf(S) ˆf(T ) x,y n ˆf(S) ˆf(T ) n i=1 T i l=0 1 T i/ i=1 n x,y i=1 (Ti l T i l=0 ( x + y ) ( ) xi + y i H Si (x i )H Ti H Si (x i )H Ti ( xi + y i ). ) H l (x i )H Ti l(y i ). (Ti ) l x H Si (x i )H l (x i ) y H Ti l(y i ). By Claim.1, y H Ti l(y i ) = 0, unless l = T i, and x H Si (x i )H l (x i ) = 0, unless l = S i. Thus, x,y ( ) x + y f(x)f = ( ) 1 S ˆf(S) S 1 ( ) 1 f =1 + f f =1, (5) 14

15 where we used ˆf( 0) = 0 that follows from anti-symmetry. inequality (5), x,y N n f(x) + f(y) + f By combining equality (4) and ( x + y ) 4 f 4 f =1 4 f f =1 = (4 ) f f =1 f f =1. 4 Dictator Testing In this section we devise a dictator test, i.e., a test that checks whether an anti-symmetric real function in L (R n, N n ) is a dictator (that is, of the form f(x) = x i for some i n) or far from a dictator. We consider a function to be close to a dictator if it satisfies the following definition: Definition 7 ((J, s, Γ)-Approximate linear junta). An anti-symmetric function f L (R n, N n ) with linear part f =1 = n i=1 a ix i, is called a (J, s, Γ)-approximate-linear-junta, if: f =1 = n i=1 a i (1 s) f. i:a i 1 J f a i Γ f. An approximate linear junta has almost all the Fourier mass on its linear part, and this linear part is concentrated on at most J coordinates: Let I = { i a i 1 J f }. Then I J, and f i I a ix i (s + Γ) f. Our test will produce equations that dictators almost always satisfy exactly. On the other hand, functions that are not even approximate linear juntas fail with large margin. Theorem 8 (Dictator testing). For every constant 0 < Γ 0.01, there are constants s, c > 0 such that the following holds. For every sufficiently small δ > 0, there is a dictator test given by a distribution over equations, where each equation depends on the value of f on at most three points in R n. The test satisfies the following properties: 1. Uniformity: The distribution over R n obtained from picking at random an equation and x such that f(x) is queried by the equation, is Gaussian N n.. Bound on coefficients: All the coefficients in the equations are in 1 b 0, b 0 in magnitude where b 0 is a universal constant (b 0 = works). 3. Completeness: If f(x) = x i for some i n, then χ eq >0 f eq δ. eq 4. Soundness: For any anti-symmetric function f L (R n, N n ), f ( 10, s, Γ)-approximate linear junta, then Γδ χ eq >c δ f eq s 100. eq = 1, if f is not a 15

16 Remark 4.1. Note that it follows from the soundness guarantee that for an anti-symmetric function f L (R n, N n ) with arbitrary non-zero norm, if f is not a ( 10, s, Γ)-approximate Γδ linear junta, then χ eq >c δ f f eq s eq 100 f. This is obtained by applying the theorem with the normalized version of f, i.e., f f. The test will consist of three steps: (i) Linearity test that rules out functions that are not well-approximated by their linear parts. (ii) Coordinatewise perturbation test that checks that the function does not change by re-sampling a small fraction of the coordinates. (iii) Random perturbation test that guarantees that the function does not change much if perturbing the input slightly in a random direction. We achieve the effect of this test by instead doing two correlated linearity tests, in order to keep the coefficients in the range 1, in magnitude. Dictator Test: Given oracle access to a function f L (R n, N n ), f anti-symmetric. With equal probability, perform one of these three tests: 1. Linearity test on f, as in Section 3.. Coordinatewise perturbation test: (a) Pick x, y N n. Pick x N n as follows: for i = 1,,..., n, independently, with probability 1 δ, set x i = x i, and with probability δ, set x i = y i. (b) Test: 3. Random perturbation test (in disguise): f(x) f( x) = 0. (a) Pick y, z N n. Let x = y+z, w = y z, and x = (1 δ)x + δ δ w ( 1 δ = + = λ 1 y + λ z (say). δ δ ) y + ( ) 1 δ δ δ z (b) Note that λ 1, λ are very close to 1. Test with equal probability: f(x) 1 f(y) 1 f(z) = 0. f( x) λ 1 f(y) λ f(z) = 0. Note that in the random perturbation test, x = (1 δ)x + δ δ w and x is independent of w. Thus x can indeed be thought of as a perturbation of x in a random direction. The uniformity property, as well as the bound on the coefficients, hold by the definition of the tests. Denote the distribution on all equations by, and the three sub-distributions by: l (linearity tests), c (coordinatewise perturbation tests), r (random perturbation tests). 16

17 Completeness: A dictator function f, being a linear function, always exactly satisfies the linearity test and the random perturbation test. As for the coordinatewise perturbation test, eq c χ eq >0 f eq δ f = δ. Soundness: In the following, O( ) and Ω( ) notations will hide universal constants. Let Γ be the given constant in Theorem 8. We will pick s and c to be constants eventually, but throughout the proof, retain the dependence on the parameters. Assume for now that c s 0.01 and 3 s Γ. The parameter δ is thought of as tending to zero. Let f L (R n, N n ) be an anti-symmetric function, f 10 = 1, f is not a (J =, s, Γ)- Γδ approximate linear junta. Assume, for the sake of a contradiction, that χ eq c δ f eq 1 s 100. eq Denote the non-linear part of f by e = f f =1 (since f is anti-symmetric, f 0 0). We handle the cases that e s and e > s separately. Case e > s: By Lemma 3.1, eq l eq e > s. By Cauchy-Schwarz inequality, for every equation, 4 we have eq 1 f eq, so s < eq χ eq >c δ 1 f eq + c δ 1 χ eq >c δ f eq + s eq l eq l eq l 3. Since the distribution is average of distributions l, c, and r, we get χ eq >c δ f eq eq This contradicts our assumption that eq 1 3 χ eq >c δ f eq eq l χ eq c δ f eq 1 s 100. > s 100. Case e s: at most c δ. We first show that in this case, almost every equation is satisfied with margin Lemma 4.1. The probability that a dictator test equation chosen at random is c δ-approximately satisfied is at least s. Proof. We begin by showing that for x N n, f(x) 3 s 4 except with probability at most 6 3 s. When x N n, except with probability at most 4 3 s, we have that e(x) s e s/3 4. Write f =1 (x) = n i=1 a ix i. When x N n, we have that f =1 (x) is normal with mean 0 and variance n i=1 a i = 1 e Thus, except with probability at most 3 s, we have that f =1 (x) s. Overall, except with probability at most 6 3 s, we have that f(x) f =1 (x) e(x) s 3 s 3 s 4. Assume, for the sake of a contradiction, that with probability at least 7 3 s, a dictator test equation has margin at least c δ. An equation has at most three variables, and each of these is distributed as N n. With probability at least 7 3 s 6 3 s = 3 s, it also holds that the 4 The linearity testing equation is of the form f(x) + f(y) f(z) = 0. Here eq = f(x) + f(y) f(z) and f eq = f(x) +f(y) +f(z) 3. 17

18 first variable, say f(x), in the equation has magnitude f(x) f eq 1 3 f(x) s/3 48. Hence, χ eq >c δ f eq 3 s s/3 eq 48 > s 100. This contradicts our assumption, and the claim follows. 3 s 4. For such an equation, In the sequel we inspect the change in e as we perturb the input. We show that our assumptions on f (made towards a contradiction) imply that e may change somewhat as a result of a perturbation in a random direction, yet changes noticeably more as a result of a coordinatewise perturbation. We will later show that these two behaviors are contradictory. Lemma 4. (e is noise-stable for random perturbation). (Under the assumptions we made towards a contradiction) Let x, x be picked as in the random perturbation test. Then, with probability at least 1 O( 3 s), e(x) e( x) O( 3 s) δ. Proof. Since the random perturbation test is performed with probability 1 3, from Lemma 4.1, with probability at least 1 O( 3 s), we have f(x) 1 f(y) 1 f(z) c δ, f( x) λ 1 f(y) λ f(z) c δ. Since f = f =1 + e, and f =1 is linear, the above inequalities are really inequalities for e: e(x) 1 e(y) 1 e(z) c δ, e( x) λ 1 e(y) λ e(z) c δ. Combining the two inequalities and substituting for λ 1 and λ, we get: e(x) e( x) c δ + O( δ)( e(y) + e(z) ). By Markov inequality, except with probability at most 3 s, it holds that e(y) e / 3 s s /3. The same applies to e(z). Therefore, with probability at least 1 O( 3 s), e(x) e( x) c δ + O( 3 s δ) = O( 3 s) δ. Lemma 4.3 (e is noise-sensitive coordinatewise). (Under the assumptions we made towards a contradiction) Let x, x N n be picked as in the coordinatewise perturbation test. Then, with probability at least Ω(1), we have e(x) e( x) Ω( Γδ). 18

19 Proof. Write f =1 = n i=1 a ix i. Since f = f =1 + e, we have e(x) e( x) f =1 (x) f =1 ( x) f(x) f( x) n = a i (x i x i ) f(x) f( x). i=1 From Lemma 4.1, we know that except with probability O( 3 s), the second term f(x) f( x) is at most c δ. Thus it suffices to show that with probability Ω(1), the first term is at least Ω( Γδ) (and to choose c and s sufficiently small). Recall that the test picks the pair (x, x) as follows: First pick a set D n by including in it every i n independently with probability δ. Pick x, y N n independently. For every i D, set x i = x i, and for every i D, set x i = y i. Thus for a fixed D, n i=1 a i (x i x i ) = i D a i (x i y i ), which is a normal variable with mean 0 and variance i D a i. We will show that the variance is at least Γδ with probability 0.9 over the choice of D. Whenever this happens, the random variable exceeds Ω( Γδ) in magnitude with probability Ω(1) and we are done. Let I = { i n a i 1 } J be the non-influential coordinates. Since f is not a (J, s, Γ)- approximate linear junta, and e s, we must have i I a i Γ. A standard Hoeffding bound now shows that for a random choice of set D, the sum i I D a i is at least half its expected value with probability at least 0.9 and the expected value is δ i I a i which is at least Γδ. Pr D a i δ i I D i I a i δ i I a i exp ( ( δ i I a i ) i I a4 i ) exp ( J ) Γδ 0.1, where we noted that i I a4 i 1 J i I a i and J = 10 Γδ. The rest of the proof is devoted to showing that Lemma 4. and Lemma 4.3 cannot both hold, i.e., a function cannot be noise stable for random perturbation, yet noise sensitive for coordinatewise perturbation. Towards this end, we will construct from e a new function e (that happens to be {0, 1}-valued) for which the expected squared change as a result of coordinatewise perturbation is much larger than the expected squared change as a result of random perturbation: Lemma 4.4. (Under the assumptions we made towards a contradiction, and in particular, assuming Lemma 4. and Lemma 4.3) There is a function e such that: 1. (x, x) R e (x) e ( x) ( 3 ) O s Γ,. (x, x) C e (x) e ( x) Ω(1), where R is the distribution over pairs in the random perturbation test, and C is the distribution over pairs in the coordinatewise perturbation test. 19

20 The proof of Lemma 4.4 appears in Section 4.1. For sufficiently small s, Lemma 4.4 leads to a contradiction by the following claim: Claim 4.5. For any function h L (R n, N n ), h(x) h( x) (x, x) R h(x) h( x), (x, x) C where R is the distribution over pairs in the random perturbation test, and C is the distribution over pairs in the coordinatewise perturbation test. Proof. The expectation (x, x) C h(x) h( x) is given by the following expression: D x D h(x) h( x), x D, x D where the set of coordinates D n is chosen by including each i n in D independently with probability δ. Using Var x F (x) = 1 (F (x) F (x x,x )) and the notion of influence as discussed in the preliminaries, the above expression can be re-written as: Var h(x) = I D (h) = ĥ(s) = ĥ(s) Pr S D φ. D x D x D D D D S S D φ For every multi-index S N n, we have: Pr D S D φ = 1 (1 δ) #S 1 (1 δ) S. Here S = n i=1 S i and #S denotes the number of S i that are non-zero, and hence we have #S S. Therefore, the expectation is at most S ĥ(s) (1 (1 δ) S ). On the other hand, the expectation (x, x) R h(x) h( x) is given by the following expression, for ρ = 1 δ: h(x) h(x)h(ρx + 1 ρ w). x x,w We have x,w h(x)h(ρx + 1 ρ w) = h, T ρ h = S ĥ(s) ρ S and x h(x) = S ĥ(s), so the expectation is ĥ(s) (1 (1 δ) S ). S This concludes the proof of Theorem 8 assuming Lemma

21 4.1 Proof of Lemma 4.4 In this section we prove Lemma 4.4. Assume that a function e L (R n, N n ) with e s satisfies: With probability at least 1 O( 3 s) over (x, x) R, it holds that e(x) e( x) d R = O( 3 s) δ. (6) With probability at least Ω(1) over (x, x) C, it holds that e(x) e( x) d C = Ω( Γδ). (7) We show how to obtain a function e L (R n, N n ) (in fact {0, 1}-valued) that satisfies: (x, x) R e (x) e ( x) ( 3 ) O s Γ. (x, x) C e (x) e ( x) Ω(1). To this end, we construct two graphs on R n, G R = (R n, R ) and G C = (R n, C ), representing the function e under random perturbation and under coordinatewise perturbation, respectively. The graphs are infinite, and we will be abusing notation in the following, but all the arguments can be made precise by replacing sums by integrals wherever appropriate. Perturbation Graphs. The graphs G R and G C have labels on their vertices and weights on their edges. The label of a vertex x R n is e(x). The graph G R has edges between pairs (x, x) such that: (i) The labels on the endpoints are bounded, e(x), e( x) 1. (ii) e(x) e( x) d R. The weight of the edge w R (x, x) is the probability that (x, x) is chosen in the random perturbation test. The total edge weight is w R ( R ) 1 O( 3 s) from Hypothesis (6) and the observation that e s and thus for x N n, e(x) 1 except with probability s. The graph G C has edges between pairs (x, x) such that: (i) The labels on the endpoints are bounded, e(x), e( x) 1. (ii) e(x) e( x) d C. The weight of the edge w C (x, x) is the probability that (x, x) is chosen in the coordinate-wise perturbation test. The total edge weight is w C ( C ) Ω(1) from Hypothesis (7) and since e s. Cuts in Perturbation Graphs. We will construct a cut C : R n {0, 1}, and this will be our function e C. Denote by w R (C) and w C (C), the weight of the edges in the graphs G R and G C respectively that are cut by C. The cut C will satisfy: ( 3 ) 1. (Small R weight is cut:) w R (C) O s Γ.. (Large C weight is cut) w C (C) Ω(1). Let us first check that this proves Lemma 4.4: When choosing (x, x) as in the random perturbation test, the probability that the pair (x, x) is separated is at most w R (C) + (1 w R ( R )) O ( 3 s Γ ). When choosing (x, x) as in the coordinatewise perturbation test, the probability the pair (x, x) is separated is at least w C (C) Ω(1). 1