Hardness of Learning Halfspaces with Noise

Transcription

1 Hardness of Learnng Halfspaces wth Nose Venkatesan Guruswam Prasad Raghavendra Department of Computer Scence and Engneerng Unversty of Washngton Seattle, WA Abstract Learnng an unknown halfspace (also called a perceptron) from labeled examples s one of the classc problems n machne learnng. In the nose-free case, when a halfspace consstent wth all the tranng examples exsts, the problem can be solved n polynomal tme usng lnear programmng. However, under the promse that a halfspace consstent wth a fracton (1 ε) of the examples exsts (for some small constant ε > 0), t was not known how to effcently fnd a halfspace that s correct on even 51% of the examples. Nor was a hardness result that ruled out gettng agreement on more than 99.9% of the examples known. In ths work, we close ths gap n our understandng, and prove that even a tny amount of worst-case nose makes the problem of learnng halfspaces ntractable n a strong sense. Specfcally, for arbtrary ε, δ > 0, we prove that gven a set of examples-label pars from the hypercube a fracton (1 ε) of whch can be explaned by a halfspace, t s NP-hard to fnd a halfspace that correctly labels a fracton (1/2 + δ) of the examples. The hardness result s tght snce t s trval to get agreement on 1/2 the examples. In learnng theory parlance, we prove that weak proper agnostc learnng of halfspaces s hard. Ths settles a queston that was rased by Blum et al. n ther work on learnng halfspaces n the presence of random classfcaton nose [10], and n some more recent works as well. Along the way, we also obtan a strong hardness result for another basc computatonal problem: solvng a lnear system over the ratonals. 1 Introducton Ths work deals wth the complexty of two fundamental optmzaton problems: solvng a system of lnear equatons over the ratonals, and learnng a halfspace from labeled examples. Both these problems are easy when a perfect soluton exsts. If the lnear system s satsfable, then a satsfyng assgnment can be found n polynomal tme by Gaussan Elmnaton. If a halfspace consstent wth all the examples exsts, then one can be found usng lnear programmng. A natural queston that arses s the followng: If no perfect soluton exsts, but say a soluton satsfyng 99% Prelmnary verson appeared n the Proceedngs of the 47th Annual IEEE Symposum on Foundatons of Computer Scence, Research supported by NSF CCF , a Sloan Research Fellowshp, and a Packard Foundaton Fellowshp. Research supported n part by NSF CCF

2 of the constrants exsts, can we fnd a soluton that s nearly as good (say, satsfes 90% of the constrants)? Ths queston has been consdered for both these problems (and many others), but our focus here s the case when the nstance s near-satsfable (or only slghtly nosy). That s, for arbtrarly small ε > 0, a soluton satsfyng at least a fracton (1 ε) of the constrants s promsed to exst, and our goal s to fnd an assgnment satsfyng as many constrants as possble. Sometmes, the problem s relatvely easy to solve on near-satsfable nstances notable examples beng the Max 2SAT and Max HornSAT problems. For both of these problems, gven a (1 ε)-satsfable nstance, t s possble to fnd n polynomal tme, an assgnment satsfyng a fracton 1 f(ε) of the clauses where f(ε) 0 as ε 0 [28, 12]. Our results show that n the case of solvng lnear systems or learnng halfspaces, we are not so lucky and fndng any non-trval assgnment for (1 ε)-satsfable nstances s NP-hard. We descrbe the context and related work as well as our results for the two problems n ther respectve subsectons below. Before dong that, we would lke to stress that for problems admttng a polynomal tme algorthm for satsfablty testng, hardness results of the knd we get, wth gap at the rght locaton (namely completeness 1 ε for any desred ε > 0), tend to be hard to get. The most celebrated example n ths ven s Håstad s nfluental result [20] whch shows that gven a (1 ε)-satsfable nstance of lnear equatons modulo a prme p, t s NP-hard to satsfy a fracton ( 1 p + δ) fracton of them (note that one can satsfy a fracton 1 p of the equatons by smply pckng a random assgnment). Recently, Feldman [16] establshed a result n ths ven n the doman of learnng theory. He proved the followng strong hardness result for weak-learnng monomals: gven a set of example-label pars a (1 ε) fracton of whch can be explaned by a monomal, t s hard to fnd a monomal that correctly labels a fracton (1/2+δ) of the examples. Whether such a strong negatve result holds for learnng halfspaces also, or whether the problem admts a non-trval weak learnng algorthm s mentoned as a notable open queston n [16], and ths was also posed by Blum, Freze, Kannan, and Vempala [10] almost 10 years ago. In ths work, we establsh a tght hardness result for ths problem. We prove that gven a set of example-label pars a fracton (1 ε) of whch can be explaned by a halfspace, fndng a halfspace wth agreement better than 1/2 s NP-hard. Ths result was also establshed ndependently (for real-valued examples) n [17]. 1.1 Solvng lnear systems We prove the followng hardness result for solvng nosy lnear systems over ratonals: For every ε, δ > 0, gven a system of lnear equatons over Q whch s (1 ε)-satsfable, t s NP-hard to fnd an assgnment that satsfes more than a fracton δ of the equatons. As mentoned above, a result smlar to ths was shown by Håstad [20] for equatons over a large fnte feld. But ths does not seem to drectly mply any result over ratonals. Our proof s based on a drect reducton from the Label Cover problem. Whle by tself qute straghtforward, ths reducton s a steppng stone to our more complcated reducton for the problem of learnng halfspaces. The problem of approxmatng the number of satsfed equatons n an unsatsfable system of lnear equatons over Q has been studed n the lterature under the label MAX-SATISFY and strong hardness of approxmaton results have been shown n [7, 15]. In [15], t s shown that unless NP BPP, for every ε > 0, MAX-SATISFY cannot be approxmated wthn a rato of n 1 ε where n s the number of equatons n the system. (On the algorthmc sde, the best approxmaton 2

3 algorthm for the problem, due to Halldorsson [19], acheves rato O(n/ log n)). The startng pont of the reductons n these hardness results s a system that s ρ-satsfable for some ρ bounded away from 1 (n the completeness case), and ths only worsens when the gap s amplfed. For the related problem of lnear equatons over ntegers Z, a strong hardness result follows easly from Håstad s work [20]. Gven a system of lnear equatons of the form x + y z = r mod p wth the promse that there exsts a soluton satsfyng (1 ε) fracton of the equatons, t s NP-hard to fnd a soluton that satsfes 1 p + ε fracton of the equatons. By mappng x + y z = r mod p to x + y z pw = r mod Z, t gves a correspondng NP-hardness result for lnear equatons over ntegers. By choosng a large enough prme p, ths reducton mples the followng result: Gven a set of lnear equatons over ntegers wth the promse that there s a soluton satsfyng (1 ε) fracton of equatons, t s NP-hard to fnd one that satsfes more than a δ fracton, for all postve ε and δ. For the complementary objectve of mnmzng the number of unsatsfed equatons, a problem called MIN-UNSATISFY, hardness of approxmaton wthn rato 2 log0.99 n s shown n [7] (see also [5]). In partcular, for arbtrarly large constants c, the reducton of Arora et al. [7] shows NP-hardness of dstngushng between (1 γ)-satsfable nstances and nstances that are at most (1 cγ)- satsfable, for some γ. One can get a hardness result for MAX-SATISFY lke ours by applyng a standard gap amplfcaton method to such a result (usng a O(1/γ)-fold product constructon), provded γ = Ω(1). However, as presented n [7], the reducton works wth γ = o(1). It s not dffcult to modfy ther reducton to have γ = Ω(1). Our reducton s somewhat dfferent, and serves as a warm-up for the reducton for learnng halfspaces, whch we beleve puts together an nterestng combnaton of technques. 1.2 Halfspace learnng Learnng halfspaces (also called perceptrons or lnear threshold functons) s one of the oldest problems n machne learnng. Formally, a halfspace on varables x 1,..., x n s a Boolean functon I[w 1 x 1 + w 2 x w n x n θ] for reals w 1,..., w n, θ (here I[E] s the ndcator functon for an event E). For defnteness, let us assume that varables x are Boolean, that s, we are learnng functons over the hypercube {0, 1} n. In the absence of nose, one can formulate the problem of learnng a halfspace as a lnear program and thus solve t n polynomal tme. In practce, smple ncremental algorthms such as the famous Perceptron Algorthm [1, 25] or the Wnnow algorthm [24] are often used. Halfspace-based learnng algorthms are popular n theory and practce, and are often appled to labeled exampled sets whch are not separable by a halfspace. Therefore, an mportant queston that arses and has been studed n several prevous works s the followng: what can one say about the problem of learnng halfspaces n the presence of nosy data that does not obey constrants nduced by an unknown halfspace? In an mportant work on ths subject, Blum, Freze, Kannan, and Vempala [10] gave a PAC learnng algorthm for halfspaces n the presence of random classfcaton nose. Here the assumpton s that the examples are generated accordng to a halfspace, except wth a certan probablty η < 1/2, the label of each example s ndependently flpped. The learnng algorthm n [10] outputs as hypothess a decson lst of halfspaces. Later, Cohen [13] gave a dfferent algorthm for random classfcaton nose where the output hypothess s also a halfspace. (Such a learnng algorthm 3

4 whose output hypothess belongs to the concept class beng learned s called a proper learner.) These results appled to PAC learnng wth respect to arbtrary dstrbutons, but assume a rather bengn nose model that can be modeled probablstcally. For learnng n more general nose models, an elegant framework called agnostc learnng was ntroduced by Kearns et al. [23]. Under agnostc learnng, the learner s gven access to labeled examples (x, y) from a fxed dstrbuton D over example-label pars X Y. However, there s no assumpton that the labels are generated accordng to a functon from a specfc concept class, namely halfspaces n our case. The goal of the learner s to output a hypothess h whose accuracy wth respect to the dstrbuton s close to that of the best halfspace n other words the hypothess does nearly as well n labelng the examples as the best halfspace would. In a recent paper [21], Kala, Klvans, Mansour and Servedo gave an effcent agnostc learnng algorthm for halfspaces when the margnal D X on the examples s the unform dstrbuton on the hypercube or sphere S n 1, or any log-concave dstrbuton on R n. For any desred ε > 0, ther algorthm produces a hypothess h wth error rate Pr (x,y) D [h(x) y] at most opt + ε f the best halfspace has error rate opt. Ther output hypothess tself s not a halfspace but rather a hgher degree threshold functon. When the accuracy of the output hypothess s measured by the fracton of agreements (nstead of dsagreements or mstakes), the problem s called co-agnostc learnng. The combnatoral core of co-agnostc learnng s the Maxmum Agreement problem: Gven a collecton of example-label pars, fnd the hypothess from the concept class (a halfspace n our case) that correctly labels the maxmum number of pars. Indeed, t s well-known that an effcent α-approxmaton algorthm to ths problem exsts ff there s an effcent co-agnostc proper PAC-learnng algorthm that produces a halfspace that has agreement wthn a factor α of the best halfspace. The Maxmum Agreement for Halfspaces problem, denoted HS-MA, was shown to be NP-hard to approxmate wthn some constant factor for the {0, 1, 1} doman n [5, 9] (the factor was 261/262 + ε n [5] and 415/418 + ε n [9]). The best known hardness result pror to work was due to Bshouty and Burroughs [11], who showed an napproxmablty factor of 84/85 + ε, and ther result appled even for the {0, 1} doman. For nstances where a halfspace consstent wth (1 ε) of the examples exsts (the settng we are nterested n), an napproxmablty result for HS-MA was not known for any fxed factor α < 1. For the complementary objectve of mnmzng dsagreements, hardness of approxmatng wthn a rato 2 O(log1 ε n) s known [7, 5]. The problem of whether an α-approxmaton algorthm exsts for HS-MA for some α > 1/2,.e., whether a weak proper agnostc learnng algorthm for halfspaces exsts, remaned open. Ths queston was also hghlghted n Feldman s recent work [16] whch proved that weak agnostc learnng of monomals was hard. In ths paper, we prove that no (1/2 + δ)-approxmaton algorthm exsts for HS-MA for any δ > 0 unless P = NP. Specfcally, for every ε, δ > 0, t s NP-hard to dstngush between nstances of HS-MA where a halfspace agreeng on a (1 ε) fracton of the example-label pars exsts and where no halfspace agrees on more than a (1/2+δ) fracton of the example-label pars. Our hardness result holds for examples drawn from the hypercube. Our result ndcates that for proper learnng of halfspaces n the presence of even small amounts of nose, one needs to make assumptons about the nature of nose (such as random classfcaton nose studed n [10]) or about the dstrbuton of the example-label pars (such as unform margnal dstrbuton on examples as n [21]). 4

5 A smlar hardness result was proved ndependently by Feldman et al [17] for the case when the examples are drawn from R n. In contrast, our proof works when the data ponts are restrcted to the hypercube {0, 1} n, whch s the natural settng for a Boolean functon. Much of the complexty of our reducton stems from ensurng that the examples belong to the hypercube. 2 Prelmnares The frst of the two problems studed n ths paper s the followng: Defnton 2.1. For constants c, s, satsfyng 0 s c 1, LINEQ-MA(c, s) refers to the followng promse problem: Gven a set of lnear equatons over varables X = {x 1,..., x n }, wth coeffcents over Q, dstngush between the followng two cases: There s an assgnment of values to the varables X that satsfes at least a fracton c of the equatons. Every assgnment satsfes less than a fracton s of the equatons. In the problem of learnng a halfspace to represent a Boolean functon, the nput conssts of a set of postve and negatve examples all from the Boolean hypercube. These examples are embedded n the real n-dmensonal space R n by the natural embeddng. The objectve s to fnd a hyperplane n R n that separates the postve and the negatve examples. Defnton 2.2. Gven two dsjont multsets of vectors S +, S { 1, 1} n, a vector a R n, and a threshold θ, the agreement of the halfspace a v θ wth (S +, S ) s defned to be the quantty {v v S +, a v θ} + {v v S, a v < θ}. where the cardnaltes are computed, by countng elements wth repetton. In the HS-MA problem, the goal s to fnd a, θ such that the halfspace a v θ maxmzes ths agreement. Notce that there s no loss of generalty n assumng the embeddng to be { 1, 1} n. Our hardness results translate to other embeddngs as well, because the learnng problem n the { 1, 1} n embeddng can be shown to be equvalent to the learnng problem on most natural embeddngs such as {0, 1} n. Further, our hardness result holds even f both the nequaltes {, <} are replaced by strct nequaltes {>, <}. To study the hardness of approxmatng HS-MA, we defne the followng promse problem: Defnton 2.3. For constants c, s satsfyng 0 s c 1, defne HS-MA(c, s) to be the followng promse problem: Gven multsets of postve and negatve examples S +, S { 1, 1} n dstngush between the followng two cases: There s a halfspace a v θ that has agreement at least c S + S wth (S +, S ). Every halfspace has agreement less than s S + S wth (S +, S ). The hardness results n ths paper are obtaned by reductons from the Label Cover promse problem defned below. 5

6 Defnton 2.4. An nstance of LABELCOVER(c, s) represented as Γ = (U, V, E, Σ, Π) conssts of a bpartte graph over node sets U,V wth the edges E between them, such that all nodes n U are of the same degree. Also part of the nstance s a set of labels Σ, and a set of mappngs π e : Σ Σ for each edge e E. An assgnment A of labels to vertces s sad to satsfy an edge e = (u, v), f π e (A(u)) = A(v). The problem s to dstngush between the followng two cases: There exsts an assgnment A that satsfes at least a fracton c of the edge constrants Π. Every assgnment satsfes less than a fracton s of the constrants n Π. The reductons n ths paper use the followng napproxmablty result for Label Cover. Theorem 2.5. [27, 8] There exsts an absolute constant γ > 0 such that for all large enough nteger 1 constants R, the gap problem LABELCOVER(1, R ) s NP-hard, even when the nput s restrcted γ to label cover nstances wth the sze of the alphabet Σ = R. From the PCP theorem [8], t s easy to show that there exsts an absolute constant ε such that LABELCOVER(1, 1 ε) s NP-hard on nstances where the sze of alphabet Σ s restrcted to a small absolute constant (say 7). Wth ths as the startng pont, one apples the Parallel Repetton theorem [27] to obtan hardness of label cover nstances over larger alphabet. On applyng k-wse parallel repetton, the 1 vs 1 ε gap s amplfed to 1 vs c k for some absolute constant c, whle the alphabet sze also grows exponentally n k. Ths yelds the above napproxmablty result wth the requred polynomal dependence between the alphabet sze R and the soundness 1 R γ. Throughout ths paper, we use the letter E to denote a lnear equaton/functon, wth coeffcents {0, 1, 1}. For a lnear functon E, we use V (E) to denote the set of varables wth non-zero coeffcents n E. We shall refer to V (E) as the arty of E. Further, the evaluaton E(A) for an assgnment A of real values to the varables s the real value obtaned on substtutng the assgnment n the equaton E. Hence an assgnment A satsfes the equaton E = 0 f E(A) = 0. For the purposes of the proof, we make the followng defntons. Defnton 2.6. An equaton tuple T conssts of a set of lnear equatons E 1,..., E k and a lnear functon E called the scalng factor. Defnton 2.7. A tuple T = ({E 1, E 2,..., E k }, E) s sad to be dsjont f the sets of varables V (E ), 1 k, and V (E) are all parwse dsjont. Defnton 2.8. An assgnment A s sad to satsfy an equaton tuple T = ({E 1,..., E k }, E), f the scalng factor s postve,.e., E(A) > 0, and for every, 1 k, E (A) = 0. For β 0, an assgnment A s sad to β-satsfy an equaton E f E (A) β E(A). An assgnment s sad to β-satsfy the tuple T f t β-satsfes all the equatons {E 1,..., E k } and the scalng factor E(A) > 0. Note that 0-satsfyng an equaton tuple s the same as satsfyng t. Defnton 2.9. An assgnment A s sad to be C-close to β-satsfyng an equaton tuple T = ({E 1,..., E k }, E), f E(A) > 0 and E (A) > β E(A) for at most C values of, 1 k. An assgnment s sad to be C-far from β-satsfyng an equaton tuple T f t s not C-close to β-satsfyng T. 6

7 3 Overvew of the Proof Both the hardness results are obtaned through a reducton from the Label Cover problem. Let us fx a Label Cover nstance Γ over the set of labels {1,..., R}. Observe that the HS-MA problem amounts to fndng an assgnment satsfyng the maxmum number of a set of homogeneous lnear nequaltes (see Defnton 2.2). Specfcally, each example v S + S yelds a homogeneous lnear nequalty a v θ or a v < θ n the varables a R n and θ. Although θ s permtted to take any real value, let us fx θ = 0 for the sake of exposton. The HS-MA problem s closely ted to solvng systems of lnear equatons over reals. Gven a homogeneous lnear equaton a v = 0 over varables a = (a 1,..., a n ), t can be encoded as two lnear nequaltes, a v + δ 0 and a v δ < 0. For the moment, let us suppose that δ s a varable forced to be a very small postve real number. An assgnment to a satsfes both nequaltes f and only f a v [ δ, δ]. In other words, an assgnment satsfes ether two or one nequalty dependng on whether the equaton (a v = 0) s approxmately satsfed or not. Roughly speakng, usng the above reducton, an NP-hardness result for LINEQ-MA(c, s) should translate n to an NP-hardness result for HS-MA( 1+c 2, 1+s 2 ). In ths lght, a natural approach would be encode the label cover nstance as systems of lnear equatons, and use ths encodng to obtan NP-hardness results for both LINEQ-MA and HS-MA problems. However, mplementng the outlned reducton from Label Cover to HS-MA poses consderable dffcultes. For the above reducton from homogeneous lnear equatons to HS-MA, we need an NP-hardness of dstngushng between perfectly satsfable equatons, and equatons that do not admt even an approxmate soluton. Further, n the above reducton, we used a varable δ takng only small postve real values. In a general HS-MA problem, no such constrant can be forced on the varables. More mportantly, notce that all the examples v S + S n HS-MA are requred to be vectors n { 1, 1} n. Thus every coeffcent n the system of homogeneous lnear equatons must take values ether 1 or 1. We shall refer to homogeneous lnear systems as equaton tuples. Specfcally, an equaton tuple T conssts of a system of homogeneous lnear equatons {E 1, E 2,..., E k } and a scalng factor E. Note that a soluton to homogeneous lnear system can be scaled to obtan a new soluton. Hence the qualty of an approxmate soluton s to be measured n terms of relatve error, rather than absolute value of the error. The scalng factor E assocated wth the tuple T serves ths purpose (see Defnton 2.8). The proof of hardness of HS-MA proceeds n three stages as descrbed below. In the frst stage, we reduce the Label Cover nstance Γ to systems of homogeneous lnear equatons. More precsely, startng wth the nstance Γ, Verfer I generates a set of equaton tuples T such that: If Γ s satsfable then there s an assgnment A that satsfes all the equaton tuples n T. If Γ s an unsatsfable nstance of Label Cover, then every assgnment A fals to satsfy most of the equaton tuples even approxmately. Specfcally, every assgnment A can β-satsfy only a tny fracton of the tuples n T. 7

8 The set of equaton tuples T s generated as follows. For each vertex u of the Label Cover nstance Γ, we ntroduce R varables {u 1,..., u R } whch ndcate the label assgned to vertex u, that s, u s 1 f and only f u s assgned label. The edge constrants n Γ can be translated to lnear equatons over the varables {u }. As each vertex u s assgned one of R labels, only one of the varables u 1,..., u R s non-zero n the ntended soluton. In other words, the ntended soluton s sparse. Towards enforcng sparsty, Verfer I ntroduces constrants u = 0 for randomly chosen varables u. The set T of equaton tuples conssts of all the tuples output by Verfer I over all ts random choces. Fx an equaton tuple T = ({E 1, E 2,..., E k }, E). Consder the followng set of 2 k+1 nequaltes obtaned by ±1 combnatons of the equatons E. k w E + E 0 =1 and k w E E < 0 for all w { 1, 1} k. =1 Gven an assgnment A, for any w { 1, 1} k, both the nequaltes correspondng to w are satsfed only f w E (A) [ E(A), E(A)]. In the completeness analyss, f all equatons are satsfed,.e., evaluate to 0 on A, then any ±1 combnaton also vanshes, and n turn belongs to [ E(A), E(A)]. Turnng to soundness analyss, f the assgnment A does not satsfy many equatons E even approxmately, then by defnton, many of the values E (A) are large n comparson to the scalng factor E(A). Intutvely, a random ±1 combnaton of large numbers(e (A)) s small( [ E(A), E(A)]) wth very low probablty. Thus f the assgnment A does not satsfy almost all equatons E approxmately, then for almost all w { 1, 1} k, t satsfes only one of the two nequaltes. Conversely, f the assgnment A satsfes more than 1 2 of the nequaltes, t must satsfy almost all equatons n T approxmately. Roughly speakng, the argument n the precedng paragraph already yelds a reducton from equaton tuples to HS-MA. However, there are two key ssues, that are addressed n the next two stages of the reducton : Verfer II and Verfer III. Frstly, the equaton tuples T need not be dsjont,.e., dfferent equatons E can share varables. Hence the coeffcents of varables n the ±1 combnaton w E could take values outsde { 1, 1}. Snce all the nequaltes n HS-MA are requred to contan only { 1, 1} coeffcents, we address ths ssue n the second stage (Verfer II). More precsely, Verfer II takes as nput the set T and creates a set of equaton tuples T. The tuples n T are dsjont, they are all over the same set of varables, and each varable appears n exactly one equaton of every tuple. Further, n the soundness case, almost all tuples are at least C-far from beng β-satsfed. Verfer II thus plays two roles: () t makes the equatons n each tuple have dsjont support, and () n the soundness case, every assgnment not just fals to β-satsfy most of the tuples, but s n fact C-far from β-satsfyng most of the tuples. Both these facts are exploted by Verfer III n the thrd stage. The number of nequaltes produced by pckng all ±1 combnatons of equatons E 1,..., E k s exponental n k. As k = Ω(n), ths would make the runnng tme of the reducton exponental. In the thrd stage (Verfer III), we wll present a carefully chosen polynomal szed sample space of ±1 combnatons whch s suffcent to perform the soundness analyss. More formally, gven a tuple T and an assgnment A, Verfer III dstngushes (by checkng sutable nequaltes) between the cases when an assgnment A satsfes a tuple T and when t s C-far from β-satsfyng T. The equaton tuples T T generated by Verfer II are gven as nput to Verfer III. Snce the 8

9 tuples n T are dsjont, the resultng nequaltes have all the varables wth coeffcents { 1, 1}. Further the nequaltes generated by Verfer III are desgned to have a common varable (a threshold θ) on the rght hand sde. Hence the nequaltes generated by the combned verfer (the three stages) correspond naturally to tranng examples n the learnng problem. To show NP-hardness for LINEQ-MA, the set of tuples T output by Verfer I are rather easly converted n to a set of equatons. Ths s acheved by creatng several equatons for each equaton tuple T T, such that a large fracton of these are satsfed f and only f T s satsfed. The detals of ths converson are descrbed n Secton 5. 4 Verfer I Let Γ = (U, V, E, Σ, Π) be an nstance of Label Cover wth Σ = R. Ths verfer produces a set of equaton tuples, whch form nput to the next stage n the reducton (Verfer II). For each vertex u U V, the equaton tuples have varables {u 1,..., u R } takng values n Q. The soluton that we are targetng s an encodng of the assgnment to the Label Cover nstance. So f a vertex u s assgned the label by an assgnment A, then we want u = 1 and u j = 0 for j, 1 j R. We construct an equaton tuple for every t-tuple of varables correspondng to vertces n U, for a sutable parameter t that wll be chosen shortly. Let W denote the set of varables W = {u u U, 1 R}. For each t-tuple X of varables from W, construct the equaton tuple T as follows: P 1 : For every par of vertces u, v U V, an equaton, R u =1 R v j = 0. P 2 : For each edge e = (u, v) E, the Label Cover constrants for the edge, ( ) u j v = 0, for all 1 R. j πe 1 () j=1 P 3 : For each varable w X, w = 0. The scalng factor s P 4 : R =1 u for an arbtrary fxed vertex u U V. Output the tuple T = (P 1 P 2 P 3, P 4 ). Theorem 4.1. For every 0 < δ 1, ε 1, γ < 1, there exsts suffcently large R, t such that f Γ = (U, V, E, Σ, Π) s an nstance of Label Cover wth Σ = R, then wth the choce of β 1 = 1 R 3 the followng holds: If Γ s satsfable, then there s an assgnment A that satsfes at least 1 ε 1 fracton of the output tuples. 9

10 If no assgnment to Γ satsfes a fracton 1 R γ less than a fracton δ 1 of the output tuples. of the edges, then every assgnment A β 1 -satsfes Proof. Let us choose parameters c 0 = ln(1/δ 1 ) and t = 4c 0 R 1 γ, for R = ( 4c 0 ε 1 ) 1/γ. We present the completeness and soundness arguments n turn. Completeness: Gven an assgnment A to the Label Cover nstance that satsfes all the edges, let A denote the correspondng nteger soluton. Clearly, the nteger soluton A satsfes: All equatons n P 1 and P 2. ( 1 1 R) fracton of the equatons n the set: {w = 0 w W }. ( Snce t ) equatons of the form w = 0 are present n each tuple, the assgnment A satsfes at least 1 1 t ( ) R of the tuples. By the choce of parameters R and t, we have 1 1 t R > 1 t R = 1 ε 1. Soundness: Suppose there s an assgnment A that β 1 -satsfes at least a fracton δ 1 of the tuples generated. Clearly A must β 1 -satsfy all the equatons P 1 and P 2, snce they are common to all the tuples. Further by defnton of β 1 -satsfacton, the scalng factor P 4 (A) > 0. Normalze the assgnment A such that the scalng factor P 4 s equal to 1. As all the equatons n P 1 are β 1 -satsfed, we get R 1 β 1 v 1 + β 1, for all v U V. (1) =1 Further, we clam that the assgnment A β 1 -satsfes at least a fracton (1 c 0 t ) of the equatons of the form w = 0 for w W. Otherwse, wth t of these equatons belongng to every tuple, less than (1 c 0 t ) t < δ 1 tuples wll be β 1 -satsfed by A. Recall that all vertces n U have same degree. Hence by an averagng argument, for at least half the edges e = (u, v), at least a (1 2c 0 t ) fracton of the constrants u = 0 are β 1 -satsfed. Let us call these edges good. For every vertex, defne the set of labels Pos as follows, { { Σ u 8β 1 } f u U, Pos(u) = { Σ u 8β 1 (R + 1)} f u V. The set Pos(w) s non-empty for each vertex w U V, because otherwse R =1 w < 8β 1 (R + 1) R 1 β 1, a contradcton to (1). Further f e = (u, v) s a good edge then for at least a fracton 1 2c 0 t of the labels 1 R, we have u β 1. Hence Pos(u) ( 2c 0 t )R = Rγ 2. Further, snce all the constrants P 2 are β 1 -satsfed, we know that u v j β 1. πe 1 (j) Thus for every label, j Pos(v), there s at least one label Pos(u) such that π e () = j. For every vertex w U V, assgn a label chosen unformly at random from Pos(w). For any good 1 edge e = (u, v), the probablty that the constrant π e s satsfed s at least Pos(u) 2 R. Snce at γ least half of the edges are good, ths shows that there s an assgnment to the Label Cover nstance that satsfes at least a fracton 1/R γ of the edges. 10

11 5 Lnear equatons over Ratonals Theorem 5.1. For all ε, δ > 0, the problem LINEQ-MA(1 ε, δ) s NP-hard. Proof. The result s obtaned by a smple reducton that converts each equaton tuple generated by Verfer I n to a set of equatons. In fact, the noton of β-satsfablty s not necessary for the followng proof. The crucal ngredent n the proof s the followng well known fact about unvarate polynomals: Fact 5.2. A real unvarate polynomal of degree at most n that s not dentcally zero has at most n real roots. Gven a Label Cover nstance Γ, the reducton analyzed n Theorem 4.1 s appled wth ε 1 = ε, δ 1 = δ 2 to obtan a set of equaton tuples T. From T, a set of equatons over Q s obtaned as follows: For each tuple T = ({E 1,..., E n }, E) T, nclude the followng set of equatons : for all values of y = 1, 2,..., t, where t = (n+1) δ 1. E 1 + y E y n 1 E n + y n (E 1) = 0, Completeness: Observe that f Γ s satsfable then the correspondng assgnment A has a scalng factor E(A) = 1. Further for every equaton tuple T that s satsfed by A, we have E (A) = 0 for all 1 n. Hence the assgnment A satsfes at least (1 ε) fracton of the equatons. Soundness: Suppose there s an assgnment A that satsfes at least a fracton δ = 2δ 1 of the equatons. Hence for at least δ 1 fracton of the tuples, at least δ 1 fracton of the equatons are satsfed. Let us refer to these tuples as nce. For an equaton tuple T = ({E 1,..., E n }, E), consder the polynomal p of degree at most n gven by p(y) = E 1 (A) + y E 2 (A) y n 1 E n (A) + y n (E(A) 1). By Fact 5.2, the polynomal p has at most n real roots unless t s dentcally zero. Further the polynomal p s dentcally zero f and only f the tuple T s completely satsfed. Hence f the tuple T s not completely satsfed by A, then at most n t < δ 1 fracton of the equatons correspondng to T can be satsfed. Thus every nce tuple T s completely satsfed by A. So the assgnment A satsfes at least a fracton δ 1 of the tuples. The coeffcents of varables n the above reducton could be exponental n n (ther bnary representaton could use polynomally many bts). Usng an alternate reducton, we can prove a smlar hardness even f all the coeffcents are ntegers bounded by a constant dependng only on ε, δ - thus requrng only constant number of bts per coeffcent. Moreover the arty of all the equatons (.e., the number of varables wth a nonzero coeffcent) can be restrcted to a constant. The proof of the followng theorem s presented n Appendx A. Theorem 5.3. For any constants ε, δ > 0, there exst B, b > 0 such that LINEQ-MA(1 ε, δ) s NP-hard even on lnear systems where each equaton has arty at most b and all coeffcents are ntegers bounded n absolute value by B. 11

12 Usng a dfferent approach, the above result has been mproved n [18] to lnear systems where each equaton has arty at most 3, all the coeffcents are {+1, 1} and all the constants are bounded. 6 Verfer II The man deas n the constructon of the second verfer are descrbed below. To obtan halfspace examples on the hypercube { 1, 1} n, our reducton requres that the equaton tuples T are dsjont. But the equaton tuples T output by Verfer I are not dsjont,.e., there are varables that occur n more than one equaton n T. Ths problem can be solved by usng multple copes of each varable, and usng dfferent copes for dfferent equatons. However, t s mportant to ensure that the dfferent copes of the varables are consstent. To ensure ths the verfer does the followng: t has a very large number of copes of each varable n comparson to the number of equatons. The equatons of the tuple T are checked on a very small number of the copes. On all the copes that are not used for equatons n T, the verfer checks parwse equalty. Any gven copy of a varable s used to check an equaton n T for only a very small fracton of cases. For most random choces of Verfer II, the copy of the varable s used for consstency checkng. Ths way most of the copes are ensured to be consstent wth each other. The parwse consstency checks made between the copes must also satsfy the dsjontness property. So the verfer pcks a matchng at random, and performs parwse equalty checks on the matchng. It can be shown that even f there are a small number of bad copes, they wll get detected by the matchng wth hgh probablty. If a sngle equaton s unsatsfed n T, at least C equatons need to be unsatsfed on the output tuple. Ths s easly ensured by checkng each equaton n T on many dfferent copes of the varables. As all the copes are consstent wth each other, f one equaton s unsatsfed n T a large number of equatons n the output tuple wll be unsatsfed. Let us say the tuple T conssts of equatons {E 1,..., E m } and a scalng factor E over varables {u 1,..., u n }. Let us denote by n 0 the maxmum arty of an equaton n T. We use superscrpts to dentfy dfferent copes of the varables. Thus u (j) refers to the varable correspondng to j th copy of the varable u. Further for an equaton/lnear functon E, the notaton E (j) refers to the equaton E over the j th copes of varables V (E). By the notaton M (j, k), we refer to the followng parwse equalty check: M (j, k) : u (j) u (k) = 0. Let M, P be parameters whch are even ntegers. The set of varables used by Verfer II conssts of M copes for varables not n V (E), M + 1 copes of varables n V (E). Gven an equaton tuple T = ({E 1, E 2,..., E r }, E), Verfer II checks each of the r equatons E on P copes of the varables. On the remanng copes, the verfer performs a set of parwse equalty checks. We now defne famly of pseudorandom permutatons that we wll use n Verfer II. Defnton 6.1. Two dstrbutons D 1, D 2 over a fnte set Ω are sad to be η-close to each other f the varaton dstance D 1 D 2 = 1 2 ω Ω D 1(ω) D 2 (ω) s at most η. 12

13 Defnton 6.2. A multset of permutatons Π of {1... M} s sad to be k-wse η-dependent f for every k-tuple of dstnct elements (x 1,..., x k ) {1... M}, the dstrbuton (f(x 1 ), f(x 2 ),..., f(x k )) for f Π chosen unformly at random s η-close to the unform dstrbuton on k-tuples. Let Π denote a set of 4-wse η-dependent permutatons of {1,..., M}. Explct constructons ) O(1)) of such famles of permutatons of sze polynomal n M (specfcally are known, see [26, 22]. Verfer II takes as nput the set of equaton tuples T generated by Verfer I. The detals of Verfer II are descrbed below. For each equaton tuple T = ({E 1, E 2,..., E r }, E) T, For each s {1,..., M + 1}, and a permutaton π n a set Π of 4-wse η-dependent permutatons, Choose E (s) as the scalng factor. Re-number the remanng M copes of V (E) wth {1,..., M} arbtrarly. Specfcally, we shall ndex the set {E () s, 1 M + 1} wth the set {1,..., M}. Construct sets of equatons P and M as follows: Output the tuple (P M, E (s) ). P = {E (π(j)) l 1 l r, P (l 1) + 1 j P l}, M = {M (π(j), π(j + 1)) u (π(j)) / V (P), j odd}. Theorem 6.3. For all 1 > ε 2, δ 2 > 0 and postve ntegers C, n 0, there exsts constants M, P, η such that: Gven a set of equaton tuples T of whch each tuple s of arty at most n 0 and has the same scalng factor E, the followng holds: If an assgnment A, satsfes a fracton 1 ε 2 of the tuples T T then there exsts an assgnment A whch satsfes a fracton 1 ε 2 of the tuples output by the verfer. If no assgnment β 1 -satsfes a fracton δ 2 2 of the tuples T T, then no assgnment A s C-close to β = β 1 9n 0 -satsfyng a fracton δ 2 of the output tuples. Proof. Fx the parameters M, P, η, C 0 as follows, C 0 = max(8c, 8/δ 2 ) + 1, (2) ) P = 9n 0 (C + 1δ2, (3) 40P rc0 M = + 1, (4) η = δ 2 ( M η δ 2 64C 0. (5) The completeness proof s clear, snce an assgnment A consstng of several copes of A satsfes the exact same tuples that A satsfes. 13

14 Suppose an assgnment A s C-close to β-satsfyng a δ 2 -fracton of the output tuples. Then for at least a fracton δ 2 2 choces of nput tuple T T, at least a fracton δ 2 2 of the output tuples correspondng to T are C-close to beng β-satsfed by A. Let us call these nput tuples T good. For a good tuple T, there are at least δ 2 4 fracton of choces of s for whch wth probablty more than δ 24 over the choce of permutaton π, the output tuple s C-close to beng β-satsfed (by A ). These values of s (and the assocated copy of the scalng factor E (s) ) are sad to be nce wth respect to T. Lemma 6.4. Let E (s) be a nce scalng factor of T. Then, for every equaton E l T, there exst at least P C values of j for whch E (j) l (A ) β E (s) (A ). Proof. Snce E (s) s a nce scalng factor, for at least one permutaton π Π, the assgnment A s C-close to β-satsfyng the generated tuple. Snce each equaton E l s checked on P dfferent copes, at least P C of the copes must be β-satsfed by A. Lemma 6.5. For C 0 = max(8c, 8/δ 2 ) + 1, the followng holds: Let E (s) be a scalng factor that s nce wth respect to some good tuple T. For every varable u that occurs n the equatons of T (ncludng E), all but C 0 of copes of u are 2β E (s) (A ) close to each other,.e., A (u (j 1) ) A (u (j 2) ) 2β E (s) (A ) for all j 1, j 2 M for a set M wth {1,..., M} M C 0. Proof. As E (s) s a nce scalng factor wth respect to T, for at least a fracton δ 2 4 choces of π Π the assgnment A s C-close to β-satsfyng the output tuple P M. In partcular, ths means that wth probablty at least δ 2 4, at most C of the consstency checks n M fal to be β-satsfed. Defne a copy u (j) to be far from u (j 1) f A (u (j) ) A (u (j 1) ) > β E (s) (A ). We defne a copy u (j) to be bad, f t s far from at least M/2 other copes. Suppose there are more than C 0 bad copes of the varable u. For notatonal convenence, we can assume that the frst C 0 copes {u (1), u (2),..., u (C 0) } are bad. We wll prove below that wth hgh probablty over the choce of π Π, at least 2C + 1 of these bad copes wll be nvolved n checks n M that are not β-satsfed. Ths wll n turn mply that more than C of the checks n M are not β-satsfed. Observe that for every varable u, at most P r of ts copes are used n equatons n P. Further, for a unformly random permutaton, the probablty that (the ndex of) a fxed bad copy s the mage of a fxed j {1,..., M} s 1 M. Hence the probablty that a fxed bad copy s used for an equaton n P s at most P r M. Snce Π s η-dependent, the probablty that one of the C 0 bad copes u (j), 1 j C 0, s used for some equaton n P s at most C 0 ( P r M + η). So, except wth ths probablty, all bad copes are assgned to consstency checks n M. A bad copy u (j), 1 j C 0 fals to β-satsfy a check n M whenever a far copy s mapped next to t by the permutaton π. Formally, let l be such that j = π(l). The varable u (j) wll fal a check n M f ether l s even and u (π(l) 1) s far from u (j) or f l s odd and u (π(l)+1) s far from u (j). Let Z j, 1 j C 0 be the 0, 1 random varable ndcatng the event that a far copy s mapped next to by the permutaton. We shall estmate the values of E π Π [Z j ] and E π Π [Z j1 Z j2 ] for dfferent values of j, j 1, j 2. u (j) Let F j denote the set of j 0, such that u (j 0) s far from u (j). Let l be such that j = π(l). Then 14

15 for each j, 1 j C 0 we have E [Z j] = 1 π Π 2 Pr [Z j = 1 l s odd] + 1 ( 1 π Π 2 Pr [Z j = 1 l s even] π Π )( Fj ) 2 M 1 η 1 3. Further for 1 j 1 < j 2 C 0, E [Z j 1 Z j2 ] = Pr [Z j 2 = 1, Z j1 = 1]. π Π π Π To estmate Pr π Π [Z j2 = 1, Z j1 = 1], we frst estmate Pr[Z j2 = 1, Z j1 = 1] where the probablty s over a unformly random permutaton π. Let l be such that π(l) = j 1. We shall estmate Pr[Z j2 = 1, Z j1 = 1] condtoned on l beng odd, the other case follows along smlar lnes. There are two cases: Case 1 : π(l + 1) = j 2. Ths happens wth probablty 1 M 1, and further n ths case Z j 1 = 1 mples Z j2 = 1. Case 2: π(l + 1) j 2. s mapped next to one of ts far copes, there are M 3 choces for π(l + 1). Hence the probablty that a far copy s mapped next to u (j 2) s at Let π(l ) = j 2 for some l l + 1. Gven that u (j 1) most F j 2 M 3. Hence, Pr[Z j2 = 1 Z j1 = 1] 1 M 1 + F j M 3. ( ) M 2 M 1 Pr[Z j2 = 1, Z j1 = 1] F j 1 M 1 F j M 3, F j 1 F j2 (M 1) M 3. F j2 M 3, Snce Π s a 4-wse η-dependent famly we get, E [Z j 1 Z j2 ] = Pr [Z j 2 = 1, Z j1 = 1], π Π π Π Pr[Z j2 = 1, Z j1 = 1] + η, F j 1 F j2 (M 1) M 3 + η, ( ) ( ) Fj1 M 1 η Fj2 M 1 η E [Z j1 ] E [Z j2 ] + 3 π Π π Π M 3 + 3η. Defne the random varable X to be C 0 =1 Z. Then, C0 E [X] = E [Z ] C 0 π Π π Π 3. = M 3 + 3η,

16 The varance σ 2 s gven by σ 2 = E [X 2 ] ( E π Π C 0 ( ) = E π Π [Z2 j 1 ] ( E [Z j1 ]) 2 + π Π j 1 =1 C ( C0 π Π [X]) 2 ) ( ) 3 2 M 3 + 3η. C 0 C 0 j 1 =1 j 2 j 1 ( ) E [Z j 1 Z j2 ] E [Z j1 ] E [Z j2 ] π Π π Π π Π Therefore σ 2 < 2C 0, for M, 1 η suffcently large compared to C 0. Usng Chebyshev s nequalty, t follows that 2C 0 Pr[X 2C] ( C 0 3 2C). 2 Puttng these facts together, t follows that the probablty over the choce of π Π (once a nce value of s s pcked) that at most C of the consstency checks n M fal to be β-satsfed s at most ( ) P r C 0 M + η 2C 0 + ( C 0 2 2C) < δ 2 2 4, by the choce of parameters M, C 0 and η. Ths contradcts the nceness of the scalng factor E (s). It must thus be the case that at most C 0 copes of the varable u are bad. Now f nether of the copes u (j 1) and u (j 2) are bad, then both A (u (j 1) ) and A (u (j 2) ) are wthn β E (s) (A ) of the the value assgned by A to more than half the copes of u. Ths mples that they must themselves be wthn 2β E (s) (A ) of each other. Thus all but C 0 copes of u are 2β E (s) (A ) close to each other. Returnng to the proof of Theorem 6.3, fx T to be an arbtrary good tuple. Defne s 0 to be ts nce value for whch the correspondng scalng factor E (s0) (A ) has the smallest absolute value. (Note that E (s) (A) > 0 for every scalng factor E (s) that s nce wth respect to T, hence E (s0) (A ) > 0.) From Lemma 6.5, we know that all but C 0 of the copes of every varable are 2β E (s0) (A ) close to each other. Delete all the bad copes (at most C 0 ) of each varable. Further, delete all the varables n V (E (s0) ). Now defne an assgnment A as follows: The value of A(u ) s the average of all the copes of u that have survved the deleton. We clam that the assgnment A β 1 -satsfes all the good tuples T T. Observe that the arty of scalng factor E s at most n 0, and at most C copes of each varable are deleted. Snce there are at least δ 2 4 M nce scalng factors and δ 2 4 M > n 0 (C 0 + 1), there exsts a nce scalng factor E (s 1) of T such that no varable of V (E (s 1) ) s deleted. Further by defnton of s 0, E (s 1) (A ) E (s 0) (A ). From Lemma 6.5, for the average assgnment A and any undeleted varable u (j) occurrng n an equaton of T, we have A(u ) A (u (j) ) 2β E (s 0) (A ) 2β E (s 1) (A ). (6) 16

17 In partcular, A(u ) A (u (s 1) ) 2β E (s 1) (A ). Usng ths for each of the varables n V (E (s 1) ), we get E(A) (1 2β V (E (s 1) ) ) E (s 1) (A ) (1 2βn 0 ) E (s 1) (A ). Substtutng back n (6), we get A(u ) A (u (j) ) 2β E(A) 4β E(A). (7) (1 2βn 0 ) Consder any good tuple T T. The same argument used for T shows that there exsts a scalng factor E (j0) that s nce wth respect to T and none of whose varables have been deleted. Usng Equaton (7) for varables u (j 0) V (E (j0) ), we have A (u (j 0) ) A(u ) + 4β E(A). Thus, E (j 0) (A ) E(A) + 4β n 0 E(A). (8) Usng Lemma 6.4, and the fact P C > n 0 C 0, we can conclude for every equaton E l T, there exsts j 1 such that E (j 1) l (A ) β E (j0) (A ), and no varable of V (E (j 1) l ) s deleted. Smlar to (8) we get E l (A) E (j 1) l (A ) + 4β n 0 E(A). Therefore, E l (A) (β + 4β 2 n 0 + 4βn 0 ) E(A) 9βn 0 E(A) = β 1 E(A), mplyng that the assgnment A β 1 -satsfes the tuple T. Hence the assgnment A β 1 -satsfes all the good tuples. Recallng that at least a fracton δ 2 /2 of the tuples are good, the result of Theorem 6.3 follows. 7 Verfer III Gven a equaton tuple T = ({E 1,..., E n }; E), Verfer III checks whether the assgnment A satsfes T or s not even C-close to β-satsfyng T. Towards ths, we defne the followng notaton: For a tuple of equatons E = (E 1,..., E n ), and a vector v { 1, 1} n, defne E v = n =1 v E. Let V for an nteger, denote a 4-wse ndependent subset of { 1, 1}. Polynomal sze constructons of such sets are well known, see for example [4, Chap. 15]. The verfer has an addtonal parameter m chosen to be some nteger m > 16. The detals of the verfer are descrbed below. δ3 2 17

18 Partton the set of equatons {E 1,..., E n } usng n random varables that are C-wse ndependent and take values {1,..., m}. Let us say the parts are E, 1 m. For each part E, pck a random vector, v V n where n = E. Compute lnear functons B, 1 m, B = E v. Construct B = (B 1, B 2,..., B m ). Pck a vector w unformly at random from { 1, 1} m. Wth probablty 1 2, check one of the followng two nequaltes: Accept f the check s satsfed, else Reject. B w + E θ, (9) B w E < θ. (10) Throughout ths artcle, we shall choose the parameter m to be a prme number. For prme values of m, polynomal sze spaces of C-wse ndependent varables takng values {1,..., m} can be obtaned usng BCH codes wth alphabet sze m, and mnmum dstance C + 1 (See [2]). Theorem 7.1. For every 0 < β, δ 3 < 1 and all m > 16, C > 4m δ3 2 β 4 δ3 2 equaton tuple T = ({E 1,..., E n }, E) and an assgnment A, the followng holds: Gven the If the assgnment A satsfes T, then wth θ = 0, the verfer accepts wth probablty 1. If the assgnment A s C-far from β-satsfyng the tuple T, then rrespectve of the value of θ, the verfer accepts wth probablty less than δ 3 2. Proof. The proof shall make use of Lemmas 7.2 and 7.3, whch we shall present later n the secton. For an assgnment A that satsfes the tuple T, we have E j (A) = 0, 1 j n, and E(A) > 0. Hence for all the random choces of Verfer III, B s the 0 vector, and E > 0. Therefore, wth the choce θ = 0, all the checks made by the verfer succeed. (In fact, the condtons hold wth a strct nequalty.) Suppose the assgnment A s C-far from β-satsfyng the tuple T. If E(A) 0, then clearly at most one of these two nequaltes checked can be satsfed, and the proof s complete. Hence, we assume E(A) > 0. There are at least C values {E j (A) 1 j n} that have absolute value greater than β E(A). Let us refer to these E j as large. The probablty that any of the parts E contans less than B 0 = 2 large values s at most m ( ) C β 2 B 0 (1 1 m )C B 0. From Lemma 7.2, for a part E that has at least B 0 large values, Pr[ B (A) > E(A) ] Assumng that all the parts have at least B 0 large values, we bound the probablty that less than m 24 parts have B (A) > E(A). Let us fx a partton {E } such that all parts contan at least B 0 large values. For a fxed partton {E }, each of the events B (A) > E(A) are ndependent by the 18

19 choce of the vectors v. Thus for a fxed partton {E }, we use the Chernoff bounds and obtan [ Pr { : B (A) > E(A) } < m ] 24 {E } e m 96. Note that the above nequalty holds for every fxed partton wth all parts contanng at least B 0 large values. Hence, condtoned on the event that all parts have at least B 0 large values we have [ { Pr : B (A) > E(A) } m ] < e m Consder the case n whch there are at least m 0 = m 24 parts wth B (A) > E(A). In ths case, from Lemma 7.3 we can conclude Pr [ B w θ [ E(A), E(A)] ] ( m0 ) m 0 /2 2 m 0 1. Overall we have, Pr [ B w θ [ E(A), E(A)] ] ( ) ( C m 1 1 ) ( C B0 m0 ) + e m m /2 B 0 m 2 m 0 1. The value of B 0 = 2 s fxed, so for large enough values of C, m wth C > m the above probablty β 2 s less than δ 3. In partcular ths holds for choces m > 16/δ3 2 and C > 4m/β4 δ3 2. Observe that f B w θ / [ E(A), +E(A)], at most one of the two checks performed by the verfer can be satsfed. Hence the probablty of acceptance of the verfer s less than δ 3 2. Lemma 7.2. For all β > 0, and a constant B 0 2 β 2, f V { 1, 1} n s a 4-wse ndependent space of vectors then for any a R n wth at least B 0 of ts components greater than β n absolute value, Pr[ a v > 1] 1 12, where the probablty s over random choce of v V. Proof. The followng proof s along the lnes of proof of Theorem 2.2 n [3], and we nclude t here for the sake of completeness. Defne a random varable x = a v 2 for v chosen unformly at random from V. Then t can be shown that, E[x] = a 2 2, E[x 2 ] = 3 a a 4 4 < 3 a 4 2. Snce at least B 0 components of a are larger than β, we have a 2 2 > B 0β 2 2. Therefore, f Pr[ a v > 1] = α < 1 12, then Usng the Cauchy-Schwartz nequalty, we know E[x x > 1] 1 α ( a 2 2 (1 α) 1) > 1 2α a 2 2. E[x 2 x > 1] (E[x x > 1]) 2 > 1 4α 2 a