On the Advantage over a Random Assignment*

Transcription

1 On he Advanage over a Random Assignmen* Johan Håsad, 1, S. Venkaesh 2, 1 Nada, KTH, SE Sockholm, Sweden; johanh@nada.kh.se 2 DIMACS Cener, CoRE Building, Rugers Universiy, 96 Frelinghuysen Road, Piscaaway, New Jersey 08854; venka@dimacs.rugers.edu Received 19 July 2002; acceped 6 November 2003; received in final form 15 May 2004 DOI /rsa Published online 23 June 2004 in Wiley InerScience ( ABSTRACT: We iniiae he sudy of a new measure of approximaion. This measure compares he performance of an approximaion algorihm o he random assignmen algorihm. This is a useful measure for opimizaion problems where he random assignmen algorihm is known o give essenially he bes possible polynomial ime approximaion. In his paper, we focus on his measure for he opimizaion problems Max-Lin-2 in which we need o maximize he number of saisfied linear equaions in a sysem of linear equaions modulo 2, and Max-k-Lin-2, a special case of he above problem in which each equaion has a mos k variables. The main echniques we use, in our approximaion algorihms and inapproximabiliy resuls for his measure, are from Fourier analysis and derandomizaion Wiley Periodicals, Inc. Random Sruc. Alg., 25: , 2004 Keywords: linear sysem of equaions; inapproximabiliy; PCP; approximaion algorihms; saisfiabiliy Correspondence o: J. Håsad *This work was done while he auhors were a he Insiue for Advanced Sudy, Princeon, NJ, USA during he academic year A preliminary version of his work appeared in he Proceedings of he 34h Annual ACM Symposium on Theory of Compuing, Suppored in par by he Göran Gusafsson Foundaion and NSF Gran CCR Suppored by a join IAS DIMACS posdocoral fellowship Wiley Periodicals, Inc. 117

2 118 HÅSTAD AND VENKATESH 1. INTRODUCTION Given any opimizaion problem, one can ask if here is an efficien algorihm ha finds he opimal soluion. The heory of NP-compleeness allows us o prove ha many explici problems do no allow for efficien algorihms assuming ha NP conains difficul problems. In paricular, i has been known for a long ime ha many naural opimizaion problems are NP-hard. If we assume ha P NP, none of hese problems have a wors case polynomial ime algorihm finding opimal soluions. Given he evidence ha i is probably hard o develop efficien algorihms ha give opimal soluions for hese problems, i is naural o ask if i is possible o design efficien algorihms for hese problems ha give reasonably good soluions for all insances. Usually, an algorihm is said o be a c-approximaion algorihm for a maximizaion problem if i, for each insance, produces a soluion whose objecive value is a leas OPT/c, where OPT is he global opimum. A similar definiion can be given for minimizaion problems. The heory of approximabiliy of NP-hard problems has been an acive area of research in he pas decade. Many new approximaion algorihms have been designed for a variey of opimizaion problems. Many inapproximabiliy resuls have also been proved based on plausible complexiy heory assumpions. Consider any algorihm ha is designed o solve a maximizaion problem. A general crierion o evaluae is performance is OPT X ALG X where OPT is he opimum, ALG is he objecive value of he soluion oupu by he algorihm and X is a parameer o be chosen. In he usual definiion, X is chosen o be zero since in mos opimizaion problems, all feasible soluions have nonnegaive values. However, here are ineresing varians ha have been sudied. For example, here are opimizaion problems like quadraic programming where he value of he opimum soluion could be negaive. In such cases, choosing X o be he minimum possible value of a feasible soluion is an appropriae choice. Such a definiion was used by Bellare and Rogaway [8] in heir inapproximabiliy resul for quadraic programming. Anoher possible varian is o fix a polynomial ime algorihm A, and le X be he value of he soluion obained by A. If A is randomized, we can le X be he expeced value of he soluion obained by A. Such a definiion compares he performance of any algorihm wih a fixed and known polynomial ime algorihm A. This definiion can paricularly lead o useful informaion for many opimizaion problems if A represens a broad class of algorihms ha is known o give he bes approximaion algorihm for a wide range of opimizaion problems. In his paper, we focus on a class of opimizaion problems called consrain saisfacion problems. A consrain saisfacion problem has an underlying Boolean predicae P. An insance of his problem is given by a collecion of consrains C and he goal is o find an assignmen o he variables ha maximizes he number of saisfied consrains in C under he predicae P. Håsad [14] has shown ha for many consrain saisfacion problems like Max-Lin-p, in which we are required o maximize he number of saisfied equaions in a sysem of linear equaions modulo a prime p, and Max-E3-Sa, in which we are required o maximize he number of saisfied clauses in a 3-CNF formula, he random

3 THE ADVANTAGE OVER A RANDOM ASSIGNMENT 119 assignmen algorihm essenially yields he bes possible approximaion raio. This makes he sudy of a new measure of approximaion ha compares he performance of an algorihm for a consrain saisfacion problem wih he random assignmen algorihm ineresing. We sudy his measure specifically for Max-Lin-2 and is special case Max-k-Lin-2 where each equaion conains a mos k variables. We also ge similar resuls for oher problems and poin ou some connecions o he quesion of learning pariy wih noise. In a relaed work, Alon, Guin, and Krivelevich [2] recenly sudied a new measure for approximaion algorihms called he dominaion raio. They obain deerminisic polynomial ime algorihms ha achieve dominaion raio 1 o(1) for he pariion problem and dominaion raio (1) for he Max-Cu problem. Some of he echniques used by hem are similar o hose in his paper Our Resuls In his paper, we focus on he following opimizaion problem: We are given a sysem of m linear equaions modulo 2 in n variables, ogeher wih posiive weighs w i,1 i m. To avoid degenerae cases, we sae our resuls for he case ha m n. The goal is o oupu an assignmen o he variables ha maximizes he oal weigh of he saisfied equaions. We use {1, 1}-noaion for Boolean values wih 1 corresponding o rue. In his noaion, addiion modulo 2 is muliplicaion and we wrie equaion i as x j b i. j i where each i is a subse of [n] and b i {1, 1}. We consider wo cases: Max-k-Lin-2, in which each i is of size a mos k and Max-Lin-2, he general case wihou any resricions. If W is he oal weigh of all equaions, ha is, W m i1 w i, our performance measure is given by max L SATOPTL W/2 SATALGL W/2, (1) where L is an insance, SAT[OPT(L)] denoes he oal weigh of equaions saisfied by he opimal soluion and SAT[ALG(L)] denoes he oal weigh of equaions saisfied by he soluion oupu by he algorihm ALG. We noe ha (1) compares he performance of an approximaion algorihm for Max-Lin-2 wih he random assignmen algorihm. I measures how much of he gap beween he opimal soluion and he soluion of he random assignmen algorihm is recovered by an approximaion algorihm. Such a definiion can be given for any consrain saisfacion problem. Throughou he discussion, we assume ha each equaion in our sysem is nonrivial. In oher words, we do no allow equaions wih A. We also assume ha i i for i i. If his is no he case, i can be achieved as follows. If he lef-hand side of wo equaions are equal, hey can be merged o one equaion wih he appropriae weigh. If he righ-hand sides are he same, he new weigh is he sum of he original weighs, and, if hey are differen, hen he new weigh is he difference of he original weighs and he righ-hand side is ha of he equaion wih higher weigh. As a saring poin, le us firs indicae how we can obain a nonnegaive objecive

4 120 HÅSTAD AND VENKATESH value. We assign values o he variables sequenially and simplify he sysem of equaions as we go along. When we are abou o give a value o x j, we consider all equaions reduced o he form x j b for a consan b. We choose a value for x j saisfying a leas half (in he weighed sense) of hese equaions. I is easy o see ha he oal weigh of he saisfied equaions is a leas he oal weigh of he falsified equaions. By being careful, a m-approximaion algorihm can be obained as follows. Pick he equaion wih larges weigh. We consruc a soluion ha saisfies his equaion and a leas half (in he weigh sense) of he res of he equaions. This is sufficien o ensure ha we have a m-approximaion algorihm. Le us assume, wihou loss of generaliy, ha i is he firs equaion and ha 1 conains he variable x 1. For any i, i 1, ha conains x 1, replace he ih equaion j i x j b i by j i 1 x j b i b 1, where denoes he symmeric difference. The resuling m 1 equaions now do no conain he variable x 1. Now obain, as described above, an assignmen o he variables x 2,...,x n such ha a leas half (by weigh) of he remaining m 1 equaions are saisfied and finally give x 1 a value o saisfy he firs equaion. Is i possible o improve he algorihm by picking anoher equaion and repeaing he variable eliminaion sep described above? The problem is ha he firs variable eliminaion sep could creae pairs of equaions wih equal weighs and wih he same lef-hand side bu wih differen values on he righ-hand side. These equaions would hen, as discussed above, cancel each oher, possibly resuling in an empy sysem of equaions. If, however, he chosen equaion is saisfied by he opimal assignmen, he siuaion is easy o conrol. This is he basis of one of our approximaion algorihms. Previous resuls give some bounds for our curren measure. In paricular, using Håsad s resuls [14], i can be shown, for k 3, ha i is hard o approximae Max-k-Lin-2 (and hence Max-Lin-2) wihin c for every c 1 unless NP P and wihin (log m) c for some consan c 0 unless NP DTIME[m O(log log m) ] Bounded Size Equaions. We firs consider he case when each equaion has a mos k variables for some small k. We sar wih a randomized approximaion scheme for our new measure. Theorem 1.1. Consider Max-k-Lin-2. There exiss a fixed consan c 1 such ha he following holds: For any k O(log n), here is a randomized polynomial ime algorihm ha, wih probabiliy a leas 3/4, oupus an assignmen ha gives an approximaion raio a mos c k m. By running he algorihm several imes and oupuing he bes assignmen, he probabiliy 3/4 of oupuing an assignmen of he desired qualiy can be made exponenially close o 1.

5 THE ADVANTAGE OVER A RANDOM ASSIGNMENT 121 For small k, his algorihm does much beer han he simple m-approximaion algorihm. The algorihm is simple and i jus repeaedly ries random assignmens. The proof of correcness of his algorihm uses a resul from Fourier analysis ha relaes he L 4 -norm of any funcion ha has all is Fourier suppor on ses of size a mos k o is L 2 norm. We improve on he inapproximabiliy resuls menioned above by using a slighly sronger assumpion. Theorem 1.2. Unless NP DTIME[2 (log m)o(1) ], for all k 3 and 0, here is no algorihm ha approximaes Max-k-Lin-2 wihin 2 (log m)1 and runs in ime 2 (log m)o(1). The proof of Theorem 1.2 shows ha he same resul also applies o he special case of Max-k-Lin-2 where each equaion conains exacly k variables. The proof of his resul is similar o he proof of he resul by Håsad ha shows ha i is NP-hard o approximae (in he old-fashioned sense, aking raios of he number of saisfied equaions) Max-E3- Lin-2 wihin 2 for any 0. I is based on he correspondence beween approximaion problems and probabilisically checkable proofs. One imporan difference beween he wo proofs is ha we canno use long codes in our resul as hey are oo long. We make use of spli codes defined recenly by Kho [12] The General Case. We now consider he general case in which here is no resricion on he number of variables in each equaion. We sar wih he inapproximabiliy resuls. Theorem 1.3. There exiss a consan 0 such ha i is NP-hard o approximae Max-Lin-2 wihin m. This proof uses an idea from derandomizaion and in paricular i is based on he walk on expanders consrucion. If we allow randomizaion, we can ge a sronger inapproximabiliy resul. Theorem 1.4. For any 0, unless NP RP, here is no randomized polynomial ime algorihm ha, wih probabiliy a leas 1 2, oupus an assignmen for Max-Lin-2 wih an approximaion raio a mos m 1 2. This resul uses a sraighforward sampling echnique. I gives evidence ha no approximaion algorihm is likely o achieve an approximaion raio much beer han m 1/2 in general. For equaions wih few unknowns, Theorem 1.1 shows ha one can almos do his well on such insances. Theorem 1.4 does no apply o he siuaion when each equaion has only a consan number of variables. However, i does no use he full power of he general case since each equaion in he proof of he heorem has only O(log n) variables. The bes upper bound we can show for he general case is raher poor. Theorem 1.5. For any c 0, here is a randomized polynomial ime algorihm ha, wih probabiliy 3/4, oupus an assignmen for Max-Lin-2 wih approximaion raio a

6 122 HÅSTAD AND VENKATESH m mos. There is also, for any c 0, a deerminisic approximaion algorihm ha c log m approximaes Max-Lin-2 wihin m c. The main idea is o use randomizaion and exend he greedy algorihm described in he beginning of Secion Some Algorihmic Implicaions of Our Work The measure we sudy for Max-Lin-2 has connecions o some well-sudied algorihmic quesions Approximaion Algorihms for Max-Lin-2. Consider he sandard measure of approximaion. I is known ha for Max-k-Lin-2, he random assignmen algorihm achieves approximaion raio 2. Håsad showed ha i is NP-hard o ge an algorihm wih approximaion raio 2 for any consan 0. Thus he only quesion is exacly how close o 2 we can ge. Theorem 1.6. For any c 0, here is a randomized polynomial ime algorihm for Max-Lin-2 ha achieves approximaion raio 2 c log m m We view his resul as a sep oward a beer undersanding of he correc value of as a funcion of m. Such resuls have been proved for problems like Max-Clique by Engebresen and Holmerin [11] and Kho [13] Learning Pariy wih Noise. Recenly, he problem of solving pariy wih noise has been sudied [10]. In his model, a random solvable sysem of linear equaions in n variables is generaed, and hen he righ-hand side of each equaion is changed wih probabiliy p where p 1 2. The ask is o reconsruc he soluion o he original sysem. If he number of equaions, m, is sufficienly large (m c p n urns ou o be enough) he soluion is, wih high probabiliy, unique and can be found in exponenial ime. No algorihm, even for unbounded m, running faser han 2 O(n/log n) is known and a major open problem is wheher his can be improved. In our performance measure, he bes soluion would ge a value around (1 2p)m while he second bes soluion would, wih high probabiliy, ge a value ha can be bounded by O(nm). Thus, an efficien approximaion algorihm for Max-Lin-2 wih an approximaion raio of m 1/2 for 0 would imply ha for m (n 1/(2) ) he opimal soluion could be recovered efficienly. However, Theorem 1.4 implies ha such an efficien approximaion algorihm for Max-Lin-2 is unlikely unless NP RP Organizaion of he Paper In he nex secion, we describe some background from proof sysems and some ideas from Fourier analysis ha are used in his paper. We hen give proofs of he heorems menioned in he inroducion in Secion 3. We end wih some conclusions in Secion 4.

7 THE ADVANTAGE OVER A RANDOM ASSIGNMENT BACKGROUND We sar by recalling some definiions of proof sysems used in his paper. Since he conceps are hopefully familiar o many readers of his paper, we keep he discussion brief and refer he reader o [7, 14] for more deails Proof Sysems A cenral player in a proof sysem is a probabilisic Turing machine V called a verifier. The goal of he verifier is o decide wheher o accep or rejec an inpu sring x. The verifier akes x as inpu and osses random coins r. In addiion, i has access o an oracle. Definiion 2.1. An oracle is a funcion * {0, 1}*. The verifier specifies a sring s * and he oracle reurns a number of bis as he answer. For us, i is convenien o view he oracle as holding a proof and he verifier as specifying a locaion for which oracle reurns he corresponding bi(s) of he proof as he answer. A he end of he compuaion, he verifier eiher acceps or rejecs x. We say ha he verifier acceps if i oupus 1 (wrien as V (x, r) 1) and ha i rejecs if i oupus 0. Definiion 2.2. Le c and s be real numbers such ha 1 c s 0. We say ha a language L has a Probabilisically Checkable Proof (PCP) wih soundness s and compleeness c if here exiss a probabilisic polynomial ime verifier V such ha For x L, here exiss an oracle such ha Pr r [V (x, r) 1] c. For x L, for every oracle Pr r [V (x, r) 1] s. We have he famous PCP-heorem ha ells us ha here are very efficien PCPs for any language in NP. Theorem 2.3 [4]. There is a universal ineger q such ha any language in NP has a PCP wih soundness 1/2 and compleeness 1, where V uses logarihmic number of random bis in he size of he inpu and makes a mos q nonadapive accesses o he oracle, each answered by a single bi. The number of bis accessed by he verifier can be reduced o 3, bu his pushes he soundness owards 1, alhough i remains a consan below 1. Moreover, he 3 queries o he oracle are nonadapive. When he verifier reads 3 bis nonadapively, he accepance condiion on each fixed random sring can be wrien as a CNF formula wih a mos 3 lierals in each clause. By adding dummy variables i is no difficul o ensure ha each clause is of lengh exacly 3, and we call such a formula an E3-CNF formula. This leads o he following varian of he PCP heorem ha is convenien for us. Theorem 2.4 [4]. Le L be a language in NP and x be a sring. There is a universal consan 1 for which he following holds: For every language L in NP, here is an algorihm ha, given an inpu sring x, runs for ime polynomial in x and produces a

8 124 HÅSTAD AND VENKATESH E3-CNF formula x,l such ha if x L, hen x,l is saisfiable while if x L, no assignmen o he variables of x,l saisfies more han a fracion of he clauses. We nex describe a wo-prover one-round ineracive proof sysem. The verifier in such a proof sysem has access o wo oracles bu has he limiaion ha i can only ask one quesion of each oracle and ha boh quesions have o be produced before eiher of hem is answered. Though we do no limi he answer size of he oracles, he verifier will no read more han polynomial number of bis since i runs in polynomial ime. We call he wo oracles P 1 and P 2 and he wo quesions q 1 and q 2. Since he oracles are only accessed hrough hese quesions we refer o he fac ha V acceps as V(x, r, P 1 (q 1 ), P 2 (q 2 )) 1. Definiion 2.5. Le c and s be real numbers such ha 1 c s 0. We say ha a language L has a wo-prover one-round proof sysem wih soundness s and compleeness c if here exiss a probabilisic polynomial ime verifier V wih wo oracles ha, on inpu x produces, wihou ineracing wih is oracles, wo srings q 1 and q 2, such ha For x L here are wo oracles P 1 and P 2 such ha Pr r [V(x, r, P 1 (q 1 ), P 2 (q 2 )) 1] c. For x L, for any wo oracles P 1 and P 2,Pr r [V(x, r, P 1 (q 1 ), P 2 (q 2 )) 1] s. Noe ha he quesions q 1 and q 2 are in boh cases he only quesions V asks he oracles. P 1 (q 1 ) depends on x, bu may no depend on q 2 and similarly P 2 (q 2 ) is independen of q 1. On many occasions, i is convenien o hink of P 1 and P 2 as wo acual dynamic provers raher han oracles or proofs. They are infiniely powerful and are cooperaing. They can make any agreemen before he ineracion wih V sars bu hen hey canno communicae during he run of he proocol. Thus i makes sense o ask P 1 and P 2 for he same informaion in differen conexs. Provers are, in general, allowed o be boh hisory-dependen and randomized. Since we only consider one-round proocols, here is no hisory and hence he quesion wheher he provers are hisory-dependen plays no role. As wih randomizaion, i can 1 be seen ha, for any x, he provers P 1 and P 2 maximizing Pr r [V(x, r, P 1 (q 1 ), P 2 (q 2 )) 1] can be made deerminisic wihou decreasing he accepance probabiliy. When proving he exisence of good sraegies for he provers we will, however, allow ourselves o design probabilisic sraegies, which hen, in principle, can be convered o deerminisic sraegies. Improving he soundness of a wo-prover proof sysem can be done by parallel repeiion where V repeas his random choices o choose u independen pairs of quesions (q (i) 1, q (i) u 2 ) i1 and sends (q (i) u 1 ) i1 o P 1 and (q (i) u 2 ) i1 o P 2, all a once. V hen receives u answers from each prover and acceps if i would have acceped in all u proocols given each individual answer. One could naively hope ha he soundness of his parallel 1 Fix an opimal sraegy, which migh be randomized, of P 1. Now, for each q 2, P 2 can consider all possible coin osses r of V producing q 2, compue q 1 and hen, since he sraegy of P 1 is fixed, exacly calculae he probabiliy ha V would accep for each possible answer. P 2 hen answers wih he lexicographically firs sring achieving he maximum. This gives an opimal deerminisic sraegy for P 2. We can hen proceed o make P 1 deerminisic by he symmeric approach.

9 THE ADVANTAGE OVER A RANDOM ASSIGNMENT 125 proocol would be s u ; bu his is no rue in general, and we have o rely on he parallel repeiion heorem of Raz [16]. Theorem 2.6 [16]. For all inegers d and s 1, here exiss c d,s 1 such ha given a wo-prover one-round proof sysem wih soundness s and answer sizes bounded by d, hen, for all inegers u, he soundness of u proocols run in parallel is bounded by c u d,s. Since we do no limi he answer size of he provers hey can of course misbehave by sending long answers which always cause V o rejec. Thus, by answer size, we mean he maximal answer size in any ineracion where V acceps Imporan Tool; Fourier Transforms We sudy funcions f :{1, 1} n 3 mapping {1, 1} n ino he se of real numbers. A key ool for us is he discree Fourier ransform. For any [n], we have he corresponding basis funcion x x i. i A general funcion can be expanded as fx fˆ x, where fˆ are he Fourier coefficiens defined by fˆ 2 n fx x. x Parseval s equaliy ells us ha fˆ 2 2 n x fx 2 and for a Boolean funcion, whose range is {1, 1}, his sum is Max-Lin-2 Fourier ransforms have been, and are also in his paper, imporan for he reason ha hey are useful in he analysis of cerain probabilisically checkable proofs. In he presen paper, Fourier ransforms play an even more cenral role because he very problem we sudy is saed very naurally in erms of he Fourier ransform. In order o see his, le us define he linear sysem formally. Remember ha we are in {1, 1}-noaion and hence exclusive-or is in fac muliplicaion.

10 126 HÅSTAD AND VENKATESH Definiion 2.7. A linear sysem L wih m equaions in n variables is given by subses of [n] ( i ) m i1, bis (b i ) m i1 and posiive weighs (w i ) m i1. The ih equaion is x j 1 bi. j i For an assignmen x 0 (x 0 n j ) j1 o he variables, le P L (x 0 ) be he oal weigh of all saisfied equaions, and le N L (x 0 ) be he oal weigh of all falsified equaions. The objecive value W(L, x 0 ) is defined as P L (x 0 ) N L (x 0 ). In he problem Max-Lin-2 we are given L, and we need o find an x 0 ha maximizes W(L, x 0 ). The measure we used in he inroducion subraced insead of N L (x 0 ) half he oal weigh bu i is easy o check ha he new measure is exacly wice he measure used in he inroducion. Since we are ineresed in raios of objecive values his change of scale is immaerial. We have he following lemma. Lemma 2.8. W(L, x 0 ) m i1 (1) b i i (x 0 )w i. Proof. Noe ha, by definiion, 1 bi x 0 j 1 bi i x 0 j i and his is 1 if he ih equaion is saisfied and 1 oherwise. The lemma follows. In oher words, we are given he Fourier coefficiens of a funcion, and we wan o find he maximum of he funcion. The only reason we do no have a general funcion mapping {1, 1} n o he real numbers is ha we require he number of (nonzero) Fourier coefficiens o be bounded by a parameer m which is usually much smaller han he maximal value 2 n. A paricular case ha we are ineresed in is he case where each equaion depends on a mos k variables. Definiion i m. An insance of Max-Lin-2 is said o belong o Max-k-Lin-2 if i k for I urns ou ha funcions ha have he suppor of he Fourier ransform on small ses have special properies and o capure one of hese le us sudy he L p -norm denoed by f p and defined by f p 2 n fx p 1/p. I is no difficul o see ha f p is an increasing funcion in p. The key resul we use is ha for funcions whose Fourier ransform is suppored on small ses, we have a reverse inequaliy proved in several conexs, bu he following is Theorem 6 from [9]. A similar resul can also be obained using Theorem 5 from [6].

11 THE ADVANTAGE OVER A RANDOM ASSIGNMENT 127 Theorem 2.10 (Bonami [9]). For any ineger k and p 2, le f be a funcion which has is Fourier suppor on ses of size a mos k; hen f p (p 1) k/2 f The Long Code and he Spli Code In PCP heory, many resuls use he long code defined by Bellare, Goldreich, and Sudan [7]. Le be he se of all funcions mapping {1, 1} o {1, 1}. Definiion Le x {1, 1}. Is long code is a vecor of lengh 2 2 wih is enries corresponding o elemens in and he value corresponding o f is f(x). The se {1, 1} has no special significance and he long code can be defined for any range M of x wih he se of all Boolean funcions on M, denoed M, aking he place of. I is obvious from he definiion ha he long code is indeed very long. The previous resuls have made good use of he huge amoun of informaion conained in hese codes. In some siuaions, like he one in his paper, a much shorer bu closely relaed code urns ou o be a good alernaive and hese are he spli codes defined by Kho [12]. Spli codes are defined for produc spaces. Definiion Le M M 1 M 2 M. The spli code of an elemen x (x 1, x 2,...x ) M is indexed by all uples g (g 1, g 2,...,g ) where g i Mi and he corresponding value is g i x i. i1 Thus, spli codes belong o he vecor space of all real-valued funcions on M1... M. We can define an inner produc on his space by B 1, B 2 1 B 1 gb 2 g. g Definiion Le ( 1,..., ), i M i. We define a characer as g g i x. i1 x i Under he inner produc defined above, he se of all characers form an orhonormal basis for he produc space. For an arbirary funcion B, we can wrie

12 128 HÅSTAD AND VENKATESH Bg Bˆ g, (2) where Bˆ 2 i1 M i Bg g. (3) g Folding was designed in [7] as a mechanism forcing a able ha is supposed o be a long code o a leas respec negaion. In paricular his mechanism prevens a able from being biased. The mechanism nicely exends o spli codes. Definiion A spli code B is folded over rue if Bg 1,...,g 1 s1 s B1 s1 g 1,...,1 s g (4) for all g 1,...,g and for all s 1,...,s {0, 1}. Folding is implemened by choosing, for each se of 2 inpus, B1 s1 g 1,...,1 s g, s i 0, 1, only one represenaive o be sored in he able. Suppose for concreeness ha B(g 1,..., g ) is chosen. Whenever he value of any oher inpu in he se is needed, B(g 1,...,g )is read and hen possibly negaed o saisfy (4). The following lemma saes a useful propery of folded ables. Lemma If a able B is folded over rue, hen Bˆ 0 for all ( 1,..., ) such ha i is even for some i. Proof. If i is even hen any erm in (3) is canceled by he erm obained by changing g i o g i while keeping he oher g j unchanged. Spli codes are crucial o our PCP consrucion as hey offer a less lenghy alernaive o long codes. The spli code on M is only of lengh 2 M 1M 2 M while a long code on M would be of lengh 2 M 1M 2 M Noaion All logarihms in his paper are o he base 2. We use boh o denoe he cardinaliy of any se and he absolue value of real numbers. The meaning would be clear from he conex.

13 THE ADVANTAGE OVER A RANDOM ASSIGNMENT PROOFS OF THEOREMS 3.1. Proof of Theorem 1.1 We are given an insance L of Max-k-Lin-2, and for breviy le f(x) W(L, x). Our ask is o find a value x 0 such ha f(x 0 ) c k m 1/2 max x f(x) wih high probabiliy. This is done by repeaedly picking uniformly random values for x 0, and his is formalized in Algorihm 1. Algorihm 1: Le N 16 3 k m. Pick N assignmens x (i),1 i N, independenly and uniformly a random. Oupu he assignmen x () wih he propery f(x () ) max j f(x (j) ) for 1 j N. To show ha Algorihm 1 achieves an approximaion raio as claimed, define X o be he random variable f(x) when x is chosen randomly. We claim ha X has he following properies: 1. E(X) E(X 2 ) m i1 w 2 i def X m. 4. E(X 4 ) 3 2k (E(X 2 )) 2 3 2k 4. The firs wo properies are sraighforward. The hird propery is shown by using Cauchy-Schwarz inequaliy as follows: m X i1 m w i i1 m 21/2 11/2 w i m 1/2. i1 The fourh propery follows from Theorem 2.10 wih p 4. We now have he following lemma. Lemma 3.1. Consider any random variable X which akes values beween m 1/2 and m 1/2 and has he following properies: (1) E(X) 0, (2) E(X 2 ) 2, and (3) E(X 4 ) 3 2k 4. Then, PrX k k m. Proof. Suppose no. Le us assume ha PrX k k m. Le X p max(0, X) and X n min(0, X). From Hölder s inequaliy, we have ha EX 2 n EX 4 n 1/3 EX n 2/3. (5)

14 130 HÅSTAD AND VENKATESH We now make he following claims using he assumpions of he lemma. 1. E(X n 4 ) 1/3 3 2k/3 4/3. 2. E(X n ) 2/ k/3 2/3. 3. E(X n 2 ) 2 /2. These claims clearly conradic (5), and hus we only have o esablish hese claims. The firs is simply an assumpion in he lemma. The second claim is esablished as follows: 1 EX n EX p k m 1/2 8 3 k m 41 3 k, and he hird follows from E(X 2 ) E(X p 2 ) E(X n 2 ) and EX p k m k m 2 /2. From Lemma 3.1, i follows ha when an assignmen x is picked uniformly a random, hen wih probabiliy a leas, he approximaion raio is bounded from above by k m max x fx fx i1 m w i /8 3 k m /8 3 k 8 3k m Algorihm 1 increases he probabiliy of success o 3/4 by picking assignmens independenly and uniformly a random and oupuing he bes. Hence, Theorem 1.1 follows. We believe ha Theorem 1.1 is no oo far from he rue behavior of Algorihm 1, and an example where i does poorly can be consruced using he discussion in Secion To be more precise, sar wih a arge soluion x 0 and m random pariies each of size k. Probabilisically choose he righ-hand sides so ha x 0 saisfies each equaion wih probabiliy 1 p. Clearly, f(x (0) ) (m) wih high probabiliy. Now look a any random assignmen x (i) generaed by Algorihm 1. Wih high probabiliy x (i) agrees wih x 0 in a mos (n O(n log n)/2 coordinaes. This implies ha when generaing an equaion he probabiliy ha x (i) and x 0 wan he same righ hand side is a mos (1 O((log n)/n)) k/2 )/2. Now, for m n s, s k i is no difficul o see ha wih high probabiliy we have max i f(x (i) ) O(m log n) while f(x (0) ) (m) Proof of Theorem 1.2 A well-known approach o prove an inapproximabiliy resul for an opimizaion problem is o sar wih a language L in NP and design a PCP for L in which he accepance crieria of he verifier closely mimics he opimizaion problem and his is also he approach used here. The consrucion of his PCP goes hrough hree sages: In he firs sage, using Theorem 2.4, checking membership in L is convered ino solving a version of a GAP-SAT problem. Nex, a wo-prover proocol is designed for GAP-SAT. Finally, a

15 THE ADVANTAGE OVER A RANDOM ASSIGNMENT 131 PCP is obained saring from he wo-prover proocol. We describe each sage of he reducion in deail below. Sar wih a language L in NP and a sring x. Apply he efficien algorihm saed in Theorem 2.4 o ge a 3-CNF formula x,l which is saisfiable if x L bu has he propery ha no assignmen saisfies more han a fracion of he clauses if x L. Noe ha his reducion convers checking membership in L o solving a GAP-SAT problem of deciding if a formula is saisfiable or far from being saisfiable. Now consider he following wo-prover proocol. The 2-prover proocol: The proocol uses parameers and u ha will be fixed laer. Sep 1: The verifier chooses u random clauses from x,l and u variables a random, one from each of he u clauses chosen. Sep 2: The verifier asks prover P 1 for he assignmen o each clause chosen. I asks prover P 2 for he assignmen o each variable chosen. Sep 3: consisen. The verifier acceps if all he clauses are saisfied and he wo assignmens are The wo-prover proocol has compleeness one. If x,l is saisfiable, P 1 and P 2 can agree on one saisfying assignmen and answer accordingly. These answers are always consisen and saisfy any clauses picked. The basic proocol wih u 1 has soundness (2 )/3 and hence, by Theorem 2.6, he soundness of his wo-prover proocol is upper bounded by c 1 u for some c 1 1. Thus, we have shown ha L has a wo-prover proocol wih compleeness 1 and soundness a mos c 1 u. We sae hese properies for fuure reference. Lemma 3.2. If x,l is consruced according o Theorem 2.4, hen, if x L, here are sraegies for P 1 and P 2 in he wo-prover proocol o make he verifier always accep. If x L, hen, for some absolue consan c 1 1, no sraegies for P 1 and P 2 can make he verifier accep wih probabiliy larger han c u 1. The nex sep of he reducion is o conver he wo-prover proocol for L described above ino a PCP. From 2-prover proocol o PCP: We firs describe he oracle (or he proof) ha is expeced by he verifier in he PCP. In oher words, we describe he proof used o convince he verifier ha a saisfiable L,x is indeed saisfiable. The verifier expecs wo proofs A and B. Boh of hem correspond o spli codes folded over rue as described in Secion 2.4. B is indexed by every possible quesion asked by he verifier o P 1 in he wo-prover proocol. Each such quesion is grouped ino groups of u clauses and he spli code of dimension for he answer o his quesion is sored in B. Similarly, A is indexed by every possible quesion asked by he verifier o P 2 in he wo-prover proocol. Each such quesion is grouped ino groups of u variables and a spli code of dimension for he answer o his quesion is sored in A. For he answers of prover P 1, he se M i, in he definiion of he spli code, is he se

16 132 HÅSTAD AND VENKATESH of saisfying assignmens of he clauses of group i. I is of cardinaliy a mos 7 u. For he answers of prover P 2, we denoe by N i, an assignmen o he chosen variables in group i. I is of cardinaliy a mos 2 u. We have and M M 1 M 2 M N N 1 N 2 N. Using he projecion operaor from M i o N i which maps an assignmen o he subassignmen, we obain a naural compound projecion operaor : M 3 N. The PCP consrucion: We now describe he PCP ha is used o derive he inapproximabiliy resul for Max-k-Lin-2. The verifier of his PCP uses a parameer 1 2. Sep 1: The verifier picks groups each conaining u clauses chosen a random resuling in u clauses (C 1, C 2,...,C u ). For each clause C i, i chooses a variable x i C i a random which are hen similarly divided ino groups. This defines M and N. Le B be he supposed spli code on M and A he supposed spli code on N, boh folded over rue. Sep 2: The verifier hen chooses random funcions f i : N i 3 {1, 1} and g i : M i 3 {1, 1} for 1 i. Sep 3: The verifier also chooses i : M i 3 {1, 1} for 1 i by seing for every x M i, i x 1 1 wih probabiliy 1, wih probabiliy. Sep 4: Define h i : M i 3 {1, 1} for 1 i by h i (x) f i ((x))g i (x) i (x). Sep 5: Le g (g 1,...,g ), f (f 1,...,f ) and h (h 1,...,h ). The verifier reads A(f), B(g), and B(h). The verifier acceps if and only if A(f)B(g)B(h) 1. We now do he compleeness and he soundness analysis of our PCP consrucion. Compuing he compleeness is sraighforward as shown below. To show ha soundness of he PCP is small, our sraegy is o prove ha if, on he conrary, i is high, hen we can exrac sraegies for he wo provers in he wo-prover proocol o convince he verifier in ha proocol o accep wih high probabiliy. Using he very good soundness of he wo-prover proocol saed above, we can conclude ha he formula is saisfiable. This helps us obain an upper bound on he soundness of he PCP. We now presen he deails. Lemma 3.3. The compleeness of he PCP is a leas Proof. If x,l is saisfiable, fix a saisfying assignmen giving sraegies for P 1 and P 2. As indicaed above he wrien proof for he PCP is he spli-codes of he answers by P 1 and P 2. We have

17 THE ADVANTAGE OVER A RANDOM ASSIGNMENT 133 AfBgBh 1 N i x i 1, i1 where x i is he ih componen of he answer by P 1. I is no difficul o see ha and he lemma follows. Pr 1 i x i i , 2 The key lemma for esablishing he soundness is given below: Lemma 3.4. Le be he parameer of he PCP and le be such ha he probabiliy ha he verifier of he PCP acceps is (1 )/2. Then here is a sraegy for P 1 and P 2 in he wo-prover proocol ha makes he verifier of ha proocol accep wih probabiliy a leas (4 2 ). Remark 3.5. As we know, by Lemma 3.2, he soundness of he wo-prover proocol is bounded by c u 1 and hence we can, by a suiable choice of u, ge soundness (1 )/2 for any desired consan for he PCP. Proof. Fix a choice of N and M. From he assumpion in he lemma, 1 AfBgBh 1 E N,M,f,g, 2. 2 This implies ha E AfBgBh. N,M,f,g, We would now like o use his fac o exrac a successful wo-prover sraegy. To his end, we evaluae he expression We need he following definiion. E AfBgBh. f,g, Definiion 3.6. For ( 1,..., ), i M i, define 2 2 1,..., 2 where 2 ( i ) is he se of poins in N i wih odd number of preimage poins in i under projecion. Replacing each funcion by is Fourier expansion, we ge

18 134 HÅSTAD AND VENKATESH E AfBgBh E f,g, f,g,, 1 2  fbˆ 1 1gBˆ 2 2h, 1 2  Bˆ 1Bˆ 2 E f 1g 2fg. f,g, Since 2(fg) 2(f) 2(g) 2(), i is no difficul o see ha he inner expecaion equals 0 unless 1 2 and 2 (). Finally E 1 2 1, giving he oal resul  2Bˆ (6) Since B is folded over rue, each nonzero erm in his sum has i, and hence 2 ( i ), odd for every i. In paricular each i is nonempy. By Cauchy-Schwarz inequaliy, we have  2Bˆ and since Bˆ 2 1, we can conclude ha E N,M 2 21  2 Bˆ 1 2 E 2 E Bˆ 2 N,M N,M 2 21  2 Bˆ  2Bˆ  2Bˆ (7) Le us now define a sraegy for he wo provers in he wo-prover proocol as follows: P 1, upon receiving a se of clauses, finds he corresponding able B and selecs a random wih probabiliy Bˆ 2. I reurns y i i,1 i, chosen a random as he answer. P 2, upon receiving a se of variables, finds he corresponding able A and selecs a random wih probabiliy  2. I reurns x i i,1 i, chosen a random. The probabiliy of he provers convincing he verifier is a leas E N,M 2  2 Bˆ 2 1. i1 i

19 THE ADVANTAGE OVER A RANDOM ASSIGNMENT 135 For any x 0, we have x 1 e x. Applying his o x (4 i ), we have 1 i 4e4i i and by now using he lower bound from (7), he lemma follows. Thus, we have shown ha L has a PCP wih compleeness a leas (1 (1 2) )/2 and soundness a mos (1 )/2. To complee he proof of Theorem 1.2, we firs broadly describe he relaionship beween PCPs and proving inapproximabiliy resuls and hen focus on he implicaions of he PCP designed above. Le us define he following proof opimizaion problem based on he consruced PCP. For every possible locaion l of he proof we have a Boolean variable y l. The goal is o find an assignmen o he variables y l such ha he corresponding proof makes he verifier of he PCP accep x wih he highes possible probabiliy. If he PCP has compleeness c and soundness s, hen he value of he proof opimizaion problem, when x belongs o L, isa leas c while i is a mos s when x does no belong o L. This implies ha if we can approximae he opimal value of he proof opimizaion problem wihin a facor beer han c/s, hen we can solve membership in L which is NP-hard in general. Now consider he PCP ha we have consruced. By doing some inernal calculaion, he verifier V finally reads hree bis of he proof nonadapively and checks ha he produc of hese hree bis has a given value. One migh be emped o hink ha his value is always 1 (in he {1, 1} noaion) bu i is no so due o he mechanism of folding ha has he affec ha someimes he bi read in he proof should be negaed before i is used. Suppose ha wih probabiliy p, V reads posiions l 1, l 2, and l 3 and checks ha he produc of hese hree bis is 1. Le us wrie his as an equaion y l1 y l2 y l3 1 and give i weigh p. Coninuing his way wih all possible choices of posiions for V, we ge a sysem of weighed linear equaions and he oal weigh of saisfied equaions is exacly he probabiliy ha he verifier acceps a given proof. Thus, he proof opimizaion problem is exacly he problem of maximizing he oal weigh of saisfied equaion and his is he problem for which we are rying o prove an inapproximabiliy resul. Noe ha i is crucial ha he verifier does no use oo many random coins as he size of he resuling sysem of linear equaions depends on he number of coin osses. In fac, he size of he produced insance of equaions is 2 r where r is he number of coins used. Hence, we conclude ha i is hard o approximae Max-3Lin-2, in our performance measure, beer han (c 1/2)/(s 1/2) on insances of size 2 r. We will now fix he various parameers,, u, and so as o ge a srong separaion. Fix 1/4. Fix u o be a large enough consan so ha c 1 u 4 3 where c 1 is he consan specified in Theorem 2.6. If m is he number of clauses in he 3-CNF formula given as inpu o he PCP, he number of equaions produced by he ransformaion described above is N m O() wih oal weigh 1 (as we are dealing wih probabiliies). As seen by he analysis above c 1/2 1 2

20 136 HÅSTAD AND VENKATESH and Therefore, c 1/2 s 1/2 s 1/ Choose l 1 and se (log m) l. The number of equaions is N m O() 2 O((log m)l1) and he raio c1/2 s1/2 2 2 (log m)l. I follows ha an approximaion algorihm wih a performance raio of clog N1 2 for a suiable consan c would be sufficien o decide membership in L. As he size of he insance is only quasi-polynomial if an approximaion algorihm wih he given raio exised and ran in quasipolynomial ime his would imply NP DTIME[2 (log m)o(1) ]. This complees he proof of Theorem 1.2 for he case k 3. The exension o larger k is no difficul and can be done along he lines used for he similar exension in [14]. We omi he deails Proof of Theorem 1.4 The proofs of Theorems 1.4 and 1.3 are quie similar. Since he proof of Theorem 1.4 is simpler, we presen i firs. The basic idea is o sar wih any 3SAT formula and produce a sysem of linear equaions L such ha he following holds: If is saisfiable, hen here is an assignmen o he variables of L which achieves a high objecive value. If is no saisfiable, hen every assignmen o he variables of L only achieves a low objecive value. We do his by aking -wise producs (sums in {0, 1} noaion) of ses of equaions obained from a 3SAT formula. We now give he deails. In Håsad s paper [14], for any 0, a reducion is shown ha akes 3SAT formulas wih l clauses o Max-3-Lin-2 sysems wih m l O(1) linear equaions on n variables, n m, such ha 1. If is saisfiable, hen here is an assignmen x 0 o he variables of he resuling sysem of linear equaions L such ha WL, x 0 1 2m. 2. If is no saisfiable, hen for every assignmen x 0 o he variables o L, WL, x 0 m. Le us inroduce some noaion.

21 THE ADVANTAGE OVER A RANDOM ASSIGNMENT 137 Definiion 3.7. A sysem, L, wih m equaions is said o be -saisfiable if W(L, x) m for every possible assignmen x o is variables. Definiion 3.8. Le L denoe a sysem of N linear equaions. Is -wise produc, L, is a sysem of (2N) linear equaions defined as follows: Choose equaions from L. For every such choice of equaions, oupu 2 linear equaions obained by aking producs of every possible subse of hese equaions. The following lemma says ha aking -wise producs help in boosing he separaion beween he wo cases: is saisfiable or no. Lemma 3.9. The -wise produc of a -saisfiable sysem is a ( 1 2 ) -saisfiable sysem. Proof. An assignmen ha saisfies all he chosen equaions saisfies all he 2 consruced equaions. An assignmen ha falsifies some equaion saisfies exacly half of he equaions consruced. I follows ha a -wise produc is ( 1 2 ) -saisfiable. The lemma implies ha in he case when is saisfiable hen he sysem L is (1 ) -saisfiable while when is no saisfiable hen L is a mos ( 1 2 ) -saisfiable. We have one small deail o address. Taking he -wise produc produces equaions of he form 1 1 and 1 1. These rivial equaions have o be dropped as hey are no allowed in our sysems. This affecs he number of equaions as well as he obained objecive value. I urns ou ha if L has a leas as many equaions of he form 1 1as 1 1 his process is only o our advanage and hence we like such sysems. Definiion A sysem of linear equaions L is said o be good if, in L, he number of equaions of he form 1 1 is greaer han or equal o he number of equaions of he form 1 1. The -wise producs are in fac good. Lemma A-wise produc, L, of any sysem of linear equaions L is good. Proof. Fix a choice of equaions and consider he se S of 2 equaions produced by aking all possible subses. Using he characerisic vecor noaion, view S as a vecor space F over F {0, 1} under he usual operaions. Le C be he subcollecion of all subses ha resul in an equaion of he form 1 1 and 1 1. Then, C is a subspace of S since C is closed under symmeric difference. Choose a basis B for C. If every equaion in B is of he form 1 1, hen every equaion in C is of he form 1 1. If here are equaions in B of he form 1 1, hen here are equally many equaions of he form 1 1 and 1 1inC. Le us consider he effec on L of dropping hese rivial equaions. The objecive value is, before dropping rivial equaions, of he form m where m is he number of equaions and eiher (1 ) or ( 1 2 ). If he sysem is good he erasing decreases he objecive value o m C for some C 0. We only need o bound he raio beween he objecive values in he wo cases and his is minimized when C 0. I follows ha

22 138 HÅSTAD AND VENKATESH we can ignore hese rivial equaions as long as we make sure ha he sysems we sudy are good. Though aking -wise producs helps us ge an improved separaion, he number of linear equaions produced would be oo many (for our choice of ) for a good inapproximabiliy resul. We decrease he number of equaions using randomizaion. For a 3SAT formula, obain L as described above. For a suiable choice of parameers s and o be fixed laer, choose N m s random ses of equaions from L each of size. For every such choice R, consruc 2 equaions as in he definiion of L and choose one of hese equaions a random. Le he resuling sysem of linear equaions be denoed by L,R.IfL,R is no good, repea he consrucion. If afer n such repeiions, a good sysem of equaions is no produced, hal. Oherwise, erase all he rivial equaions from L,R and hal. Since he probabiliy ha he sysem of linear equaions produced in an ieraion is no good is a mos 1/2, he probabiliy ha he consrucion above fails o produce a good sysem of linear equaions is upper bounded by 1/2 n. We would now like o compue he probabiliy ha a good sysem of linear equaions has he separaion propery discussed above. We do i in wo seps. Firs, we esimae his probabiliy for a random sysem of N equaions produced by an ieraion of he algorihm above. Then, we show ha his probabiliy coninues o be high condiioned on he fac ha he sysem of linear equaions is good. Suppose is saisfiable. Then, here is an assignmen x 0 o he variables of L ha saisfies (1 )m of he equaions in L. Over he various random choices, EWL,R, x 0 1 N. We need he following resul from Alon and Spencer [3]. Theorem 3.12 ([3], Theorem A.1.13). random variables such ha Le X i,1 i k, k arbirary, be independen k and le X i1 X i X i. Then, p wih probabiliy oherwise, 2, 2 Pr[X (1 )pk] e 8 pk. Le X i denoe he random variable ha is 1 if he ih equaion is saisfied by x 0 and 1 oherwise. Then, X i is 1 wih probabiliy (1 p)/2, where p (1 ) and N i1 X i WL,R, x 0.

23 THE ADVANTAGE OVER A RANDOM ASSIGNMENT 139 Therefore, using Theorem 3.12, wih 1/2, p (1 ) and k N, over he differen choices of random equaions, PrWL,R, x 0 1 N/2 1 e 1N/32. (8) Now suppose ha is no saisfiable. Then no assignmen saisfies more han ( 1 2 )m of he equaions in L. We need he following resul from Alon and Spencer [3]. Theorem 3.13 ([3], Theorem A.1.4). random variables such ha Le X i,1 i k, k arbirary, be independen k and le X i1 X i 1 wih probabiliy 1 2 p 2, 1 oherwise, X i. Then, for any a 0, a2 Pr[X kp a] e 2k. For a fixed assignmen x 0, le X i be he random variable ha is 1 if he ih equaion is saisfied by x 0 and 1 oherwise. Then, X i is 1 wih probabiliy (1 p)/2, where p ((1 )/2) and N i1 X i WL,R, x 0. Therefore, using Theorem 3.13, we ge PrWL,R, x 0 N 1 2 2nN 2 2n, and hence Pr max x 0 1 WL,R, x 0 N 2 2nN 1 2n. (9) Sar wih any 1/4. Fix /8, s 2/, and log N. Then, wih probabiliy a leas 1 2 1n over random choices in he consrucion above, boh (8) and (9) hold simulaneously and hence he resuling sysem of linear equaions has a separaion propery we are looking for. Since a random choice is good wih probabiliy a leas 1/2, i follows ha a good sysem of linear equaions has he same separaion propery wih probabiliy a leas 1 2 2n. Thus he probabiliy ha he sysem of linear equaions produced by he consrucion above is good and has he separaion propery is a leas (1 2 2n )(1 2 n ) 1 2 3n. Also for he choice of, s and above,

24 140 HÅSTAD AND VENKATESH 1 N 2 2nN 3nN 1 N/2 N 12. Since 3SAT is NP-complee, any algorihm ha gives an approximae value of he opimum wihin N 12 N1/2 3nN can be used o solve an arbirary problem in NP. This proves he heorem. Since he random sampling sep involves wo-sided error, i seems like we need he assumpion NP BPP. I is no difficul o see ha using he self-reducibiliy of SAT, one can prove ha if NP BPP, hen in fac NP RP. Hence, he assumpion NP RP is sufficien Proof of Theorem 1.3 We use a consrucion very similar o he one in Theorem 1.4. In he firs sage, as in Theorem 1.4, we ransform he 3SAT formula ino a sysem of linear equaions wih he separaion propery. In he second sage, we replace random sampling by deerminisic sampling obained using walks on expanders. We now give he deails of our consrucion. Definiion Le G be a d-regular graph on n verices and le d n1 be he eigenvalues of he adjacency marix of G. G is said o be a Ramanujan graph if 1, n1 2d 1. Explici consrucion of such graphs were firs shown by Lubozky, Philips, and Sarnak [15]. Theorem 3.15 ([15]). Pick a prime p congruen o 1 modulo 4. Then, for every prime q congruen o 1 modulo 4 differen from p and such ha he Legendre symbol ( p q ) 1, here is an explicily consrucible Ramanujan graph G p,q on n q(q 2 1)/2 verices which is d p 1-regular. Theorem 3.16 ([3], p. 142). Given any ineger m and a prime p congruen o 1 modulo 4, using Theorem 3.15, we can consruc explicily a Ramanujan graph on (1 o(1))m verices ha is p 1-regular. Thus, for m large enough, using he Theorem above, we can consruc explicily a 30-regular Ramanujan graph G on m nodes for m m 2m. Given a 3SAT formula, we label verex i m of G by equaion i from L and every verex i m by he rivial equaion 1 1 and call he new graph obained as G.