COS 522: Complexiy Theory : Boaz Barak Hadou 0: Parallel Repeiio Lemma Readig: () A Parallel Repeiio Theorem / Ra Raz (available o his websie) (2) Parallel Repeiio: Simplificaios ad he No-Sigallig Case / Thomas Holesei hp://arxiv.org/abs/cs/060739 Parallel Repeiio We sae ad prove i for 2 prover games, ca be easily geeralized o 2-query PCP s (see Exercise ). Def: I a wo-prover game:. Two provers firs coordiae wo sraegies P ad P 2 (possibly wih shared radomess). 2. The verifier selecs a radom pair of srigs (x, y) from a joi disribuio (X, Y ) ad gives each ipu o each prover respecively. 3. We deoe by a = P (x) he firs prover s aswer ad b = P 2 (y) he secod prover s aswer. 4. The verifier decides o accep or o based o some predicae V (c, y, a, b). The value of he game is he maximum over all sraegies P, P 2 of he probabiliy ha V acceps (his probabiliy is over (X, Y ) ad possibly he provers radomizaio). The -imes parallel repeiio of he game, is whe V chooses (x, y ),..., (x, y ) idepedely from he disribuio (X, Y ), gives o P he values (x,..., x ) ad o P 2 he values (y,..., y ) o obai he respecive aswers (a..., a ) ad (b,..., b ). V acceps oly if all he aswers check ou. Noe ha V acs idepedely o each isace, bu he provers may correlae heir aswers o differe isaces. Theorem : (Forow, Rompel, Sipser 88) If he origial game had value a mos ( δ) he repeaed game has value a mos ( δ). Proof : Obvious sice he all he queries are idepede ad each oe ca be saisfied wih probabiliy a mos δ, he probabiliy ha he provers ca saisfy all of hem is a mos ( δ). Couerexample o heorem (Forow, Feige): Cosider followig game: verifier chooses wo radom ad idepede bis x, y, he aswers are pairs i {, 2} {0, }. The verifier acceps he aswers a, b iff a = b = (i, σ) ad Prover i received he bi σ. Two observaios:. The value of he game is a mos /2: for he verifier o accep, he wo provers have o sed a message wih he same i, which meas ha oe prover has o guess he bi received by he oher prover. 2. The value of he repeaed game is a leas /2: cosider he followig sraegy: firs prover o x, x oupus he aswers (, x) ad (2, x) ad secod prover o ipu (y, y ) will oupu (, y ) ad (2, y ). Now if x = y (which happes wih probabiliy /2) he he verifier will accep. Parallel Repeiio Theorem (Raz 95, Holesei 07) The value of he repeaed game is a mos 2 Ω(δ3 ) where he cosa i he Ω oaio depeds (logarihmically) o he alphabe size of he provers aswers.
Proof sraegy Le P, P 2 be some arbirary provers. We wa o boud he probabiliy of he eve W W 2 W (where W i is he eve ha hey wi he i h isace). Sice 2 Ω(δ3) = ( δ) Ω(δ2), i suffices o show ha for all k cδ 2 Pr[W k+ W W k ] δ/00, for c some cosa depedig logarihmically o he provers alphabe. I fac, sice we ca reorder coordiaes as we wish, i will suffice o prove he followig: Mai Lemma: There s c > 0 (depedig logarihmically o prover s alphabe) such ha for all k cδ 2, j > k such ha Pr[W j W W k ] δ/00 () Ideed, sice we oly wa o prove ha W W is low, i suffices o prove () uder he assumpio ha Pr[W W k ] 2 (k+), where Σ is he size of he prover s alphabe. This will esure ha leig p k be he probabiliy of W W k, he p k+ max{2 (k+), p k ( δ/00)}, which suffices o prove he lemma. Rough proof idea for Mai Lemma: We ll prove he mai lemma by reducio. Tha is, we will assume ha here are provers sraegies violaig () ad will use hem o succeed i he origial (urepeaed) game wih probabiliy more ha δ. The idea is ha for some coordiae j we will show ha if Prover is give x j ad Prover 2 is give y j wih (x j, y j ) chose from (X, Y ) he Prover is able o sample values {x i } i j ad Prover 2 is able o sample values {y i } i j such ha he joi disribuio of x,..., x, y,..., y is saisically close o he disribuio of hese values codiioed o he firs k games succeedig (i.e., codiioed o he eve W W k ). Therefore, hey will be able o wi he sigle game wih probabiliy close o Pr[W j W W k ]. This is from ow o our focus: how ca he wo provers perform his samplig. Easy example: if he provers messages i oe isace do o deped o he quesios of aoher isace he disribuio of {x i } i j ad {y i } i j is sill idepede of (x j, y j ). Hece, he provers ca sample from i usig shared radomess. Two useful lemmas: The followig wo lemmas will be crucial o he proof. Lemma : Le U = U,..., U be a produc disribuio ad le Ũ = Ũ,..., Ũ be he disribuio of U codiioed o some eve ha happes wih probabiliy a leas 2 d. The j (U j, Ũi) d/. We ll use Lemma o argue ha whe d, mos idices j saisfy ha (U j, Ũj) is small. Proof: We prove he lemma for he case ha for all i, U i is he uiform disribuio o {0, } l for some l (his is wihou loss of geeraliy sice we ca map {0, } l o arbirary (X, Y ) deoes he saisical disace of X ad Y : (X, Y ) = /2 z Pr[X = z] Pr[Y = z]. 2
disribuio wihi 2 l accuracy). Le H() deoe he Shao eropy fucio. The H(U) = l ad H(Ũ) H(U) d. Bu, sice H(Ũj) H(Ũ) j= we ge ha H(Ũj) l d/ j= Thus if we le δ i = l H(Ũj) he δ i d/ (2) j= The followig fac is lef as Exercise 2: If X is a disribuio over {0, } l wih H(X) l δ he (X, U l ) 2 δ. Usig i, (2) implies which usig E[X 2 ] (E[X]) 2 implies (Ũj, U j ) 2 d/ j= j= ad akig square roos complees he proof. (Ũj, U j ) 2 d/ We ll also use a versio of Lemma where here migh be a addiioal variable T ha is correlaed wih U,..., U : Lemma : Le U = U,..., U ad T be correlaed r.v. s such ha for every Supp(T ), U is a produc disribuio ad le Ũ be he disribuio of U codiioed o some eve W ha happes wih probabiliy a leas 2 d. The, j ( T U j T, T Ũj) d/, where T = T W. Explaaio of erms i Lemma : The disribuio T Ũj is defied as follows: choose (, u... u ) from he correlaed radom variables T, U codiioig o W, ad oupu u j. The disribuio T U j T is defied as follows: choose from T codiioed o W, he choose u... u from U codiioed o T = bu o o W. The oupu u j. 3
Proof of Lemma The proof follows by applyig Jese s iequaliy o Lemma. (U j T, Ũj T ) = Pr[T = W ] Pr[U j = u T = ] Pr[Ũj = u T = = j j u Pr[T = W ] (Ũj T =, U j T = ) by Lemma j Pr[T = W ] log(/ Pr[W T =]) log ( Pr[T = W ] Pr[W T =] ) where he las iequaliy follows from he fac ha E[f(x)] f(e[x]) for a cocave fucio f, ad x log x is cocave for x >. Bu Pr[T = W ] Pr[T =] Pr[W T =] = Pr[W ] ad hece his sum is equal o log ( Pr[W ] ) Pr[T = ] = log(/ Pr[W ]) Lemma 2: There is a mehod for wo provers wih shared radomess o perform he followig: Prover is give a specificaio of a disribuio D ad Prover 2 a specificaio of D, where (D, D ) ɛ. The, leig d ad d be he oupus of Prover ad Prover 2 respecively, () he disribuio of d is wihi ɛ saisical disace o D ad (2) Pr[d = d ] 2ɛ. Proof: Firs, i s worhwhile o oe ha he obvious procedure o his whe D D (choose a radom p [0, ] ad oupu he firs i such ha j i Pr[D j] p) compleely breaks dow if D is eve slighly differe ha D. Cosider he case where D ad D are fla disribuios. I his case, we ca hik of hem as ses wih symmeric differece a mos 2ɛ compared o heir size. The provers use heir shared radomess o ake a radom orderig of he uiverse. Prover will oupu he miimal eleme i D accordig o his orderig, ad Prover 2 will oupu he miimal eleme i D accordig o his orderig. The probabiliy ha he miimal eleme falls iside he shared iersecio is a leas 2ɛ. This case is acually geeral, sice we ca make D ad D fla by cosiderig he disribuio (x, p) over all pairs (x, p) such ha x Supp(D) ad p [0, Pr[D = x]]. δ 2 log Σ 0 6 Proof of he Mai Lemma: le k (where Σ is he alphabe used i he provers resposes) ad assume () is false for some prover sraegies P, P 2. Le s fix x = (x,..., x k ), ȳ = (y,..., y k ), ā = (a,..., a k ), b = (b,..., b k ) o be a ypical wiig rascrip of he firs k rouds (i.e., a rascrip ha causes he verifier o accep all hese k rouds). By ypical we mea he followig: 4
Codiioed o he queries ad aswers fallig i his rascrip, 90% of he coordiaes j saisfy ha W j wih probabiliy a leas δ/2. Sice we assume () is false, his will happe wih a leas 0.9 probabiliy. Codiioed o he verifier s firs k queries beig x, ȳ, he probabiliy ha he wo provers will aswer wih ā, b is a leas Σ 4k, where Σ is he alphabe used i he provers resposes. (Because he provers resposes i he firs k rouds may deped o he verifier s queries i differe rouds, his probabiliy is over he choice of (x k+, y k+ ),..., (x, y ).) Ideed, because ha here are a mos Σ 2k possible aswers, wih 0.9 probabiliy, a radom wiig rascrip will saisfy ha he aswers ā, b are obaied wih probabiliy a leas Pr[W W k ]/(00 Σ 2k ), ad we assume ha Pr[W W k ] Σ k. Defie he disribuio (X, Y ),..., (X, Y ) as follows: for i k, X i = x i ad Y i = y i wih probabiliy, ad for i > k, (X i, Y i ) is disribued idepedely accordig o (X, Y ). Noe ha ha is a produc disribuio (each pair (X i, Y i ) is idepede of he oher pairs). Le W deoe he eve ha he wo provers resposes are ā ad b respecively, ad le ( X, Ỹ),..., ( X, Ỹ) be he disribuio of (X, Y ),..., (X, Y ) codiioed o W. Noe ha his is o loger ecessarily a produc disribuio. Defiiio of T : We ow make he followig crucial defiiio of a radom variable T ha is correlaed wih (X, Y ),..., (X, Y ). Give a sequece of values, x, y,..., x, y, for every j > k, we defie he radom variable T j correlaed wih (x j, y j ) as follows: wih probabiliy /2 we se T j = ( x, x j ) ad wih probabiliy /2 we se T j = ( y, y j ). We defie he variable T o be he cocaeaio of T j for all j > k ad he variable T \j o be he cocaeaio of T i for i j. We deoe T j o be he disribuio of T j codiioed o W, ad defie similarly T, T \j. The lemma will follow from he followig claim: Claim: Le ɛ = δ/0. The for 80% of he coordiaes j > k: ( X j, Ỹj) ɛ (X, Y ) (3) T \j X j, Ỹj ɛ T\j X j (4) T \j X j, Ỹj ɛ T\j Ỹj (5) The magic of he proof (i boh Raz ad Holesei s paper) is i his claim. (4) ad (5) deserve some explaaio. For example (4) meas ha if we choose ( x j, ỹ j ) from ( X j, Ỹj), he he expeced saisical disace of he followig wo disribuios is a mos ɛ: Choose x i, ỹ i for i j accordig o X, Ỹ,..., X, Ỹ X j = x j, Ỹk = ỹ j. Choose from T \j codiioed o x, ỹ,..., x, ỹ. Oupu. Choose y j codiioed o x j oly (i.e. y j does o ecessarily equal ỹ j ). Choose x i, ỹ i for i j accordig o X, Ỹ,..., X, Ỹ X j = x j, Ỹk = y j. Choose from T \j codiioed o x, ỹ,..., x j, y j,..., x, ỹ. Oupu. Proof of Lemma from Claim Choose j a radom ad fix i. followig algorihm: The wo provers will use he 5
. Verifier chooses (x, y) from (X, Y ). Prover ges x ad ses x j = x ad Prover 2 ges y ad ses y j = y. 2. Boh provers se x, y,..., x k, y k accordig o x, ȳ. 3. Prover compues he disribuio D = T \j x j ad Prover 2 compues he disribuio D = T \j y j. Due o (4) ad 5 we kow hese wo disribuios are close o oe aoher ad o T \j x j, y j. They boh use he procedure of Lemma 2 o sample he same value from his disribuio. 4. Recall ha coais for every i > k wih i j a pair (d i, z i ) where d i {, 2}. Le S be he se of i s such ha d i = x, ad S he ses of i s such ha d i = y. Boh provers se x i = z i for i S ad y i = z i for i S. A his poi for every i, each prover has a value for a leas oe of x i or y i. 5. Prover chooses x i for all i S i he followig way: i chooses x i accordig o he disribuio X Y = y i, ad he i codiios o he firs k values of P (x,..., x ) beig ā (i.e., if his codiio does hold he prover makes all choices agai). Similarly for all i S, Prover 2 chooses y i codiioed o x i ad he codiios o he firs k values of P 2 (y,..., y ) beig b. 6. Prover s aswer is he j h value of P (x,..., x ). Similarly, Prover 2 s aswer is he j h value of P 2 (x,..., x ). The key observaio is ha if (3),(4), ad (4) held wih ɛ = 0 he he values x k+,..., x,y k+,..., y would be exacly disribued accordig o he disribuio X k+,..., X, Ỹk+,..., Ỹ, ad so he provers will covice he verifier wih probabiliy δ/2. Now, sice hey hold approximaely, hey will sill covice he verifier wih probabiliy a leas δ/2 5ɛ > δ, leadig o a coradicio. Proof of Claim We ow prove he claim. By Lemma for 90% of he coordiaes j he saisical disace of X j, Ỹj from he origial disribuio (X, Y ) is a mos k/ = δ/00 = ɛ/0 ad hece (3) is saisfied. By Lemma, for 90% of he j s, he saisical disace of T, X j, Ỹj from T, (X j, Y j ) T is also a mos ɛ/0. Sice T is equal o T \j ( x ) X j wih probabiliy /2 ad T \j ( y )Ỹj wih probabiliy /2, implyig ha /2 ( T \j Xj Ỹ j, T \j Xj Y j X j ) + /2 ( T \j Xj Ỹ j, T \j X j Ỹj Ỹj) ɛ/0 T \j Xj Ỹ j ɛ/5 T\j Xj Y j X j (6) T \j Xj Ỹ j ɛ/5 T\j X j Ỹj Ỹj (7) Ye (6) ad (7) imply (4) ad (5) respecively. (6) implies (4) sice if (4) was false here would be a disiguisher ha o ipu x, y ha are seleced from X j, Ỹj ad a hird ipu maages o ell apar if is seleced from T \j x or is seleced from T \j x, y. 2 Bu a equivale way o describe his is ha he disiguisher is give x, y, ha eiher come from X j, Ỹj, T \j, or y is oly chose codiioed o x ad igorig (we use here he fac ha by (3) choosig y accordig o Y X = x is he same as choosig y accordig o Ỹj X j = x). Such a disiguisher violaes (6). The proof ha (7) implies (5) is symmeric. 2 We use he fac ha we ca always flip he oupu of a disiguisher violaig (4) o esure ha i s more likely o oupu o he lef had side disribuio. 6
Homework Assigmes (25 pois) (a) Give he PCP Theorem as a black-box (NP PCP(O(log), O())) prove ha for ay laguage i NP here exiss a wo query PCP proof sysem where verifier uses O(log ) radomess ad he prover s aswers are i a alphabe of cosa size, wih perfec compleeess, ad soudess parameer a mos ρ for some cosa ρ <. (By soudess parameer we mea he maximum probabiliy of accepig a false saeme.) (b) Usig he Parallel Repeiio Theorem, show ha for every NP-laguage ad ɛ > 0, here exiss such a sysem wih soudess parameer a mos ɛ. 2 (30 pois) Recall ha he eropy of a disribuio (p,..., p N ) is defied o be i p i log(/p i ). Prove ha for every δ (0, ), if X is a disribuio over {0, } l wih H(X) l δ he (X, U) 2 00δ (his is rue eve if he cosa 00 is replaced by, bu his versio suffices for he proof of he parallel repeiio heorem). See foooe for hi 3 3 (25 pois) Complee he proof of Lemma, by showig ha he case of geeral disribuios reduces o he case of he uiform disribuio. 4 (30 pois) Wrie dow he full proof of he Mai Lemma from he claim: (a) Prove ha if (3),(4) ad (5) held wih ɛ = 0, he he disribuio of values x,..., x, y,..., y he wo provers pick is exacly equal o he disribuio X,..., X, Ỹ,..., Ỹ. (b) Prove ha geerally, usig (3),(4) ad (5), he disribuio of values x,..., x, y,..., y he wo provers pick is wihi 0ɛ saisical disace o he disribuio X,..., X, Ỹ,..., Ỹ. 3 Hi: If p i is he probabiliy ha X = i, he wrie p i = 2 l ( + x i ) ad use he esimae log( + x) x for small x s. 7