COMP760, SUMMARY OF LECTURE 16.

Transcription

1 COMP760, SUMMARY OF LECTURE 16. HAMED HATAMI 1. Direct product ad direct sum theorems Oe class of the most iterestig ad fudametal questios of theoretical computer sciece are the closely related direct product ad direct sum questios. A direct product theorem i a particular computatioal model asserts that the probability of success of performig idepedet copies of a computatioal task decreases i. For example, a direct product theorem might say that if f is computable with a give amout of resources (e.g. computatioal, commuicatioal, space, etc) with success probability at least 1 ɛ (e.g. o every iput, or o a radom iput, etc), the give the same amout of resources, oe ca oly compute the fuctio f : (x 1,..., x ) (f(x 1 ),..., f(x )) with success probability at most δ where δ is much smaller tha 1 ɛ. Note that this is harder tha performig copies of the origial task as here we wat all the coordiates to be correct at the same time. Two fudametal results i computer sciece, Yao s XOR Lemma [Yao82] ad Raz s Parallel Repetitio Theorem [Raz98] are examples of direct product theorems. Sometimes to stay at the Boolea regime, istead of f, oe cosiders the Boolea fuctio f : (x 1,..., x ) f(x 1 )... f(x ). It is i geeral expected that this fuctio is almost as difficult as f. Ideed to have a good estimate for the parity f(x 1 )... f(x ) oe has to have good estimates for all the values f(x 1 ),..., f(x ) as chagig ay oe of them will completely chage the value of the parity. However oe has to be slightly careful as every Boolea fuctio ca be computed with success probability 1/2 i a obvious maer by just flippig a radom coi for each iput. Hece i this cotext, if the origial fuctio f ca be computed with success rate ɛ with the give resources, a direct product theorem would state that f ca be computed with the same amout of resources oly with success probability at most ɛ where ɛ is much smaller tha ɛ. O the other had the direct sum questio fixes the success probability, ad asks how much resources oe does eed to solve f compared to the amout of resources eeded to solve f (with the same success probability). I other words, a direct sum theorem asserts that the amout of resources eeded to perform idepedet copies of the same task grows with. While the direct sum questio for geeral models such as Boolea circuits has a log history, o geeral results are kow, ad ideed they caot be achieved by the stadard reductios used i complexity theory. Ideed if we start by assumig that there is a circuit C that performs idepedet copies of the task, the it is impossible to use this C as a blackbox to show that there is a much smaller circuit C that solves oe copy of the task. Icludig C as a blackbox will already make the size of the ew circuit larger. Ideed it is kow that at least the most straightforward ad optimistic formulatio of a direct sum theorem for Boolea circuits is false. The example comes from fast matrix multiplicatio. By a coutig argumet, there exists a matrix A over GF(2) such that the map x Ax requires a circuit of Ω( 2 / log ) size. But the map (x 1,..., x ) (Ax 1,..., Ax ) is just the product of the two matrices A ad X (whose colums are x 1,..., x ) ad hece ca be carried out by a circuit of size O( 2.38 ) 2 / log. Ulike i circuit complexity ad commuicatio complexity, i iformatio complexity oe ca use a protocol as a blackbox ad still be able to 1

2 2 HAMED HATAMI obtai a protocol with a much smaller iformatio cost. We will this i the proof of Theorem 1 below. Ideed this great advatage will be used i may applicatios of iformatio complexity to commuicatio complexity. I the cotext of commuicatio complexity, the direct sum questio was first raised by Karchmer, Raz, ad Wigderso [KRW95] who cojectured a certai direct sum statemet for a certai determiistic commuicatio problem. They showed that this cojectured direct sum statemet would separate the two complexity classes NC 1 ad NC 2, ad i particular show that NC 1 P. Recall that NC i is the class of problems that ca be solved by a uiform family of Boolea circuits, with polyomial size, depth O(log i ()), ad fa-i 2. This would be a icredible result as log() seems to be a serious barrier to almost all kow techiques for provig geeral circuit lower-bouds. We will discuss this problem later i the course whe we talk about the applicatios of commuicatio complexity to circuit complexity. Feder, Kushilevitz, Naor, ad Nisa [FKNN95] gave a direct sum theorem for o-determiistic commuicatio complexity, ad deduced from it a somewhat weaker result for determiistic commuicatio complexity: if a sigle copy of a fuctio f requires C bits of commuicatios, the copies require Ω( C) bits. I other words, D(f ) lim = Ω( D(f)). Feder et al also cosidered the direct sum questio for radomized commuicatio complexity ad showed that the depedece of the commuicatio o the error of the protocol for may copies ca be better tha what ca be obtaied from the aive protocol for may copies. Shaltiel [Sha03] gave a direct sum/product theorem for the discrepacy of a fuctio, which shows a direct sum/product theorem for the radomized commuicatio complexity of the fuctio if discrepacy lower-boud matches the commuicatio complexity of the fuctio. More precisely he showed that disc(f ) = disc(f) Ω(). This shows that if a fuctio f is proved to be hard o average for c-bit commuicatio protocols via the discrepacy method, the f is expoetially harder o average for Ω(c)-bit commuicatio protocols. There are subsequet works o the direct sum questio for radomized commuicatio complexity ad the distributioal commuicatio complexity. May of these works are based o studyig the exteral iformatio cost. Barak, Braverma, Che, ad Rao [BBCR10] defied the iteral iformatio cost, ad used it to prove a strog lower-boud o the direct sum problem for the distributioal commuicatio complexity. First let us see how iformatio complexity behaves with respect to performig multiple copies of the same task. Theorem 1 (Additivity of iformatio cost). For every commuicatio task T, we have IC µ (T ) = IC µ (T ). Proof. It is obvious that IC µ (T ) IC µ (T ). Ideed give ay protocol π for T, we ca defie a protocol for T by ruig π o each coordiate. Obviously the iformatio cost of this ew protocol is IC µ (T ) ad it performs T. The opposite directios is more iterestig. Cosider a protocol π for T how ca we use π as a blackbox to obtai a protocol that performs T but has iformatio cost ICµ(π)? We defie the protocol τ i the followig maer. O iput (x, y), Alice ad Bob Publicly choose i {1,..., } uiformly at radom, ad set X i = x ad Y i = y idividually.

3 COMP760, SUMMARY OF LECTURE X 1,..., X i 1 idepedetly, each accordig to µ x. Y i+1,..., Y idepedetly, each accordig to µ y. Alice privately chooses X i+1,..., X accordig to µ coditioed o the values of Y i+1,..., Y. Bob privately chooses Y 1,..., Y i 1 accordig to µ coditioed o the values of X 1,..., X i 1. They ru the protocol π o (X, Y ) with X = (X 1,..., X ) ad Y = (Y 1,..., Y ), ad output the i-th coordiate. So the above protocol chooses X ad Y i the followig maer so that the iput (x, y) is embedded i the i-th coordiate ad furthermore each (X i, Y i ) is chose idepedetly with distributio µ: X = (X 1,..., X }{{ i 1, x, X } i+1,..., X ) ad Y = (Y }{{} 1,..., Y i 1, y, X }{{} i+1,..., X ). }{{} pub private private public This protocol τ obviously performs the task T, as it was assumed that π performs T. It remais to show that the iformatio cost of τ equals IC µ (π)/. I(y; Π τ x) = I(y; Π π i, X, Y i+1,..., Y ) (coditioig o what Alice kows) = 1 I(Y i ; Π π X, Y i+1,..., Y, [i = j]) (expadig the coditioig o i) = 1 j=1 I(Y j ; Π π X, Y j+1,..., Y ) j=1 = 1 I(Y ; Π π X) (chai rule) Note that sice X i+1,..., X are determied by Alice s private radomess ad i ad X 1,..., X i 1, ad Y i+1,..., Y are determied by public radomess, the first equality basically correspods to the geeral idetity I(Y ; Π X) = I(Y ; Π X, R A, R) which follows from what we saw o lecture 14. It follows that IC µ (τ) = IC µ (π)/. Note that although τ uses π as a blockbox, it evertheless has smaller iformatio cost. Cotrast this with commuicatio protocols, or circuits. I the first glace the way that the chai rule worked out i the above proof might look rather magical, but ideed the way we costructed X ad Y are based o reversed egieerig of the chai rule I(Y j ; Π π X, Y j+1,..., Y ) = I(Y ; Π π X). j=1

4 4 HAMED HATAMI 2. Compressio Give a task T ad a protocol π for T, Theorem 1 allows oe to obtai a protocol for T with the much smaller iformatio cost of IC µ (π)/. Note however that this protocol s commuicatio complexity equals to that of π ad thus ca be very large. Hece it is desirable to use the small iformatio cost to compress this protocol to a ew protocol with small commuicatio cost. Ideed compressio is oe of the key cocepts that allows oe to apply iformatio complexity to commuicatio complexity. The followig table summarizes the curret kow results about compressio. Table 1. Compressio results: I = IC µ (π), I ext = IC ext µ (π), C = CC(π), ad k is the umber of rouds. referece worst case CC of the compressed protocol i O( ) i O ɛ ( ) [BBCR10] IC log(c/ɛ)/ɛ IC log(c) 1 [BR14] ɛ (I + k log(k) + ki) I + k log(k) + ki [Bra12] 2 20I/ɛ 2 20I/ɛ [BBCR10] I ext poly log(c/ɛ) I ext poly log(c) We will prove the first three of these results i this course. But before delvig ito the proofs let us metio some beautiful cosequeces of these compressio results Iformatio equals amortized commuicatio. I Table 1 we metioed that Braverma ad Rao [BR14] proved that oe ca compress a protocol π with k rouds to a protocol τ with worst case commuicatio complexity O ɛ (I + k log(k) + ki) such that the simulatio fails with probability at most ɛ. I fact what they prove is stroger. Let us assume for a momet that it is possible to do compressio with (1) I + O(k log(k/ɛ) + ki) bits of commuicatio. While this is ot true, it is also ot far off from the truth. Assumig this, we will give a pseudo-proof for the followig theorem, ad the we will remark how oe ca tur that ito a actual proof usig what is actually true. Theorem 2 (Iformatio equals amortized commuicatio). Let f : X Y Z, ad let µ be a probability distributio o X Y. For every ɛ > 0, we have IC µ (f, µ, ɛ) = lim CC([f, µ, ɛ] ). Semi-proof. Oe directio is obvious from Theorem 1. Ideed CC([f, µ, ɛ] ) IC µ ([f, µ, ɛ] ) = IC µ (f, µ, ɛ). For the other directio, cosider a protocol π that performs [f, µ, ɛ]. We shall apply compressio, ad sice compressio adds error, we have to first decrease the error i π. Ideed by cotiuity of iformatio there is a protocol π with iformatio cost IC µ (π ) IC µ (π) + o δ 0 (1) that performs [f, µ, ɛ δ]. Let k deote the umber of rouds of π, ad let τ be the protocol that performs [f, µ, ɛ δ] by ruig π o each coordiate separately. The IC µ (τ) = IC µ (π ),

5 COMP760, SUMMARY OF LECTURE while the umber of rouds of τ still equals to the umber of rouds of π. Ideed although we are ruig π o each coordiate separately, we ca still budle all the messages at each roud together ad sed them as oe log message. The rest of this semi-proof has a bit of cheatig. Usig the (ot quit right) assumptio of (1), we ca compress τ to aother protocol τ with CC(τ ) = IC µ (π ) + O(k log(k/δ) + kic µ (π )), by addig at most δ error. Hece the compressed protocol will perform [f, µ, ɛ], ad thus CC([f, µ, ɛ] ) IC µ (π ) + O(k log(k/δ) + kic µ (π )) lim lim lim = lim IC µ (π ) = IC µ (π). δ 0 δ 0 Takig the ifimum o the right had side will prove the result. Remark 3. The reaso that the above proof is ot a actual proof is that oe caot i geeral compress a protocol to (1) may bits ad add oly a error of ɛ. However Braverma ad Rao [BR14] showed that there is a compressio such that if thigs go right (call it the evet E), the the compressio perfectly simulates the origial protocol, ad furthermore will have expected commuicatio I +O(k log(k/ɛ)+ ki), ad furthermore E happes (simulatio makes o mistake) with probability at least 1 ɛ. Now i the above semi-proof, sice this compressio is applied to may copies, it is possible to apply a argumet based o the cetral limit theorem to show that ideed the expected commuicatio will be highly cocetrated aroud its expected value, ad thus we ca trucate the protocol by termiatig it after I +O(k log(k/ɛ)+ ki) bits of commuicatio to obtai a compressed protocol with the worst case commuicatio I + O(k log(k/ɛ) + ki), ad this will fix the above proof The direct sum theorem. Note that CC(f, µ, ɛ) = 0 for f : X Y Z if ad oly if there exists z Z such that µ(f 1 (z)) 1 ɛ. Theorem 4 (A direct sum theorem). Let f : X Y Z, ad let µ be a probability distributio o X Y. For every ɛ > 0, if CC(f, µ, ɛ) > 0, the for sufficietly large, CC([f, µ, ɛ] ) log 2 (), ad i particular D ɛ µ (f ) log 2 (). Proof. Sice CC(f, µ, ɛ) > 0, there exists δ > 0 such that CC(f, µ, ɛ + δ) > 0 too. Let π be the protocol that performs the task [f, µ, ɛ] with the smallest commuicatio cost. Obviously CC(π) CC(f, µ, ɛ). Theorem 1 shows that oe ca use π to obtai a protocol τ for the task [f, µ, ɛ] with iformatio cost ad commuicatio cost IC µ (τ) = IC µ(π) CC µ(π), CC(τ) = CC(π) CC(f, µ, ɛ).

6 6 HAMED HATAMI Now τ has a very small iformatio cost, ad we ca use the compressio of [BBCR10] to compress it to a protocol τ with commuicatio cost ( CC(τ ) = O CC(τ) IC µ (τ) log(cc(τ)/δ) ) = CC(π) ( ) log CC(π)/δ) O = CC(π) δ δ δ O(log ), ad error ɛ + δ. Hece which shows for sufficietly large. 0 < CC(f, µ, ɛ + δ) CC(π) δ O(log ), ( ) δ CC(f, µ, ɛ + δ) CC(π) = Ω log log 2, I the above theorem, the term 1/ log 2 () must be igored as a mior logarithmic fact. Thus we ca write CC([f, µ, ɛ] ) CC(f, µ, ɛ), where the upper-boud is obvious. How far ca oe push the lower-boud? It is kow that it caot be pushed all the way up to Ω(CC(f, µ, ɛ)). Ideed if Ω()CC(f, µ, ɛ) CC([f, µ, ɛ] ), the by dividig both sides by, ad takig the limit, we would get (usig iformatio equals amortized commuicatio) that Ω(CC(f, µ, ɛ)) IC µ (f, µ, ɛ), ad this is ot true i geeral as recetly Gaor, Kol, ad Raz [GKR14] have show that CC(f, µ, ɛ) ca be expoetially larger tha IC µ (f, µ, ɛ). However their result does ot eve overrule the possibility that (log CC(f, µ, ɛ)) CC([f, µ, ɛ] ). Problem 5 (Ope problem). Is it possible to improve the lower-boud i CC([f, µ, ɛ] ) CC(f, µ, ɛ)? Whe µ is a product measure, the iformatio cost ad the exteral iformatio cost are equal ad hece the compressio of [BBCR10] with respect to the exteral iformatio cost provides a much stroger direct sum theorem. Theorem 6. Let f : X Y Z, ad let µ be a product probability distributio o X Y. For every ɛ > 0, if CC(f, µ, ɛ) > 0, the for sufficietly large, log Ω(1) () CC([f, µ, ɛ] ). I particular log Ω(1) () Dµ ɛ (f ). Refereces [BBCR10] Boaz Barak, Mark Braverma, Xi Che, ad Aup Rao, How to compress iteractive commuicatio [exteded abstract], STOC 10 Proceedigs of the 2010 ACM Iteratioal Symposium o Theory of Computig, ACM, New York, 2010, pp MR [BR14] Mark Braverma ad Aup Rao, Iformatio equals amortized commuicatio, IEEE Tras. Iform. Theory 60 (2014), o. 10, MR [Bra12] Mark Braverma, Iteractive iformatio complexity, STOC 12 Proceedigs of the 2012 ACM Symposium o Theory of Computig, ACM, New York, 2012, pp MR

7 COMP760, SUMMARY OF LECTURE [FKNN95] Tomás Feder, Eyal Kushilevitz, Moi Naor, ad Noam Nisa, Amortized commuicatio complexity, SIAM J. Comput. 24 (1995), o. 4, MR (96j:68089) [GKR14] Aat Gaor, Gillat Kol, ad Ra Raz, Expoetial separatio of iformatio ad commuicatio, 55th IEEE Aual Symposium o Foudatios of Computer Sciece, FOCS 2014, Philadelphia, PA, USA, October 18-21, 2014, 2014, pp [KRW95] Mauricio Karchmer, Ra Raz, ad Avi Wigderso, Super-logarithmic depth lower bouds via the direct sum i commuicatio complexity, Comput. Complexity 5 (1995), o. 3-4, MR (97e:68034) [Raz98] Ra Raz, A parallel repetitio theorem, SIAM J. Comput. 27 (1998), o. 3, (electroic). MR (2000c:68057) [Sha03] Roe Shaltiel, Towards provig strog direct product theorems, Comput. Complexity 12 (2003), o. 1-2, MR (2005a:68091) [Yao82] Adrew C. Yao, Theory ad applicatios of trapdoor fuctios, 23rd aual symposium o foudatios of computer sciece (Chicago, Ill., 1982), IEEE, New York, 1982, pp MR School of Computer Sciece, McGill Uiversity, Motréal, Caada address: hatami@cs.mcgill.ca