Communication vs information complexity, relative discrepancy and other lower bounds

Transcription

1 Communication vs information complexity, relative discrepancy and other lower bounds Iordanis Kerenidis CNRS, LIAFA- Univ Paris Diderot 7 Joint work with: L. Fontes, R. Jain, S. Laplante, M. Lauriere, J. Roland

2 Communication complexity x {0,1} n y {0,1} n Alice and Bob want to compute f(x, y) Minimal communication needed? Trivial protocol with communication n Initiated by [Yao 79] Wide application to circuit complexity, VLSI, streaming, distributed computing, data structures, etc.

3 Measuring complexity of communication Basic measure: number of bits transmitted But bits may be useless Example: EQUALITY M = random invertible n x n matrix x {0,1} n (Mx)1 y {0,1} n (Mx)2 If (Mx)i (My)i then output 0...

4 Measuring complexity of communication Basic measure: number of bits transmitted But bits may be useless Example: EQUALITY M = random invertible n x n matrix x {0,1} n (Mx)1 y {0,1} n (Mx)2 If (Mx)i (My)i then output 0... Many bits communicated, Ω(n) (on avg) BUT, Bob learns O(1) bits about x Question: how to measure amount of information transmitted?

5 Outline 1. Introducing Information Cost and Information Complexity 2. Relating Information to Communication Complexity 3. Zero- communication protocols and Information Complexity 4. Information complexity is bigger than Relative discrepancy

6 Entropy Entropy: H(X) x supp(x) Pr[X = x] log (1/Pr[X = x]) Uncertainty about X Conditional entropy: H(X Y) y supp(y) Pr[Y = y] H(X Y = y) Uncertainty about X knowing Y Mutual information: I(X ; Y) H(X) - H(X Y) "How much information does Y reveal about X" Conditional Mutual Information: I(X ; Y Z) H(X Z) - H(X YZ) "Already knowing Z, how much additional information does Y reveal about X"

7 Information complexity Public coins rpub Private coins ra x {0,1} n y {0,1} n Private coins rb Fix protocol π, a distribution μ over inputs (X, Y, Π) : sample (X, Y) μ, let Π be transcript of π(x, Y)

8 Information complexity Public coins rpub Private coins ra x {0,1} n y {0,1} n Private coins rb Fix protocol π, a distribution μ over inputs (X, Y, Π) : sample (X, Y) μ, let Π be transcript of π(x, Y) CC(π) = maxx, y, random coins π(x, y) ICμ(π) = I(X ; Π Y) + I(Y ; Π X) Information Alice learns about Bob s input from transcript + vice versa

9 Information complexity Public coins rpub Private coins ra x {0,1} n y {0,1} n Private coins rb Fix protocol π, a distribution μ over inputs (X, Y, Π) : sample (X, Y) μ, let Π be transcript of π(x, Y) CC(π) = maxx, y, random coins π(x, y) ICμ(π) = I(X ; Π Y) + I(Y ; Π X) Information Alice learns about Bob s input from transcript + vice versa π is (μ,ε)- good for f if Pr(X,Y) μ[π(x,y) = f(x,y)] 1 ε Rμ,ε(f) = infπ (μ,ε)- good for f CC(π) Rε(f) = supμ Rμ,ε(f) ICμ,ε(f) = infπ (μ,ε)- good for f ICμ(π) ICε(f) = supμ ICμ,ε(f)

10 Information vs Communication as a Compression question Source Coding Theorem [Shannon 49]: Given a source X with entropy H(X), can encode X using H(X)+1 bits on avg Huffman coding. Asymptotically, can use H(X)bits Non- interactive One- way communication: Information C = Communication C x Encoding(x)

11 Information vs Communication as a Compression question Source Coding Theorem [Shannon 49]: Given a source X with entropy H(X), can encode X using H(X)+1 bits on avg Huffman coding. Asymptotically, can use H(X)bits Non- interactive One- way communication: Information C = Communication C x Encoding(x) What about interaction?

12 Information vs. Communication Complexity How close are IC and CC? Can I compress an interaction? Conjecture 1: for any ε, μ, f, it holds that Rμ,ε(f) = Ο(ICμ,ε(f)) Conjecture 2: for any ε, f, it holds that Rε(f) = Ο(ICε(f))

13 Information vs. Communication Complexity How close are IC and CC? Can I compress an interaction? Conjecture 1: for any ε, μ, f, it holds that Rμ,ε(f) = Ο(ICμ,ε(f)) Conjecture 2: for any ε, f, it holds that Rε(f) = Ο(ICε(f)) Main application: Conjecture 1 (or 2) implies Direct Sum for CC Rμ,ε(f k ) ICμ,ε(f k ) = k * ICμ,ε(f) Ω( k * Rμ,ε(f) ) [Braverman11]

14 Information vs. Communication Complexity How close are IC and CC? Can I compress an interaction? Conjecture 1: for any ε, μ, f, it holds that Rμ,ε(f) = Ο(ICμ,ε(f)) Conjecture 2: for any ε, f, it holds that Rε(f) = Ο(ICε(f)) Main application: Conjecture 1 (or 2) implies Direct Sum for CC Rμ,ε(f k ) ICμ,ε(f k ) = k * ICμ,ε(f) Ω( k * Rμ,ε(f) ) [Braverman11] [Braverman- Rao 11]: IC(f) = lim k R(f k ) / k [Braverman 12]: for any π, exists τ with CC(τ) 2 Ο(IC(π)) [Gavinsky, Lovett 14] logrank conj. for IC implies is for CC

15 partition relaxed partition Communication Complexity smooth rectangle factorization norm rectangle/corruption discrepancy

16 partition relaxed partition GapHamming [Chakrabarti- Regev 11, Sherstov 11] smooth rectangle factorization norm Disjointness (Quantum) [Sherstov 08] Communication Complexity VectorInSubspace [Klartag- Regev 11] Disjointness [Kalyanasundaram- Schnitger 87, Razborov 92] rectangle/corruption [Linial- Shraibman 09, Jain- Klauck 10, Laplante- Lerays- Roland 12] Inner Product discrepancy

17 Communication Complexity partition relaxed partition Information Complexity smooth rectangle factorization norm rectangle/corruption discrepancy

18 Communication Complexity partition relaxed partition Information Complexity smooth rectangle factorization norm rectangle/corruption [Braverman- Weinstein 11] discrepancy

19 Communication Complexity partition relaxed partition KLLRX 12 Information Complexity smooth rectangle factorization norm rectangle/corruption discrepancy

20 Main Theorem of KLLRX12 Theorem: for all ε, μ, f, it holds that ICμ,ε(f) Ω( log rprtμ,ε(f) ), i.e. IC subsumes relaxed partition Corollaries 1. All known CC lower bounds imply same lower bound on IC 2. All known problems with tight LBs satisfy Direct Sum Theorems Rμk,ε(f k ) ICμk,ε(f k ) k * ICμ,ε(f) = k * Rμ,ε(f) 3. Quantum 1- way communication exponentially smaller than IC VectorInSubspace problem [Klartag- Regev 11]: Ο(log n) vs. Ω(n 1/3 ) 4. ICμ,ε(GapHamming) = Ω(n) and Direct Sum [CR 11, Sherstov 11])

21 Zero Communication protocols Let P compute f with error ϵ using C bits of communication. Then there exists a zero communication protocol π which can abort st x,y π(x,y) does not abort with probability η = 2 - C Prob[π(x,y) f(x,y) non abort] ϵ

22 Zero Communication protocols Let P compute f with error ϵ using C bits of communication. Then there exists a zero communication protocol π which can abort st x,y π(x,y) does not abort with probability η = 2 - C Prob[π(x,y) f(x,y) non abort] ϵ

23 Zero Communication protocols Let P compute f with error ϵ using C bits of communication. Then there exists a zero communication protocol π which can abort st x,y π(x,y) does not abort with probability η = 2 - C Prob[π(x,y) f(x,y) non abort] ϵ

24 Zero Communication protocols Let P compute f with error ϵ using C bits of communication. Then there exists a zero communication protocol π which can abort st x,y π(x,y) does not abort with probability η = 2 - C Prob[π(x,y) f(x,y) non abort] ϵ

25 Partition bound and ZC protocols Zero communication protocol x,y π(x,y) does not abort with probability η (efficiency) Prob[π(x,y) f(x,y) non abort] ϵ Efficiency of f := ημ,ε(f) = supπ η(π)

26 Partition bound and ZC protocols Zero communication protocol x,y π(x,y) does not abort with probability η (efficiency) Prob[π(x,y) f(x,y) non abort] ϵ Efficiency of f := ημ,ε(f) = supπ η(π) Theorem: Rμ,ε(f) Ω( log 1/ημ,ε(f) )

27 Partition bound and ZC protocols Zero communication protocol x,y π(x,y) does not abort with probability η (efficiency) Prob[π(x,y) f(x,y) non abort] ϵ Efficiency of f := ημ,ε(f) = supπ η(π) Theorem: Rμ,ε(f) Ω( log 1/ημ,ε(f) ) Theorem [LLR12]: prtμ,ε(f) = 1/ημ,ε(f)

28 Relaxed Partition bound and ZC protocols Zero communication protocol x,y π(x,y) does not abort with probability η (rel. efficiency) Prob[π(x,y) f(x,y) non abort] ϵ Efficiency of f := ημ,ε(f) = supπ η(π)

29 Relaxed Partition bound and ZC protocols Zero communication protocol x,y π(x,y) does not abort with probability η (rel. efficiency) Prob[π(x,y) f(x,y) non abort] ϵ Efficiency of f := ημ,ε(f) = supπ η(π) Thm1 [KLLRX12]: rprtμ,ε(f) = 1/ημ,ε(f) Thm2 [KLLRX12]: For any protocol, a ZC protocol with rel. eff. 2 - IC

30 Compressing IC to Zero communication x {0,1} n rpub = (t 1, t 2, t 3,..., t K ), hash h y {0,1} n 1. Create set of individually accepted transcripts Using rpub, generate candidate transcripts t 1,..., t K (K 2 CC(π) + Ο(IC(π)) ) Alice & Bob each decide whether to accept each t i Conditioned on t i accepted, distribution of t i same as π(x,y) roughly 2 Ο(IC(π)) accepted transcripts by each one 2. Find common accepted transcript Using rpub, pick a hash function that gives 0 w.p. 2 - Ο(IC(π)) Alice and Bob: If there exists accepted transcript that hashes to 0, then output according to that, else Abort. Pr[not abort] 2 - Ο(IC(π))

31 Communication Complexity partition relaxed partition KLLRX 12 Information Complexity smooth rectangle factorization norm rectangle/corruption discrepancy

32 CC =? O(IC)

33 [GKR14] show exponential gap between IC and CC for a fixed distribution μ for partial Boolean function f Function Bursting noise function New Lower Bound Technique relative discrepancy f, μ IC μ (f, ϵ) CC μ (f, ϵ) CC =? O(IC) prt rk + rec CC rprt srect rdisc μ IC μ sdisc γ 2 disc

34 [GKR14] show exponential gap between IC and CC for a fixed distribution μ for partial Boolean function f Function Bursting noise function New Lower Bound Technique relative discrepancy f, μ IC μ (f, ϵ) CC μ (f, ϵ) CC =? O(IC) prt rk + rec CC rprt srect rdisc μ IC μ sdisc γ 2 Remarks IC μ =log log n vs. CC μ =log n disc

35 [GKR14] show exponential gap between IC and CC for a fixed distribution μ for partial Boolean function f Function Bursting noise function New Lower Bound Technique relative discrepancy f, μ IC μ (f, ϵ) CC μ (f, ϵ) CC =? O(IC) prt rk + rec CC rprt srect rdisc μ IC μ sdisc γ 2 Remarks IC μ =log log n vs. CC μ =log n Conjecture 1: for any ε, μ, f, Rμ,ε(f) = Ο(ICμ,ε(f)) disc

36 [GKR14] show exponential gap between IC and CC for a fixed distribution μ for partial Boolean function f Function Bursting noise function New Lower Bound Technique relative discrepancy f, μ IC μ (f, ϵ) CC μ (f, ϵ) CC =? O(IC) prt rk + rec CC rprt srect rdisc μ IC μ sdisc γ 2 Remarks IC μ =log log n vs. CC μ =log n disc New Conjecture 1: for any ε, μ, f, Rμ,ε(f) = Ο(ICμ,ε(f))*log n

37 Understanding Relative discrepancy Can relative discrepancy separate IC from CC irrespective of the distribution on the inputs? How does relative discrepancy compare to other known bounds? What methods do we know that can still separate IC from CC?

38 Part I: Non- distributional case CC prt IC rprt [This talk] rdisc

39 Partition bound Def [JK10]: (Dual LP formulation) For any boolean function f: prtε(f) = max{αxy},{βxy} β(x Y) εα(x Y) subject to β(r) α(r f 1 (z)) 1 R, z x,y α xy 0 Thm: CCε(f) log(prtε(f))

40 Relaxed Partition bound Def: (Dual LP formulation) For any boolean function f: rprtε(f) = max{αxy},{βxy} β(x Y) εα(x Y) subject to β(r) α(r f 1 (z)) 1 R, z x,y α xy 0 α xy β xy 0 Thm: CCε(f) log(prtε(f)) log(rprtε(f))

41 Relative Discrepancy Def: [GKR14] (NB: not LP) rdisc ε (f,μ) = supκ,δ,{ρxy} 1/δ(½ κ ε) subject to μ(r f 1 (z)) (½ κ)ρ(r) R,z s.t. ρ(r) δ ρ(x Y) = 1 ρxy 0 x,y 0 κ < ½, 0 < δ < 1 rdisc ε (f)= maxμ rdisc ε (f,μ)

42 Def: [GKR14] (NB: not LP) rdisc ε (f,μ) = supκ,δ,{ρxy} 1/δ(½ κ ε) subject to μ(r f 1 (z)) (½ κ)ρ(r) R,z s.t. ρ(r) δ ρ(x Y) = 1 ρxy 0 x,y 0 κ < ½, 0 < δ < 1 rdisc ε (f)= maxμ rdisc ε (f,μ) Relative Discrepancy rescale rectangle size

43 Relative discrepancy relaxed partition Consider any feasible solution for rdisc Apply change of variables to obtain a feasible solution to rprt with a higher objective value.

44 Relative discrepancy relaxed partition rdisc ε (f) = maxμ supκ,δ,{ρxy} 1/δ(½ κ ε) s.t. μ(r f 1 (z)) (½ κ)ρ(r) R,z s.t. ρ(r) δ ρ(x Y) = 1 ρxy 0 x,y 0 κ < ½, 0 < δ < 1

45 Relative discrepancy relaxed partition rdisc ε (f) = maxμ supκ,δ,{ρxy} 1/δ(½ κ ε) s.t. μ(r f 1 (z)) (½ κ)ρ(r) R,z s.t. ρ(r) δ ρ(x Y) = 1 ρxy 0 x,y 0 κ < ½, 0 < δ < 1 α β μ β ρ rprt ε (f) = max{αxy},{βxy} β(x Y) εα(x Y) s.t. β(r) α(r f 1 (z)) 1 R, z α 0 αxy βxy 0 x,y

46 Part I: Non- distributional case CC prt IC rprt rdisc Can relative discrepancy separate IC from CC irrespective of the distribution on the inputs? NO

47 Part II: Fixed μ prt ε (f; μ) = max α, {βxy} β(x Y) ε αμ(x Y) s.t. β(r) αμ(r f 1 (z)) 1 R, z α 0

48 Part II: Fixed μ prt ε (f; μ) = max α, {βxy} β(x Y) ε αμ(x Y) s.t. β(r) αμ(r f 1 (z)) 1 R, z α 0 non-distributional α β μ β ρ

49 Part II: Fixed μ prt ε (f; μ) = max α, {βxy} β(x Y) ε αμ(x Y) s.t. β(r) αμ(r f 1 (z)) 1 R, z α 0 non-distributional α β μ β ρ

50 Part II: Fixed μ prt ε (f; μ) = max α, {βxy} β(x Y) ε αμ(x Y) s.t. β(r) αμ(r f 1 (z)) 1 R, z α 0 non-distributional Distributional: exponential separation Non-distributional: collapse α β μ β ρ

51 Lower bounds with partitions pprt CC pprt [JLV14] : quadratically tight bound prt ardisc rprt IC ardisc [GKR14]: strong new bound rdisc Remark: both use partitions and not rectangles

52 ardisc vs pprt CC prt IC rprt rk + srect rec sdisc γ 2 disc

53 Information vs. Communication Complexity How close are IC and CC? Can I compress an interaction? NEW Conjecture 1: for any ε, μ, f, Rμ,ε(f) = Ο(ICμ,ε(f))*log n Conjecture 2: for any ε, f, Rε(f) = Ο(ICε(f))

54 Information vs. Communication Complexity How close are IC and CC? Can I compress an interaction? NEW Conjecture 1: for any ε, μ, f, Rμ,ε(f) = Ο(ICμ,ε(f))*log n Conjecture 2: for any ε, f, Rε(f) = Ο(ICε(f)) [GKR15]: f, IC(f, ϵ) CC(f, ϵ) Remarks: IC=log log n vs. CC=log n New Conjecture 2: for any ε, μ, f, Rμ,ε(f) = Ο(ICμ,ε(f))*log n

55 Summary LP formulation of distributional relative discrepancy : partition bound + nonnegativity constraint Non- distributional relative discrepancy is less than IC Partition bound is not a lower bound on IC for fixed distributions ardisc is quadratically tight for communication complexity

56 n vs log(n) separation of distributional IC and CC Give separation for wprt/pprt; prt/rprt Is the partition bound a lower bound on IC? Requires new compression techniques to obtain constant efficiency ZCP Is it tight for CC? Open Problems Recover a communication protocol from a ZCP

57 Thank you

58 Compressing CC to Zero communication x {0,1} n rpub (= transcript t) y {0,1} n Theorem: if π (μ,ε)- good for f using communication, then can build ZCP τ (μ,ε)- good for f with η(τ) 2 - CC(π) Proof: τ uses rpub to guess random transcript t Alice (resp. Bob) sees if t is consistent with x (resp. y) If so, output what π outputs, otherwise output Abort Pr[t not abort] 2 - CC(π) CCμ,ε(f) Ω( log 1/ημ,ε(f) )

59 Compressing IC to Zero communication x {0,1} n rpub = (t 1, t 2, t 3,..., t K ), hash h y {0,1} n Theorem: for all ε, μ, f, it holds that ICμ,ε(f) Ω( log 1/ημ,ε(f) )

60 Public coin partition bound Max efficiency of a zero- communication protocol : Players pick a partition at random; Play following a uniformly distributed rectangle in the partition, Abort if either input not in rectangle.

61 Adaptive relative discrepancy Def: [GKR14] (NB: not LP) ardisc ε (f,μ) = supκ,δ,{ρ P xy} 1/δ(½ κ ε) subject to μ(r f 1 (z)) (½ κ)ρ P (R) P, (R,z) P s.t. ρ P (R) δ ρ P (X Y) = 1 ρ P xy 0 x,y 0 κ < ½, 0 < δ < 1 ardisc ε (f)= maxμ ardisc ε (f,μ)