Lower Bound Techniques for Multiparty Communication Complexity

Transcription

1 Lower Bound Techniques for Multiparty Communication Complexity Qin Zhang Indiana University Bloomington Based on works with Jeff Phillips, Elad Verbin and David Woodruff 1-1

2 The multiparty number-in-hand (NIH) comm. A model for proving lower bounds x 1 = x 2 = x 3 = x k = They want to jointly compute f(x 1, x 2,..., x k ) Goal: minimize total bits of communication 2-1

3 The multiparty number-in-hand (NIH) comm. A model for proving lower bounds x 1 = x 2 = Blackboard (BB): One speaks, everyone hears. Message-Passing (MP): If x 1 talks to x 2, others cannot hear. x 3 = x k = They want to jointly compute f(x 1, x 2,..., x k ) Goal: minimize total bits of communication 2-2

4 The multiparty number-in-hand (NIH) comm. A model for proving lower bounds x 1 = x 2 = Blackboard (BB): One speaks, everyone hears. Message-Passing (MP): If x 1 talks to x 2, others cannot hear. x 3 = x k = Coordinator = C They want to jointly compute f(x 1, x 2,..., x k ) S 1 S 2 S 3 S k Goal: minimize total bits of communication 2-3

5 Previously, for multiparty NIH comm. model Well studied? YES and NO. 3-1

6 Previously, for multiparty NIH comm. model Well studied? YES and NO. Quite a few papers in BB model: [Alon et al. 96], [Bar-Yossef et al. 04],

7 Previously, for multiparty NIH comm. model Well studied? YES and NO. Quite a few papers in BB model: [Alon et al. 96], [Bar-Yossef et al. 04],... But, Almost nothing in MP model before

8 Previously, for multiparty NIH comm. model Well studied? YES and NO. Quite a few papers in BB model: [Alon et al. 96], [Bar-Yossef et al. 04],... But, Almost nothing in MP model before Too easy? 3-4

9 Previously, for multiparty NIH comm. model Well studied? YES and NO. Quite a few papers in BB model: [Alon et al. 96], [Bar-Yossef et al. 04],... But, Almost nothing in MP model before Too easy? Lack of motivations? 3-5

10 Too easy? Could be more difficult than BB, since in MP each player observes only a piece of protocol transcript. 4-1

11 Too easy? Could be more difficult than BB, since in MP each player observes only a piece of protocol transcript. We do not want to analyze a k-dimensional matrix directly. A 0 1 B A communication matrix of the 2-party AND function 4-2

12 Too easy? Could be more difficult than BB, since in MP each player observes only a piece of protocol transcript. We do not want to analyze a k-dimensional matrix directly. A 0 1 B A communication matrix of the 2-party AND function Even for the 2-party Gap-Hamming problem, where Alice has a n-bit vector and Bob has a n-bit vector, and they want to compute if the hamming distance of these two vectors is bigger than n/2 + n or less than n/2 n, it tooks researchers in this area about 10 years and dozens of papers (IW03, Woodruff04,..., Sherstov12) to fully understand its 2-dim communication matrix! 4-3

13 Lack of motivations? (Continous) Distributed Monitoring communication energy, bandwidth,... Sensor networks Network routers Data in the cloud 5-1

14 Lack of motivations? (cont.) Data Streams communication space Memory CPU 6-1

15 Lack of motivations? (cont.) Data Streams communication space Memory P 1 P 2 P k CPU 6-2

16 Lack of motivations? (cont.) Distributed Computation for Big Data communication bandwidth,... Input Output Map Shuffle Reduce The BSP model. E.g., Apache Giraph, Pregel. The MapReduce model. E.g., Hadoop, Google MapReduce. 7-1

17 Interesting problems Statistical problems Frequency moments F p F 0 : #distinct elements F 2 : size of self-join Heavy hitters Quantile Entropy

18 Interesting problems Statistical problems Graph problems Connectivity Frequency moments F p F 0 : #distinct elements F 2 : size of self-join Heavy hitters Quantile Entropy Bipartiteness Counting triangles Matching Minimum spanning tree

19 Interesting problems Statistical problems Graph problems Connectivity Frequency moments F p F 0 : #distinct elements F 2 : size of self-join Heavy hitters Quantile Entropy... Numerical linear algebra Bipartiteness Counting triangles Matching Minimum spanning tree... L p regression Low-rank approximation

20 Interesting problems Statistical problems Frequency moments F p F 0 : #distinct elements F 2 : size of self-join Heavy hitters Quantile Entropy... Numerical linear algebra Graph problems Connectivity Bipartiteness Counting triangles Matching Minimum spanning tree... L p regression Low-rank approximation... DB queries Conjuntive queries Distributed learning Classifiers... Geometry problems 8-4

21 This talk We will talk lower bounds. Most lower bounds match the upper bounds. Introduce two general techniques that work in both BB model and MP model for proving lower bounds, with examples. Conclude with several future directions. 9-1

22 Our new ideas and techniques Ideas 1. Reduce a k-player problem to a 2-player problem. 2. Decompose a complicated problem to several simpler problems 10-1

23 Our new ideas and techniques Ideas 1. Reduce a k-player problem to a 2-player problem. 2. Decompose a complicated problem to several simpler problems In fact, we are exploring the underlying patterns of communication matrices 10-2

24 Our new ideas and techniques Ideas 1. Reduce a k-player problem to a 2-player problem. 2. Decompose a complicated problem to several simpler problems In fact, we are exploring the underlying patterns of communication matrices Concrete techniques Symmetrization With Phillips and Verbin, 2012 Composition With Woodruff,

25 Our new ideas and techniques Ideas 1. Reduce a k-player problem to a 2-player problem. 2. Decompose a complicated problem to several simpler problems In fact, we are exploring the underlying patterns of communication matrices Concrete techniques 1. Symmetrization Example: Graph Connecitivy With Phillips and Verbin, Composition With Woodruff, 2012 Example: Distinct Elements 10-4

26 Communication complexity I choose not to introduce formal definitions/concepts of communication complexity. Just want to mention Yao s lemma: We will prove lower bounds for randomized protocols. To do that we will usually pick some input distribution, and then prove lower bounds for any deterministic protocol that succeeds w.pr 0.99 under that input distribution. 11-1

27 12-1 1st Technique: Symmetrization

28 Example: graph connectivity k sites each holds a set of edges of a graph Goal: compute whether the graph is connected. A trivial UB: O(nk log n) bits. We prove that this is tight up to a log factor. 13-1

29 Set disjointness (2-DISJ) X Y =? X {1,..., n} Y {1,..., n} 14-1

30 Set disjointness (2-DISJ) X Y =? X {1,..., n} Y {1,..., n} Exists a hard distribution µ β, under which X Y = 1 (YES instance) w.p. β and X Y = 0 (NO instance) w.p. 1 β. Lemma: (Generalization of [Razborov 90, BJKS 04]) E[CC β/100 µ β (2-DISJ)] = Ω(n) 14-2

31 2-DISJ k-conn x 1 Bob x 2 Alice and Bob want to solve the 2-DISJ (X, Y ) µ β. (set β = 1/k 2 ) running a protocol for k-conn on: x 5 x 4 Alice Alice plays a random site S I with input X I = X. Bob plays other k 1 sites, and constructs inputs for them using Y : it choose X i µ β Y (i I). Note: X 1,..., X k are i.i.d. conditioned on Y. Vertices u 1,..., u k, v 1,..., v n. An edge (u i, v j ) exists iff X i,j = 1. Let ν be the distribution of the graph. x

32 2-DISJ k-conn (cont.) v j j [r]\y v j j Y v j Y +1 v j Y +2 v j Y +3 v jn v j1 v j2 v j Y X I Y =? u 1 u 2 u 3 u k (u i, v j ) exists if and only if X i,j =

33 2-DISJ k-conn x 1 Bob x 2 Alice and Bob want to solve the 2-DISJ (X, Y ) µ β. (set β = 1/k 2 ) running a protocol for k-conn on: x 5 x 4 Alice Alice plays a random site S I with input X I = X. Bob plays other k 1 sites, and constructs inputs for them using Y : it choose X i µ β Y (i I). Note: X 1,..., X k are i.i.d. conditioned on Y. Vertices u 1,..., u k, v 1,..., v n. An edge (u i, v j ) exists iff X i,j = 1. Let ν be the distribution of the graph. Correctness: Solving k-conn w.pr. 1 β 1.5 under ν also solves 2-DISJ w.pr. 1 β/100 under µ β. x

34 2-DISJ k-conn x 1 Bob x 2 E[CC β/100 µ β (2-DISJ)] = O ( ) 1 k CC β 1.5 ν (k-conn) Alice and Bob want to solve the 2-DISJ (X, Y ) µ β. (set β = 1/k 2 ) running a protocol for k-conn on: x 5 x 4 Alice Alice plays a random site S I with input X I = X. Bob plays other k 1 sites, and constructs inputs for them using Y : it choose X i µ β Y (i I). Note: X 1,..., X k are i.i.d. conditioned on Y. Vertices u 1,..., u k, v 1,..., v n. An edge (u i, v j ) exists iff X i,j = 1. Let ν be the distribution of the graph. Correctness: Solving k-conn w.pr. 1 β 1.5 under ν also solves 2-DISJ w.pr. 1 β/100 under µ β. x

35 2-DISJ k-conn x 1 Bob x 2 E[CC β/100 µ β (2-DISJ)] = O ( ) 1 k CC β 1.5 ν (k-conn) Ω(n) Alice and Bob want to solve the 2-DISJ (X, Y ) µ β. (set β = 1/k 2 ) running a protocol for k-conn on: x 5 x 4 Alice Alice plays a random site S I with input X I = X. Bob plays other k 1 sites, and constructs inputs for them using Y : it choose X i µ β Y (i I). Note: X 1,..., X k are i.i.d. conditioned on Y. Vertices u 1,..., u k, v 1,..., v n. An edge (u i, v j ) exists iff X i,j = 1. Let ν be the distribution of the graph. Correctness: Solving k-conn w.pr. 1 β 1.5 under ν also solves 2-DISJ w.pr. 1 β/100 under µ β. x

36 2-DISJ k-conn x 1 Bob x 2 E[CC β/100 µ β (2-DISJ)] = O ( ) 1 k CC β 1.5 ν (k-conn) Ω(n) Ω(nk) Alice and Bob want to solve the 2-DISJ (X, Y ) µ β. (set β = 1/k 2 ) running a protocol for k-conn on: x 5 x 4 Alice Alice plays a random site S I with input X I = X. Bob plays other k 1 sites, and constructs inputs for them using Y : it choose X i µ β Y (i I). Note: X 1,..., X k are i.i.d. conditioned on Y. Vertices u 1,..., u k, v 1,..., v n. An edge (u i, v j ) exists iff X i,j = 1. Let ν be the distribution of the graph. Correctness: Solving k-conn w.pr. 1 β 1.5 under ν also solves 2-DISJ w.pr. 1 β/100 under µ β. x

37 A useful message We can show trivial UBs are tight for many statistical and graph problems in the MP model (e.g., #distinct elements, bipartiteness, cycle/triangle-freeness,...), when exact answers are wanted. (Woodruff, Z, 2013a) Approximation is often necessary if we want comm. efficient protocols. 18-1

38 A useful message We can show trivial UBs are tight for many statistical and graph problems in the MP model (e.g., #distinct elements, bipartiteness, cycle/triangle-freeness,...), when exact answers are wanted. (Woodruff, Z, 2013a) Approximation is often necessary if we want comm. efficient protocols. Example: Computing the diameter of a graph exactly needs Ω(kn 2 ) bits (when m = n 2 ) in the MP model. However, there is a protocol that computes the diameter up to an additive error of 2 using Õ(kn3/2 ) bits. 18-2

39 Other problems using symmetrization k-bitwise-maj k-bitwise-or/and/xor With applications in Distinct elements reporting ε-kernels (geometry) Heavy-hitters Testing bipartiteness, Testing cycle Testing triangle-freeness,

40 Other problems using symmetrization k-bitwise-maj k-bitwise-or/and/xor With applications in Distinct elements reporting ε-kernels (geometry) Heavy-hitters Testing bipartiteness, Testing cycle Testing triangle-freeness,... Key: Find the right 2-player problem for reduction 19-2

41 20-1 2nd Technique: Composition

42 Example the F 0 (distinct elements) problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute #distinct elements( k i=1 X i) up to a (1 + ε)-approx. 21-1

43 Example the F 0 (distinct elements) problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute #distinct elements( k i=1 X i) up to a (1 + ε)-approx. Current best UB: Õ(k/ε 2 ) [Cormode, Muth, Yi. 2008] Holds in MP model 21-2

44 Example the F 0 (distinct elements) problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute #distinct elements( k i=1 X i) up to a (1 + ε)-approx. Current best UB: Õ(k/ε 2 ) [Cormode, Muth, Yi. 2008] Holds in MP model Previous LB: Ω(k) And Ω(1/ε 2 ) Direct reduction from Gap-Hamming 21-3

45 Example the F 0 (distinct elements) problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute #distinct elements( k i=1 X i) up to a (1 + ε)-approx. Current best UB: Õ(k/ε 2 ) [Cormode, Muth, Yi. 2008] Holds in MP model Previous LB: Ω(k) And Ω(1/ε 2 ) Direct reduction from Gap-Hamming New LB: Ω(k/ε 2 ). [Woodruff, Z. 2012, 2013b]. Holds in MP model. Better UB in the BB model 21-4

46 The proof framework Step 1: Find two modular problems k-gap-maj and 2-DISJ of simpler structures s.t. F 0 can be thought as a composition of them. Step 2: Analyze the complexities of k-gap-maj and 2-DISJ. Step 3: Compose two modular problems so that: Complexity(F 0 ) = Complexity(k-GAP-MAJ) Complexity(2-DISJ). 22-1

47 The proof framework Step 1: Find two modular problems k-gap-maj and 2-DISJ of simpler structures s.t. F 0 can be thought as a composition of them. Step 2: Analyze the complexities of k-gap-maj and 2-DISJ. Step 3: Compose two modular problems so that: Complexity(F 0 ) = Complexity(k-GAP-MAJ) Complexity(2-DISJ). The proof presented here is for a special case when k = 2/ε

48 k-gap-maj k sites each holds a bit Z i chosen uniform at random from {0, 1} Goal: compute k-gap-maj(z 1, Z 2,..., Z k ) = 0, if i [k] Z i k/2 k, 1, if i [k] Z i k/2 + k, don t care, otherwise, 23-1

49 k-gap-maj k sites each holds a bit Z i chosen uniform at random from {0, 1} Goal: compute k-gap-maj(z 1, Z 2,..., Z k ) = 0, if i [k] Z i k/2 k, 1, if i [k] Z i k/2 + k, don t care, otherwise, Lemma: Any protocol Π that computes k-gap-maj correctly w.p has to learn Ω(k) Z i s well, that is, In other words: I(Π; Z 1,..., Z k ) = Ω(k). H(Z i Π) H b (1/100) for Ω(k) of i. 23-2

50 Set disjointness (2-DISJ) X Y =? X {1,..., n} Y {1,..., n} 24-1

51 Set disjointness (2-DISJ) X Y =? X {1,..., n} Y {1,..., n} Exists a hard distribution µ β, under which X Y = 1 (YES instance) w.p. β and X Y = 0 (NO instance) w.p. 1 β. Lemma: (Generalization of [Razborov 90, BJKS 04]) E[CC β/100 µ β (2-DISJ)] = Ω(n) 24-2

52 The proof sketch coordinator 2-DISJ C Y sites S 1 S 2 S 3 S k X 1 X 2 X 3 X k (X i, Y ) µ 1/2 (hard input dist. for DISJ) for each i [k] 25-1

53 The proof sketch coordinator 2-DISJ C Y Z i = X i Y { 1 w.p. 1/2 0 w.p. 1/2 sites S 1 S 2 S 3 S k X 1 X 2 X 3 X k (X i, Y ) µ 1/2 (hard input dist. for DISJ) for each i [k] 25-2

54 The proof sketch coordinator 2-DISJ C Y Z i = X i Y { 1 w.p. 1/2 0 w.p. 1/2 sites S 1 S 2 S 3 S k X 1 X 2 X 3 X k (X i, Y ) µ 1/2 (hard input dist. for DISJ) for each i [k] F 0 (X 1, X 2,..., X k ) k-gap-maj(z 1, Z 2,..., Z k ) (Z i = X i Y ) = learn Ω(k) Z i s well = need Ω(nk) bits (using an embedding argument) 25-3

55 The proof sketch coordinator 2-DISJ C Y Z i = X i Y { 1 w.p. 1/2 0 w.p. 1/2 sites S 1 S 2 S 3 S k Q.E.D. X 1 X 2 X 3 X k (X i, Y ) µ 1/2 (hard input dist. for DISJ) for each i [k] F 0 (X 1, X 2,..., X k ) k-gap-maj(z 1, Z 2,..., Z k ) For the reduction set universe n = Θ(1/ε 2 ) we get Ω(k/ε 2 ) LB. (Z i = X i Y ) = learn Ω(k) Z i s well = need Ω(nk) bits (using an embedding argument) 25-4

56 Other problems using composition Frequency moments F p (p > 1) Entropy l p Heavy-hitters Quantile 26-1

57 Other problems using composition Frequency moments F p (p > 1) Entropy l p Heavy-hitters Quantile Key: Find the right modular problems 26-2

58 27-1 Furture directions

59 Future directions Problem space The current understandings statistics graph linear algebra DB queries Exact computation well medium limited Approximation well limited limited limited 28-1

60 Future directions Problem space The current understandings statistics graph linear algebra DB queries Exact computation well medium limited Approximation well limited limited limited Concretely, e.g., What is the CC of computing a const approx of the size of the matching? 28-2

61 Future directions Problem space (cont.) Communication complexity of distributed property testing. Communication complexity of distributed property testing can also be used for proving LBs for traditional property testing, by extending a framework by Blais et al

62 Future directions Problem space (cont.) Communication complexity of distributed property testing. Communication complexity of distributed property testing can also be used for proving LBs for traditional property testing, by extending a framework by Blais et al Communication complexity of relations (instead of functions, useful for database queries) 29-2

63 Future directions Problem space (cont.) Communication complexity of distributed property testing. Communication complexity of distributed property testing can also be used for proving LBs for traditional property testing, by extending a framework by Blais et al Communication complexity of relations (instead of functions, useful for database queries) Need to understand more primitive problems (building blocks), e.g., breadth-first search (BFS), pointer-chasing, etc. So far building blocks are quite limited: mainly 2-DISJ, 2- GAP-HAMMING, k-gap-maj, k-or and their variants. 29-3

64 Future directions Model/Technique space Can we relax or generalize the symmetrization technique? E.g., do not require totally symmetric distributions 30-1

65 Future directions Model/Technique space Can we relax or generalize the symmetrization technique? E.g., do not require totally symmetric distributions Consider round complexity (practical needs). Consider players with limited space (practical needs). 30-2

66 The end T HAN K YOU Questions? 31-1