Tight Bounds for Distributed Streaming

Transcription

1 Tight Bounds for Distributed Streaming (a.k.a., Distributed Functional Monitoring) David Woodruff IBM Research Almaden Qin Zhang MADALGO, Aarhus Univ. STOC 12 May 22,

2 The distributed streaming model sites coordinator C S 1 S 2 S 3 S k The coordinator needs to maintain some function defined on the union of k streams at any time 2-1

3 The distributed streaming model sites coordinator C S 1 S 2 S 3 S k The coordinator needs to maintain some function defined on the union of k streams at any time Goal: minimize total bits of communication 2-2

4 The distributed streaming model sites Q: What s the total # items? coordinator C S 1 S 2 S 3 S k The coordinator needs to maintain some function defined on the union of k streams at any time e.g., total # items Goal: minimize total bits of communication 2-3

5 The distributed streaming model sites Q: What s the total # items? coordinator C S 1 S 2 S 3 S k The coordinator needs to maintain some function defined on the union of k streams at any time e.g., total # items Almost always allow approximation. Goal: minimize total bits of communication 2-4

6 Applied motivation: Distributed monitoring Sensor networks Sites: sensors Streams: e.g., environmental data Concerns: energy Abstraction/ Simplification coordinator sites C S 1 S 2 S 3 S k 3-1

7 Applied motivation: Distributed monitoring Sensor networks Network routers Data in the cloud Sites: sensors Streams: e.g., environmental data Concerns: energy Sites: routers Streams: e.g., IP addresses. Concerns: bandwidth Sites: machines Streams: e.g., queries/updates. Concerns: bandwidth Abstraction/ Simplification coordinator sites C S 1 S 2 S 3 S k 3-2

8 Problems coordinator sites C S 1 S 2 S 3 S k Frequency moments F p = i f p i f i : freq. of element i In particular: F 0 : #distinct elements F 2 : size of self-join Heavy hitters The Distributed Streaming Model Quantile Entropy... Well-studied problems in the data stream literature 4-1

9 Results k: # sites; n: input size; ε: approximation ratio Problems Upper Bound Lower Bound F p (p > 1) F 0 Heavy-hitter/ Quantile Entropy Õ(k 2p+1 n 1 2/p /poly(ε)) [CMY, SODA 08] Õ(k p 1 /poly(ε)) [This paper] Õ(k/ε 2 ) [CMY, SODA 08] Õ(min{ k/ε, 1/ε 2 }) [HYZ, PODS 12] Õ(k/ε 3 ) [ABC, ICALP 09] Ω(k) [CMY, SODA 08] Ω(k p 1 /ε 2 ) [This paper] Ω(k) [CMY, SODA 08] Ω(k/ε 2 ) [This paper] Ω(max{ k/ε, 1/ε 2 }) [HYZ, PODS 12]; [This paper] static case Ω(1/ ε) [ABC, ICALP 09] Ω(k/ε 2 ) [This paper] Improve LBs for all problems, and the UB for F p (p > 1). 5-1

10 Results k: # sites; n: input size; ε: approximation ratio Problems Upper Bound Lower Bound F p (p > 1) F 0 Heavy-hitter/ Quantile Entropy Õ(k 2p+1 n 1 2/p /poly(ε)) [CMY, SODA 08] Õ(k p 1 /poly(ε)) [This paper] Õ(k/ε 2 ) [CMY, SODA 08] Õ(min{ k/ε, 1/ε 2 }) [HYZ, PODS 12] Õ(k/ε 3 ) [ABC, ICALP 09] Ω(k) [CMY, SODA 08] Ω(k p 1 /ε 2 ) [This paper] Ω(k) [CMY, SODA 08] Ω(k/ε 2 ) [This paper] Ω(max{ k/ε, 1/ε 2 }) [HYZ, PODS 12]; [This paper] static case Ω(1/ ε) [ABC, ICALP 09] Ω(k/ε 2 ) [This paper] Improve LBs for all problems, and the UB for F p (p > 1). Our LBs even hold in the static case. Static LBs (almost) match continuous UBs. 5-2

11 By-products Implications for problems in the data stream model First lower bound for F 0 without Gap-Hamming Improve Ω(n 1 2/p /ε 2/p t) bound for estimating F p (p 2) in a stream using t passes to Ω(n 1 2/p /ε 4/p t). First LB that agrees with the UB for F 2 (p = 2), for any constant t. 6-1

12 The multiparty NIH communication model A model for (static) lower bounds x 1 = x 2 = x 3 = x k = They want to jointly compute f(x 1, x 2,..., x k ) Goal: minimize total bits of communication 7-1

13 The multiparty NIH communication model A model for (static) lower bounds x 1 = x 2 = Blackboard: One speaks, everyone else hears. Message passing: If x 1 talks to x 2, others cannot hear. x 3 = x k = They want to jointly compute f(x 1, x 2,..., x k ) Goal: minimize total bits of communication 7-2

14 The multiparty NIH communication model A model for (static) lower bounds x 1 = x 2 = Blackboard: One speaks, everyone else hears. Message passing: If x 1 talks to x 2, others cannot hear. x 3 = x k = = static version of C They want to jointly compute f(x 1, x 2,..., x k ) S 1 S 2 S 3 S k Goal: minimize total bits of communication 7-3

15 Previously, for multiparty NIH comm. model Well studied? 8-1

16 Previously, for multiparty NIH comm. model Well studied? YES and NO. Several papers in blackboard model. [Alon, Matias, Szegedy 96] [Bar-Yossef, Jayram, Kumar, Sivakumar 04]

17 Previously, for multiparty NIH comm. model Well studied? YES and NO. Several papers in blackboard model. [Alon, Matias, Szegedy 96] [Bar-Yossef, Jayram, Kumar, Sivakumar 04]... Almost nothing in message-passing model. 8-3

18 Previously, for multiparty NIH comm. model Well studied? YES and NO. Several papers in blackboard model. [Alon, Matias, Szegedy 96] [Bar-Yossef, Jayram, Kumar, Sivakumar 04]... Almost nothing in message-passing model. A technique called symmetrization is proposed [Phillips, Verbin, Zhang 12 ] which works for both variants. However, this technique cannot be applied to our problems, due to several inherent limitations. 8-4

19 Now our new technique: Composition 9-1

20 Now our new technique: Composition A Lower bound for F 0 (distinct elements) Will cheat constantly, see our paper for the real proof 9-2

21 The F 0 problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute #distinct elements( k i=1 X i) up to a (1 + ε)-approx How many distinct items? 10-1

22 The F 0 problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute #distinct elements( k i=1 X i) up to a (1 + ε)-approx Current best UB: Õ(k/ε 2 ) Holds in message-passing model Previous LB: Ω(k) How many distinct items? 10-2

23 The F 0 problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute #distinct elements( k i=1 X i) up to a (1 + ε)-approx How many distinct items? Current best UB: Õ(k/ε 2 ) Holds in message-passing model Previous LB: Ω(k) And Ω(1/ε 2 ) (reduction from Gap-Hamming) 10-3

24 The F 0 problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute #distinct elements( k i=1 X i) up to a (1 + ε)-approx How many distinct items? Tight! Current best UB: Õ(k/ε 2 ) Holds in message-passing model Previous LB: Ω(k) And Ω(1/ε 2 ) (reduction from Gap-Hamming) Our LB: Ω(k/ε 2 ). Holds in message-passing model Better UB in the blackboard model 10-4

25 F 0 hard input distribution S 1 S n S k

26 F 0 hard input distribution S 1 S n S k F 0 =

27 F 0 hard input distribution Random partition S 1 S 2 S k noise part 0/1 w. equal. prob. important part 0/1 0/1 0 0/1 Each row in important part: choose a random special column put 0/1 w. equal. prob. rest all

28 F 0 hard input distribution Random partition S 1 S 2 S k noise part 0/1 w. equal. prob. important part 0/1 0/1 0 0/1 Each row in important part: choose a random special column put 0/1 w. equal. prob. rest all 0 Let Z i be the value in the special column in i-th row. F 0 (# columns of noise part) + i Z i 12-2

29 The proof framework Step 1: Find two modular problems k-gap-maj and 2-DISJ of simpler structures s.t. F 0 can be thought as a composition of them. (Different from traditional direct-sum) Step 2: Analyze the complexities of k-gap-maj and 2-DISJ. Step 3: Compose two modular problems so that: Complexity(F 0 ) = Complexity(k-GAP-MAJ) Complexity(2-DISJ). 13-1

30 k-gap-maj k sites each holds a bit Z i chosen uniform at random from {0, 1} Goal: compute k-gap-maj(z 1, Z 2,..., Z k ) = 0, if i [k] Z i k/2 k, 1, if i [k] Z i k/2 + k, don t care, otherwise, 14-1

31 k-gap-maj k sites each holds a bit Z i chosen uniform at random from {0, 1} Goal: compute k-gap-maj(z 1, Z 2,..., Z k ) = 0, if i [k] Z i k/2 k, 1, if i [k] Z i k/2 + k, don t care, otherwise, Lemma: Any protocol Π that computes k-gap-maj correctly w.p has to learn Ω(k) Z i s well, that is, H(Z i Π) H b (1/100). 14-2

32 Set disjointness (2-DISJ) x y =? x {0, 1,..., n} y {0, 1,..., n} 15-1

33 Set disjointness (2-DISJ) x y =? x {0, 1,..., n} y {0, 1,..., n} Exists a hard distribution µ, under which X Y = 1 (YES instance) w.p. 1/2 and X Y = 0 (NO instance) w.p. 1/2. Lemma: Any protocol Π that computes 2-DISJ correctly w.p under distribution µ communicates at least Ω(n) bits. [Razborov 90, BJKS 04] 15-2

34 Next step: Compose k-gap-maj with 2-DISJ coordinator C Y 2-DISJ 2-DISJ 2-DISJ sites S 1 S 2 S 3 S k X 1 X 2 X 3 X k (X i, Y ) µ for each i = 1,..., k 16-1

35 Next step: Compose k-gap-maj with 2-DISJ coordinator C Y 2-DISJ 2-DISJ 2-DISJ Z i = X i Y { 1 w.p. 1/2 0 w.p. 1/2 sites S 1 S 2 S 3 S k X 1 X 2 X 3 X k (X i, Y ) µ for each i = 1,..., k 16-2

36 Next step: Compose k-gap-maj with 2-DISJ [n] Y Y S 1 S 2 S k 0/1 w. equal. prob. 0/1 0/1 0 0/1 Z i coordinator C Y 2-DISJ 2-DISJ 2-DISJ Z i = X i Y { 1 w.p. 1/2 0 w.p. 1/2 sites S 1 S 2 S 3 S k X 1 X 2 X 3 X k (X i, Y ) µ for each i = 1,..., k 16-3

37 Next step: Compose k-gap-maj with 2-DISJ [n] Y Y S 1 S 2 S k 0/1 w. equal. prob. 0/1 0/1 0 0/1 Z i F 0 (X 1, X 2,..., X k ) [n] Y + i Z i coordinator C Y 2-DISJ 2-DISJ 2-DISJ Z i = X i Y { 1 w.p. 1/2 0 w.p. 1/2 sites S 1 S 2 S 3 S k X 1 X 2 X 3 X k (X i, Y ) µ for each i = 1,..., k 16-4

38 Next step: Compose k-gap-maj with 2-DISJ [n] Y Y S 1 S 2 S k 0/1 w. equal. prob. 0/1 0/1 0 0/1 Z i F 0 (X 1, X 2,..., X k ) [n] Y + i fixed Z i coordinator C Y Links to k-gap-maj 2-DISJ 2-DISJ 2-DISJ { 1 w.p. 1/2 Z i = X i Y 0 w.p. 1/2 sites S 1 S 2 S 3 S k X 1 X 2 X 3 X k (X i, Y ) µ for each i = 1,..., k 16-5

39 The proof coordinator 2-DISJ C Y Z i = X i Y { 1 w.p. 1/2 0 w.p. 1/2 sites S 1 S 2 S 3 S k X 1 X 2 X 3 X k F 0 (X 1, X 2,..., X k ) k-gap-maj(z 1, Z 2,..., Z k ) (Z i = X i Y ) = learn Ω(k) Z i s well = need Ω(nk) bits (observe Z i Π are independent, learning each Z i well needs Ω(n) bits, by 2-DISJ) 17-1

40 The proof coordinator 2-DISJ C Y Z i = X i Y { 1 w.p. 1/2 0 w.p. 1/2 sites S 1 S 2 S 3 S k Q.E.D. X 1 X 2 X 3 X k F 0 (X 1, X 2,..., X k ) k-gap-maj(z 1, Z 2,..., Z k ) For the reduction set n = Θ(1/ε 2 ) we get Ω(k/ε 2 ) LB. (Z i = X i Y ) = learn Ω(k) Z i s well = need Ω(nk) bits (observe Z i Π are independent, learning each Z i well needs Ω(n) bits, by 2-DISJ) 17-2

41 Our new technique: composition Step 1: Find two (or more) modular problems A, B of simpler structures s.t. the original problem can be thought as a composition of them. Step 2: Analyze the complexities of A, B. Step 3: Compose modular problems A, B so that: Complexity(original problem) = Complexity(A) Complexity(B). 18-1

42 The F 2 problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute F 2 ( k i=1 X i) up to a (1 + ε)-approximation. Previous UB: Õ(k 2 /ε + k 1.5 /ε 3 ) Our UB: Õ(k/poly(ε)), one way protocol Previous LB: Ω(k) Our LB: Ω(k/ε 2 ). Holds in blackboard model. Tight in static case. 19-1

43 The F 2 problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute F 2 ( k i=1 X i) up to a (1 + ε)-approximation. Previous UB: Õ(k 2 /ε + k 1.5 /ε 3 ) Our UB: Õ(k/poly(ε)), one way protocol Previous LB: Ω(k) Our LB: Ω(k/ε 2 ). Holds in blackboard model. Tight in static case. LB proof ideas overview: Same framework, choose two modular problems Gap-Hamming k-disj 19-2

44 The F 2 problem k sites each holds a set X i (i {1, 2,..., k}). Goal: compute F 2 ( k i=1 X i) up to a (1 + ε)-approximation. Previous UB: Õ(k 2 /ε + k 1.5 /ε 3 ) Our UB: Õ(k/poly(ε)), one way protocol Previous LB: Ω(k) Our LB: Ω(k/ε 2 ). Holds in blackboard model. Tight in static case. LB proof ideas overview: Same framework, choose two modular problems Gap-Hamming k-disj Compose in a different way to prove a LB for F 2 Heavy use of information cost 19-3

45 Two modular problems 2-party Gap-Hamming: Alice has X = {X 1, X 2,..., X 1/ε 2}, Bob has Y = {Y 1, Y 2,..., Y 1/ε 2}. They want to compute: GHD(X, Y ) = 0, 1, if i [1/ε 2 ] X i Y i 1/2ε 2 1/ε, if i [1/ε 2 ] X i Y i 1/2ε 2 + 1/ε,, otherwise, Solving it w.r.t. uniform distribution needs to reveal Ω(1/ε 2 ) bits of the input 20-1

46 Two modular problems 2-party Gap-Hamming: Alice has X = {X 1, X 2,..., X 1/ε 2}, Bob has Y = {Y 1, Y 2,..., Y 1/ε 2}. They want to compute: GHD(X, Y ) = 0, 1, if i [1/ε 2 ] X i Y i 1/2ε 2 1/ε, if i [1/ε 2 ] X i Y i 1/2ε 2 + 1/ε,, otherwise, Solving it w.r.t. uniform distribution needs to reveal Ω(1/ε 2 ) bits of the input k-disj: We have k sites S 1,..., S k. S i holds a set Z i ( Z i = k 2 ). We promise that either Z i are all disjoint, or they intersect on one element and the rest are all disjoint (sun-flower). The goal is to find out which is the case. Solving it w.r.t. certain distribution needs to reveal Ω(k) bits of the input 20-2

47 Next step: Compose Gap-Hamming with k-disj coordinator C sites S k2 +1 S 1 S k2 S k Alice Bob X 1 X 2 Gap-Hamming Y 1 Y 2 X 1/ε 2 Y 1/ε 2 We create 2/ε 2 (k/2)-disj instances, one for each input bit of Gap-Hamming. 21-1

48 Next step: Compose Gap-Hamming with k-disj sites coordinator C S 1 S k2 S k X 1 X 2 S k2 +1 Alice Bob Gap-Hamming Y 1 Y 2 X 1/ε 2 Y 1/ε 2 The proof Solve (1 + ε)-approx F 2 Solve Gap-Hamming (GHD) Learn Ω(1/ε 2 ) input bits of GHD Learning each bit of GHD needs to solve an instance of (k/2)-disj Solving each (k/2)-disj has to reveal Ω(k) bits of the inputs Reveal Ω(k/ε 2 ) bits in total We create 2/ε 2 (k/2)-disj instances, one for each input bit of Gap-Hamming. Q.E.D. 21-2

49 F p (p > 1) upper bounds, a quick glance Previous UB: Õ(k 2p+1 n 1 2/p /poly(ε)) Our UB: Õ(k p 1 /poly(ε)) 22-1

50 F p (p > 1) upper bounds, a quick glance Previous UB: Õ(k 2p+1 n 1 2/p /poly(ε)) Our UB: Õ(k p 1 /poly(ε)) Inspired by work on sub-sampling [Indyk-Woodruff 2005] New features in our protocol: No AMS sketches One-way protocol Threshold-based sampling used to communicationefficiently implement distributed k-party heavy hitters. 22-2

51 F p (p > 1) upper bounds, a quick glance Previous UB: Õ(k 2p+1 n 1 2/p /poly(ε)) Our UB: Õ(k p 1 /poly(ε)) Inspired by work on sub-sampling [Indyk-Woodruff 2005] New features in our protocol: No AMS sketches One-way protocol Threshold-based sampling used to communicationefficiently implement distributed k-party heavy hitters. We suspect it can have more applications, as IW05 did for streaming model. e.g., for distributed EMD, distributed l p - sampling, etc. 22-3

52 Conclusion and future work We obtained: (almost) tight bounds for frequency moments, heavy hitters, quantile and entropy in the distributed streaming model. 23-1

53 Conclusion and future work We obtained: (almost) tight bounds for frequency moments, heavy hitters, quantile and entropy in the distributed streaming model. Future work F 2 is not tight in terms of ε 23-2

54 Conclusion and future work We obtained: (almost) tight bounds for frequency moments, heavy hitters, quantile and entropy in the distributed streaming model. Future work F 2 is not tight in terms of ε Generalize the model: consider the network topology; items go into multiple sites,

55 Conclusion and future work We obtained: (almost) tight bounds for frequency moments, heavy hitters, quantile and entropy in the distributed streaming model. Future work F 2 is not tight in terms of ε Generalize the model: consider the network topology; items go into multiple sites,.... Beyond statistical problems Geometry problems: range-counting, extent measures,... Graph problems 23-4

56 The end T HAN K YOU Questions? 24-1