A Deterministic Algorithm for Summarizing Asynchronous Streams over a Sliding Window

Transcription

1 A Deermnsc Algorhm for Summarzng Asynchronous Sreams over a Sldng ndow Cosas Busch Rensselaer Polyechnc Insue Srkana Trhapura Iowa Sae Unversy

2 Oulne of Talk Inroducon Algorhm Analyss

3 Tme C Daa sream: v v v3 v v 4 5 For smplcy assume un valued elemens 3

4 Mos recen me wndow of duraon C Daa sream: v v v3 v v 4 5 Curren me Goal: Compue he sum of elemens wh me samps n me wndow [ C, C ] C v C 4

5 Example I: All packes on a nework lnk, manan he number of dfferen p sources n he las one hour Example II: Large daabase, connuously manan averages and frequency momens 5

6 Daa sream: v v v3 v v 4 5 Synchronous sream : In ascendng order Asynchronous sream : No order guaraneed 6

7 hy Asynchronous Daa Sreams? Synchronous sream Nework Asynchronous sream Nework delay & mul-pah roung Synchronous Synchronous Asynchronous Merge w/o conrol 7

8 Processng Requremens: One pass processng Small workspace: poly-logarhmc n he sze of daa Fas processng me per elemen Approxmae answers are ok 8

9 Our resuls: A deermnsc daa aggregaon algorhm Tme: O logb log Space: O log B Relave Error: log log S log B X S 9

10 Prevous ork: [Daar, Gons, Indyk, Mowan. SIAM Journal on Compung, 00] Deermnsc, Synchronous Mergng buckes [Trhapura, Xu, Busch, PODC, 006] Randomzed, Asynchronous Random samplng 0

11 Oulne of Talk Inroducon Algorhm Analyss

12 C Daa sream: Tme Curren me For smplcy assume un valued elemens

13 Mos recen me wndow of duraon C Daa sream: Curren me Goal: Compue he sum of elemens wh me samps n me wndow [ C, C ] 3

14 3 4 Dvde me no perods of duraon 4

15 sldng wndow T 3 4 C The sldng wndow may span a mos wo me perods 5

16 sldng wndow S T lef S rgh 3 4 C S S S Sum can be wren as wo sub-sums In wo me perods 6

17 sldng wndow S T Dlef lef S rgh C Drgh 3 4 Daa srucure ha manans an esmae of In lef me perod S lef 7

18 S lef T D lef hou loss of Generaly, Consder daa srucure n me perod [, ] D lef 8

19 Daa srucure consss of varous levels D D lef D D L L s an upper bound of he sum n a perod 9

20 Consder level D Bucke a Level 0 Tme perod Couns up o elemens 0

21 Sream: Increase couner value

22 Sream: Increase couner value

23 Sream: Increase couner value 3

24 Sream: 3... Increase couner value 4

25 Sream: 3... Spl bucke Couner hreshold of reached 5

26 Sream: 3... New buckes have hreshold also 6

27 Sream: 3... Increase approprae bucke 7

28 Sream: 3... Increase approprae bucke 8

29 Sream: Increase approprae bucke 9

30 Sream:... m m x Spl bucke

31 Sream:... m x

32 Sream:... m m m 3 4 x Increase approprae bucke 3

33 Sream:... m m... m x Spl bucke x

34 Sream:... m m... m x x

35 Splng Tree x x k x 4 x x

36 Max deph = Leaf buckes of duraon are no spl any furher log 36

37 Leaf buckes The nal bucke may be spl no many buckes 37

38 Leaf buckes Due o space lmaons we only keep he las buckes a log 38

39 T S Suppose we wan o fnd he sum of elemens n me perod [ T, ] S 39

40 T Consder varous levels of splng hreshold S a a k k a a 40

41 T Frs level wh a leaf bucke ha nersecs melne S a a k k a a 4

42 S T Esmae of S: X x x x z k x x Consder buckes on rgh of melne a xz z a 4

43 OR T Frs level wh a leaf bucke On rgh melne S a a k k a a 43

44 Oulne of Talk Inroducon Algorhm Analyss 44

45 S T Suppose ha we use level n order o compue he esmae 45

46 Sream: k l x b x b r Consder splng hreshold level A daa elemen s couned n he approprae bucke 46

47 Sream: k l k r k l r e can assume ha he elemen s placed n he respecve bucke 47

48 Sream: k l l k l r l r r r e can assume ha when bucke spls he elemen s placed n an arbrary chld bucke 48

49 Sream: k l r l k l r l r r If: l k l r GOOD! Elemen couned n correc bucke 49

50 Sream: k l r l k l r l r r If: l r k r BAD! Elemen couned n wrong bucke 50

51 T Consder Leaf Buckes S k If T k GOOD! 5

52 T Consder Leaf Buckes S k If k T BAD! Elemen couned n wrong bucke 5

53 T Consder Leaf Buckes S k X S Z Z Z :elemens of lef par couned on rgh Z :elemens of rgh par couned on lef 53

54 T k Z elemens of lef par couned on rgh k Mus have been nally nsered n one of hese buckes 54

55 Snce ree deph log Z O ( log ) 55

56 Snce ree deph log Z O ( log ) Smlarly, we can prove Z O ( log ) Therefore: X S Z Z O ( log ) 56

57 Snce a log I can be proven S ( log ) 57

58 Snce a log I can be proven S ( log ) Combned wh X S O ( log ) e oban relave error : X S S 58