A Deermnsc Algorhm for Summarzng Asynchronous Sreams over a Sldng ndow Cosas Busch Rensselaer Polyechnc Insue Srkana Trhapura Iowa Sae Unversy
Oulne of Talk Inroducon Algorhm Analyss
Tme C Daa sream: 3 4 5 v v v3 v v 4 5 For smplcy assume un valued elemens 3
Mos recen me wndow of duraon C Daa sream: 3 4 5 v v v3 v v 4 5 Curren me Goal: Compue he sum of elemens wh me samps n me wndow [ C, C ] C v C 4
Example I: All packes on a nework lnk, manan he number of dfferen p sources n he las one hour Example II: Large daabase, connuously manan averages and frequency momens 5
Daa sream: 3 4 5 v v v3 v v 4 5 Synchronous sream : In ascendng order Asynchronous sream : No order guaraneed 6
hy Asynchronous Daa Sreams? Synchronous sream Nework Asynchronous sream Nework delay & mul-pah roung Synchronous Synchronous Asynchronous Merge w/o conrol 7
Processng Requremens: One pass processng Small workspace: poly-logarhmc n he sze of daa Fas processng me per elemen Approxmae answers are ok 8
Our resuls: A deermnsc daa aggregaon algorhm Tme: O logb log Space: O log B Relave Error: log log S log B X S 9
Prevous ork: [Daar, Gons, Indyk, Mowan. SIAM Journal on Compung, 00] Deermnsc, Synchronous Mergng buckes [Trhapura, Xu, Busch, PODC, 006] Randomzed, Asynchronous Random samplng 0
Oulne of Talk Inroducon Algorhm Analyss
C Daa sream: Tme 3 4 5 6 Curren me For smplcy assume un valued elemens
Mos recen me wndow of duraon C Daa sream: 3 4 5 6 Curren me Goal: Compue he sum of elemens wh me samps n me wndow [ C, C ] 3
3 4 Dvde me no perods of duraon 4
sldng wndow T 3 4 C The sldng wndow may span a mos wo me perods 5
sldng wndow S T lef S rgh 3 4 C S S S Sum can be wren as wo sub-sums In wo me perods 6
sldng wndow S T Dlef lef S rgh C Drgh 3 4 Daa srucure ha manans an esmae of In lef me perod S lef 7
S lef T D lef hou loss of Generaly, Consder daa srucure n me perod [, ] D lef 8
Daa srucure consss of varous levels D D lef D D L L s an upper bound of he sum n a perod 9
Consder level D Bucke a Level 0 Tme perod Couns up o elemens 0
Sream: Increase couner value
Sream: Increase couner value
Sream: 3 3 3 Increase couner value 3
Sream: 3... Increase couner value 4
Sream: 3... Spl bucke Couner hreshold of reached 5
Sream: 3... New buckes have hreshold also 6
Sream: 3... Increase approprae bucke 7
Sream: 3... Increase approprae bucke 8
Sream: 3... 3 3 Increase approprae bucke 9
Sream:... m m x Spl bucke 3 4 3 4 30
Sream:... m x 3 4 3 4 3
Sream:... m m m 3 4 x 3 4 3 4 Increase approprae bucke 3
Sream:... m m... m x Spl bucke x 4 3 3 4 4 5 8 5 8 3 4 33
Sream:... m m... m x x 4 3 4 5 8 5 8 3 4 34
Splng Tree x x k 3 4 3 4 x 4 x x 3 5 5 8 8 3 4 35
Max deph = Leaf buckes of duraon are no spl any furher log 36
Leaf buckes The nal bucke may be spl no many buckes 37
Leaf buckes Due o space lmaons we only keep he las buckes a log 38
T S Suppose we wan o fnd he sum of elemens n me perod [ T, ] S 39
T Consder varous levels of splng hreshold S a a k k a a 40
T Frs level wh a leaf bucke ha nersecs melne S a a k k a a 4
S T Esmae of S: X x x x z k x x Consder buckes on rgh of melne a xz z a 4
OR T Frs level wh a leaf bucke On rgh melne S a a k k a a 43
Oulne of Talk Inroducon Algorhm Analyss 44
S T Suppose ha we use level n order o compue he esmae 45
Sream: k l x b x b r Consder splng hreshold level A daa elemen s couned n he approprae bucke 46
Sream: k l k r k l r e can assume ha he elemen s placed n he respecve bucke 47
Sream: k l l k l r l r r r e can assume ha when bucke spls he elemen s placed n an arbrary chld bucke 48
Sream: k l r l k l r l r r If: l k l r GOOD! Elemen couned n correc bucke 49
Sream: k l r l k l r l r r If: l r k r BAD! Elemen couned n wrong bucke 50
T Consder Leaf Buckes S k If T k GOOD! 5
T Consder Leaf Buckes S k If k T BAD! Elemen couned n wrong bucke 5
T Consder Leaf Buckes S k X S Z Z Z :elemens of lef par couned on rgh Z :elemens of rgh par couned on lef 53
T k Z elemens of lef par couned on rgh k Mus have been nally nsered n one of hese buckes 54
Snce ree deph log Z O ( log ) 55
Snce ree deph log Z O ( log ) Smlarly, we can prove Z O ( log ) Therefore: X S Z Z O ( log ) 56
Snce a log I can be proven S ( log ) 57
Snce a log I can be proven S ( log ) Combned wh X S O ( log ) e oban relave error : X S S 58