Classification Spanning Correlated Data Streams

Transcription

1 lssifition Spnning orrlt Dt Strms Yo Xu, K Wng omputr Sin Shool Simon Frsr Univrsity {yxu,wngk}@s.sfu. A Wi-h Fu Dprtmnt of omputr Sin n Enginring, Th hins Univrsity of Hong Kong fu@s.uhk.u.hk Rong Sh, Jin Pi omputr Sin Shool Simon Frsr Univrsity {rsh, jpi}@s.sfu. ABSTRAT In mny pplitions, lssifirs n to uilt s on multipl rlt t strms. For xmpl, stok strms n nws strms r rlt, whr th lssifition pttrns my involv fturs from oth strms. Thus inst of mining on singl isolt strm, w n to xmin multipl rlt t strms in orr to fin suh pttrns n uil n urt lssifir. Othr xmpls of rlt strms inlu trffi rports n r ints, snsor rings of iffrnt typs or t iffrnt lotions, t. In this ppr, w onsir th lssifition prolm fin ovr sliing-winow join of svrl input t strms. As th t strms rriv in fst p n th mny-to-mny join rltionship lows up th t rrivl rt vn mor, it is imprtil to omput th join n thn uil th lssifir h tim th winow slis forwr. W prsnt n ffiint lgorithm to uil Nïv Bysin lssifir in suh ontxt. Our mtho os not n to prform th join oprtions ut is still l to uil xtly th sm lssifir s if uilt on th join rsult. It only xmins h input tupl twi, inpnnt of th numr of tupls it joins in othr strms, thrfor, is l to kp p with th fst rriving t strms in th prsn of mny-to-mny join rltionships. Th xprimnts onfirm tht our lssifition lgorithm is mor ffiint thn onvntionl mthos whil mintining goo lssifition ury. tgoris n Sujt Dsriptors H..8 [Dts Applitions]: Dt mining Gnrl Trms: lgorithms, mngmnt, prformn Kywors: lgorithm, strm t, join, lssifition, Nïv Bysin mol. INTRODUTION At tim of informtion xplosion, w s tht t not only r stor in lrg mounts, ut lso kp growing quikly ovr tim. Evry y millions of nk trnstions r ror, tlphon lls r rgistr, mils r stor, ll of whih kp ing ppn to xisting tss. Suh tss r thrfor Prmission to mk igitl or hr opis of ll or prt of this work for prsonl or lssroom us is grnt without f provi tht opis r not m or istriut for profit or ommril vntg n tht opis r this noti n th full ittion on th first pg. To opy othrwis, or rpulish, to post on srvrs or to ristriut to lists, rquirs prior spifi prmission n/or f. IKM 06, Novmr 5, 006, Arlington, Virgini, USA. opyright 006 AM /06/00...$5.00. ll t strms, s t ontinuously flow in n thr is no prtiulr orr in whih th t itms rriv. Dt strms r hrtriz s ing in high volum, unoun in siz, ynmilly hnging n rquir fst rspons tim []. As t r ontinuously volving, so r th m trns n pttrns. Sin it is impossil to stor th omplt strm for th mining strts, quikly tting volving t hrtristis is importnt for ision mking. Any lgorithm sign for t strms must hv vry low omputtion tim pr input tupl in orr to kp p with th high t rrivl rt. Morovr, thr r oftn situtions in strm mining whr multipl rlt t strms n to xmin t th sm tim in orr to isovr trns or pttrns tht involv fturs from iffrnt t strms. In, thr r mny pplitions whr th lssifition pttrns spn ross multipl strms. For xmpl, stok strms n nws strms r rlt, trffi rport strms n r-int strms r rlt, snsor rings of iffrnt typs r rlt. In suh pplitions, oourrn of rtin onitions in svrl rlt strms my jointly trmin th lss ll, thrfor rlt strms shoul xmin togthr to uil th lssifir. To illustrt this, lt us onsir simplifi xmpl. In th stok mrkt, fvorl tring rfrs to stok trnstions tht r fvorl to th ngging prty, i.., slling for stok plungs or uying for stok gos up. In orr to uil lssifition mols tht intify pttrns for fvorl tring, th stok tring strm tht rors ll tring trnstions must xmin. Howvr, stok trnstions r not isolt or inpnnt vnts; thy r rlt to mny othr t strms,.g., phon lls twn lrs n mngrs/stffs of puli ompnis. Thus it is nssry to min on multipl rlt t strms. For xmpl, th lssifition lgorithm my n to look t ll following orrlt t: Tring strm: T (, Dlr, Typ, Stok, lss) Phon ll strm: P (, llr, ll) ompny tl: (ompny, Stok) Prson tl: S (Nm, Org) whr is th timstmp, Typ is ithr sll or uy, lss ( ys / no ) rfrs to th lss ll of ing fvorl tring or not. To omput th omplt trining st, SQL qury n us to xtrt informtion from th ov t s follows: SELET * FROM P, S, T, WHERE S.Nm=P.llr AND P.ll=T.Dlr AND P.<T. AND T.Stok=.Stok

2 Nm Org Ams AAA Ry Dnnis S Dlr Typ 9:8 Jk Sll :0 Slin Sll 5:40 Sll 5:57 Buy 6:4 Ptr Sll T 9:0 P :0 4:9 5: 5:6 Stok lss A Ys A Ys B No Ys No llr Ams Ams Dnnis Ry Dnnis Stok A B ll Jk Slin Ptr ompny AAA lss Ys Ys Ys Ys No No No llr Ams Ams Ry Ry Dnnis Dnnis Dnnis Org AAA AAA ll Jk Slin Ptr Dlr Jk Slin Ptr Typ Sll Sll Sll Buy Sll Buy Sll Stok A A B B ompny AAA AAA Tl. Rlt strms / tls Tl. Th join strm Essntilly, this qury prforms join on ll rlt t n th join rsult is th trining st us to uil th lssifir. Tl shows snpshot of suh t. Th join rltionship is init y th rrows onnting th join ttriuts. Not tht th join rltionship twn P n T is mny-to-mny, whih is th most gnrl s. For xmpl, ws ll twi n tr twi, gnrting four tupls in th join strm in Tl (Th timstmps for h join ror r ignor us of th sp.), th rul Org=ompny lss=ys hols in out of 4 tupls tht hv Org=ompny, i.., with 75% onfin. It suggsts tht ftr gtting ll, th tring on th llr s ompny stok tns to mor fvorl. A lssifir ring ruls tht utiliz informtion from multipl orrlt strms/tls is likly mor urt thn thos uilt on th tring strm lon. Motivt y th ov isussions, in this ppr, w will onsir th prolm of lssifition ovr multipl t strms with gnrl join rltionships. Suh prolm is ommon in prti. Atully, in th ltr stion, our xprimnts on rllif tst (UK ro int tst) onfirm tht lssifirs uilt on multipl strms r muh mor urt thn thos uilt only on singl trgt strm. For suh pplitions, th mn on ffiiny of th strm pplitions is vn mor hllnging. Sin th siz of th t strm is unoun n nw t ontinuously flow in, it is impossil to stor th ntir t strm first for th mining strts. Inst, strm pplitions only l with th urrnt prt of strm (ll winow) whih my look lik stti tl. As nw tupls flow in, th winow lso slis forwr to inlu th ltst tupls whil som ol tupls r xpir from th winow. For onvntionl stti rltions, join is nturl oprtion to link rlt rltions y t smntis. For onlin n unoun t strms, joining th ntirty of th t is impossil. Inst, join n fin on strm winows, ll sliing-winow join [], whih gnrts nw strm, ll th join strm, rprsnting th join of ll rlt strms in thir urrnt winows. Whn th winow of ny input strm slis forwr, th join strm lso volvs to inlu nw join tupls n invlit xpir join tupls. Th prolm w r rssing in this ppr is to uil th lssifir s on suh join strm, rfrr to s th join strm lssifition prolm hrinftr. Not stti tls r lso llow (s in Tl ) in th sm lssifition prolm y onsiring thm s strms tht nvr hng. For join strm lssifition, th lssifir must upt h tim ny strm winow is upt. Th sliing of th winow n tupl-s or tims. Whn th gp twn two onsutiv winows is smll, th lssifir must ruilt t vry fst p, i.. th lgorithm tht uils th lssifir must vry ffiint. For suh join strm lssifition, tmpting strightforwr solution my to omput th join h tim winow slis forwr n thn uil th lssifir s on th urrnt join strm. Unfortuntly, this is unrlisti for t strms. As th input strms r rriving in fst p n th tupls in rlt strms rriv in no prtiulr orr, it is iffiult to optimiz th join oprtions, i.. th join oprtions r xpnsiv. Yt th lssifir must volv quikly h tim th winow slis forwr, thus thr my no tim to prform th join [0][6][]. Furthrmor, th join rltionship n mny-to-mny s shown in th smpl t in Tl, thrfor, thr r fr mor tupls in th join strm thn in th input strms. Any mtho tht xpliitly gnrts th join strm will suffr from th lowup of t rrivl rts n is unlikly to l to kp p with th inoming t. Som prvious works hv lt with hllngs in th singlstrm lssifition prolm [][4][4][5]. To th st of our knowlg, thr hs n no work on suh join strm lssifition prolm. Although thr hv n mny lssifition lgorithms tht work wll on stti tls, it is vry iffiult to pt thm to join strm lssifition. For xmpl, support vtor mhins (SVM) rquir xpliit gnrtion of th trining st, i.. th join strm, in orr to uil th lssifir. On th othr hn, w noti tht Nïv Bysin lssifir (NB)

3 [] hs som uniqu proprtis whih n xplor to voi th join. NB is on of th most wily us n sussful lssifition mthos. Although it ssums vrils r inpnnt givn th lss ll, rsrhs show tht NB is still rlil vn whn this ssumption is violt [][][9]. Thus w liv tht th NB prsnts st opportunity in orr to l with th join strm lssifition prolm. By tking vntg of th uniqu proprtis of th NB lssifir, w n ffiintly rss this prolm. To kp p with th fst input strms, w propos NB mtho tht os not n to join th input strms, whil still prouing th xt sm NB s prou on th join strm. Our insight is tht th informtion rquir y NB for th join strm n otin y omputing low-up ounts irtly from th input strms in linr tim. Our min hivmnt lis in th ft tht omputing th low-up ounts is muh hpr thn omputing th sliing-winow join itslf. Our pproh xmins h input tupl in th urrnt winow twi, inpnnt of th numr of tupls it joins in th othr strm winows. Not sin th strm winow n hl in min mmory, snning h tupl in th urrnt winow in linr tim is vry ffiint. Thus it is highly suitl to hnl high-sp strms in th prsn of mny-to-mny join. Not th i of omputing suh low-up ounts to voi th xpnsiv join oprtions is lso pplil to som othr lssifition lgorithms tht rquir similr sttistis, for xmpl, ision trs. Th rst of this ppr is orgniz s follows. In Stion, w rviw rlt works. In Stion, w fin th prolm n isuss or onpts of NB. In Stion 4, w prsnt our lgorithm. W vlut our mtho in Stion 5. Stion 6 onlus th ppr.. RELATED WORKS In t strm mngmnt [], sliing-winow join is propos to nswr quris involving th join of multipl t strms, suh s th join siz, sum [][0], join-istint [5]. Thir fous ws on how to omput ths join rsults unr rsour onstrints n us thniqus suh s smpling [7][4] or lo-shing [6][0][]. Howvr, in th join strm lssifition prolm, s w xplin in th prvious stion, it is unsirl to first omput th join of multipl strms n thn uil th lssifir. Thus ths thniqus nnot ppli. Most strm mining lgorithms onsir singl strm n simpl sttistis suh s vrg n stnr vition. lssifition on t strms ws onsir in [][4][4][5]. Othr mining prolms tht involv multipl strms r lustring [8][4], orrltion nlysis [5], squntil pttrns [8]. Howvr, non of ths works involvs gnrl join mong strms; thus, thy o not l with th low-up of t rrivl rts us y mny-to-mny join. For xmpl, th orrltion nlysis in [5] omputs th orrltion offiint of two tim sris, whih ligns th two strms y ommon ky suh s th timstmp. To our knowlg, th lssifition prolm ovr gnrl sliing-winow join hs not n stui. On th othr hn, thr hv n works on lssifition ovr multipl stti rltions. For xmpl, [] prsnt multirltionl ision tr, [0] n [5] stui rul inutions. In rltionl lrnr, th trining st nnot fin y th join of multipl tls, mking th prolm vry iffrnt from ours. In [8], sur onstrution of ision tr lssifirs from vrtilly prtition t ws prsnt, whr th join is givn y th on-to-on rltionship impli y th ommon ky intifir for ll prtitions. Tht work is not pplil to th gnrl mny-to-mny join rltionship. Rntly, [7] propos sur onstrution for ision tr lssifirs ovr istriut tls with th gnrl mny-to-mny join rltionship. Nvrthlss, th work fouss on th privy prsrving spt, not th t strm rquirmnt. [9] propos n ffiint lgorithm for uiling ision tr lssifirs for th t givn y olltion of tls rlt y hirrhil strutur of forign ky rfrns, motivt y thos foun in rltionl tss, t wrhouss, XML t, n iologil tss. It xmins th sm prformn issu us y th low-up of join. Unlik [9], this work ls with th ritrry join rltionship tht is not nssrily rprsnt y forign ky rfrns, n fouss on strm t.. PROBLEM STATEMENT In this stion, w formlly fin th prolm of join strm lssifition n rifly rviw th stnr nïv Bysin lssifition.. Join Strm lssifition Join strm lssifition rfrs to th prolm whn lssifition involvs svrl rlt t strms S,, S n, n th lssifir ns to uilt on th join strm s fin y sliing-winow join. Th spifition of th sliing-winow join ovr S,, S n inlus th join onition, th winow spifition n upt frquny of h input strm [][7]. W onsir th join onition tht is onjuntion of qulity prits S i.a=s j.b (i j), whr S i.a n S j.b, ll join ttriuts, rprsnt on or mor ttriuts from S i n S j. By llowing S i.a n S j.b to ontin mor thn on ttriut, w n t most on prit S i.a=s j.b twn h strm pir S i n S j. Th join grph is onstrut in wy suh tht thr is n g twn S i n S j if thr is prit S i.a=s j.b in th join onition. W onsir onnt n yli joins, tht is, th join grph is wll onnt n ontins no yl. This is not srious limittion sin mny joins in prti r in ft yli, i.., hin joins n str joins ovr th ommonly us str/snowflk shms. Th winow n upt spifition in join strm lssifition n tim-s or tupl-s. Th trm winow rfrs to th olltion of urrnt winows of ll strms. On of S,,S n, ll th trgt strm, ontins th lss olumn. Th tsk is to uil lssifir h tim th winow upts. This mns tht th lssifir must ruilt whnvr th winow on ny input strm slis forwr. Th sp of fstst-sliing winow trmins th rt of lssifir upts. For th urrnt winow, P( x ) = P( x ) i j=.. n j i th trining st is th st of tupls fin y th sliing-winow join. Not tht th trining st is not xpliitly givn, ut spifi y th input strms n th sliing-winow join. This istintion is importnt us th trining st is signifintly lrgr thn

4 th input strms n ing l to work on th lttr irtly hs prformn vntgs.. Nïv Bysin lssifirs onsir singl tl T (X,, X n, lss), whr lss nots th lss olumn whos omin is [,, m ]. To lssify tupl x=(x,, x n ), th Nïv Bysin lssifir (NB) ssigns x to lss i tht mximizs th onitionl lss proility P( i x) s on th following mximum postriori (MAP) hypothsis: rgmxp( i lss i x) = rgmxp( x ) P( ) i lss ls ount Strm S i J i J whr P( i ) is th lss proility n P(x i ) is th onitionl proility of x givn th lss ll i. Unr th ssumption tht vrils X,, X n r inpnnt givn th lss ll, NB stimts P(x i ) y On P(x j i ) n P( i ) r ollt from th trining t, NB is l to ssign lss ll to nw tupl x. To omput P(x j i ) n P( i ), w only n to omput th lss ount mtrix of th form (x k, <N,, N m >) for h istint vlu J lsagg <,0> <0,> <,0> Summry from S to S ls <,0> <0,> <,0> ount Strm S J J J ountagg Summry from S to S ls <,0> ount <0,> <,0> Strm S lss J ls ount Strm S J ls <,0> <0,> <,0> ount Strm S lss J ls ount Strm S J Figur. Exmpl with strms t initiliztion Figur. Aftr ottom-up propgtions x k of X j, whr N j ( j m) is th numr of tupls tht hs th vlu x k n th lss ll j. This t strutur hs siz proportionl to th numr of istint vlus in X j. Not NB rquirs ttriuts to tgoril (hving smll numr of istint vlus). ontinuous ttriuts n first isrtiz (suh s qui-with or qui-pth inning) into smll numr of intrvls for pplying NB. Th ov isussion ssums singl tl T. For join strm lssifition, T will th join rsult of th input strms in th urrnt winow. Sin suh T is muh lrgr thn th input strms, th hllng is to omput th lss ount mtrix for T without gnrting T. In th nxt stion, w prsnt suh mtho. 4. OUR APPROAH W ssum th urrnt winows of ll input strms r hl in mmory. Th join rltionships mong strms form n yli join grph, whih is in ft root tr. Any strm my rgr s th root n th trgt strm my t ny position in th tr. As our mtho involvs propgtion of informtion long th gs of th tr, w will ll this tr s th propgtion tr. Inst of gnrting th join strm, w mintin t strutur of (ls, ount) for h tupl t in th input strms, whr ls is lss vtor in th form of <N,, N m > (m is th numr of lsss, N i rors th ount of ourrns of t with lss ll i in th join strm) n ount is ountr tht kps trk of th totl ourrns of t in th join strm. Not this t strutur stors ll informtion out tupl t in th join strm, ut with th siz proportionl to th siz of th input strm. Th lss vtors (ls) orrspon to th lss ount mtrix s mntion in stion., whih is th only informtion rquir to uil NB lssifir. Th hllng is how to omput suh lss vtors for h tupl in th input strms without prforming th join. W omput th lss vtors y propgting th low-up fft of th join. Th propgtion pros in two phss. In th phs of ottom-up propgtions, ll ls s n ounts r propgt from th lf nos to th root. On rhing th root, th ls s in th root rflt th join of ll strms. Nxt, in th phs of topown propgtions, w propgt ls s from th root to ll lf nos. Whn rhing ll lf nos, th lss vtors (ls s) in h strm hv rflt th join fft of ll strms. Our min hivmnt lis in th ft tht omputing th low-up rtios through propgtions is muh hpr thn omputing th sliingwinow join itslf. Th til pross of omputing suh t struturs is xplin in th following sustions. To orrtly rry out omputtions uring propgtions, w first fin th rithmti oprtions on ls s follows: givn n oprtor n two ls s (V : <,, m > n V : <,, m >), V V = <,, m m >. For xmpl, <4,>/<,>=<,>. 4. Initiliztion Initilly, for ny tupl in th trgt strm, its ls (<N,, N m >) is trmin y its lss ll i suh tht, N i = n N j =0 whr i j. Th ount is lwys initiliz to.

5 For ny tupl in ll othr strms, its ls is initiliz to ll zros (<0,,0>) n ount is lwys. Figur givs n xmpl with strms with initil ls s n ounts vlus shown. Th join rltionships r spifi y th rrows: S n S join on J, n S n S join on J. S is th trgt strm ontining two lsss. S is th root of th propgtion tr. Not th root of th tr n ritrrily slt. W will show ltr tht hoosing th input strm with lrgst winow siz (i.. th most numr of tupls in winow) s th root n optimiz th ost of sn on input strms. Not tht suh initiliztion os not rquir sprt sn on th input strms n n omin with th ottom-up propgtion pross s isuss in th following sustion. 4. Bottom-Up Propgtion This is th phs whr th informtion of ls n ount r propgt from lf nos to th root in ottom-up orr. onsir prnt no P n hil no with join prit P.J =.J. Osrvtion : Givn tupl t in P, if t joins with k tupls in, t will our k tims in th join twn P n. Ths ourrns n rflt irtly in P y lowing up th ls n ount of t using th ggrgt ls n ount, not s lsagg n ountagg, of th k joining tupls in. Formlly, w fin low-up summry from to P s th st {(v, lsagg, ountagg)}, whr v is join vlu in. It hs th siz tht is proportionl to th numr of istint join vlus in n n ollt y on sn of. Osrvtion : If P hs n hil nos (n>), th ls n ount of t in P will lown up y ll hilrn s in Osrvtion, to rflt th join with ll hilrn strms. Not in this phs, s w propgt only in th ottom-up orr, t most on of th hilrn ontins non-zro lsagg s (th rnh ontining th trgt strm). Th following lmm follows from th ov osrvtions: Lmm. With prnt no P n its n hil nos, for h tupl t in P with join vlus (v,,v n ), whr h v i orrspons to th join ttriut twn P n th ith hil, lt (v, lsagg, ountagg ),, (v n, lsagg n, ountagg n ) not th n orrsponing summry ntris from ll hilrn, t s ount is lown up s: t. ount = ( ountagg j ); j=.. n if thr s som ntry lsagg i ( i n) ing non-zro (y Osrvtion, thr s t most on suh ntry), t s ls is upt s: t ls = ( lsagg i )* ( ountagg j ) j=.. n, j i Now w n propgt th low-up summris from hil nos to th prnt no P. Aftr riving ll hilrn s low-up summris, w sn P on n upt its ount s n ls s s in Lmm. W lso rt th low-up summry from P to its own prnt (if ny) in th sm sn. Figur shows th ottom-up propgtion following th xmpl in Figur, whr S n S r snn to prou low-up summris to propgt to S. Not trivil ggrgt ounts (lsagg is ll zros or ountagg=) r ignor n not shown in summris. On riving th summris, S lows up ls n ount of its tupls. For xmpl, onsir th tupl t in S s grysl in Figur (with J =, J =). It hs two orrsponing summry ntris: (,<0,>,) from S n (,,) from S, h ontining ll informtion on t s joining tupls in tht hil. Th low-up of t y ths ntris in ft rprsnts th fft of join: t.ount is lown up y *=, t.ls is lown up y <0,>*=<0,>. Ths rsults init tht t ours in th join twi, oth hving th lss ll, whih is xtly th sm informtion s in th join strm. In gnrl, th trgt strm n nywhr in th tr, thus thr r two ss in th ottom-up propgtion from hilrn to prnt no P: - If th trgt strm is not in P s sutr, w low up only ount t P sin lsagg is lwys mpty; ls will lown up ltr t som nstor no of P; - If th trgt strm is in P s sutr, w low up oth ls n ount t P; in this s ithr P is th trgt strm or on of its hil nos hs non-mpty lsagg s. 4. Top-Down Propgtion At th n of ottom-up propgtion, th ls in th root strm rflts th fft of join of ll strms. Howvr, th ls s in ll othr strms hv not rflt th joins prform t thir nstors. Thus w n to propgt in th top-own fshion to push th orrt join informtion to ll non-root strms. Th propgtion is s on th following osrvtions. Osrvtion : For prnt no P n hil no, if tupl t in joins with som tupl in P tht hs th join vlu v, so o ll tupls in tht hv this join vlu v. W n viw ll suh tupls s n quivln lss on th join vlu v in, not s [v]. Similrly, P[v] is fin s th orrsponing tupls in P tht shr th sm join vlu v. Th ls s of [v] tupls must upt y ristriuting th ggrgt ount of P[v] tupls with following two proprtis: () th shr of ny tupl in its own quivln lss rmins onstnt; () th ggrgt ounts in ftr ristriution must th sm s in P. Thus, to prform th top-own propgtions proprly, w fin th istriution summry from P to s th st {(v, lsagg)}, whr v is join vlu in P n lsagg is ggrgt lss ounts of ll P[v] tupls. Not thr s on istriution summry from th prnt to h hil, with th siz proportionl to th numr of istint vlus of h join ttriut. Lmm. Givn istriution summry ntry (v,.lsagg) from P, w ristriut th lsagg mong [v] tupls suh tht, for ny tupl t in [v]:. t.ls= lsagg * (t.ount / [v].ountagg) whr [v].ountagg is th ggrgtion of ount s of ll [v] tupls prior to ristriution n s suh, (t.ount / [v].ountagg) rprsnts t s shr in [v]. Hn, on riving th istriution summry from P, th ls s in r upt s in Lmm, whrs th istriution summry from to its own hilrn (if ny) r omput in th sm sn. Figur shows th top-own propgtion. At th root S, th istriution summris to S n S r gnrt whil snning

6 S in th ottom-up propgtion. On riving ths summris, S n S ristriut thir ls s. For xmpl, for th tupl t in S s grysl in Figur (with J =), t.ls=<0,> is ristriut y <0,>*(/)=<0,>, whr (,<0,>) is th summry ntry orrsponing to, n (/) is th shr of t in its own quivln lss (hving J =). Th rsult pturs xtly th sm informtion out t s in th join strm: t ours twi hving th lss ll. J lsagg <,0> <0,> <,0> Summry from S to S ls <,0> <0,> <,0> ount Strm S ls ount <,0> <0,> <,0> Strm S lss J J J ls <,0> <0,> <0,> J ount Strm S Figur. Aftr top-own propgtions lsagg <,0> <0,> Summry from S to S 4.4 ost Anlysis In th ottom-up n top-own propgtion, on summry is pss twn h prnt/hil pir n h strm (winow) is snn on. At ny tim, only th summris for urrnt prnt/hilrn r kpt in mmory. Th siz of summry is proportionl to th numr of istint join vlus, not th numr of tupls. A summry lookup oprtion tks onstnt tim in n rry or hsh tl implmnttion. Thrfor, th whol lgorithm is linr in th strm winow siz, inpnnt of th join siz. This proprty is importnt us th join siz n ritrrily lrg ompr with th winow siz, u to th mnyto-mny join rltionships. Th lgorithm sns h input strm twi, on t th ottomup propgtion phs n on t th top-own propgtion pross. Th only xption is th root strm, whr th ottomup n top-own propgtions mt, two sns n omin into on. Thrfor, hoosing th input strm of th lrgst winow siz (i.., th most numr of tupls) s th root will minimiz th ost of sns, s it svs on sn on th lrgst strm winow. 5. EMPIRIAL STUDIES Th ojtivs of our vlutions r two-fol: to vrify tht th lssifir uilt on th join strm is mor urt ompr with tht uilt on singl strm; n to stuy th slility of our lgorithm. W not our lgorithm s, s it uils NB lssifir whos trining st is fin on th join of multipl strms. Not our lgorithm os not n to tully prform th join; inst, J th lssifition pross is prform irtly on th input strms. W ompr it with following ltrntivs: - NB_Trgt: NB s on th trgt strm lon. In this s, ll non-trgt strms r ignor. - DT_Join: ision tr lssifir (4.5) on th join strm. To uil th ision tr, th join strm is first omput y tully joining th input strms. - DT_Trgt: ision tr lssifir on th trgt strm lon. To ompr ury rsults, for h winow, w trin th lssifir on th first 80% of t tupls within this winow. Th rmining 0% of t tupls in th sm winow r kpt s tsting smpls for tsting th lssifition ury. Not tht th tsting t r gnrt y sliing-winow join on th tsting smpls from ll strms n onstitut on join strm. To ompr th slility, w fous on th lssifirs tht r uilt on th join strm n msur th slility y omputing tim pr input tupl, i.., tim spnt on h winow ivi y th numr of tupls in th winow. It givs n i out th t rrivl rt tht n lgorithm is l to hnl. For DT_Join, us it hs to gnrt th join strm for uiling th lssifir, w msur th join tim n ignor th lssifir onstrution tim sin th join ost is th most xpnsiv prt. Most of sli-winow join lgorithms in litrtur r not suitl for gnrting th join strm for DT_Join us thy fous on fst omputing spil ggrgts [0][5], or prouing pproximt join rsults [] unr rsour onstrints; not th xt join rsult. Thrfor, w hv to implmnt th join lgorithm for sli winow join. For simpliity, w implmnt th nst loop join lgorithm. This hoi shoul not hv mjor prformn fft us ll tupls in th urrnt winow r kpt in mmory. All progrms wr o in ++ n run on P with GHz PU, 5M mmory n Winows XP. 5. Rl-lif Dtst For xprimnts on rl-lif tst, w otin UK ro int t from th UK t rhiv. It ollts informtion out ints, vhils n sultis, in orr to monitor ro sfty n trmin poliis to ru th ro int sulty toll. It ontins thr tls: Aint, Vhil n sulty. Th hrtristis of yr-00 t r shown in Figur 4 whr rrows init join rltionships: h int involvs on or mor vhils; h vhil hs zro or mor sultis. Eh tl n rgr s strm tht is timstmp y t of int. In vrg, out 600 Aint tupls, 700 Vhil tupls n 850 sulty tupls r vry y. Th join strm is spifi y qulity join on th ommon ttriuts mong th strms. sulty is th trgt strm with two sulty lsss --- lss : ftl/srious (% of ll tupls) or lss : slight (87% of tupls).

7 Figur 4. UK ro int t (00) 5.. Aury Figur 5 shows uris of ll lssifirs ing ompr. For ll mthos, th winow siz is th sm n rngs from 0 to 50 ys with no winow ovrlpping. Avrg ury (09 tupls) (siz: 4.MB) sulty AS_ID S_9 (lss) VEH_ID (7409 tupls) (siz: 9.6MB) DT_Join Vhil VEH_ID A_ID NB_Trgt DT_Trgt Winow siz (ys) Figur 5. lssifir ury (5069 tupls) (siz: 0.6MB) Aint A_ID It is immitly lr tht lssifirs uilt on multipl strms r muh mor urt, showing tht xmining orrlt strms is vntgous ompr with uiling th lssifir on singl strm. In ft, th ury otin y xmining th trgt strm lon is only out 80%, vn lowr thn tht otin y nïv lssifir whih simply lssifis vry tupl s longing to lss, sin 87% of tupls long to this lss. On th othr hn, th rsults lso show tht, with th sm trining st, nïv Bysin lssifir hs omprl prformn s ision trs. Kp in min tht our mtho runs irtly on th input strms, whil th ision tr is uilt on th join strm n thus is sujt to th join ost. W xmin th ffiiny of ths two mthos in th nxt st of xprimnts. 5.. Tim pr input tupl Figur 6 omprs th tim pr input tupl. For xmpl, t th winow siz of 0 ys, th join tks out 9.8 sons whrs tks only out 0. sons. Thrfor, th join tim pr input tupl is 9.8*0 6 /4,900=4 mirosons, whr 4,900 is th totl numr of tupls tht rriv in th 0- y winow. In ontrst, tks only 0.*0 6 /4,900=6.8 mirosons pr input tupl. Thus, ny mtho tht rquirs omputing th join will t lst tims slowr thn our mtho. As th winow siz inrss, th join tim inrss quikly u to th inrs join rinlity in lrgr winow; whrs th tim pr input tupl for is lmost onstnt, initing tht our pproh is linr to th winow siz. Thus our mtho n hnl muh highr sp of winow sliing thn onvntionl mthos. Thrfor, whil oth n DT_Join lssifirs xhiit similr lssifition uris, is muh mor ffiint thn DT_Join. Tim pr tupl (ms.) Join Tim Winow siz (ys) Figur 6. Tim pr input tupl 5. Synthti Dtsts To furthr vrify our lims, w lso us synthti tsts with vrious t hrtristis. Similr to th xprimnts on rl-lif tsts, w wnt to xmin whthr th orrltion of multipl strms yils nfits for lssifition unr iffrnt t hrtristis. W lso wnt to vlut if n l with strms with high t rrivl rts. As w r not wr of n xisting t gnrtor to vlut lssifition spnning svrl rlt strms, w sign our own t gnrtor. 5.. Dt Gnrtor To fous on importnt t hrtristis, w mk som simplifying ssumptions. W onsir th hin join of k strms S,, S k, whr S is th trgt strm, n h jnt pir S i n S i+ hv on join prit. All join ttriuts r tgoril n hv th sm omin siz D. All strms hv th sm numr of tupls S. All strms hv N numril n N tgoril ttriuts (xluing join ttriuts n th lss ttriut). All numril ttriuts hv th rnk omin {,,0}, i.., vlus r trt s ing isrtiz into 0 tgoril intrvls. tgoril vlus r rwn rnomly from omin of siz 0. To vrify our lim tht lssifirs uilt on th join strm r mor urt whn thr r orrltions mong strms, w n th tst to ontin rtin strutur rthr thn rnomly gnrting th t tupls. To this n, w onstrut tsts suh tht th lss ll in th join strm is trmin y whthr t lst q numril ttriuts hv high vlus, whr q is prntg prmtr. A numril vlu is high if it longs to th top hlf of its rnk omin. Sin th numril ttriuts r istriut mong multipl input strms, to nsur th

8 sir proprty on th join strm, th input strms S,,S k r onstrut s follows. - Join vlus. Eh strm S i onsists of D groups: from st to Dth group. All tupls in th jth ( j D) group of S i join with ll tupls in th jth group of S i+, ut not ny othr tupls. Th jth join group rfrs to th st of join tupls prou y th jth groups. Th siz Z j of th jth group is th sm in ll strms S,,S k, n follows Poisson istriution with th mn λ= S /D. Th jth join group hs th siz Z j k, with λ k ing th mn. Th low-up of th join is fin s λ k /λ=λ k-, i.., th rtio twn th mn of group siz on th join strm n tht on input strms. - Numril vlus. W gnrt th numril ttriuts suh tht ll join tupls in th jth join group hv th sm lss ll, y hving high vlus in th sm numr of numril ttriuts, sy hj. To nsur this proprty, w istriut th numr hj mong S,,S k rnomly, sy hj,,hj k, suh tht hj =hj + +hj k, n ll tupls in th jth group for S i r high in hj i numril ttriuts. hj follows uniform istriution in th rng [0,k*N], whr k*n is th totl numr of numril ttriuts. - lss lls. If hj q*k*n, for som prntg prmtr q, w ssign th Ys lss ll to vry tupl in th jth group of S, othrwis, ssign th No lss ll. Finlly, to simult th onpt rifting in t strms, w hng th lss istriution vry tim ftr gnrting W tupls. This is on y vrying th prmtr q: lt w th winow siz, for vry W tupls (W>>w), w rnomly trmin q vlu in th rng [0.5, 0.75) following th uniform istriution. Thus tst gnrt s ov n hrtriz y th st of prmtrs (N, S,D,λ,W), whr λ= S /D is th mn of group siz n trmins th low-up rtio of join. 5.. Aury W gnrt thr strms S, S n S with prmtrs N=0, S =,000,000, D=00,000, λ=5, W=00,000. Figur 7 shows th ury rsults with 50% winow ovrlpping. DT_Join n r mor urt thn thir ountrprts on th singl strm, whil oth hving similr uris. Figur 8 shows nothr xprimnt, whr w fix th winow siz w t 0,000 n rs W from 00,000 to 0,000, in orr to simult situtions whr lssifition pttrns hng mor frquntly. Sin th prvious xprimnts hv onfirm tht lssifirs uilt on th join strm hv ttr uris, in this xprimnt, w only show th ury rsults of n DT_Join. As xpt, th ury rops slowly s W rss, sin thr r mor winows spnning t with iffrnt hrtristis, mking it iffiult for lssifir to orrtly intify th lssifition pttrn. Avrg ury Avrg ury DT_Join Winow siz('000 tupls) Figur 7. lssifir ury NB_Trgt DT_Trgt W ('000 tupls) DT_Join Figur 8. lssifir ury with hnging t pttrns 5.. Tim pr input tupl Figur 9 shows th tim pr tupl on th sm tst s in Figur 7. Th join tim is muh lrgr thn th tim of. As th winow siz inrss, th join tim inrss u to th low-up fft of join, whil spns lmost onstnt tim pr tupl for ny winow siz. Figur 0 shows th tim pr tupl vs. th low-up of join. All prmtrs r th sm s prvious xpt λ. For th join of thr strms, th low-up rtio is λ. By vrying λ from to 7, th low-up vris from 4 to 49. Th winow siz is fix t 0,000. Agin, shows muh ttr prformn n is flt with rspt to th low-up of join. This is us it sns th winow xtly twi, inpnnt of th low-up rtio of th join. On th othr hn, th join tks mor tim pr tupl with lrgr lowup rtio us muh mor tupls r gnrt. Figur shows th tim pr tupl vs. th numr of strms. All prmtrs r still th sm s in Figur 9. Th winow siz is fix t 0,000 tupls. W vry th numr of stms from to 5. Th low-up rtio for k-strm join is trmin y 5 (k-). Th omprison of th rsults is similr to Figur 0.

9 Tim pr tupl ( mirosons) Tim pr tupl ( mirosons) Tim pr tupl ( mirosons) Join T im Winow siz('000 tupls) Figur 9. Tim pr input tupl vs. winow siz Join T im Blowup Figur 0. Tim pr input tupl vs. low-up rtio Numr of strms Join T im Figur. Tim pr input tupl vs. numr of strms 5. Disussion On oth th rl lif n synthti tsts, our mpiril stuis show tht whn th fturs for lssifition r ontin in svrl rlt strms, th propos join strm lssifition hs signifint ury vntg ovr th onvntionl mtho of xmining only th trgt strm. Thus lssifition lgorithm shoul xmin s muh rlt informtion s possil. Th min hllng is how suh lssifition n prform in p with th high-sp input strms, givn tht th join strm hs n vn highr t rrivl rt thn tht of th input strms, u to th ritrry join low-up rtios. To this n, our xprimnts show tht our propos lgorithm hs th ost linr to th siz of input strms, inpnnt of th join siz. Thus our lgorithm is sll n suprior to ll othr ltrntiv mthos. It is worthy of not tht th lssifir must ruilt h tim th winow on ny input strm slis forwr. This is rsonl whn thr is no ovrlp or only smll ovrlps twn sli winows. Howvr, whn winows r signifintly ovrlpp, this strtgy tns to rpt th work on th ovrlpp t. In this s, mor ffiint strtgy my inrmntlly upting th NB y working only on th iffrn u to th winow sliing. W i not pursu in this irtion furthr us vn ovrlpp tupls still n to join with nw tupls in th othr strms, whih mns tht th sn of ovrlpp tupls nnot voi. Sin our lgorithm sns th urrnt winow only twi, th nfit of ing inrmntl is limit, spilly onsiring th ovrh. 6. ONLUSIONS Rl lif lssifition oftn involvs multipl rlt t strms. Du to th onlin n high volum ntur of t strms, with th join tht lows up th t rrivl rt on top of th rpily rriving strms, it is prohiitiv to prform th sliingwinow join n thn onut lssifition nlysis on th join. To solv this prolm, w xplor th proprty of Nïv Bysin lssifirs n propos novl thniqu for rpily otining th ssntil join sttistis without tully omputing th join. With this thniqu, w n uil xtly th sm Nïv Bysin lssifir s using th join strm, ut with prossing ost tht is linr to th siz of th input strms n inpnnt of th join siz. Empiril stuis support our two lims: xmining svrl rlt strms in nfits th qulity of lssifition; n th propos mtho hs muh lowr prossing tim pr input tupl, thus, is l to hnl muh highr t rrivl rts, vn in th prsn of mny-to-mny join rltionships. 7. REFERENES [] Nog Alon, Phillip B. Gions, Yossi Mtis, n Mrio Szgy. Trking Join n Slf-Join Sizs in Limit Storg. In AM PODS, Philplphi, Pnnsylvni, 999. [] A. Atrmntov, H. Liv n V. Honvr, A multi-rltionl ision tr lrning lgorithm implmnttion n xprimnts. ILP 00. [] B.Bok, S. Bu, M. Dtr, R. Motwni, J. Wiom. Mol n issus in t strm systms. In AM PODS, Mison, Wisonsin, 00. [4] J. Bringr n E. Hullrmir. Onlin lustring of prlll t strms. In prss for Dt & Knowlg Enginring, 005. [5] Y. D. i, D. luttr, G. Pp, J. Hn, M. Wlg n L. Auvil. MAIDS: Mining lrming inints from t strms. In Pro. SIGMOD, monstrtion ppr, 004.

10 [6] D. rny, U. tintml, M. hrnik,. onvy, S. L, G. Simn, M. Stonrkr, N. Ttul, n S. Zonik. Monitoring strms - nw lss of t mngmnt pplitions. In Pro. VLDB, 00. [7] S. huhuri, R. Motwni, n V. R. Nrsyy. On rnom smpling ovr joins. In Pro. SIGMOD, 999. [8] G. hn, X. Wu, X. Zhu. Squntil pttrn mining in multipl strms, In Pro. IDM, 005. [9] A. Ds, J. Ghrk n M.Riwl. Approximt join prossing ovr t strms. In Pro. SIGMOD, Mison, Wisonsin, 00. [0] Alin Dor, Minos Groflkis, Johnns Ghrk, n Rjv Rstogi. Prossing omplx ggrgt quris ovr t strms. In Pro. SIGMOD, Mison, Wisonsin, 00. [] P.Domingos n G. Hultn. Mining high-sp t strms. In Pro. SIGKDD, 000. [] Pro Domingos n Mihl Pzzni. On th optimlity of th simpl Bysin lssifir unr zro-on loss. Mhin Lrning, 9:0-0, 997. [] R. O. Du n P. E. Hrt. Pttrn lssifition n sn nlysis. Nw York: John Wily & Sons, 97. [4] J. Gm, R. Rh, P.Ms. Aurt ision trs for mining high-sp t strms. In Pro. SIGKDD, 00. [5] S. Gnguly, M. Groflkis, A. Kumr n R. Rstogj. Joinistint ggrgt stimtion ovr upt strms. In Pro. AM PODS, Bltimor, Mryln, 005. [6] J. Ghrk, R. Rmkrishnn n V. Gnti. Rinforst A frmwork for fst ision tr onstrution of lrg tsts. In Pro. VLDB, Sn Frniso, 998. [7] L. Gol, M. Tmr Ozsu. Prossing sliing winow multijoins in ontinuous quris ovr t strms. In Pro. VLDB, 00. [8] S. Guh, N. Mishr, R. Motwni, n L. O llghn. lustring t strms. In FOS, 000. [9] DJ. Hn n K. Yu, Iiot's Bys - not so stupi ftr ll? Intrntionl Sttistil Rviw. 69(), 85-99, 00. [0] G. Hultn, P. Domingos n Y. A, Mining mssiv rltionl tss, 8th Intrntionl Joint onfrn on AI - Workshop on Lrning Sttistil Mols from Rltionl Dt, Apulo, Mxio, 00. [] J. Kng, J. Nughton, S.Vigls. Evluting winow joins ovr unoun strms. In Pro. IDE, 00. [] Irin Rish. An mpiril stuy of th niv Bys lssifir. IJAI 00 Workshop on Empiril Mthos in Artifiil Intllign, 00. [] U. Srivstv, J. Wiom. Mmory-limit xution of winow strm joins. In Pro. VLDB, 004. [4] H. Wng, W. Fn, P. Yu n J. Hn. Mining onpt-rifting t strms using nsml lssifirs. In Pro. SIGKDD, Wshington D, USA, 00. [5] X. Yi, J. Hn, J. Yng, n P. Yu. rossmin: ffiint lssifition ross multipl ts rltions. IDE 004. [6] Y. Zhu n D. Shsh. Sttstrm: Sttistil monitoring of thousns of t strms in rl tim. In Pro. VLDB, 00. [7] K. Wng, Y. Xu, R. Sh, P. Yu. lssifition Spnning Privt Dtss. AAAI, 006. [8] W. Du n Z. Zhn. Builing ision tr lssifir on privt t. IDM Workshop on Privy, Surity n Dt Mining, 00 [9] K. Wng, Y. Xu, P. S. Yu, n R. Sh. Builing ision trs on rors link through ky rfrns. SIAM Intrntionl onfrn on Dt Mining (SDM), 005.