Real-time Classification of Large Data Sets using Binary Knapsack

Transcription

1 Real-me Classfcao of Large Daa Ses usg Bary Kapsack Reao Bru Uversy of Roma La Sapeza AIRO h ANNUAL CONFERENCE OF THE ITALIAN OPERATIONS RESEARCH Sepember 7-0, 004, Lecce, Ialy

2 Oule The Daa Classfcao Problem Barzao of Daa Records Evaluao of Alerave Barzaos Seleco of Bary Arbues Compuaoal Resuls

3 Daa Classfcao Fdg models ha descrbe ad dsgush classes or coceps for fuure predco More specfcally: gve a rag se S of daa already paroed classes S ad S -, predc whch class each ew daa belogs o A supervsed learg problem Wh he feld of daa mg : exraco of eresg o-rval, mplc, prevously ukow ad poeally useful formao or paers from daa large daabases We deal wh massve daa ses wh real me requremes

4 Daa Records Daa are srucured o records. A record scheme s a se of felds R = { f f m } A record sace s a se of values r = { v v m } Each feld f has s doma D : he se of every possble value v, cludg errors. Example: for records represeg persos, feld ca be age, maral saus, correspodg values ca be 8, sgle, correspodg domas ca be Z U {blak}, {sgle, marred, separae, dvorced, wdow, blak}.

5 Felds ad Arbues Felds ca be: umercal or quaave caegorcal or qualave couous: real-valued dscree: eger or bary ordered o ordered Geerally, classfcao procedures requre a coverso of all felds f o bary oes. They wll here be called arbues a {0,} f {a a } R b = {a a a m a m m }

6 Barzao A basc ad maly used barzao procedure s he dervao of cu-pos e.g. LAD [Boros-Hammer-Ibarak-Koga]: gve r r such ha her values o feld f are separaed by o oher record, D r r - derve a cu-po α = [v r v r - ] / Cu-po are used o geerae arbues: above or below α May cupos are obaable. For each group of hem, we have a barzao. May alerave barzaos are possble Selecg he bes oe s a combaoral opmzao problem

7 Example S S - Wegh Hegh Class yes yes yes o o Classfcao baskeball players ad o baskeball players Wegh Hegh Possble Barzaos: usg all cu pos bad 7.5, 0.5 ad ad 0.5 4

8 Evaluao of Cu-pos We vesgae a crero for evaluag he qualy of a barzao wh a fas procedure. We evaluae each sgle cupo usg he rag se. They ca be dffere suaos: dsrbuo of dsrbuo of - a α a α a 3 α a 4 α a 5 α a 6 α a dsrbuo of dsrbuo of - b α b α b 3 α b 4 α b 5 α b dsrbuo of dsrbuo of - c α c

9 Probably Issues The odds of gvg correc posve [egave] classfcao usg α q = l Pr class α Pr class α. Pr class α Pr class α [0, We wa a evaluao of each cu-po ha ca be summed The probably of a couco of eves s a produc Therefore, we cosder he logarhm obag a sum Le N N- be he real ukow classfcao ad A A- be he supposed classfcao kow By defo of probably, N A Pr class α = N Therefore, = l N N A q. A N N A A

10 Felds wh Normal Dsrbuo We do o kow N N-, bu for felds wh a Normal Gaussa dsrbuo we guess were hey are by usg S ad S- as follows: We hypohesze hs for all couous felds ad dscree felds wh large umber of values hypohess s esable We compue mea value m m - ad devao σ σ - from S S- ad for a raso from o we have: = d e d e d e d e q m m m m α σ α σ α σ α σ σ π σ π σ π σ π. l

11 Felds wh Bomal Dsrbuo We do o kow N N-, bu for felds wh a Bomal Beroull dsrbuo we guess were hey are by usg S ad S- as follows: We hypohesze hs for all dscree felds ad ordered caegorcal felds hypohess s esable We compue umber of values - ad probably of success p p - from S S- ad for a raso from o we have: = = = = = 0 0!!!!!!.!!!!!! l m m m m p p p p p p p p q α α α α

12 Felds wh Ukow Dsrbuo For felds havg ukow dsrbuo: caegorcal felds, or felds where oher hypohess are o verfed, we smply replace N N- wh S S-, ad compue he qualy as follows = l S A q. S A S S A A Whe o dsrbuo hypohess ca be doe, we are fac uable o guess were he posve ad egave pos o he rag se should be

13 The Kapsack Model Now we eed o choose he bes barzao We assocae o cu-pos bary varables x = f α s used 0 oherwse I early real-me applcao, we ca compue he umber b of arbues we ca deal wh reasoable me max, p, x q x b x {0,} If all p are, hs kapsack becomes a easy problem: a greedy heursc fds he opmal soluo

14 Classfcao Oce daa are barzed, he acual classfcao sep s performed usg he followg weghed sum, where P s he se of arbues gvg a posve classfcao N s he se of arbues gvg a egave classfcao r =, P w a r, N w a r 0 r s classfed < 0 r s classfed - weghs w for he arbues are posve [egave] values proporoal o he cardaly of he par of S [ S-] coaed such arbues

15 Compuaoal Resuls The algorhm s mplemeed C ad esed o he larges daases wh bary classfcao of he UCI reposory: hp:// spam e-mal, adul, germa cred, musk, pma das Tess usg 0%, 5% ad 30% of daase as rag se Each resul s average o 5 rals wh radom seleco of he rag se

16 Spam E-mal Daase Classfy emal spam or o: 460 records each havg 58 umercal felds 55 real dscree 0% 5% 30% Accuracy Tme sec Bes leraure: comparable 97 98% wh much larger rag se 50% ad more me ~0x

17 Adul Daase Decde wheher aual come > 50,000 $ : 45 mssg removed records each havg 5 felds 6 real 8 caegorcal 0% 5% 30% Accuracy Tme sec Bes leraure: moderaely beer 85 86% wh much larger rag se 75% ad more me ~0x

18 Salog - Germa Cred Daase Classfy good ad bad credors: 000 records each havg 0 felds 7 umercal 3 caegorcal 0% 5% 30% Accuracy Tme sec Bes leraure: moderaely beer 75% wh much larger rag se 60% ad much more me ~50x

19 Musk clea Daase Classfy molecules musk or o: 6598 records each havg 67 real felds 0% 5% 30% Accuracy me Bes leraure: slghly beer 9% wh much larger rag se 50% ad more me ~0x

20 Pma Idas Daase Classfy paes dabec or o: 768 records each havg 8 umercal felds real 6 dscree 0% 5% 30% Accuracy Tme sec Bes leraure: moderaely beer 70 75% wh much larger rag se 50% ad much more me ~00x

21 Coclusos The proposed approach classfes exremely shor mes large daases, obag a good accuracy ad usg very reduced rag ses Compuaoal me may be decded advace, hece he procedure s suable for dealg wh real-me requremes Accuracy creases wh he dmeso of he rag se ul a cera dmeso. Whe furher creasg he rag se, hece he graulary of he barzao, logcal combaos of arbues become useful e.g. LAD