Using DP for hierarchical discretization of continuous attributes. Amit Goyal (31 st March 2008)

Transcription

1 Usng DP fo heachcal dscetzaton of contnos attbtes Amt Goyal 31 st Mach 2008

2 Refeence Chng-Cheng Shen and Yen-Lang Chen. A dynamc-pogammng algothm fo heachcal dscetzaton of contnos attbtes. In Eopean Jonal of Opeatonal Reseach ElseVe.

3 Ovevew What s Dscetzaton? Why need Dscetzaton? Isses nvolved Tadtonal Appoaches DP solton

4 Backgond Dscetzaton edce the nmbe of vales fo a gven contnos attbte by dvdng the ange of the attbte nto ntevals. Concept heaches edce the data by collectng and eplacng low level concepts sch as nmec vales fo the attbte age by hghe level concepts sch as yong mddle-aged o seno.

5 Why need dscetzaton? Data Waehosng and Mnng Data edcton Assocaton Rle Mnng Seqental Pattens Mnng In some machne leanng algothms lke Bayesan appoaches and Decson Tees. Ganla Comptng

6 Dscetzaton Isses Sze of the dscetzed ntevals affect sppot & confdence {Refnd = No Income = $51250} {Cheat = No} {Refnd = No 60K Income 80K} {Cheat = No} {Refnd = No 0K Income 1B} {Cheat = No} If ntevals too small may not have enogh sppot If ntevals too lage may not have enogh confdence Loss of Infomaton How to mnmze? Potental solton: se all possble ntevals Too many les!!!

7 Common Appoaches Manal Eqal-Wdth Patton Eqal-Depth Patton Ch-Sqae Patton Entopy Based Patton Clsteng

8 Smple Dscetzaton Methods: Bnnng Eqal-wdth dstance pattonng: It dvdes the ange nto N ntevals of eqal sze: nfom gd f A and B ae the lowest and hghest vales of the attbte the wdth of ntevals wll be: W = B- A/N. The most staghtfowad Eqal-depth feqency pattonng: It dvdes the ange nto N ntevals each contanng appoxmately same nmbe of samples

9 Ch-Sqae Based Pattonng 2 ch-sqae test χ = Obseved Expected Expected 2 2 The lage the 2 vale the moe lkely the vaables ae elated Mege: Fnd the best neghbong ntevals and mege them to fom lage ntevals ecsvely

10 Entopy Based Patton Gven a set of samples S f S s pattoned nto two ntevals S1 and S2 sng bonday T the entopy afte pattonng s E S T S S S Ent = 1 S S Ent S2 The bonday that mnmzes the entopy fncton ove all possble bondaes s selected as a bnay dscetzaton. The pocess s ecsvely appled to pattons obtaned ntl some stoppng cteon s met

11 Clsteng Patton data set nto clstes based on smlaty and stoe clste epesentaton e.g. centod and damete only Can be vey effectve f data s clsteed bt not f data s smeaed Can have heachcal clsteng and be stoed n mltdmensonal ndex tee stctes Thee ae many choces of clsteng defntons and clsteng algothms

12 Notatons val: vale of th data nm: nmbe of occences of vale val R: depth of the otpt tee b: ppe bonday on the nmbe of sbntevals spawned fom an nteval lb: lowe bonday

13 Example R = 2 lb = 2 b = 3

14 Poblem Defnton Gven paametes R b and lb and npt data val1 val2 valn and nm1 nm2 nmn o goal s to bld a mnmm volme tee sbect to the constants that all leaf nodes mst be n level R and that the banch degee mst be between b and lb

15 Dstances and Volme Inta-dstance of a node contanng data fom data to data ntadst = val x mean * nm x x= Inte-dstance b/w two adacent sblngs; fst node contanng data fom to second node contanng data fom +1 to ntedst = β val = β val + 1 val totalnm + 1 val totalnm + totalnm + 1 = ntedst L + ntedst R + 1 Volme of a tee s the total nta-dstance mns total ntedstance n the tee

16 Theoem The volme of a tee = the nta-dstance of the oot node + the volmes of all ts sb-tees - the nte-dstances among ts chlden

17 Notatons T * : the mnmm volme tee that contans data fom data to data and has depth Tk: the mnmm volme tee that contans data fom data to data has depth and whose oot has k banches D * : the volme of T * Dk: the volme of Tk

18 Notatons Cont. QDk: k-1 v= 1 mn{ k v= 1 the volme of v the ntedstance b/w v th node - th node and v + 1 th node }

19 Notatons Cont. QD M k: k-1 v= 1 mn{ v= 1 the ntedstance b/w v ntedstance k the volme of v R } th node th node and v + 1 th node

20 Algothm } ntedst ntedst 1 1 * mn{ k QD D k QD L R M M + + = <

21 Algothm Cont. QD k = mn{ D * < + QD M + 1 k 1 ntedst L }

22 Algothm Cont. D k = ntadst + QD 1 k

23 The complete DP algothm } ntedst ntedst 1 1 * mn{ k QD D k QD L R M M + + = < } ntedst 1 1 * mn{ k QD D k QD L M + + = < 1 ntadst k QD k D + = } mn { * k D D b k lb =

24 Steps Base Case =0: D*0 = ntadst QD01 = ntadst Fo k = 2 to b Compte QD M 0k Compte QD0k Fo = 1 to R Compte Dk Compte D* Compte QD M 1 Compte QD M k Compte QDk