Chapter 7 Clustering Analysis (1)

Transcription

1 Chater 7 Clusterng Analyss () Outlne Cluster Analyss Parttonng Clusterng Herarchcal Clusterng Large Sze Data Clusterng

2 What s Cluster Analyss? Cluster: A collecton of ata obects smlar (or relate) to one another wthn the same grou ssmlar (or unrelate) to the obects n other grous Cluster analyss Fnng smlartes between ata accorng to the characterstcs foun n the ata an groung smlar ata obects nto clusters Clusterng vs. classfcaton Clusterng - Unsuervse learnng No reefne classes 3 Alcatons Marketng Market segmentaton (customers) marketng strategy s tale for each segment. Market structure analyss (roucts) smlar / comettve roucts are entfe Investgaton of neghborhoo lfestyles otental eman for roucts an servces. Fnance Balance ortfolos securtes from fferent clusters base on ther returns, volatltes, nustres, an market catalzaton. Inustry analyss smlar frms base on growth rate, roftablty, market sze,, are stue to unerstan a gven nustry.

3 Alcatons Web search: cluster queres or cluster search results. Chemstry: Peroc table of the elements Bology: Organzng seces base on ther smlarty (DNA/ Proten sequences) Army: a new set of sze system for army unforms. Measure the Smlarty Dssmlarty/Smlarty metrc Smlarty s eresse n terms of a stance functon, tycally metrc: (, ) The efntons of stance functons are usually rather fferent for numercal, boolean, categorcal, ornal, an vector varables Weghts shoul be assocate wth fferent varables base on alcatons an ata semantcs 6 3

4 4 7 Smlarty an Dssmlarty Smlarty Numercal measure of how alke two ata obects are Value s hgher when obects are more alke Often falls n the range [0,] Dssmlarty (.e., stance) Numercal measure of how fferent are two ata obects Lower when obects are more alke Mnmum ssmlarty s often 0 Uer lmt vares 8 Dfference Measure for Numercal Data Numercal (nterval)-base: Contnuous measurements of a roughly lnear scale. Dstance between each ar of obects. Euclean Dstance Manhattan (cty block) Dstance Mnkowsk Dstance )... ( ), (... ), ( )... ( ), (

5 Eamle: Dstance Measures 3 ont y Manhattan Dstance y 0, 3, 5, 0, Euclean Dstance Dstance Matr 9 Dstance Measures for Bnary Varable A bnary varable has only two states: 0 or (boolean values). Symmetrc: both of ts states are equally valuable, e.g., male an female for Gener. Asymmetrc: the outcomes of the states are not equally mortant, e.g., ostve an negatve for Test. 0 5

6 Bnary Varables A contngency table for bnary ata ( s the total number of bnary varables) Dstance measure for symmetrc bnary varables: Dstance measure for asymmetrc bnary varables: sm Jaccar sym (, b c asym (, ) a b c Obect 0 sum a b ab Obect 0 c c sum ac b (, ) asym (, ) a a bc ) b c a b c Eamle of Dssmlarty between Asymmetrc Bnary Varables asym (, ) Name Gener Fever Cough Test- Test- Test-3 Test-4 Jack M Y () N (0) P () N (0) N (0) N (0) Mary F Y () N (0) P () N (0) P () N (0) Jm M Y () P () N (0) N (0) N (0) N (0) b c a bc ( Jack, Mary) 0.33 ( Jack, Jm) 0.67 ( Mary, Jm) * These measurements suggest that Mary an Jm are unlkely to have a smlar sease, an Jack an Mary are the most lkely to have a smlar sease. 6

7 Categorcal (Nomnal) Varables A generalzaton of the bnary varable n that t can take more than states, e.g., re, yellow, blue, green Metho : Smle matchng m: # of matches, : total # of varables (, ) m Metho : Use a large number of bnary varables creatng a new bnary varable for each of the M nomnal states 3 Ornal Varables An ornal varable can be screte or contnuous, an orer s mortant, e.g., scores, an levels Can be treate lke nterval-scale, f f has M f orere states, relace f by ther rank r f {,..., M } f Snce each ornal varable can have fferent M f, ma the range of each varable onto [0,.0] by relacng -th obect n the f-th varable by r f z f M comute the ssmlarty usng methos for nterval-scale varables f 4 7

8 Eamle of Ornal Varables Name Gener Pan Levels Bloo Pressure Jack M 5 40/90 Mary F 3 0/80 Jm M 60/0 Bloo Pressure (Hgh, Normal, Low): 40/90 (Hgh - 3)->(3-)/(3-)= 0/80 (Normal - )->(-)/(3-)=0.5 60/0 (Hgh-3) -> (3-)/(3-) = Pan levels (-0): 5 -> (5-)/(0-) = > (3-)/(0-) = 0. -> (-)/(0-) = 0. Name Gener Pan Levels Bloo Pressure Jack M 0.44 Mary F Jm M 0. (Jack, Mary) = (( ) +(- 0.5) ) / = 0.55 (Jack, Jm) = (( ) +(-) ) / = 0.33 (Mary, Jm) = ((0.-0.) +(0.5-) ) / = 0.5 Varables of Me Tyes A atabase may contan fferent tyes of varables symmetrc bnary, asymmetrc bnary, nomnal, ornal, nterval One aroach s to grou each tye of varable together, erformng a searate cluster analyss for each tye. One aroach s to brng fferent varables onto a common scale of the nterval [0.0,.0], erformng a sngle cluster analyss. A weghte formula 6 8

9 A Weghte Formula (, ) f ( f ) ( f ) ( f ) f Weght δ (f) = 0 f f or f s mssng or f = f =0 an varable f s asymmetrc bnary, A Weghte Formula (, ) ( f ) ( f ) f ( f ) f Otherwse, Weght δ (f) =. The contrbuton of varable f to (f) s comute eene on ts tye. f s symmetrc bnary or categorcal (nomnal): (f) = 0 f f = f, or (f) = otherwse f s ornal, comute ranks r f an treat z f as nterval-scale. f s nterval-base: use the normalze stance wth range [0,.0] ( f ) ma h f hf f mn h hf 9

10 Eamle Name Gener Pan Levels Bloo Pressure Test- Test- Test-3 Test-4 Jack M 5 40/90 P () N (0) N (0) N (0) Mary F 3 0/80 P () N (0) P () N (0) Jm M 60/0 N (0) N (0) N (0) N (0) Gener s a symmetrc attrbute, Pan levels an Bloo ressures are ornal, an the remanng attrbutes are asymmetrc bnary Name Gener Pan Levels Bloo Pressure Test- Test- Test-3 Test-4 Jack M 0.44 P () N (0) N (0) N (0) Mary F P () N (0) P () N (0) Jm M 0. N (0) N (0) N (0) N (0) 9 Name Gener Pan Levels Bloo Pressure Test- Test- Test-3 Test-4 Jack M 0.44 P () N (0) N (0) N (0) Mary F P () N (0) P () N (0) Jm M 0. N (0) N (0) N (0) N (0) When = Jack an = Mary, δ (gener) =, δ (Pan Levels) =, δ (Bloo Pressure) =, δ (Test-) =, δ (Test-) = 0, δ (Test-3) =, δ (Test-4) = ** * *0 * ( ) ( 0.5) ( Jack, Mary) *0 * * * ( ) ( 0.5) ( Jack, Jm) ** * ** ( ) ( 0.5) ( Jm, Mary)

11 Vector Obects: Cosne Smlarty Vector obects: keywors n ocuments, gene features n mcroarrays, Alcatons: nformaton retreval, bologc taonomy,... Cosne measure: If an are two vectors, then cos(, ) = ( ) /, where ncates vector ot rouct, : the length of vector Eamle: = = = 3*+*0+0*0+5*0+0*0+0*0+0*0+*+0*0+0* = 5 = (3*3+*+0*0+5*5+0*0+0*0+0*0+*+0*0+0*0) 0.5 =(4) 0.5 = 6.48 = (*+0*0+0*0+0*0+0*0+0*0+0*0+*+0*0+*) 0.5 =(6) 0.5 =.45 cos(, ) =.350