Fuzzy Cluster Centers Separation Clustering Using Possibilistic Approach

Transcription

1 Fzzy Clster Ceters Separato Clsterg Usg Possblstc Approach Xaohog W,,, B W 3, J S, Haj F, ad Jewe Zhao School of Food ad Bologcal Egeerg, Jags Uversty, Zhejag 03, P.R. Cha el.: wxh49@js.ed.c School of Electrcal ad Iformato Egeerg, Jags Uversty, Zhejag 03, P.R. Cha 3 Departmet of Iformato Egeerg, ChZho Vocatoal echology College, ChZho 39000, P.R. Cha Abstract. Fzzy c-meas (FCM) clsterg s based o mmzg the fzzy wth clster scatter matrx trace bt FCM eglects the betwee clster scatter matrx trace that cotrols the dstaces betwee the class cetrods. Based o the prcple of clster ceters separato, fzzy clster ceters separato (FCCS) clsterg s a exteded fzzy c-meas (FCM) clsterg algorthm. FCCS attaches mportace to both the fzzy wth clster scatter matrx trace ad the betwee clster scatter matrx trace. However, FCCS has the same probablstc costrats as FCM, ad FCCS s sestve to oses. o solve ths problem, possblstc clster ceters separato (PCCS) clsterg s proposed based o possblstc c-meas (PCM) clsterg ad FCCS. Expermetal reslts show that PCCS deals wth osy data better tha FCCS ad has better clsterg accracy tha FCM ad FCCS. Keywords: Fzzy c-meas; Possblstc c-meas; Nose sestvty; Clster ceters separato; Clster scatter matrx. Itrodcto Sce Zadeh trodced the cocept of fzzy set [], fzzy set theory based o membershp fcto has advaced may dscples, sch as cotrol theory, optmzato, patter recogto, mage processg, data mg, etc, whch formato s complete or mprecse. Fzzy clsterg performs data clsterg based o fzzy set theory, whle K-meas clsterg clsters data based o classcal set. he well-ow fzzy clsterg s the fzzy c-meas (FCM) algorthm []. FCM algorthm maes the membershps of a data pot across classes sm to oe by the probablstc costrats. Ad FCM s approprate to terpret membershps as probabltes of sharg. However, FCM ad ts derved algorthms are mostly based o mmzg the fzzy wth clster scatter matrx trace [3]. he fzzy wth clster scatter matrx trace Correspodece athor. Y. a, Y. Sh, ad K.C. a (Eds.): ICSI 00, Part II, LNCS 646, pp. 35 4, 00. Sprger-Verlag Berl Hedelberg 00

2 36 X. W et al. ca be terpreted as a compactess measre wth a wth-clster varato. FCM attaches mportace to the fzzy wth clster scatter matrx trace that measres the class cetrods close to data pots bt eglects the betwee clster scatter matrx trace that cosders dstaces betwee the class cetrods. From the aspect of data classfcato, both the wth clster scatter matrx ad the betwee clster scatter matrx are mportat. he cocept of volvg the betwee clster scatter matrx s sed clster valdty sch as the FS dex proposed by Fyama ad Sgeo [4], ad clsterg algorthms sch as fzzy compactess ad separato (FCS) algorthm proposed by W, Y ad Yag [5], fzzy clster ceters separato (FCCS) clsterg proposed by W ad Zho [6]. Becase the betwee clster scatter matrx trace ca be terpreted as a separato measre wth a betwee clster varato, maxmzato of the betwee clster scatter matrx trace wll dce a reslt wth well-separated clsters [5]. O the other had, FCM s sestve to oses [7]. o overcome these dsadvatages Krshapram ad Keller have preseted the possblstc c-meas (PCM) algorthm [7] by abadog the costrats of FCM ad costrctg a ovel objectve fcto. he PCM ca deal wth osy data better tha FCM. I ths paper, we fd FCCS s also sestve to oses becase the probablstc costrats. o solve ths problem, we propose possblstc clster ceters separato (PCCS) clsterg to exted the FCCS to ts possblstc model. Fzzy Clster Ceters Separato Gve a labeled data set X={x,x,,x } R p, FCCS fds the partto of X to <c< fzzy sbsets by mmzg the followg objectve fcto [6] FCCS c c m λ = = = J (, ) = D v x UV () sbject to the costrats: 0, D = x ν, ad c = =,. Where c s the mber of clsters ad s the mber of data pots, s the membershp of x class, weghtg expoet m [, ]. λ > 0 represets the extet of clster ceters separato. V = ( ν, ν, ν c ) R cp wth ν R p s the clster ceter of, c. U M fc, M fc s a fzzy c-partto space of X : M c c f c μ = = = { U R = [0,],, ; = ;0 < <, } I eqato (), the frst term s the fzzy wth clster scatter matrx trace tr( S fw), S fw s the fzzy wth clster scatter matrx: ()

3 Fzzy Clster Ceters Separato Clsterg Usg Possblstc Approach 37 c m fw = = = S ( x v )( x v ) (3) he secod term cldes the betwee clster scatter matrx trace tr( S B ), B betwee clster scatter matrx [6] he ( UV, ) FCCS c B = ( )( ) ; = S v x v x m J (, ) S s the j= j (4) x= x UV s eqal to mmze the fzzy wth clster scatter matrx trace ad maxmze the betwee clster scatter matrx trace. Eqato () s optmzed der costrats by Lagrage mltplers ad the followg eqatos are obtaed c m D =,, j= D j (5a) ν m x = m = λx =, λ (5b) Where ν s the clster ceter or prototype of. 3 Possblstc c-meas Clsterg he possblstc c-meas clsterg s obtaed by mmzg the followg objectve fcto [7]: c c PCM(, ) = + η( log ) = = = = J t D t t t V (6) Here 0 t, m >, D = x ν. Ad c s the mber of clsters, s the mber of data pots, t s the typcalty of x class, ad t s the typcalty vale that depeds o all data. Krshapram ad Keller sggest choosg the parameters η that are postve costats by comptg [7]:

4 38 X. W et al. m, FCM D = K, K > 0 = η = m, FCM Here K s always chose to be ;, FCM s the termal membershp vales of FCM. If D = x ν > 0 for all ad >, ad X cotas at least c dstct data pots, m J (, ) (, V) PCM obtaed as follows [7]: V s optmzed ad the possblstc c-meas clsterg s t (7) D = exp( ),, (8a) η t x = ν =, t = Here ν s the clster ceter or prototype of. (8b) 4 Possblstc Clster Ceters Separato Clsterg I ths secto, we propose a ovel fzzy clsterg objectve fcto that s a geeralzato of the FCCS objectve fcto by trodcg PCM algorthm. he proposed algorthm s called possblstc clster ceters separato (PCCS) clsterg. he the objectve fcto of PCCS s defed as: PCCS c c c λ η = = = = = J (, ) = t D v x + ( t log t t ) V (9) Here, we se the techqe that comes from possblstc clsterg algorthm (PCA) [8] to compte the parameters η. he objectve fcto of PCCS s rewrtte: PCCS mc = = c c λ = = = J (, V) = t D v x c σ + ( t log t t ) (0)

5 Fzzy Clster Ceters Separato Clsterg Usg Possblstc Approach 39 Here the parameterσ s a ormalzato term that measres the degree of separato of the data set, ad t s reasoable to defe σ as the sample co-varace. hat s: σ = x x wth = j= j () x= x o mmze eqato (0), sbject to the costrats m>, 0 followg eqatos t t, we obta the mcd = exp( ),, (a) σ ν t x λx = =, t λ = (b) D >0 for all ad, ad X cotas c < dstct data pots, the the algo- If rthm descrbed below s called PCCS-AO algorthm: Italzato ) R FCM tl termato to obta the class ceter V as V (0) sed by PCCS, ad se Eq.() to calclate the parameter σ ; ) Fx c, <c<; 3) Set terato coter r = ad maxmm terato r max ; Repeat Step Update membershp (r) by Eq.(a); Step Update V (r) by Eq.(b); Step 3 Icremet r; ( r) ( r ) Utl ( V V < ε ) or (r> r max ) 5 Expermets We frst do expermet o X that s a two-dmesoal data set wth data pots whose coordates are gve [9]. he data set X comes from Pal [9] ad Fgre shows ts dstrbto coordates. here are te pots (except x 6 ad x ) form two damod shaped clsters wth fve pots each o the left ad rght sdes of the y axs. We ca see x 6 ad x as osy pots ad each has the same dstaces from the two clsters. he talzato of clster ceters [9]

6 40 X. W et al V0 = (3) Comptatoal codto: ε =0.0000, maxmm mber of teratos r max =00. m=.0, λ =0.05. able. Data set X ad termal U from FCM o X Data set X Pt. x y U FCM U able shows the termal membershp vales of FCM by applyg FCM-AO ad able shows the termal membershp vales ad typcalty vales of FCCS ad PCCS by applyg FCCS-AO ad PCCS-AO to X, respectvely. he membershp vales of x 6 ad x both FCM ad FCCS are 0.50 each clster. Becase the probablstc costrats sed by FCM ad FCCS, the membershp vales of x 6 ad x reflect the share of two data pots betwee two clsters by FCM ad FCCS algorthms. So ther vales are Bt fact x 6 ad x are oses, ther membershp vales shold be very small. So both FCM ad FCCS caot reflect the fact ad they are sestve to oses. From able ad able both FCCS ad FCM provde membershp formato bt PCCS provdes typcalty formato. PCCS abados the probablstc costrats FCCS. he typcalty vales of x 6 ad x are small by PCCS algorthm. Becase x s farther away from clster ceters tha x 6, the PCCS assgs the smaller typcalty vales of x tha those of x 6. hs reflects the real statos. hat s to say, x 6 ad x are more atypcal tha other 0 data pots. So PCCS ca dstgsh the oses from datasets. So FCM ad FCCS caot dstgsh oses from pt data. I coclso, FCM ad FCCS are more sestve to oses tha PCCS. he other example s that we perform expermets o IRIS data set [0] s wdely sed expermets. It s a for-dmesoal data set that cldes three classes: Setosa, Verslcolor ad Vgca ad each class has 50 data pots. he comptatoal

7 Fzzy Clster Ceters Separato Clsterg Usg Possblstc Approach 4 codto s ε =0.0000, maxmm mber of teratos r max =00, m=.0. he clsterg accracys from FCM, FCCS ad PCCS o IRIS data set are llstrated able 3. he PCCS algorthm has better clsterg accracy tha the other two algorthms except that λ s bgger tha Pt. able. ermal U ad from FCCS ad PCCS o X U FCCS U PCCS able 3. Clsterg accracy from FCM, FCCS ad PCCS o IRIS data set λ FCM FCCS PCCS % 89.3% 9.7% % 89.3% 9.7% % 89.3% 9.7% % 89.3% 9.7% % 89.3% 9.7% % 89.3% 93.3% % 88.7% 94.0% % 89.3% 95.3% % 89.3% 89.3% % 89.3% 88.7% 6 Coclsos Based o fzzy clster ceters separato clsterg ad possblstc c-meas clsterg we propose possblstc clster ceters separato clsterg as a exteso of fzzy clster ceters separato clsterg algorthm. PCCS abados the probablstc costrats of FCCS ad costrcts a ovel objectve fcto. Frthermore, possblstc clsterg algorthm s sed to compte the parameters η PCCS objectve

8 4 X. W et al. fcto. Or Expermets o data set X show that both FCM ad FCCS are sestve to oses, bt PCCS ca deal wth osy data better. Next example s that we mae expermets o IRIS data set by rg FCM-AO, FCCS-AO ad PCCS-AO respectvely. he expermetal reslts show that the PCCS has better clsterg accracys tha FCM ad FCCS whe λ s smaller tha I ths paper, we select the vales of λ to do expermets to evalate FCM, FCCS ad PCCS algorthms. λ cotrols the extet of clster ceters separato. If λ > 0 the clster ceters separate each other. If λ < 0 the clster ceters are close to each other. he frther terestg stdy s to optmze the parameter λ to advace FCCS ad PCCS. Acowledgmets. he athors wold le to tha Cha Postdoctoral Scece Fodato fded project (No ) for facally spportg ths research. Refereces [] Zadeh, L.A.: Fzzy sets. Iformato ad Cotrol 8, (965) [] Bezde, J.C.: Patter Recogto wth Fzzy Objectve Fcto Algorthms. Plem Press, New Yor (98) [3] Bezde, J.C., et al.: Fzzy Models ad Algorthms for Patter Recogto ad Image Processg. Klwer Academc, Dordrecht (999) [4] Fyama, Y., Sgeo, M.: A New Method of Choosg the Nmber of Clsters for Fzzy C-Meas Method. I: Proceedgs of the 5th Fzzy System Symposm, pp (989) [5] W, K.L., Y, J., Yag, M.S.: A Novel Fzzy Clsterg Algorthm Based o a Fzzy Scatter Matrx wth Optmalty ests. Patter Recogto Letters 6, (005) [6] W, X.H., Zho, J.J.: Fzzy Clsterg Models Based o Clster Ceters Separato. Joral of Soth Cha Uversty of echology (Natral Scece Edto) 36(4), 0 4 (008) [7] Krshapram, R., Keller, J.: he Possblstc C-Meas Algorthm: Isghts ad Recommedatos. IEEE ras. Fzzy Systems 4(3), (996) [8] Yag, M.S., W, K.L.: Uspervsed Possblstc Clsterg. Patter Recogto 39(), 5 (006) [9] Pal, N.R., Pal, K., Bezde, J.C.: A Possblstc Fzzy C-Meas Clsterg Algorthm. IEEE ras. Fzzy Systems 3(4), (005) [0] Bezde, J.C., Keller, J.M., Krshapram, R., et al.: Wll the Real Irs Data Stad p? IEEE ras. Fzzy System 7(3), (999)