A new P system with hybrid MDE- k -means algorithm for data. clustering. 1 Introduction

Transcription

1 Wesun, Lasheng Xang, Xyu Lu A new P system wth hybrd MDE- agorthm for data custerng WEISUN, LAISHENG XIANG, XIYU LIU Schoo of Management Scence and Engneerng Shandong Norma Unversty Jnan, Shandong CHINA sdxyu@63.com Abstract:- Custerng s an mportant part of data mnng. It can mmensey smpfy data compexty and heps dscover the underyng patterns and knowedge from massve quanttes data ponts. The popuar effcent custerng agorthm has been wdey used n many feds. However, The method aso suffers from severa drawbacks. It seects the nta custer centers randomy that greaty nfuences the custerng performance. Ths study proposes a new P system wth modfed dfferenta evouton - agorthm to mprove the quaty of nta custer centers of agorthm. The P system has three types of membranes: eementary, oca store, goba store. Dfferent membranes have dfferent rues. Based on the membrane structure, the eementary membranes evove the obects wth modfed dfferenta evouton agorthm and other types of membrane update the oca best and the goba best obects synchronousy wth communcaton rues. Under the contro of the P system, the hybrd agorthm acheves a good partton for data sets, compared wth the cassca agorthm and DE- agorthm. Key-Words: - Data mnng; custerng; unsupervsed earnng; k-means; modfed DE agorthm; membrane computng. Introducton Custerng aso known as unsupervsed earnng n genera s the process of dvdng n unabeed data ponts nto k abeed groups. As a resut, the eements n the same custer have hgher smartes than those n dfferent custers. Over the years, a great number of custerng agorthms have been deveoped [9]. One of the most popuar unsupervsed custerng methods s the agorthm [4]. Ths method has the advantages of smpcty, easy to mpement and hgh effcency. Ths method has been wdey used n a varety of areas. However, the agorthm aso has some drawbacks. It may get struck at a oca optmum and s very senstve to the nta choce of the custer centers. In order to overcome these drawbacks, varous custerng agorthms based on evoutonary computng have been ntroduced n recent years, such as the generc agorthms, the dfferenta evouton (DE) agorthm, and the partce swarm optmzaton agorthm [, 4, 5, 7, 0, ]. DE s undoubtedy an effectve popuaton based stochastc optmzaton agorthm []. DE agorthm s robust, reabe, nherenty parae, easy to mpement and has great goba exporaton abty. It can acheve the goba optma souton as the nta custer centers for the agorthm and mprove the custerng performance. However, DE aso suffer some drawbacks. It has weak oca expotaton abty and ow convergence veocty. E-ISSN: Voume 5, 06

2 Wesun, Lasheng Xang, Xyu Lu Ths study proposes a modfed DE agorthm to overcome these dsadvantages. Membrane computng, frst ntroduced by Paun n 998 [5] s motvated by the structure and functon of a vng ce. There are three man types of P systems,.e., ce-ke P systems, tssue-ke P systems, and neutra-ke P systems [6]. These P systems consst of orgna obects, speca membrane structures and specfc fexbe evouton and communcaton rues. Membrane computng has the advantage of dstrbuted and maxmum paraesm and can greaty mprove the computatona effcency. In ths study, an nnovatve membrane structure s but and the nherent evouton-communcaton rues are modfed. Through these operatons, the goba best obect s eventuay obtaned. The cassca dfferenta evouton agorthm The DE agorthm, proposed by Storn and Prce [3] and used to fnd goba optma soutons for hard optmzaton probems, has drawn more attenton n recent years. The best settng of the contro parameters can be dfferent for dfferent optmzaton probems. The DE agorthm many conssts of parameters, the ftness functon and three dfferenta operatons ncudng mutaton, crossover and seecton. Let s the th obect n the popuaton, X denote a genome n the popuaton, M ndcate the vector that has gone through the mutaton operatons and R be a new vector undergone crossover operatons. Set V to represent the survva vector after a seecton operaton.. Mutaton operaton The mutaton scheme pays an mportant roe n the process of searchng for a goba optma souton and acceeratng convergence. There are fve forms of mutaton operatons as gven by, and severa studes have proved that the frst and the fourth form have better performances.. DE/rand/: M = X + F*( X X3). DE/best/: M = Xb + F*( X X3) 3. DE/current-to-best/: M = X + F*( Xb X) + F* (X-X) 3 4. DE/best/: M = Xb + F*( X X) + F* (X-X) DE/rand/: M = X + F*( X X3) + F* (X4-X) 5 In the above, X, X, X 3, X 4, X 5 are random vectors n the popuaton and are dfferent from the base vector X, F s a postve predefned constant, caed the scae factor for scang dfferent vectors, and X b s the vector wth the best ftness n the popuaton.. Crossover operaton Ths operaton can greaty ncrease the dversty of the popuaton. The crossover operaton s performed accordng to () n the foowng R = M f rand[0,] CR or = rand, () R = X otherwse where CR denotes the crossover rate, rand ( rand {,..., d} ) s a randomy chosen ndex, R s the -th component of the data pont and R M s the -th component of the newy generated data pont ensures that obect from the newy generated vector. M. Ths crossover operaton R gets at east one component.3 Seecton operaton Seecton operaton determnes whch vector to survve n the next generaton based on the changes E-ISSN: Voume 5, 06

3 Wesun, Lasheng Xang, Xyu Lu caused n the obectve functon. The obectve functon s aso known as the ftness functon. Dfferent probem has dfferent obectve functon and each of the survva vectors has a hgher ftness. In ths study, the seecton operaton s performed as foows: d f = mn( X Z ) = V = R f f( R ) f( X ), () V = X otherwse where f s the ftness functon. f( X ) s the ftness of X. 3 The modfed dfferenta evouton agorthm The DE agorthm has the dsadvantages of weak oca expotaton abty as we as ow convergence veocty [3, 8].The modfed DE agorthm overcomes these dsadvantages. The new agorthm s caed the modfed dfferenta evouton agorthm (MDE for short). 3. The modfed of mutatons operaton In the mutaton operaton, the cassca DE agorthm chooses three vectors and the base vector randomy to mantan dversty, whch sows down convergence. In ths study a drected mutaton scheme s proposed to enhance the oca expotaton and to speed up convergence. In the MDE agorthm, the mutaton operaton s performed accordng to (3) and (4) n the foowng G+ G G G G V = X + F*(X -X ), f rand(0,) 3 G G+ G G G V = X + F *( X X ), otherwse r g w m (3) Fr = 0.5*( + rand(0,)), (4) where G ndcates the number of current teraton, G m denotes the maxmum number of teratons, G V + generaton, symbozes the base vector of the next X g represents the data vector wth the best ftness, X w represents the worst vector and F r s the dynamc scang factor. In (3) and (4), F s an mportant scang factor that contros the evovng rate of dfferent vectors. In genera, F s a constant vaue n [0, ]. In the drected mutaton scheme, the base vector s mantaned randomy and the dfferent vectors are evoved foowng the drecton of the best vector and the opposte drecton of the worst vector. In order to ncrease the dversty of the popuaton, F r s treated as a random varabe as determned by (4) to perturb the base vector. The drected mutaton scheme can expore the same drecton but wth dfferent weghts under certan condtons. At the begnnng of the drected MDE, two mutaton schemes can be used wth a ager probabty for the basc scheme and a smaer probabty for the drected scheme. As more generatons are performed,.e., as G ncreases, G G m aso ncreases and the two schemes w be apped under the same possbtes. Ths strategy baances the exporaton and expotaton of the MDE agorthm. Fnay, t ncreases the probabty of the drected scheme and enhance oca expotaton. In the entre mutaton process, the expotaton and the exporaton are performed n parae. 3. The modfcaton of the crossover rate In the crossover operaton of the cassca DE, CR contros the probabty of the tra ndvdua adoptng the od ndvdua genes or the new ndvdua genes. More new genes w be adopted E-ISSN: Voume 5, 06

4 Wesun, Lasheng Xang, Xyu Lu wth a arger CR vaue. As a resut, the popuaton dversty ncreases and the convergence speeds up. However, a arge CR vaue may aso decrease the convergence rate and ead to a premature convergence resut. At the start of the search, the popuaton has better dversty and a sma CR vaue s needed. The sma vaue can hep avod convergng prematurey. Wth the contnuous deveopment of evouton, a arge CR vaue shoud be used to ncrease popuaton dversty and acceerate the convergence. In the MDE agorthm, the crossover rate CR s determned as foows: CR = CR f G = 0 mn p, (5) G CR = CR + ( CR CR )( ) otherwse mn max mn G m of teratons exceeds the maxmum number or when the changes n center vectors stays wthn a certan amount. A new P system s desgned n ths study usng a hybrd agorthm. Ths hybrd agorthm combnes the MDE and the agorthms (MDE- for short) to optmze the custer centers n the agorthm so as to mprove the custerng resuts. 4. The structure of the P system usng the hybrd agorthm The MDE- P system has a nested structure of three ayers wth (q+) membranes. where CR =0., CR = and p = 4. mn max 4 The P system wth hybrd agorthm MDE- In ths secton, the P system s ntroduced n detas. It ncudes the speca membrane structure, the membrane rues, the cacuatng process and the hat condton n P system. 4. A bref ntroducton of the agorthm The core of the agorthm s the measure of smarty between vectors based on the Eucdean dstance. The agorthm can be brefy stated as foows: () the agorthm randomy chooses the k custer centers. () each vector s assgned to the custer wth the shortest Eucdean dstance between the vector and the custer center vector among a k custers. The custer center vector s recacuated after a vector s assgned to the custer. (3) the agorthm termnates when the number Fg. The membrane structure of P system wth MDE- agorthm The new P system for custerng s defned as foows: =( O, µ, M... M, R... R, R 0... R, 0) q q q+ (6) The notaton n (6) are O represents the aphabet of the obects n the P system. Each obect s a k* d dmensona vector. µ represents the membrane structure of the P system. M,..., M mean the nta obects on the q membrane ( =...q). R,..., R symboze the evouton and the q communcaton rues contaned n the membrane E-ISSN: Voume 5, 06

5 Wesun, Lasheng Xang, Xyu Lu ( =...q). R0,..., R q+ denote the communcaton rues on the skn membrane and membrane abeed 0,..., q +. ndcates the output regon. The output 0 regon s the envronment f = The rues n the P system The rues n the P system are ntroduced n ths secton evouton rues The evouton rues on the eementary membrane consst of mutaton rues (3, 4), crossover rues (, 5) and seecton rues () wth the form µ υ where µ, ν O communcaton rues Communcaton rues mean membrane choose obect wth the hghest ftness to transmt to ther neghbor. In each teraton, eementary membrane communcate ther oca best obect wth ther upper neghbor, whch abeed,...,q. Then Membranes abeed,...,q seect the obect wth hghest ftness on ther own membrane and transfer the obect to membrane q +. Membrane q + can receve the oca best obect from,...,q and seect the goba best obect from them to pass nto the skn membrane rues d (x ( ),z) = m m (7) m= D X Z In formua (7), X ( =,..., n) symbozes the -th vector n the data sets, Z denotes the center vector of the custer ( =,..., k), and d means the features of each vector. After each teraton, recacuate the custer vectors. 0 Z X x c =, (8) n Where custer. n s the tota number of the data n C means the custer. 4.4 The cacuatng process n the P system The P system has a three ayer nested structure. The outermost membrane, named skn membrane, s abeed 0 and contans q + membranes. Each of the membranes ( =,..., q) have an eementary membrane abeed ( =,..., q). Before the cacuaton starts, the eementary membrane ( =,..., q) generates m nta obects whe other membranes do not have nta obect. Each of the m obects s consst of k* d random rea numbers that generate randomy. They a satsfy the condtons that: Z = rand *(max mn ) + mn, (9) Where max and mn are the upper and ower bounds on the -th component of Z. There are evouton rues that consst of seecton rues, crossover rues, mutaton rues and communcaton rues on the eementary membrane. In each teraton, eementary membrane pass ther oca best obect nto ther hgher neghbor. Membranes,...,q wthout nta obect ony have the communcate rues. They store the oca best obects that passed from ts eementary membrane, seect the one wth hghest ftness and communcate wth membrane q +. Membrane q + store the oca best obects from,...,q and seect the goba best obect from them to communcate wth the skn membrane. E-ISSN: Voume 5, 06

6 Wesun, Lasheng Xang, Xyu Lu 4.5 Hatng and output of the P system In the P system, each membrane s consdered as a parae computng unt wth hgh effcency. In every operaton the rues on the same membrane are apped n a maxmay parae manner unt t hats.the P system hats, When the executon steps arrve the maxmum numbers. And the obect stored n the skn membrane s regarded as the best custer centers. 4.6 Compexty anayss Consderng the tota number of records s n; The number of attrbutes s d; every membrane has m nta records; the number of teratons of crossover, mutaton and seecton s N; The detaed compexty anayss s presented as foows. Step : Normaze a data set In a data set, normaze the vaues of numerca attrbute. The compexty to fnd the maxmum or the mnmum doman vaue of a numerca attrbute s O(n). The compexty to normaze a the vaues of a numerca attrbute s O(n). Step : Inta popuaton There are m obects n every membranes and every obect have d attrbute. Compexty of record to record dstance. The compexty to cacuate dstance between two records s O(d). For a membrane wth m records, the compexty for record to record dstance cacuaton s O(m d). Step 3: Seecton operaton It computes the dstance between a pars of seeds. If there are k number of seeds the compexty s O(k d). The ftness functon aso computes the dstance between each record and ts cosest seed wth a compexty O(mkd). Therefore, the compexty of the ftness cacuaton for each record s O(mkd+k d). Once the ftness vaues of the m records are computed we need to sort them n descendng order for fndng the best records. The compexty for ths s O(m ). Therefore the tota compexty of seecton operaton s O(mkd+k d+m ). Step 4: Crossover operaton In the crossover operaton, The rearrangement compexty s O(k d) f there are k genes n a chromosome and d attrbutes n the data set. for m chromosomes the compexty of twn remova s O(k dm). Step 5: Mutaton operaton If a chromosome s chosen for mutaton, the compexty of ths s O(k),the number of gene n a chromosome s k. In the mutaton operaton we need to compute the maxmum ftness and the mnmum ftness for cacuatng the mutaton probabty. The compexty for the cacuaton s O(m) f there are m records n one membrane. Therefore, the compexty of the Mutaton operaton s O(km). If there are N teratons then the Seecton operaton, Crossover operaton, Mutaton operaton w be repeated N tmes whe the nta popuaton w be seected once randomy. Step 6: If the number of teratons n s the N, then the compexty for s O(nkdN ). Therefore, the overa compexty of MDE-K-means. For a hgh dmensona data set where d s very arge compared to a other parameters the compexty of MDE-K-means s O(d). Smary, the overa compexty of MDE-K-means for a very arge data set where the number of records n s very arge compared to a other parameters the compexty s O(n). 5 Expermenta resuts Ths secton many ntroduced the experments, whch ncuded the data sets and parameters n experments, the process of experment and the experment resuts. 5. Experment data sets and parameters In the paper, we evauate and compare the P system wth modfed dfferenta evouton- E-ISSN: Voume 5, 06

7 Wesun, Lasheng Xang, Xyu Lu agorthm wth the cassca agorthm and the dfferenta evouton- agorthm. Expermenta resuts were cacuated for artfca data sets, the rs data set and the wne data set from UCI Machne Learnng Repostory. For the artfca data set, we cacuate the compactness of a custer and the separaton between the custers to evauate the custerng performance of the agorthms. X Z X C comp = (0) C sep = mn Z Z () Where. means Eucdean dstance between the two vectors. C s the number of the vectors n custer. Z s the center of custer. Accordng to (0, ), t s easy to fnd that a smaer compactness and ager separaton mpy a better custerng souton. For the rs and the wne data sets, to evauate the performance of the modfed custerng agorthm wth membrane computng, t s necessary to use a statstca-mathematca functon, the custer vadty ndex, to measure the effectveness of the proposed nove souton. We use the Xe-Ben ndex, the PBMF ndex, the custerng quaty descrbed beow, n our experment to udge the performance of these methods. These dfferent types of data ponts make us do more experments to prove the effectveness of the new agorthm. The Xe-Ben ndex s a functon of the rato of tota varaton σ, to mnmum separaton, sep, of the custer. The XB ndex s σ XB= () n sep ( Z ) In (), k n σ = u X z (3) = = sep( Z) mn z z =, ( ) X z In (3), u = K ( ) = X z In these formua, X means the th unabeed data ponts. The z represents the center ponts of the custer C. The u s the fuzzy membershp degree of the X toc. Note that a better parttonng shoud has ower σ and hgher sep. Therefore ower vaues of XB ndex acheve a good custerng souton. The PBMF ndex E PBMF( K) = ( ) (4) In (4), D K K Ek K n K = = = E u X z, D = max z z. K K s the number of custers. E s a constant for a gven data set. In usuay arger PBMF mpes a better parttonng. The nove proposed MDE- wth P system parameters were set to: the custer number k =3, scang factor F =0.7, the number of nta obects on eementary membrane m=0, the number of generatons G m =00, crossover rate CR max =, membrane numbers q =0. CR mn =0., 5. Experment process Input: Data set, X; the number of custers, K; the number of eementary membranes, ; the number of obects n each evouton membrane, m; maxmum generaton, G m ; and crossover probabty, CR, CR ; scang factors F. max mn Output: The optma custer centers, the skn membrane. Begn Step: Intazaton q O, n gbest E-ISSN: Voume 5, 06

8 Wesun, Lasheng Xang, Xyu Lu for = to q for = to m Generate -th nta obect for eementary membrane, O ; Cacuate partton matrx, end for Update oca best obect, Z ; store membrane by communcaton rues; end for Update goba best obect, O, n -th oca best O gbest, n goba store membrane by communcaton rues; Set Generaton G = 0; Step: Obect evouton n eementary membranes for each eementary membrane, n parae do for = to m Evove obects, O dfferenta evouton rues; Cacuate partton matrx,, ( =,,q), usng the modfed Z ; end for end for Step3: Obect communcaton for each eementary membrane, ;( =,,q), n parae do Transmt the best obect n membrane to update ts oca best obect, O, usng best communcaton rues; Transmt the best obect n membrane to q+ to update the goba best obect, O, usng gbest communcaton rues; end for step4: Hatng condton udgment f G G s satsfed m G = G+ ; go to Step; end f; Export the goba best obect, Cacuate partton matrx, the goba best obect, O ; gbest O ; gbest Z, accordng to Assgn a ponts nto K custers based on End; Z. 5.3 Experment resuts For artfca data set: X=[0 5;7 ;5 9;7 69;96 78;34 39;55 7;39 46; 35;7 77;3 9;35 39;4 54; 6;35 6] DE- MDE- Fg. The custerng resut wth three methods for artfca data sets. E-ISSN: Voume 5, 06

9 Wesun, Lasheng Xang, Xyu Lu Tabe The average compactness of the three methods n custerng artfca data sets. MDEcuster DE Tabe The average separaton of the three methods n custerng artfca data sets. custer 3 Average Separaton of Average Separaton of DEk -means Average Separaton of MDEk -means Tabe3 The average performance of the three methods n custerng UCI data sets. Tabe 4 the XB and PBMF ndex of the three methods n custerng UCI data sets. Data sets Methods XB PBMF Irs DE MDE Wne DE MDE Through these expermenta resuts, the concuson coud be drawn that MDE- agorthm wth P system has smaer compactness vaues and ager separaton vaues compared wth k-means method and DE-. MDE- agorthm wth P system aso has hgher average correct rates. Contrast wth other two method, MDE- has the owest XB vaues and the hghest PBMF vaues. In a word, MDE- agorthm wth P system gets a better custerng partton and greaty shorten the computng tme. Irs Wne method Correct ponts Correct rates Correct ponts Correc t rates k - means 35 90% % DEk -means 4 94% % MDEk -means % 46 8.% 6 Concusons Ths paper presents a new P system wth speca membrane structure and modfed evouton-communcaton mechansm. The P system combned wth modfed dfferenta evouton- agorthm can fnd the goba optma custer centers to mprove the performance of agorthm. The nta obects on eementary membrane generate randomy and execute the evouton-communcaton rues n the P system. Under the contro of rues, the nta obects are evoved and transmtted n the P system. Membranes update the oca best obects and the goba best obect synchronousy unt system reach E-ISSN: Voume 5, 06

10 Wesun, Lasheng Xang, Xyu Lu the hat condtons.in the end,the goba best obect s the best approxmaton to optma custer centers for the agorthm. The expermenta resuts verfy the advantages of the proposed P system wth the hybrd agorthm. Acknowedgement Proect supported by Natona Natura Scence Foundaton of Chna (670038,6473), Jnan Cty ndependent nnovaton pan proect n Coege and Unverstes, Chna (0400), Mnstry of educaton of Humantes and soca scence research proect, Chna (YJA6305), Soca Scence Fund Proect of Shandong Provnce, Chna (CGLJ), outstandng youth scentst foundaton proect of Shandong Provnce, Chna (ZR0FM00). References: [] Peng, H., Wang, J., Pérez-Jménez, M. J., Rscos-Nunez, A. An unsupervsed earnng agorthm for membrane computng. Informaton Scences,Vo.304, 05,pp.80-9;. [] Kao Y T, Zahara E, Kao I W. A hybrdzed approach to data custerng. Expert Systems wth Appcatons An Internatona Journa, Vo.34,No.3,008, pp [3] Mohamed A W, Sabry H Z, Khorshd M. An aternatve dfferenta evouton agorthm for goba optmzaton. Journa of Advanced Research,Vo.3,No., 0, pp [4] Rahman M A, Isam M Z. A hybrd custerng technque combnng a nove genetc agorthm wth K-Means. Knowedge-Based Systems, Vo.7,No.7, 04, pp [5] Chang D X, Zhang X D, Zheng C W. A genetc agorthm wth gene rearrangement for K-means custerng. Pattern Recognton, Vo.4,No.7, 009, pp.0-. [6] Paun G. Computng wth Membranes. Journa of Computer & System Scences, Vo.6,No., 998, pp [7] Bandyopdhyay S, Mauk U. An evoutonary technque based on K-Means agorthm for optma custerng n RN. Informaton Scences, Vo.46,No., 00, pp.-37. [8] Noman N, Iba H. Acceeratng Dfferenta Evouton Usng an Adaptve Loca Search. Evoutonary Computaton IEEE Transactons on, Vo.,No., 008, pp [9] An K. Jan. Data custerng: 50 years beyond K-means. Pattern Recognton Letters, Vo.3, 00, pp [0] Tan S C, Ka M T, Teng S W. A genera stochastc custerng method for automatc custer dscovery. Pattern Recognton, Vo.44,No.0, 0, pp [] Mauk U, Sanghamtra B. Genetc agorthm-based custerng technque. Pattern Recognton, Vo.33,No.99, 000, pp [] Chang D X, Zhang X D, Zheng C W. A genetc agorthm wth gene rearrangement for K-means custerng. Pattern Recognton, Vo.4,No.7, 009, pp.0-. [3] Raner Storn, Prce K. Dfferenta Evouton A Smpe and Effcent Heurstc for goba Optmzaton over Contnuous Spaces. Journa of Goba Optmzaton, Vo.,No.4, 997, pp [4] An K. Jan. Data custerng: 50 years beyond K-means. Pattern Recognton Letters, Vo.3, 00, pp [5] Paun G. Membrane Computng[M]. Fundamentas of Computaton Theory. Sprnger Bern Hedeberg, 003. E-ISSN: Voume 5, 06