An improved attribute reduction algorithm based on mutual information with rough sets theory

Transcription

1 Available olie Joural of Chemial ad Pharmaeutial Researh, 04, 6(7:30-35 Researh Artile ISSN : CODEN(USA : JCPRC5 A improved attribute redutio algorithm based o mutual iformatio with rough sets theory Yag Su-mi,, Meg Jie, Liu Qi-mig ad Che Jig State Key Laboratory of Comple Eletromageti Eviromet Effets o Eletrois ad Iformatio System, Luoyag, Hea, Chia Departmet of Iformatio Egieerig, Ordae Egieerig College, Shijiazhuag, Chia ABSTRACT There are t ore attributes i some iformatio system, but the ore attributes are the basis of attribute redutio algorithm based o mutual iformatio, i order to solve the problem of a ew attribute importae degree oe ew method o the basis of mutual iformatio is proposed i the paper, whih osists of its ow iformatio etropy ad mutual iformatio. The the orrespodig heuristi redutio algorithm is proposed. Eperimetal results show that the algorithm a solve o-ore iformatio system attribute redutio, but also a get attribute redutio faster, ad the redutio umber is also relatively small. Key words: Rough set; attribute redutio; mutual iformatio; attribute importae degree INTRODUCTION At preset there are various kids of evaluatio attributes i the ommad ad iformatio system, to esure effiiey ad effetiveess of evaluatio it is essetial to ostitute the most oise attribute system. Attribute redutio is to remove ueessary attribute without hagig lassifiatio result of iformatio system. Rough sets theory is a valid mathematial theory developed i reet years, whih a aalyze ad deal with impreise, iomplete ad iosistet iformatio effetively, ad a dig out the ootative kowledge, ad reveal potetial rules. By rough sets theory, we a usually obtai a few redutio results for a iformatio system, so we always hope to fid the miimal redutio. The ore ad redutio of attributes are two importat topis i the researh o rough sets theory, but researhers have prove that it is NP-hard problem to look for all redutio or miimum redutio of a iformatio system. But the researhers have foud that a effiiet attribute redutio algorithm a be obtaied o the oditio that the relatioship betwee kowledge ad iformatio system established with iformatio etropy. Skowro[] put forward oe attribute redutio algorithm based o diseribility matries, Artiles [-6] preset some improved attribute redutio algorithm based o diseribility matries, whih have lower omputatioal ompleity ad storig apaity. O the basis of oditioal iformatio etropy, artiles [7-8] study the omputatio of a ore ad attribute redutio i distributed eviromet. Qia[9] aalyzed the relatioship betwee attribute redutio ad oditioal iformatio quatity ad gave oe ew oditioal iformatio quatity whih ut dow the umber of attributes ad time ompleity. Teg[0] preseted a ew redued defiitio whih itegrates the omplete ad iomplete iformatio systems ito the orrespodig redued algorithm. Liag et.[-3] studied the iomplete iformatio systems. Miao [4] proposed the kowledge redutio algorithm whih is based o the mutual iformatio betwee the oditioal attributes ad deisio attributes. Jia [5] proposed oe attribute redutio algorithm based o mutual iformatio gai. Artiles [6-0] proposed rough sets attribute redutio algorithm based o mutual iformatio, whih a make use of heuristi iformatio to redue the searh spae, ad a shorte the searh time as far as possible, ad a fially get a optimal or approimate optimal solutio. But the attribute redutio algorithm based o mutual iformatio is the bottom-up approah, whose startig poit is from the relative ore attribute of deisio table, the the most importat attributes 30

2 Yag Su-mi et al J. Chem. Pharm. Res., 04, 6(7:30-35 seleted from the other attributes are added to the relative ore, ad the omputig proessig is eded whe the ore mutual iformatio is equal with the oditioal attribute. I atual iformatio system, there will be a lak of ore attributes. Whe the ore attributes are empty, we must alulate mutual iformatio after hoosig oe attribute, so the omputatioal ompleity ireases sigifiatly. I this paper, oe ew attribute importae degree method is proposed, whih depeds o its ow iformatio etropy ad mutual iformatio,ad the author gives the orrespodig heuristi redutio algorithm. The eperimetal results show that the proposed algorithm ot oly a solve the problem of attribute redutio for the o-ore iformatio system, but also a obtai the attribute redutio results faster, ad the umber of redutio attribute is relatively small. THE BASIC CONCEPTS OF ROUGH SETS THEORY Defiitio : S = ( U, A, V, f is set to a iformatio system. Amog them, U = { U, U, L, U [ U ]} is o empty fiite sets whih is alled the domai spae, A = { a, a, L, a[ A] } is o empty fiite attribute set, whih is alled the attribute set, V is attribute s domai rage, f : U A V is the a, a V = Aa, A iformatio futio. Whe is a, has uique value i V a.o the side, for sequee C ( (, (, L, ( ad sequee D( d (, d (, L, d (, A = C D, C D = φ, S = ( U, A, V, f is alled as deisio table of the iformatio system. (, (, L, ( is alled as the oditio attribute set. Defiitio : For the give kowledge represetatio system S = ( U, A, V, f, the i-diserable relatioship of ay attribute is as follows: a IND ( B = : {(, y U U : a B ( f (, a = f ( y, a} ( Defiitio 3: For the give the kowledge represetatio system S = ( U, A, V, f, P A, X U, U, the lower ad upper approimatio set for X with regard to IND (B is as below respetively: { U : IND( C X } R( X = ; ( { U : IND( C φ} R ( X = X (3 Defiitio 4: For the give kowledge represetatio system S = ( U, A, V, f, if P, Q A, the positive domai POS p ( is defied as: POS ( = Amog them, R ( X is the lower approimatio of X. p X U / P R( X P ad U is two equivalet relatio of domai U (kowledge, id( P = {,, L, }, U id( = { y, y, L, y }, the the probability distributio that P ad Defiitio 5: U is a domai set, U / Q effet o the U is defied as follows: / y y L ym [ X : p] = ;[ ] L L Y : p = (4 y y L ym = y, the symbol E is the base of E. i j Amog them, i =, i =,, L, ; ; yi, j =,, L, m U U Defiitio 6: Aordig to the iformatio theory, the iformatio etropy of kowledge P is 3

3 Yag Su-mi et al J. Chem. Pharm. Res., 04, 6(7:30-35 H ( P = log, the oditioal etropy H ( Q P of the kowledge P relative to Q is : i= i i Q P = i m i= j= y j log y The mutual iformatio I( P; of the kowledge P relative to Q is: I ( P; H ( H ( Q P i j = ; (6 THE ATTRIBUTE IMPORTANCE BASED ON THE MUTUAL INFORMATION I the proess of deisio, we pay attetio whih oditio attribute is the most importat for the last deisio, so we must osider the mutual iformatio betwee oditio attribute ad deisio attribute. The artile [7] proposed the method that obtaied the attribute importae by the ireasig amout of mutual iformatio with addig oe attribute. It is defied as follows: i (5 SGF ( a, R, Q I ( Q; R { a} I ( Q; R = H ( Q R H ( Q R { a} = (7 Based o the above formula, the hose attributes are that there is more umber i the domai, but from the iformatio theory, it is to selet the oe whih is haoti, but the seleted attributes are ot maybe useful for the deisio. I view of the above problems, the artile [9] has made the improvemet to the importae of attributes, whih is defied as follows: SGF old ( a, R, = I( Q R { a} I( Q R/ Q a = ( Q R Q R { a}/ Q a (8 But the above alulatio proessig depeds o the ore attributes, i order to overome o ore problem, the author improves the formula (8, the result is as follows: SGF ew (( a, R, = ( I( C { }; I( C; + I( ; / D = ( DC DC { } + D/ D = ( DC DC { } + / D The improved method ot oly osiders the iremet of mutual iformatio after addig the attribute, but also osiders its ow iformatio etropy. Whe the mutual iformatio iremet is equal, the smaller H ( Q a is, the higher attribute importae degree is. The equatio (9 a be i agreemet with the atual situatio, ad also solve the attribute redutio with o ulear attribute iformatio system. AN IMPROVED ATTRIBUTE REDUCTION ALGORITHM BASED ON MUTUAL INFORMATION O the basis of the formula (9 i the setio, i this paper the author proposes a ew attribute redutio algorithm, whih does ot eed to alulate ore attribute, whether oe attribute is added ito attribute redutio set or ot is deided by the iremet betwee the mutual iformatio ad oditioal etropy. The speifi algorithm desriptio is as follows: The Iput: A ompatible deisio table system, Cis oditio attributes set, D is the deisio attribute, U is the domai. The Output: Oe redutio attribute set; ( The mutual iformatio is alulated betwee oditio attribute ad deisio attribute set; (let R = φ, the proeed is performed o the attribute set R = C R, C = C R as follows: For eah attribute i C, we alulate I ( i, / H ( D, i (9, ad selet the oe that has maimum value a, if there is the same value for multiple attributes, we hoose oe whih omes the earliest, the R = R {a} the we judge whether I ( C; ad I ( R; is equal, if they are the same, the the et step goes to 3

4 Yag Su-mi et al J. Chem. Pharm. Res., 04, 6(7:30-35 (3, otherwise goes to. (3 R is a redutio result, ad we output it. THE SIMULATION EXPERIMENT I order to verify the effetiveess of the above algorithm, we take it to ompute the redutio set of a ommad ad iformatio system, as show i table. From it, we a see that the system has 4 attributes, 4 eperts give those evaluatio results, the oditio attributes are {,, 3, 4}, deisio attributes are {d}, the value of set C is set as V={0,,}, whih orrespods good, geeral, poor state respetively. The value of set D is set as {0,}, whih orrespods good ad bad for operatio effet, after pre-treatmet o the origial data, we get the deisio table show i table. The below is the proessig i aordae with the algorithm of setio 3: We osider table as a iformatio system U = {,, 3, 4, 5, 6, 7, 8, 9, 0,,, 3, 4},oditio attribute set is C = {,, 3, 4}, deisio attribute set is D = {d}, the IND ( = {{,, 6, 8, 4},{ 3, 4, 5, 7, 9, 0,,, 3} TABLE : Deisio table of iformatio system CommuiatioSystem ehage Safety Persoel deisio samples quality quality measure 3 diathesis 4 result d Aordig to the desriptio the algorithm of setio 3, the steps are as follows: ( Firstly we alulate the mutual iformatio betwee set C ad the set D ; ( We alulate the importae degree of eah attribute aordig to the formula (9, the table lists the result, from whih we a see that the attribute has maimum value, so it was hose as the redutio elemets, R = { }, C = C R. (3 The we alulate I ( R; = 0. 6,we a see I ( R; I ( C; (4 Aordig to the algorithm of setio 3, we eed to selet aother attribute to joi redutio set from the remaiig attributes, from table we a fid that the attribute has the most importae degree amog them, so we add it ito the redutio set, R = {, }. (5 The we alulate I ( R; = ,but I( R; I( C; ; (6 The we selet aother attribute from table to joi R, aordig to the importat degree, 3 are seleted, beause the equivalee lasses of,, } is { 4 {{ lasses of {, 3, 4} is {{, 8}, { 3}, { 4 5, }, { 3 4,},{ },{, },{, },{ },{ },{ },{ },{ },{ },{ },{ }} ad the equivalee }}.Beause their kid 33

5 Yag Su-mi et al J. Chem. Pharm. Res., 04, 6(7:30-35 ad umber of attribute ombiatios is the same. At preset 4 to joi the R set without 3. We get the result as follows: R =,, }, ( R; = { 4 is already i the redutio set, therefore we selet I. (7 Due to I ( R; = I ( C;, so we termiate the proessig of the algorithm. R = {,, 4} is oe redutio set of the origial iformatio system. (8 TABLE Attribute importat degree Attribute H ( D{ C } H ( D SGF old ( SGF ew ( From table, we a see that the attribute importae degree of,, 3, 4 obtaied by formula (8 were 0.574,0,0, respetively, whih is iosistet with the atual situatio. But the attribute importae degree of,, 3, 4 is obtaied aordig to the algorithm proposed i this paper is.798, 0.384, 0.95, respetively, the attribute importae degree of is whih is osistet with the atual system. CONCLUSION Beause some iformatio systems may have o ore attributes, but the ore attributes is the foudatio of the preset attribute redutio algorithm based o mutual iformatio, i order to solve the problem the author puts forward a ew method to measure the importae degree of attribute ad ostrut the orrespodig heuristi redutio algorithm. This proposed algorithm takes ito aout the iremet of mutual iformatio after addig a attribute, but also its ow iformatio etropy, whih a sigifiatly derease the ratio that the importat attribute is take as redudat attribute to remove. The eperimetal results show that the algorithm a ot oly solve the attribute redutio of o-ore iformatio system, but also be able to get redutio attribute faster ad the redutio umber is less tha the preset algorithms. Akowledgmets This researh is supported by State Key Laboratory of Comple Eletromageti Eviromet Effets o Eletrois ad Iformatio System uder Grats No. CEMEE04K0368B. REFERENCES [] Skowro A, Rauszer C. The diseribility matries ad futios i iformatio systems[c]. I: Slowiski R(Eds.: Itelliget Deisio Support-Hadbook of Appliatios ad Advaes of the Rough Sets Theory, Kluwer Aademi Publishers. Lodo, pp: 33-36,99. [] Zhag Teg-fei, XIAO Jia-mei,WANG Xi-huai. Ata Eletroia Siia, Vol., o. 33, pp: , 005. [3] YE Dog yi. Ata Eletroia Siia, Vol. 8, o.,pp:80-8,000. [4] Cheg Jig, Zhu Jig, Zhag Fa. Joural of Hua Uiversity (Natural Siees, Vol.36, o.4, pp:86-88,009. [5] Jiag Y, Wag X, YE Z. Joural of System Simulatio, Vol. 0, o. 4, pp : , 008. [6] Che Z, Zhag Q H. Study o attribute redutio algorithm based o fuzzy rough set[d]. Huaqiao Uiversity College of Computer Siee ad Tehology,006. [7] Yag mig, Yag Pig. Cotrol ad Deisio, Vol. 3, o.0, pp: 03-07, 008. [8] Zhag WeiXu, Zhagya, Wag Xiaoyu. Computer Appliatios ad Software, Vol30, o. 7, pp:43-46,03. [9] QIAN Ji, YE Feiyue, MENG Xiag Pig, LIU Da You. Egieerig ad Eletrois, Vol., o. 9, pp:54-57,007. [0] TENG Shu hua, ZHOU Shi li, SUN Ji iag, Li Zhi yog. Joural Of Natioal Uiversity Of Defese Tehology, Vol.3, o., pp:90-94,00. [] Liag J Y, Xu Z B. Iteratioal Joural of Uertaity, Fuzziess ad Kowledge based Systems,, Vol.0, o., pp: 95-03,00. [] Yee L, Ma J.M, Zhag W.X, Li T.J. Iteratioal Joural of Approimate Reasoig, Vol.3,o., pp:63-630, 34

6 Yag Su-mi et al J. Chem. Pharm. Res., 04, 6(7: [3] Pei X B, Wag Y Z. Novel approahes to kowledge redutio i iosistet deisio systems, Natural Laguage Proessig ad kowledge egieerig. IEEE NLP-KE 05 Proeedigs of 005 IEEE Iteratoal Coferee, pp: , Jauary,005. [4] Miao Duoqia,Fa Shidog. System egieerig theory & pratie, pp:48-56, Jauary,00. [5] Jia Pig, Dai Jiahua, Pa Yuhe, et al. Joural of ZheJiag Uiversity ( Egieerig Siee, Vol.40,o.6, pp: , 006. [6] Wag guokuag. Rough theory ad kowledge aquisitio[m]. Xi a jiao tog uiversity press, 00. [7] Wag Hogkai, Yao Bigue,Hu haiqig. The method of asertaiig weight based o rough sets theory [J].Computer egieerig ad appliatio, Vol.36, pp: 0-,003. [8] Ta zogfeg, Xu zhagya, Wagshuai. Computer egieerig ad appliatio, Vol.48, o.8, pp: 5-8,008. [9] Yaya,Yag huizhog. tsighua siee ad tehology, Vol.47,o.S, pp: , 007. [0] Bao izhog,liu heg. Joural of maagemet, Vol. 6,o.6,pp:79-73,009. [] Su limi,ji iagju. Joural of Guagdog Uiversity of Petrohemial tehology, Vol., o.6, pp:66-68,