Information Sciences

Transcription

1 Ifrmati Scieces 181 (2011) Ctets lists available at ScieceDirect Ifrmati Scieces jural hmepage: Parameterized attribute reducti with aussia kerel based fuzzy rugh sets Degag Che a,, Qighua Hu b, Ygpig Yag a a Nrth Chia Electric Pwer Uiversity, Beijig , PR Chia b Harbi Istitute f Techlgy, Harbi , PR Chia article if abstract Article histry: Received 15 September 2008 Received i revised frm 24 Jue 2011 Accepted 5 July 2011 Available lie 23 July 2011 Keywrds: aussia kerels Fuzzy rugh sets Feature selecti Disceribility matrix Fuzzy rugh sets are csidered as a effective tl t deal with ucertaity i data aalysis, ad fuzzy similarity relatis are used i fuzzy rugh sets t calculate similarity betwee bjects. O the ther had i kerel tricks, a kerel maps data it a higher dimesial feature space where the resultig structure f the learig task is liearly separable, while the kerel is the ier prduct f this feature space ad ca als be viewed as a similarity fucti. It has bee reprted there is a verlap betwee family f kerels ad cllecti f fuzzy similarity relatis. This fact mtivates the idea i this paper t use sme kerels as fuzzy similarity relatis ad develp kerel based fuzzy rugh sets. First, we csider aussia kerel ad prpse aussia kerel based fuzzy rugh sets. Secd we itrduce parameterized attribute reducti with the derived mdel f fuzzy rugh sets. Structures f attribute reducti are ivestigated ad a algrithm with disceribility matrix t fid all reducts is develped. Fially, a heuristic algrithm is desiged t cmpute reducts with aussia kerel fuzzy rugh sets. Several experimets are prvided t demstrate the effectiveess f the idea. Ó 2011 Published by Elsevier Ic. 1. Itrducti As a geeral pursuit i the dmai f machie learig, kerel trick allws mappig data frm iput space it a higher dimesial feature space thrugh kerel fuctis i rder t simplify learig tasks ad make them liear (viz. slvable by liear classifiers [37]). I this way, a umber f liear learig algrithms ca be exteded t deal with liear tasks, such as liear SVMs [42], kerel perceptr [5], kerel discrimiat aalysis [37], liear cmpet aalysis [37], kerel matchig pursuit [43], etc. Mst f them emply feature selecti as a preprcessig step. Accrdig t [34], feature selecti aims at pickig ut sme f the rigial iput features (i) fr perfrmace imprvemet by facilitatig data cllecti ad reducig strage space ad classificati time, (ii) t perfrm sematics aalysis i helpig uderstad the prblem, ad (iii) t imprve predicti accuracy by avidig curse f dimesiality. Accrdig t [2,12,13,24], feature selecti appraches ca be divided it filters [8,9,14,15,28], wrappers [24,44] ad embedded appraches [1,4,51]. Acquirig feedback frm classifiers, the filter methds estimate the classificati perfrmace by sme idirect assessmets, such as distace measures which reflect hw well the classes separate frm each ther. The wrapper methds, the ther had, take the classificati perfrmace f a learig machie as a measure f gdess f a subset f features. Wrapper methds usually prvide mre accurate slutis tha filter methds [26,27,44], but are mre cmputatially expesive. Fially, embedded appraches simultaeusly perfrm feature selecti ad classificati mdelig i the traiig prcess. Crrespdig authr. address: chegdegag@263.et (D. Che) /$ - see frt matter Ó 2011 Published by Elsevier Ic. di: /j.is

2 5170 D. Che et al. / Ifrmati Scieces 181 (2011) Fuzzy rugh sets are exteded frm Pawlak s rugh sets [35] fr dealig with decisi tables with real-valued attributes rather tha symblic es. I the existig framewrk f fuzzy rugh sets a fuzzy similarity relati is emplyed t measure similarity betwee tw bjects. There are tw tpics related with fuzzy rugh sets: develpig differet mdels f fuzzy rugh sets ad perfrmig attribute reducti with fuzzy rugh sets. Sice the pieerig wrk i [6], may effrts [29 31,36,45,46] have bee put the first tpic. Detailed summarizatis mdels f fuzzy rugh sets ca be fud i [48]. O the ther had, attribute reducti with fuzzy rugh sets was first prpsed i [20], where fuzzy depedecy fucti was emplyed t measure gdess f attributes by fuzzy rugh sets prpsed i [6] ad a algrithm t cmpute a reduct was develped. Sme researches attribute reducti with fuzzy rugh sets were maily ctributed t imprve the methd i [20], ad sme key ccepts i the traditial rugh sets, such as cre f reducts, were als geeralized t fuzzy rugh sets [3,16,18 23,41,49,50]. As the rle f fuzzy similarity relati i the framewrk f fuzzy rugh sets, kerels als play the rle as similarity measures i the framewrk f kerel tricks. It was pited ut i [32] that there is a clsed relatiship betwee kerels ad fuzzy similarity relatis, i.e., kerels mappig t the uit iterval with 1 i their diagal are a class f fuzzy similarity relatis. This fact aturally mtivates the idea i this paper t csider such kid f kerels as fuzzy similarity relatis t develp kerel based fuzzy rugh sets, ad attribute reducti with kerel based fuzzy rugh sets ca be prpsed as preprcessig step fr kerel tricks. I this paper we select the well-kw aussia kerels as fuzzy similarity relatis i the framewrk f fuzzy rugh sets. We first develp aussia kerels based fuzzy rugh sets ad discuss their graular structures. Secd we defie parameterized attribute reducti with aussia kerel based fuzzy rugh sets ad develp a algrithm with disceribility matrix t cmpute all the reducts. Here we emply psitive regi i fuzzy rugh sets t measure gdess f subsets f attributes ad attributes are distiguished accrdig t their imprtace related t the decisi. Heuristic algrithm t fid reducts is als prpsed. At last, we emply the reducti algrithm fr aussia kerel SVM as a preprcessig step i classificati learig. Hwever, it is table that every kerel mappig t the uit iterval with 1 i its diagal ca als be csidered as a fuzzy similarity fucti i fuzzy rugh sets, ad differet kerels may have differet techiques t perfrm attribute reducti. We discuss aussia kerels i this paper sice they are widely used i the field f machie learig. The prpsed idea ca be emplyed as a preprcessig step f kerel trick related t aussia kerels. The rest f this paper is rgaized as fllws: Secti 2 itrduces kerel trick ad fuzzy rugh sets. Secti 3 develps attribute reducti with aussia kerel based fuzzy rugh sets; the structure f selected attribute set is characterized by apprach f disceribility matrix i this secti. Experimetal results are described i Secti 4. Cclusis are preseted i Secti Reviews kerels ad fuzzy rugh sets 2.1. Psitive defiite kerels Suppsed X R, H is a Hilbert space. k(x,x 0 ) is a ctiuus ad symmetric fucti X X, if there exists a fucti U : X? H satisfyig that k(x,x 0 )=hu(x),u(x 0 )i H, the k(x,x 0 ) is called a psitive defiite kerel [37]. With a psitive defiite kerel k(x,x 0 ), iput vectrs are mapped it a Hilbert space H, called feature space. Accrdig t [37], kerel trick meas that give a algrithm which is frmulated i terms f a psitive defiite kerel k(x,x 0 ), e ca cstruct a alterative algrithm by replacig k(x,x 0 ) with ather psitive defiite kerel kðx; ~ x 0 Þ. I view f defiiti f psitive defiite kerel, the justificati fr this prcedure is that the rigial algrithm ca be thught f as a dt prduct based algrithm peratig the data U(x 1 ),...,U(x m ). The algrithm btaied by replacig k(x,x 0 ) with kðx; ~ x 0 Þ is the exactly the same dt prduct based algrithm. The ly differece cmes frm that they perate Uðx e 1 Þ;...; Uðx e m Þ. eerally speakig, there are maily tw kids f kerels: traslati ivariat kerels ad dt prduct kerels. The traslati ivariat kerels are idepedet f the abslute psiti f iput x ad x 0. They ly deped the differece betwee x ad x 0. S we have k(x,x 0 )=k(x x 0 ). aussia kerel kðx; x 0 Þ¼exp kx x0 k 2 2r is a well kw traslati ivariat 2 kerel. Sme ther traslati ivariat kerels iclude B -splies kerels, Dirichlet kerels ad Peridic kerels. The secd imprtat family f kerels ca be efficietly described i term f dt prduct, i.e., k(x,x 0 )=k(hx,x 0 i), icludig hmgeeus plymial kerels k(x,x 0 )=hx,x 0 i p ad ihmgeeus plymial kerels k(x,x 0 )=(hx,x 0 i + c) p with c P Fuzzy rugh sets I this subsecti we first review fuzzy lgic peratrs fud i [29,31,36,48], the give a brief itrducti f fuzzy rugh sets. Triagular rms (t-rms fr shrt) have bee rigially studied withi the framewrk f prbabilistic metric spaces [38,39]. I this ctext, t-rms prved t be a apprpriate ccept whe dealig with triagle iequalities. Latter, t-rms ad their dual versi t-crms have bee used t mdel cjucti ad disjucti fr may-valued lgic [7,11,25]. A t-rm is a icreasig, assciative ad cmmutative mappig T : [0,1] [0,1]? [0,1] that satisfies the budary cditi ("x 2 [0,1], T(x,1)= x).

3 D. Che et al. / Ifrmati Scieces 181 (2011) A triagular crm (shrtly t-crm) is a icreasig, assciative ad cmmutative mappig S : [0, 1] [0, 1]? [0, 1] that satisfies the budary cditi ("x 2 [0, 1], S(x, 0) = x). ive a t-rm T, the biary perati # T (a,c) = sup{h 2 [0,1] : T(a,h) 6 c} is called a R-implicatr based T. IfT is lwer semi-ctiuus, the # T is called a residuati implicati f T, rat-residuated implicati. I [29] r is defied by r(a, b) = if{c 2 [0, 1] : S(a, c) P b} as the residuated implicati f a t-crm S. A ifrmati system is a pair A =(U,C), where U ={x 1,...,x } is a empty uiverse f discurse ad C ={a 1,a 2,...,a m } is a empty fiite set f attributes. With a subset f attributes B # C we assciate a biary relati IND(B), called B idisceribility relati defied as IND(B) ={(x, y) 2 U U : a(x) = a(y), "a 2 B}. The IND(B) is a equivalece relati ad IND(B)=\ a2b IND({a}). By [x] B we dete the equivalece class f IND(B) icludig x. Fr X # U the sets [{[x] B : [x] B # X} ad [{[x] B :[x] B \ X /} are called B lwer ad B upper apprximatis f X i A, respectively, deted by BX ad BX. Hwever, the abve traditial rugh set mdel ca just deal with databases described with symblic attributes. This limits the applicatis f rugh sets. Several geeralizatis f the traditial rugh sets were csidered. Amg these geeralizatis, the cmbiati f rugh sets ad fuzzy sets develps a pwerful tl, called fuzzy rugh sets, t deal with real-valued datasets. The defiiti f fuzzy rugh sets was first prpsed i [6]. Sice the may effrts have bee devted t develpig ad characterizig mdels f fuzzy rugh sets. Detailed summaries this tpic ca be fud i [41,45 48]. We here just ffer the basic defiitis f fuzzy rugh sets. Suppse U is a empty uiverse f discurses. As a similarity measure betwee tw bjects, a fuzzy T similarity relati R is a fuzzy set U U which is reflexive, symmetric ad T trasitive, amely R(x,z) P T(R(x,y),R(y,z)) hlds. Fr A 2 F(U), the lwer ad upper apprximatis f A are defied as fllws: (1) T upper apprximati peratr: R T AðxÞ ¼sup u2u TðRðx; uþ; AðuÞÞ; (2) S lwer apprximati peratr: R S A(x) = if u2u S(N(R(x,u)),A(u)); (3) r upper apprximati peratr: R r AðxÞ ¼sup u2u rðnðrðx; uþþ; AðuÞÞ; (4) # lwer apprximati peratr: R # A(x) = if u2u #(R(x,u),A(u)). 3. Apprximatis ad attribute reducti with aussia kerels aussia kerels are widely used i kerel tricks. I this secti we csider aussia kerels as fuzzy T similarity relatis t develp aussia kerel based fuzzy rugh sets ad csider attribute reducti with aussia kerels aussia kerel based fuzzy rugh sets Suppse U ={x 1,x 2,...,x m } is a fiite uiverse f discurses, ad every elemet x i 2 U is described by a vectr (x i1, x i2,...,x i ) 2 R. Thus U is viewed as a subset f R. Sice aussia kerel kðx i ; x j Þ¼exp kx i x j k 2 2r takes values i [0,1], it 2 ca be csidered as a fuzzy relati. We dete this fuzzy relati by R, i.e., R ðx i; x j Þ¼exp kx i x j k2 2r. Obviusly R 2 is reflexive ad symmetric. I [32] it is pited ut that R is T cs trasitive, where T cs ða; bþ ¼ p max ab ffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffi 1 a 2 1 b 2 ; 0 is a triagular rm. Thus R is a fuzzy T cs similarity relati. T btai lwer ad upper apprximatis f fuzzy sets related t R, we first derive the residuated implicati f T cs by the fllwig lemma. Lemma ([19,32]). ( 1; a 6 b # Tcs ða; bþ ¼ ffiffiffiffiffiffiffiffiffiffiffiffiffiffi : ab þ ð1 a 2 Þð1 b 2 Þ; a > b Prf. We have # Tcs ða; bþ ¼supfh 2½0; 1Š : T cs ða; hþ 6 bg, sifa6b, the # Tcs ða; bþ ¼1. Suppse a > b. h shuld satisfy ffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffiffiffiffiffiffiffiffiffiffiffiffiffiffi ah ð1 a 2 Þð1 h 2 Þ 6 b. It meas ah b 6 ð1 a 2 Þð1 h 2 Þ. ffiffiffiffiffiffiffiffiffiffiffiffiffiffi Let f 1 (h)=ah b ad f 2 ðhþ ¼ ð1 a 2 Þð1 h 2 Þ. The f 1 (h) strictly icreases [0,1], ad f 2 (h) strictly decreases [0,1]. If ffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffiffiffiffiffiffiffiffiffiffiffiffiffiffi ffiffiffiffiffiffiffiffiffiffiffiffiffiffi h ¼ ab þ ð1 a 2 Þð1 b 2 Þ, the f 1 (h)=f 2 (h). S if h 6 ab þ ð1 a 2 Þð1 b 2 Þ, the f 1 (h) 6 f 2 (h); if h > ab þ ð1 a 2 Þð1 b 2 Þ, ffiffiffiffiffiffiffiffiffiffiffiffiffiffi the f 1 (h)>f 2 (h). This implies that sup fh 2½0; 1Š : T cs ða; hþ 6 bg ¼ ab þ ð1 a 2 Þð1 b 2 Þ. It shuld be ted that result f Lemma have bee metied i [32] withut prf, here we give a prf f it. h I the rest f this paper we dete # Tcs by # cs fr shrt. With T cs ad # cs aussia kerel based fuzzy rugh sets ca be cmputed as: R AðxÞ ¼sup u2ut cs ðr ðx; uþ; AðuÞÞ; R AðxÞ ¼if u2u# cs ðr ðx; uþ; AðuÞÞ.

4 5172 D. Che et al. / Ifrmati Scieces 181 (2011) The prperties f R ad R are discussed i [48] withi the framewrk f geeral fuzzy rugh sets. Here we ly list the ecessary es fr this paper. Therem [48]. R ad R satisfy the fllwig prperties: (1) R A # A # R A; R A ¼ A () R A ¼ A; (2) R ad R are mte; (3) R ðr AÞ¼R A; R ðr AÞ¼R A; R ðr AÞ¼R A; R ðr AÞ¼R A; (4) R ð[ t2ta t Þ¼[ t2t R A t; R ð\ t2ta t Þ¼\ t2t R A t; (5) If R m # R, the R A # Rm A # A # Rm A # R A. By (1) we kw R A ad R A are a pair f fuzzy sets apprximatig A as upper ad lwer buds, respectively, ad by (5) we ca get that a smaller fuzzy relati ca ffer mre precise apprximatis. These prperties are the theretical fudati f attribute reducti described i Secti 3.2. The fllwig therem shws the graular structure f R A ad R A. Therem R A ¼[fR x k : x k # Ag; R A ¼[ R x k : R x k # A, here x k is a fuzzy set, called fuzzy pit, defied as x k ðyþ ¼ k; y ¼ x. 0; y x Prf. Sice A = [{x k : x k # A}, by (4) f Therem R A ¼[fR x k : x k # Ag is bviusly true. Suppse R A ¼[fR z c : R z c # Ag. Fr every x 2 U, suppse k ¼ R AðxÞ, the we have R x kðyþ ¼T cs ðrðx; yþ; kþ ¼T cs Rðx; yþ; sup R z cðxþ : R z c # A ¼ sup T cs ðrðx; yþ; T cs ðrðz; xþ; cþþ : R z c # A 6 sup T cs ðrðz; yþ; cþ : R z c # A ¼ R AðyÞ: Thus R x k # R A hlds. Ad fr ay k0 > k, clearly R x k 0 cat be icluded by A; therwise R AðxÞ ¼k0. It is a ctradicti. S it implies that R x k is the maximal e i the cllecti fr x g : g 2ð0; 1Šg t be icluded by A. O the ther had, fr u 2 U we have R x k # [ R x b : R x bðuþ 6 AðuÞ. Clearly R x k # \ u2u [ R x b : R x bðuþ 6 AðuÞ. Sice [R x b 2 R x g : g 2ð0; 1Š, we have \ u2u [ R x b : R x bðuþ 6 AðuÞ 2fR x g : g 2ð0; 1Šg; thus R x k ¼\ u2u [ R x b : R x bðuþ 6 AðuÞgÞ. Sice R x k is the maximal e i the cllecti fr x g : g 2ð0; 1Šg icluded by A, it implies k ¼ R x kðxþ ¼if u2u sup fb : T cs ðrðx; uþ; bþ 6 AðuÞg ¼ if u2u # cs ðrðx; uþ; AðuÞÞ. h R Accrdig t Therem 3.1.3, M ¼fR x g : x 2 U; g 2ð0; 1Šg ca be emplyed as the basic graular set t cstruct R ad. This statemet plays a key rle i subsecti 3.2 whe we characterize the structure f reducts Parameterized attribute reducti related t aussia kerel based fuzzy rugh sets Suppse U ={x 1,x 2,...,x m } is a fiite uiverse. Each elemet x i 2 U is described by a set C f attributes with umerical values. The attribute value f x i related t the jth attribute is x ij. The pair (U,C) is a ifrmati system. Suppse U is divided it several disjit parts D 1,D 2,...,D s with a decisi attribute D. The the triple (U,C,D) is called a decisi system. A subset f C iduces a fuzzy T cs similarity relati with the aussia kerel. We dete R ðjþ ðx i; x k Þ¼exp kx ij x kj k 2 2r as 2 the e cmputed with the jth attribute i C, the C ca be equivaletly writte by C ¼ R ð1þ ; Rð2Þ ;...; RðÞ. Firstly, we shuld give the aggregati peratr f multiple elemets i C. I the existig fuzzy rugh sets [3,16 23,41,50] t-rm Mi is used as the aggregati peratr f several fuzzy relatis, ad the fuzzy relati after aggregati is just the itersecti f these fuzzy relatis. Hwever, if we select Mi as the aggregati peratr f elemets i C ¼ R ð1þ ; Rð2Þ ;...; RðÞ, the resultig aggregati des t cicide with R due t the fllwig prperty f aussia kerels: R ðx i; x k Þ¼exp kx i x k k 2 2r ¼ Q 2 s¼1 RðsÞ ðx i; x k Þ. Istead f the t-rm Mi, we itrduce the algebraic prduct T P (x,y)=x y as the aggregati peratr. Clearly, i this case the resultig aggregati f elemets i C ¼ R ð1þ ; Rð2Þ ;...; RðÞ is equal t R. I the fllwig we dete the fuzzy relati aggregated by T P(x,y)=xy with elemets i P # C by R P ad still dete R C by R. Secdly, we shuld develp a methd t measure gdess f subsets f cditial attributes. Fr each D t, t =1,2,...,s, if x R D t, the we have R D tðxþ ¼0. If x 2 D t ; R D tðxþ ¼if u2u # cs ðr ðx; uþ; D tðuþþ ¼ if urdt # cs ðr ðx; uþ; D tðuþþ ¼

5 D. Che et al. / Ifrmati Scieces 181 (2011) ffiffiffiffiffiffiffiffiffiffiffi if urdt 1 ðr ðx; uþþ2. R D tðxþ ca be uderstd as the certaity degree t that x belgs t D t accrdig t the attributes i ffiffiffiffiffiffiffiffiffiffiffi C. Oe bvius bservati is that R D tðxþ is determied by the smallest e (the wrst case) f 1 ðr ðx; uþþ2; u R D t. Thus if there is u 0 R D t such that R ðx; u 0Þ is great eugh, i.e., x is quite similar t a bject i ther classes, the R D tðxþ shuld be very small. Ather bservati is that R D tðxþ ¼1 is ever true because R ðx; uþ 0 always hlds, i.e., every pair f bjects are similar i a certai degree with respect t aussia kerels. Ps C ðdþ ¼[ s t¼1 R D t is called the psitive regi f decisi attribute D related t the cditial attribute set C, we will emply psitive regi as a measure f gdess f attributes. Hwever, if we use aussia fucti as the similarity fucti, deletig ay attribute C frm C will result i Ps C D(x i )> Ps C {C} D(x i ) fr every i =1,2,...,m. S we cat emply the idea i the traditial rugh sets [35,40] ad existig fuzzy rugh sets [41] t defie attribute reduct as the miimal subset f C t keep the psitive regi ivariat. This issue is als t metied i [19]. We vercme this prblem by csiderig a threshld e f the psitive regi, ad we ca defie a parameterized attribute reduct with aussia kerel based fuzzy rugh sets by limitig the chage f psitive regi withi the give threshld e. The idea ca be frmulated as fllws. Defiiti Suppse (U,C,D) is a decisi system, e 2 [0,1]. Fr C 2 C, if Ps C D(x i ) Ps C {C} D(x i ) 6 e fr every i =1,2,...,m, the C is called e superfluus i C relative t D; therwise C is called e idispesable i C relative t D. Fr every P # C, ifps C D(x i ) Ps P D(x i ) 6 e fr every i =1,2,...,m, ad every elemet i P is idispesable, the P is called a e reduct f C relative t D. The cllecti f all the e idispesable elemets i C is called the e cre f C relative t D, deted by Cre D (C), ad we have the fllwig therem fr the cre. Therem Cre D (C)=\Red D (C), where Red D (C) is the cllecti f all the e reduct f C relative t D. Prf. If C is e idispesable i C relative t D, the C shuld be icluded i every e reduct f C. Hece Cre D (C) # \Red D (C). O the ther had, if C is e superfluus i C relative t D, the C {C}ctais a e reduct f C, thus there is a e reduct f C that des t iclude C, hece C R \Red D (C). It implies Cre D (C) \Red D (C). h Fr C 2 C, clearly C is e superfluus i C relative t D if ad ly if R D tðxþ e 6 R C fcg D t ðxþ fr every D t, t =1,2,...,l ad x 2 D t, ad if ad ly if x kðxþ # R C fcg D t fr x 2 D t ad kðxþ ¼R D tðxþ e, ad if ad ly if R C fcg x kðxþ # R C fcg D t fr x 2 D t by Therem 3.1.3, here x k(x) is a fuzzy pit. Thus we have the fllwig therem. Therem Suppse P # C. P ctais a e reduct f C if ad ly if R P x kðxþðzþ ¼0 fr x 2 D t,zr D t,t=1,2,...,l. Prf. P ctais a e reduct f C if ad ly if R P x kðxþ # R P D t fr x 2 D t.ifr P x kðxþ # R P D t, the clearly R P x kðxþðzþ ¼0 fr z R D t. h Cversely, if R P x kðxþðzþ ¼0 fr z R D t, the R P x kðxþ # D t which implies R P x kðxþ # R P D t by Therem qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi Therem Suppse P # C. P ctais a e reduct f C if ad ly if there is Q # P such that R Q ðx; zþ 6 1 k 2 ðxþ fr x 2 D t,zrd t,t=1,2,...,l. Prf. R P x kðxþðzþ ¼0 fr x 2 D t ; z R D t ; t ¼ 1; 2;...; l () sup u2u T cs ðr P ðz; uþ; x kðxþðuþþ ¼ 0 () T cs ðr P ðx; zþ; kðxþþ ¼ 0 () RP ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi qffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ðx; zþkðxþ ð1 ðr P ðx; zþþ2 Þð1 k 2 ðxþþ 6 0 () R P ðx; zþ 6 1 k 2 ðxþ () there is Q # P such that R Q ðx; zþ 6 1 k 2 ðxþ. By Therem we fiish the prf. h Therem will be applied t study the structure f e reducti f C ad desig algrithms t cmpute all the e reducts f C i the fllwig subsecti Disceribility matrix based attribute reducti Disceribility matrix is a key ccept t ivestigate attribute reducti i the rugh set framewrk [40]. A reasable defiiti f disceribility matrix ca reveal the structure f attribute reducti, furthermre, it is the theretical fudati t desig algrithms t cmpute reducts. I this subsecti we develp a apprach t fid the e reducts based disceribility matrix. Defiiti Suppse (U,C,D) is a decisi system. By M(U,C,D) we dete a m m matrix (c ij ) mm, called the disceribility matrix f (U, C, D), defied as

6 5174 D. Che et al. / Ifrmati Scieces 181 (2011) (1) if x i ad x j belg t differet decisi classes, c ij ={^P : P # C}, here ^P is the cjucti f elemets i P, ad P satisfies R P ðx i; x j Þ 6 1 k 2 ðx i Þ ad fr Q # P such that R Q ðx i; x j Þ 6 1 k 2 ðx i Þ, the Q = P. (2) c ij = /, therwise. Clearly c ij is the cllecti f all the cjuctis f elemets i P # C thus that P is a miimal e satisfyig R P ðx i; x j Þ 6 1 k 2 ðx i Þ. It is remarkable that M(U,C,D) may t be symmetric i this case. Therem Suppse (U,C,D) is a decisi system, P # C. We have the fllwig tw statemets: (1) P ctais a e reduct f C if ad ly if P \ c ij / fr c ij /, here P \ c ij defied as [{Q # P : ^Q 2 c ij }. (2) Cre D (C)=[{Q ij # C : Q ij = \{P : ^P 2 c ij },i,j=1,2,...,m}. Prf (1) If P ctais a e reduct f C, the fr x i ad x j belg t differet decisi classes, there exists a miimal Q # P such that R Q ðx i; x j Þ 6 1 k 2 ðx i Þ, thus ^Q 2 c ij ad P \ c ij /. Cversely, if P \ c ij / fr c ij /, the there exists a miimal Q # P such that R Q ðx i; x j Þ 6 1 k 2 ðx i Þ, thus P ctais a e reduct f C. (2) If C 2 Cre D (C), the there exist x i ad x j belgig t differet decisi classes such that R C fcg ðx i ; x j Þ > 1 k 2 ðx i Þ.S if P # C such that R P ðx i; x j Þ 6 1 k 2 ðx i Þ, the C 2 P must hld. This implies C 2\fP : ^P 2 c ij g ad Cre D ðcþ # [ Q ij # C : Q ij ¼\fP : ^P 2 c ij g; i; j ¼ 1; 2;...; m : Cversely, if C 2[{Q ij # C:Q ij = \{P : ^P 2 c ij }, i,j =1,2,...,m}, the there exist x i ad x j belgig t differet decisi classes such that C 2\{P : ^P 2 c ij }, s R C fcg ðx i ; x j Þ > 1 k 2 ðx i Þ hlds, which implies C 2 Cre D (C). Thus we fiish the prf. h (2) f Therem prpses a frmula t cmpute the relative cre by disceribility matrix, this frmula will play a key rle whe desig algrithm t cmpute e reduct i Secti 4.1. Crllary Suppse (U,C,D) is a decisi system, P # C. P is a e reduct f C if ad ly if P is the miimal subset f C satisfyig P \ c ij / fr c ij /. A disceribility fucti f(u,c,d) fr (U,C,D) is a Blea fucti f Blea variables C 1 ; C 2 ;...; C crrespdig t the attributes C 1,C 2,...,C i C, respectively, ad defied as f ðu; C; DÞðC 1 ; C 2 ;...; C Þ ¼ ^f_ðc ij Þ : c ij /; 1 6 i; j 6 mg, where _(c ij ) is the disjucti f all elemets i c ij as ^P. By usig f the disceribility fucti, we have the fllwig therem t cmpute all the e reducts f C. Therem Suppse (U,C,D) is a decisi system; M(U,C,D)= (c ij :i,j6 ) is the disceribility matrix f (U,C,D) ad f(u,c,d) is the disceribility fucti f (U,C,D). If f ðu; C; DÞ ¼_ l k¼1 ð^d kþðd k # CÞ is cmputed frm f(u,c,d) by applyig the multiplicati ad absrpti laws as may times as pssible such that every elemet i D i ly appears e time, the the set {D k :k6 l} is the cllecti f all the e reducts f C, i.e., Red D (C)={D 1,...,D l }. Prf. Fr every k =1,...,l, we have D k \ c ij /. Sice f ðu; C; DÞ ¼_ l ð^d k¼1 kþ, fr every D k, if we reduce a elemet C i D k ðd 0 k ¼ D k fcgþ, the f ðu; C; DÞ _ r¼1 k 1ð^D rþ_ð^d 0 k Þ_ð_l D r¼kþ1 rþ ad f ðu; C; DÞ < _ r¼1 k 1ð^D rþ_ð^d 0 k Þ_ð_l D r¼kþ1 rþ. If"c ij,we have D 0 k \ c ij /, the ^D 0 k 6 _c ij, which implies f ðu; C; DÞ P k 1 _ ð^d r Þ_ð^D 0 k Þ l D r ad f ðu; C; DÞ ¼ k 1 _ ð^d r Þ_ð^D 0 k Þ l D r r¼1 r¼kþ1 r¼1 r¼kþ1 it is a ctradicti. Hece there exists c i0 j 0 such that D 0 k \ c i 0 j 0 ¼ /, which implies D k is a reducti f (U,C,D). Fr every X 2 Red D (C), we have X \ c ij / fr every c ij /. S we have f(u,c,d) ^ (^X)=^(_c ij ) ^ (^X)=^X. This implies ^X 6 f(u,d,d). Suppse that fr every k we have D k X /. The fr every k e ca fid C k 2 D k X. By rewritig f ðu; C; DÞ ¼ð_ l k¼1 C kþ^u, we have ^X 6 _ l k¼1 C k. S there is C k0, such that ^X 6 C k0. This implies C k0 2 X, which is a ctradicti. S D k0 # X fr sme k 0, sice bth X ad D k0 are reducts. We have X ¼ D k0. Hece Red D (C)={D 1,...,D l }. h Nw we ca cclude that C ca be categrized it three parts accrdig t their imprtace related t the classificati: (1) elemets i the cre f reducts which shuld be icluded i every reduct; (2) elemets cat be icluded i ay reduct; (3) elemets belg t sme but t all reducts. This partiti als seems reasable i the practical viewpit.

7 D. Che et al. / Ifrmati Scieces 181 (2011) It is wrth pitig ut that the prpsed idea i this paper is t ly limited t aussia kerels, but als applicable t all kerels mappig t the uit iterval with 1 i its diagal. We ca develp fuzzy rugh sets ad csider attribute reducti with this kid f kerels. Hwever, differet kerels may have differet techiques t perfrm attribute reducti. Fr example, we emply the algebraic prduct T P (x,y)=x y as the aggregati peratr fr aussia kerels, ad i the existig attribute reducts [3,10,16 23,41,50] t-rm Mi is emplyed as aggregati peratr fr fuzzy Mi similarity relatis, this differece may lead t differet frmulati f disceribility matrixes. I additi, sice a kerel plays the same rle as a similarity measure i bth attribute reducti ad kerel trick, we suggest t use the same kerel whe attribute reducti is emplyed as a preprcessig step f kerel trick. 4. Experimets ad cmpariss I this secti we will desig a algrithm t cmpute reducts; we will als perfrm attribute reducti as a preprcessig step fr aussia kerel supprt vectr machies i rder t test the effectiveess f the prpsed wrk. Table 1 Descripti f experimetal data. Data Samples Features Classes 1 Credit Heart Hepatitis Hrse I Sar Wie Wpbc (a) umber f selected featues (b) classificati perfrmace f selected features Fig. 1. Variati f size f selected features ad crrespdig classificati perfrmace (sar) (a) umber f selected features (b) classificati accuracies f selected features Fig. 2. Variati f size f selected features ad crrespdig classificati perfrmace (wie).

8 5176 D. Che et al. / Ifrmati Scieces 181 (2011) Algrithm desig ad cmplexity aalysis The algrithm by disceribility matrix is helpful t fid all the reducts f the dataset, but the time cmplexity t fid all the reducts icreases expetially with the umber f attributes O(jUj 2 2 jcj ) [40], where juj is the size f the uiverse, jcj is the umber f cditial attributes. I real applicatis, it is t ecessary t fid all the reducts. It is eugh t address the real prblem by usig e f the reducts. I the fllwig we prvide a heuristic algrithm t fid a reduct. Iput: (U,C,D), Reduct {} Step 1: Cmpute the similarity relati f the set f all cditi attributes: R ; Step 2: Cmpute Ps C ðdþ ¼[ s t¼1 R D t; Step 3: Cmpute c ij by its defiiti i Secti 3; Step 4: Cmpute Cre D (C)=[ {Q ij # C : Q ij = \{P : ^P 2 c ij }, i,j =1,2,...,m}; Delete thse c ij with empty verlap withcre D (C); Step 5: Let Reduct = Cre D (C); Step 6: Add the elemet a whse frequecy f ccurrece is maximum i all c ij it Reduct; ad delete thse c ij with empty verlap with Reduct; Step 7: If there still exist sme c ij /, g t Step 6; Otherwise, g t Step 8; Step 8: If Reduct is t idepedet, delete the redudat elemets i Reduct; Step 9: Output Reduct. The cmputatial cmplexity f this algrithm is O(jUj 2 jcj) Experimetal aalysis I this subsecti, we will perfrm experimets t examie effectiveess f ur idea. We select aussia kerel SVM as a classifier t validate the quality f the features selected by ur techique. Eight datasets are dwladed frm UCI machie learig repsitry [33], described i Table 1. First, we csider the impact f parameters feature selecti. We set r frm 0.1 t 0.4 with step I the meawhile, e is set as 0.01 t 0.1 with step With these parameters, we ca get 100 subsets f attributes ad the crrespdig classificati perfrmace. We perfrm experimets data sets sar ad wie. The results are shw i Figs. 1 ad 2, where the x-axis is r ad y-axis ise. As the bjective f feature selecti is t fid a miimal subspace which has gd classificati perfrmace, s it is expected that the size f the selected feature is relatively small ad the crrespdig classificati perfrmace is gd eugh. We ca see frm the abve results that [0.1,0.2] ad [0.01,0.02] are prper value dmais fr r ad e, respectively. Table 2 Numbers f selected features. Data Raw data KFRS CFS NRS RS Credit Heart Hepatitis Hrse I Sar Wie Wpbc Table 3 Classificati accuracies based aussia kerel SVM (%). Data Raw data KFRS CFS NRS RS Credit ± ± ± ± ± 18.5 Heart ± ± ± ± 6.59 Hepatitis ± ± ± ± ± 7.24 Hrse ± ± ± ± ± 4.45 I ± ± ± ± ± 5.97 Sar ± ± ± ± 7.60 Wie ± ± ± ± ± 4.10 Wpbc ± ± ± ± ± 5.06

9 D. Che et al. / Ifrmati Scieces 181 (2011) We ca see that the classificati accuracies f the reduced data are relatively high ad the sizes f the reduced data are small if we let r ad e take values i these dmais, respectively. Nw we cmpare the umber f the selected features ad classificati perfrmaces f the reducts, shw i Tables 2 ad 3, where reduct is cmputed by the prpsed algrithm ad the classificati perfrmaces f reducts are attaied with aussia kerel SVM based the 10-fld crss validati techique ad aussia kerel SVM is implemeted with su_svm3.00 tlbx. d ad e are specified as 0.1 ad 0.02, respectively. Nw we aalyze experimetal results i Tables 2 ad 3. Cmpared with the raw data, we see that (i) amg the 8 data sets, ur prpsed attribute reducti methd perfrms well six data sets: hepatitis, hrse, wpbc, aeal, i, sar ad wie. Fr these six data sets, umbers f attributes greatly decrease after reducti cmpared with the raw data, ad perfrmaces f classifier (SVM) are imprved distictly. This implies ur prpsed attribute reducti methd ca really delete redudat attributes frm these data sets; (ii) fr data sets credit ad heart, few attributes are deleted, ad imprvemets f perfrmaces f classifier are t sigificat. Hwever, this may due t that there are less redudat attributes i these tw data sets sice the rigial umbers f attributes i these tw data sets are few. I rder t cmpare the prpsed techiques with the existig e, we use eighbrhd rugh set apprach (NRS) [18] ad crrelati based feature selecti (CFS) [14] these data sets. These techiques ca deal with umerical features directly. Frm Tables 2 ad 3, we ca als see that fuzzy rugh sets are better tha ther algrithms imst cases. I additi, we als itrduce the classical rugh set techique t select features with a frward greedy search strategy, deted by RS. As t datasets heart ad sar, feature is retured. This pheme has bee metied i the previus wrk as ay sigle feature prduces the depedecy f zer. S the algrithm stps here. I additi, we gather six cacer recgiti tasks utlied i Table 4. The umbers f features are much mre tha the umbers f samples i these tasks. The detailed descripti abut these tasks ca be gtte frm the webpage ( Overfittig is the mst imprtat challege i gee classificati. Attribute reducti may help vercme this prblem. We perfrm attribute reducti based techiques f eighbrhd rugh sets ad fuzzy rugh sets, respectively. The results are preseted i Tables 5 ad 6. We see that mst f cadidate gees are remved frm classificati learig ad ly a few gees are selected. Mrever, the gees selected by FRS are a little mre tha that by NRS; hwever; the classificati perfrmace is greatly imprved by FRS cmpared with the raw data ad thse selected by NRS. These results shw fuzzy rugh sets are useful i gee selecti fr cacer recgiti. Table 4 ee expressi data sets. Data ees Class Samples Leuk Leuk2 12, SRBCT Breast Lug2 12, DLBCL Table 5 Number f the features selected. Data Raw NRS FRS Breast DLBCL Leukemial Leukemial Lug SRBCT Table 6 Accuracy f the selected gees. Data Raw (%) NRS (%) FRS (%) Breast DLBCL Leukemial Leukemial Lug SRBCT

10 5178 D. Che et al. / Ifrmati Scieces 181 (2011) Cclusi ad future wrk Fuzzy rugh sets are a ht tpic i graular cmputig. I this paper we itrduce aussia kerel it fuzzy rugh sets fr cmputig fuzzy similarity relati ad develp a vel methd f attribute reducti with parameter based the prpsed mdel. We discuss the structure f subsets f selected attributes with fuzzy disceribility matrix. Attributes ca be gruped as three cllectis accrdig t their imprtace related t the decisi. The mai purpse f this paper is t develp attribute reducti with kerel tricks. We use the UCI machie learig data sets ad cacer classificati tasks t test the prpsed techique. The experimetal results shws aussia kerel based fuzzy rugh sets ca fid gd subsets f attributes fr classificati learig. Althugh aussia kerel is frequetly used, there are als sme ther kerel fuctis ca be itrduced it fuzzy rugh sets. We will wrk ther kerels ad develp a set f attribute reducti techiques based fuzzy rugh sets ad kerels. Ackwledgemets This paper is partly supprted by Natial Natural Sciece Fudati uder rats , , ad ad a grat f Natial Basic Research Prgram f Chia (2009CB ). Refereces [1] M.F. Balca, A. Blum, S. Vempala, Kerels as features: kerels, margis, ad lw-dimesial mappigs, Machie Learig 65 (2006) [2] O. Barzilay, V.L. Brailvsky, O dmai kwledge ad feature selecti usig a supprt vectr machie, Patter Recgiti Letters 20 (1999) [3] R.B. Bhatt, M. pal, O fuzzy rugh sets apprach t feature selecti, Patter recgiti Letters 26 (2005) [4] P.S. Bradley, O.L. Magasaria, Feature selecti via ccave miimizati ad supprt vectr machie, i: Prceedigs f the 15th Iteratial Cferece Machie Learig, Sa Fracisc, CA, USA, 1998, pp [5] J.H. Che, C.S. Che, Fuzzy kerel perceptr, IEEE Trasactis Neural Netwrks 13 (2002) [6] D. Dubis, H. Prade, Rugh fuzzy sets ad fuzzy rugh sets, Iteratial Jural f eeral Systems 17 (1990) [7] D. Dubis, H. Prade, A review f fuzzy set aggregati cectives, Ifrmati Scieces 36 (1985) [8] R. Duda, P. Hart, D. Strk, Patter Classificati, secd ed., Jh Wiley & Ss, New Yrk, NY, USA, [9] T. Evgeiu, M. Ptil, C. Papagergiu, T. Pggi, Image represetatis ad feature selecti fr multimedia database search, IEEE Trasactis Kwledge ad Data Egieerig 15 (2003) [10] S. Feradez, J.M. Murakami, Rugh set aalysis f a geeral type f fuzzy data usig trasitive aggregatis f fuzzy similarity relatis, Fuzzy Sets ad Systems 139 (2003) [11] S. ttwald, Fuzzy Sets ad Fuzzy Lgic, Vieweg, Brauschweig, [12] I. uy, A. Elisseeff, A itrducti t variable ad feature selecti, Jural f Machie Learig Research 3 (2003) [13] I. uy, J. West, S. Barhill, V. Vapik, ee selecti fr cacer classificati usig supprt vectr machies, Machie Learig 46 (2002) [14] M. Hall, Crrelati-based feature selecti fr discrete ad umeric class machie learig, i: Prceedigs f the 17th ICML, CA, 2000, pp [15] C.L. Huag, C.J. Wag, A A-based feature selecti ad parameters ptimizati fr supprt vectr machies, Expert Systems with Applicatis 31 (2006) [16] Q.H. Hu, D.R. Yu, Z.X. Xie, Ifrmati-preservig hybrid data reducti based fuzzy-rugh techiques, Patter Recgiti Letters 27 (2006) [17] Q.H. Hu, Z.X. Xie, D.R. Yu, Hybrid attribute reducti based a vel fuzzy-rugh mdel ad ifrmati graulati, Patter Recgiti 40 (2007) [18] Q.H. Hu, D.R. Yu, J. F Liu, C. X Wu, Neighbrhd rugh set based hetergeeus feature subset selecti, Ifrmati Scieces 178 (2008) [19] Q.H. Hu, L. Zhag, D.. Che, W. Pedrycz, D. Yu, aussia kerel based fuzzy rugh sets: mdel, ucertaity measures ad applicatis, Iteratial Jural f Apprximatig Reasig 51 (2010) [20] R. Jese, Q. She, Fuzzy-rugh attributes reducti with applicati t web categrizati, Fuzzy Sets ad Systems 141 (2004) [21] R. Jese, Q. She, Fuzzy-rugh sets assisted attribute selecti, IEEE Trasactis Fuzzy Systems 15 (2007) [22] R. Jese, Q. She, Sematics-preservig dimesiality reducti: rugh ad fuzzy rugh based appraches, IEEE Trasactis Kwledge ad Data Egieerig 16 (2004) [23] R. Jese, Q. She, New appraches t fuzzy-rugh feature selecti, IEEE Trasactis Fuzzy Systems 17 (2009) [24].H. Jh, R. Khavi, K. Pfleger, Irrelevat features ad the subset selecti prblem, i: Prceedigs f the 11th Iteratial Cferece Machie Learig, 1994, pp [25] E.P. Klemet, R. Mesiar, E. Pap, Triagular rms, Treds i Lgic, vl. 8, Kluwer Academic Publishers, Drdrecht, [26] J. Khavi, Wrappers fr feature subset selecti, AIJ issue relevace, 1995 [27] Y. Liu, Y.F. Zheg, FS-SFS: a vel feature selecti methd fr supprt vectr machies, Patter Recgiti 39 (2006) [28] K.Z. Ma, Feature subset selecti fr supprt vectr machies thrugh discrimiative fucti pruig aalysis, IEEE Trasactis Systems, Ma, ad Cyberetics Part B: Cyberetics 34 (2004) [29] J.S. Mi, W.X. Zhag, A aximatic characterizati f a fuzzy geeralizati f rugh sets, Ifrmati Scieces 160 (2004) [30] J.S. Mi, Y. Leug, H.Y. Zha, eeralized fuzzy rugh sets determied by a triagular rm, Ifrmati Scieces 178 (2008) [31] N.N. Mrsi, M.M. Yakut, Aximatics fr fuzzy rugh sets, Fuzzy Sets ad Systems 100 (1998) [32] B. Mser, O represetig ad geeratig kerels by fuzzy equivalece relatis, Jural f Machie Learig Research 7 (2006) [33] D.J. Newma, S. Hettich, C.L. Blake, C.J. Merz, UCI Repsitry f machie learig databases, Uiversity f Califria, Departmet f Ifrmati ad Cmputer Sciece, Irvie, CA, < [34] J. Neuma, C. Schrr,. Steidl, Cmbied SVM-based feature selecti ad classificati, Machie Learig 61 (2005) [35] Z. Pawlak, Rugh sets, Iteratial Jural f Cmputer Ifrmati Sciece 11 (1982) [36] A.M. Radzikwska, E.E. Kerre, A cmparative study f fuzzy rugh sets, Fuzzy Sets ad Systems 126 (2002) [37] B. Schlkpf, A.J. Smla, Learig with Kerels, The MIT Press, [38] B. Schweizer, A. Sklar, Assciative fuctis ad statistical triagle iequalities, Publicaties Mathematicae Debrece 8 (1961) [39] B. Schweizer, A. Sklar, Prbabilistic Metric Spaces, Nrth-Hllad, Amsterdam, 1983.

11 D. Che et al. / Ifrmati Scieces 181 (2011) [40] A. Skwr, C. Rauszer, The disceribility matrices ad fuctis i ifrmati systems, i: R. Slwiski (Ed.), Itelliget Decisi supprt, Hadbk f Applicatis ad Advaces f the Rugh Sets Thery, Kluwer Academic Publishers, [41] C.C.E. Tsag, D.. Che, S.D. Yueg, W.T.J. Lee, X.Z. Wag, Attribute reducti usig fuzzy rugh sets, IEEE Trasactis Fuzzy Systems 16 (2008) [42] V.N. Vapik, The Nature f Statistical Learig Thery, Spriger, New Yrk, [43] P. Vicet, Y. Begi, Kerel matchig pursuit, Machie Learig 48 (2002) [44] J. West, S. Mukherjee, O. Chapelle, M. Ptil, T. Pggi, V. Vapik, Feature selecti fr SVMs, i: T.K. Lee, T.. Dietterich, V. Tresp (Eds.), Advaces i Neural Ifrmati Prcessig Systems, vl. 13, MIT Press, Cambridge, MA, USA, 2001, pp [45] W.Z. Wu, W.X. Zhag, Cstructive ad aximatic appraches f fuzzy apprximati peratrs, Ifrmati Scieces 159 (2004) [46] W.Z. Wu, J.S. Mi, W.X. Zhag, eeralized fuzzy rugh sets, Ifrmati Scieces 151 (2003) [47] W.Z. Wu, Attribute reducti based evidece thery i icmplete decisi systems, Ifrmati Scieces 178 (2008) [48] S.D. Yeug, D.. Che, C.C.E. Tsag, W.T.J. Lee, X.Z. Wag, O the geeralizati f fuzzy rugh sets, IEEE Trasactis Fuzzy Systems 13 (2005) [49] D.R. Yu, Q.H. Hu, C.X. Wu, Ucertaity measures fuzzy relatis ad their applicatis, Applied Sft Cmputig 7 (2007) [50] S.Y. Zha, E.C.C. Tsag, O fuzzy apprximati peratrs i attribute reducti with fuzzy rugh sets, Ifrmati Scieces 178 (2008) [51] J. Zhu, S. Rsset, T. Hastie, R. Tibshirai, 1-Nrm supprt vectr machies, i: S. Thru, L. Saul, B. Schlkpf (Eds.), Advaces i Neural Ifrmati Prcessig Systems, vl. 13, MIT Press, Cambridge, MA, USA, 2004.