Discretization of Continuous Attributes in Rough Set Theory and Its Application*

Transcription

1

2

3 Dscretzaton of Contnuous Attrbutes n Rough Set Theory and Its Applcaton* Gexang Zhang 1,2, Lazhao Hu 1, and Wedong Jn 2 1 Natonal EW Laboratory, Chengdu Schuan, Chna dylan7237@sna.com 2 School of Electrcal Engneerng, Southwest Jaotong Unversty, Chengdu , Schuan, Chna Abstract. Exstng dscretzaton methods cannot process contnuous ntervalvalued attrbutes n rough set theory. Ths paper extended the exstng defnton of dscretzaton based on cut-splttng and gave the defnton of generalzed dscretzaton usng class-separablty crteron functon frstly. Then, a new approach was proposed to dscretze contnuous nterval-valued attrbutes. The ntroduced approach emphaszed on the class-separablty n the process of dscretzaton of contnuous attrbutes, so the approach helped to smplfy the classfer desgn and to enhance accurate recognton rate n pattern recognton and machne learnng. In the smulaton experment, the decson table was composed of 8 features and 10 radar emtter sgnals, and the results obtaned from dscretzaton of contnuous nterval-valued attrbutes, reducton of attrbutes and automatc recognton of 10 radar emtter sgnals show that the reduced attrbute set acheves hgher accurate recognton rate than the orgnal attrbute set, whch verfes that the ntroduced approach s vald and feasble. 1 Introducton Rough set theory (RST), proposed by Pawlak [1], s a new fundamental theory of soft computng. [2] RST can mne useful nformaton from a large number of data and generates decson rules wthout pror knowledge [3,4], so t s used generally n many domans [2,3,5 9]. Because RST can only deal wth dscrete attrbutes, a lot of contnuous attrbutes exstng n engneerng applcatons can be processed only after the attrbutes are dscretzed. Thus, dscretzaton becomes a very mportant extended research ssue n rough set theory. [2,6,7,8] Although many dscretzaton methods, ncludng hard dscretzaton [2,5,6] and soft dscretzaton [7,8], have been presented n rough set theory, the methods are only used to dscretze pont attrbute values (fxed values). However, n engneerng applcatons, especally n pattern recognton and machne learnng, the features obtaned usng some feature extracton approaches usually vary n a certan range (nterval values) nstead of fxed values because of * Ths work was supported by the Natonal Defence Foundaton (No ZS0502). J. Zhang, J.-H. He, and Y. Fu (Eds.): CIS 2004, LNCS 3314, pp , Sprnger-Verlag Berln Hedelberg 2004

4 Dscretzaton of Contnuous Attrbutes n Rough Set Theory and Its Applcaton 1021 several reasons, such as plenty of nose. So a new dscretzaton approach s proposed to process the decson table n whch the attrbutes vary contnuously n some ranges. 2 Defnton of Generalzed Dscretzaton The man pont of the exstng dscretzaton defnton [2] s that condton attrbute space s splt usng selected cuts. The exstng dscretzaton methods [2,5-8] always try to fnd the best cuttng set. When the attrbutes are nterval values, t s dffcult to fnd the cuttng set. So t s necessary to generalze the exstng defnton of dscretzaton. The followng descrpton gves the generalzed dscretzaton defnton. Decson system S = U, R, V, f, where R = A { d} s attrbute set, and the subsets A = { a, a,, a } and { d} are called as condton attrbute set and decson 1 2 m attrbute set respectvely. U = { x, x,, x } 1 2 n s a fnte object set,.e. unverse. For any a ( = 1, 2,, m) A, there s nformaton mappng value doman,.e. x mn mn max mn max mn max x1 x1 x2 x2 xn xn max xj a a a U V a, where Va s the V = {[ v, v ],[ v, v ],,[ v, v ]} (1) a a a a a a a j Where v, v R,( j = 1,2,, n). For attrbute ( A), all objects n unverse U are parttoned usng class-separablty crteron functon JV ( ) a a and an equvalence relaton Ra s obtaned, that s, a knd of categorzaton of unverse U s got. Thus, n attrbute set A, we can acheve an equvalence relaton famly P, ( P = { R, R,, R }) a a a 1 2 m composed of m equvalence relatons R, R,, R a a a 1 2 m. So P the equvalence relaton famly P defnes a new decson system S = U, R, P P P V, f, where f ( x) = k, x U, k = {0,1, }. After dscretzaton, the orgnal decson system s replaced wth the new one. Dfferent class-separablty crteron functons generate dfferent equvalence relaton famles that construct dfferent dscrete decson systems. The core of the defnton s that the dscretzaton of contnuous attrbutes s regarded as a functon that transforms contnuous attrbutes nto dscrete attrbutes. The new defnton s an extended verson of the old one and emphaszes on the separablty of dfferent classes n dscretzaton. The defnton provdes a good way to dscretze contnuous nterval-valued attrbutes. Addtonally, consderng the separablty of classes n the process of dscretzaton can smplfy the structure of classfer and enhance accurate recognton rate. If mn max x x j j v = v,( j = 1,2,, n), the dscretzaton of a a nterval-valued attrbutes becomes the dscretzaton of fxed-pont attrbutes. If the class-separablty crteron J ( ) s a specal functon that can partton the value doman V of attrbute a ( a A) nto several subntervals, that s, the specal functon a decdes a cuttng-pont set n the value doman V a, the defnton of generalzed dscretzaton becomes the common defnton of dscretzaton.

5 1022 G. Zhang, L. Hu, and W. Jn 3 Dscretzaton Algorthm The key problem of dscretzng nterval-valued attrbutes s to choose a good classsepablty crteron functon. So a class-sepablty crteron functon s gven frstly n the followng descrpton. When an attrbute value vares n a certan range, n general, the attrbute value always orders a certan law. Ths paper only dscusses the decson system n whch the attrbutes have a certan law. To the extracted features, the law s consdered approxmately as a knd of probablty dstrbuton. Suppose that functons f( x) and g( x) that are one-dmensonal, contnuous and non-negatve real, are respectvely the probablty dstrbuton functons of attrbute values of two objects n unverse U n decson system. The below class-separablty crteron functon s ntroduced. J = 1 f( x) g( x) dx ( ) ( ) 2 2 f x dx g x dx Functon J n (2) satsfes the three condtons of class-separablty crteron functons [10]: () the crteron functon value s non-negatve; () the crteron functon value gets to the maxmum when the dstrbuton functons of two classes have nonoverlappng; () the crteron functon value equals to zero when the dstrbuton functons of two classes are dentcal. Because f( x) and g( x) are non-negatve real functons, accordng to the famous Cauchy Schwartz nequaton, we can get ( ) ( ) ( ) ( ) f x g x dx f x dx g x dx (3) So the value doman of the crteron functon J n (2) s [0,1]. The followng descrpton gves the nterpretaton that functon J can be used to justfy whether the two classes are separable or not. When the two functons f( x) and g( x ) n (2) are regarded respectvely as probablty dstrbuton functons of attrbute values of two objects A and B n unverse U, several separablty cases of A and B are shown n fgure 1. For all x, f one of f( x) and g( x ) s zero at least, whch s shown n Fg. 1(a), A and B are completely separable and the crteron functon J arrves at the maxmal value 1. If there are some ponts of x that make f ( x) and g( x ) not equal to 0 smultaneously, whch s shown n Fg.1(b), A and B are partly separable and the crteron functon J les n the range between 0 and 1. For all x, f f( x) = k g( x), k R +, whch s shown n Fg.1(c), k = 2, A and B are unseparable completely and the crteron functon J arrves at the mnmal value 0. Therefore, t s reasonable that the crteron functon J s used to evaluate separablty of two classes. Accordng to the above class-separablty crteron functon, the dscretzaton algorthm of nterval-valued attrbutes n decson system s gven as follows. Step 1. Intalzaton: decdng the number n of objects n unverse U and the number m of attrbutes. (2)

6 Dscretzaton of Contnuous Attrbutes n Rough Set Theory and Its Applcaton 1023 Step 2. Constructng decson table: all attrbute values are arrayed nto a twodmensonal table n whch all attrbute values are represented wth an nterval values. Step 3. Choosng a lttle postve number as the threshold T of class separablty. h Step 4. For the attrbute a (n the begnnng, = 1 ), all attrbutes are sorted by the central values from the smallest to the bggest and sorted results are v, v,, v. 1 2 n Step 5. The poston, where the smallest attrbute value v 1 n attrbute a s, s encoded to zero (Code=0) to be the ntal value of dscretzaton process. Step 6. Begnnng wth v 1 n the attrbute a, the class-separablty crteron functon value J k of v and v ( k = 1,2,, n 1) s computed by the sorted k k + 1 order v 1, v 2,, v n turn. If J T, whch ndcates the two objects are separable n k h completely, the dscrete value of the correspondng poston of attrbute v k + adds 1, 1.e. Code=Code+1. Otherwse, J < T, whch ndcates the two objects are k h unseparable, the dscrete value of the correspondng poston of attrbute v k + keeps 1 unchangng. Step 7. Repeatng step 6 tll all attrbute values n attrbute a are dscretzed. Step 8. If m, whch ndcates there are some attrbute values to be dscretzed, = + 1, the algorthm goes to step 4 and contnues untl > m, mplyng all contnuous attrbute values are dscretzed. Step 9. The orgnal decson system s replaced wth the dscretzed one. f(x) f(x) f(x) g(x) g(x) g(x) (a) J =1 (b) 0< J <1 (c) J =0 Fg. 1. Three separablty cases of functons f ( x) and g( x) 4 Applcaton Example 10 radar emtter sgnals (RESs), represented wth x 1, x 2,, x 10 respectvely, are chosen to make the smulaton experment. Attrbute set s made up of 8 features [11-13] that are represented wth a 1, a 2,, a 8. When sgnal-to-nose (SNR) vares from 5 db to 20 db, 8 features extracted from 10 RESs construct the attrbute table shown n Table 1. Table 2 s the dscretzed result of Table 1 usng the proposed approach. In the process of dscretzaton, the parameters n = 10, m = 8, T h = 0.99, and all attrbute values are regarded as a Gaussan dstrbuton functons wth the parameters expectaton and varance. After dscretzaton, the reducton method usng dscernblty matrx and logc operaton [2,3] s used to reduce Table 2. The fnal result s L = a ( a + a a + a ( a + a ( a + a ) + a a )) (4)

7 1024 G. Zhang, L. Hu, and W. Jn Table 1. The attrbute table before dscretzaton Attrbutes A 1 a 2 a 3 a 4 x 1 [0.6903,0.7071] [1.4018,1.4036] [0.1217,0.1249] [0.2707,0.2724] x 2 [0.6711,0.6821] [1.4296,1.5758] [0.1151,0.1615] [0.2136,0.2774] x 3 [0.5460,0.5976] [1.5823,1.5857] [0.2924,0.3040] [0.2251,0.2261] x 4 [0.8927,0.8959] [1.4370,1.4900] [0.3735,0.6523] [0.9896,1.000] x 5 [0.8966,0.9089] [1.5610,1.7682] [0.2002,0.2230] [0.7253,0.7270] x 6 [0.4194,0.4528] [1.2190,1.2238] [0.5616,0.5856] [0.1410,0.1424] x 7 [0.3959,0.4129] [1.2805,1.2841] [0.0802,0.0818] [0.0395,0.0404] x 8 [0.6569,0.6841] [1.4788,1.4832] [0.0657,0.0671] [0.1821,0.1914] x 9 [0.5211,0.5619] [1.3938,1.4004] [0.0780,0.0806] [0.7066,0.7294] x 10 [0.9048,0.9076] [1.3841,1.4585] [0.3317,0.3481] [0.6250,0.6301] Attrbutes a 5 a 6 a 7 a 8 x 1 [0.4346,0.4365] [1.9615,2.3013] [0.9516,1.0209] [0.4201,0.4334] x 2 [0.3619,0.4278] [1.0884,2.9913] [1.1154,1.2605] [0.4055,0.4549] x 3 [0.3741,0.3755] [0.7260,1.1110] [0.6909,0.7549] [0.5343,0.5495] x 4 [0.8562,0.8672] [11.738,20.222] [2.1570,2.9600] [0.1985,0.2009] x 5 [0.3304,0.3369] [6.8174,7.7484] [2.8992,3.1304] [0.1986,0.2015] x 6 [0.2434,0.2439] [2.6183,2.9668] [0.1946,0.3095] [0.7721,0.7866] x 7 [0.0000,0.0007] [6.4044,6.7258] [0.2569,0.3264] [0.6758,0.6873] x 8 [0.1149,0.1289] [2.1474,3.3136] [1.4230,1.6642] [0.3932,0.4133] x 9 [0.5843,0.6121] [17.351,19.069] [0.9215,0.9771] [0.5043,0.5241] x 10 [0.5561,0.5606] [0.3448,4.668] [3.1142,3.7371] [0.2022,0.2108] In (4), there are 6 reducts correspondng to 6 feature subsets. The complexty of feature extracton s ntroduced to select the lowest complexty reducts. Fnally, the feature subset composed of a, a, a s regarded as the fnal result after computng To test the performances of the reduced result, three-layer BP neural network s used to desgn classfers. For every RES, 150 feature samples are generated n each of 5dB, 10dB, 15dB and 20dB. Thus, 600 samples of each RES n total are generated when SNR vares from 5dB to 20dB. The samples are classfed nto two groups: tranng group and testng group. The tranng group, whch conssts of one thrd of all samples, s appled to tran neural network classfers (NNC). The testng group, represented by other two thrds of samples, s used to test traned NNC. The structure of NNC s We choose RPROP algorthm [14] as the tranng algorthm. Ideal outputs are 1. Output tolerance s 0.05 and output error s The average accurate recognton rates of 50 experments are shown n Table 3. To brng nto comparson, the orgnal feature set composed of 8 features s also used to recognze the 10 sgnals and the results are also shown n Table 3. Attrbute reducton method not only smplfy the structure of classfers greatly, but also the average accurate recognton rate amounts to 97.23%, whch s 1.18% hgher than the orgnal feature set made up of 8 features.

8 Dscretzaton of Contnuous Attrbutes n Rough Set Theory and Its Applcaton 1025 Table 2. The attrbute table after dscretzaton Attrbutes a 1 a 2 a 3 a 4 a 5 a 6 a 7 a 8 x x x x x x x x x x Table 3. Accurate recognton rates (%) before reducton (BR) and after reducton (AR) Sg x 1 x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 x 10 BR AR Concludng Remarks Ths paper generalzes frstly the defnton of dscretzaton based on cut-splttng and gves a generalzed defnton of dscretzaton usng class-separablty functon. Also, a new dscretzaton approach s proposed to dscretze contnuous nterval-valued attrbutes. Expermental results show that the ntroduced approach can dscretze contnuous nterval-valued attrbutes effectvely. The reduced attrbute set smplfes the structure of classfers and acheves hgher accurate recognton rate than the orgnal attrbute set. References 1. Pawlak, Z.: Rough sets. Informatonal Journal of Informaton and Computer Scence. Vol.11, No.5. (1982) Ln, T.Y.: Introducton to the specal ssue on rough sets. Internatonal Journal of Approxmate Reasonng. Vo.15. (1996) Wang, G.Y.: Rough set theory and knowledge acquston. X an: X an Jaotong Unversty Press, (2001) 4. Walczak, B., Massart, D.L.: Rough sets theory. Chemometrcs and Intellgent Laboratory Systems. Vol.47. (1999) Shen, L.X., Tay, F.E.H, Qu, L.S., and Shen, Y.D.: Fault dagnoss usng rough set theory. Computers n Industry. Vol.43. (2000) Da, J.H., L Y.X.: Study on dscretzaton based on rough set theory. Proc. of the frst Int. Conf. on Machne Learnng and Cybernetcs. (2002)

9 1026 G. Zhang, L. Hu, and W. Jn 7. Roy, A., Pal, S.K.: Fuzzy dscretzaton of feature space for a rough set classfer. Pattern Recognton Letter. Vol.24. (2003) Susmaga, R.: Analyzng dscretzatons of contnuous attrbutes gven a monotonc dscrmnaton functon. Intellgent Data Analyss. Vol.1. (1997) Kusak, A.: Rough set theory: a data mnng tool for semconductor manufacturng. IEEE Trans. on Electroncs Packagng Manufacturng. Vol.24, No.1. (2001) Ban, Z.Q., Zhang, X.G.: Pattern recognton. Bejng: Tsnghua Unversty Press, (2000) 11. Zhang, G.X., Jn, W.D., and Hu, L.Z.: Fractal feature extracton of radar emtter sgnals. Proc. of 3 rd Asa-Pacfc conf. on Envronmental Electromagnetcs. (2003) Zhang, G.X., Hu, L.Z., and Jn, W.D.: Complexty feature extracton of radar emtter sgnals. Proc. of 3 rd Asa-Pacfc Conf. on Envronmental Electromagnetcs. (2003) Zhang, G.X., Rong, H.N., Jn, W.D., and Hu, L.Z.: Radar emtter sgnal recognton based on resemblance coeffcent features. LNCS. Vol (2004) Redmller M., Braun H.: A drect adaptve method for faster back propagaton learnng: the RPROP algorthm. Proc. of IEEE Int. Conf. on Neural Networks. (1993)