An Improved Prediction Model based on Fuzzy-rough Set Neural Network

Transcription

1 Internatonal Journal of Computer Theory and Engneerng, Vol.3, No.1, February, An Improved Predcton Model based on Fuzzy-rough Set Neural Network Jng Hong Abstract After the bref revew of the basc prncples and characterstcs of BP (back-propagaton) neural network and rough set theory, a novel artfcal neural network model based on fuzzy-rough set s proposed n ths paper, whch s sutable for nonlnear regresson to acheve the precse predcton result by ntroducng mproved rough set to obtan mnmal attrbute set to solve the optmzed problem of nput layer. Compared to the normal predcton model, the feasblty and effectveness of the proposed model s verfed by a practcal case study of the mne gas emsson predcton. Index Terms fuzzy-rough set, neural network, predcton model I. INTRODUCTION Inductve learnng for rules generaton s an mportant research area n artfcal ntellgence. Recently, many technques have been developed to perform nductve learnng [1,,3,4,5]. Among them rough set theory s a new tool to solve vague and uncertan data analyss whch was put forward by Polsh scholar Z. Pawlark based on the thought of borderlne of G.Frege [6],and whch to some extent overcomes the lmtatons of fuzzy sets theory and Dempster Shafer theory whle solvng uncertanty problems. The basc dea of the theory s to classfy the objects of nterest nto smlarty classes (equvalent classes) contanng ndscernble objects va the analyss of attrbute dependency and attrbute reducton [1,]. The rule nducton from the orgnal data model s data-drven wthout any addtonal assumptons, namely whch requres no external parameters and uses only the nformaton presented n the gven data, and enrches the multple dmenson data n the drecton of attrbute and case, and consstently or nconsstently reduces to fnd hdden law of the data to be treated. It s wdely used n medcal dagnoss, pattern recognton, machne learnng, expert system and etc. In practce, however, there are some lmtatons wth the classcal rough set. The orgnal rough set can deal wth the dscrete attrbutes effcently, but t cannot deal wth the contnuous attrbutes well. For contnuous attrbutes, the tradtonal decson table s normally transformed nto bnary table by parttonng the attrbute value nto several ntervals subjectvely. However, there s no crsp boundary between the neghborng concepts. On the other hand, the orgnal rough set s based on the ndscernblty relaton. The unverse s classfed nto a set of equvalent classes wth the ndscernblty relaton. The lower and upper approxmatons are generated n terms of the equvalent classes. So the orgnal rough set classfes the knowledge too fussly, whch leads to the complexty of the problem. The fuzzy set theory and rough set theory are generalzatons of classcal set theory for modelng vagueness and uncertanty [1]. Pawlak proposed that the two theores were not compettve but complementary [7]. Dubos and Prade also proposed that they were related but dstnct and complementary theores [8]. Both of the theores model dfferent types of uncertanty. The rough set theory takes nto consderaton the ndscernblty between objects, whereas the fuzzy set theory deals wth the ll-defnton of the boundary of a class through the membershp functons. Hence, t s possble to combne the two theores to enhance the reasonng power of an ntellgent system. BP network s one of the most wdely appled artfcal networks [9, 10 ], whch s a feed-forward mult-layer mappng net and has powerful mappng capablty for nonlnear problems. It converts the problem of nput and output of a set of samples to nonlnear optmzaton, makng use of the most common method of optmzaton namely gradent descent method, teratvely computng net connecton weghts whch correspond to learnng and memory problem, and ntroduces hdden nodes to add adjustable parameter to get more precse soluton. So t has ncomparable superorty such as arbtrarly approachng a none-lnear functon and parallel reasonng, and has been wdely used n pattern recognton, system dentfcaton, predcton, control, mage processng, functon ftness and etc. However, the predcton model based on normal BP network whch apples the general factors as nput nodes may result n over-large scale and mpar the convergency and accuracy, and very dffcult to predct tmely and precsely. It s necessary to explore a novel method to effectvely solve the problem nput node selecton of classcal neural network nput layer whch s generally subjectve and random., In ths paper, an mproved predcton mothod based on fuzzy-rough set neural network model s presented, whch ntroduces the fuzzy-rough set theory as a complementary tool of BP network to construct the model by usng mnmal attrbute set to solve the optmzed problem of nput layer. In secton, the fuzzy-rough set neural network model s dscussed n detal. Secton 3 descrbes one practcal case study of the mne gas emsson to llustrate how to apply the proposed model and demonstrate ts effectveness and advantages over other normal predcton models. Fnally, conclusons are drawn n secton

2 Internatonal Journal of Computer Theory and Engneerng, Vol.3, No.1, February, II. FUZZY-ROUGH SET NEURAL NETWORK MODEL A. BP neural network model In 1989, Robert Hecht Nelson proved that any contnuous functons on a close nterval can be approxmaton by BP network wth a hdden layer, that s a three-layer BP network can complete the mappng from and n dmenson to m dmenson space [11]. The model has ncomparable superorty n machne reasonng, capacty of fault tolerance, parallel reasonng, arbtrarly approachng a none-lner functon etc. The number of hdden layer, the number of nodes of each hdden layer, weghts, learnng rate, momentum constant and reecho test should be consdered when constructng a model. Accordng to some experences, the prncples of settng parameters are respectvely shown as follows. 1) Number of hdden layer Increasng the number of hdden layers can reduce the error rate effectvely, but t also brngs about some large problems, such as over-large scale network, longer tranng tme so as to mpar the system s practcalty. In fact, the precson can also be ncreased by ncreasng the number of nodes of hdden layer, whch s more adjustable than others. The BP network n ths paper has three layers. ) Number of the nodes n hdden layer To determne the number of the nodes n hdden layer s a complcated problem, whch currently does not have analytcal expresson to use. Wth the BP network used as functon approxmaton, the number of the nodes of the hdden layer relates to the precson of functon approxmaton and the fluctuaton degree of the functon. Generally speakng, f the number of nodes n hdden layer s too large, the learnng tme may be too long and the scale s too over-lager. On the other hand, f the number of nodes n hdden layer s too small, the traned network would not be robust enough to recognze the samples whch do not learn before, and has poor performance of fault tolerance. In practce, people reference some experence formula But sometmes, formulas were not proper for some problems. We compare several result of dfferent node number of hdden layer, and add lttle nodes as ntalzaton, then optmze the model wth stepwse regresson method. Some smulatons prove t feasble. 3) Connecton weght Before the tranng of a BP network have to determne ntal connecton weght of the net. Due to the nonlnearty of the system, there are normally several local mnmums on error curve. δ f (s) n standard BP. When the slop of error curve s small ( f (s) 0, then δ 0), the connecton weght can not be tuned effectvely. In ths research, we apply addtonal momentum to mprove standard BP model. The modfed formula s ΔW ( t + 1) = (1 α ) ηδ O + αδw ( t), j pj pj j whch takes the nfluence of last weght transformaton by momentum factor nto consderaton, and adjustng α based on the rate error change n the mean tme. The formulas to adjust α are shown as follows; 0.00 SSE( k) > SSE( k 1) α = (1) 0.95 SSE( k) SSE( k 1) 159 Where SSE(k) s the rate of the kth teratve absolute error change. 4) Learnng rate and momentum Learnng rate and momentum constant are two mportant parameters n the tranng of a BP network, and each tme the degree of BP network modfyng the net connecton weght manly les on these parameter. The quantty of weght transformaton depends on the learnng rate greatly. If the learnng rate s too large, the system has poor performance on stablty. On the other hand, f the learnng rate s too small, the learnng tme may be too long. Dfferent error curve matchng proper learnng rate can mprove system performance. We adjust the learnng rate by the followng formula n ths research. Smulatons prove that the network s convergent very quckly. η ( t + 1) = η( t) E( t 1) / E( t) () ( p) Where E = E / N, N s capacty of learnng sample, p E s total average error. Momentum constant has the functon of smoothng learnng procedure, whch s an ncreased momentum rato wth arthmetc mprovement. Increasng learnng rate should properly reduce the value of momentum constant to avod sharp wave crest or trough. 5) Reecho test and predcton test Reecho test checks out whether the actual output of the net satsfes the requrements by coeffcent of correlaton R. R>0.9 usually means that the traned net s acceptable, and the more R approaches 1, the better the net fts the data. Predcton test makes the smulaton of the traned BP network wth the data of the test set and computes the relatve error, and s the lttle the relatve error, the stronger s the ablty of generalzaton of the network. By reecho test and predcton test check the precson and generalzaton of BP network whether or not to meet the precson requrement. The formula of coeffcent of correlaton s show as follow, Σ( F E) R = 1 (3) Σ( E Y ) Where: R s coeffcent of correlaton, F s the actual output of the net, E s the expectaton of output of the net, Y s mean of the expectaton of output of the net. B. Fuzzy-rough set model In ths secton, the fuzzy-rough set model s dscussed n detal, ncludng fuzzfyng the contnuous (numercal) attrbutes, new defntons based on fuzzy smlarty relaton and attrbute reducton based on the fuzzy smlarty relaton. 1) Fuzzfyng the contnuous attrbutes A decson table wth 4-tuple can be represented as T=<U,C D,V,f>, where U s the unverse, C and D are sets of condton and decson attrbutes respectvely, V s the value set of the attrbute a n A, and f s an nformaton functon. Practcally, there are many contnuous or numercal attrbutes n the decson table, such as salary and experence. These attrbutes need to be fuzzfed nto lngustc terms, such as hgh, average and low. In other words, each attrbute a s fuzzfed nto k lngustc values T, =1,,k. The

3 Internatonal Journal of Computer Theory and Engneerng, Vol.3, No.1, February, membershp functon of T can be subjectvely assgned or transferred from numercal values by a membershp functon. The popular trangular membershp functon s shown n Fg. 1, where μ(x) s membershp value and x s attrbute value. The slop of trangular membershp functons are selected n the way that adjacent membershp functons cross at the membershp value 0.5, so the only parameters need to be determned are the set of k centers M={m, =1,,,k}. The center m can be calculated through Kohonen s feature-map algorthm [1]. 1.0 μ(x) m1 m m3 mk-1 mk x Fgure 1 Trangular membershp functon ) New defntons based on fuzzy smlarty relaton The classcal lower and upper approxmatons are orgnally ntroduced wth reference to an ndscernblty relaton (reflexve, symmetrc, and transtve). Practcally, t can be extended to fuzzy smlarty relaton. Defnton 1: Consderng U={u1, u,, un} s the ~ unverse, the fuzzy smlarty relaton R on U s R n n called fuzzy smlarty matrx, f each element r j. R ~ has the two followng propertes: Reflexve: r = 1, = 1,,, n; Symmetrc: rj = rj,, j = 1,,..., n. In order to constructng the fuzzy smlarty relaton, the measurement of fuzzy smlarty relaton should be ntroduced frst, namely the method to calculate the factor r j. Generally, the max-mn method, relatonal factor method and Mnkowsk dstance based closeness degree method are used. Defnton : Consderng R ~ s a fuzzy smlarty matrx and s the level value, the matrx R ~ s called normal smlarty relaton matrx wth the level value after the followng operaton. r = 1, r ; j j, j=1,,,n r = 0, r <. j j Obvously, matrx R ~ has the reflexve property and symmetrc property. In order to obtan the classfcaton of U gven the fuzzy smlarty relaton, an algorthm s desgned as follows, Algorthm 1: Input: fuzzy smlarty matrx R ~ and level value Output: U/IND( R ~ ), whch s a partton of U gven fuzzy smlarty relaton R ~ and level value Step 1 Calculate normal smlarty relaton matrx R ~ n terms of defnton ; Step x U, X φ, Y φ; Step 3 j 0; Step 4 If rj=1 and xj X, then X X {xj}, Y Y {xj} ; Step 5 j j+1; Step 6 If j<n, then GOTO Step 4; otherwse, GOTO next step; Step 7 If card(y)>1, then select x Y and Y Y-{x}, GOTO Step 3; otherwse, GOTO next step; Step 8 Output the set X and let U U-X; Step 9 If U=φ, then end; otherwse, GOTO Step. ~ { a } Accordng to the algorthm, U/IND( R { } ), the classfcaton gven the attrbute a A wth the level value, s calculated. The classfcaton of U gven the attrbute set A wth the level value set can be defned as follows, ~ { a } U/IND( R ) = {U/IND( R { } ): a A, } (4) Where A and are the attrbute set and the level value set, respectvely, and operator s defned as follows, A B={ X Y : X A, Y B, X Y Φ} (5) Defnton 3: Consderng a subset X U and a fuzzy smlarty relaton R defned on U, the lower A approxmaton of X, denoted by R ~ (X), and upper approxmaton of X, denoted by R -(X), are respectvely defned as follows, A R ~ (X)= {Y:Y U/IND( R ),Y X); (6) R -(X)= {Y:Y U/IND( R ),Y X ).(7) ~ C Defnton 4: Assumng U/IND( R ) and Y are two ~ C parttons on U, where U/IND( R ) ={X1, X,,Xk} and Y={Y1,Y,, Yr}, the postve regon POS C (Y ) s defned as follows, POS Y = { R ~ C ( ) ( Y ) : Y Y} C.(8) 3) Attrbute reducton fuzzy based on smlarty relaton In decson system, reducton s the dependence and assocaton of the decson attrbutes on condton attrbutes. In practce, there s a lot of redundant nformaton n the orgnal data source. Attrbute reducton can remove the redundant or nose nformaton successfully. In the attrbute reducton, the attrbute reducton set s not sngle. The cardnalty of reducton set determnes the dmensonalty of problem, so t s mportant to select a mnmal reducton. The mnmal reducton can be defned as follows, Defnton 5: Consderng T=<U,C D,V, f> s a decson table, the classfcaton of U wth C and D are respectvely denoted as X={X 1,X,,X m }and Y={Y 1,Y,,Y n }. The condtonal entropy of C wth D s defned as follows, m n ( j j H D C) = P( X ) P( Y X ) log P( Y X ) Wher e p( X ) = card( X ) / card( U ), = 1,,..., m.defnton 6: Consderng T=<U,C D,V, f> s a decson table, R C, the sgnfcance for each attrbute a C-R s defned as follows: 160

4 Internatonal Journal of Computer Theory and Engneerng, Vol.3, No.1, February, SIG ( a, R, = ω ( R { ( γ ( R { γ ( R, ) 1 + ω ( R { ( H ( D R) H ( D R { a}) log n) (9) Where ω R { = card( POS ( ) card( ) ; 1 ( R { a} U ω ( R { = 1 ω1( R {. In order to obtan the mnmal condton attrbute set, an algorthm s desgned as follows, Algorthm : Input: decson table Output: the mnmal attrbute set R; Step1 classfy condton attrbuter set C wth fuzzy level value; Step calculate Core(C, by dscernble matrx and let R Core(C,; Step 3 calculate SIG(a,R,for each attrbuter; Step 4 select attrbuter a C - R wth maxmum SIG (a,r,and let R R {x}; Step 5 f r(c,==r(r,,then GOTO step 6;. otherwse, GOTO step 3; Step 6 return R. Obvously, the computatonal complexty of the algorthm s O(m), where m s the number of condton attrbute set n decson table. In terms of the algorthm mplementaton, attrbute reducton can be treated as a tree traversal. Each node of the tree represents the condton attrbute. Calculatng the mnmal reducton can be transformed to pckng the best path based on some heurstc nformaton. On the other hand, operator can reduce the computaton by usng results from the prevous levels. 4) Predcton procedure based on fuzzy-rough set neural network Step 1 select proper fuzzy level value, then fuzzy classfy the orgnal data n terms of the algorthm 1; Step reduce the condton attrbuters n terms of the algorthm ; Step 3 select mnmal attrbuters as nput nodes of neural network, then set proper hdden layer and construct model; Step 4 tran and test model; Step 5 f the error meets lmts, then GOTO end; otherwse, GOTO step 1. III. EXPERIMENT RESULT AND ANALYSIS Recently, Artfcal neural network has been appled n mne gas emsson predcton [14,15].In ths secton, we use ths practcal engneerng case to llustrate the predcton process and dscuss the effectveness of the proposed method. A. Experment result Wth the research of the gas emsson predcton of a coal mne[15], the nonlnear ftness of gas emsson data ncludng 18 cases. Accordng to geographcal structure and explotaton condtons of the coal mne condton determne the components of the nput vector are depth, gas content of coal seam, space between coal seam, the daly output of workng face, thckness of coal seam, and crculatory style, and the output vector s absolute gas emsson. We select eghteen months samples to evaluate the constructed model performance, among whch the frst sxteen months samples are used for tranng, and the seventeenth month and eghteenth month samples are used for testng. On account of each sample dmensonalty changes largely, the sample data should be normalzed. The predcton model s realzed n VC In the model, the three-layer BP network s appled and average system error s , and both weght and value are ntalzed by random functons. The performance of dfferent optmzed models based on several mnmal attrbute set gven dfferent fuzzy classfcaton are shown n Fg. and Tab. 1, respectvely. TABLE 1 COMPARISON OF THE PREDICTED RESULTS 17th month floodng 18th month floodng Number quantty m 3 /mn quantty m 3 /mn Iteratv of nput Relatv Relatv e tme Predcto Predcto nodes e e error n value n value error% % Fgure Analyss of the predcted results B. Experment analyss As we can see from Fg., because the fuzzy credblty level value s dfferent, predcton model based on fuzzy-rough set neural network has several nput sets. In [15], all the factors have been used as nput nodes (namely, the number of nput nodes s sx). Due to the redundant attrbutes both the precson and effcency are not satsfed. It s shown from Tab. 1 that when the number of reduced attrbutes s mnmal, the model s convergent very quckly whle the accuracy s low; as the number of nput nodes reaches three, the model performs better. Another attrbute wth most mportance among the rest attrbutes s contnually added ntentonally and the number of nput nodes reaches four. After 3 tmes of teratve, the gas emssons of 17th month and 18th month are 5.03m3/mn and 7.86m3/mn, respectvely. Compared to practcal results, the correspondng relatve errors are.9% and 1.84%, whch shows that the precson of predcton s not obvously mproved despte the longer tranng tme. Accordng to analyss above, we can draw the followng conclusons. 1) We can get some key factors or attrbutes by usng the proposed fuzzy-rough set model. ) There s a balance between the number of nput nodes and model performance. The convergency and accuracy of the predcton model wll be mproved when the balance reaches. 3) We can select dfferent predcton model to meet 161

5 Internatonal Journal of Computer Theory and Engneerng, Vol.3, No.1, February, dfferent requrements, so the model s of great flexblty. IV. CONCLUSION The classcal rough set theory and BP network have respectvely superorty and shortage. Hence, t s possble to combne the two theores to enhance the reasonng and predctng power of an ntellgent system. The analyss of the gas emsson predcton model based on fuzzy-rough set neural network, whch usng mnmal attrbute set to solve the optmzed problem of nput layer, proves that the model can perform better than the tradtonal neural network n both accuracy and convergency. Furthermore, the model can also be appled n other complcated engneerng problems. ACKNOWLEDGEMENT The paper s supported by the Youth Foundaton of Scence, Shangha, P. R. Chna (05XPYQ45) REFERENCES [1] Z. Pawlak, AI and ntellgent ndustral applcatons: the rough set perspectve. Cybernetcs and Systems: An Internatonal Journal, 31(4), pp. 7-5, 000 [] Q. Shen and A. Chouchoulas, FuREAP: A fuzzy-rough Estmator of algae populatons, Artfcal Intellgence n Engneerng, 15, pp.13-4, 001 [3] Wang Shyue lang and Tsa Jenn Shng. Dscovery of approxmate dependences from proxmty-based fuzzy databases. The Thrd Internatonal conference on Knowledge-based Intellgent Informaton Engneerng System, Adelade, Australa, IEEE Inc.pp , 1999 [4] Y. Yuan and M. J. Shaw, Inducton of fuzzy decson trees, Fuzzy Set and System, 69(), pp , [5] Y. Y. Yao, A comparatve study of fuzzy sets and rough sets, Journal of Informaton Scences, 109(1-4), pp. 7-4, 1998 [6] Z.Pawlark, Routh sets, Theoretcal Aspects of Reasonng about Data. Nowowejska 15/19,Warsaw.poland,10 [7] Z. Pawlak, Rough sets and fuzzy sets. Fuzzy Sets and Systems, 17(1), pp.88-10, 1985 [8] D. Dubos and H. Prade, Rough fuzzy sets and fuzzy rough sets[j]. Int. J. General Systems, 17, pp ,1990 [9] Lppmann P R. Pattern Classfcaton usng Neural Networks. IEEE Comn., Magazne, Aprl 1987, 4- [10] Zhang Zhen-yu, Xe Xao-yao, Applcaton research of BP neural network n telecom plannng predcton, Computer Engneerng and Applcatons,008,44(0):45-48 [11] R. Hecht-Nelsen, Theory of the back-propagaton of neural network, IEEE INNS Internatonal Conference on Neural Network, 1, pp , 1989 [1] T. Kohonen, Self-Organzaton and Assocatve Memory, Sprnger, Berln, 1988 [13] Hong Jng, Lu Jng-gu, Sh Feng, Combnng Fuzzy Set and Rough Set for Inductve Learnng. Intellgent Informaton Processng II, Amerca: Sprnger, [14] Xue Peng-qan, Wu L-feng, L Ha-jun, Predctng the Amount of Gas Emtted Based on Wavelet Neural Network, Chna Safety Scence,006,:-5 [15] Yang Zh-y, Xong Ya-xuan, Zhang Qan-ln, Research on the predcton of gas emsson n workng face based on neural network Coal Engneerng, 004,10:73-75 [16] R. Slownsk and D. Vanderpooten, A generalzed defnton of rough approxmatons based on smlarty, IEEE Transactons on Knowledge and Data Engneerng, 1(), pp , 000 Jng Hong was born n Kunmng Cty, Yunnan Provnce, n She receved the master dploma n Computer Applcaton from Nanjng Unv. of Technology. Her current research nterests nclude research & applcaton of algorthms of data mnng and ntellgent computaton. She s workng as a lecturer n College of electronc engneerng, Shangha Unversty of Engneerng Scence. Emal:nj_jnghon00@163.com. 16