FRULEX - Fuzzy Rules Extraction Using Rapid Back Propagation Neural Networks

Transcription

1 FRULEX - Fuzzy Rules Etraton Usng Rapd Ba Propagaton Neural Networs Mohamed Farou Adel Hady Teahng Assstant Dr. Mahmoud Wahdan Proets Manager Prof. Adel Elmaghray Atng Char Informaton and Computer Sene Department Insttute of Statstal Studes and Researh Unversty of Caro mhady@man-s.aro.edu.eg Mnstry of Communatons and Informaton Tehnology mwahdan@mt.gov.eg Computer Engneerng and Computer Sene Department J. B. Speed Sentf Shool Unversty of Lousvlle adel@lousvlle.edu Astrat: In ths paper we present a new approah for etratng fuzzy rules from numeral nput output data for pattern lassfaton. The approah omnes the merts of the fuzzy log theory and neural networs. The proposed approah uses rapd a propagaton neural networ RBPNN whh an handle oth quanttatve numeral and qualtatve lngust nowledge. The networ an e regarded oth as an adaptve fuzzy nferene system wth the apalty of learnng fuzzy rules from data and as a onnetonst arhteture provded wth lngust meanng. Fuzzy rules are etrated n three phases: ntalzaton optmzaton and smplfaton of the fuzzy model. In the frst phase the data set s parttoned automatally nto a set of lusters ased on nputsmlarty and output-smlarty tests. Memershp funtons assoated wth eah luster are defned aordng to statstal means and varanes of the data ponts. Then a fuzzy f-then rule s etrated from eah luster to form a fuzzy model. In the seond phase the etrated fuzzy model s used as startng pont to onstrut an RBPNN then the fuzzy model parameters are refned y analyzng the nodes of the networ that was traned va the a propagaton gradent desent method. In the thrd phase a smplfaton method s used to redue the anteedent parts n the etrated fuzzy rules. Keywords: Feature seleton fuzzy rule ase gradent-desent loal funton neural networs mean-square error neural fuzzy systems logal rule etraton smlarty test.. Introduton System modelng s the tas of modelng the operaton of an unnown system from a omnaton of pror nowledge and measured nput-output data. It plays a very mportant role n many areas suh as ommunatons ontrol epert systems et. Through the smulated system model one an easly understand the underlyng propertes of the unnown system and handle t properly. When we are tryng to model a omple system usually the only avalale nformaton s a olleton of mprese data; t s alled fuzzy modelng whose oetve s to etrat a model n the form of fuzzy nferene rules. Zadeh [] proposed the fuzzy set theory to deal wth suh nd of unertan nformaton

2 and many researhers have pursued researh on fuzzy modelng however ths approah las a defnte method to determne the numer of fuzzy rules requred and the memershp funtons assoated wth eah rule. Also t las an effetve learnng algorthm to refne these funtons to mnmze output errors. Another approah usng neural networs was proposed whh le fuzzy modelng s onsdered to e a unversal appromator. In reent years there has een a prolferaton of methods usng ths approah See [2] and [3] for surveys of the feld. Ths approah has advantages of eellent learnng apalty and hgh preson. However t usually suffers from slow onvergene loal mnma and low understandalty. Consderale wor has een done to ntegrate neural networs wth fuzzy nferene systems resultng n neuro-fuzzy modelng approahes suh as [5] and [6] whh omne the enefts of these two powerful paradgms nto a sngle apsule.e. adaptalty qu onvergene representaton power and hgh auray. Jang and Sun [4] have shown that fuzzy systems are funtonally equvalent to a lass of radal ass funton RBF networs ased on the smlarty etween the loal reeptve felds of the networ and the memershp funtons of the fuzzy system. The rest of ths paper s organzed as follows. The net seton refly desres the rapd a propagaton neural networs. Seton 3 gves an overvew of the FRULEX approah. Self-Construtng Rule Generator SCRG s desred n Seton 4. Seton 5 desres the tranng of RBP networs. Smplfaton of the etrated fuzzy model s desred n Seton 6. Epermental results are presented n Seton 7. An evaluaton of FRULEX s presented n Seton 8. Fnally onlusons and future wor are gven n Seton Rapd Ba Propagaton Neural Networs In the feld of Artfal Neural Networs ANN s there are several types of networs that utlze unts wth loal response haratersts to solve lassfaton and funton appromaton prolems. Whle there are many methods for etratng rules from spealzed networs there are a small numer of pulshed tehnques for etratng rules from loal ass funton networs. Tresp Hollatz and Ahmed [7] desre a method for etratng rules from gaussan Radal Bass Funton RBF networ. Berthold and Huer [8] desre a method for etratng rules from a spealzed loal funton networ the Retangular Bass Funton ReBF networ. Ae and Lan [9] desre a reursve method for onstrutng hyper-oes and etratng fuzzy rules from them. Duh et. al. [0] desre a method for etraton optmzaton and applaton of sets of fuzzy rules from soft trapezodal memershp funtons. Lapedes and Faer [] gve a method for onstrutng loally responsve unts usng pars of as-parallel logst sgmod funtons. Sutratng the value of one sgmod from the other one wll onstrut suh loal response regon. They dd not however offer a tranng sheme for networs onstruted of suh unts. Geva and Stte [2] desre a parameterzaton and tranng sheme for networs omposed of suh sgmod ased hdden unts. Andrews and Geva [3] [4] propose a method to etrat and refne rsp rules from these networs. RBP networs are smlar to RBF networs n that the hdden layer onssts of a set of loally responsve unts. 2

3 Eah loal response unt LRU of the hdden layer of the RBP networ s onstruted as follows: In eah nput dmenson form a regon of loal response aordng to the equaton ; ; ; r + σ σ + σ σ e e Ths onstruton forms an as parallel rdge funton n the th dmenson of the nput spae that s almost zero everywhere eept n the regon etween the steepest part of the two logst sgmod funtons See Fgure and Fgure 2. The parameters ; r ; and of the sgmod funtons and represent the enter readth and edge steepness respetvely of the rdge and ; + σ σ s the nput value. Fgure : Construton of a rdge Fgure 2: Cylndral Etenson of a rdge The nterseton of suh N rdges wth a ommon enter produes a funton f that represents a loal pea at the pont of nterseton wth seondary rdges etendng to nfnty on ether sdes of the pea n eah dmenson See Fgure 3. The funton f s the sum of the N rdge funtons N r f ; ; 2 Fgure 3: Interseton of two Rdges Fgure 4: Produton of an LRU 3

4 To mae the funton loal these omponent rdges must e ut off y the applaton of a sutale sgmod to leave a loally responsve regon n nput spae see Fgure 4. The funton that elmnates the unwanted regons of the radated rdge funtons s shown elow l ; σ K f ; B 3 Where B s seleted to ensure that the mamum value of the funton f loated at ondes wth the enter of the lnear regon of the output sgmod. The parameter K determnes the steepness of the output sgmod l ;. The parameter B s set to produe appreale atvaton only when eah of the nput values le n the rdge defned n the th dmenson. The parameter K s hosen suh that output sgmod l ; uts off the seondary rdges outsde the oundary of the loal funton. Eperment has shown that good networ performane an e otaned f B s set equal to the nput dmensonalty B N and K s set n the range 2-4. A networ that s sutale for funton appromaton and nary lassfaton tass an e reated wth an nput layer a hdden layer of rdge funtons a hdden layer of loal funtons and an output unt. See Fgure 5 and Fgure 6 Fgure 5: Construton of a loal funton Fgure 6: Struture of an RBP Networ The atvaton for the output unt s gven as: J y w l ; 4 whh s a lnear omnaton of J loal response funtons wth enters wdths and steepness. Where w s the output weght assoated wth eah of the ndvdual loal response funtons l. Networ output s smply the weghted sum of the outputs of the loal response funtons. For mult-lass lassfaton prolems several suh networs an e omned one networ per lass wth the output lass eng the mamum of the atvatons of the ndvdual networs that omnaton s alled MCRBP Networ. 4

5 The RBP networ s traned usng gradent desent on an error surfae to adust the parameters output weghts and the ndvdual rdge enter readth and edge steepness. 3. Overvew of FRULEX Approah Our approah of aqurng fuzzy rules from a gven data set an e shown n Fgure 7. Input-Output Data Intalzaton Phase Self Construtng Rule Generator Optmzaton Phase Ba Propagaton Learnng Smplfaton Phase Feature Suset Seleton By Relevane Fnal Fuzzy Rules Fgure 7: Phases of FRULEX Approah In the ntalzaton phase a set of ntal fuzzy rules s etrated from the gven data set wth an adaptve self-onstrutng rule generator [5]. The th fuzzy rule s defned to tae the followng form: R IF IS µ AND... AND IS µ AND... AND IS µ : N N N y IS w AND... AND y IS w AND AND ym IS w M 5 THEN... Where w s a onstant representng the th onsequent part and µ are memershp funtons eah of whh s a rdge funton wth enter wdth and steepness.e. σ + σ µ r ; 6 σ σ Note that for eah rule the frst anteedent orresponds to the frst nput the seond anteedent orresponds to the seond nput et. and the onluson orresponds to the output. The frng strength of the rule s 5

6 N α r ; 7 Also we use the entrod defuzzfaton method to alulate the output of ths fuzzy system as follow: J J 4 y α. w α 8 In the parameter optmzaton phase we mprove the auray of the ntal fuzzy system wth neural networ tehnques. In the rule ase smplfaton phase FRULEX mplements faltes for smplfyng the optmzed rule set n order to mprove the omprehenslty of the rule set. A four-layer MCRBP neural networ s onstruted ased on the fuzzy rules otaned y SCRG method shown n the frst phase shown n Fgure 8. The operatons of the MCRBP neural networ s desred as follows: Layer ontans N nodes. Node of ths layer produes output y transmttng ts nput sgnal dretly to layer 2.e. for N O Layer 2 ontans J groups and eah group ontans N nodes. Eah group representng the IF-part of a fuzzy rule. Node of ths layer produes ts output y omputng the value of the orrespondng normalzed rdge funton for N and J O 2 r r ; σ σ + σ σ Layer 3 ontans J nodes. Node of ths layer produes ts output y omputng the value of the logst funton.e. for J O 3 N 2 ; σ K O B 9 0 l Layer 4 ontans M nodes. Node of ths layer produes ts output y the entrod defuzzfaton.e. J 3 O. w 4 J 3 O O Clearly and w are the parameters that an e tuned to mprove the performane of the fuzzy system. We use the apropagaton gradent desent method to refne these parameters. 2 6

7 O 4 O 4 O M 4 w w wjm Group O 3 Group O 3 O J 3 Group J O 2 O NJ 2 O N 2 O 2 Fgure 8: Arhteture of Rapd Ba Propagaton Neural Networ Traned RBP networs an e used for numer nferene or fnal fuzzy rules an e etrated from networs for symol reasonng. N 4. Self-Construtng Rule Generator Frst the gven nput-output data set s parttoned nto fuzzy overlapped lusters. The degree of assoaton s strong for data ponts wthn the same fuzzy luster and wea for data ponts n dfferent fuzzy lusters. Then a fuzzy f-then rule desrng the dstruton of the data n eah fuzzy luster s otaned. These fuzzy rules form a rough model of the unnown system and the preson of desrpton an e mproved n the phase of parameter dentfaton. Unle ommon lusterng-ased methods suh as fuzzy -means method [6] whh requre the numer of lusters and hene the numer of rules to e approprately preseleted SCRG performs lusterng wth the alty to adapt the numer of lusters as t proeeds. For a system wth N nputs and M outputs we defne a fuzzy luster as a par w l s defned as: l where l ; σ K r ; B N l 3 where [... ] [... ] [... ] [... ] N N N N K and w denote the nput vetor enter vetor wdth vetor steepness and heght vetor respetvely of the luster. Let J e the numer of estng fuzzy lusters and S e the sze of luster. 7

8 Clearly J ntally equals zero. For an nput-output nstane v p where [ p p ] p... v v vn and q [ q v v... qvm v q v ]. We alulate l for eah estng luster J. We say that nstane v passes nput-smlarty test on luster f l p ρ 4 e v For eah luster on whh nstane v has passed the nput-smlarty test. Let d q ma - q mn where q ma and q mn are the mamum and mnmum value of the th output respetvely of the gven data set. We say that nstane v passed the output-smlarty test on luster f e τ d 6 where τ 0 τ s another predefned threshold. v l t pv ma l p l 2... v p l v f p v p v where ρ 0 ρ s a predefned threshold. Then we alulate v q 5 v w We have two ases. Frst there s no estng fuzzy lusters on whh nstane v has passed oth nput-smlarty and output-smlarty tests. For ths ase we assume that nstane v s not lose enough to any estng luster and a new fuzzy luster J+ s reated wth o and p v suh that [... ] w q v 7 X X. Where X and X o o... o on and o o upper lower upper lower are the upper and lower lmt of the th nput respetvely of the gven data set and s a user-defned onstant vetor. Note that the new luster ontans only one memer nstane v at ths tme. Of ourse the numer of estng lusters s nreased y and the sze of luster should e ntaled to.e. J J+ and S. 8 Seond f there est a numer of fuzzy lusters on whh nstane v has passed oth nput-smlarty and output-smlarty tests let these lusters are 2 and f and let the luster t e the luster wth the largest memershp degree. 9 o In ths ase we assume that nstane v s losest to luster t and luster t should e modfed to nlude nstane v as ts memer. The modfaton to luster t s shown elow for N St t o + Stt + pv St + Stt + p v t + 0 St S t St 20 + Stt + pv t 2 S + t 8

9 St wt + qv w t 22 S + t S t S t + 23 Note that J s not hanged n ths ase. The aove-mentoned proess s terated untl all the nput-output nstanes have een proessed. At the end we have J fuzzy luster. Note that eah luster s desred as l w. where l ontans enter vetor and wdth vetor. We an represent luster y a fuzzy rule havng the form of 5 wth µ r ; 24 for N and the onluson s w for M. Fnally we have a set of J ntal fuzzy rules for the gven nput-output data set. Wth ths approah when new tranng data are onsdered the estng lusters an e adusted or new luster an e reated wthout the neessty of generatng the whole set of rules from the srath. 5. Ba Propagaton Gradent-Desent Learnng Algorthm After the set of J ntal fuzzy rules s otaned n phase one we mprove the auray of these rules wth neural networ tehnques n the phase of parameter optmzaton. Frst a four-layer fuzzy rules-ased RBP networ s onstruted y turnng eah fuzzy rule nto a loal response unt LRU as shown n Fgure 8. A gradent-ased optmzaton method performng the steepest desent on a surfae n the networ parameter spae s used. The goal of ths phase s to adust oth the premse and onsequent parameters so as to mnmze the mean squared error funton shown elow P E E v 25 P v M where E e 2 4 e v 2 v v y v qv and y v O p s the atual output of the v th v tranng pattern. The update formula for a gener weght α s α η α E α 26 where η α s the learnng rate for that weght. In summary gven a tranng set T of P T p q : v... P p... p q... q. tranng patterns { } { } v v v vn v vm For the sae of smplty the susrpt v ndatng the urrent sample wll e dropped n the followng. Startng at the frst layer a forward pass s used to ompute the atvty levels of all the nodes n the networ to otan the urrent output values. Then startng at the output layer a award pass s used to ompute E α for all the nodes. 9

10 The omplete learnng algorthm s summarzed as follow:.. J.. M Intalze the weghts { } and { } the SCRG phase... N w.. J wth rule parameters otaned n 2 Selet the net nput vetor p from T propagate t through the networ and determne the output 4 y O 3 Compute the error terms as follows: 4 4 δ O q 27 M δ M δ w O O 28 t δ t δ KO O 29.. J.. M 4 Update the gradents of { } and { }.. N w.. J respetvely aordng to: + + E 2 σ σ σ σ δ σ σ E 2 σ σ σ σ + δ σ σ 3 3 E 4 O + J w δ 3 O 32 t t.. J 5 After applyng the whole tranng set T Update the weghts { }.. N.. M { } respetvely aordng to: w.. J E η 33 E η 34 K o 35 E w η 36 w where η eng the learnng rate y a proper seleton of η the speed of onvergene an e vared and K o s the ntal steepness. and 0

11 6 If E < ε or mamum numer of teratons reahed stop else go to step 2. where ε s the error goal 6. Feature Suset Feature Seleton By Relevane Sne n applaton areas le medne not only the auray ut also the rule smplty and omprehenslty s mportant the etrated fuzzy model was redued y applyng a feature seleton algorthm to ope wth the hgh dmensonalty of the real-world dataset. Startng wth an ntal traned RBP networ havng the omplete set of features the algorthm teratvely produes a sequene of networs wth smaller set of nput nodes. The teratve nature of our feature seleton method allows a systemat nvestgaton of the performane of redued networ models wth fewer nput nodes. The proposed feature seleton method s omputatonally heap as the overall omputatonal ost of eah teraton depends manly on the tranng of redued networ we frst sort the features aordng to ther relevane for the lassfaton. That s for eah feature an RBP neural networ s reated y usng the full feature set eept that feature. The lassfaton auray of the networ on the test dataset s saved for that suset. The feature whose orrespondng networ produes the smallest lassfaton auray s the most relevant one. We sort features n asendng order aordng to ther orrespondng networs test lassfaton auray whh s the est feature set. Then an RBP neural networ s reated y usng the est feature the most relevant one. The lassfaton auray of the networ on the test dataset s saved for that suset. Net the est two features are tested followed y the est three features and t goes le that tll the est N features N numers of features are tested. For eample If the sorted lst s le {f f 2... f N }. we test the susets {f } {f f 2 } {f f 2 f 3 } {f f 2... f N }. We fnd the suset wth the est test set lassfaton auray. Sne we want the smallest feature we tae the full feature set auray a full as our ase and fnd the smallest suset wthn a a β. ertan range of that auray net a f ull For eample f the auray of the full feature set s 95% and est urrent suset wth 3 features has auray of 97% and net est suset wth 2 features has the auray of 92% and β 5% then we hoose the suset wth 2 features eause 92% 95% 5% and t eomes the est suset. An outlne of the feature suset seleton algorthm s gven n Fgure 9. Here s how we fnd the fnal est feature suset: In eah fold we fnd the est suset. as mentoned aove For eah feature we fnd n how many folds that feature s a memer of ts est suset. Then we fnd the average-tmes-n-est-suset value total of tmes-n-est-suset values of all the features dvded y the numer of features.

12 For the fnal feature suset we hoose the feature that appeared n more susets than the average-tmes-n-est-suset value. vstedlst emptyset; N numfeatsfullfeatureset; for 0; < N; ++ { urrentsuset fullfeatureset feature Construt an RBP Networ y usng urrentsuset and the tranng set Test the RBP Networ y usng test set Fnd the lassfaton auray a of the test set Add the par feature a to the vstedlst } sort the vstedlst n asendng order aordng to auray Now the vstedlst s sorted from the most relevant feature to the least esta -; urrentsuset emptyset; estsuset urrentsuset; for 0; < N; ++ { f esta 00 AND numfeatsestsuset STOP Add the net most relevant feature to the urrentsuset Construt an RBP Networ y usng urrentsuset and the tranng set Test the RBP Networ y usng test set Fnd the lassfaton auray urrenta of the test set f urrenta > fulla Beta AND numfeatsurrentsuset < numfeatsestsuset { esta urrenta; estsuset urrentsuset; } } return estsuset Fgure 9: Feature Suset Seleton y Relevane Algorthm 7. Epermental Results The valdty of our approah to fuzzy reasonng and rule etraton has een tested on a well-nown enhmar prolem from lterature [7]. 7. Irs Flower Classfaton Prolem The lassfaton prolem of the Irs data onssts of lassfyng three spees of rs flowers setosa versolor and vrgna. There are 50 samples for ths prolem 50 of eah lass. Eah sample s a four-dmensonal pattern vetor representng four attrutes of the rs flower See Tale and Tale 2. Results otaned wth varous methods for ths dataset are olleted n Tale 6. 2

13 ID Class Setosa 2 Versolor 3 Vrgna Tale : Classes for the rs flower lassfaton dataset ID Feature Feature values F Sepal length [ ] F2 Sepal wdth [ ] F3 Petal length [.0 6.9] F4 Petal wdth [0. 2.5] Tale 2: Features and Feature values for the rs flower lassfaton dataset A MCRBP networ wth 4 nputs and 3 outputs orrespondng to the 3 lasses was onstruted. The whole data set was dvded nto two parts. A part onsstng of 5 samples unformly drawn from the three lasses was used as a test set for the networ traned wth the remanng 35 data ponts. The SCRG method desred n Seton 4 s used to determne the ntal enters and wdths of the memershp funtons of the nput features. The results after fnshng SCRG are shown n Tale 3. We tae σ o 0.2 ρ 0.0 and τ 0.95 Irs After SCRG Tranng Set Test Set Whole Set Run Rules Class. No. of Class. No. of Class. No. of Auray Mslass. Auray Mslass. Auray Mslass Average Tale 3: Results of the 0-fold ross valdaton after SCRG phase for the rs flower lassfaton dataset The apropagaton gradent desent method Seton 5 s used to optmze the fuzzy rule ase etrated n phase one. The results otaned after 00 epohs are shown n Tale 4. We tae ε 0.0 and η.0 3

14 Irs After BP Tranng Set Test Set Whole Set Run Rules Class. No. of Class. No. of Class. No. of Auray Mslass. Auray Mslass. Auray Mslass Average Tale 4: Results of the 0-fold ross valdaton after BP learnng phase for the rs flower lassfaton dataset Fgure 0: Fuzzy rules otaned after BP learnng phase for the rs flower lassfaton dataset The Feature Suset Seleton By Relevane method Seton 6 s used to smplfy the fuzzy rule ase etrated n phase one. The results otaned after ths phase are shown n Tale 5. We tae β 0 Features Sortng y Relevane Test Classfaton Auray F F2 F3 F4 Removed Feature Fgure : Performane of RBP networ durng the removal of nput features of the rs flower lassfaton dataset 4

15 Features Suset Seleton Test Classfaton Auray F4 F3 F2 F Added Feature Fgure 2: Performane of the RBP networ wth dfferent features of the rs flower lassfaton dataset After Smplfaton Irs Tranng Set Test Set Whole Set Class. No. of Class. No. of Class. No. of Run Rules Auray Mslass. Auray Mslass. Auray Mslass. No. of Ante. Best Feature Set F F F F F F F F F F F F4 Average F4 Tale 5: Results of the 0-fold ross valdaton after Smplfaton phase for the rs flower lassfaton dataset Fgure 3: Fuzzy rules otaned after Smplfaton phase for the rs flower lassfaton dataset 5

16 Approah Classfaton Auray Rules Numer Anteedents Per rule Referene Rule Type MLP wth BP 97.36% N/A N/A Ster et al. [20] N/A BIO-RE 78.67% 4 3 Taha et al. [9] C Full-RE 97.33% 3 to 2 Taha et al. [9] C RULEX 94.0% 5 3 Andrews et al. [8] C FRULEX 94.4% 3 Our result F Tale 6: Comparng FRULEX approah to some other approahes for the rs flower lassfaton dataset 8. Evaluaton of FRULEX Approah There are s dfferent rtera for the evaluaton of our approah as follows: A. Rule Format It an e seen that FRULEX etrats fuzzy rules. In the dretly etrated fuzzy system eah fuzzy rule ontans an anteedent ondton for eah nput dmenson as well as a onsequent whh desres the output lass overed y that rule. B. Complety of the approah FRULEX unle other deomposton algorthms does not rely on any form of searh to etrat rules rather t reles on the dret analyss of the weghts of the traned networ. The ntalzaton module s lnear n the numer of fuzzy lusters or fuzzy rules and the numer of tranng patterns OJ.P. The optmzaton module s lnear n the numer of teratons numer of tranng patterns and the numer of hdden nodes OI.P.J. The module assoated wth rule smplfaton s lnear n the numer of features the numer of teratons numer of tranng patterns and the numer of hdden nodes ON.I.P.J. Therefore FRULEX s omputatonally effent and ts usage to nlude an eplanaton falty adds lttle overhead to the learnng phase of a neural networ. C. Qualty of the etrated rules As stated prevously the essental funton of rule etraton algorthms suh as FRULEX s to provde an eplanaton falty for the traned networ. The rule qualty rtera provde nsght nto the degree of trust that an e plaed n the eplanaton. Rule qualty s assessed aordng to the auray fdelty and omprehenslty of the etrated rules. C.. Auray Durng tranng phase loal response unts wll grow shrn and/or move to form a more aurate representaton of the nowledge enoded n the tranng data. 6

17 C.2. Fdelty Fdelty s losely related to auray and the fators that affet auray also affet the fdelty of the rule sets. In general the rule sets etrated y FRULEX dsplay an etremely hgh degree of fdelty wth the networs from whh they were drawn. C.3. Comprehenslty In general omprehenslty s nversely related to the numer of rules and to the numer of anteedents per rule. The RBP networ s ased on a greedy overng algorthm. Hene ts solutons are aheved wth relatvely small numers of tranng teratons and are typally ompat.e. the traned networ ontans a small numer of loal response unts. Gven that FRULEX onverts eah loal response unt nto a sngle fuzzy rule therefore the etrated rule set ontans the same numer of rules as the numer of loal response unts n the traned networ. D. Conssteny of the approah Rule etraton algorthms that generate rules y queryng the traned neural networ wth patterns drawn from the prolem doman may generate a varety of dfferent fuzzy models from any gven tranng run of the neural networ. Suh algorthms may have low onssteny. FRULEX on the other hand s a determnst algorthm that always generates the same fuzzy model from any gven tranng run. Hene FRULEX always ehts 00% onssteny. E. Translueny of the approah FRULEX s a deompostonal approah as fuzzy rules are etrated at the level of the hdden layer unts. Eah loal response unt s treated n solaton wth the output weghts eng onverted dretly nto a fuzzy rule. F. Portalty of the approah FRULEX s non-portale havng een spefally desgned to wor wth RBP networs whh s a loal funton networ. Ths means that t annot e used as a general-purpose deve for provdng an eplanaton omponent for estng traned neural networs. However the RBP networ s applale to a road range of prolem domans nludng ontnuous valued dsrete valued domans and domans whh nlude mssng values. Hene FRULEX s also applale to a road varety of prolem domans. 9. Conlusons We developed a new fuzzy rules etraton approah FRULEX. FRULEX s ale to provde an eplanaton falty for the MCRBP traned networ. It etrats fuzzy systems from traned feedforward RBP networs and smplfes the fuzzy system n a way to mamze the fdelty etween the system and the neural networ. 7

18 The man features and advantages of the proposed approah are: It s a general framewor that omnes two tehnologes namely neural networs and fuzzy systems The nowledge emedded nsde the RBP networ an e eplaned n onept of a fuzzy model and hene t an e easly understood. The numer of fuzzy rules s determned automatally and the memershp funtons math losely wth the real dstruton of the tranng data ponts The seleton of relevant features s automat. Also t learns faster and produes hgher lassfaton auray than other mahne learnng methods as shown n the seton of epermental results. In many applaton areas le medal dagnoss not only the auray ut also the rule smplty and omprehenslty s mportant. Our approah are of ths n the thrd phase. 9. Future Wor The followng are the suets of our on-gong researh:. Funton appromaton: We are plannng to apply our approah to funton appromaton prolems. 2. Mamdan-type fuzzy models: We an etend our proposed approah to e appled to other types of fuzzy models suh as Mamdan-type fuzzy models. 3. Real-world prolems: We epet that the proposed approah should e onsdered further n respet to a wder range of real-world prolems. 4. Genet Algorthms: The use of Genet Algorthms GA nstead of apropagaton learnng algorthm. GA does not suffer from onvergene prolems wth the same degree that the BP suffers. To verfy the effetveness of the new approah the approah was appled on a wellnown enhmar prolem. Referenes [] L. A. Zadeh Fuzzy sets Informaton Control Vol. 8 PP [2] R. Andrews A.B. Tle and J. Dederh A Survey and Crtque of Tehnques for Etratng Rules from Traned Artfal Neural Networs Knowledge Based Systems Vol. 8 PP [3] S. Mtra and Y. Hayash Neuro-Fuzzy Rule Generaton: Survey n Soft Computng Framewor IEEE Trans. Neural Networs Vol. No. 3 May [4] J. -S.R. Jang and C. -T. Sun Funtonal equvalene etween radal ass funton networs and fuzzy nferene systems IEEE Trans. on Neural Networs Vol. 4 PP

19 [5] Y. Ln G. A. Cunnngham III and S. V. Coggeshall Usng fuzzy parttons to reate fuzzy systems from nput-output data and set the ntal weghts n a fuzzy neural networ IEEE Trans. Fuzzy Syst. Vol. 5 PP August 997. [6] W. A. Farag V. H. Quntana and G. Lamert-Torres A genet-ased neuro-fuzzy approah for modelng and ontrol of dynamal systems IEEE Trans. Neural Networs Vol.9 PP Ot [7] V.Tresp J. Hollatz and S. Ahmed Networ Struturng and Tranng Usng Rule- Based Knowledge Advaned n Neural Informaton Proessng Systems NIPS*6 PP [8] M. Berthold and K. Huer Buldng Prese Classfers wth Automat Rule Etraton n Pro. of the IEEE Int. Conf. On Neural Networs Perth Australa Vol. 3 PP [9] S. Ae and M.S. Lan A Method for Fuzzy Rules Etraton Dretly from Numeral Data and ts Applaton to Pattern Classfaton IEEE Trans. on Fuzzy Systems Vol. 3 No. PP [0] W. Duh R.Adamza and K. Grazws Neural Optmzaton of Lngust Varales and Memershp Funtons Pro. of the 6 th Int. Conf. On Neural Informaton Proessng ICONIP 99 Perth Australa Vol. 2 PP [] A. Lapedes and R. Faer How Neural Networs Wor Neural Informaton Proessng Systems Anderson D.Z.ed Ameran Insttute of Physs New Yor PP [2] S. Geva and J. Sttle Constraned Gradent Desent Pro.of the 5 th Australan Conferene on Neural Computng Brsane Australa 994. [3] R. Andrews and S. Geva On the Effets of Intalzng a Neural Networ wth Pror Knowledge Pro. of the Internatonal Conferene on Neural Informaton Proessng Perth Western Australa PP [4] R. Andrews and S. Geva RULEX & CEBP Networs As the Bass for a Rule Refnement System n Hyrd Prolems Hyrd Solutons Hallam J. Ed IOS Press PP [5] S. J. Lee and C. S. Ouyang A Neuro-Fuzzy System Modelng wth Self- Construtng Rule Generaton and Hyrd SVD Based Learnng IEEE Trans. on Fuzzy Systems Vol. PP June [6] J. C. Bezde Pattern Reognton wth Fuzzy Oetve Funton Algorthms Plenum Press N. Y. 98. [7] C. J. Mertz and P. M. Murphy UCI repostory of mahne learnng dataases [Onlne]. Avalale: [8] R. Andrews and S. Geva Rule Etraton From Loal Cluster Neural Nets Sumtted to Neuroomputng Feruary [9] I. Taha and J. Ghosh Symol Interpretaton of Artfal Neural Networs IEEE Trans. Knowledge And Data Engneerng Vol. No. 3 PP May 999. [20] B. Ster and A. Donar Neural networs n medal dagnoss: Comparson wth other methods Pro. of Int. Conf. E`ANN 96 A. Bulsar Ed. PP