Research Article Special Issue

Transcription

1 ournl of Fundmentl nd Appled Scences ISSN Reserch Artcle Specl Issue Avlble onlne t A PERFORMANCE EVALUATION OF PRUNING EFFECTS ON HYBRID NEURAL NETWORK S. Y. Leow* 1, K. S. Yp 1, H.. Yp 2, nd S. Y. Wong 1 1 College of Grdute Studes, Unverst Teng Nsonl, Kng, Selngor, Mlys 2 Deprtment of Mechncl Engneerng, Fculty of Engneerng, Unversty of Mly, Kul Lumpur, Mlys Publshed onlne: 05 October 2017 ABSTRACT In ths pper, we explore the prunng effects on hybrd mode sequentl lernng lgorthm nmely FuzzyARTMAP-prunble Rdl Bss Functon (FAM -PRBF) tht utlzes Fuzzy ARTMAP to lern trnng dtset nd Rdl Bss Functon Network (RBFN) to perform regresson nd clssfcton. The prunng lgorthm s used to optmze the hdden lyer of the RBFN. The expermentl results show tht FAM-PRBF hs successfully reduced the complexty nd computton tme of the neurl network. Keywords: prunng; rdl bss functon network; fuzzy ARTMAP. Author Correspondence, e-ml: syleow88@hotml.com do: /fs.v94s INTRODUCTION An rtfcl neurl network (ANN) s generlly constructed by three lyers. They re nput lyer, hdden neuron lyer nd output lyer. The nputs re collected nd fed nto the network through nput lyer. The second lyer s the hdden neuron lyer. Ths hdden neuron lyers my hve one or severl hdden lyers, whch depends on the structure of the neurl network. The output s commonly decded by n overll functon of the ANN. The trnng dt tht fed to the network, trnng pproch nd weghts of the neurons decde the overll functon. Recently, hybrd models of ANNs hve been proposed. Generlly, there re three dfferent methods n the hybrdzton of ANNs. They re model, lgorthms nd dt [1]. ournl of Fundmentl nd Appled Scences s lcensed under Cretve Commons Attrbuton-NonCommercl 4.0 Interntonl Lcense. Lbrres Resource Drectory. We re lsted under Reserch Assoctons ctegory.

2 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), Rdl Bss Functon Network RBFN s type of feedforwrd neurl network tht grounded on functon pproxmton theory. Ths network serches the most outstndng mtch n multdmensonl spce wthn the knowledge tht t lerned from trnng dt. RBF hs good generlzton, onlne lernng cpblty nd tolernt to nosy nputs [2]. These dvntges mke the RBF network wdely ppled n flexble control systems, dynmc systems nd tme-seres predcton [3-7]. A smple RBFN conssts of three lyers. They re nput lyer, hdden neuron lyer wth Q hdden neurons nd output lyer. The connecton between the nput lyer nd hdden neuron lyer s non-lner. However, the connecton between the hdden neuron lyer nd output lyer s lner. The hdden neurons mplement rdl bss functons. The y s the lner output of the RBFN, whch s defned s follows: y f ( X) β 1 G ( X) Q (1) The X s nput vector, Q s the number of hdden neurons, β s the output weghts nd G (x) s rdl bss functon of the hdden neurons. There re vrous forms of bss functons tht cn be ppled to RBFN. In ths pper, Gussn functon s selected. Gussn functon s commonly used. Ths functon s locl nd only responses to the nput tht ner the center. The bsolute vlue of ths functon decreses contnuously pproches zero n ll dmensons when the nputs re wy from ts center. The G (X) s defned n Equton (2). G ( X) X c exp The c s the center of the Gussn functon. The spred, σ controls the wdth of the (2) Gussn functon. The 1 β cn be found by usng β G y. The G my not be squre mtrx. If the G s not squre mtrx, β cn be found by usng the pseudonverse of G, G T ( G) 1 G T y (3) 1.2. Fuzzy ARTMAP Adptve Resonnce Theory (ART) hs been developed by Crpenter nd Grossberg n 1987 to overcome stblty-plstcty dlemm. Fuzzy ARTMAP (FAM) s supervsed lernng member of the ART fmly. FAM embeds fuzzy [21] logc opertons to ther neurons bsed on ART lgorthm. The FAM s n ncrementl lernng system wth onlne lernng

3 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), cpblty. FAM cn preserve nformton lerned from prevous nformton (stble), whle beng flexble enough to lern new nformton ncrementlly (plstc). FAM does not forget the prevous lerned nformton,.e., t does not hve problem wth ctstrophc forgettng [8]. FAM hs pr of ART modules nd mp feld. They re ART- nd ART-b. Both ART modules hve three neuron lyers,.e. normlzton lyer, nput lyer nd recognton lyer respectvely. Trnng dt tht hve nputs nd trget outputs re mndtory to be fed to FAM. Assume {(X, T ), (X, T ),..., (X, T )} denotes nputs nd ts trget outputs. The th trnng smples, X R nd T R, re nput vector wth M-dmensons nd trget output vector wth L-dmensons. F nd F re normlzton lyers. In these lyers, nput vectors nd ts trget output vectors re ntroduced to complement codng,.e., A B ( X, X ) ( X,1 X c ( T, T ) ( T,1 T ) c ) where A R s the complemented nputs vector, nd B R s the complemented trget outputs vector. F nd F re nput lyers. These nput lyers connected n between normlzton lyers nd recognton lyers respectvely. The vector tht obtned from the A nd B re presented to recognton lyers v nput lyers. The F nd F re recognton lyers. These two lyers tke complemented nput nd trget output vectors seprtely. Every neuron encodes n nput nd ts trget outputs n these two lyers. In the trnng phse, the number of hdden neurons cn be ncresed. Ech neuron hs ts specfc dptve weghts set n vector form. The weghts wth Q hdden neurons for both ART modules re w nd w. The weghts re mgned s hyper-rectngles. Menwhle, the centers of both ART modules re c nd c. When fresh hdden neuron s creted (.e., Q Q +1), w nd c n both ART modules re fxed to be 1 nd 0 respectvely. The q, number of trnng smple whch s ssgned to neuron n F lyer, s set to be 0. Durng the trnng phse, normlzed nput vector s presented to ART-. Menwhle, ts trget output vector s presented to ART-b. An ctvton number s creted to ech neuron n by the choce functon mesurement. The hdden neuron whch hs the top ctvton number, s selected s the wnnng neuron. The wnnng neuron must ccomplsh wnner-tke-ll competton rule. (4)

4 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), T A w w (5) The α s choce prmeter nd W s weght vector of neuron n F. The wnnng neuron s chrctersed s neuron. After tht, vglnce test s performed. The wnnng neuron, W, nd A, re mtched the vglnce prmeter, ρ, A w A (6) where ρ s [0,1]. If the wnnng neuron fls vglnce test, then serchng cycle of new wnnng neuron s performed. Ths serch cycle wll be performed untl the wnnng neuron fulfls the vglnce test. The smlr procedure s presented n ART-b. Once dentfy the wnnng neuron n the reorgnzton lyer of ART-, vglnce test n ART-b s performed to ts trget output neuron n b F 2 (.e., b w ). Ths test s requred to verfy the neuron s the concludng wnner. B w B b b (7) The b s number between 0 nd 1. If fls the vglnce test, trckng mtch s presented s follows: A w A (8) In F, to prevent the sme neuron to be chosen gn, very smll postve constnt number, s dded. In ART-, new serch cycle strts wth nother threshold level of. If there s no neuron cn fulfl Equton (7), fresh neuron s creted. When the wnner neuron s exmned, the centers nd weghts of wnnng neuron n recognton lyers re updted s: w A w, w b B w b (9) q q 1 (10) c c b 1 A 1 c q (11) q 1 1 c q b B q

5 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), Prunng Algorthm Prunng lgorthm s not new to ANNs. Prunng lgorthm hs been proposed to mprove generlzton of n ANN. It cn escpe from overfttng problems n huge system [9]. A huge ANN crres unnecessry constrnts. The network nvolves longer predcton responses, nessentl knowledge storge nd hgh-prced hrdwre mplementton. The lrge ANN my overft the trnng dt nd produce poor generlzton performnce. There re mny types of prunng lgorthms nd cn be dvded nto two groups [9]. One group prunes the hdden neurons bsed on the senstvty of the error functon. Another group s dd term to the prunng obectve, for exmple, the hdden neurons wth smll weghts cn be pruned. Prunng lgorthm cn be ppled to n ANN durng trnng procedure or fter ech trnng epoch. A resource lloctng network (RAN) whch ws frstly ntroduced by Pltt n 1991 s used to llocte hdden neurons nd prune nsgnfcnt hdden neurons [10]. RAN ws further enhnced. Mnml resource lloctng network (MRAN) cn grow nd prune neurons. Prunng strtegy cn be ppled to n ANN durng trnng procedure or fter ech trnng epoch. In rdl bss functon networks wth dynmc decy dustment ( RBFN-DDA) lgorthm, the prunng strtegy s ppled to the network fter ech trnng epoch [11]. Besdes, n [12-13] ntroduced growng nd prunng RBF (GAP -RBF) nd generlzed growng nd prunng RBF (GGAP -RBF). A smlr lgorthm, FAMDDA wth temporry neurons (FAMDDA-T) tht conssts dynmc decy dustment [23] nd prunng strtegy s developed by [14]. Recently, n [15] hs proposed self-orgnzng recurrent rdl bss functon (SR-RBF) neurl network tht desgned bsed on spkng mechnsm nd mproved Levenberg-Mrqurdt (LM) lgorthm. The hdden neuron lyer n ths lgorthm cn dd or prune hdden neurons by clcultng the spkng strength between hdden nd output neurons n recurrent rdl bss functon neurl network. 2. METHODOLOGY Ths pper proposes n lgorthm tht tes the prunng of nsgnfcnt hdden neurons on the relevnce of the hdden neurons n hybrd neurl network. Ths FAM-PRBF lgorthm s usng sortng methods to mesure the relevnce of ech hdden node n the proposed lgorthm. A smple prunng technque s ppled to remove the nsgnfcnt hdden neurons n RBFN. Fg. 1 llustrtes the rchtecture of the new lgorthm. A remrkble rrngement of the proposed lgorthm the center of RBFN hdden neurons re determned by the hdden

6 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), neurons n FAM. The center of hdden neurons n RBFN re selected rndomly from the trnng dtset. A dtset s requred to be dvded nto trnng dtset nd testng dtset. The trnng dtset s presented to FAM. The tsk of FAM n ths hybrd neurl network s to lern from the trnng dtset. Durng the trnng process, the FAM hdden neurons grow s requested by the trnng dtset. When the trnng process completed, the center weghts of FAM, c re employed to compute the hdden neurons n RBFN. The c s substtuted s center of Gussn functon n Equton (2) to compute the ctvton functon of RBFN. The spred, σ s tuneble prmeter tht hs to be determned before trnng phse. Then, the output weght, β re computed. After computng the RBFN, testng dtset s ppled to the network to test the ccurcy of ths hybrd neurl network [22]. Next, the prunng lgorthm s mplemented n the RBF hdden lyer bsed on the relevnce of the hdden neurons. The prunng threshold, θ whch decdes the nsgnfcnt hdden neurons, s defned before the prunng lgorthm s executed. The number of nsgnfcnt hdden neurons to be pruned whch s defned n percentge form hs to be converted to decml form wthn the rnge 0.1, then multpled wth the totl number of hdden neurons, Q, to set the θ. The prunng process of the proposed lgorthm s descrbed s follow: Step 1: Re-llocte the bsolute vlue of output weght, β tht re rrnged frstly n RBFN of FAM-PRBF network n descendng order. The hdden neurons re re-llocted together wth the β. Step 2: If the qth hdden neurons s greter thnθ, t s output weght, β wll be remn nd store. The qth hdden neuron s defned s nsgnfcnt hdden neuron, f t s re-llocted lower thn θ. The β of the nsgnfcnt neurons wll be set to 0. The connectons tht lnked to the nsgnfcnt hdden neurons wll be trmmed., q θ 0, q θ (12)

7 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), Fg.1. Archtecture of FAM-PRBF network Step 3: Re-orgnze the β nd re-dust usng Equton (1), (2) nd (3). Then, updte the RBFN of FAM-PRBF. After prunng process, the trnng nd testng dtsets re re-ppled to the RBFN of FAM-PRBF. The trnng nd testng performnces of the RBFN re re-mesured. 3. RESULTS AND DISCUSSION In ths study, the performnce of the prunng effects on FAM-PRBF s evluted on four benchmrkng problems (two clssfcton dtset nd two regresson dtset) from UCI mchne-lernng repostory [16]. The two mult-clss clssfcton problems tht hve been consdered re Imge Segment nd Stellte Imge. However, the two regresson problems tht hve been studed re Ablone nd MPG. All the experments were performed n MATLAB 7.11 runnng on Core 5, 2.6GHz CPU wth 4G RAM surroundngs. The specfcton of the experments re presented n Tble 1. Ech dtset ws dstrbuted nto trnng nd testng dt s presented n Tble 1. All the outputs re normlzed nto the rnge [0, 1] n the experments. The nputs re normlzed nto the rnge [-1, 1] n clssfcton problems. The nputs re normlzed nto the rnge [0, 1] n regresson problems. The spred of the Gussn functon n RBFN, σ nd vglnce prmeters, ρ nd ρ of FAM hd to be determned before conductng the experments. The ρ s set to 1 nd ρ hs to be tuned the n clssfcton experments. However, ρ s set to be 0 nd ρ hs to be tuned n regresson experments.

8 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), Tble 1. Specfcton of benchmrk dtsets for experments Dtset Attrbutes Clss Trnng Dt Testng Dt Ablone 8 n/ 3,000 1,177 MPG 7 n/ Imge Segment Stellte Imge The obectve of Ablone experment s to predct the ge of blone bsed on ther physcl dmensons. The ntenton of MPG experment s to predct the fuel consumpton of numerous knds of crs n term of mles per gllon. Besdes, the mn obectve of mge segmentton experment s to clssfy ech regon nto one of the seven outdoor mges through 19 ttrbutes. The seven outdoor mges re brck fcng, sky, folge, cement, wndow, pthwy nd grss. Ech mge s 3X3 regon. The mn gol of stellte mge experment s to ctegorze ech regon nto one of the sx ctegores nmely very dump grey sol, dmp grey sol, cotton crop, sol wth vegetton stubble, red sol nd grey sol by usng 36 ttrbutes. The verge trnng nd testng ccurcy re collected bsed on 50 trls. These ccurces re ndcted n root men squre (RMS) error for regresson problems nd n ccurcy rtes (%) for clssfcton problems Comprsons on Performnce Evlutons The performnce of FAM-PRBF (before prunng) on regresson problems s reported n Tble 2. Ths performnce s compred wth other lernng lgorthms. The performnces of the compred lernng lgorthms n both regresson nd clssfcton experments re strght denoted from the lterture. Tble 2. Results of the regresson experments before prunng Dtset Algorthms RMSE Number of Trnng Testng Hdden Neurons FAM-RBF Ablone GAP-RBF OS-ELM (RBF) [17] MRAN [17] FAM-RBF MPG GAP-RBF OS-ELM (RBF) [17] MRAN [17]

9 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), The number of hdden neurons n FAM-PRBF (before prunng) for Ablone experment s smlr wth MRAN. However, the trnng nd testng RMS errors of FAM-PRBF (before prunng), nd n Ablone experment re sgnfcntly better thn the trnng nd testng RMS errors of MRAN ( nd ). In ths experment, the trnng nd testng RMSE vlues of the FAM-PRBF (before prunng) re notceble better thn other lgorthms. Before the prunng lgorthm n FAM-PRBF s performed n MPG experment, the number of hdden neurons, 64 s slghtly lrger thn the number of hdden neurons n the compred lgorthms. The trnng nd testng RMSE vlues of FAM-PRBF (before prunng) re better thn the compred lernng lgorthms. The FAM-PRBF (before prunng) hs comprble trnng nd testng RMS errors ( nd ) wth the trnng nd testng RMS errors of OS-ELM (RBF) ( nd ). Fg. 2 summrzes the trnng nd testng RMS errors on dfferent numbers of nsgnfcnt hdden neurons re pruned (%) for Ablone nd MPG experments correspondngly. Wth regrds to the Ablone experment, both RMS errors nd number of hdden neurons (fter prunng) re close to the trnng nd testng RMS error of OS-ELM ( nd ) nd hdden neurons, 25. In comprson wth GAP-RBF tht hs hdden neurons, GAP-RBF obtns hgher trnng nd testng RMS errors t nd However, the trnng RMS error of the FAM-PRBF (fter prunng) n MPG experment ncreses wth number of nsgnfcnt hdden neurons re pruned. When 10% to 70% nsgnfcnt hdden neurons re pruned, the testng RMS error stedly ncreses. When there re 70% of nsgnfcnt hdden neurons re pruned, the trnng nd testng RMS error re nd , nd the number of hdden neuron n the new RBFN s Ths result s comprble wth the OSELM (RBF) network whch hs 25 hdden neurons, obtns trnng nd testng RMS error t nd As observed from Tble 3, FAM-PRBF (before prunng) obtns comprble performnce results n both clssfcton experments. The trnng nd testng performnces of FAM-PRBF (before prunng) n both experments re better thn the compred lgorthms. However, the number of hdden neurons of FAM-PRBF (before prunng) n both experments re vsbly lrger thn the compred lernng lgorthms. The trnng nd testng ccurces on dfferent numbers of nsgnfcnt hdden neurons re pruned (%) for Imge Segmentton nd Stellte Imge experments resp ectvely re tbulted n Fg. 3. The testng ccurcy of the FAM-PRBF decreses stedly n Stellte Imge experments when 10% to 60% of nsgnfcnt hdden neurons re pruned. When 60%

10 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), of the nsgnfcnt hdden neurons re removed n Stellte Imge experment (453.2 hdden neurons n FAM-PRBF), the trnng nd testng ccurces re better thn the trnng nd testng ccurces of OS-ELM (RBF). Bsed on the performnce evlutons n Fg. 2 nd 3, t cn be notced tht there s shrp fllng on the testng ccurcy or rsng on the testng RMS error when there re bout 80% of nsgnfcnt hdden neurons re pruned. Further prunng (prune more thn 80% of nsgnfcnt hdden neurons) worsens trnng nd testng performnces of the network. Ths s nturl stoppng pont of prunng strtegy n ths lgorthm. The prunng strtegy hs successfully remove the nsgnfcnt hdden neurons n ths proposed lgorthm. Fg.2. Trnng nd testng RMS errors for () Ablone dtset (b) MPG dtset

11 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), Tble 3. Results of the clssfcton experments before prunng Dtset Algorthms Accurcy (%) Number of Hdden Neurons Trnng Testng (Before Prunng) FAM-RBF FAM-OELM [18] Imge OS-ELM (RBF) [17] Segmentton FAM [19] n/ MRAN [20] n/ FAM-RBF FAM-OELM [18] Stellte Imge OS-ELM(RBF) [17] FAM [19] n/ MRAN [20] n/ We cn conclude tht the sgnfcnt hdden neurons ply mportnt prtcptons to ths lgorthm. The prtcpton s defned by the bsolute vlue of output weght, β n RBFN. The output weght tht s nerly zero hs less nfluence to the RBFN. Therefore, t cn be pruned. The trnng error s expected to be hgh n regresson experments nd the trnng ccurcy s expected to be felled n clssfcton experments when prunng lgorthm s ppled to FAM-PRBF network. Durng the trnng phse (before prunng), trnng dt hs been used to estmte the number of hdden neurons n ths proposed lgorthm. The hdden neurons re creted s requred by the trnng dt. When prunng lgorthm s ppled, the trnng error rses becuse the FAM-PRBF network becomes underft fter prunng.

12 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), Fg.3. Trnng nd testng ccurces for () Imge Segmentton dtset (b) Stellte Imge dtset Some of the nsgnfcnt hdden neurons whch re relted nd mportnt to trnng dtset were removed n the prunng phse. In both regresson experments, the best prunng results occurs when there re 30% of nsgnfcnt hdden neurons s pruned. The testng errors n both experments t ths prunng threshold were mproved. When 20% to 30% of nsgnfcnt hdden neurons re removed, the testng error n Ablone nd MPG experments ws felled from to nd to respectvely. The testng ccurcy n Imge Segmentton experment s slghtly mproved from 95.97% to 96.11% when 10% to 30% nsgnfcnt hdden neurons

13 S. Y. Leow et l. Fundm Appl Sc. 2017, 9(4S), re pruned. Ths prunng threshold hs solved the overfttng problems nd mproved the testng performnces n these experments. The repettve hdden neurons tht detrct the testng performnce of the network hs been recognzed nd removed. Therefore, n the prevous study, [5], the prunng threshold ws set t 30% for the regresson experments Comprsons on Computton Tme Fg. 4 reports the computton tme reduced of RBFN (fter prunng) n these four experments. The del computton tme reduced tht s expected to be lnerly proportonl to the number of pruned nsgnfcnt hdden neurons s represented n dshed lne n these four experments. However, the sold lne denotes the rel-world computton tme of the new RBFN n the proposed lgorthm.

14 H.. Yp et l. Fundm Appl Sc. 2017, 9(4S), Fg.4. Computton tme reduced (%) versus number of nsgnfcnt hdden neurons pruned (%) for: () Ablone dtset, (b) MPG dtset, (c) Imge segmentton dtset nd (d) Stellte mge dtset The computton tme of RBFN reduced s clculted bsed on percentge chnge formul below: before fter C. T. reduced (%) 100 % (13) before The C.T. Reduced ndctes the computton tme reduced n percentge form, before ndctes the computton tme before prunng n seconds, nd fter ndctes the computton tme fter prunng n seconds. These four grphs hve successfully reported the reltonshp between the numbers of pruned nsgnfcnt hdden neurons nd the computton tme of the RBFN (fter prunng) s lnerly proportonl.

15 H.. Yp et l. Fundm Appl Sc. 2017, 9(4S), CONCLUSION In concluson, the prunng lgorthm hs been successfully proposed n FAM-PRBF. The prunng lgorthm hs reduced the sze of hdden neurons lyer n the RBFN of FAM-PRBF network nd computton tme of FAM-PRBF network. The defnton of prunng lgorthm n ths pper s to remove the nsgnfcnt hdden neurons tht hs output weghts nerly zero nd less nfluence to the network. The performnces of FAM-PRBF on regresson nd clssfcton benchmrkng problems re compred wth other well-known lernng lgorthms. Usng the correct prunng threshold, FAM-PRBF cn return substntl or better trnng nd testng ccurces. The prunng lgorthm n FAM-PRBF s n exctng topc for further reserch. 5. REFERENCES [1] Gutérrez P A, Hervás-Mrtínez C. Hybrd rtfcl neurl networks: models, lgorthms nd dt. In Interntonl Work-Conference on Artfcl Neurl Networks, 2011, pp [2] Poggo T, Gros F. Networks for pproxmton nd lernng. Proceedngs of the IEEE, 1990, 78(9): [3] Lu Y, Zhng T, Zeng Z, Loo. An mproved RBF neurl network for short-term lod forecst n smrt grds. In IEEE Interntonl Conference on Communcton Systems, 2016, pp. 1-6 [4] n L C, Seer M, Lm C P, Blsubrmnm P. A revew of onlne lernng n supervsed neurl networks. Neurl Computng nd Applctons, 2014, 25(3-4): [5] Leow S Y, Wong S Y, Yp K S, Yp H. A new hybrd fuzzy ARTMAP nd rdl bss functon neurl network wth onlne prunng strtegy. In 7th IEEE Control nd System Grdute Reserch Colloquum, 2016, pp [6] Bghee H R, Mrslm M, Ghrehpetn G B, Tleb H A. Nonlner lod shrng nd voltge compenston of mcrogrds bsed on hrmonc power-flow clcultons usng rdl bss functon neurl networks. IEEE Systems ournl, 2017, 99:1-11 [7] Dong F, Le X, Chou W. A dynmc model nd control method for two-xs nertlly stblzed pltform. IEEE Trnsctons on Industrl Electroncs, 2017, 64(1): [8] Crpenter G A, Grossberg S, Mrkuzon N, Reynolds H, Rosen D B. Fuzzy ARTMAP: A

16 H.. Yp et l. Fundm Appl Sc. 2017, 9(4S), neurl network rchtecture for ncrementl supervsed lernng of nlog multdmensonl mps. IEEE Trnsctons on Neurl Networks, 1992, 3(5): [9] Reed R. Prunng lgorthms-a survey. IEEE Trnsctons on Neurl Networks, 1993, 4(5): [10] Pltt. A resource-lloctng network for functon nterpolton. Neurl Computton, 1991, 3(2): [11] Petz. Reducng the number of neurons n rdl bss functon networks wth dynmc decy dustment. Neurocomputng, 2004, 62:79-91 [12] Hung G B, Srtchndrn P, Sundrrn N. An effcent sequentl lernng lgorthm for growng nd prunng RBF (GAP -RBF) networks. IEEE Trnsctons on Systems, Mn, nd Cybernetcs, Prt B (Cybernetcs), 2004, 34(6): [13] Hung G B, Srtchndrn P, Sundrrn N. A generlzed growng nd prunng RBF (GGAP-RBF) neurl network for functon pproxmton. IEEE Trnsctons on Neurl Networks, 2005, 16(1):57-67 [14] Tn S C, Ro M V, Lm C P. On the reducton of complexty n the rchtecture of fuzzy ARTMAP wth dynmc decy dustment. Neurocomputng, 2006, 69(16): [15] Lu C, Hn H, Qo, Yng C. Desgn of self-orgnzng recurrent RBF neurl network bsed on spkng mechnsm. In 35th IEEE Chnese Control Conference, 2016, pp [16] Blke C, Merz C. UCI repostory of mchne lernng dtbses. Irvne: Unversty of Clforn, 1998 [17] Lng N Y, Hung G B, Srtchndrn P, Sundrrn N. A fst nd ccurte onlne sequentl lernng lgorthm for feedforwrd networks. IEEE Trnsctons on Neurl Networks, 2006, 17(6): [18] Wong S Y, Yp K S, Yp H, Tn S C. A truly onlne lernng lgorthm usng hybrd fuzzy ARTMAP nd onlne extreme lernng mchne for pttern clssfcton. Neurl Processng Letters, 2015, 42(3): [19] Tn S C, Ro M V, Lm C P. An onlne prunng strtegy for supervsed ARTMAP-bsed neurl networks. Neurl Computng nd Applctons, 2009, 18(4): [20] L G, Lu M, Dong M. A new onlne lernng lgorthm for structure-dustble extreme

17 H.. Yp et l. Fundm Appl Sc. 2017, 9(4S), lernng mchne. Computers nd Mthemtcs wth Applctons, 2010, 60(3): [21] Ty K M, Lm C P. On the use of fuzzy nference technques n ssessment models: Prt II: Industrl pplctons. Fuzzy Optmzton nd Decson Mkng, 2008, 7(3): [22] Yp K S, Lm C P, Abdn I Z. A hybrd ART-GRNN onlne lernng neurl network wth ε-nsenstve loss functon. IEEE Trnsctons on Neurl Networks, 2008, 19(9): [23] Petz. Reducng the number of neurons n rdl bss functon networks wth dynmc decy dustment. Neurocomputng, 2004, 62:79-91 How to cte ths rtcle: Leow SY, Yp KS, Yp H, Wong SY. A performnce evluton of prunng effects on hybrd neurl network.. Fundm. Appl. Sc., 2017, 9(4S),