Trade-offs in Optimization of GMDH-Type Neural Networks for Modelling of A Complex Process

Transcription

1 Proceedngs of the 6th WSEAS Int. Conf. on Systems Theory & Scentfc Computton, Elound, Greece, August -3, 006 (pp48-5) Trde-offs n Optmzton of GDH-Type Neurl Networs for odellng of A Complex Process N. Nrmn-zdeh, E. Hghgoo, A. Jml Fculty of echncl Engneerng Islmc Azd Unversty, Testn Brnch, IRAN Abstrct:- Evolutonry Algorthms (EAs) re deployed for mult-obectve Preto optml desgn of Group ethod of Dt Hndlng (GDH)-type neurl networs tht hve been used for modellng of complex process (such s explosve cuttng process) usng some nput-output expermentl dt. In ths wy, EAs wth new encodng scheme s frstly presented to evolutonry desgn of the generlzed GDH-type neurl networs n whch the connectvty confgurtons n such networs re not lmted to dcent lyers. ult-obectve EAs (non domnted sortng genetc lgorthm, NSGA-II) wth new dversty preservng mechnsm re secondly used for Preto optmzton of such GDH-type neurl networs. Optml Preto fronts re obtned whch exhbt the trde-off between pr of conflctng obectves nd, thus, provde dfferent non-domnted optml choces of GDH-type neurl networs models for such complex process. Key-Words:- Preto optmzton, GAs, GDH, odellng Introducton System dentfcton technques re ppled n mny felds n order to model nd predct the behvours of unnown nd/or very complex systems bsed on gven nput-output dt []. Theoretclly, n order to model system, t s requred to understnd the explct mthemtcl nput-output reltonshp precsely. Such explct mthemtcl modellng s, however, very dffcult nd s not redly trctble n poorly understood systems. GDH lgorthm s self-orgnzng pproch by whch grdully complcted models re generted bsed on the evluton of ther performnces on set of mult-nput-sngle-output dt prs ( X, y ) (=,,, ) []. The mn de of GDH s to buld n nlytcl functon n feedforwrd networ bsed on qudrtc node trnsfer functon [] whose coeffcents re obtned usng regresson technque. Recently, genetc lgorthms hve been used n feedforwrd GDH-type neurl networ for ech neuron serchng ts optml set of connecton wth the precedng lyer [3-4]. In the [3], uthors hve proposed hybrd use of genetc lgorthm for smplfed structure GDH-type neurl networ n whch the connectons of neurons re restrcted to dcent lyers. Such restrcton hs been removed by recent wors of some of uthors n [5] led to generlzed-structure GDH-type neurl networs (GS-GDH) whch exhbted better performnce n terms of both modellng errors nd networ's complexty n comprsons wth those of other desgn methods [6]. All these methods devsed prevously hve been bsed on sngle obectve optmzton process n whch ether trnng error or predcton error selected to be mnmzed wth no control of other obectves. In order to obtn more robust models of such complex hgh rte energy process, t s requred to consder ll the conflctng obectves, nmely, trnng error (TE), predcton error (PE) nd number of neuron (N) (representng the complexty of the models) be mnmzed smultneously n the sense of mult-obectve Preto optmzton process. In ult-obectve optmzton problems (OPs), there re severl obectve or cost functons ( vector of obectves) to be optmzed (mnmzed or mxmzed) smultneously. These obectves often conflct wth ech other so tht mprovng one of them wll deterorte nother. Therefore, there s no sngle optml soluton s the best wth respect to ll the obectve functons. Insted, there s set of optml solutons, nown s Preto optml solutons or Preto front [7-9] for mult-obectve optmzton problems. The concept of Preto front (n the domn of obectves) or set (n the domn of desgn vrbles) of optml solutons n the spce of obectve functons n OPs stnds for set of solutons tht re non-domnted to ech other but re superor to the rest of solutons n the serch spce. Ths mens tht t s not possble to fnd sngle soluton to be superor to ll other solutons wth respect to ll obectves so tht chngng the vector of desgn vrbles n such Preto set consstng of these non-domnted solutons could not led to the mprovement of ll obectves smultneously. Consequently, such chnge wll led to deterortng of t lest one obectve. Thus,

2 Proceedngs of the 6th WSEAS Int. Conf. on Systems Theory & Scentfc Computton, Elound, Greece, August -3, 006 (pp48-5) ech soluton of the Preto front ncludes t lest one obectve nferor to tht of nother soluton n tht Preto front, lthough both re superor to others n the rest of serch spce. n n n y = 0 + x + x x + = = = n n n x x x +... = = = (4) In ths pper, EAs wth new encodng scheme re used to evolutonry desgn the generlzed structure GDH-type (GS-GDH) neurl networs n whch the connectvty confgurton n such networs s not lmted to dcent lyers for modellng nd predcton of the process. In ths wy, multobectve EAs (non domnted sortng genetc lgorthm, NSGA-II) wth new dversty preservng mechnsm re ppled for Preto optmzton of such GS-GDH-type neurl networs. The mportnt conflctng obectves of the GS-GDH neurl networs tht re consdered n ths wor re, nmely, trnng error (TE), predcton error (PE) nd number of neurons (N). Dfferent prs of these obectve functons (N-TE) nd (N-PE) re selected for -obectve Preto optmzton of GS-GDH neurl networs models. odellng Usng GDH Neurl Networs The forml defnton of the dentfcton problem s to fnd functon fˆ so tht cn be pproxmtely used nsted of ctul one, f n order to predct output ŷ for gven nput vector X = ( x,... ) s close s possble to ts 3 n ctul output y. Therefore, gven observton of mult-nput-sngle-output dt prs so tht y = f ( x,... ) (=,,,), () 3 n It s now possble to trn GDH-type neurl networ to predct the output vlues ŷ for ny gven nput vector X = ( x,... ), tht s 3 n y ˆ = fˆ( x,... ) (=,,,). () 3 n The problem s now to determne GDH-type neurl networ so tht the squre of dfference between the ctul output nd the predcted one s mnmsed, tht s = [ f ˆ ( x 3,... n ) y ] mn. (3) Generl connecton between nputs nd output vrbles cn be expressed by complcted dscrete form of the Volterr functonl seres n the form of whch s nown s the Kolmogorov-Gbor polynoml []. Ths full form of mthemtcl descrpton cn be represented by system of prtl qudrtc polynomls consstng of only two vrbles (neurons) n the form of yˆ = G ( x ) = x 3 x + 4 x x + 5 x x + (5) The coeffcent n equton (5) re clculted usng regresson technques [] so tht the dfference between ctul output, y, nd the clculted one, ŷ, for ech pr of x s nput vrbles s mnmzed. In ths wy, the coeffcents of ech qudrtc functon G re obtned to optmlly ft the output n the whole set of nputoutput dt pr, tht s ( ()) = y G E = mn. (6) The pplcton of lest-squres methods nd SVD technques for mult-regresson nlyss to fnd the coeffcent emboded n equton (5) n order to mnmze equton (6) hs been thoroughly gven n [5]. 3 Applcton of GAs n the Desgn of GDH-Type Neurl Networs In the Generl Structurl GDH (GS-GDH) neurl networs, neurons connectons cn occur between dfferent lyers whch re not necessrly very dcent ones, unle the CS-GDH neurl networs n whch such connectons only occur between dcent lyers. Usng the sme procedure of defnng chromosome descrbed n the [4], t cn now be redly modfed to nclude GS-GDH networs. Ths s ccomplshed by repetng the nme of the neuron whch drectly pssng the next lyersin fgure (), neuron d n the frst hdden lyer s connected to the output lyer by drectly gong through the second hdden lyer. Therefore, t s now very esy to notce tht the nme of output neuron (networ s output) ncludes d twce s bbcdd. In other words, vrtul neuron nmed dd hs been constructed n the second hdden lyer nd used wth bbc n the sme lyer to me the output neuron bbcdd s shown n the fgure (). It should be noted tht such repetton occurs

3 Proceedngs of the 6th WSEAS Int. Conf. on Systems Theory & Scentfc Computton, Elound, Greece, August -3, 006 (pp48-5) whenever neuron psses some dcent hdden lyers nd connects to nother neuron n the next nd, or 3 rd,or 4 th,or followng hdden lyer. In ths encodng scheme, the number of repetton of tht neuron depends on the number of pssed hdden ñ lyers, ñ, nd s clculted s. only f F(X ) p F(X). Alterntvely, t cn be redly restted s {,,..., }, X Ω {X } f (X ) f (X) {,,..., } : f (X ) < f (X). It mens tht the soluton X s sd to be Preto optml (mnml) f no other soluton cn be found to domnte X usng the defnton of Preto domnnce. Fgure : A GS-GDH Networ Structure of Chromosome 4 ult-obectve Optmzton ult-obectve optmzton whch s lso clled multcrter optmzton or vector optmzton hs been defned s fndng vector of decson vrbles stsfyng constrnts to gve optml vlues to ll obectve functons [8][]. In generl, t cn be mthemtclly defned s: Defnton of Preto front fnd the vector X = [ x,x,...,x n ] T to optmze For gven OP, the Preto front ƤŦ s set of vectors of obectve functons whch re obtned F(X) = [ f (X), f (X),..., f (X)] T, usng the vectors of decson vrbles (4) n the Preto subect to m nequlty constrnts set, Ƥ tht s, g (X) ƤŦ = {F(X) = (f.{ Ƥ 0, = to m, (7) (X), f (X),..., f (X)): X Therefore, the Preto front ƤŦ s set of the nd p equlty constrnts Defnton of Preto Set For gven OP, Preto set Ƥ s set n the decson vrble spce consstng of ll the Preto optml vectors, Ƥ = { X Ω X Ω : F(X ) p F(X) }. In other words, there s no other X n Ω tht domntes ny. Ƥ X. Ƥ vectors of obectve functons mpped from h (X) = 0, = to p, (8) n where X R s the vector of decson or desgn vrbles, nd F(X) R s the vector of obectve functons. Wthout loss of generlty, t s ssumed tht ll obectve functons re to be mnmzed. Such mult-obectve mnmzton bsed on the Preto pproch cn be conducted usng some defntons: Defnton of Preto domnnce A vector U = [ u, u,..., u ] R [ ] domntes to vector V = v, v,..., v R (denoted by U p V ) f nd only f {,,..., }, u v {,,..., } : u < v. It mens tht there s t lest one u whch s smller thn v whlst the rest u s re ether smller or equl to correspondng v s. Defnton of Preto optmlty n A pont X Ω (Ω s fesble regon n R stsfyng equtons (7) nd (8)) s sd to be Preto optml (mnml) wth respect to ll X Ω f nd Evolutonry lgorthms hve been wdely used for mult-obectve optmzton becuse of ther nturl propertes suted for these types of problems. Ths s mostly becuse of ther prllel or populton-bsed serch pproch. It s very mportnt n evolutonry lgorthms tht the genetc dversty wthn the populton be preserved suffcently. Ths mn ssue n OPs hs been ddressed by much relted reserch wor []. Consequently, the premture convergence of OEAs s prevented nd the solutons re drected nd dstrbuted long the true Preto front f such genetc dversty s well provded. The Preto-bsed pproch of NSGA-II [3] hs been recently used n wde rnge of engneerng OPs becuse of ts smple yet effcent non-domnnce rnng procedure n yeldng dfferent levels of Preto fronters. However, the crowdng pproch n such stte-ofthe-rt OEA wors effcently for two-obectve optmzton problems s dversty-preservng opertor whch s not the cse for problems wth more thn two obectve functons [9]. In our wor, ε-dversty s useed to modfy NSGA-II so tht t cn be sfely used for ny number of obectve functons (prtculrly for more thn two obectves) n OPs [9].

4 Proceedngs of the 6th WSEAS Int. Conf. on Systems Theory & Scentfc Computton, Elound, Greece, August -3, 006 (pp48-5) The evolutonry process strts by rndomly genertng n ntl populton of symbolc strngs ech s cnddte soluton. Then, the obectve functons tht hve been consdered n ths wor re trnng error (TE), predcton error (PE), nd number of neurons (N) re evluted for ech entre strng of symbolc dgts whch represents GDHtype neurl networ to model explosve cuttng process. The modfed NSGA-II [9-0] s then used for mult-obectve optmzton of GDH-type neurl networs. 5 Preto Optmzton of GDH Neurl Networ odels of Explosve Cuttng The prmeters of nterest n ths mult-nput sngleoutput system tht ffect both the performnce of the shped chrge nd the depth of penetrton, re the lner mterl, the explosve mterl, the lner shpe, the pex ngle, the lner thcness, the explosve weght nd dstrbuton, nd the stndoff dstnce. Among these prmeters, the lner nd explosve mterl together wth the lner shpe hve been ept fxed. Accordngly, there hs been totl number of 43 nput-output expermentl dt consderng 4 nput prmeters, nmely, the pex ngle, the stndoff, the lner thcness, nd the explosve weght [6]. However, n order to demonstrte the predcton blty of the evolved GDH-type neurl networs, the dt hve been dvded nto two dfferent set, nmely, trnng nd testng sets. The trnng set, whch conssts of 30 out of 43 nputs-output dt prs, s used for trnng the neurl networs models usng the evolutonry method of ths pper. The testng set conssts of 3 unforeseen nput-output dt smples durng the trnng process, s merely used for testng to show the predcton blty of such evolved GDH-type neurl networ models durng the trnng process. The GDH-type neurl networs re now used for such nput-output dt to fnd the polynoml model of depth penetrton n explosve cuttng process wth respect to ther effectve nput prmeters, nmely, pex ngle, stnd off, lner thcness, nd mss of chrge. In order to desgn GDH-type neurl networ descrbed n prevous secton from mult-obectve optmum pont of vew, populton of 60 ndvduls wth crossover probblty of 0.95 nd mutton probblty of 0. hs been used n 50 generton tht no further mprovement hs been cheved for such populton sze. In the multobectve optmzton desgn of such GDH-type neurl networs, dfferent prs of conflctng obectves (TE, N) nd (PE, N) re selected for - obectve optmzton desgn of neurl networs. The obtned Preto front for ech pr of - obgectve optmzton hve been shown n fgures () (b) for (TE, N) nd (PE, N), respectvely. It s cler from these fgures tht ll desgn ponts representng dfferent GDH-type neurl networs re non-domnted wth respect to ech other correspondng to tht pr of conflctng obectves. Fgures () nd (b) depct the Preto front of - obectve optmzton of trnng error nd number of neurons (TE, N) nd predcton error nd number of neurons (PE, N), respectvely. In ths fgure, ponts D nd G stnd for the best optmum vlues obtned for TE nd PE n ther correspondng - obectve optmzton process wth respect to the number of neuron (N). On the other hnd, ponts E nd H stnds for the smplest structure of GDHtype neurl networs (N=) wth ther correspondng vlues of (TE) nd (PE). () (b) Fgure : Preto front of trnng & predcton error nd number of neurons n -obectve optmzton. It s cler from these fgures tht ll the optmum desgn ponts (GDH-type neurl networs) n Preto front re non-domnted nd could be chosen by desgner for modellng nd predcton of explosve cuttng process. It s cler from the these fgures tht choosng better vlue for ny obectve functon n Preto front would cuse worse vlue for nother obectve. However, f the set of decson vrbles (genome structure of GDH-type neurl networs nd the ssocted coeffcents) s selected bsed on ech of the correspondng sets, t wll led to the best possble combnton of those two obectves s shown n fgures () (b). In other words, f ny other set of decson vrbles s

5 Proceedngs of the 6th WSEAS Int. Conf. on Systems Theory & Scentfc Computton, Elound, Greece, August -3, 006 (pp48-5) chosen, the correspondng vlues of the pr of obectves wll locte pont nferor to the correspondng Preto front. Such nferor re n the spce of the two obectves s n fct top/rght sde of fgures () nd (b). Clerly, there re some mportnt optml desgn fcts between the two obectve functons whch hve been dscovered by the Preto optmum desgn of GDH-type neurl networs. Such mportnt desgn fcts could not hve been found wthout the mult-obectve Preto optmzton of those GDH-type neurl networs. From fgures () nd (b) ponts F nd I re the ponts whch demonstrte these mportnt optml desgn fcts. The good behvour of GDH-type neurl networs model of pont I n trnng nd predcton dt re shown n fgure (3). Fgure 3: Comprson of ctul vlues wth the evolved GDH model correspondng to optmum pont I. 6 Concluson Evolutonry lgorthms hve been successfully used for mult-obectve Preto bsed optmzton of generlzed GDH-type neurl networs used for modellng nd predcton of complex hgh rte energy explosve cuttng process. Such multobectve optmzton led to the dscoverng of useful optml desgn prncples n the spce of obectve functons. It hs been shown tht there exst some optml structures of neurl networs (ponts F nd I of the gven Preto fronts) whch exhbt very resonble compromse between the conflctng obectve functons nd, thus, cn be confdently chosen s optmum polynoml neurl networs. Such mportnt results s useful optml desgn prncples would not hve been obtned wthout the use of mult-obectve optmzton pproch of neurl networs. References: [] Frlow, S.J. ed., 984. Self-orgnzng ethod n odellng: GDH type lgorthm, rcel Deer Inc. [] Ivhneno, A. G. 97 Polynoml Theory of Complex Systems. IEEE Trns. Syst. n & Cybern, SC-, [3] Yo, X Evolvng Artfcl Neurl Networs, Proceedngs of IEEE, 87(9):43-447, Sept. [4] Nrmn-Zdeh, N.; A. Drvzeh; nd R. Ahmd-Zdeh Hybrd Genetc Desgn of GDH-Type Neurl Networs Usng Sngulr Vlue Decomposton for odellng nd Predcton of the Explosve Cuttng Process, Proceedngs of the I ECH E Prt B Journl of Engneerng nufcture, Volume: 7, Pge: [6] Nrmn-zdeh, N.; A. Drvzeh; A. Jml; nd A. oen Evolutonry Desgn of Generlzed Polynoml Neurl Networs for odellng nd Predcton of Explosve Formng Process. Journl of terl Processng nd Technology, Vol 64-65, pp 56-57, Elsever. [7] Nrmn-zdeh, N.; A. Drvzeh;. Drvzeh; nd H. Ghrbbe. 00. odellng of explosve cuttng process of pltes usng GDHtype neurl networ nd sngulr vlue decomposton. Journl of terls Processng Technology, vol. 8, no., pp (8). [8] Coello Coello, C. A.; nd A. D. Chrstnsen ultobectve optmzton of trusses usng genetc lgorthms. Computers & Structures, 75, pp [9] Atshr, K.; N. Nrmn-zdeh; A. Plech; A. Jml; nd X. Yo Thermodynmc Preto Optmzton of Turbo Engnes usng ult-obectve Genetc Algorthms. Interntonl Journl of Therml Scence, 44, [0] Nrmn-zdeh, N.; K. Atshr; A. Jml; A. Plech; nd X. Yo Inverse odellng of ult-obectve Thermodynmclly Optmzed Turbo Engnes usng GDH-type Neurl Networs nd Evolutonry Algorthms. Engneerng Optmzton, Tylor & Frncs Group, Vol. 37, No. 5, [] Osyez, A ultcrter optmzton for engneerng desgn. In Desgn Optmzton, Gero, J.S., (ed.), pp 93-7, Acdemc Press, NY. [] Toffolo, A. nd E. Benn Genetc Dversty s n Obectve n ult-obectve evolutonry Algorthms. Evolutonry Computton.():5-67, IT Press. [3] Deb, K.; S. Agrwl; A. Prtp; nd T. eyrvn. 00. A fst nd eltst multobectve genetc lgorthm: NSGA-II. IEEE Trns. On Evolutonry Computton. 6():8-97, 00.