An Improved Genetic Algorithm with Average-Bound Crossover and Wavelet Mutation Operations

Transcription

1 An Improved Genetc Algorthm wth Average-Bound Crossover and Wavelet Mutaton Operatons S.H. Lng and F.H.F. Leung Centre for Multmeda Sgnal Processng, Department of Electronc and Informaton Engneerng, The Hong Kong Polytechnc Unversty, Hung Hom, Kowloon, Hong Kong Abstract: Ths paper presents a real-coded genetc algorthm (RCGA) wth new genetc operatons (crossover and mutaton). They are called the average-bound crossover (ABX) and wavelet mutaton (WM). By ntroducng the proposed genetc operatons, both the soluton qualty and stablty are better than the RCGA wth conventonal genetc operatons. A sute of benchmark test functons are used to evaluate the performance of the proposed algorthm. Applcaton examples on economc load dspatch and tunng an assocatve-memory neural network are used to show the performance of the proposed RCGA. Keywords: Crossover, mutaton, real-coded genetc algorthm, assocatve-memory neural network, and economc load dspatch. I. INTRODUCTION Genetc algorthm (GA) s one evolutonary computaton technque [9] that can tackle complex optmzaton problems [9, 7, 3]. It has been appled n dfferent areas such as fuzzy control [3-4], tunng of neural or neural fuzzy network [5-6], path plannng [], greenhouse clmate control [], economc load dspatch [, 7], etc. Tradtonal bnary GA [5, 9, 9, 5] has some drawbacks when applyng to multdmensonal and hgh-precson numercal problems. The stuaton can be mproved f GA n real numbers s used. Each chromosome s coded as a vector of floatng pont numbers that has the same length as the soluton vector. A large doman can thus be handled. Much research effort has been spent to mprove the performance of real-coded GA (RCGA). In general, RCGA nvolves three operatons: selecton, crossover and mutaton. The selecton operaton s used to select the chromosomes from the populaton wth respect to some probablty dstrbuton based on ftness values. The crossover operaton s used to combne the nformaton of the selected chromosomes (parents) and generate the offsprng. The mutaton

2 operaton s used to change the offsprng genes. Selecton schemes such as rank-based selecton, eltst strateges, steady-state electon and tournament selecton were reported [5]. Recently, dfferent crossover operatons for RCGA have been proposed to mprove the effcency of the algorthm. The extended ntermedate recombnaton (crossover) (EIX) was proposed by Mühlenben et. al. []. The genes (varables) of the offsprng are chosen somewhere between the genes of the parents. It s capable of producng any pont wthn a hypercube slghtly larger than that defned by the parents. The unmodal normal dstrbuton crossover (UNDX) was proposed by Ono et. al. [, ] for handlng multmodal functons and non-separablty problems. UNDX mxes the parental nformaton and shows a good searchng ablty. However, t changes the fundamental concept that the crossover operaton should combne the parents to generate offsprng, not mxng the parents. The blend crossover (BLX-α) was proposed by Eshelman et. al. [7], whch combnes the parents to reproduce offsprng. It shows a good searchng ablty for separable functons. However, BLX-α has dffculty n handlng non-separablty optmsaton problems. Also, the above crossover operatons are not sutable for optmsaton problems wth the optmal pont located near the doman boundary. For mutaton operatons, the unform mutaton and nonunform mutaton can be found [9, ]. The unform mutaton s to change the value of a randomly selected gene to a value between ts upper and lower bounds. The non-unform mutaton s capable of fne-tunng the parameters by ncreasng or decreasng the value of a randomly selected gene wth respect to a weghted random number. The weght s usually a monotonc decreasng functon of the number of teraton. In ths paper, new genetc operatons of crossover and mutaton are proposed. The crossover operaton s called the average-bound crossover (ABX), whch combnes the average crossover and bound crossover. The average crossover manpulates the genes of the selected parents, the mnmum, and the maxmum possble values of the genes. The bound crossover s capable of movng the offsprng near the doman boundary. On realzng the ABX operaton, the offsprng spreads over the doman so that a hgher chance of reachng the global optmum can be obtaned. The proposed mutaton operaton s called the wavelet mutaton (WM), whch apples the wavelet theory [3-4, 8] to realze the mutaton. Wavelet s a tool to model sesmc sgnals by combnng dlatons and translatons of a smple, oscllatory functon (mother wavelet) of a fnte duraton. The wavelet functon has two propertes: ) the functon ntegrates to zero, and ) t s square ntegrable, or equvalently has fnte energy. Thanks to the propertes of the wavelet, the convergence and soluton stablty are mproved. By ntroducng these genetc operatons, the RCGA performs more effcently and provdes a faster convergence than the RCGA wth conventonal genetc operatons n a sute of 8 benchmark test functons [6, 8, 4, 3]. In addton, the RCGA wth the proposed operatons gves smaller standard devatons of results,.e. the

3 soluton qualty of the RCGA wth the proposed operatons s more stable. An expermental study wll be made to evaluate the searchng ablty of the proposed mutaton. Also, the senstvty of the parameter n WM and the senstvty of the genes ntal range for the proposed RCGA to the searchng performance wll be dscussed. Applcaton examples on economc load dspatch and tunng an assocatve-memory neural network are also gven to show the performance of the proposed RCGA. Ths paper s organzed as follows. Secton II presents the operaton of the proposed genetc operatons. Expermental studes and analyss are dscussed n Secton III. 8 benchmark test functons wll be used to evaluate the performance of the proposed method. Applcaton examples on economc load dspatch and tunng an assocatve memory neural network are gven n Secton IV. A concluson wll be drawn n Secton V. II. AVERAGE-BOUND CROSSOVER AND WAVELET MUTATION FOR RCGA The Real-Coded Genetc Algorthm (RCGA) process [5, 9, 5] s shown n Fg.. Frst, a set of populaton of chromosomes P s created. Each chromosome p contans some genes (varables). Second, the chromosomes are evaluated by a defned ftness functon. The better chromosomes wll return hgher ftness functon values n ths process. Thrd, some of the chromosomes are selected to undergo genetc operatons for reproducton by the method of normalzed geometrc rankng []. Normalzed geometrc rankng s a selecton based on a nonstatonary penalty functon, whch s a functon of the generaton number. As the number of generaton ncreases, the penalty ncreases that puts more and more selectve pressure on the RCGA to fnd the feasble soluton. In general, a hgher-rank chromosome wll have a hgher chance to be selected. Fourth, genetc operatons of crossover are performed. The crossover operaton s manly for exchangng nformaton between two parents that are obtaned by the selecton operaton. In the crossover operaton, one of the parameters s the probablty of crossover p c whch gves the expected number p c pop _ sze (where pop _ sze s the number of chromosomes n the populaton) of chromosomes that undergo the crossover operaton n a generaton. We propose a new crossover operaton here. Frst, four chromosomes are generated (nstead of two chromosomes n the conventonal RCGA) from two selected parents. Second, the best two offsprng n terms of the ftness value wll be selected to replace ther parents. After the crossover operaton, the mutaton operaton follows. It operates wth a parameter called the probablty of mutaton ( p m ). The mutaton operaton s to change the genes of the chromosomes n the populaton such that the features nherted from ther parents can be changed. After gong through the mutaton operaton, 3

4 the new offsprng wll be evaluated usng the ftness functon. The new populaton wll be formed when the new offsprng replaces the chromosome wth the smallest ftness value. After the operatons of selecton, crossover and mutaton, a new populaton s generated. Ths new populaton wll repeat the same process. Such an teratve process wll be termnated when a defned condton s met. The detals about the proposed crossover and mutaton operatons are gven below. A. Average-bound crossover operaton The crossover operaton s manly for exchangng nformaton from the two parents, chromosomes p and p, obtaned n the selecton process. The two parents wll eventually produce two offsprng. The average-bound crossover (ABX) comprses two operatons: average crossover and bound crossover. ) Average crossover ( p p ) + o s = c ( p max + p mn )( w a ) + ( p + p ) w o a s = () c () ) Bound crossover ( p ) b o s = p ( w b ) + max w (3) 3 max, p c ( p ) b o s = p ( w b ) + mn w (4) 4 mn, p c where k = [ o k o k o k ] o, k =,, 3, 4. sc s s sno _ vars [ p p p p ] p =, =, ; =,,, no_vars, (5) no_vars paramn p para max max, (6) no _ vars [ para para para ] p =, (7) mn max max max no _ vars [ para para para ] p =, (8) mn mn no_vars denotes the number of varables to be tuned; maxmum values of mn para mn and p respectvely for all ;, [ ] a w b average crossover and bound crossover respectvely, max ( p ) para max are the mnmum and w denotes the user-defned weght for p denotes the vector wth each element obtaned by takng the maxmum between the correspondng element of p and p. For, 4

5 nstance, max( [ 3 ], [ 3 ] ) = [ 3 3]. Smlarly, mn( p ) the mnmum value. For nstance, mn( [ 3 ], [ 3 ] ) [ ] p, gves a vector by takng =. Among o to s c two wth the largest ftness values are used as the offsprng of the crossover operaton. These two offsprng are put back nto the populaton to replace ther parents. The ratonale behnd the ABX s that f the offsprng spreads over the doman, a hgher chance of reachng the global optmum can be obtaned. As seen from () to (4): The average crossover wll move the offsprng near the centre regon of the concerned doman (as w a n () o 4 s c, the approaches, approaches, o approaches ( p ) + p, whch s the average of the selected parents; and as w a s c o approaches p + p ), whch s the average of the doman boundary), c s ( max mn whle bound crossover wll move the offsprng near the doman boundary (as w b n (3) and (4) approaches, o s 3c and 4 s c o approaches p max and p mn respectvely). The result of the crossover depends on the values of the weghts w a and w b. Ther values depend on the optmsaton problem and are chosen by tral and error. Fg. shows an example ndcatng the relatonshp between the parents and the offsprng under dfferent values of the weghts. In ths fgure, the lne represents the doman of a gene. The end ponts of the lne represent the mnmum and maxmum values of the gene. The dot ( ) represents the parents and the crcle-dot ( ) represents the offsprng. The values n brackets represent the values of the genes under dfferent values of the weghts. For example, when p = and p = 4, referrng to () and (), the offsprng s c o and o should be equal s c to.5 and 3.75 respectvely when w =. 5. Accordng to () to (4), the offsprng s generated. We a can see how the offsprng spreads over the doman under dfferent values of w a and w b. Changng the value of the weght w wll change the characterstcs of the average crossover operaton. In a ths paper, the value of w a s arbtrarly set at.5. On the other hand, changng the value of the weght w wll change the characterstcs of the bound crossover operaton. b B. Wavelet mutaton operaton Before presentng the wavelet mutaton operaton, we frst dscuss the basc wavelet theory. ) Wavelet theory Certan sesmc sgnals can be modelled by combnng translatons and dlatons of an oscllatory functon wth a fnte duraton called a wavelet. A contnuous-tme functon ψ (x) s called a mother wavelet or wavelet f t satsfes the followng propertes: 5

6 Property : + ψ ( x) dx = (9) In other words, the total postve momentum of ψ (x) s equal to the total negatve momentum of ψ (x). Property : + ψ ( ) dx < () x where most of the energy n ψ (x) s confned to a fnte duraton and bounded. The Morlet wavelet (as shown n Fg.3) s an example mother wavelet, whch was proposed by Daubeches [4]: x / ( x) = e cos( 5x) ψ () The Morlet wavelet ntegrates to zero (Property ). Over 99% of the total energy of the functon s contaned n the nterval of.5 x. 5 (Property ). In order to control the magntude and the poston of ψ (x), we defne ψ ( ) as follows: a, b x x b ψ a, b ( x) = ψ () a a where a s the dlaton parameter and b s the translaton parameter. Notce that ( x) ψ, ( x) = ψ (3) As x ψ ( a, x) = ψ, (4) a a t follows that ψ ) s an ampltude-scaled verson of ψ (x). Fg. 4 shows dfferent dlatons of a, ( x the Morlet wavelet. The ampltude of ψ ) wll be scaled down as the dlaton parameter a a, ( x ncreases. Ths property s used to do the mutaton operaton n order to enhance the searchng performance. ) Wavelet mutaton The mutaton operaton s to change the genes of the chromosomes nherted from ther parents. In general, varous methods lke unform mutaton or non-unform mutaton [9, ] can be employed to realze the mutaton operaton. We propose a Wavelet Mutaton (WM) operaton based on the wavelet theory, whch exhbts a fne-tunng ablty. The detals of the operaton are as follows. Every gene of the chromosomes wll have a chance to mutate governed by a probablty of mutaton, p [ ] m, whch s defned by the user. Ths probablty gves an expected number ( p m pop _ sze no_vars) of genes that undergo the mutaton. For each gene, a random number 6

7 between and wll be generated such that f t s less than or equal to place on that gene whch s updated nstantly. If s = [ os, os, o ] chromosome and the element [ para, p m, the mutaton wll take o s the selected s no_ vars o s s randomly selected for mutaton (the value of s o s nsde mn, paramax ]), the resultng chromosome s gven by ˆ [ os oˆ s =,, s,, os no ] _ vars where,, no_vars, and ( paramax os ) ( o para ) o, os + δ f δ > oˆ s =, (5) os + δ s mn f δ δ = ψ a, ( ϕ) (6) δ = ϕ ψ a a By usng the Morlet wavelet n () as the mother wavelet, (7) δ = ϕ a e a ϕ cos 5 a where ϕ [.5,.5] s randomly generated. If δ s postve ( δ > ) approachng, the mutated (8) gene wll tend to the maxmum value of o. Conversely, when δ s negatve ( δ ) approachng s, the mutated gene wll tend to the mnmum value of searchng space for o s. A larger value of δ gves a larger o s. When δ s small, t gves a smaller searchng space for fne-tunng the gene. Referrng to Property of the wavelet, the sum of the postve δ s equal to the sum of the negatve δ when the number of samples s large and ϕ s randomly generated. That s, N N δ = for N, (9) where N s the number of samples. Hence, the overall postve mutaton and the overall negatve mutaton throughout the evoluton are nearly the same. Ths property gves better soluton stablty (smaller standard devaton of the soluton values upon many trals). As over 99% of the total energy of the mother wavelet functon s contaned n the nterval [.5,.5], ϕ can be generated from [.5,.5] randomly. The value of the dlaton parameter a can be set to vary wth the value of T τ n order to meet the fne-tunng purpose, where T s the total number of teraton and τ s the current number of teraton. In order to perform a local search when τ s large, the value of a should ncrease as T τ ncreases so as to 7

8 reduce the sgnfcance of the mutaton. Hence, a monotonc ncreasng functon governng a and τ s proposed as follows. T a = e ζ ln T + τ ( g ) ln( g ) where ζ s the shape parameter of the monotonc ncreasng functon, g s the upper lmt of the parameter a. In ths paper, g s set as. The effects of the varous values of the shape parameter ζ to a wth respect to T τ are shown n Fg. 5. The value of a s between and. τ Referrng to (8), the maxmum value of δ s when the random number of ϕ = and a= ( = ). T Then referrng to (5), the offsprng gene ( ) s s s () oˆ = o + para o = para. It ensures that a large search space for the mutated gene s gven. When the value T τ s near to, the value of a s τ so large that the maxmum value of δ wll become very small. For example, at =. 9 T max max and ζ =, the dlaton parameter a = 4. If the random value of ϕ s zero, the value of δ wll be equal to.58. Wth oˆ o +.58 ( para o ) gven for fne-tunng. s = s max s, a small searchng space for the mutated gene s C. Choosng the parameters We can regard the RCGA s seekng a balance between the exploraton of new regons and the explotaton of the already sampled regons n the search space. Ths balance, whch crtcally controls the performance of the RCGA, s governed by the rght choces of the control parameters: the probablty of crossover ( p ), the probablty of mutaton p ), the populaton sze (pop_sze), c the weghts of the proposed crossover ( w, w ) and the shape parameter ζ of WM. Some vews about these parameters are ncluded as follows: a b The probablty of crossover ( p c ) gves us an expected number ( p c pop_sze) of chromosomes whch undergo the crossover operaton n a generaton. When p c =, all ( m chromosomes n a generaton wll undergo the crossover operaton. Increasng the probablty of mutaton ( p m ) tends to transform the genetc search nto a random search. Ths probablty gves us an expected number ( p m pop_sze no_vars) of genes that undergo the mutaton. When p =, all genes wll mutate. The value of p m depends on the desred number of genes that undergo the mutaton operaton. m 8

9 Increasng the populaton sze wll ncrease the dversty of the search space, and reduce the probablty that GA wll prematurely converge to a local optmum. However, t also ncreases the tme requred for the populaton to converge to the optmal regon n the search space. Changng the value of the weght w a n the average-bounded crossover wll change the characterstcs of the average crossover operatons. It s chosen by tral and error, whch depends on the knd of the optmsaton problem. As the value of w a tends to, the offsprng tends to be the average of the selected parents. As the value of wa tends to, the offsprng tends to be the average of the doman boundary. For many optmsaton problems, the value of the weght w a can be set as.5. Changng the value of the weght w b n the average-bound crossover wll change the characterstcs of the bound crossover operatons. It s also chosen by tral and error, whch depends on the knd of the optmsaton problem. A value of w b approachng wll make the offsprng to be near the selected parents. As the value of w b tends to, the offsprng wll become near the doman boundary. Changng the parameter ζ wll change the characterstcs of the monotonc ncreasng functon of the wavelet mutaton. The dlaton parameter a wll take a value so as to perform fne-tunng faster as ζ s ncreasng. It s chosen by tral and error, whch depends on the knd of the optmsaton problem. When ζ becomes larger, the decreasng speed of the step sze (δ ) of the mutaton becomes faster. In general, f the optmsaton problem s smooth and symmetrc, the searchng algorthm s easer to fnd the soluton and process the fne-tunng n early teraton. Thus, a larger value of ζ can be used to ncrease the step sze of the early mutaton. More detals about the senstvty of ζ to WM wll be dscussed n the next secton. III. EXPERIMENTAL STUDIES AND ANALYSIS A. Benchmark test functon A sute of 8 benchmark test functons [6, 8, 4, 3] are used to test the performance of the RCGA wth the proposed genetc operatons. Many dfferent knds of optmzaton problems are covered by these benchmark test functons. They are dvded nto three categores: unmodal functons, multmodal functons wth only a few local mnma, and multmodal functons wth many local mnma. The 8 benchmark test functons are detaled n Appendx A. They can test 9

10 the searchng ablty of the proposed searchng algorthm comprehensvely. To avod the proposed crossover operaton ntroducng a strong bas to the optmal locaton at p + p ), the ranges ( max mn of the doman boundary for some test functons are set dfferent from those n [6, 8, 4, 3]. Functons f to f 7 are unmodal functons. Functons f 8 to f 3 are multmodal functons wth only a few local mnma. Functons f 4 to f 8 are multmodal functons wth many local mnma. B. Expermental setup The crossover operaton for comparson s the UNDXBXover, whch conssts of two publshed crossover operatons: Unmodal normal dstrbuton crossover (UNDX) [, ] and Blend crossover (BLX-α) [7]. The detals of these two crossovers are shown n Appendx B and Appendx C respectvely. The mutaton operaton for comparson s the non-unform mutaton (NUM) [9, ]. The detals of NUM are shown n Appendx D. The smulaton condtons are descrbed as follows. The shape parameter of NUM: It s chosen by tral and error through experments for good performance for all functons. The parameters ζ of WM: It s chosen by tral and error through experments for good performance for all functons. The weght of the ABX w a :.5 for all functons. The weght of the ABX w b :.5 for f to f 8 and f 5 to f 7 ;. for. f 9 to f 4, and f 8. Populaton sze:. Number of runs: 5. Selecton operaton: Normalzed geometrc rankng []. The probablty of selectng the best chromosome []:.8. Crossover operaton: For UNDX, the parameters β and µ are set at and.35 respectvely; for BLX-α, the parameter α s set at.336 [6]. Probablty of crossover p c :.8. Probablty of mutaton p m :.5 for f to f 6 and f 4 to f 8 ;.8 for f 7 to f 3. Intal populaton: It s generated unformly at random. In ths paper, RCGA wth Avergae-Bound Crossover and Wavelet Mutaton (), RCGA wth Avergae-Bound Crossover and Non-Unform Mutaton (), RCGA wth Unmodal Normal Dstrbuton and Blend Crossover and Wavelet Mutaton, (), and RCGA wth Unmodal Normal Dstrbuton and Blend crossover and Non-Unform Mutaton () are used to test the benchmark test functons.

11 C. Experment results. Unmodel Functons Functons f to f 6 are unmodal functons. The experment results n terms of the mean cost value, best cost value, standard devaton, and the t-test value for f to f 6 are tabulated n Table. I. The comparson between dfferent genetc operatons on f to f 6 s shown n Fg. 6. The t-test s a statstcal method to evaluate the sgnfcant dfference between two algorthms. The t- value wll be negatve f the frst algorthm s better than the second, and postve f t s poorer. When the t-value s smaller than.645 (degree of freedom = 49), there s a sgnfcant dfference between the two algorthms wth a 95% confdence level. Functon f s a sphere model whch s probably the most wdely used test functon. It s smooth and symmetrc. The performance on ths functon s a measure of the convergence rate of a searchng algorthm. For f, the results n terms of the mean and the best cost value of ABX wth WM or NUM are better than those of the correspondng UNDXBXover. Comparng ABX wth WM to UNDXBXover wth WM, the mean cost value s.5 tmes better. A much smaller standard devaton s gven by the, whch means the soluton s more stable. Comparng the mutaton operatons WM and NUM, the proposed WM s more effectve than NUM n term of the cost value and standard devaton. Both the soluton qualty and stablty offered by WM are better than those offered by NUM. In addton, the t value of.6 mples that the mproved genetc operatons (AveXover wth WM) are better than the conventonal genetc operatons (UNDXBXover wth NUM). In Fg. 6, ABX wth WM dsplays a faster convergence rate than UNDXBXover wth NUM thanks to ts better searchng ablty. It reaches approxmately. n around 5 tmes of teraton, whle t s about 3. for UNDXBXover wth NUM. Functon f s a generalzed Rosenbrock s functon whch s strongly non-separable and the optmum s located n a very narrow rdge. The tp of the rdge s very sharp, and t runs around a parabola. Algorthms that are unable to dscover good searchng drectons wll perform poorly n ths problem. The proposed genetc operatons (ABX wth WM) outperforms the UNDXBXover wth NUM. The t value s Although the best cost values on usng WM wth dfferent crossover operatons are a bt worse than those on usng NUM, the mean value, standard devaton and convergence rate offered by WM are better. Functon f 3 s a step functon that s a representatve of flat surfaces. Flat surfaces are obstacles for optmzaton algorthms because they do not gve any nformaton about the search drecton. Unless the algorthm has a varable step sze, t can get stuck n one of the flat surfaces. UNDXBXover performs poorly for f 3 because t manly searches n a small local neghbourhood, but the flat

12 surfaces do not gve any searchng drecton for UNDXBXover. On the other hand, the proposed ABX s good for f 3 because t can generate longer ump than UNDXBXover. Comparng WM to NUM wth UNDXBXover, the former also gves a better soluton. Functon f 4 s a quartc functon, whch s a smple unmodal functon padded wth nose. The Gaussan nose causes the algorthm never gettng the same value at the same pont. Many algorthms that do not do well n ths functon are due to the nosy data. In ths functon, the mean cost value, best cost value, standard devaton and the convergence rate brought by the proposed ABX and WM are sgnfcantly better than the conventonal genetc operatons. Functon f 5 s the Schwefel s problem., functon f 6 s the Schwefel s problem. and functon f 7 s the Eason s functon. In these problems, the performance on usng the proposed crossover ABX and mutaton WM s better than that on usng the UNDXBXover and NUM. The rapd convergence of the proposed genetc operatons shown n Fg. 6 supports our argument. In short, the proposed genetc operatons (ABX and WM) are good to tackle unmodal functons/problems when compared wth the conventonal genetc operatons (UNDXBXover and NUM). Both the soluton qualty and stablty are satsfactory.. Multmodel functons wth a few local mnma Functons f 8 to f 3 are multmodal functons wth only a few local mnma. The expermental results for f 8 to f 3 are tabulated n Table II. Fg. 7 shows the average values for f 8 to f 3. Among these functons, fve of them ( f 8, f - f 3 ) do not show statstcally sgnfcant dfferences for dfferent genetc operatons. They all reach or get near to the global optma. For the functon f 9, we obtan statstcally dfferent results from the proposed genetc operatons and the conventonal genetc operatons. The proposed ABX performs better than the UNDXBXover n terms of the mean, best value, standard devaton and the convergence rate. In addton, the results offered by WM are better than those by NUM n terms of the mean and the best cost values. Furthermore, WM gves a faster convergence rate. 3. Multmodel functons wth many local mnma Functons f 4 to f 8 are multmodal functons wth many local mnma, and the dmenson of each functon s comparatvely larger than that of f 8 to f 3. The dmenson of these functons s 3. The expermental results for f 4 to f 8 are tabulated n Table III. The comparson between dfferent genetc operatons s shown n Fg. 8. It can be seen from Table III that the mean results and the best results offered by the proposed genetc operatons (ABX and WM) are better than

13 those offered by the conventonal genetc operatons (UNDXBXover and NUM). Also, they have smaller standard devatons. Therefore, n terms of the soluton qualty and stablty, the proposed genetc operatons are better than the conventonal operatons. In addton, the t-test value of all functons s smaller than.645. Therefore, the proposed genetc operatons are sgnfcantly better than the conventonal operatons for solvng the optmzaton problems. From Fg. 8, we can see that the convergence rate offered by the proposed genetc operatons s better than that offered by the conventonal genetc operatons. D. The searchng ablty of wavelet mutaton In ths secton, we gve an analyss based on expermental results to llustrate that the searchng ablty of WM s better than that of NUM. The expermental settngs are the same as before, except the probablty of crossover s set at and the probablty of mutaton s set at. By usng ths settng, no chromosomes wll undergo the crossover operaton, and all genes n the populaton wll mutate under the mutaton operaton. Hence, the searchng ablty of the mutaton operaton can be evaluated. The expermental results on usng WM and NUM for f to f 8 (except f 3 and f 7 ) wthout crossover operaton are summarzed n Table IV. The comparson between WM and NUM s gven n Fg. 9 and Fg.. Functon f 3 and f 7 are not ncluded n ths experment because they do not perform well wth mutaton operaton only. As seen from Table IV, the average performance of WM s better than NUM. WM gves smaller standard devatons of results for all test functons than NUM, and hence the soluton stablty offered by WM s better. From Fg. 9 and Fg., the convergence of WM s found faster than that of NUM. In concluson, the searchng ablty of WM s better than NUM. E. Senstvty of the parameter for wavelet mutaton The mean cost values offered by WM usng dfferent shape parameter ζ for all test functons are tabulated n Table V. As can be seen from the table, all functons are tested by usng ζ =., ζ =.5, ζ =, ζ =, and ζ =5. If the optmzaton problem needs a more sgnfcant mutaton to reach the optmal pont, a smaller ζ should be gven. Conversely, f the RCGA needs to perform the fne-tunng faster, a larger ζ should be used. For example, f s a sphere model whch s smooth and symmetrc. Searchng algorthms are fast to ump to the area near the global optmum and then perform fne-tunng. Therefore, a larger ζ s set (ζ =5) so that the RCGA wll go to perform fne-tunng faster. On the other hand, ζ s set as. for f 3 when the mutaton operaton s playng a sgnfcant role at the later stage. In some cases, ζ s value s not very crtcal, e.g. f 3 and f. For f 3, the mean cost value for dfferent ζ s the same. We say that the best 3

14 performance s obtaned when ζ =.5 because the standard devaton of the RCGA for ζ =.5 s the smallest. However, n some cases, the parameter ζ s so senstve as to affect the performance of the searchng, e.g. f and f 6. In concluson, no formal method s avalable to choose the parameterζ ; t depends on the characterstcs of the optmzaton problems. F. Senstvty of the ntal range of varables Addtonal experments are carred out to test the senstvty of the ntal range of the varables to the RCGA wth the mproved genetc operatons. The settngs of these experments are exactly the same as before (secton III B). The experment results for f 4 are tabulated n Table VI. Fg. shows the results for dfferent genetc operatons on f 4. The ntal populaton s generated unformly at random n the ranges of.56 x 5. (twce the orgnal range), 6.4 x. 8 (5 tmes of the orgnal range),.8 x 5. 6 ( tmes of the orgnal range), and 5.6 x 5. ( tmes of the orgnal range), makng the average dstance to the global optmum ncreasngly large. The enlarged searchng space s expected to make the problem more dffcult to solve. As can be seen from the table and the fgures, the mean cost values offered by the proposed genetc operatons are better than those by the conventonal genetc operatons. From Fg., ABX and WM offer faster convergence than UNDXBXover and NUM. In addton, ABX and WM gve smaller standard devaton for all ntal ranges than UNDXBXover and NUM. Hence, the soluton qualty s more stable. Two more test functons are then used to test the senstvty to the ntal range of varables. The experment results for f 7 and f 6 are tabulated n Table VII to VIII respectvely. Fg. and Fg. 3 show the results for dfferent genetc operatons on f 7 and f 6 respectvely. In these tables and fgures, the results of the mproved genetc operatons n terms of the mean cost value, convergence rate, and standard devaton are better than those of the conventonal algorthms. IV. APPLICATION EXAMPLES Applcaton examples on economc load dspatch and tunng of assocatve memory are gven n ths secton. A. Economc load dspatch In a power system, mnmzng the operaton cost s mportant. Economc load dspatch (ELD) s a method to schedule power generator outputs wth respect to the load demands, and to operate a power system economcally. The nput-output characterstcs of modern generators are 4

15 nonlnear by nature because of the valve-pont loadngs and rate lmts. The problem of ELD s multmodal, dscontnuous and hghly nonlnear. RCGAs had been employed to solve the ELD problems [, 7].. Mathematc modellng of economc load dspatch wth valve-pont loadng The economc load dspatch wth valve-pont loadng problem can be formulated nto the followng obectve functon: Mn n = C ( P ) L, () where C ( P L ) s the operaton fuel cost of generator, and n denotes the number of generators. The problem s subect to a balance constrant and generatng capacty constrants as follows: D = P L n = P L P Loss, () L P P, =,,, n. (3) L, mn,max where D s the load demand, loss, P L,max and PL s the output power of the -th generator, P Loss s the transmsson P L,mn are the maxmum and mnmum output powers of the -th generator respectvely. The operaton fuel cost functon wth valve-pont loadngs of the generators s gven by, C ( P ) a P + b P + c + e sn( f ( P P ) L = L L L, mn L, (4) where a, b, and c are coeffcents of the cost curve of the -th generator, e and f are coeffcents of the valve-pont loadngs. (The generatng unts wth multvalve steam turbnes exhbt a great varaton n the fuel-cost functons. The valve-pont effects ntroduce rpples n the heat-rate curves.) RCGA can be used to solve the economc load dspatch problem. The chromosomes p s defned as follows: [ P P P P ] p, (5) = L L L3 L n From (), we have, n PL = D PL + P n Loss. (6) = In ths paper, the power loss s not consdered. Therefore, n P = D P. (7) Ln = L 5

16 To ensure f f PL falls wthn the range [ P ] n L, P n, mn Ln,max P = P + ( P P ), the followng condtons are consdered: L New L Ln,max Ln P L > P n L, (8) n,max PL = P n Ln,max ( P P ) PL = New PL Ln Ln,mn P L < P n L. (9) n,mn PL = P n Ln,mn It should be noted from (8) and (9) that f the value of PL s also outsde the constrant boundares. The exceedng porton of the power wll further be shared by other generators n order to make sure that all generators output power s wthn the safety range. Referrng to (), the ftness functon for ths ELD problem s defned as: n ftness = C ( PL ) =, (3) where C ( P L ) s defned n (4). The obectve s to maxmze the ftness functon (3).. Case Study The RCGA wth the proposed genetc operatons and the RCGA wth the conventonal genetc operatons are appled to a 4-generator system, whch was adopted as an example n []. The system s a very large one wth nonlneartes. The data of the unts for ths example wth valve-pont loadngs are tabulated n Table IX. The load demand (D) s 5MW. For comparson purpose, RCGA wth ABX and WM, RCGA wth ABX and NUM, RCGA wth UNDXBXover and WM, and RCGA wth UNDXBXover and NUM are used to solve the ELD problem. The populaton sze used for all RCGAs s. All the smulaton results are averaged ones out of 5 runs. For the proposed ABX, the parameters w a and w b are set at.5. For the UNDXBXover, the parameters β, µ, and α are set at,.35, and.336 respectvely. For the proposed mutaton WM, the parameter ζ s set at. For the NUM, the shape parameter s set at. The probabltes of crossover and mutaton for all approaches are set at.6 by tral and error. For all approaches, the number of teraton s. The statstcal results are shown n Table X and Fg. 4. It can be seen that the RCGA wth the proposed ABX and WM performs better than other RCGAs wth conventonal genetc operatons (UNDXBXover and NUM) n terms of cost, t value, and standard devaton. Both the soluton qualty and stablty are good. The average cost s $8.4 and the best (mnmum) cost s $ The optmal dspatch soluton s summarzed n Table XI. B. Tunng assocatve memory 6

17 Learnng or tranng s one of the mportant ssues of neural networks. The learnng process ams to fnd a set of optmal network parameters. The wdely-used gradent methods [8, 3], such as MRI, MRII, MRIII rules, and back-propagaton technques, adust the network parameters based on the gradent nformaton of the ftness functon n order to reduce the mean square error over all nput patterns. One maor weakness of the gradent methods s that the dervatve nformaton of the ftness functon has to be known, meanng that the ftness functon has to be contnuous. Also, the learnng process s easly trapped n a local optmum, especally when the problems are multmodal and the learnng rules are network structure dependent. To tackle ths problem, the real-code genetc algorthm (RCGA) [5, 9, 5], was proposed for the optmzaton problem n a large, complex, non-dfferentable and multmodal doman [9]. RCGA s a good tranng algorthm for neural or neural-fuzzy networks [5-6]. The same RCGA can be used to tran many dfferent networks regardless of whether they are feed-forward one, recurrent one, assocatve memory or of other structure types. Ths generally saves a lot of human efforts n developng tranng algorthms for dfferent types of networks. Assocatve memory s one type of neural network that maps ts nput vector nto tself. Thus, the desred output vector s ts nput vector. 5 nput vectors are used for the learnng. The functon of the assocatve memory s gven by: k ( t) = w z ( t) y, k =,,, (3) = k where z(t) s the nput vector and w k s the weght of the lnk between the -th nput and the k-th output. The obectve s to mnmze the mean square error (MSE), whch s defned as follows: MSE = 5 k = t= ( z ( t) y ( t) ) k 5 k The ntal range of the weght w k s from to. For comparson purpose, RCGA wth ABX and WM, RCGA wth ABX and NUM, RCGA wth UNDXBXover and WM, and RCGA wth UNDXBXover and NUM are used to solve ths problem. The populaton sze used for all RCGAs s. All the smulaton results are averaged ones out of 5 runs. For the proposed ABX, the parameters w a and w b are set at.5 and respectvely. For the UNDXBXover, the parameters β, µ, and α are set at,.35, and.336 respectvely. For the proposed mutaton WM, the parameter ζ s set at. For the NUM, the shape parameter s set at. The probabltes of crossover and mutaton for all approaches are set at.8 and. by tral and error. For all approaches, the number of teraton s. The expermental results are tabulated n Table XII, and the comparson between dfferent genetc operatons s shown n Fg. 5. As can be seen from the table, the mean and the best cost value offered by ABX and WM are better. In addton, the smaller standard (3) 7

18 devaton mples a more stable soluton. The t-value for ths functon s 4.67, whch s a relatvely large fgure. In short, the proposed RCGA s good for tunng assocatve memory. V. CONCLUSION An RCGA wth mproved genetc operatons (average-bound crossover and wavelet mutaton) has been presented. By usng the proposed crossover operaton, the offsprng spreads over the doman so that the probablty of reproducng good offsprng s ncreased. In the proposed mutaton operaton, the wavelet theory s appled. Thanks to the propertes of the wavelet, both the soluton qualty and stablty are mproved. A sut of benchmark test functons has been used to llustrate the merts of the mproved genetc operatons. Examples on economc load dspatch and tunng assocatve memory have also been gven. ACKNOWLEDGEMENT The work descrbed n ths paper was substantally supported by a grant from the Hong Kong Polytechnc Unversty (PhD Student Account Code: RG9T). REFERENCES [] Caponetto R., Fortuna L., Nunnar G., Occhpnt L., and Xbla M.G., Soft computng for greenhouse clmate control, IEEE Trans., Fuzzy Systems, vol. 8, no. 6, pp , Dec.. [] Chen P.H. and Chang H.C., Large-scale economc dspatch by genetc algorthm, IEEE Trans. Power Syst., vol., pp. 7-4, Feb [3] Daubeches I., The wavelet transform, tme-frequency localzaton and sgnal analyss, IEEE Trans. Informaton Theory, vol. 36, no.5, pp. 96-5, Sep. 99. [4] Daubeches I., Ten lectures on wavelets. Phladelpha, PA: Socety for Industrcal and Appled Mathematcs, 99. [5] Davs L., Handbook of genetc algorthms. NY: Van Nostrand Renhold, 99. [6] De Jong K.A., An analyss of the behavor of a class of genetc adaptve systems, Ph.D. Thess, Unversty of Mchgan, Ann Arbor, MI, 975. [7] Eshelman L.J. and Schaffer J.D., Real-coded genetc algorthms and nterval-schemata, Foundatons of Genetc Algorthms, pp. 87-, 993. [8] Goldsten A.A. and Prce I.F., On descent from local mnma, Math. Comput., vol. 5, no. 5, 97. [9] Holland J.H., Adaptaton n natural and artfcal systems. Unversty of Mchgan Press, Ann Arbor, MI,

19 [] Jones J. and Houck C., On the use of non-statonary penalty functons to solve constraned optmzaton problems wth genetc algorthm, n Proc. 994 Int. Symp. Evolutonary Computaton, Ordando, 994, pp [] Judette H. and Youlal H., Fuzzy dynamc path plannng usng genetc algorthms, Electroncs Letters, vol. 36, no. 4, pp , Feb.. [] Kta H., Ono I., and Kobayash S., Theoretcal analyss of the unmodal normal dstrbuton crossover for real-coded genetc algorthms, n Proc. of the Congress on Evolutonary Computatonal (CEC998), World Congress on Computatonal Intellgence (WCCI 998), May 4-9, 998, pp [3] Lam H.K., Leung F.H.F., and Tam P.K.S., Desgn and stablty analyss of fuzzy model based nonlnear controller for nonlnear systems usng genetc algorthm, IEEE Trans. Syst., Man and Cybern, Part B: Cybernetcs, vol. 33, no., pp. 5-57, Feb. 3. [4] Leung F.H.F., Lam H.K., Lng S.H., and Tam P.K.S., "Optmal and stable fuzzy controllers for nonlnear systems usng an mproved genetc algorthm," IEEE Trans. Industral Electroncs, vol. 5, no., pp.7-8, Feb. 4. [5] Leung F.H.F, Lam H.K., Lng S.H., and Tam P.K.S., "Tunng of the structure and parameters of neural network usng an mproved genetc algorthm," IEEE Trans. Neural Networks, vol.4, no., pp.79-88, Jan. 3. [6] Lng S.H., Leung F.H.F, Lam H.K., and Tam P.K.S., "Short-term electrc load forecastng based on a neural fuzzy network," IEEE Trans. Industral Electroncs, vol. 5, no. 6, pp.35-36, Dec. 3. [7] Lu B.D., Chen C.Y., and Tsao J.Y., Desgn of adaptve fuzzy logc controller based on lngustchedge concepts and genetc algorthms, IEEE Trans. Systems, Man and Cybernetcs, Part B, vol. 3 no., pp. 3-53, Feb.. [8] Mallat S.G., A theory for multresoluton sgnal decomposton: the wavelet representaton, IEEE Trans. Pattern analyss and machne ntellgence, vol., no.7, pp , Jul [9] Mchalewcz Z., Genetc Algorthm + Data Structures = Evoluton Programs, nd extended ed. Sprnger-Verlag, 994. [] Mühlenken H. and Schlerkamp-Voosen D., Predctve models for the breeder genetc algorthm I. contnuous parameter optmzaton, Evolutonary Computaton, vol., no., pp. 5-49, 993. [] Neubauer A., A theoretcal analyss of the non-unform mutaton operator for the modfed genetc algorthm, n Proc. IEEE Int. Conf. Evolutonary Computaton, 997, Indanapols, pp [] Ono I. and Kobayash S., A real-coded genetc algorthm for functon optmzaton usng unmodal normal dstrbuton crossover, n Proc. 7 th ICGA, 997, pp [3] Pham D.T. and Karaboga D., Intellgent Optmzaton Technques, Genetc Algorthms, Tabu Search, Smulated Annealng and Neural Networks. Sprnger,. [4] Schwefel H.P., Numercal optmzaton of computer models. Chchester, Wley & Sons, 98. [5] Srnvas M.and Patnak L.M., "Genetc algorthms: a survey," IEEE Computer, vol. 7, ssue 6, pp. 7-6, June

20 [6] Takahash M. and Kta H., A crossover operator usng ndependent component analyss for realcoded genetc algorthms, n Proc. of the Congress on Evolutonary Computaton, (CEC),, pp [7] Walter D.C. and Sheble G.B., Genetc algorthm soluton of economc dspatch wth valve pont loadng, IEEE Trans. Power Syst., vol. 8, pp.35-33, Aug [8] Wdrow B. and Lehr M.A., "3 years of adaptve neural networks: Perceptron, madalne, and backpropagaton," Proceedngs of the IEEE, vol. 78, no. 9, pp , Sept. 99. [9] Yao X., Evolvng artfcal networks, Proceedngs of the IEEE, vol. 87, no. 7, pp , 999. [3] Yao X. and Lu Y., Evolutonary programmng made faster, IEEE Trans. Evolutonary Computaton, vol. 3, no., pp. 8-, July 999. [3] Zurada J.M., Introducton to Artfcal Neural Systems. West Info Access, 99. Procedure of the RCGA begn τ // τ : teraton number Intalze P(τ) // P(τ) : populaton for teraton τ Evaluate f(p(τ)) // f(p(τ)) :ftness functon whle (not termnaton condton) do begn τ τ+ Select parents p and p from P(τ ) Perform crossover operaton wth p c Four chromosomes wll be generated Select the best two offsprng n terms of the ftness value Perform mutaton operaton wth p m Reproduce a new P(τ) Evaluate f(p(τ)) end end Fg.. RCGA process. ( ) (.5) (4) ( ) () p O O p s c s c paramn paramax w a = paramn ( ) (4) ( ) () p 3 O p O 4 sc s c paramax w b = paramn ( ) ( ) (.5) (3.3) (4) () p O O s p s c c paramax w a ( ) (.5) (3.75) (4) ( ) () w para mn a p O O para p max s s c c =.75 =.5 (.75) ( ) (4) (5.5) ( ) () para mn w b O p p para 3 max s c O s 4 c (.5) ( ) (4) (7) ( ) () para mn w para b O p p O max 3 4 s c sc =.75 =.5 ( ) ( ) (.5) (4) (4.38) () p O p O s c paramn s c paramax w a =.5 (.5) ( ) (4) (8.5) ( ) ) paramn O 3 sc p p O 4 sc ( paramax w b =.5 ( ) (.5) (4) (5) ( ) () w para mn para a p O p max s c O s c = ( ) ( ) (4) () ( ) () para mn w para b O p p max O s 3 s c 4 c = parent offsprng p O s c Fg.. Parents and offsprng under dfferent values of the weghts wa and w ( w, w =,.5,.5,.75 and.) b a b

21 ψ ( x) x Fg. 3. The Morlet Wavelet. a= a=5 a= a= a= a=5 a= a= Fg. 4. A Morlet wavelet dlated by dfferent values of the parameter a (x-axs: x, y-axs: ψ ).) a, ( x 4 a 3 ζ = 5 ζ = dlaton parameter ζ = ζ =.5 ζ = τ T Fg. 5. The effect of the shape parameter ζ to a wth respected to τ. T

22 ftness cost value ftness cost value value teraton number f teraton number f ftness cost value value 3 5 cost ftness value value UNDXXover+WM teraton number f teraton number f 4 cost value ftness value - UNDXXover+WM ftness cost value value UNDXXover+NUM teraton number f teraton number f cost value ftness value teraton number f 7 Fg. 6. Comparsons between dfferent genetc operatons for f to f 7. All results are averaged ones over 5 runs.

23 3 - ftness cost value cost value ftness value teraton number f teraton number f ftness cost value cost value ftness value teraton number -3.4 f teraton number -.6 f cost value ftness value cost ftness value teraton number f teraton number f 3 Fg. 7. Comparson between dfferent genetc operatons for f 8 to f 3. All results are averaged ones over 5 runs. 3

24 3 ftness cost value value 5 cost ftness value value teraton number teraton number f 4 f 5 cost value ftness value - - ftness cost value teraton number teraton number f 6 f ftness cost value value AveBXover+NUM AveBXover+WM teraton number f 8 Fg. 8. Comparson between dfferent genetc operatons for f 4 to f 8. All results are averaged ones over 5 runs. 4

25 cost ftness value value WM NUM ftness cost value 3 NUM - WM teraton number f teraton number f 3 ftness cost value NUM ftness cost value value WM NUM - WM teraton number f teraton number f ftness cost value WM NUM ftness cost value NUM teraton number f 6 WM teraton number f cost ftness value value ftness cost value value NUM NUM WM teraton number WM teraton number f 9 Fg. 9. Comparson between WM and NUM for f to f (except f 3 and f 7 ) wthout the crossover operaton. All results are averaged ones over 5 runs. f 5

26 cost ftness value value NUM cost ftness value value WM NUM teraton number -.6 f WM teraton number f cost ftness value ftness cost value value 4 NUM -.8 NUM WM teraton number f 3 3 WM teraton number f 4 3 NUM cost value ftness value WM ftness cost value - WM NUM teraton number f teraton number f cost value ftness value - WM NUM cost ftness value NUM teraton number f 7 - WM teraton number f 8 Fg.. Comparson between WM and NUM for f to f 8 wthout the crossover operaton. All results are averaged ones over 5 runs. 6

27 4 3 ftness cost value - cost value ftness value teraton number 5 (a) teraton number 6 (b) cost ftness value cost value ftness value 3 - ABXr+NUM teraton number (c) teraton number (d) Fg.. Comparson between dfferent genetc operatons for f 4 wth dfferent ntal ranges of varables. All results are averaged over 5 runs cost value ftness value ftness cost value teraton number (a) teraton number (b) cost value ftness value ftness cost value teraton number teraton number (c) (d) Fg.. Comparson between dfferent genetc operatons for f 7 wth dfferent ntal ranges of varables. All results are averaged ones over 5 runs. 7

28 4 6 3 cost ftness value value - ftness cost value value teraton number 6 (a) teraton number 6 (b) ftness cost value ftness cost value teraton number (c) teraton number (d) Fg. 3. Comparson between dfferent genetc operatons for f 6 wth dfferent ntal ranges of vrables. All results are averaged ones over 5 runs..3 x cost value ftness value teraton number Fg. 4. Comparsons between dfferent genetc operatons for ELD. All results are averaged ones over 5 runs. 8