A stochastc algorthm for functon mnmzaton DONGCAI SU School of Communcatons, Jln Unversty, Changchun, Jln 3002, Chna E-mal: suntree452@gmal.com JUNWEI DONG Bradley Department of Electrcal & Computer Engneerng, Vrgna Polytechnc Insttute and State Unversty, VA 22043, USA E-mal: dwe@vt.edu ZUDUO ZHENG Department of Cvl & Envronmental Engneerng, Hoha Unversty and Arzona State Unversty, 5 E. Lemon St. 40 E Apt., Tempe, AZ 8528 USA Emal: zhengzuduo@gmal.com Phone: +-480-727-9805 Abstract Focusng on what an optmzaton problem may comply wth, the so-called convergence condtons have been proposed and sequentally a stochastc optmzaton algorthm named as DSZ algorthm s presented n order to deal wth both unconstraned and constraned optmzatons. Its prncple s dscussed n the theoretcal model of DSZ algorthm, from whch we present a practcal model of DSZ algorthm. Practcal model s effcency s demonstrated by comparson wth smlar algorthms such as Enhanced Smulated Annealng (ESA), Monte Carlo Smulated Annealng (MCS), Snffer Global Optmzaton (SGO), Drected Tabu Search (DTS), and Genetc Algorthm (GA), usng a set of well-known both unconstraned and constraned optmzaton test cases. Meanwhle, further attenton goes to the strategy how to optmze the hgh-dmensonal unconstraned problems usng DSZ algorthm. Keywords: Global optmzaton, unconstraned optmzaton, constraned optmzaton
Introducton Assumng the optmzaton task s to mnmze the obectve functon, and pont p s the global soluton. Regardng to the condton whch the obectve functons concerned n ths paper satsfy: (secton 2.): "The smaller a pont s functon value, the hgher probablty that ths pont s closer to p." DSZ algorthm (secton 2.4) s presented nvolvng two strateges:. Pont set evoluton accordng to ts functon value; 2. Shrnkng operaton accordng to the shrnkng coeffcent c s employed durng the pont set evoluton. Accordng to the computatonal eperments (secton 2.5, 4.4), the effcency of DSZ algorthm s encouragng. Furthermore, the strategy of handlng hgh dmensonal, unconstraned problem was dscussed n secton 3 2 Optmzaton Prncples and Procedure 2. Condtons on the optmzaton problems The consdered obectve functon s denoted as f (), :[, L,, L, n ] where ( n) s real number. Let D the regon of ( n), where D: l u, ( l u, n). And l + u o = s the center of D; 2 Wthout losng generalty, the optmzaton task s to search for mnma of f (). The correspondng soluton for the global mnma s p :[ p, L, p, L, pn ],( n). D = ( k( D o) + ) D,( D,0 < k 2), whose central pont s, the rato of k smltude between D k and D s k. r 2 p = ma, n, ( D,0 < k 2 ) k( u l ) k 2
Optmzaton problems consdered n ths paper shall meet the followng three condtons: () Contnuty: Gven thatε s a postve number however small, and 2 are any two ponts n D,f 0 < 2 < δ,when δ s suffcently small, the f () always satsfes f ( ) f ( 2) < ε. (2) Convergence Condtons I: θ ( k) s a stochastc value whch s correlated to k ( k (0,2] ), defned as θ ( k) = p ' p where p D k, and ' as a random pont n D s subected to f ( ') < f ( ). Then for any postve k nteger N 0, whch s large enough, and N > N 0, we have N C(k) θ < ε ' = where ε ' s a postve number however small, and k ( N ) s any number from ( 0,2 ]. (3) Convergence Condtons II: Assume λ 0 (0,) and P (0, 0 ) are fed constants and ' s a pont randomly selected n D. If k p D k and rk λ0, then prob[ f ( ') < f ( )] P0, where prob [*] denotes the probablty of event *. 2.2 The Theoretcal Model The theoretcal procedure of DSZ algorthm s shown below: () Intalzng: set = and ntalze the teratve number and the mamum teraton (ma_ter); randomly generate from regon D and ntalze k (0, 2] to enforce p D k and rk λ ; 0 (2) Randomly generate ' from regon D k. If f ( ') < f ( ), then let = + ', otherwse, let = + ; (3) Choose c to ensure k+ p D + and + k λ + 0 r, where k = + c k ; 3
(4) Let =+, return to step (2) untl reaches ma_ter ; (5) Take ma_ter as the optmum soluton. 2.3 Optmzaton Prncple Theorem: Accordng to the theoretcal model, f the mamum teraton ma_ter s suffcently large, then f ( p) f ( ) < ε, where ε s a postve number ma_ ter however small, ma_ ter s the optmum soluton. Proof: [, L,, L, ] be the mamum subset of [, L,, L, ma_ ter ] Let τ () τ ( ) τ ( M ) whch holds: (ⅰ) τ ( ) [, ma_ ter] s a postve nteger andτ( +)> τ( ), ( M ) ; f ( + ) < f ( ). (ⅱ) τ ( ) τ ( ) Accordng to Convergence Condtons II as k k p D, r λ ( ma_ ter), when ma_ter s suffcent large, we have 0 M P ma_ ter > N +. 0 0 Also, from the defnton of θ ( k) n Convergence Condtons I, we have p τ ( + ) θ ( kτ ( ) ) =,( M ). p τ ( ) Thus, M D ma_ ter ma θ ( ) = D p d ( k τ ), where dma between p and any other pont n regon D. s the mamum dstance Accordng to Convergence Condtons I as well as the fact that M > N0, we have D ma_ ter p < ε ' dma < δ, 4
Where δ ε ' < d D ma. Due to the contnuty condton, t can be found: f ( p) f ( ) < ε, ma_ ter Where ε s a postve however small number, whch proves that ma_ter s actually the optmum soluton. 2.4 The Practcal Model Because of the uncertanty of pont p before optmzaton, values of k and c ( ma_ ter ) cannot be precsely determned so as to satsfy the condtons requred n step () and (3) of the theoretcal procedure. To avod ths dffculty, the practcal procedure of DSZ algorthm s nstead proposed here. Frst, ntalze k = 2 so that D,we have p D k.[the defnton of D k s n secton 2.]. And c = c,( ma_ ter) s assgned n our research, where c (0<c<) s the shrnkng coeffcent gven n the ntalzaton step of the algorthm, whch s related to the specfc optmzaton obectve functon. k Unfortunately, p D,( ma_ ter) wll be no longer guaranteed by dong so. Concernng to enclose p n search scope as much as possble durng the process of DSZ algorthm, the practcal model of DSZ algorthm s desgned as bellow: () Let = and randomly generate m ponts n D to form the ntal set S : s,,,, L s L s m ; ntalze k =2, the shrnkng coeffcent c (0<c<) and the mamum teraton ma_ter; (2) For each pont s ( m) n S, a correspondng random pont s ' s s generated from D, thus a new set s formed as S ':[ s ', L, s ', L s '] ; k m (3) From S S ', choose m ponts accordng to ther obectve functon values as the new set S +.The mamal value of these chosen ponts' functon values should be smaller than the mnmal value of the rest ponts' functon values; 5
(4) Let k = c + k, (0<c<); (5) Let =+, return to step (2) untl =ma_ter; (6) Choose the soluton 0 from S ma_ter whch mnmzes f() as the output; For certan optmzaton functons, f we can choose values of c, m and ma_ter to hold ma_ ter D 0 p 2c dma () Then 0 p wll be controlled by varyng c ma_ ter, whch meets the need to control the optmzaton precson. 2.5 Epermental Test The effcency of DSZ algorthm has been tested by usng a set of well known functons. It ncludes eght classes of problems: Brann [2, 3, 8-4], Eason [4, 8, 9, 3], Goldsten-Prce [2, 4, 8-3], Shubert [3, 4, 8, 9, -4], Hartmann [, 2, 4, 8, 9, 3], Rosenbrock [, 4, 8, 9, 2, 3], Shekel [, 2, 4, 8-0, 2, 3], and Zakharov [, 4, 8, 9, 3]. Accordng to Hedar and Fukushma [8], the characterstcs of these test functons are dverse enough to cover many knds of dffcultes that arse n global optmzaton problems. Other three functons: rastrgn [8, 0], s hump [8, 5, 6], covlle [5] have also been tested. Snce the global mnma s known for each of these functons, as dd n [, 4, 8, 3], we defne suffcently close by ~ * * < ε + ε 2 f f f Where f * refers to the global mnma of the obectve functon, and ~ f refers to the result acheved by DSZ algorthm, ε and ε 2 are set to 0-4 and 0-6, respectvely. Thus for each eecuton of DSZ algorthm, f the above nequty s satsfed, we say DSZ algorthm s successful for ths eecuton. Table shows the results of the tests, and the functon evolutons "ma_ter m" s lmted wthn 20 5. For each test functon, the success rato ρ s defned as the number of successes out of 00 ndependent eecutons. The optmzaton error s epressed as 6
er = f * ~ f * f ~ f f = *,( 0) f *,( 0) Med(er) n the table stands for the medan of the er set that obtaned from the 00 ndependent eecutons of DSZ algorthm. Table DSZ algorthm epermental results for a set of test functons Functon Dmenson n ρ Med(er) ma_ter Intal sample : m c^ma_ter Brann 2 00.2063e-6 60 0 0^(-4) Rastrgn-2 2 89.908e-7 500 0 0^(-4) Easom 2 99 3.528e-6 00 0 0^(-4) Goldsten-prce 2 00 3.0373e-7 50 0 0^(-4) Shubert 2 99 9.8209e-7 50 0 0^(-4) S-hump 2 00 2.7558e-5 50 0 0^(-4) Colvlle 4 94 3.2967e-8 2000 20 0^(-5) Hartmann-3 3 84 3.0660e-7 300 0 0^(-4) Hartmann-6 6 87 6.2395e-7 300 0 0^(-4) Rosenbrock-2 2 96.48e-9 300 20 0^(-4) Rosenbrock-5 5 Rosenbrock-0 0 Skekel-(4,5) 4 74 4.3520e-7 300 0 0^(-4) Skekel-(4,7) 4 95 3.482e-6 300 0 0^(-4) Skekel(4,0) 4 97 5.0862e-7 300 0 0^(-4) Zakharov-5 5 00 8.0444e-9 300 0 0^(-5) Zakharov-0 0 00 2.5854e-8 000 0 0^(-5) Zakharov-50 50 00.0649e-8 20000 0 0^(-6) Net, the test results are compared wth some selected estng optmzaton algorthms. The comparson results are demonstrated n Table 2. From Table, ecept Rosenbrock-5 and Rosenbrock-0, DSZ algorthm has optmzed the test functons wth success rato of 74% or above. It s also seen from Table 2 that DSZ algorthm s compettve wth respect to optmzaton 7
success rato. It s worth pontng out that accordng to the procedure of the practcal model, the process of generatng new set value calculaton for S ' n the step (2) and functon S ' n the step (3) can easly be acheved through the parallel computng desgn. In that sense, f m parallel computatonal components are appled, the complety of the computatonal tme can be reduced to O( ma _ ter ). Table 2 Comparson between DSZ algorthm and other optmzatons Functon Enhanced Smulated Annealng (ESA) Monte Carlo Smulated Annealng (MCSA) Snffer Global Optmzaton (SGO) Drected Tabu Search (DTS) DSZ algorthm Brann -- 00/557 00/205 00/22 00/600 Easom 82/223 99/000 Goldsten-prce 00/783 99/86 00/664 00/230 00/500 Shubert 92/274 99/500 Hartmann-3 00/698 00/224 99/534 00/438 84/3000 Hartmamnn-6 00/638 62/94 99/760 83/787 87/3000 Rosenbrock-2 00/254 96/6000 Rosenbrock-5 85/684 Rosenbrock-0 85/9037 Shekel-(4,5) 54/487 54/390 90/3695 75/89 74/3000 Shekel(4,7) 54/66 64/342 96/2655 65/82 95/3000 Shekel(4,0) 50/363 8/3078 95/3070 52/828 97/3000 Zakharov-5 00/003 00/3000 Zakharov-0 00/4032 00/0000 Note: */* stands for Optmzaton success number/number of functon evolutons. 8
3 Strategy of Handlng Hgh Dmensonal, Unconstraned problems As mentoned n [5], the unconstraned and bound constraned problems n hgher dmensons are one of challenges n the near future. In practce, when the optmzaton problems are complcated functons wth hgh dmensonal ndependent varables, chose parameters m, c and ma_ter to meet equaton () n secton 2.4 may ntroduce unacceptable computatonal cost. The subsecton wll dscuss a strategy to handle hgh-dmensonal, unconstraned problems subect to partcular condton. 3. Condton on the Hgh-dmensonal, Unconstraned Problem The consdered obectve functon s denoted as f (), :[, L,, L, n ] where ( n) s real number. D s the regon of the ndependent varables, where D: l u, ( l u, n). If p, p 2 are any two random ponts n D ( D,0 < k 2), k and f ( p) < f ( p2), then we have prob[ p p < p p2 ] > 0.5 for any ( n). 3.2 Dstrbuton of the Results Now we study dstrbuton of DSZ algorthm s output for each dmenson of the coordnate 0, ( n) by the predetermned proper value of c, m and ma_ter. Assumng that parameters c, m and ma_ter are chosen for acceptable computatonal cost, after ndependently eecutng DSZ algorthm to optmze the target functon for K tmes, we then get a set of ndependent results: { 0 k n, k K}, where 0 k denotes the eecuton result. th coordnate of the th k tme 9
s dstrbuton n only [ p, u ] ( n) wll be dscussed here, because ts 0 dstrbuton n [ l, p ] s subected to the same manner. Let ma_ ter t = 2 c ( u l ). Evenly dvde [ p, u ] by t nto fnte subntervals. Let q ' ( ' ) be the number of ponts from set { 0 k k K} that falls nto the ' th subnterval. When the algorthm runs nto the mamum teraton, the adacent subntervals q ' and q '+ would have the transtonal relatonshp as shown n Fgure. q q ' ' + prob 2 ' q ' prob ' q ' + Fgure Transton relatonshp between q ', q '+ at the fnal teraton cycle Accordng to the practcal model, the values of set { S } are smaller than ma_ ter or equal to the ones of the set { Sma_ }. Accordng to condton n secton 3., t ter gves prob ' prob2 ' because ples n the frst subnterval of[ p, u ]. When the transton reaches ts equlbrum, we have Thus ' = q ' = r q q, ' >, where prob2 q ' ' + = prob ' r prob2 ' =, 0 r prob <. If r s a constant number equal to r, whch s ndependent of the subnterval where t located, then we have: 0
q ' = r ' q Furthermore, f 0 < r <, and ma_ ter c s set to a suffcently small number, whch also leads to a suffcently small t, the fnal output result n the th coordnate 0 would yeld to the eponental dstrbuton wthn the nterval [ l, u ]. Wth the mamum probablty occurrng at the subnterval contanng p, t has a decreasng negatve eponental dstrbuton along both sdes of ths subnterval. 3.3 Test Eperments n Test functon n ths secton s f ( ) =,0. The optmzaton task s to = determne the mnmum value of f ( ). As to ths functon, condton n secton 3. s satsfed. In ths optmzaton, the dmenson of s 3 n = 0. The algorthm parameters 3 / 40000 are set as ma_ter=4000, m= and the shrnkng constant c= 0. When the algorthm runs nto the last teraton, the varable subntervals are scaled to the magntude of c ma_ ter = 3 0. After all the nput parameters are set up, ndependently runnng 0 tmes gves a 0 000 output matr M. Each row of M refers to the result of one ndependent eecuton of DSZ algorthm. Because of the specalty of the functon f ( ) consdered n ths paper, all elements n M can be regarded as the output of any dmenson 0 ( n) after ndependently runnng the DSZ algorthm for 0 000 tmes. Hence, hstograms for each dmenson of the varables are shown n Fgure 2 by usng all the elements of M. Fgure 3 shows the plot of all the elements fell nto nterval [0, 0.], and the length of ts subnterval s 3 0. Fgure 3 ndcates the maorty of ponts fall nto the regon of [0, 0.02]. Thus f we zoom n ths regon, the correspondng result s shown n Fgure 4. The logarthmc curve of Fgure 4 s shown n Fgure 5. The smooth trend of curve n Fgure 5 dscloses that the dagram n Fgure 3 appromates an eponental dstrbuton, as what has been derved n the prevous secton.
Fgure 2 The hstogram for all the elements of M Fgure 3 Hstogram of all the elements fell nto nterval [0, 0.] Fgure 4 Hstogram of all the elements fell nto nterval [0, 0.02] Fgure 5 The logarthm curve of fgure 4. as: subnterval number; y as: log value of the number of ponts fell n each subnterval 3.4 Strategy Accordng to the dstrbuton of the output results n each coordnate 0 ( n) of DSZ algorthm dscussed above, we descrbe the strategy of usng DSZ algorthm to optmze certan hgh-dmensonal, unconstraned problems as followng: Intalze parameters c, m and ma_ter at acceptable computatonal cost. Run DSZ algorthms to optmze the obectve functon ndependently for K tmes. Eamne the hstograms of the output soluton set at each dmenson { 0 k k K} ( n ).Use the nformaton hstograms provde (the subnterval n whch maorty of set { 0 k k K} ( n ) locate) to narrow the nterval, n whch p ( n) locates, hence narrow the regon D p where 2
p locates. Obvously, p Dp D. Use Dp as the new regon for the varables: small. new D D p. Repeat the above steps untl D p becomes suffcently 4 Applcatons n the Constraned Optmzaton Contet 4. Condton on the optmzaton problem The optmzaton target s functon f (), :[, L,, L, n ] where ( n) s real number. Defne D: l u, ( l u, n) as the regon for varable. E D s the constraned regon on whch s nonempty. The target functon f () also satsfes the three condtons descrbed n secton 2., namely, () Contnuty; (2) Convergence condton I; (3) Convergence condton II. Wthout losng generalty, the task s to look for the global mnma of f (). The correspondng soluton for the global mnma s p :[ p, L, p L, p ], ( n)., n 4.2 Optmzaton Methodology For the constraned optmzaton problems, DSZ algorthm employs almost the same approach as done for the unconstraned optmzaton. In the constraned optmzaton contet, frstly, randomly generate N legal ponts n E by evenly samplng sz ponts n D, the set of legal ponts s denoted as S. In ths way, on the one hand, S s randomly dstrbuted n the regon E; on the other hand, the rato of volume between space E and D could be estmated by N/sz. Thus, f ntalze k as N k = sz / n (n s the dmenson of varable ), the entre E space can be epected to covered by usng the ntal random set U D. k S After that, the same method of processng the unconstraned optmzaton s 3
adopted. One thng noted s that llegal ponts should be removed once they appear durng the process. 4.3 Procedure n the Constraned Optmzaton Contet () Let =, and randomly generate m legal ponts by samplng sz ponts randomly n regon D to form the ntal set S : s,,,, L s L s m, thus S E. Intalze k m = sz teraton ma_ter; / n, shrnkng coeffcent c (0<c<), and the mamum (2) For each pont s,( m) n S, a correspondng random pont s ' s s generated from D, thus new set s formed as S ':[ s ', L, s ', L sm '] ; k (3) From set S S ', choose m legal ponts accordng to ther obectve functon values as the new set S +. The mamal value of these chosen m ponts' functon values should smaller than the mnmal value of the rest legal ponts functon values; (4) Let k = + c k (0<c<); (5) Let =+, return step (2) untl =ma_ter; (6) Choose the soluton from S ma_ ter whch yelds f ( ) mnma as the output. 4.4 Test Results In ths secton, DSZ algorthm s appled to test cases from reference [6], as shown n Table 3. Table 3 Summary of test case Functon N Type of f ρ LI NE NI A Mn G 3 Quadratc 0.0% 9 0 0 6 Ma G2 K Nonlnear 99.8474% 0 0 2 Ma G3 K Polynomal 0.0000% 0 0 Mn G4 5 Quadratc 52.230% 0 0 6 2 Mn G5 4 Cubc 0.0000% 2 3 0 3 4
Mn G6 2 Cubc 0.0066% 0 0 2 2 Mn G7 0 Quadratc 0.0003% 3 0 5 6 Ma G8 2 Nonlnear 0.8560% 0 0 2 0 Mn G9 7 Polynomal 0.552% 0 0 4 2 Mn G0 8 Lnear 0.000% 3 0 3 6 Mn G 2 Quadratc 0.0000% 0 0 Note: LI-lnear nequaltes, NE-nonlnear Equaltes, NI-Nonlnear nequaltes, A-actve constrants and ρ - feasblty; for both G2 and G3, k=50. For each test case we lst number of varables n, type of the functon f, the relatve sze of the feasble regon n the search space gven by ρ, the number of constrants of each categores such as LI (lnear nequaltes), NE (nonlnear equatons) and NI (nonlnear nequaltes). The feasblty ρ s determned epermentally n reference [7] by calculatng the percentage of feasble solutons among the,000,000 randomly generated ndvduals. Table 5 and Table 6 are the comparsons between DSZ algorthm and genetc algorthm [6, 7]. The ndependent varables of functon G2 and G3 are 20 dmensons and 0 dmensons, respectvely. Parameters of DSZ algorthm are shown n Table 4. Table 4 Parameters of DSZ algorthm Functon n m ma_ter G 3 30 000 G2 20 00 000 G3 0 20 00 G4 5 20 000 ma_ ter c 0 5 0 3 0 3 0 3 G5 4 G6 2 5 500 G7 0 20 000 G8 2 0 500 G9 7 20 000 G0 8 50 000 0 3 0 3 0 3 0 3 0 3 5
G 2 5 20 0 5 Note: n - dmensons of functon s ndependent varables; m-the populaton sze of the ponts set; ma_ter- the mamum teratons; c- the shrnkng coeffcent. Table 5 Test Results () Functon Optmum value GA n[6] (Eperment #3) DSZ algorthm Worst Best Average Worst Best Average G -5-4.5732-4.784-4.6478-4.9606-4.9999-4.9609 G2 0.803553 0.78279 0.79486 0.78722 0.75686 0.80339 0.7867 G3.0 0.9960 0.9978 0.9970.0000.0000.0000 G4-30665.5-30645.6-3066.5-30653. -30664.7-30665. -30665.0 G5 526.498 G6-696.8-6390.6-6944.4-6720.4-696.4-696.7-696.6 G7 24.306 26.82 25.090 25.545 24.702 24.40 24.493 G8 0.095828* 0.0958246 0.0958250 0.0958248 0.05459 0.05460 0.05459 G9 680.63 683.58 68.72 682.56 680.99 680.79 680.85 G0 7049.33 7685.8 732.2 7498.6 7297.5 7053.4 706.3 G 0.75 0.75 0.75 0.75 0.75 0.75 0.75 Note: Independently run the genetc algorthm n [6] and DSZ algorthm 0 tmes, respectvely; for the genetc algorthm, the mamum number of generatons s 5,000, the populaton sze s 70; for G3, k=0 and for G2, k=20. Parameters of DSZ algorthm are shown n Table 4. Table 6 Test Results (2) Functon Optmum value GA n [7] DSZ algorthm Worst Best Medan Worst Best Medan G -5 -.9999-4.9999-4.9997-4.605-4.9999-4.9950 G2 0.803553 0.67269 0.80390 0.755332 0.72074 0.80347 0.785 G3.0 0.785582.00009 0.94899.0000.0000.0000 G4-30665.5-30652.0-30665.5-30663.4-30664.7-30665.2-30665.0 G5 526.498 62.22 526.5 572.53 G6-696.8-6954.32-696.78-6959.57-696.4-696.8-696.7 G7 24.306 35.8820 24.40 26.7357 24.94 24.340 24.448 6
G8 0.095828* 0.0958246 0.0958250 0.0958248 0.05459 0.05460 0.05460 G9 680.63 684.3 680.762 68.706 68.5 680.79 680.83 G0 7049.33 2097.4 7060.55 7723.7 7297.5 7053.4 7070.4 G 0.75 0.8094 0.7490 0.7493 0.75 0.75 0.75 Note: Run the genetc algorthm n [7] and DSZ algorthm 50 tmes ndependently; for the genetc algorthm, the mamum number of generatons s 5,000, the populaton sze s 0; for G3, k=0 and for G2, k=20. Parameters of DSZ algorthm are shown n Table 4. The test results clearly ndcate that by usng less than /5 of the teratons of genetc algorthm, DSZ algorthm s capable of determnng better results than GA. It should be ponted out that DSZ algorthm has found a new soluton = [.227800729835, 3.7448865967657] for G8, whch gves G8() =0.05460>0.095828. Besdes, when the constrants contan equaltes, DSZ algorthm handles them by transformng them nto a format of nequaltes or bound constraned problems. Take G3 for an eample: n n G3:Mamze G3( ) = ( n), where 2 =,0, for( n). G3 could be transformed nto an equvalent form as followng: = n n n 2 MamzeG3( ) = ( n), where 2,0, for( n ). = = In ths way, the n-dmensonal constraned optmzaton problem wth equaltes s transformed nto the (n-)-dmensonal constraned optmzaton problem wth only nequaltes. Take G for another eample. 2 2 2 G:Mamze G( ) = + ( 2 ), where 2 = 0,,( =, 2). G s equvalent to the problem below: n = 2 2 2 Mamze G( ) = + ( ), where. The above eamples show that the two-dmensonal problem constraned by equaltes can be transformed nto the one-dmensonal unconstraned problem. If constrants contan multple nonlnear equaltes e.g. G5, the nonlnear equaton n = 7
toolkt shall be added to current verson of DSZ algorthm so as to transform equalty constrants nto ether nequalty constrants or unconstraned condtons. 5 Summary In the present paper a smple stochastc global optmzaton method, DSZ algorthm, has been proposed based on Convergence Condtons I and Convergence Condtons II, whch are generally gnored by prevous researchers. The epermental results demonstrate that DSZ algorthm s effectve and capable of handlng both unconstraned and constraned optmzaton problems. As to DSZ algorthm s prncple, we demonstrated ts valdty based on ts theoretcal model. For the practcal model of DSZ algorthm, ts valdty and effcency wll need further nvestgaton when appled to more complcated obectve functons. Therefore, future work wll contnue on: () Mechansm on practcal DSZ algorthm and what condtons the obectve functon should meet to guarantee the effcency of ts optmzaton; (2) In ths paper we proposed a strategy of handlng the hgh-dmensonal optmzaton problem by usng the hstogram nformaton of DSZ algorthm outputs to narrow the optmal regon for p. The effcency of ths strategy needs to be testfed by more hgh-dmensonal and complcated cases; (3) Although the current verson of DSZ algorthm s capable to handle scenaros wth nequalty constrants, n order to cover the equalty and nequalty med condtons, a toolkt of equaton solver would need to be ncluded; (4) DSZ algorthm s also epected to be appled to other optmzaton ssues, e.g. dscrete varable optmzaton and combnaton optmzaton n future. Acknowledgement We are thankful to Professor Wael H. Abulshohoud n Jln Unversty- Lambton College (prevous wth The George Washngton Unversty) for hs nsghtful comments and suggestons. 8
6 References [] P. Sarry and F. Durdn, "Enhanced smulated annealng for globally mnmzng functons of many-contnuous varables, ACM Transactons on Mathematcal Software (TOMS), vol. 23, pp. 209-228, 997. [2] D. VANDERBILT and S. LOUIE, "A Monte Carlo smulated annealng approach to optmzaton over contnuous varables, Journal of Computatonal Physcs (Prnt), vol. 56, pp. 259-27, 984. [3] R. R. Butler and E. Slamnka, "An evaluaton of the snffer global optmzaton algorthm usng standard test functons," Journal of Computatonal Physcs, vol. 99, pp. 28-32, 3. 992. [4] A. R. Hedar and M. Fukushma, "Hybrd Smulated Annealng and Drect Search Method for Nonlnear Unconstraned Global Optmzaton, Optm. Methods Software, vol. 7, pp. 89-92, 2002. [5] A. Neumaer, "Complete search n contnuous global optmzaton and constrant satsfacton, Acta Numerca, vol. 3, pp. 27-369, 2004. [6] S. Kozel and Z. Mchalewcz, "Evolutonary algorthms, homomorphous mappngs, and constraned parameter optmzaton, Evl. Compute, vol. 7, pp. 9-44, Sprng. 999. [7] S. Venkatraman and G. Yen, "A Generc Framework for Constraned Optmzaton Usng Genetc Algorthms, Evolutonary Computaton, IEEE Transactons on, vol. 9, pp. 424-435, 2005. [8] A. R. Hedar and M. Fukushma, "Tabu Search drected by drect search methods for nonlnear global optmzaton star, open, Eur. J. Oper. Res., vol. 70, pp. 329-349, 2006. [9] M. J. Hrsch, C. N. Meneses, P. M. Pardalos and M. G. C. Resende, "Global optmzaton by contnuous grasp, Optmzaton Letters, vol., pp. 20-22, 2007. [0] J. Barhen, "TRUST: A Determnstc Algorthm for Global Optmzaton, Scence, vol. 276, pp. 094-097, 997. [] D. Cvovc and J. Klnowsk, "Taboo Search: An Approach to the Multple Mnma Problem, Scence, vol. 267, pp. 664-666, 995. [2] T. B. Trafals and S. Kasap, "A novel metaheurstcs approach for contnuous global optmzaton, J. Global Optmz., vol. 23, pp. 7-90, 2002. [3] A. Hedar and M. Fukushma, "Mnmzng multmodal functons by smple codng genetc algorthm, Optm. Methods Software, vol. 8, pp. 265 282, 2003. [4] G. H. Koon and A. V. Sebald, "Some nterestng test functons for evaluatng evolutonary programmng strategy," Proceedngs of the Fourth Annual Conference on Evolutonary Programmng, pp. 479 499, 995. [5] Z. Mchalewcz, Genetc Algorthms Data Structures= Evoluton Programs. Sprnger, 996, [6] R. Chelouah and P. Sarry, "A Contnuous Genetc Algorthm Desgned for the Global Optmzaton of Multmodal Functons, J. Heurstcs, vol. 6, pp. 9-23, 2000. 9
[7] Z. Mchalewcz and G. Nazhyath, "Genocop III: a co-evolutonary algorthm for numercal optmzatonproblems wth nonlnear constrants, Evolutonary Computaton, 995, IEEE Internatonal Conference on, vol. 2. 20