A PENALTY-CONJUGATE GRADIENT ALGORITHM FOR SOLVING CONSTRAINED OPTIMIZATION PROBLEMS

Transcription

1 IJRRAS 35 () April 8 wwwarpapresscom/volumes/vol35issue/ijrras_35 pdf A PENALY-CONJUGAE GRADIEN ALGORIHM FOR SOLVING CONSRAINED OPIMIZAION PROBLEMS Hoia Zha Zhuohua Lia & Weli Zhao School of Mathematics ad StatisticsShado Uiversity of echoloy Zibo Shado Chia @63com @63com zwlsdj@63com Correspodi Author zwlsdj@63com ABSRAC his paper presets a ew alorithm for costraied optimizatio problems it is called pealty-cojuate radiet method he method apply oliear cojuate radiet method to pealty fuctio by choosi a cojuate factor with ood umerical performace ad coverece ad selecti the search directio p with the descet property he alorithm has lobal coverece uder the Wolfe lie search Key Words: pealty-cojuate radiet method lobal coverece pealty fuctio INRODUCION Costraied optimizatio eist i may fields such as optimal cotrol parameter estimatio Nash equilibrium ad sesitivity aalysis I order to solve costraied optimizatio problems we usually eed to deal with a lare umber of state variables cotrol variables or costraied variables herefore it is very ecessary to fid out efficiet umerical solutios quicly For the process of fidi the solutios of costraied optimizatio problem we ca divide it ito two steps First the cotrol vector ca be parameterized to mae the oriial problem become a oliear ucostraied optimizatio problem he the eeral ucostraied optimizatio alorithms are used to solve this problem I these alorithms the cojuate radiet method is a id of practical method with ood properties simple structure ad less computatio Fletcher ad Reeves proposed a cojuate radiet method [] for solvi ucostraied miimizatio problems ad it is used to solve lare-scale problems I the literature [] AI-Baali proved that the Fletcher-Reeves method had lobal coverece Literature [3] eeralized the above coclusios to the case of stro Wolfe lie search However there were always a lot of small steps so the FR method sometimes performed poorly i umerical calculatios he PRP method proposed by Polar-Ribiere-Polyar ad the HS method proposed by Hestece-Stiefel were two ids of cojuate radiet alorithm with ood umerical performace I [4] Powell poited out that there may be o lobal coverece eve the eact lie was used to search for the Polar-Ribiere- Polyar method But this method had ood umerical performace(see [5])Subsequetly may scholars had doe a more profoud study of the lobal coverece of this id of alorithm he DY method was proposed by Dai YH ad Yua YX was studied detailedly It had lobal coverece ad itrisic properties I literature[6] Dai systematically itroduced lobal coverece of the DY- method uder eeral lie search Adrei had cosidered the ood umerical performace of HS method ad the ood coverece of DY- method he hybrid DY-HS cojuate radiet method which was proposed i literature[78] had ood coverece but it ca t uaratee the descet of search directio I recet years a lot of proress has bee made i the research of various cojuate radiet methods [4 9] Narushima [] ad Xu Do [3] put forward two ids of cojuate radiet methods with sufficiet descet respectively such that the search directio has a descet property ad is ot affected by the cojuate radiet factor Zhao ad Yao [45] proposed the eometric oliear cojuate radiet method ad Riema Fletcher-Reeves cojuate radiet method to solve the stochastic eievalue problem i literature respectively Liu ad Di propose two ew descet cojuate radiet alorithms i the literature [67] was used to solve cove costraied mootoe equatios O the basis of the eisti research a ew pealty-cojuate radiet alorithm is preseted i this paper It combies oliear cojuate radiet method with pealty fuctio he structure of search directio p ca esure every search directio is descet ad the cojuate radiet factor we choose ca esure the

2 IJRRAS 35 () April 8 Zha et al A Pealty-Cojuate Gradiet Alorithm alorithm has a ood coverece Ad we prove that the ew alorithm has lobal coverece uder the Wolfe lie search At the ed of the paper two umerical results have bee ive PENALY-CONJUGAE GRADIEN ALGORIHM Now we cosider the followi iequality costraied optimizatio problems: where f ci are cotiuous differetiable cove fuctios Let mi f s t c i m i R m mi f ma c i i () () where is a pealty factor Moreover the lobal optimal solutio of the problem () is the optimal solutio of the problem () whe it satisfies certai coditios For coveiecelet c i D ma ( ) s Defiitio It is said that the quadratic cotiuous differetiable fuctio : is a uiform cove R R fuctio (a stro cove fuctio) If the eistece of for y R the followi iequalities are established: y y y y R From the literature [8] it is ow that the above formula is equivalet to y y y y y R (4) Form (3) (4) we have the coclusio ad Net the pealty-cojuate radiet alorithm is ive (3) s y s (5) s s (6) Alorithm : Step Select the iitial poit R ive the termiatio error let p :

3 IJRRAS 35 () April 8 Zha et al A Pealty-Cojuate Gradiet Alorithm Step Calculate p : s p ( ) s (7) where y s s y s y s (8) Step 3 Calculate the step size where Step 4 Let p If by Wolfe lie search coditio p p p p p calculate radiet the stop otherwise o bac to step 3 HE CONVERGENCE ANALYSIS OF HE ALGORIHM Now we prove that p is the descet directio Accordi to the property of the descet directio we oly eed to verify p that is s p ( ) s therefore p is the descet directio Lemma 3([]) Suppose is the iitial poit ad p Where p is obtaied by the alorithm ad satisfies the Wolfe lie search coditio the p (3) p From (7) we ca et p the (3) is equivalet to the followi formula p 4 3

4 IJRRAS 35 () April 8 Zha et al A Pealty-Cojuate Gradiet Alorithm I order to esure the ood coverece of the alorithm ad prove the coverece of the alorithm we eed to assume several coditios Assumptio he level set R is bouded where Assumptio I a cove eihborhood N of the level set satisfies the Lipschitz coditio that is the eistece costat Assumptio 3 Assume that fuctio : R R is the iitial poit is cotiuously differetiable the radiet L satisfies y L y y N (3) is a uiformly cove fuctio heorem 3 Suppose the objective fuctio satisfies the Assumptio ad 3 the we have lim or limif Proof Assume that for ay there is Because p the accordi to is eerated by the alorithm is a uiform cove fuctio we ca obtai s y s s (33) is the descet directio there is p From (6) (8) ad ( 3 ) we ca obtai y s s y s y s s y s y s s s L s s s s L s that is herefore form (7) we ca obtai L s 4

5 IJRRAS 35 () April 8 Zha et al A Pealty-Cojuate Gradiet Alorithm s p s L L L Let L the the above formula is equivalet to p herefore form Lemma 3 we ca obtai 4 p So limif Net we prove that the lobal optimal solutio of fuctio f ( ) heorem33 is the lobal optimal solutio of objective fuctio is the lobal optimal solutio of the costraied problem () he pealty factor if is a sequece of solutios of fuctio () the ay cluster poit ~ the costraied problem Proof Suppose ~ be a cluster poit of suppose that the coverece sequece is ca obtai Sice is a feasible poit for the costraied problem therefore ad of must be the lobal optimal solutio of the there is the coverece subsequece coveres to ~ Ad we may Accordi to ~ is the lobal optimal solutio of fuctio we D ( ) % ( ) ( ) f ( ) D( ) (34) % ( ) f ( ) (35) that is 5

6 IJRRAS 35 () April 8 Zha et al A Pealty-Cojuate Gradiet Alorithm Whe Net let s prove that ( 36 ) meas ~ D ( %) so ~ is a feasible poit f % D % f ( ) ( ) ( ) is the lobal optimal solutio Accordi to (36) ad D ( ) (36) we ca obtai f ( % ) f ( ) whe O the other had because %is feasible solutio ad is lobal optimal solutio the there are f ( ) f ( % ) therefore we ca obtai f ( % ) f ( ) 4 NUMERICAL RESULS Net we ive two eamples of the above alorithm Eample Accordi to () there is We tae the iitial poit we ca obtai 4 4 mi f 4 st (4) 4 ma (4) as (- ) he parameters are as follows: Because ad the above parameters we ca mae 5 fially et 6 so there is 4 p 4 4 Because optimal solutio () of (4) It's clear that () is also the optimal solutio for (4) Eample 3 mi f st We tae the iitial poit as (- 4) he parameters 3 MALAB4a to solve the above fuctio the result is as follows: By the Wolfe step rule 6 5 able so we et the 4 We use 6

7 IJRRAS 35 () April 8 Zha et al A Pealty-Cojuate Gradiet Alorithm Solutio sequece Iterative directio Gradiet Step size (-4) (-6) ( ) 94 (5935) (-3-77) (35376) 7 () () () () 5 ACKNOWLEDGEMENS his research was supported by Natioal Natural Sciece Foudatios of Chia (No 7755) ad Natural Sciece Foudatio of Shado Provice (ZR6AM7 ZR5AL) 5 REFERENCES [] Fletcher R Reeves Fuctio miimizatio by cojuate radiet[j] Computer Joural 964 7(): [] Al-Baali M Descet property ad lobal coverece of the Fletcher-Reeves method with ieact lie searches[j] Joural of Numerical Aalysis 9855(): -4 [3] Byrd R H Nocedal J Yua Y X Global coverece of a class of Quasi-Newto methods o cove problems[j] Siam Joural o Numerical Aalysis 987 4(5):7-9 [4] Kou CX A improved oliear cojuate radiet method with a optimal property[j] Sciece Chia Mathematics 4 57(3): [5] Jore N Stephe J Wriht Numerical optimizatio[m] Sprier 6 [6] Dai Y Yua Y Noliear cojuate radiet method[m] Sciece ad echoloy Press of Shahai [7] Adrei N Aother hybrid cojuate radiet alorithm for ucostraied Optimizatio[J] Numerical Alorithms 8 47: [8] Adrei N Accelerated hybrid cojuate radiet alorithm with modified secat coditio for ucostraied optimizatio[j] Numerical Alorithms 54(): 3-46 [9] Yao S He D Shi L A improved perry cojuate radiet method with adaptive parameter choice[j] Numerical Alorithms 7(3): -5 [] Babaie-Kafai S A ote o the lobal coverece of the quadratic hybridizatio of Pola Ribière Polya ad Fletcher Reeves cojuate radiet methods[j] Joural of Optimizatio heory & Applicatios 3 57(): [] Al-Bayati AY Al-Kawaz RZ A ew hybrid WC-FR cojuate radiet-alorithm with modified secat coditio for ucostraied optimizatio[j] JMath Comput Sci (4): [] Narushima Y Yabe H Ford J A A three-term cojuate radiet method with sufficiet descet property for ucostraied optimizatio[j] Computatioal Optimizatio & Applicatios 4 6(): 89- [3] Do X He Y A ew class of cojuate radiet methods with sufficiet descet coditio ad stro coverece[j] Joural of Mathematics 7 37(): 3-38 [4] Zhao Z Ji XQ Bai ZJ A eometric oliear cojuate radiet method for stochastic iverse eievalue problems[j] Siam Joural o Numerical Aalysis 6 54(4): 5-35 [5] Yao Bai ZJ Zhao Z et al A Riemaia Fletcher--Reeves cojuate radiet method for doubly stochastic iverse eievalue problems[j] 6 37(): 5-34 [6] Liu SY Hua YY Jiao HW Sufficiet descet cojuate radiet methods for solvi cove costraied oliear mootoe equatios[j] Abstract & Applied Aalysis 4 4(): - [7] Di Y Xiao Y Li J A class of cojuate radiet methods for cove costraied mootoe equatios[j] Optimizatio 7(): - [8] Wa YJ Xiu N H Noliear optimizatio theory ad alorithm[m] Sciece Publishi Compay Chia 7