International Journal of All Researh Euation an Sientifi Methos IJARESM ISSN: 55-6 Volume Issue 5 May-0 Extene Spetral Nonlinear Conjuate Graient methos for solvin unonstraine problems Dr Basim A Hassan Asst Professor Collee of Computers Sienes an Math University of Mosul Iraq ABSRAC In this paper we present extension forms of Dai Yuan DY Flether Reveres FR an Conjuate Desent CD CG alorithms he extene metho have the suffiient esent an lobally onverene properties uner ertain onitions hese new alorithms are teste on some stanar test funtions an ompare with the oriinal FR alorithm showin onsierable improvements over all these omparisons Keywors: Conjuate raient metho Spetral Conjuate raient metho suffiient esent property lobal onverent methos INRODUCION he nonlinear onjuate raient CG metho is esine to solve the followin unonstraine optimization problem n min f x x R where f : R n R is a ontinuously ifferentiable nonlinear funtion whose raient is enote by Due to its simpliity an its very low memory requirement the CG metho has playe a speial role for solvin lare sale nonlinear optimization problems he iterative formula of the CG metho is iven by x x where 0 is a step lenth whih is ompute by arryin out a line searh an satisfies the stanar Wolfe SW onitions : f x 3 f x x with 0 an is the searh iretion efine by where parameter 5 is a esent iretion Different onjuate raient alorithms orrespon to ifferent hoies for the salar see [7] he well-nown formula of are iven by FR 6 y CD 7 DY 8a whih are alle Flether an Reeves FR [] Conjuate Desent CD [3] an Dai an Yuan DY [] respetively In fat utilizin 5 an be rewritten as : DY Pae 0
International Journal of All Researh Euation an Sientifi Methos IJARESM ISSN: 55-6 Volume Issue 5 May-0 DY 8b In [6] moifie onjuate raient methos are iven by the rule 9 where is one of the values in 6 or 7 or 8 Zhan an Wan [8] propose a eneral form of onjuate raient methos are iven by the rule y 0 where is one of the values in 6 or 7 or 8 he paper is oranize as follows In setion is the introution In setion we present the extene spetral metho an alorithm Setion 3 show that the searh iretion enerate by this propose alorithm at eah iteration satisfies the suffiient esent onition an establishes the lobal onverene analysis Setion establishes some numerial results to show the effetiveness of the propose CG-metho an Setion 5 ives a brief onlusions Extene spetral onjuate raient metho an alorithm In this paper we suest a new type of spetral onjuate raient methos for solution of the min f x In [5] we onsier a onition that a esent searh iretion is enerate an we exten the DY metho We mae suh a iretion inutively Suppose that the urrent searh iretion is a esent iretion namely 0 at the th iteration Now we nee to fin a that proues a esent searh iretion his requires that Lettin be a positive parameter we efine Equation is equivalent to 3 ain the positively of in to onsieration we have max 0 herefore if onition is satisfie for all the onjuate raient metho with proues a esent searh iretion at every iteration From we an et various ins of onjuate raient metho by hoosin various Hieai an Yasushi propose a new onjuate raient metho whih was obtaine by moifyin the DY metho an alle MDY metho A nie property of the MDY metho is that it enerates suffiient esent iretions he parameter in MDY metho is iven by where he efinition of searh iretion an Formula MDY 5 f f 6 5 ensure that Pae
International Journal of All Researh Euation an Sientifi Methos IJARESM ISSN: 55-6 Volume Issue 5 May-0 an hene MDY MDY 7 0 8 performs more effetive More etails an be foun in [5] Let us try to erive a new type metho we nee the next iretion this we have for any 0 ] an from 3 the followin inequality hols : ie to be esent Assume that 0 9 0 0 Now we an rewrite the above inequality as 0 Hene we obtain our new iretions as follows : hen we an rewrite as where 3 his metho inlues the Zhan an Wan ZW metho as a speial ase By settin y iretion reues to the Zhan an Wan ZW metho whih efine in 0 Now we an obtain the a new onjuate raient alorithms as follows : he New Alorithm n x R Step Initialization : Selet an the parameters 0 an set the initial uess / Step est for ontinuation of iterations If Compute x 0 6 then stop else step3 f an Step 3 Line searh : Compute 0 satisfyin the Wolfe line searh onition 6 an upate the variables x x Step Conjuate raient parameter whih efine in 6 or 7 or 8 Step 5 Diretion omputation whih efine in If the restart riterion of Powell 0 is satisfie then set otherwise efine GLOBAL CONVERGENCE Consier In this setion we establish onverene of the propose metho the followin assumptions for the objetive funtion are neee Assumption 3 By Pae
International Journal of All Researh Euation an Sientifi Methos IJARESM ISSN: 55-6 Volume Issue 5 May-0 n i- he level set L x R f x f x is boune 0 ii- In some neihborhoo U of f x L is ontinuously ifferentiable an its raient is Lipshitz ontinuous namely there exists a onstant 0 suh that x x L x x x x U 5 Assumption 3 imply that there exists a positive onstant suh that Here we have to present suffiient esent property heorem Let x an of the suffiient esent property x U 6 be enerate by an where satisfies Wolfe line searh onitions then hols Proof : Suppose that 7 0 0 0 hen onlusion an be prove by inution When 0 we have 0 From 3 an we have 8 9 where hus the theorem is prove he followin Lemma [9] is the result for eneral iterative methos : Lemma Suppose that Assumption is satisfie an onsier any metho with Eq where satisfies Eqs an hen i 30 From the previous analysis we an et the followin lobal onverene result for new Alorithm heorem Suppose that Assumption 3 hols an these methos have the satisfies suffiient esent onition with hen these metho are lobally onverent one has lim inf 0 or 3 Proof : Now we will prove lobal onverene We suppose that the theorem is not true Suppose by ontraition that there exists 0 suh that Pae 3
International Journal of All Researh Euation an Sientifi Methos IJARESM ISSN: 55-6 Volume Issue 5 May-0 Pae 0 0 an 3 By squarin the two sies of an transferrin an trimmin we et : 33 Diviin the previous in equation by we et : 3 35 36 37 a When DY hen by 8 b DY 38 an applyin 37 we have 39 Notin that 0 With this from 3 we have i i hen we et
International Journal of All Researh Euation an Sientifi Methos IJARESM ISSN: 55-6 Volume Issue 5 May-0 Pae 5 Whih iniates 3 his is a ontraition to the 30 b When FR hen from 37 an suffiient esent onition with 5 we also have When CD hen from 37 an suffiient esent onition with 6 we also have whih ontraits Lemma herefore we et this theorem NUMERICAL RESULS In this setion we will report the numerial performane of Alorithm We test Alorithm by solvin the 5 benhmar problems from [] an ompare its numerial performane with that of the other similar metho whih inlue the stanar FR onjuate raient metho in [3] All oes of the omputer proeures are written in Fortran he parameters are hosen as follows : 09 000 05 0 6 In ables an we use the followin enotations : n : the imension of the objetive funtion NOI : the number of iterations NOF : the number of funtion evaluations FR : the stanar FR onjuate raient metho in [3] SFR : the new spetral FR metho presente in this paper SDY : the new spetral DY metho presente in this paper SCD : the new spetral CD metho presente in this paper
International Journal of All Researh Euation an Sientifi Methos IJARESM ISSN: 55-6 Volume Issue 5 May-0 F : If NOF or NOI exeee 000 then enote F able Comparison of the alorithms for n 00 est problems FR SFR SDY SCD NOI NOF NOI NOF NOI NOF NOI NOF rionometri 0 0 36 39 39 Extene Rosenbro 9 98 53 0 56 3 58 5 Extene White & Holst 69 5 5 90 6 7 69 3 Penalty 7 00 3 33 30 3 3 Generalize riiaonal 6 90 5 6 8 8 ExtenehreeExpo erms 3 33 3 9 30 Generalize riiaonal 05 9 69 6 73 5 75 Diaonal 33 5 7 3 8 8 Extene Himmelblau 6 3 5 8 3 5 8 Broyen riiaonal 5 7 3 5 35 56 35 53 DENSCHNA CUE 3 5 3 36 3 35 DENSCHNF CUE 39 3 0 9 33 38 Extene Blo-Diaonal 8 3 3 6 3 6 Generalize quarti GQ 0 9 0 9 0 9 0 Generalize quarti GQ 50 8 39 69 8 76 5 86 otal 603 3038 386 70 799 65 85 est problems able Comparison of the alorithms for n 000 FR SFR SDY SCD NOI NOF NOI NOF NOI NOF NOI NOF rionometri 67 33 58 3 57 70 Extene Rosenbro 60 3 65 6 6 Extene White & Holst 37 99 58 5 7 36 7 9 Penalty 6 09 6 3 30 3 37 Generalize riiaonal 3 390 30 57 58 993 39 376 ExtenehreeExpo erms 38 583 9 809 Generalize riiaonal 3 7 9 78 6 5 79 Diaonal 3 50 7 3 7 5 7 5 Extene Himmelblau 37 6 30 6 3 Broyen riiaonal 7 78 69 68 0 67 DENSCHNA CUE 58 0 8 30 9 3 7 9 DENSCHNF CUE 39 8 3 9 3 Extene Blo-Diaonal 3 50 5 8 3 6 3 6 Generalize quarti GQ 8 0 8 0 8 0 8 0 Generalize quarti GQ 5 83 39 63 6 75 50 8 otal 7 665 396 73 7 7 387 009 From the above numerial experiments it is shown that the new alorithms in this paper is promisin CONCLUSIONS AND DISCUSSIONS In this paper a new spetral onjuate raient alorithm has been evelope for solvin unonstraine minimization problems Uner some mil onitions the lobal onverene has been prove Compare with the other similar alorithm the numerial performane of the evelope alorithm is promisin able 5 ives a omparison between the new-alorithm an the Flether an Reeves FR-alorithm for onvex optimization this table iniates that the new alorithm saves 58 66% NOI an 30 3% NOF overall aainst the stanar Flether an Reeves FR-alorithm espeially for our selete test problems Relative Effiieny of the Different Methos Disusse in the Paper Pae 6
International Journal of All Researh Euation an Sientifi Methos IJARESM ISSN: 55-6 Volume Issue 5 May-0 ools NOI NOF FR- alorithm 00 % 00 % SFR- alorithm 85 % 6953 % SDY- alorithm 3397 % 350 % SCD- alorithm 3665 % 600 % REFERENCES [] Anrie N 008 ' An Unonstraine Optimization est funtions olletion' Avane Moelin an optimization0 pp7-6 [] Dai Y an Yuan Y 999 ' A nonlinear onjuate raients metho with a stron lobal onverene property' SIAM J Optimization 0 pp77-8 [3] Flether R 987 ' Pratial Methos of Optimization seon eition John Wiley an Sons New Yor [] Flether R an Reeves C 96 ' Funtion minimization by onjuate raients ' Computer J 7 pp9-5 [5] Hieai I Yasushi N 0 Conjuate raient methos usin value of objetive funtion for unonstraine optimization Optimization Letters V6 Issue 5 9-955 [6] Matonoha C Lusan L an Vle J 008 ' Computational experiene with onjuate raient methos for unonstraine optimization ' ehnial report no038 pp-7 [7] Yabe H an M aano 00 Global onverene properties of nonlinear onjuate raient methos with moifie seant onition Comput Optim Appl 8 03 5 [8] Zhan Y an Wan K 0 ' A new eneral form of onjuate raient methos with uarantee esent an stron lobal onverene properties' Numer Alor 60 pp35 5 [9] Zoutenij G : Nonlinear prorammin In: Abaie J e Inteer an Nonlinear Prorammin North-Hollan Amsteram 3 pp 37 86 Pae 7