Global Convergence of a New Conjugate Gradient Method with Wolfe Type Line Search +

Transcription

1 ISSN , Ela, UK Joural of Iformatio a Computi Sciece Vol 7, No,, pp 67-7 Global Coverece of a New Cojuate Graiet Metho with Wolfe ype Lie Search + Yuayua Che, School of Maaemet, Uiversity of Shahai for Sciece a echoloy, Shahai 93, Collee of Mathematics, Qiao Uiversity, Qiao, 667 (Receive Auust 7,, accepte September 5, Abstract I this paper, a ew oliear cojuate raiet methos with wolfe type lie search for solvi ustraie optimizatio problems is propose he irectio eerate by the ew methos prouce sufficiet escet search irectio Uer some coitios, we ive the lobal coverece results for the ew oliear cojuate raiet methos with Wolfe type lie search Fially, the umerical results show that the ew metho is also efficiet for eeral ucostraie optimizatios Key Wors: Ucostraie optimizatio; oliear cojuate raiet metho; lie search; lobal coverece Itrouctio I this paper, we cosier the followi lare scale ucostraie optimizatio problem mi f ( x x R where f : R R is cotiuously ifferetiable a its raiet ( x f ( x is available his problem has bee well stuie Iterative methos are wiely use for solvi ( a the iterative formula is ive by x x + α ( + ( where x R is the -th approximatio to the solutio of (, α is a stepsize obtaie by carryi out a lie search a is a search irectio Due to the simplicity of its iteratio a low memory requiremets, the oliear cojuate raiet metho are oe of the most famous methos for solvi the above ucostraie optimizatio problem (, especially i case of the imesio of f (x is lare, which are ofte propose i scietific a eieeri computatio he search irectio is efie by,, (3 +, where is a scalar a ( x Differet cojuate raiet methos correspo to ifferet scalar i (3, ie [-6] FR, (Fletcher-Reeves, HS ( (, (Hestees-Stiefel, + Supporte by atioal sciece fouatio of Chia(3, 7, A project of Shao provice hiher eucatioal sciece a techoloy proram(jla5, Shahai muicipal committee of sciece a techoloy(5558 Publishe by Worl Acaemic Press, Worl Acaemic Uio

2 68 Yuayua Che: Global Coverece of a New Cojuate Graiet Metho with Wolfe ype Lie Search PRP LS ( (, (Pola-Ribiere-Polya,, (Liu-Storey, DY CD (, (Cojuate Descet,, (Dai-Yua he oliear cojuate raiet methos a the lobal coverece results about Fletcher-Reeves(FR metho, Pola-Ribiere-Polya(PRP metho, Hestees-Stiefel(HSmetho, Dai-Yua(DY metho, Cojuate Descet(CD metho a Liu-Storey(LS metho ca see [5-] Quite recetly, a ew oliear cojuate raiet metho was propose i [] he search irectio are ive by the followi way y ( y, ( y y where y, stas for the Eucliea orm I this paper, base o the above ew oliear cojuate raiet metho i [], uer some mil coitios, we ive the lobal coverece of the ew oliear cojuate raiet metho with Wolfe type lie search he rest of the paper is oraize as follows I Sectio, we preset the Wolfe type lie search a the ew oliear cojuate raiet metho I Sectio 3, we preset the lobal coverece results of the ew oliear cojuate raiet metho with Wolfe type lie search Numerical results a some iscussios are ive i the Sectio New oliear cojuate raiet metho I this sectio, we will ive the followi assumptios o objective fuctio f (x i ( he two assumptios have bee ofte use i the literature to prove the lobal coverece of oliear cojuate raiet methos with exact a iexact lie searches for ucostraie optimizatio problems Assumptio (i he level set L x R f ( x f ( } is boue (ii I some eihborhoo { x U of L, f (x is cotiuously ifferetiable a its raiet is Lipschitz cotiuous, amely, there exists a costat L > such that A (x also satisfies the followi coitio ( x ( y L x y, x, y U ( ( x ( y( x y τ x y, for τ > a x, y U Now we cosier the moifie ew cojuate raiet metho for ( with Wolfe type lie search Firstly, we ive the followi Wolfe type lie search (we have propose i [] he lie search is to choose α > such that where < ρ < σ < f ( x α f ( x ρα +, ( ( x + α σα, ( Now we preset the ew cojuate raiet methos as follows Alorithm Step Give x R, set If, the stop Step Fi α > satisfyi the Wolfe type lie search (, (, by (, x + is ive If +, the stop JIC for cotributio: eitor@jicoru

3 Joural of Iformatio a Computi Sciece, Vol 7 ( No, pp Step Compute by (3 a ( Step 3 Set : +, o to Step I orer to establish the lobal coverece of the Alorithm, we ive the followi lemmas Lemma Suppose that Assumptio hols, the the Wolfe type lie search (, ( is feasible he proof is essetially the same as Lemma of [], hece we omit it Lemma Suppose irectio is ive by (3 a (, the we have 7, 8 hols for ay, ie is escet search irectio he proof is essetially the same as heorem of [], hece we omit it Lemma 3 Suppose that Assumptio hols, α is etermie by Wolfe lie search (, (, we have ( < + Proof From (, (, Lemma a Assumptio, we obtai he, we ow (σ + L α Squari both sies of above formula, we et From (, we have ( ( σ + L α α ( ( σ + L (σ + L ρ (σ + L α { f ( x f ( x + } < + So we et (3, this completes the proof Lemma Suppose that Assumptio hols, is eerate by (3 a (, α is etermie by Wolfe lie search (, (, we have Proof From Lemma a Lemma 3, we obtai ( 3 Global coverece result for the metho (3 < + ( From the above aalysis of Sectio, we ive the followi result of lobal coverece for the ew cojuate raiet metho he eeral iea for provi the lobal coverece of the ew cojuate raiet metho with Wolfe type lie search is to assume first by cotraictio, the erive cotraictory relatios We ow ive the lobal coverece theorem for the ew cojuate raiet metho i a similar way to the lobal coverece results i [-3] uer mil assumptios heorem 3 Cosier Alorithm, suppose that Assumptios hols he we have lim if Proof Firstly, we suppose that the coclusio is ot true Suppose by cotraictio that there exists a JIC for subscriptio: publishi@wauoru

4 7 Yuayua Che: Global Coverece of a New Cojuate Graiet Metho with Wolfe ype Lie Search positive costatε >, such that ε hols for all From (3, ( a Assumptios, we et Divii both sies of the above formula by L L where ξ ( + + τ τ ( L L + + τ τ , we have ξ We obtai ε +, ξ which cotraicts ( herefor, we complete the proof of this theorem Numerical experimets a iscussios ξ ε I this sectio, we ive some umerical experimets for the above ew oliear cojuate raiet metho a some fiical iscussios he problem that we teste are from [] Our lie search subroutie computes α such that the lie search coitio ( a ( hol with ρ a σ We also use the coitio 6 + as the stoppi criterio i Alorithm We use MALAB7 to test the chose problems he umerical results of Alorithm are liste i able, where the items i each colum have the followi meais: Name: the ame of the test problem; Dim: the imesio of the problem; NI: the umber of iteratios; NF: the umber of fuctio evaluatios; NG: the umber of raiet evaluatios able est results for the Alorithm Name im NI NF NG LIN 5 BAND BAND 3 3 RID RID IE RIG Discussios From the aalysis of the lobal coverece of Alorithm, we ca see whe satisfie the property of efficiet escet search irectio, we ca et the lobal coverece of the correspoi cojuate raiet metho with Wolfe type lie search without other assumptios Such as the followi metho, which have bee iscusse i [], JIC for cotributio: eitor@jicoru

5 Joural of Iformatio a Computi Sciece, Vol 7 ( No, pp τ, ( N N 7 where τ [,max{,}], is compute by ( From the fact of + + +, we ca et the 8 lobal coverece result as heorem 3 Remar Cosier Alorithm, suppose that Assumptios hols, is compute by ( he we have lim if 5 Refereces [] RFletcher, CReeves Fuctio Miimizatio by Cojuate Graiets J Comput 96, 7: 9-5 [] BPolya he cojuate raiet metho i extreme problems USSR CompMath Math Phys 969, 9: 9- [3] YHDai, YXYua A oliear cojuate raiet metho with ice lobal coverece properties SIAM J Optim 999, : 77-8 [] MRHestees, EStiefel Metho of cojuate raiet for solvi liear equatios JResNatBurSta 95, 9: 9-36 [5] RFletcher Practical methos of optimizatio ucostraie optimizatio New Yor: Wiley, 987 [6] Y Liu, C Storey Efficiet eeralize cojuate raiet alorithms Part : heory, J Optim heory Appl 99, 69: 9-37 [7] M Raya he Barzilai a Borwei raiet metho for the lare ucostraie miimizatio problem SIAM J Optim 997, 7: 6-33 [8] EG Biri, JM Martiez A spectral cojuate raiet metho for ucostraie optimizatio Appl Math Optim, 3: 7-8 [9] JC Gilbert, J Noceal Global Coverece properties of cojuate raiet methos for optimizatio SIAM J Optim 99, : - [] J Su, J Zha Coverece of cojuate raiet methos without lie search Aals Operatios Research, 3: 6-73 [] L Zha, W Zhou, D Li Global coverece of a moifie Fletcher-Reeves cojuate raiet metho with Armijo-type lie search Numer Math 6, : [] WW Haer, H Zha A ew cojuate raiet metho with uaratee escet a a efficiet lie search SIAM J Optim 5, 6: 7-9 [3] C Wa, Y Che, S Du Further isiht ito the Shamasii moificatio of Newto metho Appl Math Comput 6, 8: 6-5 [] JJ More, BS Garbow, KE Hillstrom esti ucostraie optimizatio software ACM ras Math Software 98, 7: 7- JIC for subscriptio: publishi@wauoru