A steepest descent method with a Wolf type line search

Transcription

1 ISSN , Ela, UK Joural of Iformatio a Computi Sciece Vol 9, No 4, 14, pp 5-61 A steepest escet metho with a Wolf type lie search Ju Ji-jie,Pa De-ya,Du Shou-qia Collee of Mathematics, Qiao Uiversity,Qiao 6671 (Receive February 1, 14, accepte September 11, 14) Abstract he steepest ecet metho is propose Frech mathematicia Cauchy a it is oe of the simplest a olest methos for solvi ucostraie optimizatio problems he steepest escet metho, eative raiet irectio is chose as the search irectio, also ow as the raiet metho Usually, the step leth α of the steepest ecet metho ca be compute some ieact lie search I this paper, we will use a Wolfe type lie search to evaluate the step leth a its coverece property will be ive uer mil assumptios From the umerical results, we ca see that the steepest escet metho with this Wolfe type lie search is very promisi Fially, we ive a applicatio of the metho to solve the oliear complemetarity problem Keywors: the steepest ecet metho, lie search, lobal coverece 1 Itrouctio We cosier the followi ucostraie optimizatio problem mi f ( ), (1) R where f : R R is cotiuously ifferetiable I the past four ecaes, may theories a alorithms i lobally optimal problems have bee evelope ([1-15]) Amo these methos, the steepest ecet metho propose Frech mathematicia Cauchy is oe of the well-ow a practical methos for lobal optimizatio problems Because it has the simple structure, the less amout of calculatio a the fast coverece spee whe it away from the miimum value of the problem So the steepest escet metho has become oe of the most famous methos for solvi lare-scale optimizatio problem he steepest escet metho has bee stuie eeply may scholars, such as [1-4] I practice, it ofte use i commuicatio, computer a iformatio eieeri a other fiels herefore, it has a importat meai for the stuy of the steepest escet metho he classical steepest escet metho for the cotiuously ifferetiable fuctio f : R R is efie the iteratio +1 =, where is the steepest escet irectio at a α is the step leth at Sice we use a eative raiet irectio as the search irectio, so the steepest ecet metho is also sometimes calle the raiet metho I practice, the step leth is compute a ieact lie search, this esures a appropriate reuctio i the objective fuctio Usually, we use Armijo, Wolfe a other ieact lie search rule to compute the step leth α (such as []) I this paper, we use a Wolfe type lie search to computi the step leth he umerical results i Sectio 4 iicate that, for some problems, the steepest escet metho with Wolfe type lie search ca obtai the lobal miimum solutios most close to the fuctio, as well as the steepest escet with Armijo lie search he Wolfe type lie search (see [5]) as follows: Chooseα > such as f ( ) f ( ) ρα () ( ) σα (3) Publishe Worl Acaemic Press, Worl Acaemic Uio

2 Joural of Iformatio a Computi Sciece, Vol 9 (14) No 4, pp where < ρ < σ < 1 his paper is oraize as follows I Sectio, we will ive the alorithm of the steepest escet metho with the Wolfe type lie search a the steepest escet metho with Armijo lie search, respectively he lobal coverece of the methos will be ive i Sectio 3 I Sectio 4, some umerical results are reporte for both the steepest escet metho with the Wolfe lie search, the steepest escet metho with Armijo lie search a some cojuate raiet methos Fially, i Sectio 5, we ive the coclusio a a applicatio of the steepest escet metho for solvi oliear complemetarity problem I this paper, we eote = ) = f ( ) he orm is the Eucliea orm Alorithm ( I this sectio, we will ive the steepest escet metho with the Wolfe type lie search a the Armijo lie search, respectively he steepest escet metho with Armijo lie search has bee etail itrouce i [], so we will oly ive the alorithm Alorithm 1 Step Give R, β (,1), σ (,5), < ε << 1, = 1 Step 1 Compute If ε, stop Output as the approimate optimal solutio Step Choose = Step 3 Computeα m m f ( + β ) f ( ) + σβ Step 4 Set +1 =, = + 1, o to Step 1 I the followi, we preset the steepest escet metho with the Wolfe type lie search Alorithm Step Give R, β (,1), σ (,5), < ε << 1, = 1 Step 1 Compute If ε, stop Output as the approimate optimal solutio Step Choose = Step 3 Computeα () a (3) Step 4 Set +1 =, = + 1, o to Step 1 As a compariso, i Sectio 4, we will ive the umerical results of the above two alorithms, respectively 3 Coverece aalysis of the alorithms he coverece result of Alorithm 1 has bee ive i [] i etail I the followi, we oly ive the lobal coverece result of Alorithm I orer to establish the lobal coverece of Alorithm, firstly, we ive the followi Assumptio 31 a Lemma 31 Assumptio 31 f () is boue below o the level set L { R f ) f ( )} ( = Lemma 31 Suppose that Assumptio 31 hols, the the Wolfe type lie search () a (3) is feasible (see [5]) Now, we ive the followi lobal coverece theorem for the steepest escet metho with the Wolfe type lie search uer mil assumptios JIC for subscriptio: publishi@wauoru

3 54 Ju Ji-jie etal : A steepest escet metho with a Wolf type lie search heorem 31 Suppose that Assumptio 31 hol, () is Lipschitz cotiuous o the level set L Ifα satisfies () a (3) a{ } is eerate the Alorithm, the lim ( ) = (4) Proof From () a Assumptio 31, we ow that By (3), we have ie, lim α = (5) ( ) σα, ( ) ( ) ( ( ) ( )) ( ) Because () L is Lipschitz cotiuous o the level set, we ow ( where L is the Lipschitz costat So Sice = ( ), we have Let, (5), we have he, we ow that herefore (4) hols σα ) ( ) L = L α L ( ) lim ( ) α σα ( ) - ( L + σ) α ( ) lim = = lim ( ( ) ) = lim ( ) =, 4 Numerical results I this sectio, we will ive some umerical results hey are respectively from the Alorithm 1, the Alorithm, the FR cojuate raiet metho, the PRP cojuate raiet metho a the spectral cojuate raiet metho I this sectio, the testi fuctios we will use are well-ow fuctios that are ofte use [, 6] he test fuctios are the follow fuctios, (i) 6-hump camel bac fuctio JIC for cotributio: eitor@jicoru

4 Joural of Iformatio a Computi Sciece, Vol 9 (14) No 4, pp f ( ) = , 3 1, 3 3 he lobal miimum solutios = (898,717) or = ( 898, 717) a f = (ii) fuctio f ( ) = 1( 1 ) + ( 1 1), he lobal miimum solutio = (1,1) a f = (iii) Rastrii fuctio f ( ) = cos(181 ) cos(18 ), 1 1, 1 he lobal miimum solutios = (,) or = (,) a f = First presets the efiitio of several otatios i the table, iicates the iitial poit, iicates the calculate lobal miimizer, f ( ) iicates the calculate fuctio value of the lobal miimizer I alorithms we set β = 5, ρ =, σ = 4 aε = PartⅠ he calculatio results are ive i the tables below he ata i the table (a) a (b) is compute the Alorithm 1 a Alorithm, respectively Fuctio (i) able 411 (a) f ( ) (1,8) ( , ) (44,86) ( , ) (-89,-633) ( , ) able 411 (b) f ( ) (1,8) ( , ) (44,86) ( , ) (-89,-633) ( , ) able 41 (a) Fuctio (ii) f ( ) (,) ( , ) e-1 (,1) ( , ) e-1 (1,-1) ( , ) e-1 able 41 (b) f ( ) (,) ( , ) e-11 (,1) ( , ) e-11 (1,-1) ( , ) e-11 JIC for subscriptio: publishi@wauoru

5 56 Ju Ji-jie etal : A steepest escet metho with a Wolf type lie search able 413 (a) Fuctio (iii) f ( ) (9,9) ( , ) (87,5) 1e-7 * ( , ) (898,91) ( , ) able 413 (b) f ( ) (9,9) 1e-7 *( , ) (87,5) 1e-8 *( , ) (898,91) 1e-7 *( , ) PartⅡ he alorithm of the FR cojuate raiet metho is as followi, Alorithm 41 Step Give R, < ε << 1 Compute = f ( ), let = 1 Step 1 If ε, stop Output as the approimate optimal solutio Step Compute, =, = + β 1 1, 1, where β 1 = ( 1) 1 1 Step 3 Computeα m m f ( + β ) f ( ) + σβ Step 4 Set +1 =, compute + 1 = f ( + 1) Step 5 Set = + 1a o to Step 1 he calculatio results are ive i the tables below he ata i the table (a) a (b) is compute the Alorithm 41 a Alorithm, respectively Fuctio (i) able 41 (a) f ( ) (1,8) ( , ) (44,86) ( , ) (-89,-633) ( , ) able 41(b) f ( ) (1,8) ( , ) (44,86) ( , ) (-89,-633) ( , ) JIC for cotributio: eitor@jicoru

6 Joural of Iformatio a Computi Sciece, Vol 9 (14) No 4, pp able 4 (a) Fuctio (ii) f ( ) (,) ( , ) e-11 (,1) ( , ) e-11 (1,-1) ( , ) e-11 able 4 (b) f ( ) (,) ( , ) e-11 (,1) ( , ) e-11 (1,-1) ( , ) e-11 able 43 (a) Fuctio (iii) f ( ) (9,9) ( , ) (87,5) 1e-8 * ( , ) (898,91) ( , ) able 43 (b) f ( ) (9,9) 1e-7 *( , ) (87,5) 1e-8 *( , ) (898,91) 1e-7 *( , ) PartⅢ he alorithm of the PRP cojuate raiet metho is as followi, Alorithm 4 Step Give R, < ε << 1 Compute = f ( ), let = 1 Step 1 If ε, stop Output as the approimate optimal solutio Step Compute, =, = + β 1 1, 1, ( 1 ) where β 1 = ( 1) 1 1 Step 3 Computeα m m f ( + β ) f ( ) + σβ Step 4 Set +1 =, compute + 1 = f ( + 1) Step 5 Set = + 1a o to Step 1 he calculatio results are ive i the tables below he ata i the table (a) a (b) is compute the Alorithm 4 a Alorithm, respectively JIC for subscriptio: publishi@wauoru

7 58 Ju Ji-jie etal : A steepest escet metho with a Wolf type lie search able 431 (a) Fuctio (i) f ( ) (1,8) ( , ) (44,86) ( , ) (-89,-633) ( , ) able 431(b) f ( ) (1,8) ( , ) (44,86) ( , ) (-89,-633) ( , ) able 43 (a) Fuctio (ii) f ( ) (,) ( , ) e-1 (,1) ( , ) e-1 (1,-1) (157771, ) e-1 able 43 (b) f ( ) (,) ( , ) e-11 (,1) ( , ) e-11 (1,-1) ( , ) e-11 able 433 (a) Fuctio (iii) f ( ) (9,9) ( , ) (87,5) 1e-7 * ( , ) (898,91) ( , ) able 433 (b) f ( ) (9,9) 1e-7 *( , ) (87,5) 1e-8 *( , ) (898,91) 1e-7 *( , ) PartⅣ he alorithm of the spectral cojuate raiet metho is as followi (see [16]), Alorithm 43 Step Give R, < ε << 1 Compute = f ( ), let = a = 1 Step 1 If ε, stop Output as the approimate optimal solutio Step Computeα JIC for cotributio: eitor@jicoru

8 Joural of Iformatio a Computi Sciece, Vol 9 (14) No 4, pp m m f ( + β ) f ( ) + σβ Set +1 = s s ( y s ) + 1 Step 3 Computeθ = a β = θ Defie s y s y = θ β s he compute + 1 3, if = 1 1 θ + 1, others Step 4 Set = + 1a o to Step 1 he calculatio results are ive i the tables below he ata i the table (a) a (b) is compute the Alorithm 43 a Alorithm, respectively Fuctio (i) able 441 (a) f ( ) (1,8) ( , ) (44,86) ( , ) (-89,-633) ( , ) able 441(b) f ( ) (1,8) ( , ) (44,86) ( , ) (-89,-633) ( , ) able 44 (a) Fuctio (ii) f ( ) (,) ( , ) e-14 (,1) ( , ) e-1 (1,-1) ( , ) e-13 able 44 (b) f ( ) (,) ( , ) e-11 (,1) ( , ) e-11 (1,-1) ( , ) e-11 able 443 (a) Fuctio (iii) f ( ) (9,9) ( , ) (87,5) 1e-7 * ( , ) (898,91) ( , ) JIC for subscriptio: publishi@wauoru

9 6 Ju Ji-jie etal : A steepest escet metho with a Wolf type lie search able 443 (b) f ( ) (9,9) 1e-7 *( , ) (87,5) 1e-8 *( , ) (898,91) 1e-7 *( , ) CONCLUSIONS Accori to the umerical results of Sectio 4 we ca see that, the umerical performace of the steepest escet metho with the Wolfe type lie search is ecellet, as well as the steepest escet metho with Armijo lie search, the cojuate raiet methos a the spectral cojuate raiet metho So, the steepest escet metho with the Wolfe type lie search is promisi Hece, we ca use this metho to solve the optimizatio problems Net, we ive some iscussios about the applicatio of the steepest escet metho with the Wolfe type lie search As we ow, the oliear complemetarity problem, NCP( f ) for short, ca be trasforme ito ucostraie optimizatio problem [1] he oliear complemetarity problem is to fi a R, such that, f ( ), f ( ) =, where f () is cotiuously ifferetiable We ca use the Fischer-Burmeister fuctio to trasform the oliear complemetarity problem to ucostraie optimizatio problem he Fischer-Burmeister fuctio, φ : R R, φ ( a, b) = a + b a b, heφ is cotiuously ifferetiable, ecept the poit (,) A φ ( a, b) = a, b, ab = We eoteψ : R R, 1 ψ ( u) = φ( u), u R where, φ ( u) = u e u, u = ( a, b) R Deote the merit fuctio 1 θ ( ) = θ i ( ), θ i ( ) = ψ ( i, f i ( )) = φ( i, f i ( )), i = 1,, L, i= 1 whereθ : R R is cotiuously ifferetiable he, we ow that is the solutio of miθ ( ) if a oly if is the solutio of NCP ( f ) So, the NCP( f ) is trasforme ito ucostraie optimizatio problem he etaile alorithm a coverece property will be ive i the future Acowleemets Supporte Natioal Sciece Fouatio of Chia (111131), A project of Shao provice hiher eucatioal sciece a techoloy proram (J1LA5) R JIC for cotributio: eitor@jicoru

10 Joural of Iformatio a Computi Sciece, Vol 9 (14) No 4, pp Refereces [1] G C Beto, O P Ferreira, P R Oliveira, Ucostraie steepest escet metho for multicriteria optimizatio o Riemaia maifols[j], Joural of Optimizatio heory a Applicatios, 1(154):88-17 [] C F Ma, Optimizatio methos a matlab prorammi[m], Beiji: Sciece press, 1 [3] Y X Yua, W Y Su, Optimizatio theory a metho [M], Beiji: Sciece press, 1997 [4] Y J Wa, N H Xiu, Noliear optimizatio theory a metho [M], Beiji: Sciece press, 1 [5] C Y Wa, Y Y Che, S Q Du, Further isiht ito the Shamasii moificatio of Newto metho [J], Applie Mathematics a Computatio, 6(18):46-5 [6] W Wa, Y J Ya, L S Zha, Uificatio of Fille Fuctio a uelli Fuctio i Global Optimizatio [J], Acta Mathematicae Applicatae Siica, Elish Series, 7(3):59-66 [7] F Alvarez, A Cabot, Steepest escet with curvature yamical system [J], Joural of Optimizatio heory a Applicatios, 4(1):47-73 [8] K C Kiwiel, K Murty, Coverece of the Steepest Descet Metho for Miimizi Qussicove Fuctios [J], Joural of Optimizatio heory a Applicatios, 1996(89):1-6 [9] R Fletcher, A limite memory steepest escet metho [J], Mathematical Prorammi, 1(135): [1] C Y Wu, G Q Che, A cojuate raiet alorithm for lare scale oliear complemetarity problems [J], Mathematics i Practice a heory, 1(4): [11] Y X Yua, Y H Dai, J Y Yua, Moifie two-poit stepsize raiet methos for ucostraie optimizatio [J], Computatioal Optimizatio a Applicatios, (): [1] Y X Yua, Y H Dai, Alterate miimizatio raiet metho [J], IMA J Num Aal, 3(3): [13] G H Yu, L Gua, W F Che, Spectral cojuate raiet methos with sufficiet escet property for larescale ucostraie optimizatio [J], Optimizatio Methos a Software, 8(3):75-93 [14] Y X Yua, A ew stepsize for the steepest escet metho [J], Joural of Computatioal Mathematics, 6(4): [15] Y X Yua, A short ote o the Q-liear coverece of the steepest escet metho [J], Mathematical Prorammi, 1(13): [16] E G Biri, J M Martiez, A spectral cojuate metho for ucostraie optimizatio [J], Appl Math Optim, 1(43): JIC for subscriptio: publishi@wauoru