A New Nonlinear Conjugate Gradient Coefficient. for Unconstrained Optimization

Transcription

1 Applie Mathematical Sciences, Vol. 9, 05, no. 37, 83-8 HIKARI Lt, A New Nonlinear Conjuate Graient Coefficient for Unconstraine Optimization * Mohame Hamoa, Moh Rivaie, 3 Mustafa Mamat an Zabiin Salleh School of Informatics an Applie Mathematics, Universiti Malaysia erenanu (UM, 030 Kuala erenanu, Malaysia Department of Computer Science an Mathematics, Univesiti enoloi MARA (UIM 3000 erenanu, Malaysia 3 Department of Computer Science an Mathematics, Faculty of Informatics an Computin, Universiti Sultan Zainal Abiin, 00 erenanu, Malaysia Copyriht 04 Mohame Hamoa et al. his is an open access article istribute uner the Creative Commons Attribution License, which permits unrestricte use, istribution, an reprouction in any meium, provie the oriinal wor is properly cite. Abstract In this paper, we suest a new nonlinear conjuate raient metho for solvin lare scale unconstraine optimization problems. We prove that the new conjuate raient coefficient with exact line search is lobally converent. Preliminary numerical results with a set of 6 unconstraine optimization problems show that is very promisin an efficient when compare to the other conjuate raient coefficients Fletcher - Reeves (FR an Pola -Ribiere Polya (PRP. Keywors: Conjuate raient coefficient, exact line search, lobal converence, lare scale, unconstraine optimization. Introuction In this paper, we focus our attention on the unconstraine optimization problem, min f ( x n x R (. Where f R n n : R is continuously ifferentiable function an R enotes an n-imensional Eucliean space. We enote by (x, the raient of f at x. he

2 84 Mohame Hamoa et al. conjuate raient (CG metho is the best methos for solvin (., especially when the imension is lare. he iterates of the CG metho for solvin (. are obtaine by x x α, 0,,,... (. Where x is current iterate point an the α is step size. he step size is compute by carryin out some line search, for example, the exact line search where, α a r min f ( x α α 0 (.3 he is the search irection efine by if 0, if, (.4 Where ( x an is a scalar. he most well-nown classical formula for are the Hestenes-Stiefe (HS metho []. he Fletcher Reeves (FR metho [7]. he Pola-Ribiere -Polya (PRP metho [5, 6]. he conjuate escent (CD metho[6]. he Liu Storey (LS metho [4] an the Dai Yuan (DY metho []. he parameters of these as follows HS ( ( (.5 FR (.6 PRP ( (.7 CD (.8 LS ( (.9 DY ( (.0 he most stuie properties of CG methos are its lobal converence properties. Zoutenij [] an Powell [7] prove that FR metho with exact line search is lobally converent. Zhan et al [3], Propose a moifie FR metho MFR which

3 A new nonlinear conjuate raient coefficient 85 is lobally converent uner inexact line search. Polya [6] an Powell [8] showe that PRP has a oo numerical performance, but oes not have such oo converence property. ouati-ahme an Storey [0], Gilbert an Noceal [8] ave another way to iscuss the lobal converence of the PRP metho with the wea Wolfe Powell line search, where the parameter in (.6 is not allowe to be neative, max{ PRP,0}, therefore, urin the past few years, many authors has been investiate to create new formula for,[3, 4, 9, 0, 9, ]. In this paper, we will show a new in section. In section 3, we will stuy the sufficient escent conition an the lobal converence proof of the new. In section 4, we present the numerical results an iscussion. Finally, we present the conclusions in section 5.. New parameter an alorithm In this section, we present a moifie of PRP metho which is nown as, where enotes Mohame, Rivaie an Mustafa, is efine by, ( (. he followin alorithm is a eneral alorithm for solvin optimization by CG methos. Alorithm (. n Step : Given x 0 R, ε 0, set 0 0 if 0 ε then stop. Step : Compute α by exact line search (.3. Step 3: Let x x α, ( x if < ε then stop. Step 4: Compute by formula (., an enerate by (.4. Step 5: Set o to Step. he followin assumptions are often use in the stuies of the conjuate raient methos. Assumption A. f (x is boune from below on the level set n Ω { x R, f ( x f ( x0 }, where x0 is the startin point. Assumption B. In some neihbourhoo N of, the objective function is continuously ifferentiable, an its raient is Lipschitz continuous, that is there exists a constant L > 0 such that x ( y L x y x, y N (. (.

4 86 Mohame Hamoa et al. 3. he Global Converence properties In this section, we stuy the lobal converent properties of, first we nee to simplify the, so that the proof will be easier. From (. we now that ( 0 hus we et, 0 Also Hence we obtain 0 he followin lemmas are very useful in the process of the stuies on the conjuate raient methos Lemma 3.. Suppose that Assumptions A an B hol, let x be enerate by Alorithm. where, satisfies 0 < for all, an α is obtaine by (.3, then, < 0 ( (3. his theorem show that B has an avancement that the irections will approach to the steepest escent irections while the step lenth α is small. heorem 3.. Suppose that Assumptions A an B hol, } { x enerate by the Alorithm., where the step size α is etermine by the exact line search (.3. hen (3. hols for all 0.

5 A new nonlinear conjuate raient coefficient 87 Proof. Suppose that for all, 0. If 0 then ( 0 c 0. If at a point x, is not a escent irection, then by the exact line search, we have x x which implies []. From (., we have B 0. his means that at those points the irections will turn out to be the steepest escent irections. hose points are enote by P { x : B 0} an the other points are enote by P { x : B 0}. For all the points in P, since the irections are the steepest escent irections, from Lemma., we have ( x P he same as the above proof, for the points in P, we also have < (3. From (3. an (3.3 we have, ( < x P ( 0 ( ( x P x P < (3.3 he proof is complete. heorem 3. Suppose that Assumptions A an B hol, the sequence { x } is enerate by Alorithm., if S α 0 while, then lim inf 0 (3.4 Proof. Let θ θ be the anle between an, where cos, then by the exact line search, we have 0, where the search irection efine by (.4, the followin relations hol true: secθ., tan θ. So we have tanθ ( secθ secθ secθ secθ secθ tan secθ θ (3.5

6 88 Mohame Hamoa et al. If (3.4 oes not hol, then, for all, there exists c > 0 such that c (3.6 By S 0 an Lipschitz conition (., there must exist an inteer M 0 for all M, such that c (3.7 4 Combinin (3.5 an (3.7, we obtain tanθ secθ (3.8 π We now that, for all θ [0,, the followin inequality hols: sec θ tanθ (3.9 From (3.8 an (3.9 we et, tanθ ( tanθ (3.0 Utilizin (3.0 inuces, tanθ m m... tanθm 4 8 tanθ From this result, we note that the anle θ must be always less than some anle θ π where, θ <, but by the heorem 3., we have ( 0 0.(cosθ < his implies lim inf 0, which contraicts (3.6. he proof is complete. 4. Numerical results an iscussions In this section, we present the computational performance of a MALAB proram on a set of 6 unconstraine optimization test problems. We selecte 4 test functions consiere in Anrei [], each of them is teste in ifferent variables. We performe a comparison with two CG methos Fletcher Reeves 6 (FR an Pola-Ribiere-Polya (PRP, we consiere ε 0 an the raient value as the stoppin criteria as Hillstrom [] sueste that ε as the stoppin criteria. For each of the test functions problem, we use four initial points, startin from a closer point to the solution an movin on to the one that is furthest from it. A list of problem functions an the initial points use are shown m

7 A new nonlinear conjuate raient coefficient 89 in table, where the exact line search was use to compute the step size. he CPU processor use was Intel (R Core M i3-m350 (.7GHz, with RAM 4 GB. In some cases, the computation stoppe ue to the failure of the line search to fin the positive step size, an thus it was consiere a failure. Numerical results are compare relative on the CPU time an number of iteration. he performance results are shown in Fis. an respectively, usin a performance profile introuce by Dolan an More [5]. No Function Dimension Initial points hree Hump (-0,-0,(0,0,(0,0,(40,40 Six Hump (-0,-0,(-8,8,(8,8,(0,0 3 Booth (0,0,(5,5,(50,50,(00,00 4 reccani (5,5,(0,0,(0,0,(50,50 5 Zettl (5,5,(0,0,(0,0,(50,50 6 Diaonal 4,4, 0,00,500,000 (,..,,(3,,3,(6,,6,(,, 7 Perturbe Qua.,4, 0,00,500,000 (,,,(3,,3,(5,,5, (0,,0 8 E-Himmelblau 0,00,500,000,0000 (50,..,50,(70,..,70,(00,..,00, (5,..,5 9 E-Rosenbroc 0,00,500,000,0000 (3,,3,(5,,5,(30,,30,(50,,50 0 Shallow 0,00,500,000,0000 (0, 0,(5,,5,(50,,50,(70,,70 E-riiaonal 0,00,500,000,0000 (6,,6,(,,,(7,,7,(0,,0 G-riiaonal,4,0,00 (7,,7,(0,,0, (3,,3,(,, 3 E-white-Holst,4,0,00,500,000,0000 (3,,3,(5,,5,(7,,7, (0,,0 4 G- Quartic,4,0,00,500,000,0000 (,,,(,,,(5,,5,(7,,7 5 E- Powell 4,0,00,500,000 (,,,(4,,4,(6,,6,(8,,8 6 E-Denschnb,4,0,00,500,000,0000 (8,,8,(3,,3,(30,,30,(50,50 7 Haer,4,0,00 (7,,7,(0,,0, (5,,5, (3,,3 8 E- Penalty,4,0,00 (80,..,80,(0,..,00,(,..,,(50,..,50 9 Quartic QF,4,0,00,500,000 (5,,5,(0,,0,(50,,50,(00,..,00 0 E - QP,4,0,00,500,000 (0,..,0,(0,..,0, (30,..,30,(50,..,50 E- Beale,4,0,00,500,000,0000 (-,,-,(3,,3,(7,,7, (0,,0 Diaonal,4,0,00,500,000 (,,,(5,,5,(0,,0,(5,,5 3 Rayan,4,0,00 (,,,(3,,3,(7,,7, (0,,0 4 Sum Squares,4,0,00,500,000 (,,,(3,,3,(7,,7, (0,,0 Accorin to the rules consiere in [5], we now that the metho whose performance profile plot on top riht will be better than the rest of the other methos. Fiures - show that the performances of these methos are relative to the number of iteration an the CPU time. From fiures -, it is easy to see that is the best amon the two methos FR an PRP, the performance of PRP seems to be faster than, but it can solve only 93% of the problems an FR solve only 70%, where can solve all the test problems an reach 00%, the performance of lies between FR an PRP. In other wors, metho is competitive to the other two methos an its notable formula.

8 80 Mohame Hamoa et al P S (t P S (t FR PRP 0.0 e 0 e e e 3 t 0. t FR PRP 0.0 e 0 e e e 3 Fi.. Performance profile relative to the number of iterations. Fi.. Performance profile relative to the CPU time. 5. Conclusion an further research his paper ives a new conjuate raient metho for solvin unconstraine optimization problems. Uner the exact line search, this possesses the lobal converence conition. Numerical results show that our metho is competitive to other two conjuate raient methos, Fletcher Reeves (FR an Pola Ribiere Polya (PRP, an come out with best numerical results. For further research, we shoul stuy the new metho with the stron Wolfe-Powell line search. Acnowleements. he authors woul lie to than the University of Malaysia erenanu (Grant no FRGS Vot 5956 an Alasmrya University of Libya. References [] N. Anrei, "An unconstraine optimization test functions collection," Avance Moelin an Optimization, 0 (008, [] Y.-H. Dai an Y. Yuan, "A nonlinear conjuate raient metho with a stron lobal converence property," SIAM Journal on Optimization, 0 (999, [3] Y. H. Dai, "Converence of nonlinear conjuate raient methos," Journal of Computational Mathematics, 9 (00, [4] Y. H. Dai an Y. Yuan, "Further stuies on the Pola-Ribiere-Polya metho," Research report ICM , Institute of Computational Mathematics an Scientific/Enineerin Computin, Chinese Acaemy of Sciences995.

9 A new nonlinear conjuate raient coefficient 8 [5] E. D. Dolan an J. J. Mor, "Benchmarin optimization software with performance profiles," Mathematical Prorammin, 9 (00, [6] R. Fletcher, Practical Metho of Optimization, e. I. New Yor, [7] R. Fletcher an C. M. Reeves, "Function minimization by conjuate raients," he computer journal, 7 (964, [8] J. C. Gilbert an J. Noceal, "Global converence properties of conjuate raient methos for optimization," SIAM J. Optim., (99, [9] L. Grippo an S. Lucii, "A lobally converent version of the Pola-Ribière conjuate raient metho," Math. Proram., 78 (997, [0] H. Hai, Y. Shenwei, an L. Suihua, "A new conjuate raient metho combine HS an DY formulas," M. a. I. Science, E., e: Guanxi University, 006. [] M. R. Hestenes an E. Stiefel, "Metho of conjuate raient for solvin linear systems," J. Res. Nat. Bur. Stan., 49 (95, [] K. E. Hillstrom, "A simulation test approach to the evaluation of nonlinear optimization alorithms," ACM ransactions on Mathematical Software 3 (977, [3] W. J. Z. L. Zhan, D.H. Li, "Global converence of a moifie Fletcher Revves conjuate raient metho with Armijo-type line search," Numer. Math., 04 (006, [4] Y. Liu an C. Storey, "Efficient eneralize conjuate raient alorithms, Part : heory," Journal of Optimization heory an Applications, 69 (99, [5] E. Pola an G. Ribiere, "Note Sur la converence e irections conjue`es," Rev. Francaise Informat Recherche Operationelle, 3E (969, [6] B.. Polya, "he conjuate raient metho in extreme problems," USSR Computational Mathematics an Mathematical Physics, 9 (969, 94.

10 8 Mohame Hamoa et al. [7] M. J. Powell, "Nonconvex minimization calculations an the conjuate raient metho," in Numerical analysis, e: Spriner, -4, [8] M. J. D. Powell, "Restart proceures for the conjuate raient metho," Mathematical Prorammin (977, [9] M. Rivaie, M. Mamat, L. W. June, an I. Moh, "A new class of nonlinear conjuate raient coefficients with lobal converence properties," Applie Mathematics an Computation, 8 (0, [0] D. ouati-ahme an C. Storey, "Globally converent hybri conjuate raient methos," J. Optim. heory Appl., 64 (990, [] Z. X. Wei, S. W. Yao, an L. Y. Liu, "he converence properties of some new conjuate raient methos," Applie Mathematics an Computation, 83 (006, [] G. Zoutenij, "Nonlinear prorammin, computational methos," in Inteer an nonlinear prorammin, e North-Hollan, Amsteram, 37-86, 970. Receive: December 0, 04; Publishe: March 7, 05