Research Article Nonlinear Conjugate Gradient Methods with Wolfe Type Line Search

Transcription

1 Abstract and Applied Analysis Volume 013, Article ID 74815, 5 pages Research Article Nonlinear Conjugate Gradient Methods with Wolfe Type Line Search Yuan-Yuan Chen and Shou-Qiang Du College of Mathematics, Qingdao University, Qingdao 66071, China Correspondence should be addressed to Yuan-Yuan Chen; usstchenyuanyuan@163.com Received 0 January 013; Accepted 6 February 013 Academic Editor: Yisheng Song Copyright 013 Y.-Y. Chen and S.-Q. Du. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original wor is properly cited. Nonlinear conjugate gradient method is one of the useful methods for unconstrained optimization problems. In this paper, we consider three inds of nonlinear conjugate gradient methods with Wolfe type line search for unstrained optimization problems. Under some mild assumptions, the global convergence results of the given methods are proposed. The numerical results show that the nonlinear conjugate gradient methods with Wolfe type line search are efficient for some unconstrained optimization problems. 1. Introduction In this paper, we focus our attention on the global convergence of nonlinear conjugate gradient method with Wolfe typelinesearch.weconsiderthefollowingunconstrained optimization problem: min x R n f (x). (1) In (1), f is continuously differentiable function, and its gradient is denoted by g(x) = f(x). Of course, the iterative methods are often used for (1). The iterative formula is given by x +1 =x +α d, () where x, x +1 R n is the th and (+1)th iterative step, α is astepsize,andd is a search direction. Here, in the following, we define the search direction by d ={ g, g +β d 1, if =1, if. In (3), β is a conjugate gradient scalar, and the well-nown useful formulas are β FR, βprp, β HS,andβDY (see [1 6]). Recently, some inds of new nonlinear conjugate gradient methods are given in [7 11]. Based on the new method, we give some new inds of nonlinear conjugate gradient (3) methods and analyze the global convergence of the methods withwolfetypelinesearch. The rest of the paper is organized as follows. In Section, we give the methods and the global convergence results for them. In the last section, numerical results and some discussions are given.. The Methods and Their Global Convergence Results Firstly, we give the Wolfe type line search, which will be used in our new nonlinear conjugate gradient methods. In the following section of this paper, stands for the -norm. We have used the Wolfe type line search in [1]. The line search is to compute α >0such that f(x +α d ) f(x ) ρα d, (4) g(x +α d ) T d σα d, (5) where ρ, σ (0, 1), ρ < σ. Now, we present the nonlinear conjugate gradient methods as follows. Algorithm 1. We have the following steps. Step 0. Given x 0 R n,setd 0 = g 0, := 0.If 0 =0,then stop.

2 Abstract and Applied Analysis Step 1. Find α >0satisfying (4) and(5), and by (), x +1 is given. If +1 =0,thenstop. Step. Compute d by the following equation: d =β d 1 (1+β g T d 1 )g, (6) in which β = /d T 1 g 1.Set:=+1,andgoto Step 1. By (4), we now that =1 (g T d ) d =1 (σ + L) α d (σ + L) ρ <+. =1 {f (x ) f(x +1 )} (13) Before giving the global convergence theorem, we need the following assumptions. Assumption 1. (A1) The set L 0 ={x R n f(x) f(x 0 )} is bounded. (A) In the neighborhood of L 0, denoted as U, f is continuously differentiable. Its gradient is Lipschitz continuous; namely, for x, y U, there exists L>0such that g (x) g(y) L x y. (7) In order to establish the global convergence of Algorithm 1, we also need the following lemmas. Lemma. Suppose that Assumption 1 holds; then, (4) and (5) are well defined. TheproofisessentiallythesameasLemma1of[1]; hence, we do not rewrite it again. Lemma 3. Suppose that direction d is given by (6);then,one has d T g = g 0 (8) holds for all 0.So,onenowsthatd is descent search direction. Proof. From the definitions of d and β,wecangetit. Lemma 4. Suppose that Assumption 1 holds, and α is determined by (4) and (5);one has (g T d ) =1 d <+. (9) Proof. By (4), (5), Lemma 3, andassumption 1, wecanget Then, we now that (σ + L) α d g T d. (10) (σ + L) α d gt d d. (11) By squaring both sides of the previous inequation, we get (σ + L) α d (gt d ) d. (1) So,weget(9), and this completes the proof of the lemma. Lemma 5. Suppose that Assumption 1 holds, d is computed by (6),andα is determined by (4) and (5);onehas 4 <+. (14) d 0 Proof. From Lemmas 3 and 4,wecanobtain(14). Theorem 6. Consider Algorithm 1, and suppose that Assumption 1 holds. Then, one has g =0. (15) Proof. We suppose that the theorem is not true. Suppose by contradiction that there exists ε>0such that holds for 0. From (6)and Lemma 3,we get g ε (16) d =(β ) d 1 g T (1+β d 1 ) g (1+β g T d 1 )dt g. Dividing the previous inequation by (g T d ),weget d 4 = d (g Td ) = β d 1 (g T d ) (1 + β T g d 1/ ) (d Tg ) (1+β g T d 1/ ) d T g (17)

3 Abstract and Applied Analysis 3 = d So, we obtain β (gt d 1) / d i i=0 ε. (18) 4 1 d ε 1 =+, (19) 1 which contradicts (14).Hencewegetthistheorem. Remar 7. In Algorithm 1, wealsocanusethefollowing equations to compute d : d =β d 1 β g T d 1 g, (0) where β = max{min{β FR,βPRP }, 0}; d =β d 1 β g T d 1 g, (1) where β = max{min{β LS,βCD }, 0}. Algorithm 8. We have the following steps. Step 0. Given x 0 R n,setd 0 = g 0, =0.If 0 =0,then stop. Step 1. Find α >0satisfying (4) and(5), and by (), x +1 is given. If +1 =0,thenstop. Step. Compute β by formula β DY = g T g (g g 1 ) T d 1, { g 1 g T d 1, β = { β FR = { 1, g 1 >g T d 1, and compute d +1 by (3). Set :=+1,andgotoStep1. () Lemma 9. Suppose that Assumption 1 holds, and β is computed by ();if =0,thenonegetsg T d <0for all and β FR β (see [9]). Lemma 10. Suppose that l>0, and] is a constant. If positive series ζ satisfied one has ζ i i 1 i ζ i l+], (3) ζ =+, 1 ζ =+. i (4) From the previous analysis, we can get the following global convergence result for Algorithm 8. Theorem 11. Suppose that Assumption 1 holds, and (x) c,wherec is a constant. Then, one has g =0. (5) Proof. Suppose by contradiction that there exists ε>0such that g ε (6) holds for all.from(3), we have d =β d 1 g. (7) Squaring both sides of the previous equation, we get d =(β d 1 ) g g T d. (8) Let θ = d / 4 and r = g T d / ;from(), we have θ θ 1 By θ 1 =1/ 1, r 1 =1,wenowthat By (30), we get r 1. (9) θ ε r i c. (30) θ ε r i, r i ε c. From (31)and Lemma10,we have (31) (g T d ) 1 d =+, (3) which contradicts Lemma 4. Therefore, we get this theorem.

4 4 Abstract and Applied Analysis Algorithm 1. We have the following steps. Step 0. Given x 0 R n, μ > 1/4,setd 0 = g 0, := 0.If 0 =0,thenstop. Step 1. Find α >0satisfying (4) and(5), and by (), x +1 is given. If +1 =0,thenstop. Step. Compute d by g, = 0, { d = g T { β d 1 (1+β d (33) 1 { )g, 1, where β = gt (g g 1 ) 1 μ g 1 g T d (34) Set :=+1,andgotoStep1. Lemma 13. Suppose that direction d is given by (33) and (34); then, one has holds for any 0. d T g = g 0 (35) Lemma 14. Suppose that Assumption 1 holds, d is generated by (33) and (34),andα is determined by (4) and (5);onehas 4 <+. (36) d 0 Proof. From Lemma 4 and Lemma 13,weobtain(36). Lemma 15. Suppose that f is convex. That is, d T f(x)d 0, for all d R n,where f(x) is the Hessian matrix of f.let{x } and {d } be generated by Algorithm 1;onehas Proof. By Taylor s theorem, we can get ρα d g T d. (37) f(x +1 )=f(x )+g T s + 1 st G s, (38) where s =x +1 x,andg = 1 0 f(x +τs )dτs. By Assumption 1,(4), and (38), we get ρα d f(x +1 ) f(x ) g T s =α g T d. (39) So, we get (37). Theorem 16. Consider Algorithm 1, and suppose that Assumption 1 and the assumption of Lemma 15 hold. Then, one has g =0. (40) Proof. We suppose that the conclusion is not true. Suppose by contradiction that there exists ε>0such that g ε (41) holds for all 0. By Lemma 13,wehave β = gt (g g 1 ) g T 1 d 1 μ( g 1 g 1 T d ) g T d 1. (4) 1 From Assumption 1, Lemma 15, and(4), we now that Therefore, by (33), we get We obtain β (μl +ρl ρ ) d g +(μl +ρl ρ ) g =(1+ L ρ +μl ρ ) g. g d. (43) 1 (44) 4 +, (45) d 1 which contradicts (36). Therefore, we have So, we complete the proof of this theorem. g =0. (46) Remar 17. In Algorithm 1, β can also be computed by the following formula: β = gt (g g 1 ) 1 where μ>1/4. min { gt (g g 1 ) 1,μ g 1 g T d }, 3. Numerical Experiments and Discussions (47) In this section, we give some numerical experiments for the previous new nonlinear conjugate gradient methods with Wolfe type line search and some discussions. The problems that we tested are from [13]. We use the condition as the stopping criterion. We use MATLAB 7.0 to test the chosen problems. We give the numerical results of Algorithms 1 and 1 to show that the method is efficient for unconstrained optimization problems. The numerical results of Algorithms 1 and 1 arelistedintables1 and.

5 Abstract and Applied Analysis 5 Table 1: Test results for Algorithm 1. Name Dim NI NF NG GULF VARDIM LIN LIN LIN LIN LIN LIN Name: the test problem name; Dim: the problem dimension; NI: the iterations number; NF: the function evaluations number; NG: the gradient evaluations number. Table : Test results for Algorithm 1. Name Dim NI NF NG GULF BIGGS IE IE TRIG BV BV Name: the test problem name; Dim: the problem dimension; NI: the iterations number; NF: the function evaluations number; NG: the gradient evaluations number. Discussion 1. From the analysis of the global convergence of Algorithm 1, wecanseethatifd satisfies the property of efficient descent search direction, we can get the global convergence of the corresponding nonlinear conjugate gradient method with Wolfe type line search without other assumptions. Discussion. In Algorithm 8, we use a Wolfe type line search. Overall, we also feel that nonmonotone line search (see [14]) also can be used in our algorithms. References [1] B. T. Polya, The conjugate gradient method in extremal problems, USSR Computational Mathematics and Mathematical Physics,vol.9,no.4,pp.94 11,1969. [] R. Fletcher and C. M. Reeves, Function minimization by conjugate gradients, The Computer Journal,vol.7,pp , [3] M. R. Hestenes, E.Stiefel. Method of conjugate gradient for solving linear equations, JournalofResearchoftheNational Bureau of Standards,vol.49,pp ,195. [4] Y. H. Dai and Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence property, SIAM Journal on Optimization,vol.10,no.1,pp ,1999. [5] R. Fletcher, Practical Methods of Optimization, Unconstrained Optimization,Wiley,NewYor,NY,USA,ndedition,1987. [6] M. Raydan, The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM Journal on Optimization,vol.7,no.1,pp.6 33,1997. [7] L. Zhang and W. Zhou, Two descent hybrid conjugate gradient methods for optimization, Computational and Applied Mathematics,vol.16,no.1,pp.51 64,008. [8] A. Zhou, Z. Zhu, H. Fan, and Q. Qing, Three new hybrid conjugate gradient methods for optimization, Applied Mathematics, vol., no. 3, pp , 011. [9] B. C. Jiao, L. P. Chen, and C. Y. Pan, Global convergence of a hybrid conjugate gradient method with Goldstein line search, Mathematica Numerica Sinica,vol.9,no.,pp ,007. [10] G. Yuan, Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems, Optimization Letters, vol. 3, no. 1, pp. 11 1, 009. [11] Z. Dai and B. Tian, Global convergence of some modified PRP nonlinear conjugate gradient methods, Optimization Letters, vol. 5, no. 4, pp , 011. [1] C. Y. Wang, Y. Y. Chen, and S. Q. Du, Further insight into the Shamansii modification of Newton method, Applied Mathematics and Computation,vol.180,no.1,pp.46 5,006. [13] J. J. Moré, B. S. Garbow, and K. E. Hillstrom, Testing unconstrained optimization software, ACM Transactions on Mathematical Software,vol.7,no.1,pp.17 41,1981. [14]L.Grippo,F.Lampariello,andS.Lucidi, Anonmonotone line search technique for Newton s method, SIAM Journal on Numerical Analysis,vol.3,no.4,pp ,1986. Discussion 3. From the analysis of the global convergence of Algorithm 1,wecanseethatwhend is an efficient descent search direction, we can get the global convergence of the corresponding conjugate gradient method with Wolfe type line search without requiring uniformly convex function. Acnowledgments This wor is supported by the National Science Foundation of China ( and ), Project of Shandong Province Higher Educational Science and Technology Program (J10LA05), and the International Cooperation Program forexcellentlecturersof011byshandongprovincialeducation Department.

6 Advances in Operations Research Advances in Decision Sciences Applied Mathematics Algebra Probability and Statistics The Scientific World Journal International Differential Equations Submit your manuscripts at International Advances in Combinatorics Mathematical Physics Complex Analysis International Mathematics and Mathematical Sciences Mathematical Problems in Engineering Mathematics Discrete Mathematics Discrete Dynamics in Nature and Society Function Spaces Abstract and Applied Analysis International Stochastic Analysis Optimization