Research Article A Nonmonotone Weighting Self-Adaptive Trust Region Algorithm for Unconstrained Nonconvex Optimization

Transcription

1 Discrete Dynamics in Nature and Society Volume 2015, Article ID , 8 pages Research Article A Nonmonotone Weighting Self-Adaptive Trust Region Algorithm for Unconstrained Nonconvex Optimization Yunlong Lu, Weiwei Yang, Wenyu Li, Xiaowei Jiang, and Yueting Yang School of Mathematics and Statistics, Beihua University, Jilin , China Correspondence should be addressed to Yueting Yang; yangyueting@163.com Received 14 August 2015; Revised 25 October 2015; Accepted 26 October 2015 AcademicEditor:JuanR.Torregrosa Copyright 2015 Yunlong Lu et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. A new trust region method is presented, which combines nonmonotone line search technique, a self-adaptive update rule for the trust region radius, and the weighting technique for the ratio between the actual reduction and the predicted reduction. Under reasonable assumptions, the global convergence of the method is established for unconstrained nonconvex optimization. Numerical results show that the new method is efficient and robust for solving unconstrained optimization problems. 1. Introduction Consider the following unconstrained optimization problem: min x R n f (x), (1) where f: R n R is continuously differentiable. The trust region methods calculate a trial step d k by solving the subproblem at each iteration, min q k (d) =f(x k )+ 1 2 dt B k d+g T k d s.t. d Δ k, where g k = f(x k ) and B k is symmetric matrix approximating the Hessian of f(x) at x k and Δ k >0is a trust region radius at x k.throughoutthispaper, denotes the Euclidean norm on R n.definetheratio (2) r O k = f(x k) f(x k +d k ), (3) q k (0) q k (d k ) andthenumeratorandthedenominatorarecalledtheactual reduction and the predicted reduction, respectively. The nonmonotone line search technique is firstly proposedbygrippoetal.[1]inlinesearchframeworkfor Newton s method. At each iteration, the selected function value is taken as f l(k) = max f(x k j), (4) 0 j m(k) where m(0) = 0, 0 m(k) min{m(k 1) + 1, M}, M is a positive integer. Although many algorithms based on (4) work well in many cases, a good function value generated as the iteration process is not selected because of the max function in (4), and the choice of M is sensitive to some numerical tests sometimes. To overcome shortages, Zhang and Hager [2] proposed a new nonmonotone line search technique and they used C k to replace the function in (4), where C k = ζ k 1Q k 1 C k 1 +f(x k ), (5) Q k where Q k =ζ k 1 Q k 1 +1, Q 0 =1,andC 0 =f(x 0 ), ζ k 1 [0, 1]. Numerical tests have shown that the new nonmonotone algorithm is more effective. Many researchers proposed many trust region methods by considering the ratio r k and the updating trust region radius to solve effectively unconstrained optimization problem. Dai and Xu [3] proposed the following weighting formula: r k = min{k,m} i=0 ω ki r O k i, (6)

2 2 Discrete Dynamics in Nature and Society where m is some positive integer and ω ki [0, 1] is the weight of r k i,suchthat min{k,m} i=0 ω ki =1. (7) Many self-adaptive adjustment strategies are developed to update the trust region radius, such as [4 14]. In addition, many adaptive nonmonotonic trust region methods have been proposed in literatures [15 21]. In this paper, we propose a new self-adaptive weighting trust region method based on the nonmonotone technique (4) in [2], the weighting technique (5) in [3], and L-function in [6]. The rest of the paper is organized as follows. In Section 2, we define L-function to introduce a new update rule and a new nonmonotone self-adaptive trust region algorithm is presented. In Section 3, the convergence properties of the proposed algorithm are investigated. In Section 4, numerical results are given. In Section 5 conclusions are summarized. 2. L-Function and the New Nonmonotone Self-Adaptive Trust Region Algorithm To obtain the new trust region radius update rules, we recall L-function L(t), t R. Definition 1 (see [6]). A function L(t) is called an L-function if it satisfies the following. (1) L(t) is nondecreasing in (, η 2 ] and nonincreasing in (2 η 2,+ ), L(t) = β 2,fort [η 2,2 η 2 ], (2) lim t L(t) = c 1, (3) L(0) = c 2, (4) lim t η2 L(t) = 1, (5) lim t 0 +L(t) = β 1, (6) L(t) < 1,fort>η 3, (7) lim t + L(t) = β 3, where the constants β 1, β 2, β 3, η 2, η 3, c 1,andc 2 are positive constants such that 0<c 1 <c 2 <β 1 β 3 <1<β 2, η 3 >2 η 2, 0.5 η 2 <1. Now we describe the new nonmonotone self-adaptive trust region algorithm. Algorithm 2. Step 1.Givenx 0 R n, B 0 R n n, 0<η 2 <1, 0<c 1 <c 2 < 1, 0<β 1 β 3 <1 β 2,and Δ 0 >0; m is a given small (8) positive integer; ε 0; 0 ζ min ζ max <1, Q 0 =1;andset k:=0. Step 2.If g k ε,stop. Step 3. Solve subproblem (2) to get d k. Step 4.Compute r k = Ared k = C k f(x k +d k ) Pred k φ k (0) φ k (d k ) and r k by (6). If r k 0,gotoStep5.ElsegotoStep6. Step 5. Choose some L-function and compute Δ k+1 = L( r k )Δ k, and then go to Step 3. Step 6.Setx k+1 =x k +d k. Update the trust region radius Δ k+1 =L( r k )Δ k. (10) Step 7.Computeg k+1 and B k+1 ;chooseζ k+1 [ζ min,ζ max ],set k:=k+1,andgotostep2. 3. Convergence of Algorithm 2 In the section, we consider the convergence properties of Algorithm 2. We give the following assumption. Assumption 3. (i) The function f is bounded on the level set S = {x f(x) f(x 0 )} and twice continuously differentiable. (ii) The sequence {B k } is uniformly bounded in norm; that is, for some constant M, B k M. (iii) The solution d k of subproblem (2) satisfies q k (0) q k (d k ) σ g k min {Δ k, g k B }, (11) k where σ (0, 1]. Lemma 4. Supposethat(i)and(ii)inAssumption3hold.Then f(x k +d k ) q(d k ) 1 2 M d k 2 +C( d k ) d k, (12) where C( d k ) arbitrarily decreases with d k decreasing. Proof. Since, from Taylor theorem, we have f(x k +d k )=f(x k )+g T k d k 1 + [ f (x k +td k ) g(x k )] T d k dt, 0 it follows from the definition of q k (d) in (2) that f(x k +d k ) q k (d k ) 1 1 = 2 dt k B kd k [ f (x k +td k ) g(x k )] T d k dt M d k 2 +C( d k ) d k, where C( d k ) arbitrarily decreases with d k decreasing. (9) (13) (14)

3 Discrete Dynamics in Nature and Society 3 Lemma 5. Assume that the sequence {x k } is generated by Algorithm2.Thenthesequence{x k } S. Proof. From Lemma 3.1 in [22] and x 0 Sin Assumption 3 (i), we have {x k } S. The next lemma shows that the loop through Step 3 to Step 5 cannot cycle infinitely and the sequence {x k } is well defined. Lemma 6. Suppose that Assumption 3 holds. Assume also g k = 0, and there exists a sufficiently small constant Δ>0such that Then holds. Proof. By Assumption 3 and g k positive index k 0 such that Δ k Δ. (15) Δ k+1 Δ k (16) =0, there exist ε>0and a g k ε, k k 0. (17) Combining (11), we have that, for k k 0, g k q k (0) q k (d k ) σ g k min {Δ k, B } k σεmin {Δ k, Combining (12) and (18), we have ro k 1 = f(x k +d k ) q k (d k ) q k (0) q k (d k ) ε M }. (1/2) M d k 2 +C( d k ) d k σε min {Δ k,ε/m} Δ k (MΔ k +2C( d k )). σε min {Δ k,ε/m} By (15), we can choose sufficiently small Δ such that Δ k Δ ε M, MΔ k +2C( d k ) (1 η 2)σε, and furthermore, for sufficiently large k k 0, (18) (19) (20) ro k 1 (1 η 2). (21) For the above k,weknowthat ro k+j 1 1 η 2, j=1,2,...,m. (22) From (6) and (22), for sufficiently large k k 0 +m,we know that r k formulate always the form m r k 1 = ω ki r O m k i i=0 m i=0 i=0ω ki m i=0 ω ki (1 η 2 ) 1 η 2. ω ki ro k i 1 (23) From (23), for sufficient large k, wehave2 η 2 r k η 2. By Algorithm 2 and the definition of L-function, we have Δ k+1 Δ k,whereδ k falls below Δ. We will show the global convergence of Algorithm 2. Theorem 7. Suppose that Assumption 3 holds. Let the sequence {x k } be generated by Algorithm 2. Then lim inf k g k =0. (24) Proof. For the purpose of deriving a contradiction, suppose thatthereexistsapositiveconstantδ >0such that g k δ>0. (25) For convenience, we denote one index set as follows: J={k r k η 2 }. (26) First, assume that the set J has infinite elements. That is, for any k J, r k η 2 holds. For any k J,usingAlgorithm2 and (12), we have that C k f(x k+1 ) η 2 [q k (0) q k (d k )] Thus, from (27), ση 2 g k min {Δ k, g k B } k ση 2 δ min {Δ k, f(x k+1 ) C k ση 2 δ min {Δ k, From (5) and (28), we have that C k+1 = ζ kq k C k +f(x k+1 ) Q k+1 δ M }. ζ kq k C k +C k ση 2 δ min {Δ k,δ/m} Q k+1 =C k ση 2δ min {Δ k,δ/m} Q k+1. (27) δ }. (28) M (29) From Lemma 3.1 in [22] and Assumption 3 (i), we know the sequence {C k } is nonincreasing and convergent. Then lim k Δ k =0, (30)

4 4 Discrete Dynamics in Nature and Society which contradicts (16). Next, we assume that the set J has finite elements. Then, for sufficient large k,wehavethat r k < η 2. From the definition of L-function and Steps 5 and 6 in Algorithm2,wehavethatthetrustregionΔ k is decreasing as the iteration process. Furthermore, the limit 4. Numerical Experiments In this section, we present preliminary numerical results to illustrate the performance of Algorithm 2, denoted by NTRW. In Algorithm 2, we choose lim k Δ k =0 (31) holds, which gives a contradiction to (16). The proof is completed. c 1 +(c 2 c 1 ) exp (r k ), if r k 0, 1 β 1 exp (η 2 ) { (1 β 1) exp (η 2 ) exp ((r 1 exp (η L(r k )= 2 ) 1 exp (η 2 ) k η 2 )), if 0<r k <η 2, β 2, if η 2 r k 2 η 2, { β 3 +(β 2 β 3 ) exp ( ( r 2 k +η 2 2 ) ), if r { η 2 2 k >2 η 2, (32) where β 1 = 0.5, β 2 = 2, β 3 = 0.7, c 1 = 0.12, c 2 = 0.14, andη 2 = In the framework of Algorithm 2, we compare NTRW with the following algorithms: the basic trust region method, denoted by BTR; the basic trust region method with Grippo s nonmonotone technique, denoted by NTR1, m = 3; the basic trust region method with Hager s nonmonotone technique, denoted by NTR2, ζ = 0.5.Alltests areimplementedbyusingmatlabr2008aonapcwithcpu 2.40 GHz and 2.00 GB RAM. The test problem collections for unconstrained minimization in Table 1 are taken from More etal.in[23],thecutercollection[24,25]. In all algorithms in this paper, the matrix B k is updated by BFGS formula [26, 27]. The trial step d k, for smallscaleproblems,iscomputedbytrustmfileinoptimization ToolboxofMatlab,formiddle-scaleandlarge-scaleproblems, and is computed by CG-Steihaug algorithm in [26]. The iteration is terminated by the following condition: g k ε, (33) where ε=10 5.InTables1,2,3,and4,wegivethedimension (Dim) of each test problem (P), the number iter of iterations, the number nf of function evaluations, and the CPU (cpu) time for solving the test problem. InTable2,wecompare43small-scaleproblemsforthe four algorithms, and the results are concluded as follows: (i) 19 problems where NTRW was superior to BTR, (ii) 11 problems where BTR was superior to NTRW, (iii) 13 problems where NTR2 was superior to NTR1, (iv) 5 problems where NTR1 was superior to NTR2, (v) 25 problems where NTRW was superior to NTR2, (vi) 10 problems where NTR2 was superior to NTRW. For problems 12, 18, 24, 30, and 36 especially, the iterations of four algorithms are similar while nf of NTRW are much less than the others. It means that the number of subproblem evaluations of NTRW is much less than the others. Therefore, our self-adaptive technique is efficient. For problems 10, 20, 27, 32, and 38, NTRW is superior to the others clearly. And cpu of NTRW is less than the others. So the performance of our algorithm is better than the others. In Table 3, we compare 25 middle-scale problems of the four algorithms. There are 12 problems that show NTRW is much superior than the others, 7 problems that show the performance of the four algorithms is similar, and only 4 problems that show NTRW is bad. InTable4,wecompare10large-scaleproblemsofthe four algorithms. There are 5 problems that show NTRW is much superior than the others, 4 problems that show the performance of the four algorithms is similar, and only 1 problem that shows NTRW is bad. Note that, for problems 33 and 35, the iteration of our algorithm is similar to NTR2, whilethecputimeismuchmorethanntr2.exponential function called in Matlab environment maybe consume more time, which is contained in L-function in (32). Further result is shown in Figures 1 and 2, which is characterized by means of performance profile proposed in [28]. The performance ratio q(τ) is the probability for solver s for the test problems, where a log-scale ratio is not greater than the factor τ. More details are founded in [28]. As we can see from Figures 1 and 2, NTRW is obviously superior than NTR1 and NTR2 in the number of iterations and function evaluations. NTRW is superior than the other three algorithms in the number of function evaluations. 5. Conclusion This paper presents a nonmonotone weighting self-adaptive trust region algorithm for unconstrained nonconvex optimization. The new algorithm is very simple and easily implemented. The convergence properties of the method

5 Discrete Dynamics in Nature and Society 5 Table 1: Test problems. Number Problem name 1 Helicalvalleyfunction 2 Biggs EXP6 function 3 Gaussian function 4 Boxfunction 5 Variable dimension function 6 Watson function 7 PenaltyfunctionI 8 PenaltyfunctionII 9 Brown badly scaled function 10 Brown and Dennis function 11 Gulf function 12 Extended Rosenbrock function 13 Beale function 14 Wood function 15 Chebyquad function 16 Boundary value function 17 Separable cubic function 18 Powell singular function 19 Linear function, full rank 20 Linear function, rank 1 21 FLETCHCR function 22 BDQRTIC function 23 TRIDIA function 24 ARGLINB function 25 ARWHEAD function 26 NONDIA function 27 NONDQUAR function 28 Generalized Rosenbrock function 29 Broyden tridiagonal function 30 Allgower function 31 EG2 function 32 CURLY20 function 33 LIARWHD function 34 POWER function 35 ENGVAL1 function 36 ARGLINC function 37 NONSCOMP function 38 VARDIM function 39 QUARTC function 40 Extended DENSCHNB function 41 Extended DENSCHNF function 42 DIXON3DQ function 43 BiGGSB1 function 44 Nearly separable function 45 Schittkowski function Discrete integral equation function 47 DQDRTIC function 48 EDENSCH function 49 Bdexp function 50 COSINE function 51 HIMMELBG function Table 2: Numerical comparisons for some small-scale test problems. P/Dim BTR NTR1 NTR2 NTRW iter/nf iter/nf iter/nf iter/nf 1/3 32/43 31/51 30/51 38/45 2/6 38/45 37/42 36/41 40/41 3/3 3/4 3/4 3/4 3/4 4/3 33/47 33/46 33/46 38/41 5/5 21/30 11/28 11/28 14/19 6/5 23/31 23/32 24/35 24/27 7/5 188/240 19/36 19/36 65/76 8/5 18/26 18/26 18/26 8/11 9/2 69/ /117 10/4 72/92 219/ /261 44/57 11/3 9/10 9/10 9/10 9/10 12/6 41/62 44/76 44/72 51/58 13/2 12/15 17/20 17/20 15/17 14/4 34/50 140/254/ 140/ /125 15/5 8/10 8/10 8/10 9/11 16/10 23/27 23/34 23/28 20/22 17/10 8/9 8/9 8/9 8/9 18/4 32/44 26/36 33/43 27/31 19/10 3/4 3/4 3/4 3/4 20/10 7/20 14/48 14/48 3/8 21/10 38/58 43/94 44/98 49/62 22/10 35/48 35/79 35/80 39/50 23/10 33/44 32/64 32/57 52/64 24/10 25/10 9/13 9/14 9/14 11/15 26/10 43/80 24/58 24/58 76/93 27/10 98/102 98/105 98/105 85/93 28/20 110/ / / /147 29/20 30/56 30/82 30/82 57/72 30/20 65/93 69/138 75/127 64/86 31/20 22/24 22/24 24/26 23/25 32/20 64/145 63/131 65/81 33/20 30/58 30/72 30/72 40/51 34/20 74/107 77/157 77/145 92/115 35/20 28/55 26/77 25/70 42/50 36/30 37/30 346/ / / /321 38/30 60/70 39/30 17/18 17/18 17/18 19/20 40/30 8/9 8/9 8/9 8/9 41/30 37/76 38/100 38/100 47/57 42/30 37/64 34/68 36/69 53/63 43/30 36/59 37/73 38/79 52/62 means that the algorithm reaches 500 iterations.

6 6 Discrete Dynamics in Nature and Society Table 3: Numerical comparisons for some middle-scale test problems. P/Dim BTR NTR1 NTR2 NTRW iter/nf /cpu iter/nf /cpu iter/nf /cpu iter/nf /cpu 5/500 45/114/ /534/ /1192/ /92/2.85 7/ /228/ /361/ /700/ /99/ /500 6/11/ /36/0.78 9/23/0.44 7/8/ /500 8/10/0.64 8/10/0.69 8/10/0.75 9/11/ / /500/ /450/ / /7001/ /11668/ /8214/ /500 16/31/ /28/ /28/ /35/ /500 21/40/ /552/ /310/ /15/ / /236/ /471/ /358/ /142/ /500 13/24/ /30/ /41/ /55/ /500 26/38/ /56/ /80/ /43/ /500 42/96/ /144/ /134/ /66/ /500 66/224/ /227/ /65/ /500 62/135/ /666/ /1182/ /92/ /500 14/18/ /18/ /18/ /14/ /500 8/13/ /16/ /16/0.70 7/9/ /500 37/183/ /33/ / /280/ /3879/ /62/ /500 49/108/ /457/ /852/ /96/ /500 76/208/ /486/ /500 67/116/ /224/ /162/ /500 35/80/ /131/ /112/ /500 18/19/ /19/ /19/ /25/ /500 90/154/ /127/ /204/ /71/ /500 7/8/0.45 7/8/0.39 7/8/ /29/1.50 means that the algorithm reaches 5000 iterations Variable q(τ) Variable q(τ) Variable τ Variable τ BTR NTR1 NTR2 NTRW BTR NTR1 NTR2 NTRW Figure 1: Performance profile comparing the number of iterations. Figure 2: Performance profile comparing the number of function evaluations. are established under reasonable assumptions. Numerical experiments show that the new algorithm is quite robust and effective, and the numerical performance is comparable to or better than that of other trust region algorithms in the same frame. Conflict of Interests The authors declare that they have no conflict of interests.

7 Discrete Dynamics in Nature and Society 7 Table 4: Numerical comparisons for some large-scale test problems. P/Dim BTR NTR1 NTR2 NTRW iter/nf /cpu iter/nf /cpu iter/nf /cpu iter/nf /cpu 26/ /54/ /1072/ /358/ /260/ / /75/ /3148/ /929/ /582/ / /12/ /12/ /12/3.16 8/10/ / /13/ /13/ /13/ /9/ / /35/ /65/ /199/ /30/ / /115/ /86/ /59/ /95/ / /40/ /70/ /87/ /50/ / /186/ /197/ /186/ /69/ / /57/ / /123/ / /202/ /3335/ /47/ / /45/ / /22/ /20/ /20/ /20/ / /23/ /22/ /22/ /31/ / /23/ /22/ /22/ /31/ /1000 7/13/2.45 8/18/2.45 8/18/2.54 8/11/ /3000 7/15/ /21/ /21/ /12/ /5000 7/14/ /2482/ /29/ /13/ / /196/ /38/ / /236/ /15/ / /246/ /30/ / /327/ /327/ /113/ / /472/ /98/ / /95/ / /89/ /143/ /158/ /73/ / /77/ /87/ / /250/ /66/ means that the algorithm does not end in 30 minutes; means that the algorithm reaches 5000 iterations. Acknowledgment This research is partly supported by Chinese NSF under Grant no References [1] L. Grippo, F. Lampariello, and S. Lucidi, A nonmonotone line search technique for Newton s method, SIAM Journal on Numerical Analysis,vol.23,no.4,pp ,1986. [2]H.C.ZhangandW.W.Hager, Anonmonotonelinesearch technique and its application to unconstrained optimization, SIAM Journal on Optimization, vol. 14, no. 4, pp , [3] Y.H.DaiandD.C.Xu, Anewfamilyoftrustregionalgorithms for unconstrained optimization, Computational Mathematics,vol.21,pp ,2003. [4] A. R. Conn, N. I. M. Gould, and P. L. Toint, Trust Region Methods, vol.1ofmps/siam Series on Optimization, SIAM, Philadelphia, Pa, USA, [5] J.H.Fu,W.Y.Sun,andR.J.DeSampaio, Anadaptiveapproach of conic trust region method for unconstrained optimization problems, Applied Mathematics & Computing, vol. 19,no.1-2,pp ,2005. [6] Y.L.Lu,W.Y.Li,M.Y.Cao,andY.T.Yang, Anovelself-adaptive trust region algorithm for unconstrained optimization, Journal of Applied Mathematics, vol.2014,articleid610612,8pages, [7] N. I. M. Gould, D. Orban, A. Sartenaer, and P. L. Toint, Sensitivity of trust-region algorithms to their parameters, 4OR,vol.3,no.3,pp ,2005. [8] L. Hei, A self-adaptive trust region algorithm, Computational Mathematics,vol.21,no.2,pp ,2003. [9] A. Sartenaer, Automatic determination of an initial trust region in nonlinear programming, SIAMJournalonScientific Computing,vol.18,no.6,pp ,1997. [10] Z.-J. Shi and J.-H. Guo, A new trust region method for unconstrained optimization, Computational and Applied Mathematics,vol.213,no.2,pp ,2008. [11] Z. Y. Sang and Q. Y. Sun, A self-adaptive trust region method with line search based on a simple subproblem model, Journal of Computational and Applied Mathematics, vol.232,no.2,pp , [12] J. M. B. Walmag and E. J. M. Delhez, A note on trust-region radius update, SIAMJournalonOptimization, vol. 16,no. 2, pp , 2005.

8 8 Discrete Dynamics in Nature and Society [13] Z. S. Yu and Q. Li, A self-adaptive trust region method for the extended linear complementarity problems, Applications of Mathematics,vol.54,no.1,pp.53 65,2009. [14] X. Zhang, J. Zhang, and L. Liao, An adaptive trust region method and its convergence, Science in China. Series A. Mathematics,vol.45,no.5,pp ,2002. [15] M. Ahookhosh and K. Amini, A nonmonotone trust region method with adaptive radius for unconstrained optimization problems, Computers & Mathematics with Applications, vol. 60, no. 3, pp , [16] Z. C. Cui and B. Y. Wu, A new modified nonmonotone adaptive trust region method for unconstrained optimization, Computational Optimization and Applications,vol.53,no.3,pp , [17] J. H. Fu and W. Y. Sun, Nonmonotone adaptive trust-region method for unconstrained optimization problems, Applied Mathematics and Computation, vol.163,no.1,pp , [18] Z. J. Shi and S. Q. Wang, Nonmonotone adaptive trust region method, European Operational Research,vol.208,no. 1, pp , [19] Z. Sang and Q. Sun, A new non-monotone self-adaptive trust region method for unconstrained optimization, Applied Mathematics and Computing,vol.35,no.1-2,pp.53 62, [20] J.-L. Zhang and X.-S. Zhang, A nonmonotone adaptive trust region method and its convergence, Computers & Mathematics with Applications, vol. 45, no , pp , [21] J. Zhang, K. C. Zhang, and S. J. Qu, A nonmonotone adaptive trust region method for unconstrained optimization based on conic model, Applied Mathematics and Computation, vol. 217, no. 8, pp , [22] J.T.Mo,C.Y.Liu,andS.C.Yan, Anonmonotonetrustregion method based on nonincreasing technique of weighted average of the successive function values, Computational and Applied Mathematics,vol.209,no.1,pp ,2007. [23] J. J. Moré, B. S. Garbow, and K. E. Hillstrom, Testing unconstrained optimization software, ACM Transactions on Mathematical Software,vol.7,no.1,pp.17 41,1981. [24] N. I. M. Gould, D. Orban, and P. L. Toint, CUTEr and SifDec: a constrained and unconstrained testing environment, revisited, ACMTransactionsonMathematicalSoftware,vol.29,no.4,pp , [25] N. Andrei, An unconstrained optimization test functions collection, Advanced Modeling and Optimization,vol.10,no.1, pp , [26] J. Nocedal and S. T. Wright, Numerical Optimization, Springer, Berlin, Germany, [27] W. Sun and Y. Yuan, Optimization Theory and Methods. Nonlinear Programming, Springer, New York, NY, USA, [28] E. D. Dolan and J. J. Moré, Benchmarking optimization software with performance profiles, Mathematical Programming, vol.91,no.2,pp ,2002.

9 Advances in Operations Research Advances in Decision Sciences Applied Mathematics Algebra Probability and Statistics The Scientific World Journal International Differential Equations Submit your manuscripts at International Advances in Combinatorics Mathematical Physics Complex Analysis International Mathematics and Mathematical Sciences Mathematical Problems in Engineering Mathematics Discrete Mathematics Discrete Dynamics in Nature and Society Function Spaces Abstract and Applied Analysis International Stochastic Analysis Optimization