A Derivative-Free Algorithm for Constrained Global Optimization based on Exact Penalty Functions

Transcription

1 Noname manuscript No. (will be inserted by the editor) A Derivative-Free Algorithm for Constrained Global Optimization based on Exact Penalty Functions Gianni Di Pillo Stefano Lucidi Francesco Rinaldi the date of receipt and acceptance should be inserted later Abstract Constrained global optimization problems can be tackled by using exact penalty approaches. In a preceding paper, we proposed an exact penalty algorithm for constrained problems which combines an unconstrained global minimization technique for minimizing a non-differentiable exact penalty function for given values of the penalty parameter, and an automatic updating of the penalty parameter that occurs only a finite number of times. However, in the updating of the penalty parameter, the method requires the evaluation of the derivatives of the problem functions. In this work, we show that an efficient updating can be implemented also without using the problem derivatives, in this way making the approach suitable for globally solving constrained problems where the derivatives are not available. In the algorithm, any efficient derivative-free unconstrained global minimization technique can be used. In particular, we adopt an improved version of the DIRECT algorithm. In addition, to improve the performances, the approach is enriched by resorting to derivative-free local searches, in a multistart framework. In this context, we prove that, under suitable assumptions, for every global minimum point there exists a neighborhood of attraction for the local search. An extensive numerical experience is reported. Keywords Nonlinear Optimization Global Optimization Exact Penalty Functions Derivative-Free Minimization DIRECT Algorithm This work has been partially funded by the UE(ENIAC Joint Undertaking) in the MODERN project (ENIAC ). G. Di Pillo, S. Lucidi (corresponding author) Dipartimento di Ingegneria Informatica, Automatica e Gestionale A. Ruberti, Sapienza Università di Roma, Via Ariosto 25, Rome (Italy) dipillo@dis.uniroma1.it, lucidi@dis.uniroma1.it F. Rinaldi Dipartimento di Matematica, Università di Padova, Via Trieste 63, Padova (Italy) rinaldi@math.unipd.it

2 2 Gianni Di Pillo et al. Mathematics Subject Classification (2000) 90C26 90C30 90C56 1. Introduction The search for global rather than local solutions of optimization problems has been receiving growing attention in many fields like Engineering, Economics and Applied Sciences. Many real-world problems in these fields are of the constrained type, while most of the algorithms in global optimization are for unconstrained or simply bound-constrained problems. For the unconstrained case, many algorithmic approaches, either deterministic or probabilistic, have been developed (see [1 7] and references therein). The more difficult case of nonlinearly-constrained global optimization problems has been investigated more recently and various approaches have been described in, e.g., [1, 7, 8]. In this context, a particular emphasis is given to the use of some Augmented Lagrangian function to deal with the general constraints (see [9 11]). However, the Augmented Lagrangian approach is sequential in nature and requires, in principle, an infinite number of global minimizations. Therefore, we adopt an exact penalty approach so that the number of unconstrained global minimizations needed to globally solving the original constrained problem results to be finite. Moreover, by adopting an exact penalty approach we may take advantage of efficient derivative-free unconstrained algorithms. In this way, we can deal with constrained global optimization problems, where the derivatives of the functions are not available, as happens, for instance, in black-box simulation-based optimization, a topic that is receiving growing interest in the literature (see [12] and references therein). In particular, based on the approach described in [13], we develop a new strategy having the two following distinguishing features: - differently from [13], derivatives of the objective and constraints functions are not required for the penalty parameter updating; - a local search phase is used for speeding-up the convergence to a global solution of the constrained problem: indeed, we prove that, under suitable assumptions, for every global minimum point there exists a neighborhood of attraction for the local search. The paper is organized as follows. In Section 2, we give some preliminary results that will be used throughout the paper. In section 3, we describe the Global Optimization Framework and we give some convergence results. In Section 4, we discuss the local search properties. In Section 5, we report an extended numerical experience using the proposed algorithm either alone, or combined with a derivative-free local minimization technique, and we discuss on the efficiency of our approach. Finally, we draw some conclusions in Section 6.

3 A Derivative-Free Algorithm for Constrained Global Optimization 3 2. Preliminaries In this paper, we are interested in the global solution of the general nonlinear programming problem: min f(x) s.t. g(x) 0, l x u (1) where f : R n R, g : R n R p, l,u R n both finite, and we assume that f and g are continuously differentiable functions. We assume that no global information (convexity, Lipschitz constants,...) on the problem is available. To simplify notations, we denote the bound constraints as [ ] l x s(x) := 0. x u We denote by F the feasible set of problem (1): F := {x R n : g(x) 0, s(x) 0}. In order to solve problem (1), we adopt an exact penalty function approach like that described in [13]. In particular, We use a non-differentiable penalty function having the following form: where 1 < q <, and P q (x;ε) = f(x)+ 1 ε [max{0,g(x)},max{0, s(x)} ] q (2) with α j > 0, j = 1,...,2n. s j (x) = s j(x) α j s j (x), As pointed out in [13], even if, in general, the penalty function P q (x;ε) is non-differentiable, it turns out to be continuously differentiable at infeasible points. We denote by D the set Then, we consider the problem D := {x R n : s j (x) α j, j = 1,...,2n}. We make use of the following Assumptions: Assumption 2.1 D is a compact set. min x D P q(x;ε). (3) Assumption 2.2 The Mangasarian-Fromovitz Constraint Qualification (MFCQ) holds at every global solution x of Problem (1).

4 4 Gianni Di Pillo et al. We note that Assumption 2.1 is common in global optimization and is always satisfied in real-world problems and Assumption 2.2 is only needed in global minimizers, therefore they are very weak assumptions. In the following theorem (see [13]), we recall the main exactness property of the penalty function P q, which is at the basis of the proposed approach: Theorem 2.1 Under Assumption 2.2, there exists a threshold value ε such that, for any ε ]0, ε], every global solution of Problem (1) is a global solution of Problem (3), and conversely. Remark 2.1 We note that by construction, lim P q(x;ε) = +, x D where D denotes the boundary of D. Therefore Problem (3) has global solutions contained only in the interior of D, so that determining a global solution of Problem (3) is essentially an unconstrained problem. Remark 2.2 Theorem 2.1 implies that, for a sufficiently small value ε, Problem (3) has only unconstrained global minimizers. Remark 2.3 The global solution of Problem (3) can be obtained by using any method for unconstrained global optimization. On these bases, in [13] we were able to describe an algorithm for the global solution of the constrained Problem (1), by combining any algorithm that globally solves the unconstrained global optimization Problem (3) with an automatic updating rule for the penalty parameter which makes use of first order derivatives of the problem functions. In the paper, given a vector v R n, we denote by v its transpose, by v q its q-norm, and by v + = max{0,v} the n-vector (max{0,v 1 },...,max{0,v n }). 3. Derivative-Free Exact Penalty Global Optimization Algorithm In this section, we introduce the DF-EPGO (derivative-free Exact Penalty Global Optimization) algorithm model for finding a global solution of Problem (1) using the exact penalty function (2), and we analyze its convergence properties. We assume that F k R n approximates the gradient of the objective function f calculated in x k, in the sense that there exists a value τ k 0 such that: F k f(x k ) q M 1 τ k, (4) with M 1 > 0. In the same way, we assume that V k approximates the gradient of the function [g + (x), s + (x) ] q (5)

5 A Derivative-Free Algorithm for Constrained Global Optimization 5 calculated in x k / F, in the sense that there exists a value τ k 0 such that: with M 2 > 0. V k [g + (x k ), s + (x k ) ] q q M 2 τ k, (6) In Figure 1 we report the scheme of the DF-EPGO Algorithm, where we use the following condition to check if an updating of the penalty parameter is timely: ε k ( F k q + [g + (x k ), s + (x k ) ] q ) > ρ V k q. (7) At iteration k of the algorithm, we first calculate x k as a δ k -global minimizer of Problem (3), defined as the point x k such that P q (x k,ε k ) P q (x,ε k )+δ k, x D. In principle, any global unconstrained optimization method can be used to compute aδ k -globalminimizerx k.then, wecheckfeasibilityofx k forproblem (1) and, if x k is not feasible, we check if an updating of the penalty parameter is timely by means of condition (7). Finally, we reduce the value of δ k in order to find a better approximation of the global solution of Problem (3) and the valueτ k toobtainabetter approximationofthe problemfunctionsderivatives. We point out that the DF-EPGO is a conceptual algorithm model; in practice, as usual in global optimization, it will be stopped when a δ k -global minimizer x k which is reputed a good approximation of a global minimizer x is found. In order to state the convergence properties of the algorithm, we preliminary prove the following Lemma. Lemma 3.1 Every accumulation point x of a sequence {x k } produced by DF- EPGO Algorithm belongs to the set F. Proof We consider two different cases: Case 1) an index k exists such that, for any k k, ε k = ε: We assume, by contradiction, that there exists an accumulation point x / F. We have, for k sufficiently large, that the following inequality holds: [g + (x k ), s + (x k ) ] q ρ ε [g+ (x k ), s + (x k ) ] q q f(x k ) q +(M 1 +M 2 )τ k. Let x be the limit of the subsequence {x k } K. Taking the limit for k on both sides, as τ k 0, we get [g + ( x), s + ( x) ] q ρ ε [g+ ( x), s + ( x) ] q q f( x) q. (8) From this point, the proof follows the same steps of the proof of Lemma 1 in [13].

6 6 Gianni Di Pillo et al. begin iteration k, ρ (0,1), ε k,δ k,τ k > 0, σ,θ,ξ (0,1) Compute a δ k -global minimizer x k of Problem (3) Set ε k+1 = ε k yes x k F no Set ε k+1 = σε k Condition (7) Set ε k+1 = ε k. yes no Set δ k+1 = θδ k, τ k+1 = ξτ k end iteration k Fig. 1 Scheme of DF-EPGO Algorithm (iteration k). Case 2) {ε k } 0: The proof is a verbatim repetition of the proof of Case 2 of Lemma 1 in [13]. Then we can state the first convergence result. Theorem 3.1 Every accumulation point x of a sequence {x k } produced by DF-EPGO Algorithm is a global minimizer of Problem (1). Proof See proof of Theorem 2 in [13].

7 A Derivative-Free Algorithm for Constrained Global Optimization 7 In next theorem, we prove that, if Assumption 2.2 holds, the penalty parameter ε is updated a finite number of times. Theorem 3.2 Let us assume that Assumption 2.2 holds. Let {x k } and {ε k } be the sequences produced by DF-EPGO Algorithm. Then an index k and a value ε > 0 exist such that, for any k k, ε k = ε. Proof By contradiction, let us assume {ε k } 0. Then, from the scheme of the DF-EPGO Algorithm, there exists a subsequence {x k } K such that, for all k K, x k / F and the test is satisfied. By rewriting (9) as ε k ( F k q + [g + (x k ), s + (x k ) ] q ) > ρ V k q (9) ε k ( f(x k ) q +M 1 τ k + [g + (x k ), s + (x k ) ] q ) > ρ [g + (x k ), s + (x k ) ] q q M 2 τ k, and considering the fact that τ k 0, we obtain: lim k K, k [g+ (x k ), s + (x k ) ] q = 0. (10) From this point, the proof follows the same steps of the proof of Theorem 3 in [13]. 4. Enriching the Global Search by a Derivative-Free Local Approach In practice, we can enrich the global search by means of a derivative-free local minimization algorithm using as starting point the δ k -global minimizer x k obtained at iteration k by the DF-EPGO algorithm. In this section, we motivate analytically this enrichment by proving that, under reasonable assumptions, the sequence generated by the derivative-free local algorithm is attracted by a global solution of Problem (3). To simplify notation, we will develop the analysis making reference to a locally Lipschitz function φ : R n R and to the problem min x R n φ(x). (11) Let us denote by d φ(x,d) the Clarke-directional derivative of φ at x R n in the direction d R n : d φ(y +td) φ(y) φ(x,d) := limsup ; y x t 0 t + then we can recall the definition of stationary point for the nonsmooth Problem (11):

8 8 Gianni Di Pillo et al. Definition 4.1 x is a Clarke stationary point for φ, iff the following condition is satisfied: d φ(x,d) 0, d R n. (12) An alternative definition of Clarke stationary point for Problem (11) can by given by using the Clarke subdifferential φ(x) of φ at xl given as: φ(x) = conv{v R n {x k } : x k T,x k x, φ(x k ) v}, T being the subset of R n where φ is differentiable. Even if we will not use the alternative definition, we will make use of the Clarke subdifferential in order to exploit some second order properties of function φ. To this aim we need some additional notations and definitions. Definition 4.2 The sequence {x k } converges to x in the direction d, denoted by x k d x, iff x k x and the sequence { (xk x) d x k } converges to x d. Then, we can denote by d φ(x) the subset of φ(x), defined as follows: d φ(x) := {g : x k and g k φ(x k ) such that x k d x and g k g}. Definition 4.3 Let g d φ(x). Then φ (x,g,d) is defined as the infimum of all numbers 1 lim inf (t k ) 2{φ(xk ) φ(x) g T (x k x)} (13) taken over all triples of sequences {x k }, {g k } and {t k }, for which t k > 0 for each k and {t k } converges to 0, {x k } converges to x and (x k x)/t k converges to d, {g k } converges to g with g k in φ(x k ) for each k. Moreover,let us denote by d + φ(x,d) the lower Dini-directional derivative at x R n in the direction d R n : d + φ(x,d) := lim inf d d t 0 φ(x+td ) φ(x). t We can now introduce the definition of second order Dini stationary point for Problem (11). Definition 4.4 x is a second order Dini stationary point for φ, iff the following conditions are satisfied: (i) d + φ(x,d) 0 for all unit vectors d R n ; (ii) φ (x,0,d) β > 0 for all unit vectors d Rn for which d + φ(x,d) = 0. We state a preliminary result [14] that will be used to prove the main result:

9 A Derivative-Free Algorithm for Constrained Global Optimization 9 Lemma 4.1 Suppose that φ(x) φ( x) for all x B( x,δ). Let 0 d R n, t > 0, α > 1 and 0 < η < (α d ) 2 such that φ( x+td) φ( x) tη and t( d +η 1/2 ) < δ. Then there exist z x in R n and g φ(z) such that (i) z x td tη 1/2 α 1 (< tη 1/2 ), (ii) φ(z) φ( x+td) and (iii) g αη 1/2. Now we can prove the following result: Proposition 4.1 Let x R n be a local minimum point of φ which satisfies the conditions for being a second order Dini stationary point. Then there exists a neighborhood B( x,η) and a value γ > 0 such that x x 2 1 (φ(x) φ( x)), x B( x,η). γ Proof Suppose, by contradiction, that η k > 0 and γ k > 0 there exists a point x k B( x,η k ) such that Let x k x 2 > 1 γ k(φ(xk ) φ( x)). (14) x k x x k x = dk, we consider the sequences {γ k },{η k },{d k } such that γ k 0, η k 0, d k d. Now, by (14) and (i) of Definition 4.4, when k is large enough, we can write the following x k x > 1 (φ(x k ) φ( x)) γ k x k (φ(xk ) φ( x)) x x k d + φ( x,d) 0. (15) x Taking the limit for k, we have Now, we can denote d + φ( x,d) = 0. (16) d k = xk x x k x and t k = x k x, (17) and by (14), we can write φ( x+t k d k ) < φ( x)+γ k (t k ) 2. (18) Using Lemma 4.1 with α = 2 and η = γ k t k, there exists a sequence {z k } such that

10 10 Gianni Di Pillo et al. (i) z k x t k d k tk (γ k t k ) 1/2, 2 (ii) φ(z k ) φ( x+t k d k ) and (iii) g k 2(γ k t k ) 1/2. By using (i) and (iii), we have that By using (ii) and (14), we have Then, we can write lim k zk = x, (19) z k x lim k t k = lim k dk = d, (20) z k x lim k t k = 1, (21) lim k gk = g = 0 d φ( x). (22) φ(z k ) φ(x k ) < φ( x)+γ k (t k ) 2. (23) φ(z k ) φ( x) z k x 2 z k x 2 (t k ) 2 γ k. By using (21) and the fact that γ k 0, we have φ(z k ) φ( x) lim k z k x 2 0. (24) Finally, taking into account (ii) of Definition 4.4, (13) and (16), we get the following contradiction φ(z k ) φ( x) g T (z k x) 0 lim k z k x 2 f ( x,0,d) β > 0. Now, we extend to the case of locally Lipschitz functions a result established in [15]. Theorem 4.1 Let φ be a locally Lipschitz function and {x k } be a sequence of points generated by an iterative method satisfying the following condition x k+1 = x k +β k d k φ(x k+1 ) φ(x k ) ν(β k d k ) 2, for all k, where ν > 0, and such that every accumulation point is a Clarke stationary point. Then, for every global minimum point x of φ(x), which is an isolated Clarke stationary point and a second order Dini stationary point, there exists an open set L containing x such that, if x k L for some k 0, then we have:

11 A Derivative-Free Algorithm for Constrained Global Optimization 11 - x k L for all k k; - {x k } x. Proof Let B(x,δ 2 ) be a neighborhood introduced in Proposition 4.1, where we have φ(x k) φ(x ) γ x k x 2, x B(x,δ 2 ). (25) We consider now the following open set: L = {x B(x,δ),φ(x) < φ(x )+ δ2 γν }, (26) 2(2ν +γ) where δ = min{δ 1, δ 2 }. Now we prove that, if x k L for some k 0, then x k L for all k k and {x k } x. Recalling the hypothesis φ(x k+1 ) φ(x k ) ν(β k ) 2 d k 2 we have (β k) 2 d k 2 1 ν (φ(x k) φ(x k+1 )) (27) 1 ν (φ(x k) φ(x )), where the second inequality follows from the hypothesis that x is a global minimum. By using (25) and (27), we have x k+1 x 2 = x k x +β kd k 2 (28) 2 x k x 2 +2(β k) 2 d k 2 4 γ (φ(x k) φ(x ))+ 2 ν (φ(x k) φ(x )) = 2(2ν +γ) (φ(x k) φ(x )). γν Recalling that x k L we have φ(x k) φ(x ) < δ2 γν 2(2ν +γ). Combining the above inequality with (28), we obtain x k+1 x < δ. (29) Furthermore, by using again the hypothesis φ(x k+1 ) φ(x k ) ν(β k ) 2 d k 2, we have that φ(x k+1 ) φ(x k ) for all k and hence φ(x k+1 ) φ(x k) < φ(x )+ δ2 γν 2(2ν +γ). (30) Using the above two inequalities, it follows that x k+1 B(x,δ) and similarly x k L for all k k. In particular, using the fact that L B(x,δ), then x k B(x,δ) for all k k. Since, by definition, x k L C = {x R n : x x δ 1 },

12 12 Gianni Di Pillo et al. and the set C is compact, the sequence {x k } will have at least one limit point which must be a stationary point of Problem (11). Since within B(x,δ) the only Clarke stationary point is x, then x k x. As in the derivative-free local search used to enrich the global algorithm we minimize the exact penalty function P q, which is a locally Lipschitz function, we can apply to it all the results given for φ in this section. Then, recalling Remark 2.1, we have that, under the assumptions of Theorem 4.1 there exists a neighborhood L of a global solution of Problem (3), where the sequence produced by the derivative-free local algorithm applied to P q remains and is attracted by the global solution. 5. Numerical Results In the numerical experimentation of the derivative-free exact penalty approach we used: - the set A of 20 small dimensional test problems reported in [9], where a global optimization algorithm for general constrained problems, which uses first and second order derivatives of the problem functions, is described; - the set B of 97 problems from the GLOBALLib collection of COCONUT [16] with dimension n 10. These choices are motivated by the fact that derivative-free approaches cannot tackle large dimensional problems. All test problems are of the kind min f(x) s.t. g(x) 0, h(x) = 0, l x u (31) with h(x) : R n R t, with t n. Of course, the DF-EPGO Algorithm can be easily adapted in order to take into account also the equality constraints h(x) = 0. In the practical application of the proposed approach, we must choose: - A value q for the vector norm. We chose q = 2. - A method for approximating the derivatives. We chose the simple forward difference: P q (x k ;ε) x i P q(x k +γe i ;ε) P q (x k ;ε) γ k, (32) with γ k+1 = ξγ k,γ 0 = 10 6 and ξ = 0.9. Of course, by this choice the conditions (4) and (6) are satisfied. - A method for globally solving the problem with bounded feasible set. In principle, any method could be used. In our numerical experience we used: 1) the DIRECT method as improved in [17]; 2) the DIRECT method as improved in [17], enriched by a derivative-free local search phase (DFNS method) based on the same exact penalty function (2) used in the global search [18].

13 A Derivative-Free Algorithm for Constrained Global Optimization 13 - Values for the data of the exact penalty approach. We set ε 0 = 0.1, σ = 0.9 and ρ = 0.5. Our choice in case 1) is motivated by the the fact that DIRECT is reputed to be one of the most efficient methods for problems with simple bounds. DIRECT belongs to the class of derivative-free deterministic methods, which perform the sampling of the objective function in points generated by the previous iterates of the method. The derivative-free algorithm used in case 2) seems to be very effective when first order information is not available. The practical computation of a δ k -global minimizer x k of Problem (3) has been performed in case 1) using as stopping rule the standard stopping rule of the DIRECT Algorithm [19]. Namely, the DIRECT algorithm stops when the diameter of the hyperrectangle containing the best found value of the objective function is less than a given threshold ζ k. We set ζ 0 = and r k+1 = θζ k with θ = 0.1. In case 2) the local search phase starts from the point obtained bythedirectalgorithmasdescribedbeforeandstopsassoonasthestepsize α in [18] is lower than As concerns the checking of feasibility for the new point x k, we consider x k feasible if the constraints violation cv = max{ g + (x), h(x) }, is lower than or equal to 10 3, infeasible otherwise. The numerical experimentation was carried out on an Intel Core 2 Duo 3.16 GHz processor with 3.25 GB RAM. DF-EPGO and DFNS Algorithm were implemented in Fortran 90 (double precision). The results for the problem set A are collected in Tables 1 and 2, which refer respectively to case 1) and 2). In both tables we report the name of the problem, the number n of variables, the number me and mi of equality and inequality constraints (excluding simple bounds), the CPU time required to attain the stop condition, the optimal function value f(x ), the constraints violation cv, and the reference value f reported in [9]. In Table 1, we note that the values f(x ) and f are practically the same for all problems 3-12, 15 and 16, f(x ) is lower than f for problem 2b, while f is lower than f(x ) for problems 1, 2a, 2c, 2d, 13 and 14. These results can be considered satisfactory considering that the results reported in [9] are obtained using first and second order derivatives. Furthermore, we point out that the results reported in Table 1 are comparable with the ones reported in [13] where first order derivatives of the problem functions are used. In particular, the values of f(x ) and cv are practically the same, while there is a slight increase of the CPU time, as expected. In Table 2, we note that the values f(x ) are practically the same as the ones reported in Table 1, but we have better results in terms of constraints violation. The results for the problem set B are collected in Tables 3-6. Again in Tables 3-6 we report the name of the problem, the number n of variables, the number

14 14 Gianni Di Pillo et al number of feasible solutions constraints violation x 10 3 Fig. 2 Performance Comparison between the DF-EPGO Algorithm (dotted line) and the DF-EPGO+DFNS Algorithm (dash-dot line) on the test set B: number of feasible solutions. me and mi of equality and inequality constraints (excluding simple bounds), the CPU time required to attain the stop condition, the optimal function value f(x ), the constraints violation cv. In order to synthesize the results we make reference to Figures 2 and 3. The plot in Figure 2 gives on the y-axis the number of obtained solutions whose constraints violation is smaller or equal than the value given in the x-axis. The plot in Figure 3 gives on the y-axis the number of obtained solutions whose objective function value has a relative error, given by f(x ) f max{1, f }, where f is the best known value reported in [16], smaller or equal than 0.01 and whose constraints violation is smaller or equal than the value given in the x-axis. From both Figures, we note that the DF-EPGO algorithm guarantees quite good performances in terms of feasibility and optimality and that these performances are much improved by combining the DF-EPGO algorithm with a local search phase based on the DFNS algorithm. 6. Conclusions In this paper, we have shown that the approach for constrained global optimization developed in [13], and based on the use of a non-differentiable exact

15 A Derivative-Free Algorithm for Constrained Global Optimization number of best feasible solutions constraints violation x 10 3 Fig. 3 Performance Comparison between the DF-EPGO Algorithm (dotted line) and the DF-EPGO+DFNS Algorithm (dash-dot line) on the test set B: number of best feasible solutions. penalty function, can be fruitfully modified in such a way that the derivatives of the problem functions are not required, thus making the approach derivative-free. Moreover, the approach has been enriched by a local search phase that uses the same penalty function used in the global search phase. This is motivated by the fact that the local searches are performed in such a way that the sequence produced by the algorithm, under suitable assumptions, is attracted by any global minimum point. Avoiding the use of derivatives makes the approach suitable to deal with realworld problems where the analytical expressions of the problem functions are not available, as happens for instance in black-box simulation-based optimization. The extensive numerical experimentation, which has been carried out on a well-known set of problems, shows that the new method is efficient and the results are comparable with those obtained by other methods which instead rely on the use of derivatives. References 1. Floudas, C.A.: Deterministic global optimization: theory, methods and application. Kluwer Academic Publishers, Dordrecht, The Netherlands (1999) 2. Horst, R., Tuy, H.: Introduction to Global Optimization. Kluwer Academic Publishers, Dordrecht, The Netherlands (2000)

16 16 Gianni Di Pillo et al. 3. Horst, R., Pardalos, P.M., Thoai, N.V.: Global Optimization: Deterministic Approaches. Springer-Verlag, Berlin, Germany (1990) 4. Jones, D.R., Perttunen, C.D., Stuckman, B.E.: Lipschitzian optimization without the lipschitz constant. Journal of Optimization Theory and Applications 79(1), (1993) 5. Jones, D.R.: The direct global optimization algorithm. C. Floudas, P. Pardalos (edit.) Encyclopedia of Optimization pp (2001) 6. Pinter, J.D.: Global optimization in action. Kluwer Academic Publishers, Dordrecht, Boston, London (1996) 7. Tawarmalani, M., Sahinidis, N.V.: Convexification and Global Optimization in Continuous and Mixed-Integer Nonlinear Programming. Kluwer Academic Publishers, Dordrecht, The Netherlands (2002) 8. Neumaier, A., Shcherbina, O., Huyer, W., Vinkó, T.: A comparison of complete global optimization solvers. Mathematical Programming 103, (2005) 9. Birgin, E.G., Floudas, C.A., Martinez, J.M.: Global minimization using an augmented lagrangian method with variable lower-level constraints. Mathematical Programming Serie A 125(1), (2010) 10. Luo, H.Z., Sun, X.L., Li, D.: On the convergence of augmented lagrangian methods for constrained global optimization. SIAM Journal on Optimization 18, (2007) 11. Wang, C.Y., Li, D.: Unified theory of augmented lagrangian methods for constrained global optimization. Journal of Global Optimization 44, (2009) 12. Cassioli, A., Schoen, F.: Global optimization of expensive black box problems with a known lower bound. Journal of Global Optimization (2011). doi : /s Di Pillo, G., Lucidi, S., Rinaldi, F.: An approach to constrained global optimization based on exact penalty functions. Journal of Global Optimization 54(2), (2012) 14. Huang, L.R., Ng, K.F.: Second-order necessary and sufficient conditions in nonsmooth optimization. Mathematical Programming 66(3), (1994) 15. Liuzzi, G., Lucidi, S., Piccialli, V., Sotgiu, A.: A magnetic resonance device designed via global optimization techniques. Mathematical Programming 101, (2004) 16. Neumaier, A., Sam-Haroud, D., Vu, X.H., Nguyen, T.V.: Benchmarking global optimization and constraint satisfaction codes. Ch. Bliek, Ch. Jermann and A. Neumaier (eds.), Global Optimization and Constraint Satisfaction pp (2003) 17. Liuzzi, G., Lucidi, S., Piccialli, V.: An efficient data structure for the direct algorithm. Tech. rep., IASI (2012) 18. Fasano, G., Liuzzi, G., Lucidi, S., Rinaldi, F.: An exact penalty method for constrained lipschitz optimization. Tech. rep., DIIAG (2013) 19. Gablonsky, J.M.: Direct version 2.0, user guide. Tech. rep., North Carolina State University (2001)

17 A Derivative-Free Algorithm for Constrained Global Optimization Tables PROBLEM n me mi CPU time f(x ) cv f problem E problem02a E problem02b E problem02c E problem02d E problem03a E problem03b E problem E problem E problem E problem E problem E problem E problem E problem E problem E problem E problem E problem E problem E Table 1 Performace of the DF-EPGO Algorithm on the test set A. PROBLEM n me mi CPU time f(x ) cv f problem E problem02a E problem02b E problem02c E problem02d E problem03a E problem03b E problem E problem E problem E problem E problem E problem E problem E problem E problem E problem E problem E problem E problem E Table 2 Performace of the DF-EPGO+DFNS Algorithm on test set A).

18 18 Gianni Di Pillo et al. PROBLEM n me mi time val funct viol f ex E ex E chance E circle E dispatch E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex522case E ex522case E ex522case E ex E ex E ex E Table 3 Performace of the DF-EPGO Algorithm on the test set B (Part I).

19 A Derivative-Free Algorithm for Constrained Global Optimization 19 PROBLEM n me mi time val funct viol f ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E himmel E house E least E like E meanvar E mhw4d E process E rbrock E sample E wall E Table 4 Performace of the DF-EPGO Algorithm on the test set B (Part II).

20 20 Gianni Di Pillo et al. PROBLEM n me mi time val funct viol f ex E ex E chance E circle E dispatch E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex522case E ex522case E ex522case E ex E ex E ex E Table 5 Performace of the DF-EPGO+DFNS Algorithm on the test set B (Part I).

21 A Derivative-Free Algorithm for Constrained Global Optimization 21 PROBLEM n me mi time val funct viol f ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E ex E himmel E house E least E like E meanvar E mhw4d E process E rbrock E sample E wall E Table 6 Performace of the DF-EPGO+DFNS Algorithm on the test set B (Part II).