A filled function method for constrained global optimization

Transcription

1 DOI 0.007/s ORIGINAL PAPER A filled funtion method for onstrained global optimization Z. Y. Wu F. S. Bai H. W. J. Lee Y. J. Yang Reeived: 0 Marh 005 / Aepted: 5 February 007 Springer Siene+Business Media B.V. 007 Abstrat In this paper, a filled funtion method for solving onstrained global optimization problems is proposed. A filled funtion is proposed for esaping the urrent loal minimizer of a onstrained global optimization problem by ombining the idea of filled funtion in unonstrained global optimization and the idea of penalty funtion in onstrained optimization. Then a filled funtion method for obtaining a global minimizer or an approximate global minimizer of the onstrained global optimization problem is presented. Some numerial results demonstrate the effiieny of this global optimization method for solving onstrained global optimization problems. Keywords Filled funtion Filled funtion method Constrained global optimization Z. Y. Wu B F. S. Bai Shool of Mathematis and Computer Siene, Chonging Normal University, Chonging , China zywu@nu.edu.n F. S. Bai fsbai@nu.edu.u H. W. J. Lee Department of Applied Mathematis, The Hong Kong Polytehni University, Kongloon, Hong Kong, China Joseph.Lee@inet.polyu.edu.hk Y. J. Yang Department of Mathematis, Shanghai University, Shanghai 00444, China yjyang@mail.shu.edu.n Present Address: Z. Y. Wu F. S. Bai Shool of Information Tehnology and Mathematial Sienes, University of Ballarat, Ballarat, VIC 3353, Australia

2 Introdution We onsider the following ineuality onstrained global optimization problem: P min f x. s.t. g i x 0, i =,..., m, x X, where f : X R, g i : X R, i =,,..., m, andx is a box. The existene of loal minimizer other than global one makes global optimization a great hallenge. As one of the main methods to solve general unonstrained global optimization problem without speial strutural property, filled funtion method has attrated extensive attention, see, e.g., [5,6,9,,8 0]. The essential idea of filled funtion method is to onstrut an auxiliary funtion alled filled funtion via the urrent loal minimizer of the original optimization problem, with the property that the urrent loal minimizer is a loal maximizer of the onstruted filled funtion and a better initial point of the primal optimization problem an be obtained by searhing the onstruted filled funtion loally. Both the theoretial and the algorithmi studies show that the filled funtion method is ompetitive with the other existing global optimization methods, suh as tunneling funtion methods [,0] and stohasti methods [4,5]. Constrained global optimization is an even tougher area to takle. For some global optimization problems with nie speial strutural property, suh as onave minimization problem, D.C. programming problems, reverse onvex programming problems and monotone programming problems, there are a number of well developed methods, see, e.g., [,7,8,3,7]. For general onstrained global optimization, i.e., onstrained global optimization without speial strutural property, there is lak of well developed methods. In this paper, we generalize the appliable area of filled funtion method to general onstrained global optimization. The key idea here is to ombine the filled funtion method in unonstrained global optimization proposed in Ref. [8] with penalty funtion methods in onstrained optimization. It is should be noted that all the existing filled funtion methods proposed in Refs. [5,6,9,,,8 ] fous only on solving unonstrained global optimization problems or box onstrained global optimization problems, whereas the filled funtion method proposed in this paper fouses on solving general onstrained global optimization problems. The rest of this paper is organized as follows. In Set., a filled funtion for onstrained global optimization problem P is introdued, and then some basi properties of the proposed filled funtion are disussed. Espeially, it is shown that an improved feasible point an be obtained by solving loally an unonstrained or a box onstrained optimization problem onstruted via the given filled funtion. In Set. 3, we propose a global optimization method named filled funtion method using the presented filled funtion to obtain a global minimizer of the onstrained global optimization problem P. Finally, some illustrative numerial examples are given in Set. 4. Filled funtion of onstrained global optimization and its properties To begin with, we have the following assumptions. Assumption. f and g i, i =,..., m, are ontinuously differentiable.

3 Let where inta denotes the interior of set A. S ={x X g i x 0, i =,..., m},. S ={x intx g i x <0, i =,..., m},. Assumption. S =,ls = S, wherela denotes the losure of set A. By Assumption., we know that for any x 0 S, there exists a seuene {x n } S, suh that lim n x n = x 0. Let Y ={x S x is a loal minimizer of problemp}..3 Assumption.3 The set {f x x Y} is a finite set. Definition. A point x X is said to be a vertex of box X if x = λx + λx with x, x X and λ 0, implies that x = x = x. Let x S be the urrent loal minimizer of problem P. Definition. A funtion px is said to be a filled funtion of problem P at a loal minimizer x if it is ontinuously differentiable and satisfies the following onditions: x is a strit loal maximizer of px on X; i any point x with p x = 0and x = x satisfies f x <f x and x S; ii any loal minimizer x of px on X satisfies a. f x <f x and x S, or b. x is a vertex of X. 3 If x is not a global minimizer of problem P, then there exists a point x S, suh that x is a loal minimizer of px on X with f x <f x. 4 For any x, x X with i f x f x or x X \ S and ii f x f x or x X \ S, it holds that x x > x x if and only if px <px. By the above definition, we know that if px is a filled funtion of problem P at a loal minimizer x, then any infeasible point or any x X with x = x and f x f x, exept the vertexes of X, is neither a loal minimizer of px on X, nor a stationary point of px. Also we note that the urrent loal minimizer x is a global minimizer of problem P if and only if any loal minimizer of funtion px on X is a vertex of X. Condition 4 reveals some kind of algorithmi onsideration for an appliable filled funtion. In what follows, we will introdue a funtion p r,,,x x whih satisfies Definition.. To begin with, we design two ontinuously differentiable funtions f r, t and t. Funtion f r, t should has the following properties: f r, t euals to a onstant when t > 0, and f r, t euals to zero when t is less than a negative number r. f r, t plays the role of filling in the onstrution of filled funtion. t should have the following properties: It is larger than a positive number d when t > 0 and euals to zero when t is less than a negative number r, in addition, it is inreasing. t an be regarded as

4 a modified penalty funtion and used for dealing with onstraints. More speifially, for any given > 0andr > 0, let, t 0 f r, t = r 3 t3 3 r t +, r < t 0,.4 0, t r t + t 0 r 4 t = r 3 t 3 + r 6 r t + t + r < t < t r Note that the reuirement for ontinuous differentiability of f r, t and t justifies the use of ubi polynomials in f r, t and t. Conseuently we have that 0, t 0 f r, t = 6 r 3 t 6 t, r < t 0,.6 r 0, t r t 0 t = 3r r 3 t 4r + r t + r < t < t r Obvilousy f r, t and t are ontinuously differentiable on R. Define p r,,,x x = x x + f r, f x f x m + g i x r,.8 where > 0, r > 0, and > 0 are parameters. Remark. Funtion p r,,,x x is essentially different from the filled funtion H,r,x x proposed in Ref. [8]. Here p r,,,x x inludes m g i x as a penalty term to penalize unfeasible points, while H,r,x x just plays the role of filling. The following theorems shows that p r,,,x x satisfies Definition.. Theorem. For any > 0, > 0, and0 < r, x is a strit loal maximizer of p r,,,x x on X. Proof Sine x is a loal minimizer of the original problem P, there exists δ 0 > 0, suh that f x f x for any x S Nx, δ 0,whereNx, δ 0 denotes the neighborhood of x with radius δ 0, i.e., Nx, δ 0 ={x R n x x <δ 0 }. Then, for any x X Nx, δ 0, we have f x f x or x X \ S, whih yields f x f x + m g i x.

5 Thus, when > 0and0< r, we have that m f r, f x f x + g i x r =. Therefore, we have that p r,,,x x = x x + < = p r,,,x x for any x X Nx, δ 0 \{x } when 0 < r. Thus, x is a strit loal maximizer of p r,,,x x on X. In what follows we assume that,, and r > 0. Theorem. i Any x X with x = x and p r,,,x x = 0 satisfies that f x <f x and x S; ii when r, any loal minimizer x Xofp r,,,x x on X satisfies that f x < f x and x S, or x isavertexofx. Proof i By.8, we have that x x p r,,,x x = f r, f x f x m + x x + m + x x + f r, g r f x f x + f x f x f x + For any x X satisfying f x f x or x X \ S, we have that m f x f x + g i x. When r, we have that m f x f x + g i x r 0, whih yields and f r, f x f x + f r, Thus, we have that f x f x + m m g i x r g i x r m g r g i x g i x..9 g i x r = g i x r = 0. p r,,,x x = x x x x +.

6 Therefore, for any x X with x = x and f x f x,orx X \ S, we have that p r,,,x x = 0, furthermore, T p r,,,x xdx <0, where dx is defined as dx = x x x x. Thus, any point x = x with p r,,,x x = 0 satisfies that f x <f x and x S. ii If x is a loal minimizer of p r,,,x x on X, and x is neither a point satisfying f x <f x and x S, nor a vertex of X, then there exist two different points z, z X and a positive number α with 0 <α<suh that x = αz + αz and p r,,,x x = x x + when 0 < r. Sine, X is a box, we have that [z, z ] := {λz + λz 0 λ } X. Let d 0 = z z, α 0 = min{α, α}. Then, we an easily verify that for any s with s α 0, it holds x + sd 0 [z, z ] X. Let z,λ = x + λd 0 and z,λ = x λd 0,where0< λ α 0.Thenz,λ, z,λ X and p r,,,x z,λ z,λ x + = x x + λd 0 +, p r,,,x z,λ z,λ x + = x x λd 0 +. By x x + λd 0 + x x λd 0 = x x + λ d 0 + λ d 0, x x + x x + λ d 0 λ d 0, x x = x x + λ d 0 > x x, we have that x x + λd 0 or x x λd 0 is larger than x x. Thus, p r,,,x z,λ or p r,,,x z,λ is less than p r,,,x x. Sine,λ an approah 0 to any extent, we obtain that x is not a loal minimizer of p r,,,x x on X. This is a ontradition. Therefore, if x is a loal minimizer of p r,,,x x on X, then we have f x <f x and x S,or x is a vertex of X. Theorem.3 If x is not a global minimizer of the original problem P, then there exist r 0 > 0, 0 > 0,and x S, suh that x is a loal minimizer of p r,,,x x on X with f x <f x when r r 0 and 0.

7 Proof i If x is not a global minimizer of the original problem P, then there exists x 0 S suh that f x 0 <fx.letr 0 = f x f x 0,thenr 0 > 0. By Assumption., there exists x S suh that f x f x 0 <r 0. For this r 0, there exists 0 > 0, suh that g i x < r 0 0 for any i =,..., m. Thus, when r r 0 and 0, we have f x f x = f x f x 0 + f x 0 f x < r 0 r 0 r, and g i x < r for any i =,..., m, whih yield p r,,,x x = 0. Note that p r,,,x x 0foranyx X and p r,,,x x =, it follows that x = x is a global minimizer of p r,,,x x on X. Conseuently, by x S, we have that p r,,,x x = 0, whih yields f x <f x. Theorem.4 For any x, x X with i f x f x or x S and ii f x f x or x S, we have that when 0 < r, x x > x x if and only if p r,,,x x < p r,,,x x. Proof For suh x i, p r,,,x x i =, i =,, when 0 < r. Thus, we have x i x + that x x > x x if and only if p r,,,x x <p r,,,x x when 0 < r. 3 Algorithm We introdue here the following box onstrained optimization problem named filled funtion problem: P min p r,,,x x. x X Based on the proposed filled funtion p r,,,x x, we an obtain a filled funtion method for the onstrained global optimization problem P. The general idea of this filled funtion method is as follows: if the urrent loal minimizer x is not a global minimizer of P, then we an manage to obtain a point x S with f x <f x by applying some loal searh shemes to problem P. Conseuently we an obtain a better loal minimizer of P by applyingloal searh shemesto problem P starting from x. Finally, a global minimizer or an approximate global minimizer of P an be obtained. The orresponding algorithm is denoted by Algorithm FFMC and detailed as follows: 3. Algorithm FFMC Step 0 a Choose a small positive numbers µ, and a large positive number M in the examples of Set. 4,wetakeµ = 0 5 and M = 0 5. Choose a positive integer number K and diretions e,..., e K in the numerial examples in Set. 4, weletk = n

8 and let e i, i =,..., K, be the oordinate diretions, where n is the number of dimensions of the variable. Choose the initial values,,andr for the parameters,, and r, respetively in the numerial examples of Set. 4, wetake = 00, =, and r =. b Choose an initial point x 0 Xhere x0 may not be a feasible point, then use penalty funtion methods to find the first loal minimizer x of the original problem P. Letk :=, j :=, and λ :=, and go to Step. Step Let p rk, k, k,x x = k x x k + f r k, k k f x f x k + m k g i x r k, k 3. where f r, t and t are defined in.4 and.5, respetively. Go to Step. Step If j K, hoose a nonnegative λ with λ suh that y j k := x k + λe j X, and go to Step 3; otherwise, go to Step 5. Step 3. Searh for a loal minimizer of the following filled funtion problem starting from y j k in the numerial examples of Set. 4, we use the onjugate gradient method to searh for a loal minimizer of the following problem: min p r x X k, k, k,x x. 3. k One a point y k X with f y k <fx k and g iy k 0, i =,..., m, is obtained in the proess of searh, set x 0 k+ := y k, k := k +,andgotostep4;otherwiseontinue the proess. Let x k be an obtained loal minimizer of problem 3.. If x k satisfies f x k <fx k and x k S, thenletx0 k+ := x k, k := k +, and go to Step 4; otherwise, let j := j + and go to Step. Step 4 Find a loal minimizer x k of the original onstrained problem P by loal searh methods starting from x 0 k in the numerial examples of Set. 4, weusethe SQP method to obtain a loal minimizer of the original problem P. Go to Step. Step 5 If k M,let k := 0 k,andj :=, go to Step ; otherwise, go to Step 6. Step 6 If k M, let k := 0 k, k :=, j :=, go to Step ; otherwise, go to Step 7. Step 7 If r k µ, letr k := r k 0, k :=, k :=,gotostep;otherwise,stopand x k is a global minimizer or an approximate global minimizer of problem P. By Assumption.3, we know that the set V := {f x x Y} is a finite set, where Y is defined in.3. Let V : ={v,..., v l }, 3.3 η : = min{ v i v j v i, v j V, i = j}. 3.4 For any v i V, there exists x i Y suh that f x i = y i.ifx i is not a global minimizer of problem P, then by Assumption., there exists x i S suh that f x i fx i < η.

9 For this η, there exists i > 0 suh that g j x i < η i for any j =,..., m. Otherwise, if x i is a global minimizer of problem P, thenlet i = 0. Let : = max{ i, i =,..., l}. 3.5 The following assumption together with Assumptions..3 an ensure the onvergene of Algorithm FFMC. Assumption 3. µ η, M,andtheset{e,..., e K } is large enough to ensure that if problem P has a better loal minimizer than the urrent loal minimizer x k,then x 0 k+ in Step 3 an be obtained. Remark 3. Theoretially we an obtain a global optimizer in finite steps under Assumptions..3 and 3.. However, in pratie, some of these assumptions are freuently violated, espeially Assumption 3.. Hene we often obtain approximate global minimizers instead of global minimizers by implementing Algorithm FFMC to solve onstrained global optimization problems. 4 Numerial examples In this setion we give four numerial examples to illustrate the effiieny of Algorithm FFMC. For eah example, we take several initial points to start the searh proess. In the list of tables, we use the same notations as in Algorithm FFMC, and moreover, we use NFFE to denote the number of filled funtion evaluations. FORTRAN 95 is used to ode the algorithm, and the NCONF subroutine, whih is based on the SQP method, is used to find loal minimizers of the original onstrained problem when k. The onjugate gradient method is used to searh for loal minimizers of the filled funtion problems. Example 4. min f x = x + x os7x os7x + 3 s.t. g x = x + x.6 0, g x = x + x 3.7 0, x X ={x, x 0 x i, i =, }. This example is taken from Ref. [6]. It is a nononvex problem with loal optimal solutions in the interior of the feasible region. By Sun and Li [6], we know that x = 0.755, is a global minimum of Example 4. with global optimal value f = Table reords the numerial results of Example 4..

10 Table Numerial results for Example 4. k 3 3 x 0 k f x 0 k x k f x k NFFE Sum: Sum: Sum: Sum: Sum: 3 Example 4. min f x = 5x x x 3 x 4 4 x 5 x 6 4 s.t. x x 4 4 x x 6 4 x 3x x + x x + x 6 x + x 0 x 6 0 x 8 x X = x x x 4 6 x x 6 0 This example is taken from Ref. [4]. We know from Ref. [4] that there exist 8 loal minima and that x = 5,, 5, 0, 5, 0 is a global minimum of Example 4. with global optimal value f = 30. Table reords the numerial results of Example 4..

11 Table Numerial results for Example 4. k x 0 k f x 0 k x k f x k NFFE 3, 3, 3, 3, 3, ,, 5, , 5, ,, 5, , 5, ,,5, ,5, ,.,5, ,5, ,,5,0,5, ,.00000,5,0,5, ,,5,0,5, Sum: 60 4,4,4,4,4,4 5,,5,4,5, ,, 5, , 5, ,, 5, 6, 5, ,, 5, , 5, ,, 5, 0, 5, ,, 5, 0, 5, ,, 5, 0, 5, Sum: 400 3, 3, 4, 4, 3, ,,5,4,5, ,, 5, , 5, ,, 5, 0, 5, Sum: 6,, 3,, 3, 6 5,,5,6,5, ,, 5, , 5, ,, 5, 0, 5, Sum: 6 4, 7, 4, 5, 4, ,,5,6,5, ,, 5, , 5, ,, 5, 0, 5, Sum: 6 Table 3 Numerial results for Example 4.3 k x 0 k f x 0 k x k f x k NFFE 0, , , , , , Sum: , , , , , , , , Sum: , , Example 4.3 min x y s.t. y x 4 8x 3 + 8x + y 4x 4 3x x 96x x 3 0 y 4. This problem is taken from Ref. [3]. We know from Ref. [3] that the best known global solution is x =.395, with funtion value f x = Table 3 reords the numerial results of Example 4.3.

12 Example 4.4 min x x x x s.t x 3 x x x x x x 3 x x x x x x x x x x x x x x x x 3 x x 3 x x x x 3 x x 3 x x x x 0 33 x 45 x X = x 7 x x x 5 45 This problem is also taken from Ref. [3]. We know from Ref. [3] that the best known global solution is x = 78, 33, , 45, with funtion value f x = Table 4 Numerial results for Example 4.4 k x 0 k f x 0 k x k f x k NFFE Sum:

13 Table 4 reords the numerial results of Example 4.4. Aknowledgements This work is partly supported by Australia Researh Counil Projet Grant and by Program for New Century Exellent Talents in University of China. Referenes. Barhen, J., Protopopesu, V., Reister, D.: TRUST: A deterministi algorithm for global optimization. Siene 76, Benson, H.P.: Deterministi algorithm for onstrained onave minimization: a unified ritial surver. Naval Res. Logist. 43, Floudas, C.A., Pardalos, P.M.: A Colletion of Test Problems for Constrained Global Optimization Algorithms. Springer, Berlin Heidelberg Floudas, C.A., Pardalos, P.M., Adjiman, C.S., Esposito, W.R., Gumus, Z.H., Harding, S.T., Klepeis, J.L., Meyer, C.A., Shweiger, C.A.: Handbook of Test Problems in Loal and Global Optimization. Kluwer Dordreht, The Netherlands Ge, R.: A filled funtion method for finding a global minimizer of a funtion of several variables. Math. Program. 46, Ge, R.: The globally onvexized filled funtions for global optimization. Appl. Math. Comput. 35, Hoffman, K.L.A.: A method for globally minimizing onave funtions over onvex set. Math. Program. 0, Horst, R.: Deterministi methods in onstrained global optimization: some reent advanes and new fields of appliation. Naval Res. Logist. 37, Kanzow, C.: Global optimization tehniues for mixed omplementarity problems. J. Glob. Optim. 6, Levy, A.V., Montalvo, A.: The tunneling algorithm for the global minimization of funtion. SIAM J. Si. Stat. Comput. 6, Liu, X.: Finding global minimia with a omputable filled funtion. J. Glob. Optim. 9, Luidi, S., Piialli, V.: New lasses of globally onvexized filled funtions for global optimization. J. Glob. Optim. 4, Pardalos, P.M., Rosen, J.B.: Constrained Global Optimization, Algorithms and Appliations. Springer, Berlin Rinnoy Kan, A.H.G., Timmer, G.T.: Stohasti global optimization methods, part I: lustering methods. Math. Program. 39, Rinnoy Kan, A.H.G., Timmer, G.T.: Stohasti global optimization methods, part II: multi-level methods. Math. Program. 39, Sun, X.L., Li, D.: Value-estimation funtion method for onstrained global optimization. J. Optim. Theory Appl. 0, Tuy, H.: Convex Analysis and Global Optimization. Kluwer, Dordreht, Boston, London Wu, Z.Y., Lee, H.W.J., Zhang, L.S., Yang, X.M.: A novel filled funtion method and uasi-filled funtion method for global optimization. Comput. Optim. Appl. 34, Xu, Z., Huang, H.X., Pardalos, P.: Filled funtions for unonstrained global optimization. J. Glob. Optim. 0, Zhang, L.S., NG, C.K., Li, D., Tian, W.W.: A new filled funtion method for global optimization. J. Glob. Optim. 8, Zhu, W.X.: A lass of filled funtions for box onstrained ontinuous global optimization. Appl. Math. Comput 69, Zhu, W.X.: Globally onavized filled funtion method for the box onstrained ontinuous global minimization problem. Optim. Methods Softw. 4,