A NOVEL FILLED FUNCTION METHOD FOR GLOBAL OPTIMIZATION. 1. Introduction Consider the following unconstrained programming problem:

Transcription

1 J. Korean Math. Soc. 47, No. 6, pp DOI.434/JKMS A NOVEL FILLED FUNCTION METHOD FOR GLOBAL OPTIMIZATION Youjiang Lin, Yongjian Yang, and Liansheng Zhang Abstract. This paper considers the unconstrained global optimization with the revised filled function methods. The minimization sequence could leave from a minimizer to a better minimizer of the objective function through minimizing an auxiliary function constructed at the minimizer. Some promising numerical results are also included.. Introduction Consider the following unconstrained programming problem:. min{fx : x R n }, where f : R n R. Many results devoted to global optimization are available in the literature. See, for example []-[8]. In order to ensure the ability to escape from minimum, many global optimization algorithms would include in their consideration a subproblem of transcending optimality, namely: given a minimizer x, find a better minimizer, or showing that x is a global minimizer upon termination. Among all the different types of global optimization algorithms available in the literature, one popular approach is called the auxiliary function approach. In this approach, the resolution to the subproblem under concerned is to replace the original cost function with an auxiliary function. This replacement procedure, in principle, should ensure any search applied to the auxiliary function starting from x, would lead to a lower minimum of the original cost function, if there exists one. Thus, global minimizer can be obtained just by implementing search methods to the auxiliary function and the original function. However, it is rather difficult to construct such an auxiliary function. Received February 8, 9; Revised June 3, 9. Mathematics Subject Classification. 9C6, 9C3. Key words and phrases. filled function, unconstrained global optimization, global minimizer. 53 c The Korean Mathematical Society

2 54 YOUJIANG LIN, YONGJIAN YANG, AND LIANSHENG ZHANG Filled function method is a typical auxiliary function method. The primary filled function was proposed by Ge []. The definition of the filled function is as follows: Definition.. Let x be a current minimizer of fx. A function P x is called a filled function of fx at x if P x has the following properties: x is a maximizer of P x and the whole basin B of fx at x becomes a part of a hill of P x; P x has no minimizers or saddle points in any higher basin of fx than B ; 3 if fx has a lower basin than B, then there is a point x in such a basin that minimizes P x on the line through x and x. For the definitions of basin and hill, refer to []. The filled function given at x in [] has the following form:. P x, x, r, ρ r + fx exp x x ρ, where the parameters r and ρ need to be chosen appropriately. However, the filled function algorithm described in [] still has some unexpected features. Such as: The efficiency of the filled function algorithm strongly depend on two parameters r and ρ; When the domain is large or ρ is small, the factor exp xx ρ will be approximately zero, This smoothing increases with this factor, the filled function. will become very flat. This maes the efficiency of the filled function algorithm decrease. Although some other filled functions were proposed later, all of them are still not satisfactory for global optimization due to the above features. In paper [8], a new definition of the filled function is given as following: Definition.. P x, x is called a filled function of fx at a minimizer x if P x, x has the following properties: x is a maximizer of P x, x ; P x, x has no stationary point in the region S {x : fx fx, x Ω \ {x }}. 3 If x is not a global minimizer of fx, then P x, x does have a minimizer in the region S {x : fx < fx, x Ω}. The new filled function algorithm overcomes the disadvantages mentioned above in a certain extent. In this paper, we construct a novel filled function satisfying the new definition. Numerical results indicate the filled function is very efficient and reliable. The paper is organized as follows: In Section, we state the problem under some assumptions and give a novel filled function which has two adjustable

3 A NOVEL FILLED FUNCTION METHOD FOR GLOBAL OPTIMIZATION 55 parameters is proposed and its properties are investigated. In Section 3, we give a new filled function algorithm. In Section 4, test functions and numerical experiments are reported. Finally, in Section 5, we give some concluding remars.. Some assumptions and a new filled function Consider the following unconstrained programming problem: min fx such that x R n. Throughout this paper we mae the following assumptions: Assumpution. fx is Lipschitz continuous on R n, i.e., there exists a constant L > such that fx fy L x y holds for all x, y R n. Assumpution. fx is coercive, i.e., fx +, as x +. Notice that Assumption implies the existence of a robust compact set Ω R n whose interior contains all minimizers of fx. We assume that the value of fx for x on the boundary of Ω is greater than the value of fx for any x inside Ω. Then the original problem is equivalent to the following problem: P min fx such that x Ω. Assumpution 3. fx has only a finite number of minimums in Ω. Let LP stand for the set of minimizers of fx and the function { arctan q ϕ q t t + π, if t,., if t is given. It is easy to prove that ϕ q t is a continuously differentiable function. The new filled function given at x has the following form:. F x, x, q, r where q > and r satisfies q + x x ϕ q fx fx + r, < r < max x,x LP fx <fx fx fx. The following theorems show that F x, x, q, r is a filled function satisfying Definition.. Theorem.. Suppose that fx holds Assumption. Further suppose that x is a minimizer of fx. For any r >, when q > is satisfactorily small, x is a maximizer of F x, x, q, r.

4 56 YOUJIANG LIN, YONGJIAN YANG, AND LIANSHENG ZHANG Proof. Since x is a minimizer of fx, there exists a neighborhood Nx,δ of x with δ > such that fx fx for all x Nx,δ. Then, F x, x, q, r F x, x, q, r q q + x x arctan fx fx + + π r arctan q q r + π q + x x arctan q fx fx + q arctan q r r + x x π qq + x x q + x x arctan q r arctan arctan + q q + x x x x < q q + x x q q r q fx fx + r q fx fx + r q r π + x x arctan π when x y, arctanx arctany x y; and let q < r q q + x x q 3 fx fx fx fx + r r fx fx + x x π. r 4 Since any x Nx, δ have fx fx, thus fx fx + r r >, therefore fx fx fx fx + r r fx fx + r fx fx + rfx fx r fx fx + r fx fx r fx fx + + rfx fx r r fx fx + r fx fx r fx fx + r + rfx fx r 4 L x x r 3 + L x x r 3 3L x x r 3.

5 A NOVEL FILLED FUNCTION METHOD FOR GLOBAL OPTIMIZATION 57 Then, we have F x, x, q, r F x, x, q, r < q q + x x q 3 3L x x r 3 x x π 4 q q + x x q 3L x x x x r π 4 because q < r x x q q + x x when q < πr L. q 3L r π 4 Above of all, when < q < min{r, πr L } and x Nx, δ, x x, we have, F x, x, q, r F x, x, q, r <, x is a maximizer of F x, x, q, r. Theorem.. Suppose that fx is continuously differentiable. If x is a minimizer of fx, then the function F x, x, q, r has no stationary points in the region S {x : fx fx, x Ω/{x }} when r > and q > is satisfactorily small. Proof. Let x S, i.e., fx fx and x x, and let L sup{ fx : x Ω}. We have F x, x, q, r T x x x x q q + x x arctan fx fx + + π r q fx T x x + x x q + x x fx fx + r3 q + fxfx +r q + x x arctan q r + π + q + max x Ω x x arctan q r L q q + x x r 3 + L q r 3. + π Let M max x X x x r, and < q < min{, 3 4L+M }. We have F x, x, q, r T x x x x < + M arctan q r + π + + M.

6 58 YOUJIANG LIN, YONGJIAN YANG, AND LIANSHENG ZHANG r Thus, when < q < min{ 3 4L+M, r tan π, }, we have F x, x, q, r T x x x x <. It implies that the function F x, x, q, r has no stationary points in the region S {x : fx fx, x Ω \ {x }} when r > and q > is satisfactorily small. Theorem.3. If x LP and it is not a global minimizer of fx in Ω, then there exists a minimizer x of F x, x, q, r in the region S {x : fx < fx, x Ω}. Proof. Since fx is continuous and x is not its global minimizer, there exists a point x, such that fx fx + r, namely, F x, x, q, r. On the other hand F x, x, q, r from the form of the filled function. Therefore, F x, x, q, r F x, x, q, r. Thus x is a minimizer of F x, x, q, r. Theorem.4. Suppose that Assumption is satisfied. If x, x Ω and satisfy the following conditions: fx fx and fx fx, x x > x x. Then, when r > and q is positive and satisfactorily small, F x, x, q, r > F x, x, q, r. Proof. Consider the following two cases: If fx fx fx, then it is obvious that the result follows. If fx fx fx, we will show F x, x, q, r > F x, x, q, r also holds. When fx fx fx, we have F x, x, q, r F x, x, q, r q q + x x arctan fx fx + r q q + x x arctan fx fx + r q + x x arctan q fx fx + r + q + x x arctan q fx fx + r x x x x π q + x x q + x x + π + π

7 A NOVEL FILLED FUNCTION METHOD FOR GLOBAL OPTIMIZATION 59 and lim F x, x, q, r F x, x, q, r x x x x π q x x x x <. Then, there must exist a constant q > such that F x, x, q, r > F x, x, q, r while q < q, and q is not related to the values of fx at x and x. Theorem.5. If x, x Ω and satisfy the following conditions: x x > x x, fx fx > fx and fx fx + r >. Then, we have F x, x, q, r < F x, x, q, r. Proof. By Conditions and, we have q + x x < q + x x and Hence < fx fx + r < fx fx + r. F x, x, q, r < F x, x, q, r. Now we mae some remars. First, in the phase of minimizing the new filled function, Theorems.4 and.5 guarantee that the present minimizer x of the objective function is escaped and the minimum of the new filled function will be always achieved at a point where the objective function value is not larger than the objective function value of the current minimum. Second, the parameters q and r are easier to be appropriately chosen than those of the original filled function.. When the parameter q is small and fx fx, q the factor arctan fxfx + π +r will be approximately π, therefore, the new filled function does not become flat. In the next section, a new filled function algorithm is given, it has a simple termination criteria. 3. The filled function algorithm In the above section, we discussed some properties of the filled function. Now, we present an algorithm in the following: Algorithm. Initial Step Choose r, and < r < as the tolerance parameters for terminating the minimization process of problem P. Choose < q < and M >. Choose direction e i, i,,..., with integer > n, where n is the number of variable. Choose an initial point x Ω.

8 6 YOUJIANG LIN, YONGJIAN YANG, AND LIANSHENG ZHANG Let.. Main Step. Obtain a minimizer of prime problem P by implementing a downhill search procedure staring from the x. Let x be the minimizer obtained. Let i, r, q r ln.. If i, then goto 5, otherwise goto If r r, then terminate the iteration, the x is the global minimizer of problem P, otherwise, goto If q q, then let r r/, q r ln, i, goto 5, otherwise, let q q/, i, goto x x +σe i where σ is a very small positive number, if f x < fx then let +, x x and goto ; otherwise, goto Let F x, x, q, r q + x x ϕ qfx fx + r and y x. Turn to inner loop. 3. Inner Loop. Let m.. y m+ ϕy m, where ϕ is an iteration function. It denotes a downhill search method for the following problem: min F x, x, q, r such that x Ω. Such as F-R method, BFGS method, etc. 3. If y m+ / Ω, then let i i +, goto main step, otherwise goto If fy m+ fx, then let +, x y m+ and goto main step, otherwise let m m + and goto. The idea and mechanism of algorithm are explained as follows: There are two phrases in the algorithm. One is that of minimizing the original function f, the other is that of minimizing the new filled function F x, x, q, r in the inner loop. We let r and q r ln in the initialization, afterwards, r and q are gradually reduced via the two-phase cycle until they are less than sufficiently small positive scales. If the parameters r and q are sufficiently small, we cannot find the point x with fx < fx yet, then we believe that there does not exist a better minimizer of fx. The algorithm is terminated. 4. Numerical examples In this section, we apply the filled algorithm to several test examples. The proposed algorithm is programmed in Fortran 95 for woring on the windows XP system with Intel cl.7g CPU and 56M RAM. Numerical results prove that the method is efficient. The computational results are summarized in tables for each example problem. The symbols used in the tables are given as follows:

9 A NOVEL FILLED FUNCTION METHOD FOR GLOBAL OPTIMIZATION 6 : The iteration number in finding the -th minimizer; x : The starting point in the -th iteration; q,r: The parameters used for finding the -th minimizer; x : The -th minimizer; fx : The function value of the -th minimizer. Example 4. Two-dimensional n function in [9]. min fx [ x + c sin4πx x ] + [x.5 sinπx ] such that x, x, where c.,.5,.5. The proposed filled function approach succeeds in identifying the global minimum solutions: fx for all c. The computational results are summarized in Tables -3, for c.,.5,.5, respectively. An illustration with c.5 is given in Fig.. Example 4. Six-hump bac camel n function in []. min fx 4x.x x6 x x 4x + 4x 4 such that 3 x 3, 3 x 3. Three initial points x,,,, and, are used. The proposed filled function approach succeeds in identifying the global minimum solutions: x.89843, or.89843, , where fx The computational results are summarized in Table 4-6, respectively. An illustration is given in Fig..

10 6 YOUJIANG LIN, YONGJIAN YANG, AND LIANSHENG ZHANG Table 4.. numerical results for example 4. with c. x minimizer x fx q, r Table 4.. Numerical results for Example 4. with c.5 x minimizer x fx q, r Example 4.3 The Rastrigin n function in []. min fx x + x cos8x cos8x such that x, x. The initial points x, are used. The proposed filled function approach succeeds in identifying the global minimum solutions x.,., where fx. The computational results are summarized in Table 7, respectively. An illustration is given in Fig.3.

11 A NOVEL FILLED FUNCTION METHOD FOR GLOBAL OPTIMIZATION 63 Table 4.3. Numerical results for Example 4. with c.5 x minimizer x fx q, r Table 4.4. Numerical results for Example 4. with initial point, x minimizer x fx q, r Example 4.4 n-dimensional Sine-square n, 3, 5, 7, function in []. min fx π n [ sin πx + gx + x n ]

12 64 YOUJIANG LIN, YONGJIAN YANG, AND LIANSHENG ZHANG Table 4.5. Numerical results for Example 4. with initial point, x minimizer x fx q, r Table 4.6. Numerical results for Example 4. with initial point, x minimizer x fx q, r Table 4.7. Numerical results for Example 4.3 x minimizer x fx q, r such that x i, i,,..., n,

13 A NOVEL FILLED FUNCTION METHOD FOR GLOBAL OPTIMIZATION 65 Table 4.8. Numerical results for Example 4.4 with n x minimizer x fx q, r Table 4.9. Numerical results for Example 4.4 with n 3 x minimizer x fx q, r Table 4.. Numerical results for Example 4.4 with n 5 x minimizer x fx q, r where gx n i [x i + sin πx i+ ]. The function is tested for n, 3, 5, 7,. The global minimum solution is uniformly expressed as: x.,.,...,. and fx.. The computational results are summarized in Tables 8-, respectively. An illustration with n is given in Fig Conclusions Based on the new definition of the filled function, we give a novel filled function which satisfying the definition and develop an algorithm for unconstrained

14 66 YOUJIANG LIN, YONGJIAN YANG, AND LIANSHENG ZHANG Table 4.. Numerical results for Example 4.4 with n 7 x minimizer x fx q, r Table 4.. Numerical results for Example 4.4 with n x minimizer x fx q, r global optimization. The computational results show that this filled function is quite efficient and reliable.

15 A NOVEL FILLED FUNCTION METHOD FOR GLOBAL OPTIMIZATION 67 References [] X. Chen, L. Qi, and Y. F. Yang, Lagrangian globalization methods for nonlinear complementarity problems, J. Optim. Theory Appl., no., [] R. Ge, A filled function method for finding a global minimizer of a function of several variables, Math. Program , no., Ser. A, 9 4. [3] R. Ge and Y. Qin, The globally convexized filled functions for global optimization, Appl. Math. Comput , no., part II, [4] X. Liu, Several filled functions with mitigators, Appl. Math. Comput. 33, no. -3, [5] Z. Y. Wu, H. W. J. Lee, L. S. Zhang, and X. M. Yang, A novel filled function method and quasi-filled function method for global optimization, Comput. Optim. Appl. 34 6, no., [6] Z. Y. Wu, M. Mammodov, F. S. Bai, and Y. J. Yang, A filled function method for nonlinear equations, Appl. Math. Comput. 89 7, no., [7] Z. Xu, H. X. Huang, and P. Pardalos, and C. X. Xu, Filled functions for unconstrained global optimization, J. Global Optim., no., [8] Y. Yang and Y. Shang, A new filled function method for unconstrained global optimization, Appl. Math. Comput. 73 6, no., 5 5. [9] L. S. Zhang, C. K. Ng, D. Li, and W. W. Tian, A new filled function method for global optimization, J. Global Optim. 8 4, no., [] Q. Zheng and D. Zhuang, Integral global minimization: algorithms, implementations and numerical tests, J. Global Optim , no. 4, Youjiang Lin Department of Mathematics Shanghai University Shanghai 444, P. R. China address: linyoujiang@shu.edu.cn Yongjian Yang Department of Mathematics Shanghai University Shanghai 444, P. R. China address: yjyang@mail.shu.edu.cn Liansheng Zhang Department of Mathematics Shanghai University Shanghai 444, P. R. China address: Zhangls@staff.shu.edu.cn