Characterizations of Pseudoconvex Functions and Semistrictly Quasiconvex Ones

J Glob Optim DOI 10.1007/s10898-012-9956-6 manuscript No. (will be inserted by the editor) Characterizations of Pseudoconvex Functions and Semistrictly Quasiconvex Ones Vsevolod I. Ivanov Received: date / Accepted: date The final publication is available at http://link.springer.com Abstract In this paper we provide some new necessary and sufficient conditions for pseudoconvexity and semistrict quasiconvexity of a given proper extended real-valued function in terms of the Clarke-Rockafellar subdifferential. Further we extend to programs with pseudoconvex objective function two earlier characterizations of the solutions set of a set constrained nonlinear programming problem due to O.L. Mangasarian [Oper. Res. Lett. 7 (1988) 21 26]. A positive function p appears in the most results. It is replaced by the number 1 if the function is convex and its domain of definition is convex, too. Keywords Pseudoconvex functions Semistrictly quasiconvex functions Nonsmooth analysis Characterizations of the solution set of a set constrained optimization problem Optimality conditions Mathematics Subject Classification (2000) MSC 26B25 MSC 90C26 90C46 1 Introduction Pseudoconvex functions play important role in mathematics. They admit a lot of applications in optimization, economics, mechanics, and other disciplines. This concepts originated from Eugenio Elia Levi in 1910 within a research on analytic functions; see [31]. Independently of him, Tuy [46] and Mangasarian [35], introduced the same notion in the field of optimization. The properties of pseudoconvex functions were studied by Ortega and Rheinboult [39], Thompson and Parke [45], Mereau and Paquet [38], Avriel and Schaible [3], Diewert, Avriel and Zang [14], Crouzeix and Ferland [10], Karamardian and Schaible [25], Komlosi [28], Ivanov [21], Crouzeix, Eberhard and Ralph [9] in the case of Fréchet or directional differentiability. Some results were populated by the books [2, 4, 7, 17, 36]. Generalizations to V.I. Ivanov Department of Mathematics, Technical University of Varna, 1 Studentska Str., 9010 Varna, Bulgaria Tel.: +359-52-383436 E-mail: vsevolodivanov@yahoo.com

2 Vsevolod I. Ivanov nonsmooth functions are derived by Komlosi [26, 29, 30], Luc [34], Penot [40], Penot and Quang [41], Aussel [1], Ginchev and Ivanov [15], Yang [47], Soleimani-damaneh [43, 44], Hassouni, Jaddar [19]. Higher-order pseudoconvex functions are investigated by Ginchev and Ivanov [16], Ivanov [23]. Several necessary and sufficient conditions for strict pseudoconvexity of functions are obtained by Ivanov [20]. The main result of Crouzeix and Ferland was extended to pseudoconvex nonlinear programming problems by Ivanov [22]. In this paper, we obtain three new complete characterizations of proper extended pseudoconvex real-valued functions in terms of the Clarke-Rockafellar subdifferential. We derive a derivative-free complete characterization of pseudoconvex functions in terms of some positive function b. Now, we compare our characterizations of pseudoconvex functions with the respective properties of convex ones. An arbitrary positive function appears in all of them. It is equal identically to the constant 1 if the function is convex. Recall that a function f, defined on a convex set S R n is called convex if f [λy + (1 λ)x] λ f (y) + (1 λ) f (x), x,y S, λ [0,1]. (1) A differentiable convex function, which is defined on an open convex set S is completely characterized by each one of the following inequalities: and f (y) f (x) f (x),y x, x,y S (2) f (y) f (x),y x 0, x,y S. (3) These conditions do not hold anymore if the function is not convex. We prove that in the case when f is pseudoconvex (Theorems 2 and 4) Inequalities (2) and (3) are transformed into the following ones: p(x,y) > 0 : f (y) f (x) p(x,y) f (x),y x, x,y S. p : S S (0,+ ) : p(y,x) f (y) p(x,y) f (x),y x 0, x,y S. Condition (1) is generalized to the following inequality (Theorem 6): for all x, y S and each λ [0,1] there exists a positive number b, which depends on x, y, λ such that 0 λb 1 and f [λy + (1 λ)x] λ b f (y) + (1 λ b) f (x). (4) Further we consider some relations between pseudoconvex and semistrictly quasiconvex functions. We characterize them. We prove that semistrictly quasiconvex functions are the largest class completely characterized by Inequality (4) and the condition 0 < b < 1/λ, 0 < λ < 1. We derive necessary and sufficient condition for pseudoconvexity of a semistrictly quasiconvex function. Some other results concerning these functions are obtained by Hadjisavvas and Schaible [18], Daniilidis and Hadjisavvas [11], Daniilidis and Ramos [13]. One can find more properties in the books [4,7,17]. Another purpose of the paper is to study the solution set of the nonlinear programming problem Minimize f (x) subject to x S, (P) provided that a fixed minimizer x is known. Such characterizations originated by Mangasarian [37]. The problem that he considered was convex. The results of Mangasarian were generalized to proper extended real-valued

Characterizations of Pseudoconvex Functions and Semistrictly Quasiconvex Ones 3 functions by Burke and Feris [6]. Denote the solution set of the problem (P) by S. Mangasarian obtained in [37] that if f and S are convex, then S = {x S f ( x),x x = 0, f (x) = f ( x)}. We generalize the characterizations of Mangasarian to problems with locally Lipschitz pseudoconvex objective function over nonconvex set in terms of the Clarke generalized gradient. As a consequence of our Theorem 9 we have that if in the problem (P) the function f is Fréchet differentiable and pseudoconvex and the set S is convex, then the solution set S of (P) can be completely characterized by the following equality: S = {x S f ( x),x x = 0, p(x) > 0 : f (x) = p(x) f ( x)}. The paper is organized as follows: In Section 2 we derive two first-order complete characterizations of pseudoconvex functions. In Section 3 we obtain a derivative-free necessary and sufficient condition for pseudoconvexity. The results concerning semistrictly quasiconvex functions are considered in Section 4. In Section 5 we obtain two characterizations of the solution set of a pseudoconvex program. 2 First-order characterizations of pseudoconvex functions In this section, we derive two necessary and sufficient conditions for a given function to be pseudoconvex. In the sequel, we suppose that E is a Banach space. We denote by E its dual and the duality pairing between the vectors a E and b E by a,b, by R the set of reals, by R the union R {+ }, by B(x,r) the closed ball of a center x with a radius r. Let f : E R be a proper extended real-valued function, whose domain is the set dom( f ) := {x E f (x) < + }. Definition 1 Let f : E R be a proper extended real-valued function and x dom( f ). The Clarke-Rockafellar generalized derivative of f at x in direction v is defined by f (x,v) = sup lim sup inf ε>0 (y,α) f x;t 0 u B(v,ε) [ f (y +tu) α]/t, where (y,α) f x means that y x, α f (x), α f (y) (see [42]), and y x implies that the norm y x approaches 0. If f happens to be lower semicontinuous at x the definition can be expressed in the slightly simpler form f (x,v) = sup lim sup ε>0 y f x;t 0 inf u B(v,ε) [ f (y +tu) f (y)]/t, where y f x means that y x, f (y) f (x). When f is locally Lipschitz, this derivative coincides with the Clarke generalized derivative [8], which is defined by f 0 (x,v) = limsup[ f (y +tv) f (y)]/t. y x;t 0 The Clarke-Rockafellar subdifferential of f at x is defined as follows: f (x) = {x E x,v f (x,v), v E} with the convention that f (x) = /0 if x / dom( f ).

4 Vsevolod I. Ivanov The following lemma was established in [33]. Lemma 1 Assume that f : E R is lower semicontinuous and that f (b) > f (a). Then there exists a sequence {x i } i=1 in E converging to some x 0 [a,b) and a sequence {x i } i=1, x i f (x i ) such that, for any c = a +t(b a) with t 1, and for every positive integer i, one has x i,c x i > 0. Definition 2 A proper extended real-valued function f : E R is called pseudoconvex (in terms of the Clarke-Rockafellar directional derivative) iff the following implication is satisfied: x E, y E, f (y) < f (x) x,y x < 0, x f (x). (5) Recall that a proper extended real function is said to be quasiconvex iff, f [x +t(y x)] max{ f (x), f (y)}, x E, y E, t [0,1]. The following result is due to Daniilidis, Hadjisavvas [11, Proposition 2.2]. Lemma 2 Let f : E R be a lower semicontinuous pseudoconvex function with a convex domain. Then f is quasiconvex. Lemma 3 Let f : E R be a lower semicontinuous pseudoconvex function with a convex domain. Then the following implication holds x E, y E, f (y) f (x) x,y x 0, x f (x). Proof The claim is trivially satisfied when x / dom( f ), because in this case f (x) = /0 by definition. Suppose that x dom( f ) and x f (x) : x,y x > 0. (6) We prove that f (y) > f (x). It follows from the definition of pseudoconvexity that f (y) f (x). Therefore, we have to prove that the case f (y) = f (x) is impossible. Assume that f (y) = f (x). Then, by (6) we have f (x,y x) > 0. It follows from here that a number ε > 0 and sequences {x i } i=1, x i E, {t i } i=1, t i > 0 can be chosen such that x i x, t i 0 and inf [ f (x i +t i u) f (x i )]/t i > 0, u B(y x,ε) i. Taking the number i sufficiently large, we ensure that x i B(x,ε). Therefore, we have y x i B(y x,ε) and f [x i +t i (y x i )] > f (x i ). Using that f is lower semicontinuous and pseudoconvex we conclude from Lemma 2 that it is quasiconvex. Therefore, f (x i ) < f [x i +t i (y x i )] f (y). According to the pseudoconvexity of f we have 0 / f (y). Indeed, it follows from f (x i ) < f (y) that y,x i y < 0, y f (y). Hence, 0 / f (y). It follows from (6) that the number ε could be chosen such sufficiently small that x,y x > 0, y B(y,ε). Due to the pseudoconvexity of f we have f (y ) f (x) = f (y). Consequently, the point y is a local minimizer, which contradicts the relation 0 / f (y).

Characterizations of Pseudoconvex Functions and Semistrictly Quasiconvex Ones 5 Theorem 1 Let f : E R be a lower semicontinuous proper extended real-valued function with a convex domain. Then f is pseudoconvex if and only if there exists a positive function p : E E E (0,+ ) with f (y) f (x) p(x,y,x ) x,y x, x dom( f ), y dom( f ), x f (x) (7) such that f (x) /0. Proof Suppose that f is pseudoconvex. We prove that for all x dom( f ), y dom( f ), x f (x) with f (x) /0 there exists p > 0 such that inequality (7) holds. We construct the function p explicitly in the following way: { f (y) f (x) p(x,y,x ) = x,y x, if f (y) < f (x) or x,y x > 0, (8) 1, otherwise. If f (y) < f (x), then x,y x < 0 by pseudoconvexity. If x,y x > 0, then by Lemma 3 we have f (y) > f (x). Therefore the inequalities f (y) < f (x) and x,y x > 0 cannot be satisfied together and the function p is well defined. If f (y) f (x) and x,y x 0, then p(x,y,x ) = 1. It is clear that in all cases p is strictly positive, and it satisfies inequality (7). Conversely, let inequality (7) hold. It is obvious that implication (5) is fulfilled, i.e. f is pseudoconvex. The classical notion of pseudoconvexity is more useful when the function is Fréchet differentiable with an open domain. The next definition, in terms of the Fréchet derivative, is given in [36]. Definition 3 Let f be a finite real-valued function, which is defined on some open set in R n containing the set S. Suppose that h(x,d) is some directional derivative of f at the point x S in direction d R n. Then f is called pseudoconvex on S in terms of the derivative h iff the following implication holds: x S, y S, f (y) < f (x) h(x,y x) < 0. In the case when dom( f ) is an open set and f is Fréchet differentiable on dom( f ) the Clarke-Rockafellar subdifferential f (x) can be replaced in (7) by the set containing a single point { f (x)}. Theorem 2 Let f be a finite real-valued function, which is Fréchet differentiable on some open set in R n containing the convex set S. Then f is pseudoconvex in terms of the Fréchet derivative on S if and only if there exists a positive function p : S S (0,+ ) such that f (y) f (x) p(x,y) f (x),y x, x S, y S. Proof The proof follows the arguments of the proof of Theorem 1. Everywhere in the proof we replace dom( f ) by S. We take into account that S is not the domain of f but a convex subset of this open set. The function p is defined for all x S, y S as follows: { f (y) f (x) p(x,y) = f (x),y x, if f (y) < f (x) or f (x),y x > 0, (9) 1, otherwise. Instead of Lemma 3 we apply the following property of pseudoconvex functions [27, Proposition 1]: If f is pseudoconvex on S, then x S, y S, f (y) f (x) imply f (x),y x 0. (10)

6 Vsevolod I. Ivanov Theorem 3 Let f : E R be a lower semicontinuous and radially continuous proper extended real-valued function with a convex domain. Then f is pseudoconvex in terms of the Clarke-Rockafellar subdifferential if and only if there exists a positive function p : E E E (0,+ ) with p(x,y,x ) x,y x + p(y,x,y ) y,x y 0, (x,y) dom( f ) dom( f ), (x,y ) f (x) f (y) such that f (x) /0, f (y) /0. (11) Proof Let f be pseudoconvex. We prove that inequality (11) holds. Choose arbitrary x dom( f ), y dom( f ). If f (x) = /0 or f (y) = /0, then we have nothing to prove. Suppose that f (x) /0 and f (y) /0. It follows from Theorem 1 that there exists a function p : E E E (0,+ ) with and f (y) f (x) p(x,y,x ) x,y x, x f (x) (12) f (x) f (y) p(y,x,y ) y,x y, y f (y). (13) For example, the function p, which is defined by construction (8), satisfies both (12) and (13). If we add (12) and (13), then we obtain inequality (11). Let inequality (11) be satisfied. Choose arbitrary points x E, y E such that f (y) < f (x). We prove that x,y x < 0 for all x f (x). If f (x) = /0, then we have nothing to prove. Suppose that x dom( f ) and f (x) /0. It follows from f (y) < f (x) that y dom( f ). According to Lemma 1 we obtain that there exist sequences {u i } i=1, u i u, where u (x,y], and {u i }, u i f (u i ) such that u i,x u i > 0. We infer from inequality (11) that there exists a function p : E E E (0, ) such that p(x,u i,x ) x,u i x + p(u i,x,u i ) u i,x u i 0, x f (x). It follows from u i,x u i > 0, p(u i,x,u i ) > 0, p(x,u i,x ) > 0 that x,u i x < 0. Taking the limits when i + we obtain that x,u x 0 for all x f (x). It follows from u (x,y] that x,y x 0 for all x f (x). We prove that the case x,y x = 0 is impossible. Assume the contrary, that is, there exists ξ f (x) with ξ,y x = 0. Thanks to the inequality x,u i x < 0 for all x f (x) we obtain that 0 / f (x). Therefore ξ 0 and there exists d E such that ξ,d > 0. Consider the point z(t) = y + td, t > 0. We have 0 < ξ,td = ξ,z(t) y = ξ,z(t) x + ξ,y x = ξ,z(t) x. Since the function f is radially continuous, then we get from f (x) > f (y) that f (x) > f (z(t)) for all sufficiently small t > 0. It follows from Lemma 1 that there exist sequences {v i (t)}, v i (t) v(t) where v(t) [z(t),x), and {v i (t)}, v i (t) f (v i (t)) such that v i (t),x v i (t) > 0. According to (11) the following inequality holds p(x,v i (t),ξ ) ξ,v i (t) x + p(v i (t),x,v i (t)) v i (t),x v i (t) 0. We conclude from v i (t),x v i(t) > 0, p(v i (t),x,v i (t)) > 0, p(x,v i(t),ξ ) > 0 that ξ,v i (t) x < 0. (14) It follows from ξ,z(t) x > 0 and v [z(t),x) that ξ,v x > 0. Using that v i (t) v(t), we obtain that ξ,v i (t) x > 0 for all sufficiently large numbers i, which contradicts the inequality (14). Therefore f is pseudoconvex.

Characterizations of Pseudoconvex Functions and Semistrictly Quasiconvex Ones 7 Theorem 4 Let f be a finite real-valued function, which is Fréchet differentiable on some open set in R n containing the convex set S. Then f is pseudoconvex in terms of the Fréchet derivative on S if and only if there exists a positive function p : S S (0,+ ) such that p(x,y) f (x),y x + p(y,x) f (y),x y 0, (x,y) S S. Proof The proof is based on the arguments of the proof Theorem 3 and it refers to Theorem 2. The mean-value theorem in terms of the Fréchet derivative should be applied. We finish this section with the remark that inequality (11) is equivalent to the pseudomonotonicity of the Clarke-Rockafellar subdifferential, which is known from [41] (see Theorem 4.1 in this reference). 3 A derivative-free characterization of pseudoconvex functions The next result holds in both cases if we suppose that 0.(+ ) = + or 0.(+ ) = 0. Theorem 5 Let E be a Banach space. Suppose that f : E R is a lower semicontinuous pseudoconvex function in terms of the Clarke-Rockafellar subdifferential with a convex domain, and f (x) /0 for all x dom( f ). Then, for all x E, y E, λ [0,1] and ξ f (x + λ(y x)) there exists a number b > 0, which depends on x, y, ξ, λ such that the following conditions are satisfied: f [x + λ(y x)] λ b(x,y,ξ,λ) f (y) + [1 λ b(x,y,ξ,λ)] f (x), (15) 0 < b(x,y,ξ,λ) 1/λ, λ (0,1]. (16) Proof If x / dom( f ) or y / dom( f ) with λ > 0, then the inequality (15) is satisfied for arbitrary positive number b with 0 < λb < 1. It follows from Lemma 2 that f is quasiconvex. The claim is trivial if x = y or λ = 0 taking into account the convention 0.(+ ) = + or 0.(+ ) = 0. If λ = 1, then b(x,y,ξ,λ) = 1 satisfies (15). Choose arbitrary x dom( f ), y dom( f ), x y and λ (0,1). Denote z(λ) = x + λ(y x). We have f (z(λ)) /0. Take arbitrary ξ f (z(λ)). It follows from Theorem 1 that there exists a positive function q : E E E (0,+ ) such that and q(z(λ),x,ξ )[ f (x) f (z(λ))] ξ,x z(λ) = λ ξ,x y (17) q(z(λ),y,ξ )[ f (y) f (z(λ))] ξ,y z(λ) = (1 λ) ξ,y x (18) where q = 1/p. Let us multiply (17) by (1 λ), (18) by λ, and add the obtained inequalities. Then we have λ q(z(λ),y,ξ )[ f (y) f (z(λ))] + (1 λ)q(z(λ),x,ξ )[ f (x) f (z(λ))] λ(1 λ)( ξ,x y + ξ,y x ) = 0. We conclude from here that inequality (15) holds where b = q(z(λ),y,ξ )/[λ q(z(λ),y,ξ ) + (1 λ)q(z(λ),x,ξ )]. (19) It follows from (19) that 0 < λ b < 1 if 0 < λ < 1 and x y.

8 Vsevolod I. Ivanov Remark 1 The class of functions f : E R such that f (x) /0 are called subdifferentiable functions in terms of the Clarke-Rockafellar subdifferential. It is well known that every locally-lipschitz function is subdifferentiable, because for locally-lipschitz functions the Clarke generalized gradient coincides with the Clarke-Rockafellar subdifferential [42], and the Clarke generalized gradient f (x) is nonempty for every x E [8]. Example 1 Consider the function of one variable f (x) = 1 if x = 0, and f (x) = 0 if x 0. The inequality f (y) < f (x) is satisfied only if x = 0. On the other hand f (0,v) = for every v R. Therefore, f is pseudoconvex. There is no b > 0 which satisfies inequality (15) when x = 1, y = 1, and λ = 1/2. We can immediately see that the function is not lower semicontinuous at x = 0 and f (0) = /0. Remark 2 In the case when the function f is Fréchet differentiable and f (x) { f (x)}, we denote the dependence of the functions p, q, and b on the points x, y, and the number λ by p(x,y), q(x,y), b(x,y,λ) instead of p(x,y, f (x)), q(x,y, f (x)), b(x,y, f (x),λ), because the subdifferential contains a single point. The next theorem gives us a derivative-free complete characterization of pseudoconvex functions. Theorem 6 Suppose that S R n is a convex set and f is a continuously differentiable function, defined on some open set Γ, which contains S. Then the following claims are equivalent: (i) f is pseudoconvex on S in terms of the Fréchet derivative; (ii) there is a function b : S S [0,1] (0,+ ) such that for all x S, y S there exists the limit q(x,y) = lim λ 0 b(x,y,λ), q(x,y) is strictly positive, and the following inequalities are satisfied: f [x + λ(y x)] λ b(x,y,λ) f (y) + [1 λ b(x,y,λ)] f (x), (x,y,λ) S S [0,1], (20) 0 < b(x,y,λ) 1/λ, (x,y,λ) S S (0,1], x y. (21) Proof We prove the implication (i) (ii). Let f be pseudoconvex on S. It follows from Theorem 2 that the arguments of Theorem 5 are satisfied. We should replace ξ f (z), z = x + λ(y x) by f (z). It follows from the arguments of Theorem 5 that the function defined by b(x,y,λ) = q(z(λ),y)/[λ q(z(λ),y) + (1 λ)q(z(λ),x)], x S, y S, λ [0,1], (22) where z(λ) = x + λ(y x), q(x, y) = 1/p(x, y) and the function p is defined by (9), satisfies (20) and (21). We prove that a stronger condition holds under the stronger assumptions of continuous differentiability that is there exists the limit lim λ 0 b(x,y,λ) and it is strictly positive. Take arbitrary points x, y S. We prove that lim λ 0 q(z(λ),x) = 1. It follows from the explicit construction of the function p in the proof of Theorem 2 that q(z(λ),x) = λ f (z(λ)),x y /[ f (x) f (z(λ))] if f (x) < f (z(λ)) or f (z(λ)),x y > 0. Otherwise q(z(λ),x) = 1. On the other hand we have f (z(λ)) f (x) f [x + λ(y x)] f (x) lim = lim = f (x),y x. λ 0 λ λ 0 λ

Characterizations of Pseudoconvex Functions and Semistrictly Quasiconvex Ones 9 Therefore, using that f is continuously differentiable, we obtain that λ f (z(λ)),x y f (z(λ)),x y lim = lim = 1. λ 0 f (x) f (z(λ)) λ 0 f (x),x y We conclude from here and from the construction of the function q that in all possible cases To prove that lim q(z(λ),x) = 1. (23) λ 0 limb(x,y,λ) = q(x,y) > 0 (24) λ 0 we consider several cases: First, f (y) < f (x). Then f (y) < f (z(λ)) for all sufficiently small λ > 0. It follows from q = 1/p and (9) that q(z(λ),y) = (1 λ) f (z(λ)),y x /[ f (y) f (z(λ))]. (25) According to the continuous differentiability we obtain that f (x),y x lim q(z(λ),y) = = q(x,y). (26) λ 0 f (y) f (x) Then we conclude from (22), (23), (26) that (24) holds. Second, f (x), y x > 0. By the assumption f is continuously differentiable we have f (z(λ)),y x > 0 for all sufficiently small λ > 0. Therefore, f (z(λ)),y z(λ) > 0. By continuous differentiability of f we obtain that (26) is satisfied again, where f (y) > f (x) according to f (x),y x > 0, pseudoconvexity of f, and (10). By (22), (23) we have that (24) is satisfied. Third, f (y) > f (x) and f (x),y x < 0. We have f (y) > f (z(λ)) and f (z(λ)),y x < 0 for all sufficiently small λ > 0. It follows from here that f (z(λ)),y z(λ) < 0. Thanks to (9) we obtain that q(z(λ),y) = 1 = q(x,y). Therefore (24) is fulfilled again. Fourth, f (y) = f (x). Since f is quasiconvex, we obtain from f [x + λ(y x)] max{ f (x), f (y)} = f (x) that f (x),y x 0. Hence, arbitrary positive function b satisfies (20). It follows from (9) that p(x,y) = 1 = q(x,y). If we take b(x,y,λ) = 1 for all λ [0,1], then the required equality (24) is satisfied. It remains to consider the last fifth case when f (y) > f (x) and f (x),y x = 0. Since we have f [x + λ(y x)] f (x) lim = f (x),y x = 0 < f (y) f (x), λ 0 λ f [x + λ(y x)] < λ f (y) + (1 λ) f (x) for all sufficiently small positive numbers λ, which implies that b(x,y,λ) = 1 for all sufficiently small λ > 0. According to (9) we conclude that p(x,y) = 1 = q(x,y). Consequently, (24) is satisfied again.

10 Vsevolod I. Ivanov We prove the inverse claim (ii) (i). It follows from inequality (20) that f [x + λ(y x)] f (x) λ b(x,y,λ)[ f (y) f (x)]. Taking the limits when λ approaches zero with positive values, and taking into account that q(x,y) > 0 for all x, y S we obtain that f (x),y x q(x,y)[ f (y) f (x)], x S, y S. Therefore f is pseudoconvex. The following notion was introduced by Bector and Singh [5]. Definition 4 Let S R n be a convex set. A function f, defined on S, is called b-vex iff there exists a function b : S S [0,1] [0,+ ) such that f [λy + (1 λ)x)] λb(x,y,λ) f (y) + [1 λb(x,y,λ)] f (x), (27) and 0 λ b(x,y,λ) 1 for all x,y S, λ [0,1]. We can see from Theorem 6 that every pseudoconvex function is b-vex. On the other hand a function f, which is defined on a convex set S, is b-vex if and only if it is quasiconvex; see [32]. If f is a quasiconvex function, defined on the convex set S, then the following function b satisfies inequality (15); see [32]: { 1/λ, if f (y) f (x) and λ (0,1] b(x,y,λ) = 0, if f (y) < f (x) or λ = 0. The theorem of Crouzeix and Ferland [10, Theorem 2.2] is a condition for pseudoconvexity of a differentiable quasiconvex function. It follows from Definition 4 and Theorem 6 that we could consider inequalities (20), (21) and the conditions that for all x S, y S there exists the limit q(x,y) = lim λ 0 b(x,y,λ), q(x,y) is strictly positive, as derivative-free conditions for pseudoconvexity of a quasiconvex function. 4 Semistrict quasiconvexity and pseudoconvexity We proved in Theorem 5 that (15) and (16) provide a characterization of pseudoconvex functions. It is interesting which is the largest class of functions such that the characterization by inequalities (15) and (16) is a necessary and sufficient condition. Definition 5 ([12]) A proper function f : E R is called semistrictly quasiconvex iff dom( f ) is convex and for all x dom( f ), y dom( f ), λ (0,1) the following implication holds: f (y) < f (x) f [x + λ(y x)] < f (x). The following claim is known from [24], where the function is taken to be lower semicontinuous, finite real-valued and differentiable. It follows from the proof given in [24] that the radial lower semicontinuity is enough. Lemma 4 Let f : E R be a proper radially lower semicontinuous semistrictly quasiconvex function. Then f is quasiconvex.

Characterizations of Pseudoconvex Functions and Semistrictly Quasiconvex Ones 11 Remark 3 It is well known that every differentiable pseudoconvex function, defined on a convex set, is semistrictly quasiconvex on this set; see, for example, [4, Theorem 3.5.11]. A generalization of this claim to non-differentiable functions follows directly from Theorem 5. Theorem 7 Let E be a Banach space. Suppose that f : E R is a proper extended radially lower semicontinuous function. Then f is semistrictly quasiconvex if and only if for all x dom( f ), y dom( f ), and λ (0,1) there exists a number b, which depend on x, y, λ such that 0 < λ b(x,y,λ) < 1. (28) and f [x + λ(y x)] λ b(x,y,λ) f (y) + [1 λ b(x,y,λ)] f (x) (29) Proof Suppose that f is semistrictly quasiconvex. We prove that (28) and (29) hold. Take arbitrary x dom( f ), y dom( f ), and λ (0,1). Define the function b as follows: f [x+λ(y x)] f (x) λ[ f (y) f (x)], if f (x) f (y), f [x + λ(y x)] > f (x) b(x,y,λ) = f [x+λ(y x)] f (x) λ[ f (y) f (x)], if f (x) f (y), f [x + λ(y x)] > f (y) 1, otherwise. We prove (28). Suppose that f (x) f (y) and f [x + λ(y x)] > f (x). It follows from here, by semistrict quasiconvexity, that f (y) > f (x). Therefore λ b > 0. According to f (x) < f (y), by semistrict quasiconvexity, we obtain that f [x + λ(y x)] < f (y). Hence { f [x + λ(y x)] f (x)}/[ f (y) f (x)] < 1, and λ b < 1. Suppose that f (x) f (y) and f [x+λ(y x)] > f (y). Thanks to semistrict quasiconvexity we get f (x) > f (y) and f [x + λ(y x)] < f (x). Therefore { f [x + λ(y x)] f (y)}/[ f (x) f (y)] < 1. By easy manipulations we conclude that (28) is again satisfied. In the third case we have b = 1. Consequently 0 < λ b = λ < 1. We prove that f [x + λ(y x)] f (x) λ b(x,y,λ)[ f (y) f (x)]. (30) Consider the function b. In the cases when f (x) f (y) and f [x + λ(y x)] > f (x) or if f (x) f (y) and f [x + λ(y x)] > f (y) inequality (30) is satisfied as equality. Consider the third case. Let f (y) > f (x) and f [x + λ(y x)] f (x). Then f [x + λ(y x)] f (x) 0 < λ b(x,y,λ)[ f (y) f (x)]. Therefore (30) holds. Let f (x) > f (y) and f [x + λ(y x)] f (y). We obtain f [x + λ(y x)] f (y) 0 < [1 λ b(x,yλ)][ f (x) f (y)]. Therefore (30) is satisfied again. The last case that we have to consider is when f (x) = f (y). By radial lower semicontinuity and Proposition 4 f is quasiconvex. Then it follows from quasiconvexity that f [x + λ(y x)] f (x) 0 = λ b(x,y,λ)[ f (y) f (x)]. Thus (29) holds in all cases. The converse claim easy follows from the definition of semistrict quasiconvexity.

12 Vsevolod I. Ivanov Theorem 8 Let E be a Banach space. Suppose that f : E R is a proper lower semicontinuous function with a convex domain, and the assumptions of Theorem 5 hold. Then f is pseudoconvex in terms of the Clarke-Rockafellar subdifferential if and only if f is semistricyly quasiconvex and it satisfies the following implication: x dom( f ), y dom( f ), f (y) < f (x) ξ,y x = 0, ξ f (x). (31) Proof It follows from Remark 3 that every pseudoconvex function, which fulfills the hypothesis of the theorem is semistrictly quasiconvex. By pseudoconvexity it satisfies implication (31). Consider the converse claim. Let f be semistrictly quasiconvex and implication (31) holds. We prove that f is pseudoconvex. Take arbitrary x E, y E, and x f (x) with x,y x > 0. We have x dom( f ), because it is supposed that the subdifferential at x is nonempty. We prove that f (y) f (x). It follows from x,y x > 0 that f (x,y x) > 0. It follows from here that there exists a number ε > 0, sequences {x i } i=1, x i E, {t i } i=1, t i > 0 such that x i x, t i 0 and inf [ f (x i +t i u) f (x i )]/t i > 0, u B(y x,ε) i. Taking the number i sufficiently large we ensure that x i B(x,ε). Therefore, we have y x i B(y x,ε) and f [x i + t i (y x i )] > f (x i ). Using that f is lower semicontinuous and semistrictly quasiconvex we conclude from Lemma 4 that it is quasiconvex. Therefore, f (x i ) < f [x i +t i (y x i )] f (y). Hence, f (x) liminf i f (x i ) f (y). It follows from the converse implication that x E, y E, f (y) < f (x) implies x,y x 0, x f (x). Then, by (31), we obtain that f is pseudoconvex. The following example illustrates the last theorem. Example 2 Consider the function f : R 2 R such that f (x 1,x 2 ) = (x 2 1 x 2) 3. It is semistrictly quasiconvex. Indeed, let f (y) < f (x) where x = (x 1,x 2 ) and y = (y 1,y 2 ). Therefore y 2 1 y 2 < x 2 1 x 2. It follows from here that y 2 x 2 > y 2 1 x2 1 2x 1(y 1 x 1 ). By y 2 1 y 2 < x 2 1 x 2, we have for every λ (0,1) [x 1 + λ(y 1 x 1 )] 2 [x 2 + λ(y 2 x 2 )] < x 2 1 x 2. Therefore f [x + λ(y x)] < f (x), which implies that f is semistrictly quasiconvex. This function is not pseudoconvex on the whole plane. Indeed, it is continuously differentiable and the Clarke-Rockafellar derivative of f coincides with the usual directional derivative. If d = (d 1,d 2 ) is a direction, then Hence f (x,d) = f (x),d = 3(x 2 1 x 2 ) 2 (2x 1 d 1 d 2 ). f (x,y x) = 3(x 2 1 x 2 ) 2 (2x 1 (y 1 x 1 ) + x 2 y 2 ) 0. (32) If x 2 1 = x 2, then f (x,y x) = f (x),y x = 0. Therefore, the function is not pseudoconvex on every open set in R 2 which intersects the curve x 2 1 = x 2. On the other hand the function is pseudoconvex on every set which does not intersect the plane curve x 2 1 = x 2.

Characterizations of Pseudoconvex Functions and Semistrictly Quasiconvex Ones 13 5 Characterizations of the solution set of a pseudoconvex problem Let S R n be a given set and f be a finite-valued real function, which is defined on some open set Γ such that S Γ. Consider the problem Minimize f (x) subject to x S (P) Denote by S the solution set argmin { f (x) x S} of (P), and let it be nonempty. Suppose that x is any fixed element of this set. Consider the following notations of sets: S 0 = {x S f ( x),x x = 0, f (x) = f ( x)}; S 0 1 = {x S f ( x),x x 0, f (x) = f ( x)}. The following theorem is useful for problems with multiple solutions. Proposition 1 ([37]) Assume that f is a twice continuously differentiable convex function on some open convex set Γ R n containing the convex set S. Let x be any fixed point from S. Then S = S 0 = S 0 1. The following example shows that Proposition 1 does not hold anymore when the function is not convex. Example 3 Consider the function of two variables f (x 1,x 2 ) = x 2 /x 1. It is pseudoconvex in terms of the Fréchet derivative on the set Γ = {x = (x 1,x 2 ) R 2 x 1 > 0}, but not convex on the convex set S = {(x 1,x 2 ) R 2 1 x 1 2, 0 x 2 1}. The set of minimizers of f on S is the line segment S = {(x 1,x 2 ) R 2 1 x 1 2, x 2 = 0}. We can see immediately that f (x) is not constant on S. We generalize Proposition 1 to the case when the function is locally Lipschitz and pseudoconvex in terms of the Clarke generalized gradient. We suppose that E is a Banach space and f is locally Lipschitz on some open convex set in E, containing S. Denote by f (x) the Clarke generalized gradient of f at x. Consider the sets Ŝ = {x S ξ f ( x) : ξ,x x = 0, ξ,v 0 f 0 (x,v) 0}, Ŝ 1 = {x S ξ f ( x) : ξ,x x 0, ξ,v 0 f 0 (x,v) 0}. Theorem 9 Let f : Γ R be locally Lipschitz and pseudoconvex on some open convex set Γ E in terms of the Clarke generalized gradient. Suppose that S Γ is an arbitrary convex set and x is any fixed point from the solution set S. Then S = Ŝ = Ŝ 1.

14 Vsevolod I. Ivanov Proof It is trivial that Ŝ Ŝ 1. We prove that Ŝ 1 S. Let x Ŝ 1. There exists ξ f ( x) such that ξ,x x 0. Therefore ξ, x x 0. It follows from x Ŝ 1 that f 0 (x, x x) 0. According to the pseudoconvexity of f we have f ( x) f (x). Therefore x S. We prove that S Ŝ. Suppose that x S. We prove that there exists ξ f ( x) with ξ,x x = 0. Indeed, we have f (x) = f ( x), because x S. It follows from Lemma 3 that x,x x 0 for all x f ( x), because Lemma 3 remains valid when the function is finite-valued locally Lipschitz on some open convex set Γ instead of the whole space E. On the other hand we have from x S, by S is convex that x + t(x x) S for all t [0,1]. According to x S we obtain f [ x +t(x x)] f ( x). Therefore, f 0 ( x,x x) = limsup y x;t 0 f [y +t(x x)] f (y)] t limsup t 0 f [ x +t(x x)] f ( x) t Since f 0 ( x,x x) = max{ x,x x x f ( x)} and the Clarke generalized gradient is weakly compact (see [8]), then we conclude from here that there exists ξ f ( x) with ξ,x x = 0. We prove that ξ,v 0 implies f 0 (x,v) 0. Let ξ,v 0 where v E. Thanks to ξ,x x = 0 we obtain that ξ,(x +tv) x = t ξ,v + ξ,x x 0, t 0. It follows from the pseudoconvexity of f that We conclude from here that f (x +tv) f ( x) = f (x). f 0 (x,v) limsup t 0 f (x +tv) f (x) t 0. 0. Thus S Ŝ and the proof is complete. The following lemma is well known and it is a particular case of Farkas lemma. Lemma 5 Let a, b R n, b 0. If for all d R n then there exists p > 0 such that b = pa. a,d 0 implies b,d 0, Definition 6 A locally Lipschitz function f defined on some open set Γ is called strictly differentiable at the point x Γ with a strict derivative D s f (x), which is a linear continuous operator from R n to R n [8], iff there exists the limit D s f (x),v = lim y x;t 0 [ f (y +tv) f (y)]/t, v Rn. If the function f is strictly differentiable on Γ, then f (x) {D s f (x)} for all x Γ [8, Proposition 2.2.4]. The following result is a consequence of Theorem 9 when the function is pseudoconvex in terms of the strict derivative. Consider the sets S = {x S D s f ( x),x x = 0, p(x) > 0 : D s f (x) = p(x)d s f ( x)}; S 1 = {x S D s f ( x),x x 0, p(x) > 0 : D s f (x) = p(x)d s f ( x)}.

Characterizations of Pseudoconvex Functions and Semistrictly Quasiconvex Ones 15 Corollary 1 Suppose that f is strictly differentiable and pseudoconvex in terms of the strict derivative on some open convex set Γ R n. Let x be any fixed point from S, and S be a convex set such that S Γ. Then S = S = S 1. Proof The inclusion S 1 S is a consequence of Theorem 9. We prove the inclusion S S. Let x S. By Theorem 9 we obtain that D s f ( x),x x = 0 and D s f ( x),v 0, v R n D s f (x),v 0. If D s f (x) 0, then we conclude from Lemma 5 that there exists p(x) > 0 with D s f (x) = p(x)d s f ( x). If D s f (x) = 0, since f is pseudoconvex on Γ, then x is a global minimizer of f on Γ. By x S, we obtain that f ( x) = f (x). Therefore, x is also a global minimizer of f on Γ. By Proposition 2.3.2 in [8] we obtain that 0 f ( x), because Γ is open. It follows from f ( x) = {D s f ( x)} (see [8, Proposition 2.2.4]) that D s f ( x) = 0. If follows from here that an arbitrary positive number p satisfies the equation D s f (x) = pd s f ( x). Acknowledgements The author is thankful to both referees for their careful reading of the manuscript and valuable remarks, which improved the paper. The work is partially supported by Technical University of Varna. Acknowledgements The author thanks to the organizers of the 10th International Symposium GCM-10, Publishing House Springer and the editors of the Journal of Global Optimization for publishing this paper. References 1. Aussel, D.: Subdifferential properties of quasiconvex and pseudoconvex functions: Unified approach. J. Optim. Theory Appl. 9, 29 45 (1998) 2. Avriel, M., Diewert, W.E., Schaible, S., Zang, I.: Generalized Concavity. Plenum Press, New York (1988) 3. Avriel, M., Schaible, S.: Second order characterizations of pseudoconvex functions. Math. Program. 14, 170 185 (1978) 4. Bazaraa, M.S., Shetty, C.M.: Nonlinear Programming - Theory and Algorithms. John Wiley & Sons, New York (1979) 5. Bector, C.R., Singh, C.: B-vex functions. J. Optim. Theory Appl. 71, 237 253 (1991) 6. Burke, J.V., Ferris, M.C.: Characterization of the solution sets of convex programs. Oper. Res. Lett. 10, 57 60 (1991) 7. Cambini, A., Martein, L.: Generalized Convexity and Optimization. Lecture Notes in Econom. and Math. Systems 616, Springer, Berlin (2009) 8. Clarke, F.H.: Optimization and Nonsmooth Analysis. John Wiley & Sons, New York (1983) 9. Crouzeix, J.-P., Eberhard, A., Ralph, D.: A geometrical insight on pseudoconvexity and pseudomonotonicity. Math. Program., Ser. B 123, 61 83 (2010) 10. Crouzeix, J.-P., Ferland, J. A.: Criteria for quasi-convexity and pseudo-convexity: relations and comparisons. Math. Program. 23, 193 205 (1982) 11. Daniilidis, A., Hadjisavvas, N.: On the subdifferentials of quasiconvex and pseudoconvex functions and cyclic monotonicity. J. Math. Anal. Appl. 237, 30 42 (1999) 12. Daniilidis, A., Hadjisavvas, N.: Characterization of nonsmooth semistrictly quasiconvex and strictly quasiconvex function. J. Optim. Theory Appl. 102, 525 536 (1999) 13. Daniilidis, A., Ramos, Y.G.: Some remarks on the class of continuous (semi-)strictly quasiconvex functions. J. Optim. Theory Appl. 133, 37 48 (2007) 14. Diewert, W.E., Avriel, M., Zang, I.: Nine kinds of quasiconcavity and concavity. J. Econom. Theory 25, 397 420 (1981) 15. Ginchev, I., Ivanov, V.I.: Second-order characterizations of convex and pseudoconvex functions. J. Appl. Anal. 9, 261 273 (2003)

16 Vsevolod I. Ivanov 16. Ginchev, I., Ivanov, V.I.: Higher-order pseudoconvex functions. In: Konnov, I., Luc, D.T., Rubinov, A.M.(eds.). Proc. 8th International Symposium on Generalized Convexity and Monotonicity. Lecture Notes in Econom. and Math. Systems. 583, 247 264 (2007) 17. Giorgi, G., Guerraggio, A., Thierfelder, A.: Mathematics of Optimization. Elsevier, Amsterdam (2004) 18. Hadjisavvas, N., Schaible, S.: On strong pseudomonotonicity and (semi)strict quasimonotonicity. J. Optim. Theory Appl. 79, 139 155 (1993) 19. Hassouni, A., Jaddar, A.: On pseudoconvex functions and applications to global optimization. ESAIM: Proc. 20, 138 148 (2007) 20. Ivanov, V.I.: On strict pseudoconvexity. J. Appl. Anal. 13, 183 196 (2007) 21. Ivanov, V.I.: On variational inequalities and nonlinear programming problem. Studia Sci. Math. Hungar. 45, 483 491 (2008) 22. Ivanov, V.I.: On a theorem due to Crouzeix and Ferland. J. Global Optim. 46, 31 47 (2010) 23. Ivanov, V.I.: Optimization and variational inequalities with pseudoconvex functions. J. Optim. Theory Appl. 146, 602 616 (2010) 24. Karamardian, S.: Strictly quasi-convex (concave) functions and duality in mathematical programming. J. Math. Anal. Appl. 20, 344 358 (1967) 25. Karamardian, S., Schaible, S.: Seven kinds of monotone maps. J. Optim. Theory Appl. 66, 37 46 (1990) 26. Komlósi, S.: Some properties of nondifferentiable pseudoconvex functions. Math. Program. 23, 232 237 (1983) 27. Komlósi, S.: First and second order characterizations of pseudolinear functions. European J. Oper. Res. 67, 278 286 (1993) 28. Komlósi, S.: On pseudoconvex functions. Acta Sci. Math. (Szeged) 57, 569 586 (1993) 29. Komlósi., S.: Generalized monotonicity in nonsmooth analysis. In: Komlósi, S., Rapcsák, T., Schaible, S. (eds.). Proc. 4th International Symposium on Generalized Convexity and Monotonicity. Lecture Notes in Econom. and Math. Systems. 405, 263 275 (1994) 30. Komlósi, S.: Generalized monotonicity and generalized convexity. J. Optim. Theory Appl. 84, 361 376 (1995) 31. Levi, E.E.: Studi sui punti singolari essenziali delle funzioni analitiche di due o più variabili complesse. Ann. Mat. Pura Appl. 17, 61 68 (1910) 32. Li, X.F., Dong, J.L., Liu, Q.H.: Lipschitz b-vex functions and nonsmooth programming. J. Optim. Theory Appl. 93, 557 573 (1997) 33. Luc, D.T.: Characterizations of quasiconvex functions. Bull. Austral. Math. Soc. 48, 393 405 (1993) 34. Luc, D.T.: On generalized convex nonsmooth functions. Bull. Austral. Math. Soc. 49, 139 149 (1994) 35. Mangasarian, O.L.: Pseudo-convex functions. SIAM J. Control 3, 281 290 (1965) 36. Mangasarian, O.L.: Nonlinear Programming. Mc Graw-Hill, New York (1969) 37. Mangasarian, O.L.: A simple characterization of solution sets of convex programs. Oper. Res. Lett. 7, 21 26 (1988) 38. Mereau, P., Paquet, J.-G.: Second order conditions for pseudoconvex functions. SIAM J. Appl. Math. 27, 131 137 (1974) 39. Ortega, J.M., Rheinboldt, W.C.: Iterative Solution of Nonlinear Equations of Several Variables. Academic Press, New York (1970) 40. Penot, J.-P.: Generalized convexity in the light of nonsmooth analysis. Lecture Notes in Econom. and Math. Systems. 429, 269 290 (1995) 41. Penot, J.-P., Quang, P.H.: Generalized convexity of functions and generalized monotonicity of set-valued maps. J. Optim. Theory Appl. 92 343 356 (1997) 42. Rockafellar, R.T.: Generalized directional derivatives and subgradients of nonconvex functions. Canad. J. Math. 32, 257 280 (1980) 43. Soleimani-damaneh, M.: Characterization of nonsmooth quasiconvex and pseudoconvex functions. J. Math. Anal. Appl. 330, 1387 1392 (2007) 44. Soleimani-damaneh, M.: On generalized convexity in Asplund spaces. Nonlinear Anal. 70, 3072 3075 (2009) 45. Thompson, W.A., Parke, D.W.: Some properties of generalized concave functions. Oper. Res. 21, 305 313 (1973) 46. Tuy, H.: Sur les inégalités linéaires. Colloq. Math. 13, 107 123 (1964) 47. Yang, X.Q.: Continuous generalized convex functions and their characterizations. Optimization 54, 495 506 (2005)