Pseudonormality and a Lagrange Multiplier Theory for Constrained Optimization 1

Transcription

1 JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol. 114, No. 2, pp , August 2002 ( 2002) Pseudonoality and a Lagange Multiplie Theoy fo Constained Optiization 1 D. P. BERTSEKAS 2 and A. E. OZDAGLAR 3 Counicated by P. Tseng Abstact. We conside optiization pobles with equality, inequality, and abstact set constaints, and we exploe vaious chaacteistics of the constaint set that iply the existence of Lagange ultiplies. We pove a genealized vesion of the Fitz John theoe, and we intoduce new and geneal conditions that extend and unify the ajo constaint qualifications. Aong these conditions, two new popeties, pseudonoality and quasinoality, eege as cental within the taxonoy of inteesting constaint chaacteistics. In the case whee thee is no abstact set constaint, these popeties povide the connecting link between the classical constaint qualifications and two distinct pathways to the existence of Lagange ultiplies: one involving the notion of quasiegulaity and the Fakas lea, and the othe involving the use of exact penalty functions. The second pathway also applies in the geneal case whee thee is an abstact set constaint. Key Wods. Pseudonoality, infoative Lagange ultiplies, constaint qualifications, exact penalty functions. 1. Intoduction We conside finite-diensional optiization pobles of the fo in f(x), (1a) s.t. x C, (1b) 1 This eseach was suppoted by NSF unde Gant ACI Pofesso, Depatent of Electical Engineeing and Copute Science, Massachusetts Institute of Technology, Cabidge, Massachusetts. 3 Gaduate Student, Depatent of Electical Engineeing and Copute Science, Massachusetts Institute of Technology, Cabidge, Massachusetts Plenu Publishing Copoation

2 288 JOTA: VOL. 114, NO. 2, AUGUST 2002 whee the constaint set C consists of equality and inequality constaints as well as an additional abstact set constaint x X, CGX {x h 1 (x)g0,...,h (x)g0} {x g 1 (x) 0,...,g (x) 0}. (2) We assue thoughout the pape that f, h i, g j ae sooth (continuously diffeentiable) functions fo R n to R, and that X is a nonepty closed set. In ou notation, all vectos ae viewed as colun vectos, and a pie denotes tansposition, so x y denotes the inne poduct of the vectos x and y. We will use thoughout the standad Euclidean no x G(x x) 1 2. Necessay conditions fo the above poble can be expessed in tes of tangent cones, noal cones, and thei polas. In ou teinology, a vecto y is a tangent of a set S R n at a vecto x S if eithe yg0 o thee exists a sequence {x k } S such that x k x fo all k and x k x, (x k Ax) x k Ax y y. An equivalent definition often found in the liteatue [e.g., Bazaaa, Sheali, and Shetty (Ref. 1), Rockafella and Wets (Ref. 2)] is that thee exist a sequence {x k } S, with x k x, and a positive sequence {α k } such that α k 0 and (x k Ax) α k y. The set of all tangents of S at x is denoted by T S (x) and is also efeed to as the tangent cone of S at x. The pola cone of any cone T is defined by T*G{z z y 0, y T}. Fo a nonepty cone T, we will use the well-known elation T (T*)*, which holds with equality if T is closed and convex. Fo a closed set X and a point x X, we will also use the noal cone of X at x, denoted by N X (x), which is obtained fo the pola cone T X (x)* by eans of a closue opeation. In paticula, we have z N X (x) if thee exist sequences {x k } X and {z k } such that x k x, z k z, and z k T X (x k )* fo all k. Equivalently, the gaph of N X ( ), viewed as a pointto-set apping, {(x, z) z N X (x)}, is the closue of the gaph of T X ( )*. The noal cone, intoduced by Modukhovich (Ref. 3), has been studied by seveal authos, and is of cental ipotance in nonsooth analysis [see the books by Aubin and Fankowska (Ref. 4), Rockafella and Wets (Ref. 2), and Bowein and Lewis (Ref. 5); fo the case whee X is a closed subset of R n, ou definition of N X (x) coincides with the ones used by these authos]. In geneal, we have T X (x)* N X (x), fo any x X.

3 JOTA: VOL. 114, NO. 2, AUGUST Howeve, N X (x) ay not be equal to T X (x)*, and in fact it ay not even be a convex set. In the case whee T X (x)*gn X (x), we will say that X is egula at x. The te egula at x in the sense of Clake is also used in the liteatue [see, Rockafella and Wets (Ref. 2, p. 199)]. Two popeties of egulaity that ae ipotant fo ou puposes ae that (i) if X is convex, then it is egula at each x X, and (ii) if X is egula at soe x X, then T X (x) is convex [Rockafella and Wets (Ref. 2, pp. 203 and 221)]. A classical necessay condition fo a vecto x* C to be a local iniu of f ove C is f(x*) y 0, y T C (x*), (3) whee T C (x*) is the tangent cone of C at x* [see e.g. Bazaaa, Sheali, and Shetty (Ref. 1), Betsekas (Ref. 6), Hestenes (Ref. 7), Rockafella (Ref. 8), Rockafella and Wets (Ref. 2)]. Necessay conditions that involve Lagange ultiplies elate to the specific epesentation of the constaint set C in tes of the constaint functions h i and g j. In paticula, we say that the constaint set C of Eq. (2) adits Lagange ultiplies at a point x* C if, fo evey sooth cost function f fo which x* is a local iniu of poble (1), thee exist vectos λ*g(λ* 1,...,λ* ) and µ*g(µ* 1,...,µ* ) that satisfy the following conditions: f(x*)c λ*i h i (x*)c µ* j g j (x*) y 0, y TX (x*), (4) µ* j 0,,...,, (5) µ* j G0, j A(x*), (6) whee A(x*)G{ j g j (x*)g0} is the index set of inequality constaints that ae active at x*. Condition (6) is efeed to as the copleentay slackness condition (CS fo shot). A pai (λ*, µ*) satisfying Eqs. (4) (6) will be called a Lagange ultiplie vecto coesponding to f and x*. When thee is no dange of confusion, we efe to (λ*, µ*) siply as a Lagange ultiplie vecto o a Lagange ultiplie. We obseve that the set of Lagange ultiplie vectos coesponding to a given f and x* is a (possibly epty) closed and convex set. The condition (4) is consistent with the taditional chaacteistic popety of Lagange ultiplies: endeing the Lagangian function stationay

4 290 JOTA: VOL. 114, NO. 2, AUGUST 2002 at x* [cf. Eq. (3)]. When X is a convex set, Eq. (4) is equivalent to f(x*)c λ*i h i (x*)c µ* j g j (x*) (xax*) 0, x X. (7) This is because, when X is convex, T X (x*) is equal to the closue of the set of feasible diections F X (x*), which is in tun equal to the set of vectos of the fo α(xax*), whee αh0 and x X. IfXGR n, Eq. (7) becoes f(x*)c λ*i h i (x*)c µ* j g j (x*)g0, which togethe with the nonnegativity condition (5) and the CS condition (6), copise the failia Kaush Kuhn Tucke conditions. In the case whee XGR n, it is well-known [see e.g. Betsekas (Ref. 6, p. 332)] that, fo a given sooth f fo which x* is a local iniu, thee exist Lagange ultiplies if and only if f(x*) y 0, y V(x*), whee V(x*) is the cone of fist-ode feasible vaiations at x*, given by V(x*)G{y h i (x*) yg0,,...,, g j (x*) y 0, j A(x*)}. This esult, a diect consequence of the Fakas lea, leads to the classical theoe that the constaint set adits Lagange ultiplies at x* if T C (x*)gv(x*). In this case, we say that x* is a quasiegula point o that quasiegulaity holds at x* {othe tes used ae that x* satisfies the Abadie constaint qualification [Abadie (Ref. 9), Bazaaa, Sheali, and Shetty (Ref. 1)], o that x* is a egula point [Hestenes (Ref. 7)]}. Since quasiegulaity is a soewhat abstact popety, it is useful to have oe eadily veifiable conditions fo the adittance of Lagange ultiplies. Such conditions ae called constaint qualifications, and have been investigated extensively in the liteatue. Soe of the ost useful ones ae the following: (CQ1) (CQ2) XGR n and x* is a egula point in the sense that the equality constaint gadients h i (x*),,...,, and the active inequality constaint gadients g j (x*), j A(x*), ae linealy independent. XGR n, the equality constaint gadients h i (x*),,...,, ae linealy independent, and thee exists a y R n such that h i (x*) yg0, g j (x*) yf0,,...,, j A(x*).

5 JOTA: VOL. 114, NO. 2, AUGUST Fo the case whee thee ae no equality constaints, this is known as the Aow Huwitz Uzawa constaint qualification, intoduced in Ref. 10. In the oe geneal case whee thee ae equality constaints, it is known as the Mangasaian Foovitz constaint qualification, intoduced in Ref. 11. (CQ3) XGR n, the functions h i ae linea and the functions g j ae concave. It is well-known that all of the above constaint qualifications iply the quasiegulaity condition T C (x*)gv(x*), and theefoe iply that the constaint set adits Lagange ultiplies [see e.g. Betsekas (Ref. 6), o Bazaaa, Sheali, and Shetty (Ref. 1); a suvey of constaint qualifications is given by Peteson (Ref. 12)]. These esults constitute the classical pathway to Lagange ultiplies fo the case whee XGR n. Howeve, thee is anothe equally poweful appoach to Lagange ultiplies, based on exact penalty functions, which has not eceived uch attention thus fa. In paticula, let us say that the constaint set C adits an exact penalty at the feasible point x* if, fo evey sooth function f fo which x* is a stict local iniu of f ove C, thee is a scala ch0 such that x* is also a local iniu of the function F c (x)gf(x)cc ove x X, whee we denote g + j (x)gax{0, g j (x)}. hi (x) C g + j (x), Note that, like adittance of Lagange ultiplies, adittance of an exact penalty is a popety of the constaint set C, and does not depend on the cost function f of poble (1). We intend to use exact penalty functions as a vehicle towad asseting the adittance of Lagange ultiplies. Fo this pupose, thee is no loss of geneality in equiing that x* be a stict local iniu, since we can eplace a cost function f(x) with the cost function f(x)c xax* 2 without affecting the poble s Lagange ultiplies. On the othe hand, if we allow functions f involving ultiple local inia, it is had to elate constaint qualifications such as the peceding ones, the adittance of an exact penalty, and the adittance of Lagange ultiplies, as we show in Exaple 7.7 of Section 7.

6 292 JOTA: VOL. 114, NO. 2, AUGUST 2002 Note two ipotant points, which illustate the significance of exact penalty functions as a unifying vehicle towad guaanteeing the adittance of Lagange ultiplies. (a) (b) If X is convex and the constaint set adits an exact penalty at x*, it also adits Lagange ultiplies at x*. [This follows fo Poposition of Bonnans and Shapio (Ref. 13); see also the subsequent Poposition 4.4, which genealizes the Bonnans Shapio esult by assuing that X is egula at x* instead of being convex.] All of the above constaint qualifications CQ1 CQ3 iply that C adits an exact penalty. [The case of CQ1 was teated by Pietzykowski (Ref. 14); the case of CQ2 was teated by Zangwill (Ref. 15), Han and Mangasaian (Ref.16), and Bazaaa and Goode (Ref. 17); the case of CQ3 will be dealt with in the pesent pape see the subsequent Popositions 3.1 and 5.1.] Figue 1 suaizes the elationships discussed above fo the case XGR n, and highlights the two distinct pathways to the adittance of Lagange ultiplies. The two key notions, quasiegulaity and adittance of an exact penalty, do not see to be diectly elated (see Exaples 7.2 and 7.3 in Section 7), but we will show in this pape that they ae connected though the new notion of constaint pseudonoality, which iplies both while being iplied by the constaint qualifications CQ1 CQ3. Anothe siila connecting link is the notion of constaint quasinoality, which is iplied by pseudonoality. Unfotunately, when X is a stict subset of R n, the situation changes significantly because thee does not appea to be a satisfactoy extension of the notion of quasiegulaity, which iplies adittance of Lagange ultiplies. Fo exaple, the classical constaint qualification of Guignad (Ref. 18) esebles quasiegulaity, but equies additional conditions that ae not easily veifiable. In paticula, Guignad (Ref. 18, Theoe 2) has shown that the constaint set adits Lagange ultiplies at x* if V(x*) conv(t X (x*))gconv(t C (x*)), (8) and if the vecto su V(x*)*CT X (x*)* is a closed set [hee, conv (S) denotes the closue of the convex hull of a set S]. The Guignad conditions ae equivalent to V(x*)*CT X (x*)*gt C (x*)*, which in tun can be shown to be a necessay and sufficient condition fo the adittance of Lagange ultiplies at x* based on the classical esults

7 JOTA: VOL. 114, NO. 2, AUGUST Fig. 1. Chaacteizations of the constaint set C that iply adittance of Lagange ultiplies in the case whee XGR n. of Gould and Tolle (Refs ). In the special case whee XGR n, we have T X (x*)gr n, and the condition (8) becoes V(x*)Gconv(T C (x*)) T X (x*)*g{0}, [o equivalently, V(x*)*GT C (x*)*], which is a siila but slightly less estictive constaint qualification than quasiegulaity. Howeve, in the oe geneal case whee X R n, condition (8) and the closue of the set V(x*)*CT X (x*)* see had to veify. [Guignad (Ref. 18) has teated only the cases whee X is eithe R n o the nonnegative othant.] In this pape, we focus on the connections between constaint qualifications, Lagange ultiplies, and exact penalty functions. Much of ou analysis is otivated by an enhanced set of Fitz John necessay conditions that ae intoduced in the next section. Weake vesions of these conditions wee shown in a lagely ovelooked analysis by Hestenes (Ref. 7) fo the case whee XGR n, and in the fist autho s ecent textbook (Ref. 6) fo the

8 294 JOTA: VOL. 114, NO. 2, AUGUST 2002 case whee X is a closed convex set (see the discussion in Section 2). They ae stengthened and futhe genealized in Section 2 fo the case whee X is a closed but not necessaily convex set. In paticula, we show the existence of Fitz John ultiplies that satisfy soe additional sensitivity-like conditions. These conditions otivate the intoduction of two new types of Lagange ultiplies, called infoative and stong. We show that infoative and stong Lagange ultiplies exist when the tangent cone is convex and the set of Lagange ultiplies is nonepty. In Section 3, we intoduce the notions of pseudonoality and quasinoality, and we discuss thei connection with classical esults elating constaint qualifications and the adittance of Lagange ultiplies. Quasinoality seves alost the sae pupose as pseudonoality when X is egula, but fails to povide the desied theoetical unification when X is not egula (copae with Fig. 6). Fo this eason, it appeas that pseudonoality is a theoetically oe inteesting notion than quasinoality. In addition, in contast with quasinoality, pseudonoality adits an insightful geoetical intepetation. In Section 3, we intoduce also a new and natual extension of the Mangasaian Foovitz constaint qualification, which applies to the case whee X R n and iplies pseudonoality. In Section 4, we ake the connection between pseudonoality, quasinoality, and exact penalty functions. In paticula, we show that pseudonoality iplies the adittance of an exact penalty, while being iplied by the ajo constaint qualifications. In the pocess, we pove in a unified way that the constaint set adits an exact penalty fo a uch lage vaiety of constaint qualifications than has been known hitheto. We note that, taditionally, exact penalty functions have been viewed as a coputational device and they have not been integated ealie within the theoy of constaint qualifications in the anne descibed hee. Let us also note that exact penalty functions ae elated to the notion of calness, intoduced and suggested as a constaint qualification by Clake (Refs ). Howeve, thee ae soe ipotant diffeences between the notions of calness and adittance of an exact penalty. In paticula, calness is a popety of the poble (1) and depends on the cost function f, while adittance of an exact penalty is a popety of the constaint set and is independent of the cost function. Moe ipotantly fo the puposes of this pape, calness is not useful as a unifying theoetical vehicle because it does not elate well with othe ajo constaint qualifications. Fo exaple CQ1, one of the ost coon constaint qualifications, does not iply calness of poble (1), as is indicated by Exaple 7.7 of Section 7; evesely, calness of the poble does not iply CQ1. In Section 5, we discuss soe special esults that facilitate the poofs of adittance of Lagange ultiplies and of an exact penalty. In Section

9 JOTA: VOL. 114, NO. 2, AUGUST , we genealize soe of ou analysis to the case of a convex pogaing poble and we povide a geoetic intepetation of pseudonoality. Finally, in Section 7, we povide exaples and counteexaples that claify the inteelations between the diffeent chaacteizations that we have intoduced. 2. Enhanced Fitz John Conditions The Fitz John necessay optiality conditions (Ref. 23) ae used often as the stating point fo the analysis of Lagange ultiplies. Unfotunately, these conditions in thei classical fo ae not sufficient to deive the adittance of Lagange ultiplies unde soe of the standad constaint qualifications, such as when XGR n and the constaint functions h i and g j ae linea [cf. CQ3]. Recently, the classical Fitz John conditions have been stengthened though the addition of an exta necessay condition, and thei effectiveness has been significantly enhanced [see Hestenes (Ref. 7) fo the case XGR n, and Betsekas (Ref. 6, Poposition ) fo the case whee X is a closed convex set]. The following poposition extends these esults by allowing the set X to be nonconvex, and also by showing that the Fitz John ultiplies can be selected to have soe special sensitivity-like popeties [see condition (iv) below]. Poposition 2.1. Let x* be a local iniu of poble (1) (2). Then, thee exist scalas µ* 0, λ* 1,...,λ*, and µ* 1,...,µ*, satisfying the following conditions: (i) A µ* 0 f(x*)c λ*i h i (x*)c µ* j g j (x*) N X (x*). (ii) µ* j 0, fo all jg0,1,...,. (iii) µ* 0, λ* 1,...,λ*, µ* 1,...,µ* ae not all equal to 0. (iv) If the index set I J is nonepty, whee IG{i λ* i 0}, JG{ j 0 0 µ* j H0}, thee exists a sequence {x k } X that conveges to x* and is such that, fo all k, f(x k )Ff(x*), λ* i h i (x k )H0, i I, µ* j g j (x k )H0, j J, (9) h i (x k ) Go(w(x k )), i I, g + j (x k )Go(w(x k )), j J, (10)

10 296 JOTA: VOL. 114, NO. 2, AUGUST 2002 whee w(x) Gin{in i I h i (x), in g + j (x)}. (11) j J Poof. We use a quadatic penalty function appoach. Fo each kg1,2,...,weconside the penalized poble whee in s.t. F k (x) f(x)c(k 2) (hi (x)) 2 x X S, C(k 2) (g + j (x)) 2 C(1 2) xax* 2, SG{x xax* (} and H0 is such that f(x*) f(x) fo all feasible x with x S. Since X S is copact, by the Weiestass theoe, we can select an optial solution x k of the above poble. We have, fo all k, f(x k )C(k 2) (hi (x k )) 2 C(k 2) (g + j (x k )) 2 C(1 2) x k Ax* 2 GF k (x k ) F k (x*)gf(x*), (12) and since f(x k ) is bounded ove X S, we obtain li h i (x k ) G0, k S li g + j (x k ) G0, k S,...,,,...,; othewise, the left-hand side of Eq. (12) would becoe unbounded fo above as k S. Theefoe, evey liit point x of {x k } is feasible; i.e., x C. Futheoe, Eq. (12) yields f(x k )C(1 2) x k Ax* 2 f(x*), fo all k, so by taking the liit as k S, we obtain f(x )C(1 2) x Ax* 2 f(x*). Since x S and x is feasible, we have f(x*) f(x ), which when cobined with the peceding inequality yields x Ax* G0 so that x Gx*. Thus, the sequence {x k } conveges to x*, and it follows that x k is an inteio point of the closed sphee S fo all k geate than soe k.

11 JOTA: VOL. 114, NO. 2, AUGUST Fo k k, we have by the necessay condition (3), o equivalently, F k (x k ) y 0, fo all y T X (x k ), F k (x k ) T X (x k )*, which is witten as f(x k )C ξ k i h i (x k )C ζ k j g j (x k )C(x k Ax*) T X (x k )*, (13) whee ξ k i Gkh i (x k ), ζ k j Gkg + j (x k ). (14) Denote δ G1 k 1C (ξ k i ) 2 C (ζ k j) 2, (15) µ k 0G1 δ k, λ k i Gξ k i δ k,,...,, µ k j Gζ k j δ k,,...,. (16) Then, by dividing Eq. (13) with δ k, we obtain µ k 0 f(x k )C λ k Since by constuction we have (µ k 0) 2 C i h i (x k )C µ k j g j (x k )C(1 δ k )(x k Ax*) T X (x k )*. (17) (λ k i ) 2 C (µ k j) 2 G1, (18) the sequence {µ k 0, λ k 1,...,λ k, µ k 1,...,µ k } is bounded and ust contain a subsequence that conveges to soe liit {µ* 0, λ* 1,...,λ*, µ* 1,...,µ* }. Fo Eq. (17) and the defining popety of the noal cone N X (x*) [x k x*, {x k } X, z k z*, and z k T X (x k )*, fo all k, iply that z* N X (x*)], we see that µ* 0, λ* i, and µ* j ust satisfy condition (i). Fo Eqs. (14) and (16), µ* 0 and µ* j ust satisfy condition (ii), and fo Eq. (18), µ* 0, λ* i, and µ* j ust satisfy condition (iii). Finally, to show that condition (iv) is satisfied, assue that I J is nonepty, and note that, fo all sufficiently lage k within the index set K of the convegent subsequence, we ust have λ* i λ k i H0, fo all i I,

12 298 JOTA: VOL. 114, NO. 2, AUGUST 2002 and µ* j µ k j H0, fo all j J. Theefoe, fo these k, fo Eqs. (14) and (16), we ust have and λ* i h i (x k )H0, fo all i I, µ* j g j (x k )H0, fo all j J, while fo Eq. (12), we have f(x k )Ff(x*), fo k sufficiently lage (the case whee x k Gx* fo infinitely any k is excluded by the assuption that I J is nonepty). Futheoe, the conditions and ae equivalent to and h i (x k ) Go(w(x k )), g + j (x k )Go(w(x k )), λ k i Go(in{in i I µ k j Go(in{in i I fo all i I, fo all j J, λ k i, in µ k j }), i I, j J λ k i, in µ k j }), j J, j J espectively, so they hold fo k K. This poves condition (iv). Note that, if X is egula at x*, i.e., N X (x*)gt X (x*)*, condition (i) of Poposition 2.1 becoes o equivalently, µ* 0 f(x*)c µ* 0 f(x*)c λ*i h i (x*)c λ*i h i (x*)c µ* j g j (x*) T X (x*)*, µ* j g j (x*) y 0, y TX (x*). If in addition, the scala µ* 0 can be shown to be stictly positive, then by noalization we can choose µ* 0 G1, and condition (i) of Poposition 2.1 becoes equivalent to the Lagangian stationaity condition (4). Thus,

13 JOTA: VOL. 114, NO. 2, AUGUST if X is egula at x* and we can guaantee that µ* 0 G1, the vecto (λ*, µ*)g{λ* 1,...,λ*, µ* 1,...,µ* } is a Lagange ultiplie vecto that satisfies condition (iv) of Poposition 2.1. A key fact is that this condition is stonge than the CS condition (6). [If µ* j H0, then accoding to condition (iv), the coesponding jth inequality constaint ust be violated abitaily close to x* (cf. Eq. (9)), iplying that g j (x*)g0]. Fo ease of efeence, we efe to condition (iv) as the copleentay violation condition (CV fo shot). 4 This condition will tun out to be of cucial significance in the next section. To place Poposition 2.1 in pespective, we note that its line of poof, based on the quadatic penalty function, oiginated with McShane (Ref. 24). Hestenes (Ref. 7) obseved that the McShane poof can be used to stengthen the CS condition to asset the existence of a sequence {x k } such that λ* i h i (x k )H0, i I, µ* j g j (x k )H0, j J, (19) which is slightly weake than CV as defined hee [thee is no equieent that x k, siultaneously with violation of the constaints with nonzeo ultiplies, satisfies f(x k )Ff(x*) and Eq. (10)]. McShane and Hestenes consideed only the case whee XGR n. The case whee X is a closed convex set was consideed in Betsekas (Ref. 6), whee a genealized vesion of the Mangasaian Foovitz constaint qualification was also given. The extension to the case whee X is a geneal closed set and the stengthened vesion of condition (iv) ae given in the pesent pape fo the fist tie. To illustate the use of the genealized Fitz John conditions of Poposition 2.1 and the CV condition in paticula, conside the following exaple. Suppose that we convet a poble with a single equal- Exaple 2.1. ity constaint, in f(x), h(x)g0 to the inequality constained poble in f(x), s.t. h(x) 0, Ah(x) 0. 4 This te is in analogy with copleentay slackness, which is the condition that, fo all j, µ* j H0 iplies g j (x*)g0. Thus, copleentay violation eflects the condition that, fo all j, µ* j H0 iplies g j (x)h0 fo soe x abitaily close to x* (and siultaneously fo all j with µ* j H0).

14 300 JOTA: VOL. 114, NO. 2, AUGUST 2002 The Fitz John conditions asset the existence of nonnegative ultiplies µ* 0, λ +, λ, not all zeo, such that µ* 0 f (x*)cλ + h (x*)aλ h (x*)g0. (20) The candidate ultiplies that satisfy the above condition, as well as the CS condition λ + h(x*)gλ h(x*)g0, include those of the fo µ* 0 G0 and λ + Gλ H0, which povide no elevant infoation about the poble. Howeve, these ultiplies fail the stonge CV condition of Poposition 2.1, showing that, if µ* 0 G0, we ust have eithe λ + 0 and λ G0 o λ + G0 and λ 0. Assuing h(x*) 0, this violates Eq. (20), so it follows that µ* 0 H0. Thus, by dividing Eq. (20) by µ* 0, we ecove the failia fist-ode condition with f(x*)cλ* h(x*) G0, λ*g(λ + Aλ ) µ* 0, unde the egulaity assuption h(x*) 0. Note that this deduction would not have been possible without the CV condition. If we can take µ* 0 G1 in Poposition 2.1 fo all sooth f fo which x* is a local iniu, and X is egula at x*, then the constaint set C adits Lagange ultiplies of a special type, which satisfy the stonge CV condition in place of the CS condition. The salient featue of such ultiplies is the infoation which they ebody egading constaint violation with coesponding cost eduction. This is consistent with the classical sensitivity intepetation of a Lagange ultiplie as the ate of eduction in cost as the coesponding constaint is violated. Hee, we ae not aking enough assuptions fo this stonge type of sensitivity intepetation to be valid. Yet it is eakable that, with hadly any assuptions (othe than thei existence), Lagange ultiplies of the type obtained though Poposition 2.1 povide a significant aount of sensitivity infoation: they indicate the index set I J of constaints that, if violated, a cost eduction can be effected [the eaining constaints, whose indices do not belong to I J, ay also be violated, but the degee of thei violation is abitaily sall elative to the othe constaints as pe Eqs. (10) and (11)]. In view of this intepetation, we efe to a Lagange ultiplie vecto (λ*, µ*) that satisfies, in

15 JOTA: VOL. 114, NO. 2, AUGUST addition to Eqs. (4) (6), the CV condition [condition (iv) of Poposition 2.1] as being infoative. An infoative Lagange ultiplie vecto is useful, aong othe things, if one is inteested in identifying edundant constaints. Given such a vecto, one ay siply discad the constaints whose ultiplies ae 0 and check to see whethe x* is still a local iniu. While thee is no geneal guaantee that this will be tue, in any cases it will be; fo exaple, in the special case whee f and X ae convex, the g j ae convex, and the h i ae linea, x* is guaanteed to be a global iniu, even afte the constaints whose ultiplies ae 0 ae discaded. Now, if we ae inteested in discading constaints whose ultiplies ae 0, we ae also otivated to find Lagange ultiplie vectos that have a inial nube of nonzeo coponents (a inial suppot). We call such Lagange ultiplie vectos inial, and we define the as having suppot I J that does not stictly contain the suppot of any othe Lagange ultiplie vecto. Minial Lagange ultiplies ae not necessaily infoative. Fo exaple, think of the case whee soe of the constaints ae duplicates of othes. Then, in a inial Lagange ultiplie vecto, at ost one of each set of duplicate constaints can have a nonzeo ultiplie, while in an infoative Lagange ultiplie vecto, eithe all o none of these duplicate constaints will have a nonzeo ultiplie. Nonetheless, inial Lagange ultiplies tun out to be infoative afte the constaints coesponding to zeo ultiplies ae neglected, as can be infeed by the subsequent Poposition 2.2. In paticula, let us say that a Lagange ultiplie (λ*, µ*) is stong if, in addition to Eqs. (4) (6), it satisfies the condition below: (iv ) If the set I J is nonepty, whee IG{i λ* i 0} and JG{ j 0 µ* j H0}, then given any neighbohood B of x*, thee exists a sequence {x k } X that conveges to x* and is such that, fo all k, f(x k )Ff(x*), λ* i h i (x k )H0, i I, g j (x k )H0, j J. (21) This condition esebles the CV condition, but is weake in that it akes no povision fo negligibly sall violation of the constaints coesponding to zeo ultiplies, as pe Eqs. (10) and (11). As a esult, infoative Lagange ultiplies ae also stong, but not evesely.

16 302 JOTA: VOL. 114, NO. 2, AUGUST 2002 Fig. 2. Relations of diffeent types of Lagange ultiplies, assuing that the tangent cone T X (x*) is convex (which is tue in paticula if X is egula at x*). The following poposition, illustated in Fig. 2, claifies the elationships between diffeent types of Lagange ultiplies. Poposition 2.2. Let x* be a local iniu of poble (1) (2). Assue that the tangent cone T X (x*) is convex and that the set of Lagange ultiplies is nonepty. Then: (a) (b) The set of infoative Lagange ultiplie vectos is nonepty, and in fact the Lagange ultiplie vecto that has iniu no is infoative. Each inial Lagange ultiplie vecto is stong. Poof. (a) We suaize the essence of the poof aguent in the following lea (a elated but diffeent line of poof of this lea is given in Ref. 25). Lea 2.1. Let N be a closed convex cone in R n, and let a 0, a 1,..., a be given vectos in R n. Suppose that the closed and convex set M R, given by MG µ 0 a 0 C µ j a j N, is nonepty. Then, thee exists a sequence {d k } N* such that a 0 d k µ* 2, (22) (a j d k ) + µ* j,,...,, (23) whee µ* is the vecto of iniu no in M.

17 JOTA: VOL. 114, NO. 2, AUGUST Poof. Fo any γ 0, conside the function L γ (d, µ)g a 0 C µ j a j dcγ d A(1 2) µ 2. Ou poof will evolve aound the saddle point popeties of the convex concave function L 0 ; but to deive these popeties, we will wok with its γ - petubed and coecive vesion L γ fo γ H0, and then take the liit as γ 0. With this in ind, we fist establish that, if γ H0, L γ (d, µ) has a saddle point ove d N* and µ 0. Indeed, fo any fixed µ 0, L γ (, µ ) is convex ove d N* and, if µ M, we have so that a 0C µ ja j d 0, fo all d N*, L γ (d, µ ) γ d A(1 2) µ 2, d N*. Hence, L γ (, µ ) is coecive ove N*. Also, fo any fixed d N*, L γ (d, ) is concave and AL γ (d, ) is coecive ove µ R. It follows fo a theoe given by Hiiat-Uuty and Leaechal (Ref. 26, p. 334; see also Ref. 25) that, fo each γ H0, thee exists a saddle point (d γ, µ γ )ofl γ ove d N* and µ 0, satisfying L γ (d γ, µ γ )Gax L γ (d γ, µ) µ 0 Gin L γ (d, µ γ ) d N* Gax µ 0 in L γ (d, µ). (24) d N* We will now calculate soe of the expessions in the above equations. We have fo Eq. (24) L γ (d γ, µ γ )Gax L γ (d γ, µ) µ 0 Ga 0 d γ Cγ d γ Cax µ 0 µ j a j d γ A(1 2) µ 2. The axiu in the ight-hand side above is attained when µ j is equal to (a j d γ ) + fo all j [to axiize µ j a j d γ A(1 2)µ 2 j subject to the constaint µ j 0, we calculate the unconstained axiu, which is a j d γ, and if it is negative we set it to 0, so that the axiu subject to µ j 0 is attained fo µ j G(a j d γ ) + ]. Thus, we have L γ (d γ, µ γ )Ga 0 d γ Cγ d γ C(1 2) ((a j d γ ) + ) 2 (25)

18 304 JOTA: VOL. 114, NO. 2, AUGUST 2002 and µ γ G[(a 1 d γ ) +,...,(a d γ ) + ]. (26) We also have fo Eq. (24) whee L γ (d γ, µ γ )Gq γ (µ γ )A(1 2) µ γ 2 Gax {q γ (µ)a(1 2) µ 2 }, (27) µ 0 q γ (µ)g inf d N* a 0C To calculate q γ (µ), we let bg a 0 C µ j a j, and we use the tansfoation µ j a j dcγ d. to wite dgαξ, q γ (µ)g whee α 0 and ξ G1, inf α 0 ξ 1, ξ N* {α(γ Ab ξ)} (28) if ax b ξ γ, G ξ 1, ξ N* 0, S, othewise. We will show that ax b ξ γ, if and only if b NCS(0, γ ), (29) ξ 1,ξ N* whee S(0, γ ) is the closed sphee of adius γ that is centeed at the oigin. Indeed, if b NCS (0, γ ), then bgbˆcb, with bˆ N and b γ, and it follows that, fo all ξ N* with ξ 1, we have so that bˆ ξ 0 and b ξ γ, b ξgbˆ ξcb ξ γ,

19 fo which we obtain ax b ξ γ. ξ 1, ξ N* Convesely, assue that b ξ γ, JOTA: VOL. 114, NO. 2, AUGUST fo all ξ N* with ξ 1. If b N, then clealy b NCS (0, γ ). If b N, let bˆ be the pojection of b onto N and let b GbAbˆ. Because N is a convex cone, the nonzeo vecto b belongs to N* and is othogonal to bˆ. Since the vecto ξgb b belongs to N* and satisfies ξ 1, we have γ b ξ, o equivalently, Hence, γ (bˆcb ) (b b ) G b. bgbˆcb, with bˆ N and b γ, iplying that b NCS(0, γ ), and copleting the poof of Eq. (29). We have thus shown [cf. Eqs. (28) and (29)] that q γ (µ)g 0, S, if a 0C othewise. µ j a j NCS (0, γ ), (30) Cobining this equation with Eq. (27), we see that µ γ is the vecto of iniu no on the set M γ G µ 0 A a 0 C µ j a j NCS(0, γ ). Futheoe, fo Eqs. (27) and (30), we have L γ (d γ, µ γ )G (1 2) µ γ 2, which togethe with Eqs. (25) and (26), yields a 0 d γ Cγ d γ G µ γ 2. (31) We now take the liit in the above equation as γ 0. We clai that µ γ µ*. Indeed, since µ* M γ, we have µ γ µ*, so that {µ γ γ H0} is bounded. Let µ be a liit point of µ γ, and note that µ 0 and µ µ*.

20 306 JOTA: VOL. 114, NO. 2, AUGUST 2002 We have µ γ j a j Ga 0 Cν γ Cs γ, fo soe vectos ν γ N and s γ S(0, γ ), so by taking the liit as γ 0 along the elevant subsequence, it follows that ν γ conveges to soe ν N, and we have µ ja j Ga 0 Cν. It follows that µ M, and since µ µ*, we obtain µ Gµ*. The peceding aguent has shown that evey liit point of µ γ is equal to µ*, so µ γ conveges to µ* asγ 0. Thus, Eq. (31) yields We have li sup a 0 d γ A µ* 2. (32) γ 0 Conside now the function L 0 (d, µ)g a 0 C µ j a j da(1 2) µ 2. a 0 d γ C(1 2) It can be seen that ((a j d γ ) + ) 2 Gsup L 0 (d γ, µ) µ 0 sup inf L 0 (d, µ) µ 0 d N* inf L 0 (d, µ*). d N* inf L 0 (d, d N* µ)g (1 2) µ 2, S, if a 0 C µ j a j N, othewise. Cobining the last two equations, we have a 0 d γ C(1 2) ((a j d γ ) + ) 2 A(1 2) µ* 2, and since (a j d γ ) + Gµ γ j [cf. Eq. (26)], a 0 d γ A(1 2) µ* 2 A(1 2) µ γ 2.

21 JOTA: VOL. 114, NO. 2, AUGUST Taking the liit as γ S, we obtain li inf a 0 d γ A µ* 2, γ 0 which, togethe with Eq. (32), shows that a 0 d γ µ* 2. Since we have also shown that (a j d γ ) + Gµ γ j µ* j, the poof is coplete. We now etun to the poof of Poposition 2.2(a). Fo siplicity, we assue that all the constaints ae inequalities that ae active at x* (equality constaints can be handled by convesion to two inequalities, and inactive inequality constaints ae inconsequential in the subsequent analysis). We will use Lea 2.1 with the following identifications: NGT X (x*)*, a 0 G f(x*), a j G g j (x*),,...,, MGset of Lagange ultiplies, µ* GLagange ultiplie of iniu no. If µ*g0, then µ* is an infoative Lagange ultiplie and we ae done. If µ* 0, by Lea 2.1, fo any (H0, thee exists a d N*GT X (x*) such that whee a 0 d F0, (33) a j d H0, j J*, a j d ( in a l d, j J*, (34) l J* J*G{ j µ* j H0}. By suitably scaling the vecto d, we can assue that d G1. Let {x k } X be such that x k x* fo all k and x k x*, (x k Ax*) x k Ax* d. Using the Taylo theoe fo the cost function f, we have that, fo soe vecto sequence ξ k conveging to 0, f(x k )Af(x*)G f(x*) (x k Ax*)Co( x k Ax* ) G f(x*) (d Cξ k ) x k Ax* Co( x k Ax* ) G x k Ax* [ f(x*) d C f(x*) ξ k Co( x k Ax* ) x k Ax* ].

22 308 JOTA: VOL. 114, NO. 2, AUGUST 2002 Fo Eq. (33), we have so we obtain f(x*) d F0, f(x k )F f(x*), fo k sufficiently lage. Using also the Taylo theoe fo the constaint functions g j, we have fo soe vecto sequence ξ k conveging to 0, g j (x k )Ag j (x*)g g j (x*) (x k Ax*)Co( x k Ax* ) G g j (x*) (d Cξ k ) x k Ax* Co( x k Ax* ) G x k Ax* [ g j (x*) d C g j (x*) ξ k Co( x k Ax* ) x k Ax* ]. This, cobined with Eq. (34), shows that, fo k sufficiently lage, g j (x k )is bounded fo below by a constant ties x k Ax* fo all j such that µ* j H0 [and hence g j (x*)g0], and satisfies g j (x k ) o( x k Ax* ) fo all j such that µ* j G0 [and hence g j (x*) 0]. Thus, the sequence {x k } can be used to establish the CV condition fo µ*, and it follows that µ* is an infoative Lagange ultiplie. (b) We suaize the essence of the poof aguent of this pat in the following lea. Lea 2.2. Let N be a closed convex cone in R n, let a 0, a 1,...,a be given vectos in R n. Suppose that the closed and convex set M R, given by MG µ 0 A a 0 C µ j a j N, is nonepty. Aong the index subsets J {1,..., } such that, fo soe µ M, we have JG{ j µ j H0}, let J {1,...,} have a inial nube of eleents. Then, if J is nonepty, thee exists a vecto d N* such that a 0 d F0, a j d H0, fo all j J. (35) Poof. We apply Lea 2.1 with the vectos a 1,...,a eplaced by the vectos a j, j J. The subset of M, given by M G µ 0 A a 0 C µ j a j N, µ jg0, j J, j J is nonepty by assuption. Let µ be the vecto of iniu no on M. Since J has a inial nube of indices, we ust have µ jh0 fo all j J. If J is nonepty, Lea 2.1 iplies that thee exists a d N* such that Eq. (35) holds.

23 JOTA: VOL. 114, NO. 2, AUGUST Given Lea 2.2, the poof of Poposition 2.2(b) is vey siila to the coesponding pat of the poof of Poposition 2.2(a). Sensitivity and Lagange Multiplie of Miniu No. Let us fist intoduce an inteesting vaiation of Lea 2.1. Lea 2.3. Let N be a closed convex cone in R n, and let a 0,...,a be given vectos in R n. Suppose that the closed and convex set M R, given by MG µ 0 A a 0 C µ j a j N, is nonepty, and let µ* be the vecto of iniu no on M. Then, µ* 2 a 0 dc(1 2) ((a j d) + ) 2, d N*. Futheoe, if d is an optial solution of the poble we have in a 0 dc(1 2) ((a j d) + ) 2, (36a) s.t. d N*, (36b) a 0 d G µ* 2, (a j d ) + Gµ* j,,...,. (37) Poof. Fo the poof of Lea 2.1, we have that, fo all γ H0, (1 2) µ* 2 Gsup inf L 0 (d, µ) µ 0 d N* inf sup L 0 (d, µ) d N* µ 0 G inf d N* a 0dC(1 2) ((a j d) + ) 2. (38) If d is an optial solution of poble (36), we obtain inf d N* a 0dC(1 2) ((a j d) + ) 2 Ga 0 d C(1 2) ((a j d ) + ) 2 a 0 d γ C(1 2) ((a j d γ ) + ) 2.

24 310 JOTA: VOL. 114, NO. 2, AUGUST 2002 Since (accoding to the poof of Lea 2.1) a 0 d γ µ* 2 and (a j d γ ) + µ* j, asγ 0, by taking the liit above as γ 0, we see that equality holds thoughout in the above two inequalities. Thus, (d, µ*) is a saddle point of the function L 0 (d, µ) ove d N* and µ 0. It follows that µ* axiizes L 0 (d, µ) ove µ 0, so that µ* j G(a j d ) +, fo all j, and µ* 2 Gα 0 d. The diffeence between Leas 2.1 and 2.3 is that, in Lea 2.3, thee is the exta assuption that poble (36) has an optial solution (othewise, the lea is vacuous). It can be shown that, assuing that the set M is nonepty, poble (36) is guaanteed to have at least one solution when N* is a polyhedal cone. To see this, note that poble (36) can be witten as in a 0 dc(1 2) z 2 j, s.t. d N*, 0 z j, a j d z j,,...,, whee the z j ae auxiliay vaiables. Thus, if N* is polyhedal, then poble (36) is a quadatic poga with a cost function that is bounded below by Eq. (38), and hence it has an optial solution [see Bonnans and Shapio (Ref. 13, Theoe 3.128)]. Thus, when N* is polyhedal, Lea 2.3 applies. An ipotant context whee this is elevant is when XGR n, in which case N X (x*)*gt X (x*)gr n, o oe geneally when X is polyhedal, in which case T X (x*) is polyhedal. Anothe condition that guaantees the existence of an optial solution of poble (36) is that thee exists a vecto µ in the set such that MG µ 0 A a 0 C µ j a j N a 0 C µ ja j i(n), whee i(n) denotes the elative inteio of N. The elevant analysis, which is due to Xin Chen (pivate counication), is given in Ref. 25. When poble (36) can be guaanteed to have an optial solution and Lea 2.3 applies, the line of poof of Poposition 2.2(a) can be used to

25 JOTA: VOL. 114, NO. 2, AUGUST show that, if the Lagange ultiplie that has iniu no, denoted by (λ*, µ*), is nonzeo, thee exists a sequence {x k } X and a positive constant c such that f(x k )Gf(x*)A λ*i h i (x k )A µ* j g j (x k )Co( x k Ax* ), (39) h i (x k )Gcλ* i x k Ax* Co( x k Ax* ),,...,, (40) g j (x k )Gcµ* j x k Ax* Co( x k Ax* ), if µ* j H0, (41) g j (x k ) o( x k Ax* ), if µ* j G0. (42) These equations suggest that the iniu-no Lagange ultiplie has a sensitivity intepetation. In paticula, the sequence {x k } above coesponds to the vecto d T X (x*) of Eq. (37), which solves poble (36). Fo this, it can be seen that a positive ultiple of d solves the poble in s.t. f(x*) d, [ g j (x*) d) + ] 2 Gβ, [ hi (x*) d] 2 C j A(x*) d T X (x*), fo any given positive scala β. Thus, d is the tangent diection that axiizes the cost function ipoveent (calculated up to fist ode) fo a given value of the no of the constaint violation (calculated up to fist ode). Fo Eq. (39), this fist-ode cost ipoveent is equal to λ*i h i (x k )C µ* j g j (x k ). Thus, the ultiplies λ* i and µ* j expess the ate of ipoveent pe unit constaint violation, along the axiu ipoveent (o steepest descent) diection d. This is consistent with the taditional sensitivity intepetation of Lagange ultiplies. Altenative Definition of Lagange Multiplies. Finally, let us ake the connection with Rockafella s teatent of Lagange ultiplies fo Refs. 2, 8. Conside vectos λ*g(λ* 1,...,λ* ) and µ*g(µ* 1,...,µ* ) that satisfy the conditions f(x*)c λ*i h i (x*)c µ* j g j (x*) N X (x*), (43) µ* j 0,,...,, µ* j G0, j A(x*). (44)

26 312 JOTA: VOL. 114, NO. 2, AUGUST 2002 Such vectos ae called Lagange ultiplies by Rockafella, but in this pape we will efe to the as R-ultiplies, to distinguish the fo Lagange ultiplies as we have defined the [cf. Eqs. (4) (6)]. When X is egula at x*, the Rockafella s definition and ou definition coincide. In geneal, howeve, the set of Lagange ultiplies is a (possibly stict) subset of the set of R-ultiplies, since T X (x*)* N X (x*) with inequality holding when X is not egula at x*. Also, the existence of R- ultiplies does not guaantee the existence of Lagange ultiplies. Futheoe, even if Lagange ultiplies exist, none of the ay be infoative o stong, unless the tangent cone is convex (cf. Poposition 2.2 and Exaple 2.2 given below). Note that ultiplies satisfying the enhanced Fitz John conditions of Poposition 2.1 with µ* 0 G1 ae R-ultiplies, and they still have the exta sensitivity-like popety ebodied in the CV condition. Futheoe, Lea 2.1 can be used to show that, assuing that N X (x*) is convex, if the set of R-ultiplies is nonepty, it contains an R-ultiplie with the sensitivity-like popety of the CV condition. Howeve, if X is not egula at x*, an R-ultiplie ay not ende the Lagangian function stationay. The following is an illustative exaple. Exaple 2.2. In this 2-diensional exaple, thee ae two linea constaints a 1 x 0 and a 2 x 0 with the vectos a 1 and a 2 linealy independent. The set X is the (nonconvex) cone XG{x (a 1 x)(a 2 x)g0}. Conside the vecto x* G(0, 0). Hee, T X (x*)gx and T X (x*)*g{0}. Howeve, it can be seen that N X (x*) consists of the two ays of vectos that ae colinea to eithe a 1 o a 2, N X (x*)g{γ a 1 γ R} {γ a 2 γ R}; see Fig. 3. Because N X (x*) T X (x*)*, X is not egula at x*. Futheoe, both T X (x*) and N X (x*) ae not convex. Fo any f fo which x* is a local iniu, thee exists a unique Lagange ultiplie (µ* 1, µ* 2 ) satisfying Eqs. (4) (6). The scalas µ* 1, µ* 2 ae deteined fo the equieent that f(x*)cµ* 1 a 1 Cµ* 2 a 2 G0. (45) Except in the cases whee f(x*) is equal to 0 o to Aa 1 o to Aa 2,we have µ* 1 H0 and µ* 2 H0, but the Lagange ultiplie (µ* 1, µ* 2 ) is neithe

27 JOTA: VOL. 114, NO. 2, AUGUST Fig. 3. Constaints of Exaple 2.2. We have T X (x*)gxg{x (a 1 x)(a 2 x)g0} and N X (x*) is the nonconvex set consisting of the two ays of vectos that ae colinea to eithe a 1 o a 2. infoative no stong, because thee is no x X that siultaneously violates both inequality constaints. The R-ultiplies hee ae the vectos (µ* 1, µ* 2 ) such that f(x*)cµ* 1 a 1 Cµ* 2 a 2 is eithe equal to a ultiple of a 1 o to a ultiple of a 2. Except fo the Lagange ultiplies, which satisfy Eq. (45), all othe R-ultiplies ae such that the Lagangian function has negative slope along soe of the feasible diections of X. 3. Pseudonoality, Quasinoality, and Constaint Qualifications Poposition 2.1 leads to the intoduction of a geneal constaint qualification unde which the scala µ* 0 in Poposition 2.1 cannot be zeo. Definition 3.1. We say that a feasible vecto x* of poble (1) (2) is quasinoal if thee ae no scalas λ 1,...,λ, µ 1,...,µ, and a sequence {x k } X such that: (i) A λ i h i (x*)c µ j g j (x*) N X (x*). (ii) µ j 0, fo all,...,. (iii) λ 1,...,λ and µ 1,...,µ ae not all equal to 0. (iv) {x k } conveges to x* and, fo all k, λ i h i (x k )H0, fo all i with λ i 0, and µ j g j (x k )H0, fo all j with µ j 0.

28 314 JOTA: VOL. 114, NO. 2, AUGUST 2002 If x* is a quasinoal local iniu, the Fitz John conditions of Poposition 2.1 cannot be satisfied with µ* 0 G0, so that µ* 0 can be taken equal to 1. Then, if X is egula at x*, the vecto (λ*, µ*) G (λ* 1,...,λ*, µ* 1,...,µ* ) is an infoative Lagange ultiplie. Quasinoality was intoduced fo the special case whee XGR n by Hestenes (Ref. 7), who showed how it can be used to unify vaious constaint qualifications. The extension fo the case whee X R n is investigated hee fo the fist tie. A elated notion, also intoduced hee fo the fist tie, is given in the following definition. Definition 3.2. We say that a feasible vecto x* of poble (1) (2) is pseudonoal if thee ae no scalas λ 1,...,λ, µ 1,...,µ, and sequence {x k } X such that: (i) (ii) (iii) λ i h i (x*)c µ j g j (x*) N X (x*). µ j 0, fo all,...,, and µ j G0, fo all j A(x*). {x k } conveges to x* and λ i h i (x k )C µ j g j (x k )H0, k. (46) It can be seen that pseudonoality iplies quasinoality. The following exaple shows that the evese is not tue. We will show late in this section (Poposition 3.2) that, unde the assuption that N X (x*) is convex (which is tue in paticula if X is egula at x*), quasinoality is in fact equivalent to a slightly weake vesion of pseudonoality. Exaple 3.1 Let the constaint set be specified by CG{x X g 1 (x) 0, g 2 (x) 0, g 3 (x) 0}, whee XGR 2 and g 1 (x)gx 2 1C(x 2 A1) 2 A1, g 2 (x)g[x 1 Acos(π 6)] 2 C[x 2 Csin(π 6)] 2 A1, g 3 (x)g[x 1 Ccos(π 6)] 2 C[x 2 Csin(π 6)] 2 A1;

29 JOTA: VOL. 114, NO. 2, AUGUST Fig. 4. Constaints of Exaple 3.1. see Fig. 4. Conside the feasible vecto x* G(0, 0). Because thee is no x that siultaneously violates all thee constaints, quasinoality is satisfied. Howeve, a staightfowad calculation shows that we have while 3 gj (x*)g0, g 1 (x)cg 2 (x)cg 3 (x)g3(x 2 1Cx 2 2)H0, x x*, so by using µg(1, 1, 1), the conditions fo pseudonoality of x* ae violated. Thus, even when XGR n, quasinoality does not iply pseudonoality. We now give soe additional constaint qualifications, which togethe with CQ1 CQ3, given in Section 1, will be seen to iply pseudonoality of a feasible vecto x*. (CQ4) XGR n and, fo soe nonnegative intege F, the following supeset C of the constaint set C, C G{x h i (x)g0,,...,, g i (x) 0, jg C1,...,}, is pseudonoal at x*. Futheoe, thee exists a y R n such that h i (x*) yg0,,...,, g j (x*) y 0, j A(x*), g j (x*) yf0, j {1,..., } A(x*).

30 316 JOTA: VOL. 114, NO. 2, AUGUST 2002 Since CQ1 CQ3 iply pseudonoality, a fact to be shown in the subsequent Poposition 3.1, we see that CQ4 genealizes all the constaint qualifications CQ1 CQ3. (CQ5) (a) The equality constaints with index above soe, h i (x)g0, ig C1,...,, (b) ae linea. Thee does not exist a vecto λg(λ 1,...,λ ) such that λ i h i (x*) N X (x*), (47) (c) and at least one of the scalas λ 1,...,λ is nonzeo. The subspace V L (x*)g{y h i (x*) yg0, ig C1,...,} (d) has a nonepty intesection with the inteio of N X (x*)*. Thee exists a y N X (x*)* such that h i (x*) yg0,,...,, g j (x*) yf0, j A(x*). We efe to CQ5 as the genealized Mangasaian Foovitz constaint qualification, since it educes to CQ2 when XGR n and none of the equality constaints is assued to be linea. The constaint qualification CQ5 has seveal special cases, which we list below. (CQ5a) (a) Thee does not exist a nonzeo vecto λg(λ 1,...,λ ) such that λ i h i (x*) N X (x*). (b) Thee exists a y N X (x*)* such that h i (x*) yg0,,...,, g j (x*) yf0, j A(x*). (CQ5b) Thee ae no inequality constaints, the gadients h i (x*),,...,, ae linealy independent, and the subspace V(x*)G{y h i (x*) yg0,,...,} contains a point in the inteio of N X (x*)*.

31 (CQ5c) (CQ5d) JOTA: VOL. 114, NO. 2, AUGUST X is convex, thee ae no inequality constaints, the functions h i,,...,, ae linea, and the linea anifold {x h i (x)g0,,...,} contains a point in the inteio of X. X is convex, the functions g j ae convex, thee ae no equality constaints, and thee exists a feasible vecto x satisfying g j (x )F0, j A(x*). CQ5a is the special case of CQ5 whee all equality constaints ae assued nonlinea. CQ5b is a special case of CQ5 (whee thee ae no inequality constaints and no linea equality constaints) based on the fact that, if h i (x*),,...,, ae linealy independent and the subspace V(x*) contains a point in the inteio of N X (x*)*, then it can be shown that assuption (b) of CQ5 is satisfied. Finally, the convexity assuptions in CQ5c and CQ5d can be used to establish the coesponding assuption (c) and (d) of CQ5, espectively. Note that CQ5d is the well-known Slate constaint qualification, intoduced in Ref. 27. Let us also ention the following constaint qualification. (CQ6) The set WG{(λ, µ) λ 1,...,λ, µ 1,...,µ satisfy conditions (i) and (ii) of the definition of pseudonoality} (48) consists of just the vecto 0. CQ6 is the constaint qualification intoduced by Rockafella (Ref. 8), who used the McShane line of poof to deive the Fitz John conditions in the classical fo whee CS eplaces CV in Poposition 2.1. Clealy, CQ6 is a oe estictive condition than pseudonoality, since the vectos in W ae not equied to satisfy condition (iii) of the definition of pseudonoality. If the set of R-ultiplies [Eqs. (43) and (44)] is a nonepty closed convex set, its ecession cone is the set W of Eq. (48) [this is shown in a less geneal context by Bonnans and Shapio (Ref. 13, Poposition 3.14), but thei poof applies to the pesent context as well]. Since copactness of a closed, convex set is equivalent to its ecession cone containing just the 0 vecto [Rockafella (Ref. 28, Theoe 8.4)], it follows that, if the set of R- ultiplies is nonepty convex and copact, then CQ6 holds. In view of Poposition 2.1, the evese is also tue, povided the set of R-ultiplies ultiplies is guaanteed to be convex, which is tue in paticula if N X (x*) is convex. Thus, if N X (x*) is convex, CQ6 is equivalent to the set of R-ultiplies being nonepty and copact. It can also be shown that, if X

32 318 JOTA: VOL. 114, NO. 2, AUGUST 2002 is egula at x*, then CQ6 is equivalent to CQ5a. This is poved by Rockafella and Wets (Ref. 2) in the case whee XGR n and can be veified in the oe geneal case whee X R n by using thei analysis given in p. 226 of Ref. 2 [in fact, it is well-known that, fo XGR n, CQ5a is equivalent to noneptiness and copactness of the set of Lagange ultiplies; this is a esult of Gauvin (Ref. 29)]. Howeve, CQ3, CQ4, CQ5 do not peclude the unboundedness of the set of Lagange ultiplies and hence do not iply CQ6. Thus, CQ6 is not as effective in unifying vaious existing constaint qualifications as pseudonoality, which is iplied by all the constaint qualifications CQ1-CQ6, as shown in the following poposition. Poposition 3.1. A feasible point x* of poble (1) (2) is pseudonoal if any one of the constaint qualifications CQ1 CQ6 is satisfied. Poof. We will not conside CQ2, since it is a special case of CQ5. It is also evident that CQ6 iplies pseudonoality. Thus, we will pove the esult fo the cases CQ1, CQ3, CQ4, CQ5 in that ode. In all cases, the ethod of poof is by contadiction, i.e., we assue that thee ae scalas λ i,,...,, and µ j,,...,, which satisfy conditions (i) (iii) of the definition of pseudonoality. We then assue that each of the constaint qualifications CQ1, CQ3, CQ4, and CQ5 is in tun also satisfied, and in each case we aive at a contadiction. (CQ1) Since XGR n, iplying that N X (x*)g{0}, and since we also have µ j G0 fo all j A(x*) by condition (ii), we can wite condition (i) as i i µ j g j (x*)g0. λ h (x*)c j A(x*) The linea independence of h i (x*),,...,, and g j (x*), j A(x*), iplies that λ i G0 fo all i and µ j G0 fo all j A(x*). This, togethe with the condition µ j G0 fo all j A(x*), contadicts condition (iii). (CQ3) all x R n, By the lineaity of h i and the concavity of g j, we have that, fo h i (x)gh i (x*)c h i (x*) (xax*), g j (x) g j (x*)c g j (x*) (xax*),,...,,,...,.