A note on solution sensitivity for Karush Kuhn Tucker systems*

Transcription

1 Math Meth Oper Res (25) 6: DOI.7/s86449 A note on solution sensitivity for Karush Kuhn ucker systems* A. F. Izmailov, M. V. Solodov 2 Faculty of Computational Mathematics and Cybernetics, Department of Operations Research, Moscow State University, Leninskiye Gori, GSP-2, 9992, Moscow, Russia. ( izmaf@ccas.ru) 2 Instituto de Matemática Pura e Aplicada, Estrada Dona Castorina, Jardim Botânico, Rio de Janeiro, RJ, , Brazil. ( solodov@impa.br) Manuscript received: January 24 / Final version received: September 24 Abstract. We consider Karush-Kuhn-ucker (KK) systems, which depend on a parameter. Our contribution concerns with the existence of solution of the directionally perturbed KK system, approximating the given primaldual base solution. o our knowledge, we give the first explicit result of this kind in the situation where the multiplier associated with the base primal solution may not be unique. he condition we employ can be interpreted as the 2-regularity property of a smooth reformulation of the KK system. We also give a strictly sharper, compared to other statements in the literature, estimate for the contingent derivative of the KK solution multifunction. Key words: KK system, Perturbation, Sensitivity analysis, 2-regularity AMS subject classifications: 9C3, 65K5. Introduction Let U : R s R n! R n and G : R s R n! R m be sufficiently smooth mappings. We consider the parametric Karush Kuhn ucker (KK) system: Find ðx; lþ 2R n R m such that *Research of the first author is supported by the Russian Foundation for Basic Research Grant and the RF President s Grant NS for the support of leading scientific schools. he second author is supported in part by CNPq Grants 3734/95-6(RN) and 4778/ 23-, and by FAPERJ.

2 348 A.F. Izmailov, M.V. Solodov l ¼ ; l ; G ; hl; Gi ¼ ; ð:þ where r 2 R s is a parameter, and h; i is the Euclidean inner product. System (.) is a case of the mixed complementarity problem with a special (primal-dual) structure. If for some smooth function f : R s R n! R it holds that U ; r 2 Rs ; x 2 R n ; ð:2þ then, as is well-known, (.) is the KK optimality system for the parametric optimization problem minimize f ð:3þ subject to x 2 DðrÞ; where DðrÞ ¼fx 2 R n jg g: ð:4þ We note that all the developments given below extend in a straightforward manner to the case when equality constraints are present. Let KK be the set comprised by all triples ðr; x; lþ 2R s R n R m satisfying (.). We define the KK solution multifunction by KK : R s! 2 Rn R m ; KK ðrþ ¼fðx; lþ 2R n R m jðr; x; lþ 2 KK g: For a given (base) parameter value r 2 R s, let ðx; lþ 2KK ðrþ. he sensitivity theory is concerned with the local structure of the set KK or, to put in other words, with the behavior of the multifunction KK for the values of r 2 R s close to the base value r. here are two principal issues in stability/sensitivity analysis, which are to some extent independent of each other (and are typically considered separately in the sensitivity literature). One problem is that of approximation of the base solution by the solutions of the perturbed problems. It concerns with the properties of the map KK, assuming that solutions exist, at least for some forms of perturbations. (his assumption is usually not made explicitly, but without it, the sensitivity statements become vacuous.) Such studies usually deal with the question whether the set KK ðrþ approximates in some sense the set KK ðrþ as r! r ( stability ), and give some quantitative characterization of the approximation properties ( sensitivity ). Sensitivity information concerning the KK multifunction can be presented in various (equivalent) forms. One relevant object is the contingent cone C KK ðr; x; lþ to the set KK at the point ðr; x; lþ, or the (smaller, in general) tangent cone KK ðr; x; lþ. In the terminology of [6], the multifunction from R s to 2 Rn R m whose graph coincides with C KK ðr; x; lþ, is called the contingent (outer graphical) derivative of KK at r for ðx; lþ. Moreover, KK is said to be protodifferentiable at r for ðx; lþ if C KK ðr; x; lþ ¼ KK ðr; x; lþ. he essence of this branch of sensitivity analysis can therefore be stated in terms of the contingent and tangent directions. In Section 3, using the the notion of 2-regularity [6, 4], we present an estimate of the contingent derivative of the KK multifunction, which is sharper than other statements in the literature.

3 A note on solution sensitivity 349 he second major issue of stability/sensitivity analysis is that of existence of solutions of the perturbed problems, i.e., whether KK ðrþ 6¼ ;for a given (or all) r 2 R s close enough to r. his issue had been studied by many authors under quite mild assumptions (in particular, not implying the uniqueness of the multiplier associated with the base primal solution); see, for example, [2, 2, 23, 2, 3, 22, 7,8, 8], and the recently published books [, 3]. (Note that the existence results usually appear in conjunction with some kind of assertions on approximation properties, as discussed above.) In this paper, we are concerned with the following more specific question: we are looking for mild conditions guaranteeing, for given primal-dual base solution ðx; lþ, the existence of an arc of solutions of the form ðxðtþ; lðtþþ ¼ ðx þ tn; l þ tmþþoðtþ corresponding to the parameter values rðtþ ¼r þ td þ qðtþ, t, where d 2 R s and ðn; mþ 2R n R m. the persistence of a given primal-dual base solution ðx; lþ under perturbations. Moreover, we deal with a quite specific (though natural) form of solutions of perturbed problems. o our knowledge, this kind of analysis was previously known only in the context of Robinson s strong regularity, see [, 6]. Strong regularity implies that ðx; lþ is an isolated point of KK ðrþ, and in particular, l is the unique multiplier associated with x. In Section 2, we prove existence results for directional perturbations under conditions which do not require the uniqueness of the multiplier. hose conditions can be interpreted in terms of the 2-regularity property [6, 4] of a certain smooth reformulation of the KK system. he latter relation is discussed in Section 3. A few words about our notation. Given a finite set I, jij stands for its cardinality. For y 2 R m and an index set I f;...; mg, y I stands for the vector with components y i, i 2 I. For a matrix (linear operator) K, imk is its range (image space), and ker K is its kernel (null space). For a directionally differentiable mapping F : R n! R m,byf ðx; dþ we denote the usual directional derivative of F at x 2 R n in the direction d 2 R n. Given a set D in R n, the contingent cone to D at a point x 2 D is given by C D ðxþ ¼ fn 2 R n j9ft k gr such that ft k g!þ; dist ðx þ t k n; DÞ ¼oðt k Þg. he tangent cone to D at x is defined as D ðxþ ¼fn 2 R n j dist ðx þ tn; DÞ ¼ oðtþ; t g, where dist ðx; DÞ ¼inf z2d kx zk. For the base value r and the given x, we define the index sets associated with the active and inactive constraints in the usual way: I ¼ I ¼fi ¼ ;...; mjg i ¼g; N ¼ N ¼f;...; mgni: ð:5þ For a given l such that ðr; x; lþ 2 KK, the active constraints are further partitioned into the weakly and strongly active, as follows: I ¼ I ðr; x; lþ ¼fi2Ijl i ¼ g; I þ ¼ I þ ðr; x; lþ ¼I n I ¼fi2Ijl i > g: ð:6þ We next state some constraint qualifications and second-order conditions that will be used in the paper. All those conditions are associated with the nonperturbed KK system (i.e., for the base value r of the parameter). he linear independence constraint qualification (LICQ): I ¼ jij.

4 35 A.F. Izmailov, M.V. Solodov he Mangasarian Fromovitz constraint qualification (MFCQ): 9h 2 R n such I h >. he strict Mangasarian Fromovitz constraint qualification (SMFCQ): Iþ ¼jI þj and 9h 2 R such that I h I þ h ¼. he weak linear independence constraint qualification (WLICQ): Iþ ¼jI þj. As is well known, SMFCQ is equivalent to the uniqueness of the multiplier. For the sake of convenience, we define the mapping associated with the equations in (.) (leaving out the equation associated with the complementarity condition): W : R s R n R m! R n ; Wðr; x; lþ l: he second-order conditions have the form ðr; x; lþn; n 6¼ 8n 2 K nfg; with different choices of the cone K R n : he strong second-order sufficiency condition (SSOSC) uses K ¼fn 2 R n Iþ n ¼ g. he second-order condition (SOC) uses K ¼fn 2 R n I n ¼ g. Note that the second-order conditions mean that h ðr; x; lþn; ni has the same sign for all n in the corresponding K. 2. Existence of solutions under directional perturbations Let the cone L¼Lðr; x; lþ be the solution set (with respect to ðd; n; mþ 2R s R n R m ) of the following linearization of the KK system (.): ðr; x; lþd þ ðr; x; m ¼ ; m i ; hg i ; ðd; nþi ; m ihg i ; ðd; nþi ¼ ; i 2 I ; m N ¼ ; G I þ ðd; nþ ¼: he following inclusion is well known: C KK ðr; x; lþ L: ð2:þ ð2:2þ For the special case of directional perturbations, this fact is stated in [, heorem 5.] he same result, but in terms of the contingent derivative of KK at r for ðx; lþ, was given in [6, Proposition 2.5.]. he early related references are [5, 9, 4], and some recent related statements can be found in [, 8, 9, ], and also [Lemma 4. and heorem 5.]. In this section, for a given triple ðd; n; mþ 2L and a given mapping q : R þ! R s such that qðtþ ¼oðtÞ, we consider the arc of the form r þ td þ qðtþ in the space of parameters, and solutions of the form

5 A note on solution sensitivity 35 ðx þ tn; l þ tmþþoðtþ of the corresponding perturbed KK system. We are concerned with the existence and uniqueness of such solutions for the values t small enough. As mentioned above, the question of the existence and uniqueness of solutions of perturbed KK systems, approximating the given primal-dual base solution, was previously studied in the context of Robinson s strong regularity; see [, 6]. Recall that in the optimization setting, strong regularity is equivalent to LICQ combined with SSOSC [, Proposition 5.38]. In particular, it implies uniqueness of the multiplier associated with the given x for the base value r of the parameter. he assumptions of the existence heorem 2. below do not presume uniqueness of the multiplier. his issue will be further illustrated by examples in Section 4. We also note that the assumptions of Corollary 2. below are actually a certain 2-regularity property. We state the assumptions here in the algebraic form, leaving their conceptual interpretation until the next section. For any partition ði ; I 2 Þ of I (i.e., a pair of index sets such that I [ I 2 ¼ I, I \ I 2 ¼;), define the branch KK ði ;I 2 Þ ¼ KK ði ;I 2 Þðr; x; lþ of the set KK, as the solution set of the following system: l ¼ ; l I ; G I ¼; l I2 ¼ ; G I2 ; l N ¼ ; G Iþ ¼ As is easy to see, near ðr; x; lþ, the set KK can be represented as the union of such branches for (the finite number of) all the possible partitions. Similarly, the cone L is the union of the branches L ði ;I 2 Þ ¼L ði ;I 2 Þðr; x; lþ, given by ðr; x; lþd þ ðr; x; m I ; G I ðd; nþ ¼; m I2 ¼ ; G I 2 ðd; nþ ; m N ¼ ; G I þ ðd; nþ ¼ m ¼ ; Define the following index sets associated with the given triple ðd; n; mþ: I ¼ I ðd; n; mþ ¼fi2I jm i ¼ ; hg i; ðd; nþi ¼ g; I þ ¼ Iþ ðd; n; mþ ¼fi2I jm i > ; hg i; ðd; nþi ¼ g; ð2:3þ I N ¼ IN ðd; n; mþ ¼fi2I jm i ¼ ; hg i ; ðd; nþi > g: Let ði ; I2 Þ be any partition of I. hen ði ; I 2 Þ, with I ¼ I [ Iþ and I 2 ¼ I 2 [ IN, is a partition of I. Furthermore, ðd; n; mþ 2L ði ;I 2 Þ. he needed result makes use of Gollan s regularity condition [2] at ðr; x; lþ for the constraints defining the branch KK ði ;I 2 Þ. After some computations, this condition can be expressed in the form

6 352 A.F. Izmailov, M.V. Solodov ðr; x; I I [Iþ A 6¼ ; 9ðn; mþ 2R n R m such that ðd; n; m I [I þ Þ2ker K ði ;I 2 Þ; m I > ; G I 2 ðd; nþ > ; ð2:4þ ð2:5þ where K ði ;I 2 I [Iþ ðr; x; lþ ðr; x; I I [Iþ A: ð2:6þ he following theorem is now implied by [, heorem 3.4]. heorem 2.. For ðr; x; lþ 2 KK and a given ðd; n; mþ 2L, assume that Gollan s condition holds at ðr; x; lþ for the constraints defining the branch KK ði [Iþ ;I 2 [I N Þ for some partition ði ; I2 Þ of I. hen for every mapping q : R þ! R s such that qðtþ ¼oðtÞ, there exists a mapping r : R þ! R n R m such that for t it holds that ðx þ tn; l þ tmþþ rðtþ 2KK ðr þ td þ qðtþþ, rðtþ ¼oðtÞ. Under the additional assumption that I ¼;, the existence result in heorem 2. can be complemented by the following uniqueness result. Corollary 2.. For ðr; x; lþ 2 KK ¼;(see (2.3)) and I and a given ðd; n; mþ 2 L, assume that B det@ ðr; x; I þ [I I þ [I þ C A 6¼ : ð2:7þ hen for every mapping q : R þ! R s such that qðtþ ¼oðtÞ, and for every t > small enough, there exists the unique element rðtþ 2R n R m such that ðx þ tn; l þ tmþþrðtþ 2KK ðr þ td þ qðtþþ, rðtþ ¼oðtÞ. Proof. he assumptions of heorem 2. are certainly satisfied here. Indeed, the equality I ¼; implies that the only suitable partition of I is ði ; I 2 Þ¼ðI þ; IN Þ, and for this partition (2.4) reduces to (2.7). Moreover, from (2.), (2.3) and (2.6), it follows that (2.5) holds, e.g., with n ¼ n, m ¼ m. It remains to show that for all t > small enough, the element rðtþ ¼ðxðtÞ; lðtþþ 2 R n R m defined according to heorem 2. is unique. From (.5), (.6) and (2.3) it follows that this element satisfies ðl þ tm þ lðtþþ I þ [I > ; þ G I N [N ðr þ td þ qðtþ; x þ tn þ xðtþþ > ; and hence, by necessity,

7 A note on solution sensitivity 353 Wðr þ td þ qðtþ; x þ tn þ xðtþ; l þ tm þ lðtþþ ¼ ; G I þ [I ðr þ td þ qðtþ; x þ tn þ xðtþþ ¼ ; þ l I N [N ðtþ ¼: Condition (2.7) evidently means that the Jacobian of the latter system of equations with respect to ðx; lþ is nonsingular. his implies the needed uniqueness. ( he equality I ¼; can be interpreted as the strict complementarity condition at the solution ðd; n; mþ of the linearized KK system (2.) defining L. Under this condition, the following two limit cases can be pointed out: If I þ ¼ I (i.e., I N ¼;), then (2.7) implies LICQ, and is implied by LICQ combined with SOC. If I N ¼ I (i.e., I þ ¼;), then (2.7) implies WLICQ, and is implied by WLICQ combined with SSOSC. We omit the proofs, as they are quite direct. 3. Connections with 2-regularity and the contingent derivative In this section, we exhibit the connections between some of the key conditions which appeared above and the property of 2-regularity of a nonlinear mapping [6, 4]. We also obtain a sharper estimate for the contingent derivative of the KK multifunction, see Proposition 3.. When introducing 2-regularity, we simplify the setting to what is needed in the context of this paper. In particular, we state everything in finite dimensions. Let the following hypotheses be satisfied: (H) Z and W are (finite-dimensional) Euclidean spaces, LðZ; W Þ is the space of linear operators from Z to W, V is a neighborhood of a point z in Z. (H2) F : V! W is Fre chet-differentiable on V, and the mapping (H3) F : V! LðZ; W Þ is continuous at z. W ¼ im F ðzþ, W 2 is some complementary subspace of W in W, P is the projector in W onto W 2 parallel to W. (H4) he mapping PF : V! LðZ; W Þ is Lipschitz-continuous on V and directionally differentiable at z with respect to every direction in Z. Definition 3.. he mapping F is referred to as 2-regular at the point z with respect to a direction f 2 Z, if im ðf ðzþþðpf Þ ðz; fþþ ¼ W : Furthermore, F is said to be 2-regular at the point z, if it is 2-regular at this point with respect to every direction f 2 2 nfg, where 2 ¼ff 2 ker F ðzþjðpf Þ ðz; fþf ¼ g: Among other things, we have the following.

8 354 A.F. Izmailov, M.V. Solodov heorem 3.. [6, heorem 2.] Under the hypotheses (H) (H4), the following statements hold: (a) C F ðf ðzþþðzþ 2, where F ðf ðzþþ ¼ fz 2 ZjF ðzþ ¼F ðzþg. (b) If f 2 2, and the mapping F is 2-regular at z with respect to f, then f 2 F ðf ðzþþðzþ. In particular, if the mapping F is 2-regular at z, then C F ðf ðzþþðzþ ¼ F ðf ðzþþðzþ ¼ 2. Note that since the tangent cone is always a closed set, in the last assertion of heorem 3. it is sufficient to assume that F is 2-regular at z with respect to every element in some dense subset of 2. As is well known (e.g., [7]) and easy to see, the set KK can be equivalently represented as KK ¼ fz 2 R s R n R m jf ðzþ ¼g; where z ¼ðr; x; lþ, F : R s R n R m! R n R m ; F ðzþ ¼ WðzÞ ; SðzÞ S : R s R n R m! R m ; S i ðzþ ¼l i G i ðminf; G i þl i gþ 2 =2; i ¼ ;...; m: By direct computation, for z ¼ðr; x; lþ we have that ðl i i Si ðzþ ¼ ; l i < ; Þ; i 2 I þ; ð; ; G i e i Þ; i 2 N; : ; i 2 I ; where e i is the i-th vector of the canonic basis in R m. herefore (possibly after the rearrangement of the indices), we have that F ðzþ ¼ K ; where the matrix K of dimension ðn þji þ jþjnjþ ðs þ n þ mþ is given ðr; x; lþ ðr; x; lþ K A B A; C and A, B and C are matrices of dimensions ji þ js, ji þ jn and jnjm, respectively, with the following i A i ¼ l i ; i i ¼ l i ; i 2 I þ; C i ¼ G i e i ; i 2 N: aking into account that l i > 8i 2 I þ, and G i > 8i 2 N, it is easy to see that kerf ðzþ¼kerk ¼ff¼ðd;n;mÞ2R s R n R m jðd;n;m I Þ2kerC; m N ¼ g; ð3:þ where

9 A note on solution sensitivity 355 C ¼ ðr; x; I I þ ðr; x; I C A: ð3:2þ In particular, from (2.) it follows that Lker F ðzþ. We further obtain that W ¼ im F ðzþ ¼im K fg: Hence, we can take W 2 as follows: W 2 ¼ im F ðzþ? ¼ im K? R jij : With this choice, P is the orthogonal projector in R n R m onto W 2 : Pw ¼ðPðu; v Iþ [NÞ; v I Þ; w ¼ðu; vþ 2R n R m ; where P is the orthogonal projector onto ð im KÞ?. Observe that for i 2 I þ [ N, we have that S i ðzþ ¼l i G i for all z ¼ðr; x; lþ close to z. In particular, S Iþ [N is sufficiently smooth (say, twice differentiable at z). Hence, for any f ¼ðd; n; mþ 2R s R n R m, we have that ðpf Þ ðz; fþ ¼ P W SI þ [N ðzþf A: ð3:3þ ðsi Þ ðz; fþ For i 2 I, we obtain ðsi Þ ðz; fþ ¼ðm i minf; hg i ; ðd; nþi þ m i i ; þðhg i ; ðd; nþi minf; hg i ; ðd; nþi þ m igþð; ; e i Þ; ð3:4þ and ðsi Þ ðz; fþf ¼ 2m i hg i ; ðd; nþi ðminf; hg i ; ðd; nþi þ m igþ 2 : In particular, ðsi Þ ðz; fþf ¼ () m i ; hg i ; ðd; nþi ; m ihg i; ðd; nþi ¼ : Hence, by (2.), (3.), (3.2) and (3.3), we conclude that 2 ¼L\Q; where Q¼ f 2 R s R n R m P W ðzþ½f; fš SI ¼ : þ [N ðzþ½f; fš In particular, taking into account assertion (a) of heorem 3., we obtain the following estimate for the contingent derivative of the KK multifunction, sharper than (2.2) (see also Example 3.). Proposition 3.. Under the hypotheses (H) (H4), for ðr; x; lþ 2 KK, the following inclusion holds: C KK ðr; x; lþ L\Q: ð3:5þ

10 356 A.F. Izmailov, M.V. Solodov In the special case when rank K ¼ n þji þ jþjnj, or equivalently, rank C ¼ n þji þ j; ð3:6þ it holds that P ¼, so that Q¼R s R n R m, and thus 2 ¼L. In this case, inclusions (2.2) and (3.5) are the same. We next show that (3.6), and thus 2 ¼L, hold in the special case of canonical perturbations. he KK system is said to be canonically perturbed if the parameterization includes arbitrary right-hand side perturbations of U and G: U ¼U ðr ; xþþr ; G ¼G ðr ; xþþr 2 ; r ¼ðr ; r ; r 2 Þ2R s R n R m ; x 2 R n ð3:7þ ; with r ¼ðr ; ; Þ, r 2 R s. Here, U : R s R n! R n, G : R s R n! R m are sufficiently smooth mappings. It is easy to see that in this case rank ðr; x; ¼ n þ m; ð3:8þ and hence, (3.6) holds (see (3.2)). hus, for the case of canonical perturbations, 2 ¼L, and (3.5) coincides with (2.2). But beyond the case of canonical perturbations, 2 can be a strictly sharper estimate of the contingent derivative than L, as illustrated by the following example, which is obtained by introducing the (non-canonical) perturbation in [5, Example 2]. Example 3.. Let s ¼, n ¼ m ¼ 2, U ¼ðrþx ; x 2 2Þ, G ¼ ðx x 2 2 =2; x þ x 2 2 =2Þ, r ¼, x ¼ l ¼. By direct computations it can be shown that L is the solution set of the following system of equations in variables ðd; n ; n 2 ; m ; m 2 Þ: d þ n m m 2 ¼; minfn ; m g¼; minfn ; m 2 g¼: In particular, n 2 is arbitrary. At the same time, elements of 2 should satisfy the additional equation n 2 ðn 2 m þ m 2 Þ¼; so that n 2 is no longer arbitrary. his shows that the estimate (3.5) is sharper than the estimate (2.2). We next consider the conditions which ensure that a given f ¼ðd; n; mþ 2L belongs to KK ðr; x; lþ. Recall that I ¼ I [ Iþ, I 2 ¼ I 2 [ IN, where ði ; I2 Þ is a partition of I,andI, Iþ, IN are defined in (2.3). Assume that MFCQ holds at ðr; x; lþ for the constraints defining the branch KK ði ;I 2 Þ: rank K ði ;I 2 Þ ¼ n þji jþji þ j; 9ðd; n; m I [I þ Þ2kerK ði ;I 2 Þ such that m I > ; G I 2 ðr;xþðd; nþ ð3:9þ > ; where K ði ;I 2 Þ is defined in (2.6). ake f ¼ðd; n; mþ, where m I2 [N ¼. Note that the first condition in (3.9) implies (3.6). Hence, by (3.3),

11 A note on solution sensitivity 357 F ðzþþðpf Þ ðz; fþ¼ K ðsi Þ ðz; fþ : ð3:þ By the second condition in (3.9), it can be seen that f 2L ði ;I 2 Þ, and that the strict complementarity condition holds in (2.) at f. In particular, m i minf; hg i ; ð d; nþi þ m i g¼ m i > ; i 2 I ; ; i 2 I 2 ; hg i ; ð d; nþi minf; hg i ; ð d; nþi þ m i g ¼ ; i 2 I ; hg i ; ð d; nþi > ; i 2 I 2 : By (2.6), (3.4) and (3.), it is now evident that (3.9) implies 2-regularity of F at z with respect to f. Moreover, it can be seen that (3.9) actually implies that F is 2-regular at z with respect to every direction in some dense subset of L ði ;I 2 Þ. In particular, piecewise MFCQ (that is, MFCQ (3.9) for every partition ði ; I 2 Þ of I ) implies that F is 2-regular at z with respect to every direction in some dense subset of L. From assertion (b) of heorem 3. we now obtain Proposition 3.2. For ðr; x; lþ 2 KK and a given ðd; n; mþ 2L, assume that MFCQ (3.9) holds at ðr; x; lþ for the constraints defining the branch KK ði [Iþ ;I 2 [I N Þ for some partition ði ; I2 Þ of I. hen ðd; n; mþ 2 KK ðr; x; lþ. In particular, piecewise MFCQ for KK at ðr; x; lþ implies KK ðr; x; lþ ¼C KK ðr; x; lþ ¼L: ð3:þ Of course, the result of Proposition 3.2 can be obtained by the standard argument combined with piecewise analysis. We include this proposition merely as one of the illustrations for the use of the 2-regularity concept. If we assume (3.6), then it can be seen that 2-regularity of F at z with respect to f is actually equivalent to (3.9). In particular, strict complementarity in (2.) at f is a necessary condition for 2-regularity of F at z with respect to f. Consider again the case of canonical perturbations (3.7) and assume that MFCQ holds. In this case, from (3.8) it easily follows that piecewise MFCQ holds for KK at ðr; x; lþ, which implies (3.). In particular, under these assumptions, the KK multifunction is protodifferentiable at r for ðx; lþ, and its contingent derivative at r for ðx; lþ is a multifunction from R s to 2 Rn R m whose graph coincides with L. his fact was established in [6, Proposition 2.5.]. We next discuss the assumptions of Corollary 2. which is our existence and uniqueness result. Recall that in this setting, we consider the existence of solutions for a given perturbation of the form r þ td þ qðtþ, where the direction d 2 R s and the mapping q : R þ! R s, qðtþ ¼oðtÞ, are fixed. herefore, the relevant object to study is ~F : R R n R m! R n R m ; ~F ðpþ ¼F ðzþ; p ¼ðt; x; lþ; z ¼ðr þ td þ qðtþ; x; lþ:

12 358 A.F. Izmailov, M.V. Solodov By the analysis similar to the above (possibly after the rearrangement of the indices), for p ¼ð; x; lþ it can be seen that ~F K ðpþ ¼ ~ ; where the matrix K ~ of dimension ðn þji þ jþjnjþ ð þ n þ mþ is given by ~K ðr; x; lþd ðr; x; lþ Ad B C In particular, im ~F ðzþ ¼ im K ~ fg: ð3:2þ Observe that if there exists ðn; mþ 2R n R m satisfying ðd; n; mþ 2L, then (2.) implies that the first column in K ~ can be obtained as a linear combination of the other columns. Hence, im K ~ ðr; x; lþ B C C A: A: ð3:3þ Evidently, (2.7) implies that the matrix in the right-hand side has full row rank. herefore, ðim KÞ ~? ¼fg, and by (3.2), we have that im ~F ðzþþ? ¼fgR jij : It is now easy to see that under the assumptions of Corollary 2., 2-regularity of ~F at p with respect to q ¼ð; n; mþ is equivalent to saying that the matrix ðr; x; I þ [I þ d ðr; x; I þ [I I þ [I þ C A has full row rank. aking again into account that the first column above can be represented as a linear combination of the other columns (by (2.) and (2.3)), the latter condition is equivalent to (2.7). Moreover, if we assume that the I þ ðr; x; I C A has full row rank, which is equivalent to the assumption that the matrix in the right-hand side of (3.3) has full row rank, then 2-regularity of ~F at p with respect to q is equivalent to the assumptions of Corollary 2., i.e., I ¼; and (2.7) (recall that according to the discussion above, the strict complementarity condition I ¼;is necessary for 2-regularity of ~F at p with respect to q).

13 A note on solution sensitivity Some examples We start this section with the following result exhibiting some further properties of the set L. Related pairs of dual linear programs are known to be very useful in sensitivity analysis. But Lemma 4. appears to be new. Conclusions which can be deduced by using Lemma 4. will be given after the proof and illustrated by the examples below. Lemma 4.. If for a given ðd; nþ 2R s R n, there exists m 2 R m such that ðd; n; mþ 2L¼Lðr; x; lþ, then n is a solution of the LP problem minimize x hu; xi subject to hg i ; ðd; xþi ; i 2 I; ð4:þ while l is a solution of the dual LP problem maximize l l subject to ¼ U; ð4:2þ l I ; l N ¼ : Proof. By the definition of L (see (2.)), n is feasible in (4.) since hg i ; ðd; nþi 8i 2 I ; hg i ; ðd; nþi ¼ 8i 2 I þ: At the same time, l is feasible in (4.2) since the constraints of (4.2) can be stated in the form * ðr; x; lþ 2 KK. Furthermore, the duality relation holds: hu; ni + l; n ¼ n ¼ i l i ; n i2i þ ¼ i l i ; d i2i þ ¼ d ; where the inclusion ðr; x; lþ 2 KK and the definition of L (see (2.)) were taken into account. Lemma 4. leads to the following conclusions. d 6¼ for a given d 2 R s, and there exist n 2 R n and m 2 R m such that ðd; n; mþ 2L, then the objective function of (4.2) is non-constant, and it is quite likely that l is the unique solution of (4.2). In that case, l is a vertex of the polyhedral set of multipliers by necessity. Suppose that SMFCQ is not satisfied, i.e., the set of multipliers (of the nonperturbed KK system) associated with x is not a singleton. hen we can conclude that I 6¼;, i.e., the strict complementarity condition is violated at the solution ðx; lþ of the nonperturbed KK system. Indeed, if it were the case that l is a vertex and I ¼;, then l would have been

14 36 A.F. Izmailov, M.V. Solodov the unique solution of the equality-part of constraints in (4.2), which contradicts nonuniqueness of the multiplier. he examples presented in this section highlight the situation where SMFCQ is violated, i.e., the multiplier associated with x at the base value r of the parameter is not unique. In particular, we demonstrate that the branches of solutions of the perturbed KK system may depend drastically on the specific choice of the multiplier (which should be already clear from Lemma 4.). Example 4. Let s ¼, n ¼ 2, m ¼ 3, f ¼x þ x 2 2 =2, G ¼ðx þ r; x þ x 2 ; x x 2 Þ. Consider the parametric optimization problem (.3) with the feasible set defined in (.4). When r ¼ r ¼, this problem has the unique solution x ¼, ¼ A; A: he KK system for this problem takes the form (.) with U defined in (.2): U ¼ð; x 2Þ, and the set of multipliers associated with x is fl 2 R 3 jl ¼ð 2h; h; hþ; h 2½; =2Šg. For d <, problem (4.2) has the unique solution l ¼ð; ; Þ. With this choice of the multiplier, I ¼f2; 3g, I þ ¼fg, N ¼;, ðr; x; lþ ¼ ; ðr; x; lþ ¼ ; and L is the solution set of the following system of equations: m þ m 2 þ m 3 ¼; n 2 m 2 þ m 3 ¼; minfn þ n 2 ; m 2 g¼; minfn n 2 ; m 3 g¼; d þ n ¼: Consider next the branches of L. If I ¼;, I 2 ¼ I, then the corresponding branch is the ray spanned in R s R n R m by ð ; ð; Þ; Þ, and for every non-zero element of this ray, I ¼;and (2.7) holds. If I ¼ I and I 2 ¼;, the corresponding branch is the ray spanned by ð; ; ð 2; ; ÞÞ, and for every non-zero element of this ray, I ¼;and (2.7) holds as well. As is not difficult to check, the other two branches (corresponding to I ¼f2g and I 2 ¼f3g, I ¼f3g and I 2 ¼f2g, respectively) are trivial. Hence, for the given choice of l, the tangent cone KK ðr; x; lþ consists of two rays (the first two branches above); this follows from (2.2) and Proposition 3.2. Moreover, according to Corollary 2., for every t > small enough, the perturbed KK system corresponding to the parameter values r ¼ tþoðtþ has the unique solution of the form ððt; Þ; ÞþoðtÞ. Similarly, for the parameter values r ¼ oðtþ, there is a unique solution of the form ð; ð 2t; t; tþþ þ oðtþ.

15 A note on solution sensitivity 36 For d >, problem (4.2) has the unique solution l ¼ð; =2; =2Þ. With this choice, I ¼fg, I þ ¼f2; 3g, N ¼;, and it is easy to see that L is the ray spanned in R s R n R m by ð; ; Þ, and for every non-zero element of this ray, I ¼;, and (2.7) holds. We conclude that KK ðr; x; lþ coincides with this ray, and for every t > small enough, the perturbed KK system corresponding to the parameter values r ¼ t þ oðtþ has the unique solution of the form oðtþ. Note that according to Lemma 4., any other choice of the multiplier l will result in the cone LfgR n R m. Example 4.2 ([, Example 4.99])Let s ¼ n ¼ m ¼ 2, f ¼ððx Þ 2 þ x 2 2 Þ=2, G ¼ð x ; x r x 2 r 2 Þ. When r ¼ r ¼, optimization problem (.3) with the feasible set defined in (.4) has the unique solution x ¼ ; ¼ : he KK system for this problem takes the form (.) with U defined in (.2): U ¼ðx ; x 2 Þ, and the set of multipliers associated with x is fl 2 R 2 jl ¼ð h; hþ; h 2½; Šg. For d ¼ðd ; d 2 Þ with d 2 <, problem (4.2) has the unique solution l ¼ð; Þ. With this choice of the multiplier, I ¼f2g, I þ ¼fg, N ¼;, ðr; x; lþ ¼ ; ðr; x; lþ ¼ (note that ðr; x; lþ does not actually depend on the choice of l), and L is the solution set of the following system of equations: n þ m þ m 2 ¼; n 2 ¼; minf d 2 n ; m 2 g¼; n ¼: Obviously, L consists of the following two branches in R s R n R m : fðða; bþ; ; Þja 2 R; b 2 R þ g and fðða; Þ; ; ð c; cþþja 2 R; c 2 R þ g; their intersection is the straight line spanned by ðð; Þ; ; Þ. For every element of the two branches with b > and c >, respectively, we have that I ¼; and (2.7) holds. It follows that KK ðr; x; lþ consists of these two branches. For d ¼ðd ; d 2 Þ with d 2 >, problem (4.2) has the unique solution l ¼ð; Þ. With this choice, I ¼fg, I þ ¼f2g, N ¼;, ðr; x; lþ ¼ ; and L is the solution set of the following system of equations: ;

16 362 A.F. Izmailov, M.V. Solodov n þ m þ m 2 ¼; d þ n 2 ¼; minf n ; m g¼; d 2 n ¼: he two branches of L are and fðða; bþ; ð b; aþ; ð; bþþja 2 R; b 2 R þ g fðða; Þ; ð; aþ; ðc; cþþja 2 R; c 2 R þ g; their intersection is the straight line spanned by ðð; Þ; ð; Þ; ÞÞ. For every element of the two branches with b > and c >, respectively, we again have that I ¼;and (2.7) holds. Hence, KK ðr; x; lþ consists of these two branches. Finally, let d ¼ðd ; Þ. In this case, every l ¼ð h; hþ, h 2½; Š, provides a solution to (4.2). ake h 2ð; Þ (the values h ¼ andh¼ were already considered above). With this choice, I ¼ N ¼;, I þ ¼f; 2g, ðr; x; lþ ¼ ; h and L is the solution set of the following system of linear equations: n þ m þ m 2 ¼; hd þ n 2 ¼; n ¼; d 2 n ¼: Hence, L is the subspace fðða; Þ; ð; hmaþ; ðc; cþþja; c 2 Rg: Summarizing the above analysis, we see that for every direction d ¼ðd ; d 2 Þ with d 2 6¼, the corresponding primal part of the tangent vector to KK at ðr; x; lþ is uniquely defined: n ¼ if d 2 < and n ¼ð d 2 ; d Þ if d 2 >. For d ¼ðd ; Þ, situation is more complicated, as for every h 2½; Š there exists the tangent vector with the primal part n ¼ð; hd Þ. his can be explained (in some sense) by the following observation: the behavior of the solution of the original optimization problem under a perturbation of the parameter r along such directions depends drastically on the higher-order terms of such perturbation. For instance, if for t we take r ¼ðt; ÞþqðtÞ with some q : R þ! R s such that qðtþ ¼oðtÞ, then the branches of the solutions corresponding to different choices of q are not necessarily tangent to each other. Acknowledgment. We would like to thank A. Shapiro for encouragement and fruitful discussion on the subject. We also thank the anonymous referees whose comments helped us to improve the presentation.

17 A note on solution sensitivity 363 References [] Bonnans JF, Shapiro A (2) Perturbation Analysis of Optimization Problems. Springer Verlag, New York [2] Gollan B (984) On the marginal function in nonlinear programming. Mathematics of Operations Research 9:28 22 [3] Facchinei F, Pang J-S (23) Finite-Dimensional Variational Inequalities and Complementarity Problems. Springer Verlag, New York [4] Izmailov AF, Solodov MV (2) Error bounds for 2-regular mappings with Lipschitzian derivatives and their applications. Mathematical Programming 89: [5] Izmailov AF, Solodov MV (23) Karush-Kuhn-ucker systems: regularity conditions, error bounds and a class of Newton-type methods. Mathematical Programming 95:63 65 [6] Izmailov AF, Solodov MV (22) he theory of 2-regularity for mappings with Lipschitzian derivatives and its applications to optimality conditions. Mathematics of Operations Research 27: [7] Kanzow C (994) Some equation-based methods for the nonlinear complementarity problem. Optimization Methods and Software 3: [8] Klatte D (2) Upper Lipschitz behaviour of solutions to perturbed C ; programs. Mathematical Programming 88:285 3 [9] Klatte D, Kummer B (2) Contingent derivatives of implicit (multi-) functions. Annals of Operations Research :33 33 [] Klatte D, Kummer B (999) Generalized Kojima functions and Lipschitz stability of critical points. Computational Optimization and Applications 3:6 85 [] Klatte D, Kummer B (22) Nonsmooth Equations in Optimization: Regularity, Calculus, Methods and Applications. Kluwer Academic Publishers, Dordrecht [2] Kojima M (98) Strongly stable stationary solutions in nonlinear programs. In: Analysis and Computation of Fixed Points, edited by S.M.Robinson, Academic Press, New York, pp [3] Kummer B (984) Generalized equations: Solvability and regulaity. Mathematical Programming Studies 2:99 22 [4] Kyparisis J (992) Parametric variational inequalities with multivalued solution sets. Mathematics of Operations Research 7: [5] Kyparisis J (99) Sensitivity analysis for nonlinear programs and variational inequalities. Mathematics of Operations Research 5: [6] Levy A (2) Solution sensitivity from general principles. SIAM Journal on Control and Optimization 4: 38 [7] Levy A, Rockafellar R (995) Sensitivity analysis of solutions in nonlinear programming problems with nonunique multipliers. In: Recent Advances in Optimization, edited by D. Du, L. Qi, and R. Womersley, World Scientific Publishers, Singapore, pp. 38 [8] Liu J (995) Strong stability in variational inequalities. SIAM Journal on Control and Optimization 33: [9] Qiu Y, Magnanti L (992) Sensitivity analysis for variational inequalities. Mathematics of Operations Research 7:6 76 [2] Robinson SM (979) Generalized equations and their solutions. Part I: Basic theory. Mathematical Programming Study :28 4 [2] Robinson SM (982) Generalized equations and their solutions. Part II: Applications to nonlinear programming. Mathematical Programming Study 9:2 22 [22] Robinson SM (987) Local epi-continuity and local optimization. Mathematical Programming 37: [23] Robinson SM (98) Some continuity properties of polyhedral multifunctions. Mathematical Programming Study 4:26 24 [24] Shapiro A (23) Sensitivity analysis of generalized equations. Journal of Mathematical Sciences 5: