arxiv: v1 [math.oc] 30 Mar 2018

Transcription

1 A siple canonical for for nonlinear prograing probles and its use arxiv: v1 [ath.oc] 30 Mar 2018 Walter F. Mascarenhas April 2, 2018 Abstract We argue that reducing nonlinear prograing probles to a siple canonical for is an effective way to analyze the, specially when the proble is degenerate and the usual linear independence hypothesis does not hold. To illustrate this fact we solve an open proble about constraint qualifications using this siple canonical for. 1 Introduction In this article we look at the classical nonlinear prograing proble (1 iniize f(x subject to h(x = 0 R, g(x 0 R r+p in situations in which the derivatives Dh(0 and Dg(0 of the constraints are defective, in the sense that the rank of cobined atrix ( Dh(x Dhg(x := (2 Dg(x is + r +r+ p (by derivative Dh(x here we ean the linear transforation represented by the Jacobian atrix of the function h. To siplify the notation, we look at proble (1 for x in an open set A containing 0 R n and assue that all constraints are active at 0, that is h(0 = 0 and g(0 = 0. The analysis of proble (1 is usually based on a constraint qualification. For instance, the well known Mangasarian-Froovitz constraint qualification assues that Dh(0 has rank and require the existence of a strictly decreasing direction d for the inequality constraints, which is also copatible with the equality constraints: Definition 1 (The Mangasarian-Froovitz Constraint Qualification For functions h and g such that h(0=0 and g(0=0, we say that 0 R n satisfies the Mangasarian- Froovitz constraint qualification for proble (1 if rank(dh(0 = and there exists d R n such that Dh(0d=0 and Dg(0d<0. Instituto de Mateática e Estatística, Universidade de São Paulo, Cidade Universitária, Rua do Matão 1010, São Paulo SP, Brazil. CEP Tel.: , Fax: , walter.ascarenhas@gail.co. 1

2 Unfortunately, even under conditions like Mangasarian-Froovitz, the analysis of proble (1 can be tricky, and it is not uncoon to find articles in which incorrect results or proofs are presented, as pointed out in [2]. In fact, we are all huan and our capability to deal with the details involved in the analysis of proble (1 is liited, and we need tools to handle the. In the present article we argue that by reducing proble (1 to a siple canonical for we have a better chance of understanding degenerated cases, and we illustrate this point by proving the following conjecture by Andreani, Martinez and Schuverdt [1] regarding second order constraint qualifications. Theore 1 (Andreani s Conjecture Suppose the functions f, h and g in proble (1 are of class C 2 in a neighborhooda of 0 R n and h(0=0 and g(0=0. If the Mangasarian-Froovitz constraint qualification is satisfied and rank(dhg(x rank(dhg(0 + 1 for x A then there exist λ R and µ R p, with µ 0, such that S(d := d ( T 2 f (0+ λ j 2 r+p h j (0+ µ l 2 g l (0 for all d R n with Dh(0d=0 and Dg(0d=0. d 0 (3 The reduction of coplicated probles to sipler ones is a standard procedure in atheatics. Frequently, the Inverse Function Theore and its variations, like the Iplicit Function Theore, are used to obtain changes of variables which reduce a nonlinear proble to a linear one. The results can be striking, as in Malgrange s Preparation Theore, and Mather s Division Theore [4], which generalize the Iplicit Function Theore and have their roots in Weierstrass work. The Hartan-Groban Theore [11, 13] is another rearkable result along these lines, and allows us to understand the behavior of a nonlinear dynaical syste in ters of its linearization, under ild technical conditions. The Stable Manifold Theore [11] is yet another exaple. In our research about nonlinear prograing we have used it to analyze the convergence of significant algoriths, like the Affine Scaling Method [6], Newton s Method [9], the BFGS Method [7], and even to general failies of ethods [8, 10]. In these articles, by looking at low diensional probles fro the right perspective, we were able to provide satisfactory answers to relevant open probles. Here we present yet another instance in which the proper siplification leads to a better understanding of nonlinear prograing probles. In Section 2 we ephasize that usual optiality conditions in nonlinear prograing are invariant under changes of variables. In particular the Lagrange ultipliers are the sae in different coordinate systes for the independent variables, and the Mangasiran-Froovitz constraint qualification is invariant under changes of these variables. Section 2 is obvious but relevant. In fact, we believe that changes of variables do not receive the attention they deserve in the atheatical prograing literature. For instance, ost textbooks do not ention the invariance above explicitly. Authors usually have these changes of variables on the back of their ind, and build good exaples to illustrate their points based upon the. However, readers ay not notice that, with the proper changes of variables, ore general situations can be reduced to these good exaples, and in any cases such exaples are ore enlightening than proofs. 2

3 In Section 3 we present a convenient change of the independent variables for nonlinear prograing probles in which the rank of the derivative of the inequality constraints drops at the point in which we are interested. Of course, we cannot rely on the usual linear independence hypothesis in this case, but we show that we can still siplify proble considerably by using the proper choice of variables. Finally, in Section 4 we prove Andreani s conjecture, using the results fro the previous sections and two linear algebraic leas proved in the appendix. 2 Changes of variables This section calls the reader s attention to the fundaental fact that the usual optiality conditions in atheatical prograing are invariant under changes of variables. In other words, in theory we can pick any coordinate syste we please, as long as the coordinates are consistent and reflect correctly the proble we want to understand. We ephasize that, for theoretical purposes, changes of coordinates do not need to be explicit: it suffices to know that they exist and have the nice properties required to deal with the proble at hand. Of course, things are different in practice, because we usually cannot afford, or know how, to copute such changes of coordinates. The siplest changes of variables are the linear ones: we replace the coordinates x by Ay, where A is a nonsingular square atrix. As a result, we can replace a function f by a ore appropriate, or sipler, function ˆf, as in The chain rule yields ˆf(y= f(ay. (4 ˆf (y=a T f (Ay. (5 and applying the forula above to the function ˆf k (y := k f(ay and taking one ore derivative and doing the algebra we obtain that 2 ˆf (y=a T 2 f (AyA. (6 Equations (5 and (6 are quite useful, but they are not enough to explore the full power of changes of variables. For that we we need to replace x by a nonlinear function q(y of a ore convenient variable y, as in the nonlinear version of Equation (4: ˆf(y := f(q(y. (7 In Equation (7, the function q is a local diffeoorphis, that is a differentiable function defined in a neighborhood of the point with which we are concerned, and such that its inverse q 1 (in the sense that q 1 (q(x=x exists and is also differentiable. In this nonlinear setting we have the following version of Equation (5 ˆf (y=dq(y T f (q(y. (8 Equation (8 is alost the sae as (5: we only need to replace A by Dq(y and Ay by q(y, and keep in ind that the atrix Dq(y is square and non singular. Things are a bit ore coplicated for the Hessian. In this case we need to introduce an extra su due to the curvature in q, and the resulting forula is: 2 ˆf (y=dq(y T 2 f (q(ydq(y+ 3 n k=1 k f(q(y 2 q k (y. (9

4 With Equations (8 and (9 we can find how the optiality conditions behave under nonlinear changes of coordinates x = q(y. To see why this is true, we consider the classical nonlinear prograing proble (1. By aking the change of variables x = q(y we do not affect the satisfiability of the equalities and inequalities in (1, that is ĥ j (y := h j (q(y will be equal to zero as long as h j (x is equal to zero. This is obvious, but the analogous obvious property would not hold if we were to take cobinations of the dependent instead of the independent variables in proble (1, as when replace the equations by g 1 (x 0 and g 2 (x 0 (10 g 1 (x+ g 2 (x 0 and g 1 (x+ 2g 2 (x 0. Due to this obvious fact we ust think carefully before using techniques like the SVD to analyze the nonlinear prograing proble (1. In particular, when we say change of variables in this article we refer to changes in the dependent variables x, but not to cobinations of the equality constraints or changes in the order of the inequality constraints. Let us now analyze the first order optiality conditions for proble (1. These conditions are written in ters of the Lagrange ultiplies λ j R and µ l 0: f (x+ r+p λ j q j (x+ Given a local diffeoorphis q with q(y=x, by defining µ l g l (x=0, (11 ˆf(y := f(q(y, ĥ(y := h(q(y, and ĝ(y := g(q(y, (12 Equation (8 yields and ˆf (y=dq(y T f (q(y (13 ĥ j (y=dq(y T h j (q(y and ĝ l (y=dq(y T g l (q(y. (14 Since our Dq(y is always non singular, it is easy to see that the first order condition (11 holds if and only if ˆf (y+ r+p λ j ĥ j (y+ µ l ĝ l (y=0. (15 In other words, the first order conditions are invariant with respect to changes of coordinates, and we ay study the by using Equation (11, or Equation (15, or both. In particular, the Lagrange ultipliers do not change as we change coordinates as above. For the second order conditions, we consider directions d such that Dh(0d=0 and Dg(0d=0 (16 Equation (14 shows that Equation (16 is equivalent to Dĥ(0 ˆd=0 and Dĝ(0 ˆd=0, (17 4

5 for ˆd := Dq(y 1 d, or d=dq(y ˆd, and the orthogonality condition (16 is invariant under changes of coordinates, that is, Equations (16 for d and Equation (17 for ˆd are equivalent. Using the equations above, we can write the second order ter as S(d := S(d, f,h,g,λ,µ := d T ( 2 f (x+ Equation (9 yields for = ˆd T ( n k=1 λ j 2 r+p h j (xd+ S(d= ˆd T( Dq(y T 2 f (h(ydq(y + λ j Dq(y T 2 r+p h j (q(ydq(y+ n j λ k=1 ( µ l 2 g l (x µ l Dq(y T 2 g l (q(ydq(y S(d, f,h,g,λ,µ=s (ˆd, ˆf,ĥ,ĝ,λ,µ := ˆd T ( n k=1 k f(h(y 2 q k (y+ k h j (q(y 2 r+p n q k (y+ µ l k g l (q(y 2 q k (y ˆd k=1 k f(q(y+ r+p λ j k h j (q(y+ When the first order conditions (11 hold we have that k f(h(y+ r+p λ j h j (q(y+ µ l k g l (q(y µ l g l (q(y=0 2 q k (y and = 0. Therefore, when the first order conditions hold, the second order ter S(d in Equation (18 is invariant with respect to changes of variables, and it can be evaluated using the expression S(d=Ŝ (ˆd := ˆd T ( 2 ˆf (y+ λ j 2 r+p ĥ j (y+ An analogous arguent starting fro the conditions Dh(0d=0 and Dg(0d<0 µ l 2 ĝ l (y ˆd. ˆd. d, (18 ˆd. (19 instead of Equation (16 shows that the Mangazarian-Froovitz constraint qualification is invariant under changes of coordinates, and siilar arguents apply to any other constraint qualifications. In suary, when trying to answer any theoretical questions regarding the first and second order optiality conditions, Lagrange ultipliers and constraint qualifications for the nonlinear prograing proble (1, we can analyze the in other coordinate systes, and reach correct conclusions by considering only siplified probles. As we show in the next sections, this obvious observation has far reaching consequences, and can be used to give sipler proofs for soe results one finds in the nonlinear optiization literature. 5

6 3 The canonical for In this section we present a siple canonical for for the classical nonlinear prograing proble (1, which can be used when the derivative of the equality constraints has full rank but the inequality constraints are degenerated. In this canonical for the variables are y=(y 1,...,y T, z=(z 1,...,z r T and w=(w 1,...,z n r T, where The equality constraints are given by r := rank(dhg(0. ĥ(y,z,w=y=0, and there are two groups of inequality constraints. The first one is given by ĝ(y,z,w=z=0. The second group of inequalities is is given by a potentially coplicated function c: about which, in principle, we know that c(y,z,w 0 R p, c(0,0,0= 0 and D w c(0,0,0=0, (20 but have no control over D y c(0,0,0 or D z (0,0,0. The doain of these functions is a product of open sets Y Z W containing 0. In suary, we have the canonical nonlinear prograing proble iniize f(y, z, w subject to y = 0 R, z 0 R r, c(x,y,w 0 R p, y Y R, z Z R r w W R n r. (21 In this proble, the derivative of the constraints has the siple for I 0 0 Dhgc(x,y,z= 0 I r r 0, (22 D y c(y,z,w D z c(y,z,w D w c(y,z,w with D w c(0,0,0 = 0, where I is the identity atrix. The next theore shows that nonlinear prograing probles can be reduced to the canonical for (21 under ild assuptions, and the discussion in the previous section shows that the for of the first and second order conditions and the Lagrange ultipliers do not change in this reduction. In any relevant situations we can then use Theore 2 below and say rigorously without loss of generality, we can assue that our linear prograing proble is of the for (21 The purpose of the present article is to call the readers attention to this siple intuitive idea, which is foralized by Theore 2 and its proof, which we now present. 6

7 Theore 2 (The Canonical For Suppose that the functions h and g in proble (1 are of class C s, with s 1, in a neighborhooda of 0 R n, and h(0= 0 and g(0=0. If the n (+r atrix ( h 1 (0,..., h (0, g 1 (0,..., g r (0 (23 has rank +r then there exist open sets Y R, Z R r and W R n r, with (0,0,0 Y Z W, and a diffeoorphi q :Y Z W q(y Z W A of class C s with q(0,0,0= 0 such that, for 1 l r, ĥ(y,z,w := h(q(y,z,w=y and ĝ l (y,z,w := g l (q(y,z,w=z l and for 1 l p the functions c l (y,z,w := g r+l (q(y,z,w are such that c(0,0,0=0 and D w c(0,0,0=0. Theore 2 is a direct consequence of the Projection Lea below, which is a corollary of the Inverse Function Theore (see Th 2-13 in page 43 of [12]. Besides Lea 1, we only need to note that the equality D w (0,0,0= 0 follows fro Equation (22 and the assuption that Dhg(0 has rank +r. Lea 1 (The Projection Lea Let A be a neighborhood of 0 R n+k and let f :A R n a function of class C s. If D f(0 has rank n then there exists a neighborhood Y of 0 R n, a neighborhoodz of 0 R k and a diffeoorphis of class C s such that f(q(y,z=y. q :Y Z q(y Z A 4 A proof of Andreani s Conjecture In this section we use the canonical for (21 to prove the conjecture by Andreani, Martinez and Schuverdt entioned in the introduction. This is an interesting application of the canonical for because there were several failed attepts to find a proof of this conjecture by other eans. For instance, the authors of [2] attepted to obtain an appropriate coordinate syste, by using a version of the Singular Value Decoposition, and succeeded in proving new particular cases of Andreani s conjecture with this approach. However, they did not prove the conjecture because their decoposition is not as effective as the canonical for: it has high order ters in places in which the canonical decoposition has exact zeros, and the technicalities required to handle these ters precluded the fro obtaining a proof for which they had found all the other ingredients. This shows that good choices of variables go beyond controlling high order ters : we actually want to eliinate the, and Andreani s conjecture is one of the fortunate cases in which this is possible. We now prove Andreani s Conjecture using the arguents presented in the previous sections. Along the proof we resort to two linear algebraic leas, which are proved 7

8 in the appendix. These leas are variations of results already presented in other references [2, 5], and we provide their proofs to ake the article self contained. The first step to prove Theore 1 is to use Theore 2 to reduce proble (1 to the canonical for (21. In order to that we note that the Mangasarian-Froovitz constraint qualification requires that Dh(0 has rank and recall that we use +r to denote the rank of the atrix Dhg(0 in Equation (2. Therefore, by changing the order of the inequality constraints if necessary, we can assue that the atrix in Equation (23 in the stateent of Theore 2 has rank +r. We can then use this theore to analyze Andreani s conjecture and assue without loss of generality that our linear prograing proble has the for (21, because in Section 2 we have shown that Equation (3 and the Mangasarian-Froovitz constraint qualification are invariant under changes of coordinates. Due to Equations (16 and (22, we can assue that the vector d in Equation (3 has the for d= 0 0 r d and defining f :W R and c :W R by with d R p, f(w := f(0,0 r,w and c(w := c(0,0 r,w, we can rewrite Equation (3 as Equation (22 yields d T ( 2 f (0 + p µ r+l 2 c l (0 d 0. (24 rank(dhgc(y,z,w +r+1 rank(d w c(y,z,w 1, and cobining the observation that D c(w=d w c(0,0,w with the next lea we conclude that there exist α 1,...,α p R and a syetric atrix H R p p such that 2 c l (0=α l H for,... p. Lea 2 (Hessians with rank at ost one LetAbe a neighborhood of 0 R n, and let c 1,...,c be functions fro A to R of class C 2. If Dc(0=0 and rank(dc(x 1 for all x A then there exist α 1,...,α R and a syetric atrix H R n n such that 2 c l (0=α l H for,...,. It follows that Equation (24 is equivalent to S ( d,γ := d T( 2 f (0 + γh d 0 for γ = p α l µ r+l. (25 The Mangasarian-Froovitz constraint qualification iplies that for every d there exists λ and µ > 0 such that S(d 0 in Equation (3 (see [2]. Since this equation is equivalent to Equation (25, for every d there exists γ such that S ( d 0 in Equation (25, and in order to coplete the proof we use the following lea: 8

9 Lea 3 (Seidefinite separation Let I R be a copact interval. If the syetric atrices A,B R n n are such that for all x R n there exists γ x I such that x T (A+γ x Bx 0 then there exists γ I such that for all x R n. Lea 3 iplies that there exists x T (A+γ Bx 0 γ = p α l µ r+l such that S ( d,γ 0 in Equation (25 for all d. The full set λ and µ of Lagrange ultipliers containing these µ r+1,..., µ r+p is as required by Andreani s Conjecture and we are done. A Linear Algebra In this appendix we prove the results involving Linear Algebra used in Section 4. The proof of Lea 2 is based on the next two leas: Lea 4 (Rank one coluns If the syetric atrices H 1,...,H R n n are such that the n atrix A v :=(H j v,...,h v has rank at ost one for all v R n then there exist α 1,...α R and a syetric atrix H R n n \{0} such that H j = α j H for j = 1... Lea 5 (Directional derivatives of rank one Let A be a neighborhood of 0 R n and let h :A R n be a function of class C 1, and for 1 l let h l (x be the lth colun of h(x. If h(0=0 and rank(h(x 1for all x A then for every v R n the n atrix A v :=(Dh 1 (xv,...,dh (xv has rank at ost one. The proof of Lea 3 uses the following auxiliary results: Theore 3 (Dines Theore If the syetric atrices A,B R n n are such that for all x R n \{0} either x T Ax 0 or x T Bx 0 then the set R(A,B := {( x T Ax, x T Bx, x R n} R 2 (26 is either R 2 itself or a closed angular sector of angle less than π. Lea 6 (Definite separation Let I R be a copact interval. If the syetric atrices A,B R n n are such that for all x R n \{0} there exists γ x I such that x T (A+γ x Bx>0 then there exists γ I such that for all x R n \{0}. x T (A+γ Bx>0 9

10 Dines Theore is proved in [3], and in the rest of this appendix we prove our leas in the order in which they were stated. Proof of Lea 2. Applying Lea 5 to the function h(x := Dc(x we conclude that for every v R n the n atrix A v := ( 2 c 1 (0v,..., 2 c (0v has rank at ost one, and Lea 4 yields the coefficients α j and the atrix H. Proof of Lea 3. For every k N, Lea 6 yields γ k I such that x (A+γ T k B+ 1 k I n n x 0 for all x R n. Since the sequence γ k is bounded, it has a subsequence which converges to soe γ I. This γ is as required by Corollary 3. Proof of Lea 4. Lea 4 holds when n=1 or =1. Let us then assue that it holds when for n 1 1 or 1 1 and show that it also holds for and n. If soe H j is zero then we can take α j = 0 and use induction for H 1,...,H j 1,H j+1,...h. Therefore, we can assue that H j 0 for all j. It follows that H 1 has an eigenvalue decoposition H 1 = QDQ T with d By replacing H j by Q T H j Q for all j, we can assue that Q=I n n. Taking v=1/d 11 e 1 we obtain that H 1 v=e 1 and the hypothesis that the atrix A v has rank one iplies that H j v=α j e 1 for all j. Since the atrices H j are syetric, all of the have the for H j =( α j 0 T n 1 0 n 1 H j for syetric atrices H j R(n 1 (n 1 \{0} which satisfy the hypothesis of Lea 4 with n=n 1. Since H 1 e 1 = d 11 e 1 0, we have that α 1 0. Let then v R n 1 be such that H v 0, and write ( 1 v= v, We have that H 1 v=α 1 ( 1 α 1 α 1 H v ( α and H j v= j α j H v for j > 1, and the H 1 v and H j v are aligned by hypothesis. Moreover, either α j 0 or α j 0, because H j 0. It follows that α j = α 1 α j/α 1 for all j. As a result, H j = α j H where ( 1 0 T n 1 H := α 1 0 n 1 α 1 H and we are done. Proof of Lea 5. h(0=0 iply that For every v R n and 1 l, the facts that h C 1 and 1 li δ 0 δ h l(δv=dh l (0v=A v. 10

11 Let ρ > 0 be such that if B A v ρ then rank(b rank(a v. Taking δ such that 1 δ h(δv A v < ρ we obtain that ( 1 1 rank(h(δv=rank δ h(δv rank(a v. Therefore, rank(a v 1, and we are done. Proof of Lea 6. Let us write I =[a,b]. If a=b then we could siply take γ = a and we would be done. Therefore, we can assue that a<b. If(u,v R(A,B then u+γv 0 for soe γ [a,b]. This iplies that u γ v (1+ a + b v Therefore,( 2(1+ a + b,1 R(A,B and by Dines TheoreR(A,B is a closed pointed cone. For each γ I the set C γ := { (u,v R 2, with u+γv 0 } (27 is a closed convex cone, and C := C γ = C γ γ I γ [a,b] is also a closed cone. Moreover, C is pointed because a < b. The hypothesis tells us that for every (u,v R(A,B\{(0,0} there exists γ I such that (u,v C γ, and this iplies thatc R(A,B={(0,0}. Therefore, there exists(c,d R 2 such that and cu+dv> 0 for (u,v R(A,B\{(0,0} (28 cu+dv< 0 for (u,v C\{(0,0}. (29 Equation (27 shows that ( 1,0 C γ for all γ. Therefore, ( 1,0 C and Equation (29 iplies that c > 0, and by dividing Equations (28 and (29 by c we can assue that c=1. The point (a, 1 belongs to C γ for all γ a. Therefore, (a, 1 C and Equation (29 iplies that a d< 0, that is d> a. Siilarly, The point( b,1 belongs toc γ for all γ b, and b+d < 0, that is, d < b. In suary, we have that d [a,b]. Finally, we take γ = d, and Equation (28 with c=1 shows that this is a valid choice. References [1] R. Andreani, J. M. Martínez, and M. L. Schuverdt, On second-order optiality conditions for nonlinear prograing, Optiization 56 (2007. [2] R. Behling, G. Haeser, A. Raos, and D.S. Viana, On a conjecture in secondorder optiality conditions, arxiv: v1 [ath.oc] 23 Jun [3] L. L. Dines, On the apping of quadratic fors, Bull. Aer. Math. Soc. 47 (1941,

12 [4] M. Golubitsky and V. Guillein, Stable appings and their singularities, Graduate Texts in atheatics 14, Springer-Verlag, [5] G. Haeser, An extension of Yuan s lea and its applications in optiization, J. Opti. Theory Appl. 174 (2017, [6] W. F. Mascarenhas, The affine scaling algorith fails for stepsize 0.999, SIAM J. Opti. 7 (1997, [7], The BFGS algorith with exact line searches fails for nonconvex functions, Math. Progra. 99 (2004, [8], On the divergence of line search ethods, Coput. Appl. Math. 26 (2007. [9], Newton s iterates can converge to non-stationary points, Math. Progra. 112 (2008, [10], The divergence of the BFGS and Gauss Newton ethods, Math. Progra. 147 (2014, [11] L. Perko, Differential equations and dynaical systes, Springer-Verlag, New York, [12] M. Spivak, Calculus on anifolds, a odern approach to classical theores of advanced calculus, Addison-Wesley, [13] S.H. Strogatz, Nonlinear dynaics and chaos with applications to physics,biology, cheistry, and engineering, Perseus Books Publishing, Cabridge,MA,