QPCOMP: A Quadratic Programming Based Solver for Mixed. Complementarity Problems. February 7, Abstract

Transcription

1 QPCOMP: A Quadratc Programmng Based Solver for Mxed Complementarty Problems Stephen C. Bllups y and Mchael C. Ferrs z February 7, 1996 Abstract QPCOMP s an extremely robust algorthm for solvng mxed nonlnear complementarty problems that has fast local convergence behavor. Based n part on the NE/SQP method of Pang and Gabrel[14], ths algorthm represents a sgncant advance n robustness at no cost n ecency. In partcular, the algorthm s shown to solve any solvable Lpschtz contnuous, contnuously derentable, pseudo-monotone mxed nonlnear complementarty problem. QP- COMP also extends the NE/SQP method for the nonlnear complementarty problem to the more general mxed nonlnear complementarty problem. Computatonal results are provded, whch demonstrate the eectveness of the algorthm. 1 Introducton Ths paper descrbes a new algorthm for solvng the mxed nonlnear complementarty problem (MCP), whch provdes a sgncant mprovement n robustness over prevous superlnearly or quadratcally convergent algorthms, whle preservng these fast local convergence propertes. The MCP s dened n terms of a box IB := Q n =1 [l ; u ] and a functon f : IB! IR n, where for each, 1 l < u 1. The problem MCP(f; IB) s to nd x IB such that (x l) > f(x) + = (u x) > f(x) = 0; where f(x) + represents the projecton of f(x) onto the postve orthant, and f(x) := f(x) + f(x). Further, n the above denton, we agree that 1 0 = 0. Note that by choosng l = 0 and u = 1, the MCP reduces to the standard nonlnear complementarty problem (NCP), whch s to nd x 0 such that f(x) 0 and x > f(x) = 0: Complementarty problems (both MCP and NCP) arse n many applcatons [4, 7] and are the subject of much recent computatonal work. Indeed n recent years, a sgncant number of algorthms have been developed to solve complementarty problems. Most of these algorthms can be classed as descent methods; they work to mnmze a nonnegatve mert functon, whch s Ths materal s based on research supported by Natonal Scence Foundaton Grant CCR , Department of Energy Grant DE-FG03-94ER61915, and the Ar Force Oce of Scentc Research Grant F y Department of Mathematcs, Unversty of Colorado, Denver, Colorado 8017 z Computer Scences Department, Unversty of Wsconsn, Madson, Wsconsn

2 chosen so that zeros of the mert functon correspond to solutons of the complementarty problem. Among the algorthms ncluded n ths class are PATH [5, 15], MILES [17], SMOOTH [3], NE/SQP [14], and BDIFF [10]. Wthn ths basc framework, there are substantal derences between the algorthms; the algorthms der n the choce of mert functon, the technques used for determnng search drectons, and the globalzaton strateges used to guarantee descent of the mert functon. However, because all of these algorthms work to mnmze a mert functon, ther global convergence behavor s lmted by the same fundamental dculty: the mert functon may have local mnma that are not solutons of the complementarty problem. Ths dculty manfests tself n derent ways for derent algorthms. In PATH and MILES, t arses as a rank-decent bass or as a lnear complementarty subproblem whch s not solvable. In SMOOTH, t appears as a sngular Jacoban matrx. In NE/SQP t arses as convergence to a pont that fals some regularty condton. Due to ths dculty, the best these algorthms can hope for, n terms of global convergence behavor, s to guarantee ndng a soluton only when the mert functon has no strct local mnmzers that are not global mnmzers. In general, ths means that the functon f must be monotonc n order to guarantee convergence from arbtrary startng ponts. Ths paper descrbes and mplements an algorthm QPCOMP that does not suer from the above dculty, and hence s more robust than many other MCP algorthms. QPCOMP s based upon a strategy presented n Secton of ths paper. Ths strategy provdes a means of extendng any algorthm whch relably solves strongly monotone MCPs so that t wll solve a much broader class of problems. In partcular, t wll solve any problem whch satses a pseudo-monotoncty condton at a soluton. Applyng ths strategy to the NE/SQP algorthm[14], results n the QPCOMP algorthm. NE/SQP s an algorthm for solvng nonlnear complementarty problems that has a number of theoretcal advantages. We present ths algorthm n Secton 3, along wth extensons to the MCP framework that are necessary for ts use n our context. When we tested ths algorthm on our sute of test problems, we found that NE/SQP compares poorly to PATH, SMOOTH, and MILES n terms of robustness. In fact, we shall show n Secton 3 that the algorthm cannot relably solve even one dmensonal monotone lnear complementarty problems. However, NE/SQP works well on strongly monotone problems, whch s all that s requred for our strategy to work. In Secton 4, we present the QPCOMP algorthm. The man convergence result for ths algorthm s gven n Theorem 4.1, whch shows global convergence under the assumpton of pseudomonotoncty at a soluton, whenever f s a Lpschtz contnuous, contnuously derentable functon. The eectveness of the algorthm s demonstrated convncngly by the test results gven n Secton 5. Ths s n spte of the poor performance of the NE/SQP algorthm on whch QPCOMP s based. Before we begn, a word about notaton s n order. Iteraton numbers appear as superscrpts on vectors and matrces and as subscrpts on scalars. Subscrpts on a vector (or matrx) represent ether subvectors (or submatrces) or components of the vector or matrx. For example, f M s an n n matrx wth elements M jk ; j; k = 1; : : :; n, and J and K are ndex sets such that J; K f1; : : :; ng, then M J;K denotes the jjj jkj submatrx of M consstng of the elements M jk ; j J; k K. Smlarly, x j represents the jth component of the vector x. The notaton x + and x refers to the postve and negatve components of the vector x. Speccally, x + s the vector whose th component s gven by max(x ; 0), and x := x + x. The drectonal dervatve of a functon f : IB! IR n evaluated at the pont x n the drecton d s denoted by f 0 f(x + d) f(x) (x; d) := lm ; #0

3 provded the lmt exsts. Note that f x s a statonary pont of f on IB, then f 0 (x; d) = 0 8 d such that x+d IB. The Eucldean and max norms are denoted by kk and kk 1, respectvely. Throughout the paper, we use the standard dentons of monotone and strongly monotone functons [13, Denton 5.4.]. Smlarly, n dscussng convergence rates, we use the standard dentons of Q-superlnear and Q-quadratc convergence [13, Chapter 9]. Fnally, we use the symbol IR+ to represent the nonnegatve real numbers. The Basc Idea As mentoned n the ntroducton, numerous algorthms exst whch are extremely procent at solvng monotone or strongly monotone mxed complementarty problems. The challenge then s to develop an ecent algorthm that solves a broader class of problems. In ths secton we present a strategy for takng algorthms whch work well on strongly monotone MCPs and extendng them to solve MCPs for whch a consderably weakened monotoncty condton s satsed. To state ths condton, we rst need to dene the concept of pseudo-monotoncty: Denton.1 Gven a set IB IR n, the mappng f : IB! IR n s sad to be pseudo-monotone at a pont x IB f 8y IB, f(x ) > (y x ) 0 mples f(y) > (y x ) 0: (1) f s sad to be pseudo-monotone on IB f t s pseudo-monotone at every pont n IB. It s known [9] that f a functon g : IR n! IR s pseudo-convex [11, Denton 9.3.1], then rg s a pseudo-monotone functon. However, f g s only pseudo-convex at a pont x, t does not necessarly follow that rg s pseudo-monotone at x. Pseudo-monotoncty s a weaker condton than monotoncty. In partcular, every monotone functon s pseudo-monotone. But the converse s not true. For example, consder the functon f(x) := x= + sn(x). Ths functon s pseudo-monotone, but s not monotone. Note further that the natural mert functon kf(x)k = has strct local mnma that are not global mnma. Thus, we see that the natural mert functon of a pseudo-monotone functon can have local mnma that are not global mnma. In order to guarantee global convergence of our algorthm we shall requre that the followng assumpton be satsed: Assumpton. MCP(f; IB) has a soluton x such that f s pseudo-monotone at x. If MCP(f; IB) satses Assumpton., we say that MCP(f; IB) s pseudo-monotone at a soluton. However, for smplcty, we wll abuse termnology somewhat and say smply that MCP(f; IB) s pseudo-monotone. Ths should not cause any confuson snce all of our dscusson wll refer to problems whch satsfy Assumpton.. The strategy we present for pseudo-monotone MCPs s based upon extendng a descent-based algorthm for strongly monotone MCPs. The dea behnd a descent-based algorthm s to reformulate the MCP as a mnmzaton problem nvolvng a nonnegatve mert functon : IB! IR+. The mert functon s dened n such a way that (x) = 0 f and only f x s a soluton to MCP(f; IB). If f s strongly monotone, t s easy to construct a mert functon whch has no local mnma. It s then a smple task to nd the global mnmzer of, thereby gvng a soluton to the MCP. If however f s not monotone, then the mert functon chosen wll, n all lkelhood, contan local 3

4 mnma for whch 6= 0. The algorthm may then termnate at such a local mnmum, rather than at the soluton. To overcome ths dculty, we would lke to nd some way to \escape" from ths local mnmum. Ths can be accomplshed by constructng an mproved startng pont ~x where (~x) s smaller than the value of at the local mnmum. Snce the descent-based algorthm never allows the value of to ncrease, the algorthm can be restarted from ~x wth the guarantee that t wll never return to the local mnmum. Obvously, ndng such an mproved startng pont s not a straghtforward task. However, ths can be acheved when the problem s pseudo-monotone. The remander of ths secton descrbes how to construct ths mproved startng pont. We begn by denng a partcular mert functon for our algorthm: To do ths, we rst ntroduce the mappng H : IB! IR n dened by H (x) := mn(x l ; max(x u ; f (x))): () It s easly shown that H(x) = 0 f and only f x solves MCP(f; IB). Usng ths functon, we dene the mert functon (x) := 1 H(x)> H(x): (3) Clearly, x s a soluton to MCP(f; IB) f and only f x s a mnmzer of wth (x) = 0. In Secton 3 we wll present a basc algorthm for solvng strongly monotone MCPs, whch s based on mnmzng ths partcular choce of. However, for now, we smply assume that such an algorthm exsts. Moreover we assume that the algorthm wll fal n a nte number of teratons whenever t cannot solve the problem. Now suppose the basc algorthm fals at a pont x 0. Our strategy wll be to solve a sequence of perturbed problems, generatng a sequence of solutons fx k g that leads to an mproved startng pont ~x. The perturbed problems we solve are based on the followng perturbaton of f: gven a centerng pont x IB, and a number > 0, let f ;x (x) := f(x) + (x x): If f s Lpschtz contnuous, then for large enough, f ;x s strongly monotone. Thus, the basc algorthm wll be able to solve the perturbed problem MCP(f ;x ). Wth a sucently large we can then generate a sequence of terates as follows: gven a pont x 0, then for k = 0; : : :, choose x k+1 as the soluton to MCP(f ;xk ; IB). Note that every subproblem n the sequence uses the same choce of, but a derent choce of centerng pont. In partcular the centerng pont for one subproblem s the soluton of the prevous subproblem. Ths s very remnscent of the proxmal pont algorthm [16] and of Tkhonov regularzaton [18]. The followng lemma gves sucent condtons for a subsequence of these terates to converge to a soluton of MCP(f; IB). Theorem.3 Let > 0 and let fx k g; k = 0; 1; ::: be a sequence of ponts n IB such that for each k, x k+1 s a soluton to MCP(f ;xk ; IB). If MCP(f; IB) satses Assumpton., then 1. fx k g has a subsequence that converges to a soluton x of MCP(f; IB);. every accumulaton pont of fx k g s a soluton of MCP(f; IB); 3. f f s pseudo-monotone at any accumulaton pont x of fx k g, then the terates converge to x. 4

5 Proof Let x be the soluton to MCP(f; IB) gven by Assumpton.. Snce x k+1 s a soluton to MCP(f ;xk ; IB), then for each component, exactly one of the followng s true: 1. x k+1 = l and f (x k+1 ) + (x k+1 x k ) 0,. l < x k+1 < u and f (x k+1 ) + (x k+1 x k ) = 0, 3. x k+1 = u and f (x k+1 ) + (x k+1 x k ) 0, Let I l, I f and I u be the sets of ndces whch satsfy the rst, second, and thrd condtons respectvely. For I l, t follows that 0 x k xk+1 f (x k+1 )=. Also, x k+1 x = l x 0, so (x k+1 x )(x k x k+1 ) f (x k+1 )(x k+1 x )=: (4) By smlar reasonng, ths nequalty holds for I u. Fnally, for I f, f ;xk (x k+1 ) = 0, so x k xk+1 = f (x k+1 )=, whereupon t follows that (4) s satsed as an equalty. Thus n all cases, nequalty (4) s satsed, whch gves us the followng. (x k x ) = (x k+1 Summng over all components, x + xk xk+1 ) = (x k+1 x ) + (x k+1 x )(xk xk+1 ) + (x k xk+1 ) (x k+1 x ) + f (x k+1 )(x k+1 x )= + (xk xk+1 ) by (4). x k x x k+1 x + f(x k+1 ) > x k+1 x = + x k x k+1 : Under Assumpton., the nner product term above s nonnegatve. Thus, x k x x k+1 x + x k x k+1 ; x k x k+1! 0, we also see that x k j +1! x. Fnally, snce x k j +1 solves so fx k x g s a decreasng sequence, and x k x k+1! 0. It follows that fx k g has an accumulaton pont. Let x be any accumulaton pont. Then there s a subsequence fx k j : j = 0; 1; : : :g convergng to x. Snce MCP(f ;xk j ; IB), we conclude by a straghtforward contnuty argument that x solves MCP(f ;x ; IB), whch mples that x solves MCP(f; IB). Ths proves tems 1 and. To prove tem 3, note that f f s pseudo-monotone at an accumulaton pont x, then by tem, x s a soluton, so the above analyss can be repeated wth x replaced by x. We can then conclude n that x k x o s a decreasng sequence. But snce x s an accumulaton pont of fx k g, t follows that x k x! 0. Note that Theorem.3 dd not make any assumptons on the choce of. Thus, even f s too small to ensure that f ;x s strongly monotone, the strategy wll stll work so long as each subproblem s solvable. To llustrate the technque, t s useful to look at a smple example. Let IB := IR+ and let f : IR+! IR be dened by f(x) = (x 1) 1:01: 5

6 Table 1: Iterates produced by solvng sequence of perturbed problems, wth ( = 1:1) k x k f(x k ) (x k ) Ths deceptvely smple problem proved ntractable for all of the descent-based methods we tested. In partcular, we tred to solve ths problem usng PATH, MILES, NE/SQP, and SMOOTH. All four algorthms faled from a startng pont of x = 0. But ths should not be surprsng snce f s not monotone. However, f s pseudo-monotone on IB. Thus, t s easly solved by our technque. For example, usng = 1:1 and a startng pont x 0 = 0, the strategy generates the sequence of terates shown n Table 1. Note that at the 7th teraton, an mproved startng pont s found, (.e, (x 7 ) < (x 0 )). At ths pont, a basc algorthm (e.g., Newton's method) can be used to obtan the nal soluton. In ths secton, we have ntroduced a basc strategy for takng descent-based algorthms that solve strongly monotone MCPs, and extendng them to solve pseudo-monotone MCPs. Ths s, n fact, the man dea presented n ths paper. However, to turn ths strategy nto a workng algorthm, a number of detals must be addressed: 1. We must ensure that the basc algorthm (for solvng the strongly monotone MCPs) termnates n a nte number of teratons. Ths ssue wll be addressed n detal n Secton 3.. Snce we requre nte termnaton of the basc algorthm, we must allow nexact solutons of the perturbed subproblems. We shall therefore need to ncorporate control parameters nto our strategy whch govern the accuracy demanded by each subproblem. In the our actual mplementaton of the algorthm we demand very lttle accuracy for each subproblem. In fact, except n extreme crcumstances, we allow only one step of the basc algorthm before updatng the perturbed problem. To guarantee convergence of ths approach requres more laborous analyss whch we defer untl Secton Snce we have no a pror nformaton regardng the Lpschtz contnuty of f, we shall have to ncorporate some adaptve strategy for choosng n order to ensure that, eventually, the subproblems all become solvable. The next two sectons of the paper are devoted to addressng these detals. 3 Subproblem Soluton In ths secton, we present an algorthm for solvng strongly monotone MCPs, whch s based on the NE/SQP algorthm [14]. NE/SQP was orgnally developed as a method for solvng the nonlnear 6

7 complementarty problem. When t was rst ntroduced, NE/SQP oered a sgncant advance n the robustness of NCP solvers because the subproblems t needs to solve at each teraton are convex quadratc programs, whch are always solvable. Today, ts robustness has been greatly surpassed by PATH, MILES, and SMOOTH (see Secton 5). However, NE/SQP s stll a vable technque for solvng strongly monotone MCPs. Moreover, NE/SQP has the very desrable feature of evaluatng the functon f only on ts doman IB. Ths s n marked contrast to the SMOOTH algorthm whch requres f to be dened on all of IR n. In ths secton, we rst present the NE/SQP algorthm extended to the MCP framework. Snce the development closely parallels that gven n [14], we are delberately terse n our presentaton. Moreover, we omt the proofs to Proposton 3.1, Theorem 3.4 and Lemma 3.5. However, detaled proofs for these results are gven n [1, Chapter ]. Once the extended NE/SQP algorthm s presented we wll then modfy t to ensure nte termnaton. We note that Gabrel [8] also extended NE/SQP to address the upper bound nonlnear complementarty problem, a specal case of MCP where l = 0 and u > 0 s nte. 3.1 Extenson of NE/SQP to the MCP Framework Recall that a vector x solves MCP(f; IB) f and only f (x) = 0, where s dened by () and (3). The NE/SQP algorthm attempts to solve ths problem by solvng the mnmzaton problem mn xib (x). We wll use as a mert functon for the MCP. To descrbe the algorthm n detal we need to partton the ndces f1; : : :; ng nto ve sets as follows: I l (x) = f : x l < f (x)g I el (x) = f : x l = f (x)g I f (x) = f : x u < f (x) < x l g I eu (x) = f : x u = f (x)g I u (x) = f : x u > f (x)g: It wll at tmes be convenent to refer also to the ndex sets J l (x) := I l (x) S I el (x) and J u (x) := I u (x) S I eu (x). As n the orgnal descrpton of NE/SQP, the subscrpts of these sets are chosen to reect ther meanng. For example, the subscrpts l; f, and u correspond to the ndces where H (x) = (x l ); f (x), and (x u ) respectvely. The subscrpts el and eu correspond to the ndces where f (x) s equal to l and u, respectvely. These ndex sets are used to dene an teraton functon : IB IR n! IR as follows: (x; d) := P n=1 (x; d), where (x; d) := 8 >< >: 1 (x l + d ) I l (x) S I el (x) 1 (x u + d ) I u (x) S I eu (x) 1 (f (x) + rf (x) > d) I f (x) = 1; : : :; n: Gven a pont x k IB, the algorthm chooses a descent drecton d k by solvng the convex quadratc programmng problem (QP k ) gven by QP k : mn (x k ; d): x k +dib We note that n the orgnal NE/SQP algorthm, an addtonal constrant was added to ths quadratc program, namely, d = 0 f f (x k ) = 0 and x k = l or x k = u. However, ths constrant s unnecessary for the convergence results, so we omt t from our algorthm. 7

8 To ensure descent of the mert functon, we wll need to perform a lnesearch along the drecton d k. To descrbe ths lnesearch, we use a forcng functon z : IB IR n! IR+, dened by z(x; d) := P n =1 z (x; d), where z (x; d) := 8 < : 1 d 6 I f (x) 1 (rf (x) > d) I f (x) = 1; : : :; n: Ths forcng functon wll be used to guarantee sucent decrease n the mert functon at each teraton. The followng proposton summarzes some essental propertes of the functons and z: Proposton 3.1 ([1], Lemmas..5 and..6) The followng propertes hold: 1. If x k IB, then (QP k ) has at least one optmal soluton.. (x; d) (x; 0) z(x; d) 0 (x; d) for all (x; d) IB IR n. 3. If d k s an optmal soluton to (QP k ) and (x; d k ) < (x; 0), then for any (0; 1), there exsts a scalar > 0 such that for all [0; ] (x + d k ) (x) z(x; d k ): 4. If d k s an optmal soluton to (QP k ), then z(x k ; d k ) (x k ): Item (1) n the above proposton ensures that each QP s solvable. Item () ensures that the soluton to the QP wll be a descent drecton for unless x s a statonary pont of. Item (3) allows us to use a Armjo type lnesearch whch wll be guaranteed to termnate n a nte number of teratons. Item (4) wll be used n the proof of Theorem We now state the algorthm. Algorthm NE/SQP Step 1 [Intalzaton] Select ; (0; 1), and a startng vector x 0 IB. Set k = 0. Step [Drecton generaton] Solve (QP k ), gvng the drecton d k. If (x k ; d k ) = (x k ), termnate the algorthm; otherwse, contnue. Step 3 [Steplength determnaton] Let m k be the smallest nonnegatve nteger m such that set x k+1 = x k + m kd k. (x k + m d k ) (x k ) m z(x k ; d k ); (5) Step 4 [Termnaton check] If x k+1 satses a prescrbed stoppng rule, stop. Otherwse, return to Step, wth k replaced by k + 1. The convergence results of ths algorthm are based on two regularty condtons: b-regularty and s-regularty. It s convenent to partton the ndex sets as follows n order to dene these regularty condtons. I + el (x) = f I el : x l > 0g I 0 el (x) = f I el : x l = 0g I l f (x) = f I f : x l = 0g I n f (x) = f I f : x u < 0 < x l g I u f (x) = f I f : x u = 0g I 0 eu(x) = f I eu : x u = 0g I eu (x) = f I eu : x u < 0g: Note that for x IB, the sets I l (x); I + el (x); I 0 el (x); I l f (x); I n f (x); I u f (x); I 0 eu; I eu(x), and I u (x) form a partton of the ndces f1; : : :; ng. 8

9 Denton 3. A nonnegatve vector x s sad to be b-regular f for every ndex set satsfyng the prncpal submatrx r f (x) s nonsngular. I n f (x) I f (x) [ I el (x) [ I eu (x); Denton 3.3 A nonnegatve vector x s sad to be s-regular f the followng lnear nequalty system has a soluton n y: x l + y = 0 x u + y = 0 f (x) + rf (x) > y = 0 x l + y 0 f (x) + rf (x) > y 0 x u + y 0 I l (x) I u (x) If n(x) I l f (x) I l f (x) I u f (x) f (x) + rf (x) > y 0 x l + y 0 f (x) + rf (x) > y 0 x u + y 0 f (x) + rf (x) > y 0 y = 0 I u f (x) I + el (x) I + el (x) I eu(x) I eu(x) I 0 el (x) S I 0 eu(x): Note that when l = 0; u = 1 the above denton s dentcal to the concept of s-regularty [14, Denton 1]. The followng theorem parallels the convergence results of [14, Theorems 1 and ] and establshes the fact that the NE/SQP algorthm has very good local convergence behavor. Theorem 3.4 ([1], Theorems..1 and..15) Let f : IB! IR n be a once contnuously derentable functon. Let x 0 IB be arbtrary. The followng statements hold: 1. NE/SQP generates a well dened sequence of terates fx k g, wth x k IB, along wth a sequence of optmal solutons fd k g for the subproblems (QP k );. f x s an accumulaton pont of fx k g, and f x s both b-regular and s-regular, then the followng hold: (a) x s a soluton of MCP(f; IB). (b) there exsts an nteger K > 0 such that for all k K, the stepsze k = m k = 1, hence, x k+1 = x k + d k ; (c) the sequence fx k g converges to x Q-superlnearly; (d) f rf s Lpschtzan n a neghborhood of x, then the convergence s Q-quadratc. The global convergence results contaned above are not very useful from a practcal standpont. The problem s that the s-regularty and b-regularty condtons are dependent not only on the problem, but also on the algorthm. In partcular, they depend on the partcular choce of mert functon used. A result that wll be more useful for our purposes s avalable as a result of the followng lemma: Lemma 3.5 ([1], Lemma..17) If f s strongly monotone, then all ponts x IB are both b-regular and s-regular. It should be noted that the strong monotoncty assumpton above s essental. For example, consder the monotone functon f : IR+! IR gven by f(x) = 1, and let IB := IR+. For ths choce of f and IB, t s easly vered that 8x > 1, x s nether b-regular or s-regular. As a consequence, even though MCP(f; IB) has the trval soluton x = 0, NE/SQP fals to nd t wth any startng pont x > 1. Thus, we see that NE/SQP cannot be reled upon to solve monotone lnear complementarty problems. We now state our man convergence result of the NE/SQP algorthm. 9 (6)

10 Theorem 3.6 Suppose f s strongly monotone. If x s an accumulaton pont of the terates fx k g produced by the NE/SQP algorthm, then x s a soluton of MCP(f; IB) and the sequence fx k g converges to x wth the local convergence rates speced n Theorem 3.4. Proof By Lemma 3.5, x s both b-regular and s-regular. Therefore, by Theorem 3.4, x s a soluton of MCP(f; IB) and the terates fx k g generated by the NE/SQP algorthm converge to x wth convergence rates speced n Theorem Modcaton of NE/SQP to Guarantee Fnte Termnaton The NE/SQP algorthm has the drawback that t does not necessarly termnate n a nte number of teratons unless t converges to a soluton. In partcular, whle the algorthm guarantees descent of at every teraton, the sequence f(x k )g may not converge to 0. Ths can happen ether by generatng an unbounded sequence of ponts, or by convergng slowly to an rregular pont. Ths wll clearly be unacceptable f we are to use the algorthm to solve a sequence of perturbed subproblems. We therefore present a moded NE/SQP algorthm whch has the same local convergence propertes as the orgnal NE/SQP algorthm, but whch also guarantees nte termnaton, even when t fals. Moded NE/SQP Algorthm Step 1 [Intalzaton] Gven a startng vector x 0 IB, a convergence tolerance tol, and termnaton parameters (0; 1), and 11, select ; (0; 1), and set k = 0. Step [Drecton generaton] Solve (QP k ), gvng the drecton d k. If (x k ; d k ) (1 )(x k ), or f d k > (x0 ), then termnate the algorthm, returnng the pont x k along wth a falure message; otherwse, contnue. Step 3 [Steplength determnaton] Let m k be the smallest nonnegatve nteger m such that Set x k+1 = x k + m kd k and contnue. (x k + m d k ) (x k ) m z(x k ; d k ): (7) Step 4 [Termnaton check] If (x k+1 ) tol termnate the algorthm, returnng the soluton x k+1. Otherwse, return to Step, wth k replaced by k + 1. Note that by settng = 0 and = 1, the moded algorthm s dentcal to NE/SQP, wth the addton of a partcular stoppng crtera n Step 4. However, by choosng (0; 1) and < 1, we can ensure that the algorthm wll termnate n a nte number of teratons, whch we wll prove n Theorem Ths has the drawback that the moded algorthm may fal when the orgnal algorthm would have succeeded. However, we shall overcome ths drawback n the QPCOMP algorthm by carefully controllng the parameter. Moreover, the moded algorthm also has the same local convergence propertes as the orgnal algorthm. To establsh ths fact, we use the followng two lemmas to show that f x k s near a b-regular soluton of MCP(f; IB), then the tests n Step can never cause falure. Lemma 3.7 ([1], Lemma..14) Let x be a soluton of MCP(f; IB). If x s b-regular, then there exsts a constant c > 0 such that for any vector x k IB close enough to x, d k H(x c k ) where d k s any soluton to the quadratc program (QP k ). 10 ;

11 Observe, that when x k s close enough to a b-regular soluton, H(x k ) d k H(x0 ) d, and therefore, k H(x0 ) =c, so (x 0 ). Thus, when x k s close to a b-regular soluton, the second test n Step of the Moded NE/SQP algorthm cannot cause falure. We now show that the rst test n Step cannot cause falure ether. Lemma 3.8 ([1], Lemma..19) Let x be a soluton of MCP(f; IB). If x s b-regular, then for any (0; 1=), there s a neghborhood N IB of x such that f x k N, then where d k s an optmal soluton of (QP k ). (x k ; d k ) (x k ); The above lemmas show that for x k close enough to x, the moded algorthm wll not termnate n Step, as long as x s b-regular. Thus, the moded algorthm has the same local convergence propertes as the orgnal algorthm. Ths establshes the followng theorem: Theorem 3.9 Under the condtons of Theorem 3.4, the Moded NE/SQP algorthm generates a well dened sequence of terates fx k g IB, along wth a sequence of optmal solutons fd k g for the subproblems (QP k ). Furthermore, f x s an accumulaton pont of fx k g, and f ether f s strongly monotone, or x s both b-regular and s-regular, then x s a soluton of MCP(f; IB) and the terates converge to x at the rates speced n Theorem 3.4. The remander of ths secton s amed at provng that the Moded NE/SQP algorthm termnates. Ths s accomplshed by consderng what happens f the algorthm does not termnate. In ths case, we shall show that the terates fx k g converge to a pont x. Usng ths fact, we wll place bounds on certan quanttes, whch wll then be used to establsh a mnmum rate of decrease for the mert functon. Ths wll then force the mert functon to zero, whch means that the algorthm wll termnate after all, by the test n Step 4. For ease of dscusson, we dene the functon x (d) := (x; d). The followng lemma s a techncal result needed n several ensung proofs. Lemma 3.10 ([1], Lemma..1) If x (d) (1 )(x) then z(x; d) 1 (x). We now prove that the terates converge. Lemma 3.11 Suppose f s contnuously derentable. If the Moded NE/SQP algorthm, wth (0; 1) and < 1, fals to termnate, then the terates fx k g produced by the algorthm wll converge to a pont x IB wth (x ) > 0. Proof Let k (d) := (x k ; d) and let z k (d) := z(x k ; d). By the test n Step of the algorthm, k (d) (1 )(x k ). Thus, by Lemma 3.10, z k (d) 1 (x k ). Let f k g be the sequence of steplengths generated n step 3 of the algorthm,.e., k := m k. Then, (x k+1 ) = (x k + k d k ) (x k ) k z k (d k ) (by the lnesearch test (7)) (x k ) k (x k )= (by Lemma 3.10) = (1 k )(x k ): 11

12 Let ^ k := k =. Then (x k+1 ) (x 0 ) Snce (x k ) s bounded away from 0, t follows that 1Y k=0 ky j=0 (1 ^ k ) > 0: (1 ^ j ): But ths mples that P 1 ^ =0 k s nte, whch means that P 1 =0 k s nte. Now, by the test n Step of the algorthm, d k (x0 ). Thus, d k s bounded, so 1X d k k < 1: k=0 From ths t follows that the sequence of terates fx k g converges to some pont x. Clearly, (x ) > 0, or the algorthm would termnate n Step 4. Usng the fact that the terates converge, together wth straghtforward contnuty arguments, bounds can be placed on several quanttes, whch wll be useful n provng Lemma Lemma 3.1 ([1], Lemma..3) Under the hypotheses of Lemma 3.11, there exst constants M 1, M, and L, dependng on the startng pont x 0, such that for all [0; 1], the followng nequaltes hold: jf (x k + d k )j M 1 ; jrf (x k + d k )j M (8) and f (x k ) L d k f (x k + d k ) f (x k ) + L d k : (9) Furthermore, for any > 0, we can choose ^() > 0 such that for k sucently large, the followng holds for all [0; ^()]: jf (x k + d k )j f (x k ) + rf (x k ) > d k + d k : (10) We are now able to establsh a mnmum rate of decrease for the mert functon. Lemma 3.13 Under the hypotheses of Lemma 3.11, there exsts a constant ^ (0; 1) such that (x k+1 ) ^(x k ); 8 k sucently large. Proof Suppose (0; 1), and let [0; ^()] where ^() s chosen accordng to Lemma 3.1. Suppose that k s large enough that (10) holds. We shall examne the terms H (x k + d k ) n order to establsh an upper bound on (x k+1 ) = P H (x k + d k ) =: To smplfy notaton, we drop the superscrpts k. Thus, we let x := x k and d := d k, etc. We shall also nd t convenent to dene the scalar functon ^ : IR+! IR+, as follows: Observe that ^ 00 (0) = z (x; d), so ^ () := (x; d): To bound H (x + d), we have to look at two derent cases: nx =1 ^ 00 (0) = z(x; d): (11) 1

13 Case 1: I f (x). Note that jh (x + d)j jf (x + d)j. Thus, by (10), H (x + d) (f (x) + rf (x) > d) + f (x) + rf (x) > d kdk + kdk : But, (f (x) + rf (x) > d) = ^ () = ^ (0) + ^ 0 (0) + ^00(0), so H (x + d) ^ (0) + ^ 0 (0) + ^00 (0) + f (x) + rf (x) > d kdk + kdk : (1) Case : 6 I f (x). We look only at the case I l (x) S I el (x); the argument for I u (x) S I eu (x) s smlar. where and If H (x + d) s negatve, then Thus, H (x + d) = f (x + d) f (x) L kdk by (9) x l + d (d + L kdk) snce f (x) x l x l + d (L + 1) kdk : H (x + d) (x l + d ) (x l + d )(L + 1) kdk + (L + 1) kdk (x l + d ) + (L + 1) kdk : If H (x + d) s nonnegatve, ths nequalty holds trvally snce H (x + d ) x l + d. Fnally, (x l + d ) = ^ () = ^ (0) + ^ 0 (0) + ^00 (0), so H (x + d) ^ (0) + ^ 0 (0) + ^00 (0) + (L + 1) kdk : (13) Summng over all components, we get (x + d) = 1 X H (x + d) x (0) + 0 x (0; d) + + ; (14) := nx =1 := X I f (x) ^ 00 (0) + f (x) + rf (x) > d kdk ; X 6I f (x) We now establsh bounds for and. By (8), X I f (x) (L + 1) kdk + X I f (x) kdk : f (x) + rf (x) > d kf(x)k + M kdk M 1 + M q(x 0 ) =: C 1 : Thus, C 1 kdk C 1 p (x0 ) =: K 1. For, we deduce from (11) that nx =1 ^ 00 (0) = z(x; d) (x); by tem 4 of Proposton 3.1: 13

14 Thus, (x) + kdk n(l + 1) + n 1 + n((l + 1) + ) (x 0 ); snce kdk (x 0 ) K ; where K := (1 + n((l + 1) + 1))(x 0 ). Ths last nequalty holds snce 1. Returnng to (14), By Item of Proposton 3.1, Thus, (x + d) x (0) + 0 x(0; d) + K 1 + K = (x) + 0 (x; d) + K 1 + K : 0 (x; d) x (d) x (0) z(x; d) (1 )(x) (x) z(x; d); by the test n Step = (x) z(x; d): (x + d) (x) ( (x) z(x; d)) + K 1 + K : Note that the dentons of K 1 and K are ndependent of. We can therefore consder a partcular choce of : let := mn(1; (x )=(K 1 )) and let := mn(^(); (x )=(K )). Note that > 0 and > 0, snce (x ) > 0. It follows that for all, and for k sucently large, (x + d) (x) z(x; d) (x) + (x )= + K z(x; d) (x) + (x)= + (x)=; snce (x ) (x) = z(x; d) z(x; d); 8 1: (15) Observe that the steplength m generated by Step 3 of the algorthm s chosen such that m s the smallest nteger satsfyng (7). Thus, := m 1 cannot satsfy (15). But ths means that m 1 ; whch mples m : It follows by the lnesearch test (7) and Lemma 3.10 that (x + m d) (x) z(x; d) (1 )(x): By settng ^ := 1 =, the proof s complete. Theorem 3.14 If (0; 1) and < 1, then the moded NE/SQP algorthm wll termnate n a nte number of teratons provded that f s contnuously derentable on IB. Proof Let tol > 0 be the stoppng tolerance used n the algorthm. If the algorthm does not termnate, then by Lemma 3.13, there exsts ^ (0; 1) such that for k sucently large, (x k+1 ) ^(x k ). Thus, after sucently many teratons, (x k ) < tol, and the algorthm wll termnate n Step 4. 14

15 4 The QPCOMP Algorthm The basc dea behnd QPCOMP s smple. The algorthm rst tres to solve the problem usng the moded NE/SQP algorthm. If ths fals, QPCOMP then solves a sequence of perturbed problems n order to nd a pont wth an mproved value of the mert functon. Once ths pont s found, QPCOMP returns to runnng the moded NE/SQP algorthm on the orgnal problem, startng from ths mproved pont. One complcaton of the algorthm s that the subproblems must be solved nexactly n order to guarantee that they are each completed n a nte amount of tme. To handle ths we have ntroduced a sequence of tolerances f j g whch control the accuracy demanded by each subproblem. Another complcaton s that the best choces of the parameters and cannot be known n advance. We now state the algorthm, ncludng a descrpton of how these parameters are adaptvely chosen. Algorthm QPCOMP Step 1 [Intalzaton] Gven a startng vector x 0 IB and a convergence tolerance > 0, choose > 0, (0; 1), (0; 1), (0; 1), and set k = 0. Step [Attempt NE/SQP] Run the Moded NE/SQP algorthm wth startng pont x k, wth tol =, to generate a pont ~x. Step 3 [Termnaton check] If ~x solves MCP(f; IB), stop; otherwse contnue wth step 4. Step 4 [Generate better startng pont] Set best := (~x), set y 0 = ~x, set j = 0, and choose > 0, and choose a postve sequence f j g # 0. Step 4a Run the Moded NE/SQP algorthm to solve the perturbed problem MCP(f ;yj ; IB) from startng pont y j, wth tol = j =(1+ y j ). Ths generates a pont ~y. Step 4b If ~y fals to solve the perturbed problem to the requested accuracy, set + and, and goto step 4a; otherwse, contnue. Step 4c [Check pont] If (~y) best, set x k+1 = ~y and return to step, wth k replaced by k + 1. Otherwse, set y j+1 := ~y and return to step 4a, wth j replaced by j + 1. Observe, that the QPCOMP algorthm has the same local convergence propertes as NE/SQP. In partcular, by Theorem 3.9, for any b-regular soluton x, there s a neghborhood such that the moded NE/SQP algorthm s dentcal to NE/SQP wthn ths neghborhood. Thus, n Step of the QPCOMP algorthm, f x k s sucently close to x, then the moded NE/SQP algorthm wll converge to x at the rates speced by Theorem 3.4. We now establsh global convergence propertes for the algorthm: Theorem 4.1 If f s Lpschtz contnuous and contnuously derentable on IB, and f MCP(f; IB) satses Assumpton., then for any > 0 the QPCOMP algorthm generates an terate x k satsfyng (x k ) < n a nte number of teratons. The remander of ths secton s devoted to provng ths theorem. As an ntroducton to the proof, note that f Step 4 s always successful at generatng an mproved startng pont, then even 15

16 f the Moded NE/SQP always fals n Step, the mert functon values f(x k )g wll converge to 0 at least lnearly, snce (x k+1 ) (x k ) for all k. Thus, our convergence analyss s reduced to provng that Step 4 always generates an mproved startng pont. In the analyss that follows, t wll be convenent to dene perturbed ndex sets by l (x) := f : x l < f ;x (x)g el (x) := f : x l = f ;x (x)g f (x) := f : x u < f ;x (x) < x l g I ;x I ;x I ;x I ;x eu (x) := f : x u = f ;x (x)g u (x) := f : x u > f ;x (x)g: I ;x We shall also use the followng obvous perturbatons of the functons H,,, and z: H ;x (x) := mn(x l ; max(x u ; f ;x (x))); ;x (x) := 1 H ;x (x) ; ;x x (d) := ;x (x; d) := P ;x (x; d); 8 where 1 >< (x ;x l + d ) I ;x l (x) S I ;x el (x) 1 (x; d) := >: (x u + d ) Iu ;x (x) S Ieu ;x (x) 1 ;x (f (x) + rf ;x (x) > d) I ;x f (x) zx ;x (d) := z;x (x; d) := P z ;x (x; d); where ( 1 z ;x (x; d) := d 6 I ;x f (x) ;x (rf (x) > d) I ;x = 1; : : :; n: (x) 1 f = 1; : : :; n: To show that Step 4 s always successful at generatng an mproved startng pont, we begn by assumng that the Moded NE/SQP algorthm n Step 4a of QPCOMP fals at most a nte number of tmes. Later, we wll remove ths assumpton. It follows that after a nte number of teratons, ~y always solves the perturbed problem to the desred accuracy, so the algorthm always contnues past Step 4b to Step 4c. Thus, ether an mproved pont wll eventually be found, or the algorthm wll generate a sequence of terates fy j g such that H ;yj (y j+1 ) j 1 + ky j k : We then use the fact that f j g converges to 0 to show that (y j )! 0. Ths result s proved n the followng lemma: Lemma 4. Let f be a Lpschtz contnuous functon and let f k g be a sequence of postve numbers that converges to 0. Let > 0 and let fx k g be a sequence of ponts n IB such that H ;xk (x k+1 ) k ; 8k: (16) 1 + kx k k Suppose MCP(f; IB) satses Assumpton., then for any > 0, there exsts an terate x j fx k g such that (x j ). Proof Let x be the soluton to MCP(f; IB) guaranteed by Assumpton. whch satses (1), and let y k := H ;xk (x k+1 ). In the same sprt as the proof to Theorem.3, we establsh a lower bound on the term (x k+1 x )(xk xk+1 ). 16

17 Case 1: y k = x k+1 l and x < xk+1. Observe that where (x k+1 x )(x k x k+1 ) = (x k+1 x ) f (x k+1 ) y + w k ; (17)! w k := (x k+1 x ) x k x k+1 f (x k+1 ) y : Now, 0 < (x k+1 x ) xk+1 l = y k. Also, xk xk+1 l x k+1 = y k. Thus, w k = (x k+1 x ) x k xk+1 + y = y y k k + yk = (y k) jy kj f (x k+1 ) =: Returnng to (17), we get (x k+1 Case : y k = x k+1 f (x k+1 ) + (x k+1 jy k j f (x k+1 ) (x k+1 x )(f (x k+1 )=) = x )(xk xk+1 ) (x k+1 x ) f (x k+1 ) y = (y k) j jyk l ; and x xk+1 x k ) yk, so xk xk+1. In ths case, f ;xk (x k+1 ) x k+1 (f (x k+1 ) y k (x k+1 x )(x k x k+1 ) (x k+1 x ) f (x k+1 ) : (18) l = y k. Thus, )=. Snce xk+1 x 0, we get f (x k+1 ) y =: (19) Case 3: y k = f ;xk (x k+1 ). In ths case, y k = f (x k+1 ) + (x k+1 )=. Thus, y k (x k+1 x k ), so xk xk+1 x )(xk xk+1 ) = (x k+1 x f ) (x k+1 ) y k =: = (f (x k+1 ) Case 4: y k = x k+1 u ; x k+1 x. By smlar arguments to Case, nequalty (19) s satsed. Case 5: y k = x k+1 u ; x k+1 < x. By smlar arguments to Case 1, nequalty (18) s satsed. In every case above, nequalty (18) holds. Thus, (x k x ) = (x k+1 Summng over all components, we get x k x x + xk xk+1 ) = (x k+1 x ) + (x k+1 x )(xk xk+1 ) + (x k xk+1 ) (x k+1 x ) + (x k+1 x f ) (x k+1 ) y k = (y k) jyk j f (x k+1 ) + (x k xk+1 ) ; by (18). x k+1 x + f(x k+1 ) > x k+1 x = y k > x k+1 x = y k y n k f(x k+1 ) = + x k x k+1 : 17

18 Now, let L be the Lpschtz constant for f. Then f(x k+1 ) f(x k+1 ) f(x ) + kf(x )k L x k+1 x + kf(x )k. Further, by Assumpton., f(x k+1 ) > x k+1 x 0. Thus, x k x where Note that = k := x k+1 x y k x k+1 x y = k n y L k x k+1 x + kf(x )k = + x k x k+1 x k+1 x k x k+1 x = 1 + x k k 1 = + x k by (16) n k L x k+1 x + kf(x )k = 1 + x k + x k x k+1 x k+1 x x + k x k+1 k k ; x k+1 x (1 + kx k k) + k (1 + kx k k) + n k L x k+1 x + kf(x )k (1 + kx k k) : (0) x k+1 x (nl + 1)= n kf(x )k =: (1) x k+1 x 1. Now, Let C := (nl + 1)= n kf(x )k =. Then k C mples that let f k : k g be the subsequence of f k g for whch k C; 8k. It follows then that x k+1 x x 1; 8k. If we dvde each sde of (1) by k+1 x, t s easly seen that f k = x k+1 x : k g s bounded. However, dvdng (0) by x k+1 x gves x k x x k x k+1 kx k+1 x k 1 + kx k+1 x k k k kx k+1 x k : Snce k # 0, the last term above converges to 0. Thus, for k large enough, and Observe that x k x kx k+1 x k > 1 ; k x k x (1 + kx k k) + k (1 + kx k k) + n x k x x k L x k x + kf(x )k (1 + kx k k) (1 + kx k k) kx k + (1 + kx k k) kx k + 1: Thus, the subsequence f k : k g s bounded, from whch t follows that f k g s bounded. Now, assume the lemma s false. Then there exsts an > 0 such that for all k, (x k ) > =, whch mples H(x k ) >. Furthermore, for k large enough, k < : Wthout loss of generalty, we can assume that ths nequalty holds for all k. : 18

19 Snce f s Lpschtz contnuous, H ;xk s also Lpschtz contnuous wth some Lpschtz constant K. But then, < H(x k ) k H ;xk (x k ) H ;xk (x k+1 ) 1 + x k Thus, for small enough, K x k+1 x k H ;xk (x k ) H ;xk (x k+1 ) =(K) < ( )=K < x k+1 x k : Fnally, snce the sequence f k k g converges to 0, then for all k sucently large, k k < =(8K). Thus, from (0) x k x But, then = x k x 1X x k+1 x x + k+1 x k k k x k+1 x + =(K) =(4K) x k+1 x + =(4K): k+1 The lemma s thus proved by contradcton. =(4K) = 1 > x k x : Note that Lemma 4. dd not make any assumpton on the choce of other than that t s greater than 0. Thus, even f s smaller than the Lpschtz constant, we can guarantee convergence. The next stage n our analyss s to prove that the Moded NE/SQP algorthm can fal at most a nte number of tmes n Step 4a of QPCOMP. Ths s accomplshed by observng that after each falure, the value of s ncreased, whle the value of s decreased. Thus, the result wll be proved f we can show that for large enough, and small enough, the Moded NE/SQP algorthm wll always solve the perturbed problem MCP(f ;yj ; IB). Ths s accomplshed by the followng two lemmas. Lemma 4.3 ([1], Lemma.3.3) Suppose f s Lpschtz contnuous wth Lpschtz constant L, and let x and x be arbtrary ponts n IB. If > L +, and f d satses ;x x ( d) ;x x (0), then d < 11 ;x (x): Lemma 4.4 Suppose f s Lpschtz contnuous. There exst constants > 0, and 0, such that for any, the moded NE/SQP algorthm appled to MCP(f ;x ) wll not termnate n Step for any and x IB. Proof Suppose the lemma s false. Then there must exst a sequence f j ; j g, wth! 1 and # 0 such that for each j there exsts a perturbed problem MCP(f j;x j ; IB) where the moded NE/SQP algorthm wth := j fals n Step when run on MCP(f j;x j ; IB)g. Dene f j (x), H j (x), j (x), and j (x; d), to be the f, H,, and functons correspondng to the jth perturbed problem. For example f j (x) := f j;x j (x), etc. Then for the jth problem to fal n Step, there must exst a pont x j and a drecton d j such that d j s an optmal soluton to the quadratc program (QP j ) dened by mn j (x j ; d) x j +dib 19

20 and also d j fals one of the two tests n Step of the algorthm. Wthout loss of generalty, we can assume j L + ; 8j. By Lemma 4.3, d j < 11 j (x j ) j (x j ). Thus, the falure must occur because of the rst test n Step. In other words, Snce j (x j ; d j ) j (x j ; 0) = j (x j ), and also, j # 0, we see that j (x j ; d j ) (1 j ) j (x j ); 8j: () lm j(x j ; d j ) j (x j ) = 1: (3) Let I j := I j;x j f (x j ), J j be the set of ndces not n I j, H j I j (x j ) A j := kh j (x j )k H j J j (x j ) and B j := kh j (x j )k : We rst show that lm j!1 A j = 0. To do ths, we examne a partcular choce of j. Let H j := H j (x j ). We can then rewrte j (x j ; d), as follows: j (x j ; d) := 1 (M j + D j )d + H j where ( rf j M; j := (xj ) > f I j 0 f J j : ( f D j := Ij 1 f J j : Observe that x j u H j xj l. Note that for d ~ dened by ( H j ~d := = f I j 0 f J j t follows that x j + ~ d IB, snce 1. Furthermore (M j + D j ) ~ d + H j = Now, snce d j s an optmal soluton to (QP j ), Thus, by (3), j (x j ; d j ) j (x j ; ~ d) = = lm j(x j ; d j ) j (x j ) 3 4 (M j d) ~ Ij 5 : H j J j (M j + D j ) d ~ + H j = 1 (M j d)ij ~ + H j J j L d ~ H + j J j 1 H j L A j = j + B j lm nf L A j j But, snce fa j g s bounded, and j! 1, we see that 1 lm nf B j. Furthermore, B j 1, so lm B j = 1, whch mples that A j! 0. + B j! : : 0

21 Let us now examne the drecton ndng subproblem (QP j ) for large j. For some [0; 1], dene ~ d by Here we see that Thus, j (x j ; d j ) 1 ~d := ( 0 f Ij H j f J j : (M j + D j ) ~ d + H j = 4 H j I j + (M j ~ d)ij (1 )H j J j (M j + D j ) d ~ + H j H = 1 j I j + (M j d)ij ~ + (1 ) H j J j H 1 j I j + H j M j ~ I j d + M j d ~ + (1 ) H j J j 1 (A j H j ) + A j H j L d ~ + L d ~ + (1 )B j H j 1 H j A j + A jl + L + (1 ) Bj (x j ) A j + A jl + (1 + (L + 1) ) 3 5 : ; snce B j 1: Choosng = 1=(1 + L ), we get j (x j ; d j ) (x j ) A j (L=(1 + L ) + A j ) + 1 1=(1 + L ). But snce lm A j = 0, we see that lm sup j(x j ; d j ) (x j ) L < 1; contradctng (3). Thus, the lemma s proved by contradcton. We can now combne the results of the prevous three lemmas to prove that Step 4 always generates an mproved startng pont. Lemma 4.5 Suppose that f s Lpschtz contnuous and contnuously derentable on IB and that MCP(f; IB) satses Assumpton.. If the QPCOMP algorthm fals to termnate, t wll execute Step an nnte number of tmes. Proof Assume the lemma s false. It then follows that after a nte number of statements are executed, the algorthm never returns to Step. But ths means that, thereafter, the test n Step 4c of the algorthm s never satsed. By Theorem 3.14, the moded NE/SQP algorthm wll always termnate n a nte number of steps. Thus, Step 4b of the QPCOMP algorthm wll be executed an nnte number of tmes. But the test n Step 4b can fal only a nte number of tmes. After that, wll be large enough and wll be small enough that by Lemma 4.4 the Moded NE/SQP algorthm wll always nd a soluton to the perturbed problems. Thus we see that Step 4c s vsted an nnte number of tmes, and moreover, after a nte number of teratons, the value of s xed. But then Lemma 4. guarantees that the test n Step 4c wll be satsed after a nte number of teratons. But ths contradcts our orgnal assumpton, so the lemma s true. We are now ready to prove Theorem 4.1 1

22 Proof (of Theorem 4.1) By Lemma 4.5 ether the algorthm wll termnate wth a soluton n Step 3, or Step wll be executed an nnte number of tmes. But f Step s executed an nnte number of tmes, then we have (x k+1 ) < (x k ) =) (x k ) < k (x 0 ); so (x k ) converges to zero. 5 Implementaton and Testng The QPCOMP algorthm was coded n ANSI C, usng double precson arthmetc. The Fortran package MINOS [1] was used to solve the quadratc subproblems. The algorthm allows for a great deal of exblty n the choce of parameters, whch can be speced n an optons le. For testng purposes, we used the followng choces of parameters n the QPCOMP and Moded NE/SQP algorthms: = :9, = 1:0e4, = :5, = :5. The sequence f j g used n Step 4 of the QPCOMP algorthm was gven by j+1 = 0:999 j, wth 0 set to Ths eectvely caused the Moded NE/SQP algorthm to perform only one teraton before returnng control back to QPCOMP. The parameter was updated as follows: 1. In Step 4, s set to best.. In Step 4b, f ~y fals to solve the perturbed problem, s set to max(:1; 10); otherwse, t s multpled by :9. Fnally, the parameter s ntally chosen to be :01. Thereafter, n Step 4b, t s set to mn(1=; ). For practcal consderatons, we also placed a lmt on the number of allowable teratons of the lnesearch n Step 3 of the moded NE/SQP algorthm. Ths lmt s set to 10 when the Moded NE/SQP algorthm s called from Step of QPCOMP, and s ncreased by 4 whenever the Moded NE/SQP algorthm fals, up to a maxmum of 30. QPCOMP was nterfaced wth the GAMS modelng language [, 6], allowng problems to be easly speced n GAMS, and the algorthm to be tested usng MCPLIB [4] and GAMSLIB []. Speccally, we tested QPCOMP on every problem wth fewer than 110 varables n MCPLIB and GAMSLIB. Larger problems were excluded because our mplementaton of QPCOMP uses a dense solver for the QP subproblems. Table summarzes the features of the problems tested. We also tested NE/SQP, PATH verson.8 [5], and SMOOTH verson 3.0 [3] on the problems n Table. To run NE/SQP, we smply used the QPCOMP algorthm wth = 1 and = 0. A comparson of the performance of the algorthms s gven n Table 3. Many of the problems n the lbrary are speced wth more than one startng pont. The partcular startng pont used s shown n the second column of the table. For each problem we report the executon tme (n seconds) and the number of functon and Jacoban evaluaton, f and J. To save space, we have omtted from ths table any problems whch all four algorthms solved n less than a second. All of the problems were solved to an accuracy of Speccally, for QPCOMP the stoppng crtera was kh(x)k The results of the testng demonstrate the hgh degree of robustness of the QPCOMP algorthm. We note that although t dd not solve the Von Thunen problems, QPCOMP was able to solve these problems to an accuracy of Expermentaton wth the Von Thunen problems suggests that the Jacoban matrx s sngular at the soluton. Thus, near the soluton, the Jacoban matrx s poorly condtoned. Ths ll-condtonng s exacerbated n QPCOMP by the fact that the QP subproblems are formulated usng the square of the Jacoban matrx, resultng n extremely ll-condtoned QP

23 Table : Models GAMS le Model orgn Type Sze Nonzeros bertsekas.gms Trac assgnment NCP bllups.gms Secton NCP 1 1 cafemge.gms GAMSLIB (139) MCP cho.gms Nash equl. NCP colvncp.gms Colvlle # NLP colvdual.gms Colvlle # (Dual) NLP ehl k60.gms Lubrcaton MCP ehl k80.gms " MCP ehl kost.gms " MCP freebert.gms Trac assgnment MCP gafn.gms " MCP 5 5 hanskoop.gms Captal stock NCP hansmcp.gms GAMSLIB (135) MCP hansmge.gms "(147) MCP harkmcp.gms GAMSLIB (18) MCP harmge.gms "(148) MCP 9 81 hydroc06.gms Dstllaton NE 9 3 hydroc0.gms " NE josephy.gms MCPLIB NCP 4 16 kehomge.gms GAMSLIB (149) MCP 9 81 kojshn.gms MCPLIB NCP 4 16 kormcp.gms GAMSLIB (130) MCP math*.gms Walrasan NCP 4 11 methan08.gms Dstllaton NE 31 6 nash.gms Nash equl. MCP olgomcp.gms GAMSLIB (133) MCP 6 16 pgvon105.gms Von Thunen NCP pgvon106.gms " NCP pes.gms PIES model MCP powell.gms Powell NLP powell mcp.gms " NCP 8 47 sammge.gms GAMSLIB (151) MCP scarfa*.gms Walrasan NCP scarfb*.gms " NCP scarfmge.gms " NCP shovmge.gms GAMSLIB (153) MCP sppe.gms Spatal prce MCP 7 84 tobn.gms " MCP 4 0 transmcp.gms GAMSLIB (16) MCP 11 4 two3mcp.gms GAMSLIB (131) MCP 6 4 unstmge.gms GAMSLIB (155) MCP 5 5 vonthmge.gms Von Thunen MCP wallmcp.gms GAMSLIB (17) MCP 6 0 3