Infeasible path following algorithms for linear complementarity problems

Transcription

1 INSTITUT NATIONAL DE RECHERCHE EN INFORMATIQUE ET EN AUTOMATIQUE Infeasible path following algorithms for linear complementarity problems J. Frédéric Bonnans, Florian A. Potra N 445 Décembre 994 PROGRAMME 5 Traitement du signal, automatique et productique apport de recherche ISSN

2

3 Infeasible path following algorithms for linear complementarity problems J. Frederic Bonnans, Florian A. Potra Programme 5 Traitement du signal, automatique et productique Projet Programmation mathematique Rapport de recherche n445 Decembre pages Abstract: A generalized class of infeasible-interior-point methods for solving horizontal linear complementarity problem is analyzed and sucient conditions are given for the convergence of the sequence of iterates produced by methods in this class. In particular it is shown that the largest step path following algorithms generates convergent iterates even when starting from infeasible points. The computational complexity of the latter method is discussed in detail and its local convergent rate is analyzed. The primal-dual gap of the iterates produced by this method is superlinearly convergent to zero. A variant of the method has quadratic convergence. Key-words: Linear complementarity problem, infeasible central path, interior-point algorithm, predictorcorrector algorithm, largest step algorithm, shifted analytic center (Resume : tsvp) INRIA, B.P. 05, 7853 Rocquencourt, France. Frederic.Bonnans@inria.fr Department of Mathematics, The University of Iowa, Iowa City, Iowa 54, USA, The work of this author was supported in part by NSF, Grant DMS The main results of this paper were obtained during a visit of the second author to INRIA in June 994, supported by the projects PROMATH and MENUSIN. Unité de recherche INRIA Rocquencourt Domaine de Voluceau, Rocquencourt, BP 05, 7853 LE CHESNAY Cedex (France) Téléphone : (33 ) Télécopie : (33 )

4 Algorithmes de suivi de chemin non realisable pour le probleme de complementarite lineaire Resume : L'article analyse une classe generale de methodes de points interieurs non realisables pour resoudre des problemes de complementarite lineaire, et donne des conditions susantes de convergence de la suite des points produite par les methodes de cette classe. On montre en particulier que les algorithmes de plus grand pas et suivi de chemin generent des iteres convergeant m^eme lorsque le point de depart n'est pas realisable. La complexite de ces methodes et leur taux de convergence asymtotique sont discutes en detail. Le saut primal-dual des iterees converge superlineairement vers zero. Une variante de la methode converge de facon quadratique. Mots-cle : Probleme de complementarite lineaire, algorithme de point interieur, algorithme predicteurcorrecteur, algorithme de plus grand pas, centre analytique decale

5 Infeasible path following algorithms for linear complementarity problems 3 AMS (MOS) subject classication: 49M5, 65K05, 90C33. Introduction. In this paper we study the convergence of a class of infeasible interior point methods for solving the horizontal linear complementarity problem (HLCP): (.) xs = 0 Qx + Rs = b x; s 0; where b IR n, and Q; R IR nn are such that for any u; v IR n, (.) Qu + Rv = 0 implies u T v 0: If Q; R is a pair of n n-matrices satisfying (.) we say that Q; R is a positive semidenite pair. It is known (see e.g. [] ) that this problem trivially includes the linear programming problem (LP) and the convex quadratic programming problem (QP) in their usual formulations, and thus provides a quite general framework for the study of algorithms. Of course any LP and QP can also be written as a standard linear complementarity problem (SLCP) which is a HLCP where R is the identity matrix and?q is positive semidenite. As mentioned by Wright [3] little loss of eciency is involved in solving LPs and QPs by embedding them in an algorithm for SLCP, provided the linear algebra takes into account the specic structure of the individual problem. However, as will be seen below, in the analysis of the algorithms it is very convenient to permute the components of x and s such that x denotes \the large variables" and s denotes \the small variables". Then the SLCP becomes an HLCP which gives one more reason (besides the symmetry of the formulation) to consider HLCP. For recent theoretical work on the relationship between dierent formulations of linear complementarity problems we refer to [6], [], []. The present paper is organized as follows. In Section we present a generic infeasible path following algorithm and study its basic properties. In Section 3 we propose an extension of the largest step path following algorithm for arbitrary positive infeasible starting points and prove its global convergence. The computational complexity of this infeasible path following algorithm depends on the quality of the starting point. If the starting points are large enough then the algorithm has O(nL) iteration complexity. If a certain measure of feasibility at the starting point is small enough then the algorithm has O( p nl) iteration complexity. In particular, for feasible starting points we recover the result of McShane [8]. In Section 4 we give sucient conditions for convergence of the sequences generated by the generic infeasible path following algorithm. Our results generalize the corresponding results obtained by Bonnans and Gonzaga [] in the feasible case. In the last section we use these general results to prove that for problems having a strictly complementary solution the sequence (x k ; s k ) of iterates generated by the largest step infeasible path following algorithm is convergent and that the sequences (x k ) T s k and r k := b? Qx k? Rs k measuring \optimality" and \feasibility" are superlinearly convergent to zero. Moreover we show that the largest step infeasible path following algorithm can be modied a la Gonzaga [4] so that the latter sequences are quadratically convergent while preserving the same global convergence and polynomial complexity as the original algorithm. This result is proved by generalizing the analysis of the largest step path following algorithm done by Gonzaga [4] in the feasible case. To our knowledge this algorithm is the rst infeasible interior point algorithm for LCP with quadratic convergence and O(nL) iteration complexity that uses only one matrix factorization and two RR n445

6 4 J. Frederic Bonnans, Florian A. Potra backsolves per iteration. We mention that the infeasible interior point algorithms for LCP of Potra [9] and Potra and Sheng [0] have also quadratic convergence and O(nL) iteration complexity but the former requires two matrix factorizations and three backsolves while the latter requires two matrix factorizations and two backsolves. Also, we note that the algorithms from [4], [], [3] and [6] have only O(n L)-iteration complexity. Kojima, Mizuno and Todd [7], mention that the O(nL) infeasible{interior{point algorithms for linear programming considered in that paper can be generalized for linear complementarity problems, but the superlinear convergence of the resulting algorithms has not been yet established. Very recently, Ye [5] has proposed a homogeneous self-dual reformulation of a standard form linear complementarity problem, for which a feasible starting point is always available, and has proposed an O( p nl) iteration algorithm for this problem, but has not proved any superlinear convergence results. Conventions. Given a vector x, the corresponding upper case symbol denotes as usual the diagonal matrix X dened by the vector. The symbol e represents the vector of all ones, with dimension given by the context. We denote component-wise operations on vectors by the usual notations for real numbers. Thus, given two vectors u; v of the same dimension, uv, u=v, etc. will denote the vectors with components u i v i, u i =v i, etc. This notation is consistent as long as component-wise operations always have precedence in relation to matrix operations. Note that uv Uv and if A is a matrix, then Auv AUv, but in general Auv 6= (Au)v. We denote the nullspace and range space of a matrix A by N (A) and R(A) respectively. We frequently use the O() and () notation to express the relationship between functions. Our most common usage will be associated with a sequence fx k g of vectors and a sequence f k g of positive real numbers. In this case x k = O( k ) means that there is a constant K (dependent on problem data) such that for every k IN, x k Kk. Similarly, if x k > 0, x k = ( k ) means that (x k )? = O(= k ). Finally, x k k means that x k = O( k ) and x k = ( k ). We use the same notations for a point x in a set parameterized by, say E. We say that x = O() (resp. x = (), x ) whenever there is a constant K such that kxk K (resp. x? = O(=); x ) for all x E, and all small enough. In particular, x in E means that there are constants K > K > 0, such that any x E satises K x i K, i = ; : : :; n. Given two vector functions x and y, x y means that x i y i for i = ; : : :; n.. A Generic Infeasible Path-Following Algorithm. We dene the measure of optimality of (x; s) IR n + as (.3) = n xt s ; and its measure of feasibility as krk where r is the residual in the linear part of (.), (.4) r = b? Qx? Rs : It is easily seen that the measure of optimality of (x; s) is in fact the normalized -norm of the residual of the nonlinear part of (.), n = kxsk, so that nding a solution of HLCP (.) means nding a pair (x; s) IR n + with = 0 and r = 0. We consider algorithms for solving the HLCP (.) that follow approximately the infeasible central path pinned on a, dened as the set of triplets (x; s; ) that satisfy (.5) xs = e; Qx + Rs = b? a; INRIA

7 Infeasible path following algorithms for linear complementarity problems 5 where a is a constant vector related to the starting point (.6) by (.7) w 0 = (x 0 ; s 0 ; 0 ) IR n+ ++ a = r 0 = 0 ; with r 0 being the residual in the linear part of (.) at the starting point, (.8) r 0 = b? Qx 0? Rs 0 : It is easily seen that if a is dened as above then the second equation of (.5) is satised at the starting point. If we choose arbitrary s 0 > 0; 0 > 0 and take x 0 = 0 =s 0 then the rst equation of (.5) is also satised i.e., the starting point chosen in this way belongs to the infeasible central path pinned on a. It is easily seen that for any triplet (x; s; ) belonging to the infeasible central path we have =. Even if the starting point is on infeasible central path, the subsequent points (x; s; ) produced by the algorithm will not be on this path and therefore and will be dierent in general. Nevertheless, because the second equation in (.5) is linear, and we consider algorithms based on Newton's method, it follows that if the second equation in (.5) is satised by the starting point, then it will be satised by all subsequent points. We will assume that the algorithms under consideration produce points (x; s; ) with decreased values of (i.e., 0 ) and which belong to a certain neighborhood of the infeasible central path. More precisely we assume that the points belong to a \large" neighborhood of the form V := fw = (x; s; ) IR n + IR n + IR + ; Qx + Rs = b? a; e xs? e; 0 g where 0 < < is a given constant, or to a \small" neighborhood N := fw = (x; s; ) IR n + IR n + IR + ; Qx + Rs = b? a; k xs? ek ; 0g; where > 0 is another given constant. It is easily seen that (.9) and N V for all 0 <? (.0) V N for all p n(? ): We also note that (.)? for all (x; s; ) V ; so that in V the parameter and the optimality measure have the same size, while, as remarked before, on the infeasible central path these two quantities coincide. Moreover (.) Qx + Rs = b? a implies r = a = (= 0 )r 0 ; RR n445

8 6 J. Frederic Bonnans, Florian A. Potra so that on V the feasibility measure krk has also the same size as. At a typical point of an algorithm belonging to the class to be studied in this paper we have already computed a point (x; s; ) V, and we want to compute a new point in V with a smaller value of. Such a point can be obtained as an approximate solution of (.5) with replaced by where [0; ] is the desired reduction factor. The associated Newton direction (u; v) is given by the following linear system: (.3) su + xv =?xs + e; Qu + Rv = (? )a; which under the monotonicity assumption (.) has a unique solution. The new point is obtained by taking a steplength (0; ] along this direction i.e., (.4) x ] = x + u; s ] = x + v: It is easily seen that (.5) Qx ] + Rs ] = b? (? + ): If we take by denition (.6) ] := (? + ); then ] 0, so that the new point (x ] ; s ] ; ] ) belongs to V, provided the \centering parameter" [0; ] and the steplength (0; ] are chosen such that (.7) ] e x ] s ]? ] e: On the other hand if we want the new point (x ] ; s ] ) to belong to N then the steplength (0; ] has to be chosen such that (.8) kx ] s ]? ] ek ] : We stress the fact that the algorithms we intend to study do not necessarily choose rst a parameter, then solve the Newton equations (.3) to obtain direction (u; v), and determine steplength such that (.7) is satised. For example, in the largest step path following algorithm to be considered in the next section we solve (.3) with = 0 to obtain the ane scaling direction (u a ; v a ), and with = to obtain the ane centering direction (u c ; v c ), and then we set (u; v) = (u c ; v c ) + (? )(u a ; v a ) which clearly veries (.3) for any. The new point is obtained by taking = and determining such that (.8) is satised. We will see that this involves the solution of a quartic equation in. According to (.9) this ensures that the new point belongs to V for all?. With this observation we are ready to state the Generic Infeasible Path-Following Algorithm (GIPFA): Algorithm.. Data:, (x 0 ; s 0 ; 0 ) V k := 0 INRIA

9 Infeasible path following algorithms for linear complementarity problems 7 repeat x := x k ; s := s k ; := k ; Compute by some procedure scalars k [0; ], k (0; ] and vectors u; v such that (.3) is satised and the point w ] = (x ] ; s ] ; ] ) dened by (.4), (.6) belongs to V ; x k+ := x ] ; s k+ := s ] ; k+ = ] ; k := k +. This general algorithm includes many infeasible interior point algorithms, such as the predictor-corrector algorithm of Potra and Sheng [0], where centering steps alternate with ane-scaling steps and the largest step infeasible path following method that was briey described above and will be studied in detail in the next section. We will assume that GIPFA produces a sequence f k g that converges R-linearly to zero, in the sense that (.9) lim sup( k ) =k < : k! All infeasible interior point algorithms we know of have this property. It is easily seen that if (.9) is satised then any accumulation point of the sequence f(x k ; s k )g produced by GIPFA converges to a solution of (.). We will prove in the next theorem that if (.) has a solution then the set V is bounded and therefore the sequence f(x k ; s k )g has accumulation points. The problem is then under what conditions is there a unique accumulation point, that is under what conditions does f(x k ; s k )g converge to a solution of our problem? In Section 4 we will prove that if our problem has a strictly complementary solution then there exists > 0 such that if for all suciently large k we have either w k N P or k k then f(x k ; s k )g converges to a strictly complementary solution (x ; s ) of (.). Moreover if k=0 k k = +, then (x ; s ) is the shifted analytic center of the optimal facet. Before ending this section let us introduce some useful notation and prove a theorem showing that the existence of a solution to the HLCP (.) implies the boundedness of V. We denote the feasible set, and set of solutions of (.) as (.0) F := f(x; s) IR n + IRn + ; Qx + Rs = bg; and (.) S := f(x; s) F; xs = 0g: Also, we denote the set of strictly complementary solutions by (.) S 0 := f(x; s) S; x + s > 0g: Finally, the set of -approximate solutions of (.) is dened by (.3) S := f(x; s) IR n + IRn + ; xt s ; kqx + Rs? bk g: It is well known that if the HLCP (.) represents a linear programming problem, then if S is nonempty so is S 0. This is not true in general, since it is easy to construct an HLCP with nonempty S and empty S 0. RR n445

10 8 J. Frederic Bonnans, Florian A. Potra The existence of a strictly complementary solution will be an essential assumption in Section 4. However in the following theorem and in the next section we do not make this assumption. Theorem.. If S is nonempty then for any (x; s; ) V we have (.4) (x T s 0 + s T x 0 ) (? + )n 0 ; where (.5) = inf ((x 0 ) T s + (s 0 ) T x )=(n 0 ) : (x ; s ) S : Proof. For any (x ; s ) S and any (x; s; ) V we can write Q( 0 x? x 0? ( 0? )x ) + R( 0 s? s 0? ( 0? )s ) = 0; and since Q; R is a positive semidenite pair (see (.)) we have ( 0 x? x 0? ( 0? )x ) T ( 0 s? s 0? ( 0? )s ) 0; where-from, by using the fact that (x; s; ) V implies x T s? n and 0, we deduce 0 (x T s 0 + s T x 0 ) 0 (x T s 0 + s T x 0 ) + 0 ( 0? )(x T s + s T x) Dividing by 0, we obtain the desired inequality. 0 xt s + x 0T s 0 + ( 0? )(x T s 0 + s T x 0 ) n 0? + n 0? + 0 (x T s 0 + s T x 0 ): 3. The Largest Step Infeasible Path Following Algorithm. As mentioned in Section the Largest Step Infeasible Path Following Algorithm (LSIPFA) can be obtained from GIPFA by taking =, and the smallest value such that the new point is still in N. This smallest value is computed in the algorithm as the root of a quartic. In order to dene the quartic, one has to nd the solution of (.3) by rst solving the following two linear systems: (3.) su a + xv a =?xs; Qu a + Rv a = r (3.) su c + xv c =?xs + e; Qu c + Rv c = 0: The parameter has to be determined such that (3.3) kx()s()? ek ; where (3.4) x() = x + u(); s() = s + v(); INRIA

11 Infeasible path following algorithms for linear complementarity problems 9 and (3.5) u() = u c + (? )u a ; v() = v c + (? )v a : It is easily seen that (3.6) x()s() = e + u()v(): Since (3.7) u()v() = u c v c + (? )(u a v c + u c v a ) + (? ) u a v a ; the function (3.8) () := k? u()v()k? is a quartic. Condition (3.3) is equivalent to () 0. We will show that (0) 0 and () < 0 so that the quartic () has either one or three zeros in the open interval [0; [. We denote by the largest one. It follows that is uniquely dened by (3.9) () = 0; () < 0 for all ]; ]: In what follows we assume that is computed exactly. However, the whole theory can be rewritten in terms in appropriately chosen upper bounds of that are computable in O() arithmetic operations (see also [4]). By a continuity argument (3.9) ensures that (x(); s()) R n ++, 8 ]; ]; so that (x(); s(); ) N, 8 ]; ]. Therefore by dening (3.0) x ] := x + u(); s ] := s + v(); ] =:= we have (3.) (x ] ; s ] ; ] ) N : The complete algorithm can be written as follows: Algorithm 3.. Data: 0 < :5 > 0; (x 0 ; s 0 ; 0 ) N, where (3.) j 0 = 0? j? ; 0 = n (x0 ) T s 0 ; = + = p n : k := 0 repeat x := x k ; s := s k ; := k ; Solve the linear systems (3.) and (3.) ; Compute k the largest root of the quartic (3.8) from the interval ]0; [ ; x k+ := x ] ; s k+ := s ] ; k+ := ], according to (3.0) and (3.5) ; k := k +. RR n445

12 0 J. Frederic Bonnans, Florian A. Potra First let us note that for the standard choice of starting points x 0 = e; s 0 = e we have (x 0 ; s 0 ; ) N 0 N. More generally we consider a permutation matrix and starting points of the form L = [L L ] ; L i IR ni ; i = ; ; n + n = n; (3.3) x 0 = L e + L e ; s 0 = L e + L e ; where e i IR ni ; i = ; are the vectors of all ones of corresponding dimension. Obviously x 0 s 0 = e; 0 = ; so that we have again (x 0 ; s 0 ; ) N 0 N. In what follows we will prove that the above algorithm is globally convergent in the sense that the sequence f k g converges to zero at a global Q-linear rate. This implies that the sequence f k g converges to zero at a global R-linear rate. Under some assumptions on the starting points we will derive polynomial complexity bounds. In our proofs we will often use the following technical results: Proposition 3.. If Q; R is a positive semidenite pair then the matrix Q? R is invertible. Proposition 3.3. If Q; R is a positive semidenite pair then Qu + Rv = 0 implies u R(R T ) and v R(Q T ). In the next proposition we show how to reduce a HLCP to a SLCP. A more general result is given by Guler [6]. Our reduction scheme is inspired from []. Proposition 3.4. Let Q; R be a positive semidenite pair and denote Let ^n = rank(q) ; n = rank(r) ; ^n = n? ^n ; n = n? n : ^L = [^L ^L ] ; L = [ L L ] ; ^Li IR n^ni ; Li IR nni ; i = ; be two permutation matrices, such that rank(q^l ) = ^n and rank(r L ) = n. Consider the matrices and for any vector y IR n denote Then the matrices are invertible and the matrices ^Q i = Q^L i ; ^Ri = R^L i ; Qi = Q L i ; Ri = R L i ; i = ; ^y i = ^L T i y ; y i = L T i y; i = ; : ^W = [ ^Q ^R ] ; W = [ R Q ] ^M =? ^W? [ ^R ^Q ] ; M =? W [ Q R ] INRIA

13 Infeasible path following algorithms for linear complementarity problems are positive semidenite. Moreover the following three equations are equivalent: (3.4) Qu + Rv = g ; (3.5) (3.6) h ^u i h ^v = ^v i ^M ^u + ^W? g ; h v i h u = M u i v + W? g : Proof. Assume that ^W is not invertible. Then there is a vector z IR n? f0g such that (3.7) ^Q ^z + ^R ^z = 0 : Since ^Q is full rank we must have ^z 6= 0. The matrices ^Q = Q^L, ^R = R^L form clearly a positive semidenite pair and equation (3.7) can be written as ^Q h ^z0 i h + ^R 0^z According to Proposition 3.3 there is a vector y R n such that i = 0 : 0 = ^Q T y ; ^z = ^Q T y : Since rank( ^Q =rank(q) there is a matrix M such that ^Q = ^Q M and therefore we obtain the contradiction 0 6= ^z = ^Q T y = M T ^Q T y = 0 : Hence ^W is invertible. Then the equivalence in between (3.4) and (3.5) is easily veried. Taking g = 0 in (3.4) and (3.5) we deduce that for any h IR n there are vectors u; v IR n such that It follows that h = h ^v ^u i ^Mh = h ^u ^v i h T ^Mh = u T v 0 : ; Qu + Rv = 0 : Hence ^M is positive semidenite. Thus all the claims related to permutation ^L are proved. The corresponding claims for L are proved similarly. Lemma 3.5. If Q; R is a positive semidenite pair of n n-matrices and x; s; f; g are four n-dimensional vectors with x > 0 and s > 0, then the linear system (3.8) (3.9) su + xv = f Qu + Rv = g RR n445

14 J. Frederic Bonnans, Florian A. Potra has a unique solution (u; v) and the following relations are satised (3.0) (3.) where ky uk + Y? v + #(# + ); kuvk #( + #)( + #( + #)); Y = X?= S = ; = (XS)?= f ; # = minn ^y( ^W? g) ; y( W? g) o ; the matrices ^W, W are dened in Proposition 3.4 and?= ^x ^s = ^y = ^x = ^s?= ; y = x?= s = x = s?= Proof. Apply Proposition 3.4 and Corollary 3.8 of [9]. We are ready to prove now that the point from (3.9) exists and is unique, so that Algorithm 3. is well dened. Lemma 3.6. The quartic (3.8) has at least one zero in the interval [0; [, and its largest zero in [0; ] satises the inequality ; (3.) 0 b < ; where b is the unique zero in ]0; [ of the quadratic (3.3) () := cc + (? ) ac + (? ) aa? ; with coecients (3.4) cc = ku c v c k=; ac = ku c v a + v a v c k=; aa = ku a v a k=: Proof. First, let us prove that (0) 0 and () < 0 so that (3.9) denes indeed a unique number [0; ]. From (u; v) N and Lemma 3.5 it follows that: ku c v c k p 8 k(xs)? k kxs? ek p = p : 8(? ) 8(? ) Using the fact that :5 implies =(? ) =3 we obtain (3.5) cc 3 p 8 : INRIA

15 Infeasible path following algorithms for linear complementarity problems 3 Therefore and () 7? < 0; (0) = ku a v a k = 0: The quartic () has either one or three zeros in [0; [. Therefore the largest root of () = 0 in [0; ] exists and is characterized by (3.9). Finally, it is easily seen that the quadratic function (3.3) satises (0) 0, () < 0 so that the equation () = 0 has a unique root ^ ]0; [. Also () 0 ) () 0 which shows that (3.) is true, as required. We can prove now the following general result about the largest step infeasible path following algorithm. Theorem 3.7. Algorithm 3. is well dened and for any integer k 0 we have: (3.6) (3.7) (3.8) where (3.9) (x k ; s k ; k ) N ;? pn k k + pn k ; k = k 0 ; r k = k r 0 ; k+ = ky j=0 j : Proof. The fact that the algorithm is well dened follows from Lemma 3.6. The relation (x k ; s k ; k ) N follows from the choice of k. We have by denition k+ = k k, and it is easily veried that r k+ = k r k. This shows that (3.8) is satised. From the denition of the algorithm (see (3.)) we have j 0 = 0?j? so that (3.7) is satised for k = 0. Using the notation of Algorithm 3. and equation (3.6) we can write ] = ] + u T v=n: By using (3.8) and the fact that the orthogonal projection of uv onto span(e) is n? u T ve we deduce that ] kuvk n? (u T v)e = ju T vj p n : From the last two equations it follows that (3.7) is satised for all k 0. Let us note that in the above theorem we have not supposed that HLCP (.) has a solution. In what follows we will prove that if a solution exist then the sequence f k g converges to zero at a global Q-linear rate by showing that the quantity ^ from Lemma 3.6 has an upper bound strictly less than one. We rst have to nd bounds for the coecients (3.4). RR n445

16 4 J. Frederic Bonnans, Florian A. Potra Lemma 3.8. The coecients cc, ac and aa dened by (3.4) satisfy: s cc 3 p 8 ; ac p 8 (3.30) p 3 ; aa ; where (3.3) = n p :5 + ( + )(:5 + ( + )); (3.3) = p n((? )? + ) p =((? )) 0 ; with dened by (.5) and (3.33) 0 = min ( ^x 0 ^s 0? ^W? r 0 ; s 0 x 0?? W? r 0 ) : Proof. The rst inequality in (3.30) has already been veried in the proof of Lemma 3.6 (see (3.5)). In order to prove the last inequality in (3.30) we apply Lemma 3.5 with f =?Xs and g = r = b? Qx? Rs. Using (3.7) we deduce immediately that (3.34) = (XS) = e = p p n n : In order to obtain a bound for # we rst note that with the notation introduced in Proposition 3.4 we can write: (3.35) Let us denote ^y( ^W? r) = ^y( ^W? r 0 ) (xs)?= h ^s i 0 0 ^x ^W? r 0 p p h ^s i 0? ^x ^W? r 0 : ^W? r 0 = w 0 = w 0 w 0 : We have ^s w 0 ^s w 0 0??^x w 0 ^s^x 0 st x?^x 0 0? w 0 : In a similar manner we get ^x w 0 x T s?^s 0 0? w 0 ; INRIA

17 Infeasible path following algorithms for linear complementarity problems 5 and consequently we obtain h i ^s ^x ^W? r 0 ^s w 0 + ^x w 0 (s T x 0 + x T s 0 ) ^x 0 ^s 0? ^W? r 0 : From (3.36) and Theorem. we deduce that ^y( ^W? r) We can prove similarly that p ((? )? + ) =(? ) y( W? r) ((? )? + ) p =(? ) ^x 0 ^s 0 s 0 x 0? ^W? r 0 n p :?? W? r 0 n p ; and therefore we get (3.36) # p n : Finally, by substituting (3.34), (3.36) in (3.) we get aa which proves the last equation in (3.30). Let us note that according to Lemma 3.5 the above inequalities imply that (3.37) ky u a k + Y? v a p 8 : Similarly we have (see (3.5)): (3.38) Using the above inequalities we can write ky u c k + Y? v c 3 : ku c v a + u a v c k ky u c k jjy? v a k + ky u a k ky? v a k (ky u c k + ky? v c k ) = (ky u a k + ky? v a k ) = s p 8 p 3 ; where-from we deduce the required bound for ac. We can give now an upper bound for the parameters k generated by Algorithm 3.. Lemma 3.9. If S is nonempty then, with the notation introduced in Lemma 3.6, we have p 0 < b < e :=? 8 p : RR n445

18 6 J. Frederic Bonnans, Florian A. Potra Proof. Using (3.30) we have for any 0 < < Since n p :5 p :5 > :35 we infer Therefore () 3 p 8 + (? )p + (? )? (~) = 3 p 8? + (? ) p + (? ) : ~ =? p p :5 8 p? 8 p :35 :8 : 3 p 8? ~ + ~ ? < 0 : 64 Because is decreasing on [0; ] and (^) = 0 we deduce that ^ < ~, which completes the proof of our lemma. From the above lemma it follows that if the problem has p a solution, then the sequence f k g converges to zero at a global Q-linear rate less than or equal to? 8 p, i.e., (3.39) k+ (? p 8 p ) k: Using Theorem 3.7, this implies that our algorithm will produce an approximate solution in a nite number of steps. More precisely we have the following result. Theorem 3.0. If S is nonempty then Algorithm 3. nds a (x ; s ) S in at most K = 6 ln j ln(= 0 )j? p p 8 steps, where 0 = maxf(x 0 ) T s 0 ; kr 0 kg. We can obtain O( p nl)-iteration complexity in case the starting point is almost feasible. Corollary 3.. If S is nonempty and if there is a constant 0 independent of n such that then ( + ) 0 n?= 0 ; K = O( p n ln( 0 =)): 3 7 INRIA

19 Infeasible path following algorithms for linear complementarity problems 7 If we take starting points of the form (3.3) with L = ^L or L = L we obtain polynomiality if the points are \large enough". Corollary 3.. If S is nonempty and the starting points are chosen of the form (3.40) x 0 = ^ ^L e + ^ ^L e ; s 0 = ^ ^L e + ^ ^L e ; 0 = ^ ^ ; with for some (x ; s ) S, or of the form ^ n? k^x + ^s k ; ^ maxf^ k ^Mek ; k ^W? bk ; n? k^x + ^s k g; (3.4) x 0 = L e + L e ; s 0 = L e + L e ; 0 = ; with n? kx + s k ; maxf k Mek ; k W? bk ; n? kx + s k g; for some (x ; s ) S, then K = O(n ln( 0 =)): Proof. We give a proof only for the choice (3.40). For this choice we have n? (k^x + ^s k =^ + k^x + ^s k =^ ) and 0 ^x 0 ^s 0? ^W? r 0 = ^? ^W? b + ^ ^Me? e 3; and the claim follows from Lemma 3.8 and Theorem General Sucient Conditions for Convergence of the Iterates. Large variables and small variables. For the remaining of this paper we will assume the our HLCP (.) has a strictly complementary solution. It is well known that in this case there is a unique partition B [ N = f; ; ng; B \ N = ;; such that for any (x; s) S 0 we have ([x] i > 0; [s] i = 0; 8i B) and ([x] i = 0; [s] i > 0; 8i N ). This means that the \small" and \large" variables are invariant for any strictly complementary solution. Let us denote the corresponding partition of Q and R by Q = [Q B Q N ]; R = [R B R N ]: RR n445

20 8 J. Frederic Bonnans, Florian A. Potra Also, for any vector y IR n we denote by y B the vector of components [y] i ; i B; and by y N the vector of components [y] i ; i N. With this notation the constraint Qx + Rs = b can be rewritten as [Q B R N ]h xb sn i + [R B Q N ] h sb xn i = b: Following Bonnans and Gonzaga [] we rename the variables in the following sequence: Q [Q B R N ]; R [R B Q N ]; x h xb sn i ; s h sb xn i ; N ;; B f; : : :; ng: With this reordering, the solution set (also called optimal face) of (.) is written simply as (4.) S = f(x; s) IR n j s = 0; Qx = b; x 0g: The relative interior of the optimal face is composed by the set of all strictly complementary solutions, (4.) S 0 = f(x; s) IR n j s = 0; Qx = b; x > 0g: We note that after reordering Q; R remains a positive semidenite pair. In addition, the Newton directions as well as the neighbourhoods V and N of the the infeasible central path as dened in Section are invariant with respect to this transformation. This means that all algorithms based on the Newton step and those neighbourhoods of the infeasible central path (in particular GIPFA) are invariant with respect to permutation of variables. Of course, the algorithms never use the knowledge of the optimal partition, which is unknown: the algorithms are always dened in terms of the original problem and we assume that the optimal face is characterized by (4.) only when analyzing the algorithms. Therefore in the analysis we will always refer to x as the vector of large variables and to s as the vector of small variables. At a solution of our problem the small variables vanish. In the proof we need a pair (~x; ~s) such that (4.3) Q~x + R~s = a: Lemma 4.. If w = (x; s; ) V, then x and s. Proof. According to Theorem. we have x = O(), and s = O() and from the denition of V it follows that xs. Let (x; s) be a strictly complementary solution i.e., x > 0 and s = 0. Using and (4.3) we get It follows that 0 (x? x + ~x) T (s + ~s) i.e., Qx + Rs + a = b = Qx; Q(x? x + ~x) + R(s + ~s) = 0: (x? ~x) T s x T (s + ~s)? (x? ~x) T ~s: INRIA

21 Infeasible path following algorithms for linear complementarity problems 9 Set := min in x i. Whenever is small enough, we have x? ~x 0:5e, so that s (=)x T (s + ~s). By using xs and x = O() we deduce that s = O(). The conclusion follows by using again x = O() and xs. We want to study the limit-points of sequences f(x k ; s k ; k )g of elements of V when k! 0. They are related to the shifted analytic barrier function of the optimal face which is dened as follows. (4.4) ~s (x) :=? nx i= This is to be compared to the standard barrier function (4.5) (x) :=? log x i? ~s T x: nx i= log x i : We note that the shifted analytic barrier function depends on ~s, not only on a. However, for two dierent choices of ~s, the two corresponding barrier functions dier only by a xed constant on the set of all x having the same image through Q. This is a consequence of Proposition 3.3. We dene the shifted analytic center of S as the argument of the minimum of ~s over the relative interior S 0 of S. For future reference, we note that ~s is an analytical function with gradient (4.6) r ~s (x) =?x?? ~s: In addition, ~s is strictly convex so that the shifted analytic center of S is the unique point x that satises the optimality system r ~s (x ) R(Q T ); Qx = b; x > 0: Let us denote the projected gradient of the shifted analytic barrier function by (4.7) g(x) =?P Q (x? + ~s); where P Q denotes the orthogonal projection on N (Q). It follows that the shifted analytic center of S is the unique point x a such that (4.8) g(x a) = 0; Qx a = b; x a > 0: We note that from Proposition 3.3 it follows that x a depends only on a and not on the particular choice of ~s. Now we can state and prove a result concerning the asymptotic behavior of sequences of elements of V. Lemma 4.. Let w k = (x k ; s k ; k ) V for k = 0; ; : : :, and denote q k := xk s k k? e: RR n445

22 0 J. Frederic Bonnans, Florian A. Potra If k! 0 then there is a subsequence K IN such that, for some (x; q) IR n IR n lim kk (xk ; s k ; q k ) = (x; 0; q) and r ~s (x) + q x R(QT ): Proof. By Theorem. fw k g is bounded. According to (.0), the amount q k is also bounded. Extracting a convergent subsequence if necessary, we may assume that (x k ; s k ; q k )! (x; s; q). By virtue of Lemma 4., x > 0 and s = 0, hence Qx = b. Using and substituting b = Qx and a = Q~x + R~s, we obtain Qx k + Rs k + k a = b; Q(x k? x + k ~x) + R(s k + k ~s) = 0: By Proposition 3.3 we have s k + k ~s R(Q T ); or equivalently Passing to the limit, we get R(Q T ) 3 sk k + ~s = (x k )? + qk x k + ~s: x? + ~s + q x R(QT ); as desired. Corollary 4.3. If f(x k ; s k x ; k )g V with lim k! k = 0 and lim k s k k! toward the shifted analytic center of S. k = e; then x k converges Asymptotic study of the Newton step. Now we come back to the study of the Newton step with a generic right-hand side, namely (4.9) su + xv = f; Qu + Rv = r; We use the scaling vectors d = r x s ; = r xs : According to Lemma 4., d and whenever w = (x; s; ) V. We often use the relation d = x. The scaled equations are deduced from (4.9) by multiplying the rst equation by d x =, which results in d s x u + d v = df x = f? ; INRIA

23 Infeasible path following algorithms for linear complementarity problems and then using the denition of d to obtain, (4.0) d? u + d v = f? : Dening now the scaled variables and operators (4.) u := d? u ; v := d v ; (4.) Q := QD ; R := RD? ; we obtain the scaled Newton equations (4.3) u + v = f? ; Qu + Rv = r: The scaling used in the present section is advantageous when a strictly complementary solution exists and variables are permuted as in the previous section so that x and s are the big and small variables respectively. This will allow us to represent the solution of the original Newton equation (.5) as O() perturbations of quantities that are easily analyzed. In order to do so we represent the solution of the scaled equations (4.3) in terms of orthogonal projections. First we summarize some facts about orthogonal projections (for proofs see []). Given A IR mn ; q IR m, we dene the projection operators P A;q and P A by x 7! P A;q x = argminfkw? xk j Aw = qg; and P A = P A;0. Since P A is the orthogonal projection on N (A), and therefore a linear operator in IR n, the same notation will be used for its matrix representation. Similarly, we denote by PA ~ = I?P A the orthogonal projection on R(A T ) and the matrix representing it. Lemma 4.4. For any x IR n and q IR m, P A;q x = P A x + P A;q 0. Lemma 4.5. Let D IR n be such that d whenever d D. Then for y IR n, q R(A), d D P AD;q 0 = O(kqk); P AD;q y = O(kqk) + O(kyk); kdp AD Dyk kp A yk : Lemma 4.6. ([5]) Let g IR n be such that kg? ek ^, where ^ (0; ). Let p IR n ; set G = diag(g), and consider the projections ^h = P A p, h = gp AG gp. Then h? ^h ^( + ^)? ^ ^h :? ^ RR n445

24 J. Frederic Bonnans, Florian A. Potra Now we can prove the following result. Lemma 4.7. Consider the system (4.3). Let u; v be such that Qu + Rv = r. Then (4.4) u = P Q ( f?? d v ) + P Q;r?Rv 0; Proof. Let us write (4.3) as u? ( f?? d v ) = d v? v; Qu = r? Rv: From Q(u? u)+r(v? v) = 0 we deduce with Proposition 3.3 that v?v R(Q T ) hence d (v? v) = v?d v R(Q T ), i.e. u? ( f?? d v ) R(QT ); Qu = r? Rv: from which the result follows. Let us consider now the Newton equation (.3). The scaled system associated with (.3) is (4.5) u + v =?? ; Qu + Rv = (? )a: We consider again a pair (~x; ~s) satisfying (4.3). Applying the above results, we obtain the following important lemma: Lemma 4.8. If (x; s; ) V, then the solutions of the linear systems (.3) and (4.5) satisfy (i) v = v() = O() ; v = O(); (ii) u = u() = dp Q (? + d~s) + O(); v (iii) = d? [ PQ ~ (? + d~s)?? d~s] + O() Proof. Dene Then, by using (4.5) we get ^u := u? (? )d? ~x; ^v := v? (? )d ~s : ^u + ^v =??? (? )(d? ~x + d ~s ) = O(); Q^u + R^v = 0: INRIA

25 Infeasible path following algorithms for linear complementarity problems 3 By applying Lemma 3.5 with X = S = I, f = O() and r = 0 it follows that ^u and ^v. Coming back to the denition of ^u and ^v, we deduce that v = O(). Hence v = d?^v = O(), which proves (i). Now let us prove that (4.6) u = dp Q (? + d~s) + (? )dp Q;a 0? dp Q;Rv 0: We apply Lemma 4.7 with v = (? ) ~s ; f =?xs + e = (? +? ): Using the linearity of projections (see Lemma 4.4), and the fact that r = a, we get u = dp Q (? +?? (? )d~s) + (? )dp Q;a 0? dp Q;Rv 0: We obtain (4.6) if we can prove that P Q ( + d~s) = 0 i.e., (4.7) + d~s N (Q T )? = R(Q T ): As =d = s=, this boils down to s + ~s R(Q T ), which is a consequence of Q(x + ~x) + R(s + ~s) = b R(Q) and Proposition 3.3. Hence (4.6) is satised. By Lemma 4.5, dp Q;a 0 = O(kak) = O() and dp Q;Rv 0 = O(kvk) = O(), so that (ii) holds. Now, according to (4.5) we have and by using (ii) we get u + v =?? = (? + d~s)? ( + d~s); v = ~ PQ (? + d~s)? ( + d~s) + O(): As v = d? v, we obtain (iii). Let us prove now a very useful result about the new point produced by a step of GIPFA. Lemma 4.9. Given (x; s; ) V, let (u; v) be the solution of the Newton equation (.3) and dene the new point (x ] ; s ] ; ] ) as in (.4) and (.6). Then (i) (ii) (iii) x ] s ] = (? )xs + e + uv; uv = [ ~ P Q (? + d~s)?? d~s]p Q (? + d~s) + O( ); u T v = (? )~st dp Q (? + d~s) + O( ); RR n445

26 4 J. Frederic Bonnans, Florian A. Potra (iv) x ] s ] ]? e = Proof. Using (.3), we obtain?? + (xs? e) + uv? + : (x + u)(s + v) = xs + (xv + su) + uv; = (? )xs + e + uv: This proves (i). Relation (ii) follows from Lemma 4.8. We deduce that u T v =?( + d~s)t P Q (? + d~s) + O( ): Using (4.7), we get (iii). Relation (iv) is an easy consequence of (i). The shifted analytic barrier function associated with large variables. The direction of displacement for large variables is, up to O(), equal to l, with (4.8) l := dp Q (? + d~s): This vector l is, up to O(), equal to the centering step u c (see (3.)). If the variables are well centered, then u c is close to 0. It is easily seen that l = 0 if and only if Since d = x, this is equivalent to? + d~s R((Q) T ) = R(DQ T ): R(Q T ) 3 d? (? + d~s) = x? + ~s: The last expression is opposite to the gradient of the shifted barrier function ~s (see (4.6)). Therefore l = 0 if and only if g(x) = 0, where g is the projected gradient of the shifted analytic barrier function given by (4.7). We now interpret l as a perturbed Newton step for minimizing the shifted barrier function over some ane space. Consider the nonlinear problem parameterized by x: (4.9) min yir n ~s(y); Qy = Qx: As the Hessian of ~s at x is X?, the Newton step associated with this minimization problem is dened by (4.0) whose solution is characterized by min ^lir ^l T X?^l? ^lt (x? + ~s); Q^l = 0; n X?^l x? + ~s + R(Q T ); Q^l = 0: INRIA

27 Infeasible path following algorithms for linear complementarity problems 5 Writing the equivalent system X?^l x(x? + ~s) + R((QX) T ); QX(X?^l) = 0; we obtain the following expression for the Newton step (4.) ^l = xp QX (x(x? + ~s)) = xp QX (e + x~s): If we change the Hessian of problem (4.0) to D?, we obtain the modied problem whose optimality system is equivalent to min lir n lt D? l? l T (x? + ~s); Ql = 0; D? l d(x? + ~s) + R((QD) T ); QD(D? l) = 0: The unique primal solution of the latter system is given by (4.8). In short, l is the displacement obtained by applying a Newton-like step for minimizing ~s (x) with the Hessian being approximated by D?. The distance between the Newton-like step l and the exact Newton step ^l may be estimated using Lemma 4.6. Let us dene the proximity of the large variables x > 0 as the norm of x?^l, (the scaled Newton centering step for solving (4.9)): (x) := kp QX (e + x~s)k : The next lemma shows that the proximity is never larger than the usual measure of centering. Lemma 4.0. If (x; s) V, then (x) xs? e Proof. Because P QX is the orthogonal projection onto N (QX) we have kp QX (e + x~s)k = minfke + x~s? zk j z R(XQ T )g: By (4.7), R(XQ T ) 3 ( + d~s) = sx + x~s. Hence, from the relation above, kp QX(e + x~s)k e? xs. Now we can prove the following useful two Lemmas. Lemma 4.. Let l and ^l given by (4.8) and (4.), respectively. If (x; s) N, with 0:5, then ~s (x + l) ~s (x)? 0:06k^lk : Proof. The method of proof consists in comparing l and the Newton direction ^l. We have ~s (x + l)? ~s (x) = (x + l)? (x)? ~s T l = (e + x? l)? ~s T l: If we set h := x? l, ^h := x?^l, then (x + l)? (x)? ~s T l = (e + h)? ~s T l; RR n445

28 6 J. Frederic Bonnans, Florian A. Potra where is the (standard) barrier function (4.5). The quadratic approximation of the barrier function gives us the following property (shown for instance in [3]): if khk <, then and consequently (4.) (e + h)?e T h + khk + khk 3 3? khk ; ~s (x + l)? ~s (x)?(e + x~s) T h + khk + khk 3 3? khk : We want to apply Lemma 4.6 to h := x? l, ^h := x?^l, g := dx? =?, p = e + x~s, and A := QX. As (x; s) N,? e 0:5 and consequently :5e 0:75e. It follows that?:06e <?? e < :55e. Finally, by taking ^ = :55 in Lemma 4.6 we obtain h? ^h 0:4^h. By Lemma 4.0, ^h = (x) 0:5, so that khk ^h + kqk :4^h 0:35; where we have denoted q := h? ^h. Since khk < we can use (4.). We have (e + x~s) T h = (e + x~s) T P QX (e + x~s) = ^h, so that we obtain ~s (x + l)? ~s (x)? ^h ^h + kqk + ^h T q? (e + x~s) T q + + khk 3 3? khk : Let us prove that q N (QX). As q = h? ^h and ^h N (QX), we have to show that h N (QX). This follows immediately by writing (4.3) QXh = QX? P Q (? + d~s) = QP Q (? + d~s) = 0: Since ^h = P QX (e + x~s), we have ^h T q = (e + x~s) T q, and by using kqk 0:4k^hk we deduce ^h ~s (x + l)? ~s (x)? + kqk + khk 3 3? khk ; (? + 0:6 + khk khk 3? khk ^h Using again ^h 0:5 and khk :4^h 0:35, we obtain the conclusion. ) ^h Lemma 4.. If w = (x; s; ) V, then there exists > 0, > 0 such that : (4.4) ~s (x + l) ~s (x)? kg(x)k ; for all 0 < : INRIA

29 Infeasible path following algorithms for linear complementarity problems 7 Proof. In V, we have x. Since the Hessian of ~s is equal to X? it follows that there are constants M > 0 and > 0 such that Also, ~s (x + l)? ~s (x)?l T (x? + ~s) + M klk ; for all 0 < : l = DP QD D(x? + ~s); ^l = XPQX X(x? + ~s); g(x) = P Q (x? + ~s); so that we can apply Lemma 4.5 to deduce (4.5) l ^l g(x): From the denition of an orthogonal projection we have d? l?d(x? +~s) R(DQ T ): Hence d? l?(x? +~s) R(Q T ). From (4.3) it follows that l = xh N (Q), and therefore we can write l T (x? + ~s) = l T D? l: On the other hand, since d and l g(x), there exists > 0 and M > 0 such that If we dene then we can write for any 0, as was to be proved.?l T D? l? kg(x)k ; M klk M kg(x)k : := M ; := M ; ~s (x + l)? ~s (x)? kg(x)k + M kg(x)k =?? M kg(x)k?kg(x)k ; We end this section by stating and proving our main result. Theorem 4.3. Suppose that the linear complementarity problem (.) has a strictly complementary solution (i.e., S 0 is not empty) and that the sequence f k g produced by Algorithm. converges R-linearly to zero. Under this hypothesis there exists > 0 such that if for all suciently large k we have either w k N or P k k, then f(x k ; s k )g converges to a strictly complementary solution (x ; s ) S 0 of (.). Moreover if k=0 k k = + then (x ; s ) is the shifted analytic center (x a; s a) of the optimal face. Proof. Set l k := d k P QD k(( k )? + d k ~s): RR n445

30 8 J. Frederic Bonnans, Florian A. Potra By Lemma 4.8, we have u k = k k l k + O( k ): As l k is bounded in V, and k converges P P linearly to 0, we deduce that x k converges whenever k= k k < +. It remains to prove that if k= k k = +, then x k converges to the shifted analytic center of the optimal face. By Lemma 4., Lemma 4., the fact that l k g(x k ), and the hypothesis of our theorem we deduce that there is a constant e such that ~s (x k + k k l k ) ~s (x k )? k k e kg(x k )k for all suciently large k. The shifted analytic barrier ~s is Lipschitz continuous in V, so that there exists a constant f M such that (4.6) ~s (x k+ ) ~s (x k )? k k e kg(x k )k + f Mk : Since k converges R-linearly, and the shifted barrier function is bounded from below on the compact set V, we obtain X k= k k kg(x k )k < : Since P k= k k = there is a subsequence fx k g kk such that lim kk kg(x k )k = 0. The set V is compact and therefore we may assume without loss of generality that fx k g kk is convergent. Its limit clearly veries (4.8), so that we can write lim kk xk = x a: Using (.9) and lim kk ~s (x k ) = ~s (x a ) we deduce that for any " > 0, there exists k0 IN such that By virtue of (4.6) we get ~s (x k0 ) < ~s (x a) + " and X k=k 0 k < ": ~s (x k0 +j ) ~s (x k0 ) + f M X k=k 0 k ~s (x a ) + ( f M + )"; 8j IN:, i.e. lim supf ~s (x k )g ~s (x a ). On the other hand, any limit-point x of fx k g satises x > 0 and ~s (x ~s (x a ). It follows that ~s(x k )! ~s (x a ), which by the compactness of V implies the convergence of x k towards x a. INRIA

31 Infeasible path following algorithms for linear complementarity problems 9 5. Asymptotic convergence analysis for the largest step infeasible path following algorithm. In what follows we rst use Theorem 4.3 to deduce that the sequence fw k g produced by the largest step infeasible path following algorithm converges. Then it is easy to to prove that the sequences f r k g and f(x k ) T s k g measuring feasibility and optimality are superlinearly convergent to zero. We shall end the paper by showing that the largest step infeasible path following algorithm can be modied in such a way that the latter sequences are quadratically convergent. Theorem 5.. Assume that the set of strictly complementary solutions S 0 is nonempty and that 0:5. Then the sequence f(x k ; s k ; k )g produced by Algorithm 3. converges. Moreover the sequence f k g converges to zero Q-superlinearly, and consequently the sequences f r k g and f(x k ) T s k g also converge Q- superlinearly to zero. Proof. We assume that the variables have been permuted such that x contains the large variables and s contains the small variables. From Lemma 4. we have s k k and therefore according to (3.39) it follows that the small variables converge to zero, s k! 0. We know by Theorem 4.3 that fx k g also converges, x k! x. Hence u k! 0. It follows that = k xk+ s k+ k k? ek = k uk v k k k k = o( vk k k ): As v k k, we deduce that = o( k ), which implies k! 0. Since k = k+ = k, this means that k! 0 Q- superlinearly. From (3.8) it follows that r k converges Q-superlinearly to zero as well (even component-wise!). Finally, from (3.7) we deduce that k+ k + pn =? p k+ ; n k which shows that f(x k ) T s k g also converges to zero Q-superlinearly. We now consider a variant of this algorithm in which a safeguard is added so as to obtain quadratic convergence. Our basic tool is Lemma 5.4, due (in the framework of feasible algorithms) to Gonzaga [4]. This Lemma gives a precise estimate of k xs()? ek, which is itself strongly related to the centering of the large variable. We dene ] () as the proximity obtained at the point (x(); s(); ), i.e. ] () := k x()s()? ek: Algorithm 5.. Data: 0 < :5, > 0, 0 (0;? p n ), (x0 ; s 0 ; 0 ) N, where (5.7) 0 = 0 = = (x 0 ) T s 0 =(n); = + = p n: k := 0 repeat x := x k ; s := s k ; := k ; RR n445

32 30 J. Frederic Bonnans, Florian A. Potra Solve the linear systems (3.) and (3.) ; Compute the largest root of the quartic (3.8) from the interval ]0; [ ; Safeguard If ] (0:) 0:, then set k :=, else k := max( 0 ; ) ; x k+ := x ] ; s k+ := s ] ; k+ := ], according to (3.0) and (3.5) ; k := k +. Theorem 5.3. Assume that the set of strictly complementary solutions S 0 is nonempty and that 0:5. Then the sequence f(x k ; s k ; k )g produced by Algorithm 5. converges, and the safeguard is activated only a nite number of times. The complexity results stated in Corollary 3. and Corollary 3. hold also for Algorithm 5.. Moreover the sequence f k g converges to zero quadratically, and consequently the sequences f r k g and f(x k ) T s k g also converge quadratically to zero. Here is a general idea of the proof. We have to prove that = O(). We rst prove the following Lemma, that establishes a strong connection between the proximity at a new point and the amount k xs()? ek. In the proof of the Theorem, we show that if the large variables are suciently well centered, then k xs()? ek is small enough to obtain = O() using (5.8). Now if the safeguard was activated an innite number of times, the sequences of points would converge to the analytical center by Theorem 4.3, and we would get a contradiction using (5.8). Knowing that the safeguard is not active after a nite number of steps, we deduce with (5.9) an estimate of k xs()? ek that allows to check with (5.8) that = O(), as desired. The Lemma below was proved by [4] in the framework of feasible algorithms. Although the proof needs no modication, we give it for the self-containedness of the paper. Lemma 5.4. Consider ; w N, (0; ]. Then (5.8) ] () (? )k xs()? ek + ] () + O( ) ; (5.9) ] () (? )k xs()? ek? ] () + O( ) : Proof. By Lemma 4.8, we have x(0) = x + u(0) = x + O() and s(0) = s + v(0) = s + d? + O( ) = O( ). it follows that Multiplying these expressions, we obtain (5.30) x() = x() + (? )x(0) = x() + (? )x + O() ; s() = s() + (? )s(0) = s() + O( ) : x()s() = x()s() + (? ) xs() The result follows by subtracting e = e + (? )e and taking norms. + O( ): Proof of the Theorem We rst prove that the safeguard cannot be activated an innite number of times. Indeed, if the safeguard was activated an innite number of times, then P k= k k =, as k = at all INRIA