ON A CLASS OF SUPERLINEARLY CONVERGENT POLYNOMIAL TIME INTERIOR POINT METHODS FOR SUFFICIENT LCP

Transcription

1 ON A CLASS OF SUPERLINEARLY CONVERGENT POLYNOMIAL TIME INTERIOR POINT METHODS FOR SUFFICIENT LCP FLORIAN A POTRA AND JOSEF STOER Abstract A new class of infeasible interior point ethods for solving sufficient linear copleentarity probles requiring one atrix factorization and backsolves at each iteration is proposed and analyzed The algoriths fro this class use a large N neighborhood of an infeasible central path associated with the copleentarity proble and an initial positive, but not necessarily feasible, starting point The Q-order of convergence of the copleentarity gap, the residual, and the iteration sequence is + for probles that adit a strict copleentarity solution and + / for general sufficient linear copleentarity probles The ethods do not depend on the handicap κ of the sufficient LCP If the starting point is feasible or alost feasible the proposed algoriths have O + κ + log + κ n L iteration coplexity, while if the starting point is large enough the iteration coplexity is O + κ +/ + log + κ n L Key words linear copleentarity, interior point, affine scaling, large neighborhood, superlinear convergence AMS subject classifications 90C5, 65K05, 49M5, 90C05, 90C0 Introduction Interior point ethods have provided the first polynoialtie algoriths for solving linear prograing LP and other classes of convex optiization probles Nuerical experients perfored over the past two decades show that the ost efficient interior point ethods for LP are prial-dual path following interior point ethods The fact that these ethods, as opposed to the ellipsoid ethod, perfor uch better in practice than indicated by the worst case coputational coplexity bounds, is explained in part by their superlinear convergence The first interior point ethod having both polynoial coplexity and superlinear convergence was the predictor-corrector ethod of Mizuno, Todd and Ye MTY This ethod was proposed for LP in [6], where it was shown to have O nl iteration coplexity Shortly after that, Ye et al [45], and independently Mehrotra [4], proved that the duality gap of the iterates produced by MTY converges quadratically to zero MTY was generalized to LCP in [9], and the resulting algorith was proved to have O nl iteration coplexity under general conditions, and superlinear convergence under the assuption that the LCP has a perhaps not unique strictly copleentary solution ie, the LCP is nondegenerate and the iteration sequence converges Fro [3] it follows that the latter assuption always holds Subsequently Ye and Anstreicher [44] proved that MTY converges quadratically assuing only that the LCP is nondegenerate The nondegeneracy assuption is not restrictive since according to [7] a large class of interior point ethods, which contains MTY, can have only linear convergence if this assuption is violated In [9, 4, 6, 45, 44] one assues that the starting point for the MTY algorith is strictly feasible A generalization of the MTY algorith for infeasible starting points was proposed in [, 0] for LP, and in [] for onotone LCP The proposed algorith requires two atrix factorizations and at ost three backsolves per iteration Its coputational coplexity depends Departent of Matheatics and Statistics, University of Maryland Baltiore County, 000 Hilltop Circle, Baltiore, MD 50, USA potra@athubcedu The work of this author was supported by the National Science Foundation under Grants No 03970, , and by the National Institute of Health under Grant No R0GM Institut für Angewandte Matheatik und Statistik Universität Würzburg A Hubland D Würzburg GERMANY jstoer@atheatikuni-wuerzburgde

2 FLORIAN POTRA AND JOSEF STOER on the quality of the starting point If the starting point is large enough, then the algorith has OnL iteration coplexity If a certain easure of feasibility at the starting point is sall enough, then the algorith has O nl iteration coplexity At each iteration, both feasibility and optiality are reduced exactly at the sae rate Moreover, the algorith is quadratically convergent for probles having a strictly copleentary solution The generalizations of the MTY algorith for LCP with infeasible starting points fro [0, 33] require two atrix factorizations and two backsolves per iteration and have the sae properties According to Ostrowski [8], the asyptotic efficiency index is defined as ord /c, where ord is the Q-order of the iterative procedure and c is the cost of an iteration Since the ain cost associated with interior point ethods is represented by atrix factorizations, it follows that the asyptotic efficiency index of the MTY type algoriths entioned above is equal to The proble arises then to find a superlinearly convergent interior point ethod with polynoial coplexity that requires only one atrix factorization per step The first result of this type was obtained by McShane [3] who proved that the so-called largest step path-following ethod LSPF has O nl iteration coplexity for general onotone LCPs and superlinear convergence under the assuption that the LCP is nondegenerate and the iteration sequence converges No results about the Q-order were given Gonzaga [8] proved superlinear convergence assuing only nondegeneracy, and showed that with the addition of a coputationally trivial safeguard LSPF achieves Q-quadratic convergence We ention that the convergence of the iteration sequence generated by LSPF follows fro the general results of Bonnans and Gonzaga [3] The results of [3, 8] were extended for the infeasible case in [4] An infeasible interior point ethod with polynoial and quadratic convergence for nondegenerate LCP that uses only one atrix factorization per iteration was proposed by Wright [4] It is based on the so-called fast step safe step strategy At each iteration the algorith first tries to take a fast step along the prial-dual affine scaling direction If the fast step fails to reduce the copleentarity gap by a given factor, then the algorith reverts to a safe - step along a direction consisting of a convex cobination of the affine-scaling direction and the centering direction Higher order variants of this strategy are given in [9, 43] The fast step safe step strategy is also used in a variant of the Mehrotra predictor-corrector ethod proposed by Zhang and Zhang [46], which has polynoial coplexity, superlinear convergence, and requires one atrix factorization per step The first superlinear convergence result for degenerate LCP was obtained by Mizuno [5], who used the so-called step variant of the Tapia indicator [7] to identify the variables that are not strictly copleentary, and odified the MTY algorith in order to accelerate the convergence to zero of those variables The resulting algorith has Q-order 5 Mizuno s result was refined in [30, 3, 3] Subsequently, by using second order derivatives of weighted central paths, Stur [40] obtained a MTY type algorith of order 5 for degenerate LCP His approach was generalized in [39], where th order derivatives were used to construct MTY type algoriths with Q-order + for nondegenerate LCPs and + / for degenerate LCPs The coplexity of the predictor-corrector algorith for degenerate LCPs fro [39] is analyzed in [36] The algoriths fro [36, 39] require two atrix factorizations and + backsolves per iteration The algorith proposed in [35] requires only one atrix factorization and backsolves per iterations and has Q-order in the nondegenerate case, and / in the degenerate case The results of [35] were iproved in the unpublished technical report [34], where an algorith with the sae coputational cost is shown

3 INTERIOR POINT METHODS FOR SUFFICIENT LCP 3 to have Q-order + in the nondegenerate case and +/ in the degenerate case No coputational coplexity results are available for the algoriths of [34, 35] We ention that the algoriths of [36, 39] use a sall N neighborhood of the central path while the algoriths of [34, 35] use a large N neighborhood of the central path Algoriths using large neighborhoods of the central path are ore difficult to analyze and, in general, their coputational coplexity estiates are worse than the corresponding estiates for algoriths using sall neighborhoods In fact this is one of the paradoxes of interior point ethods, since it is known that the best practical results are achieved by interior point ethods acting in a large neighborhood of the central path For any years this fact has been accepted as an inherent difference between theory and practice However, recent research has shown that it is possible to design superlinearly convergent interior point ethods in the large neighborhood of the central path that have optial, or close to optial, coputational coplexity In [, 9] one iproves the coplexity by using new large neighborhoods of the central path and first order inforation, while in [4, 6] we use the classical N large neighborhood and the coplexity is reduced by the use of higher order inforation The latter approach ensures superlinear convergence even in the absence of strict copleentarity The algoriths of [4, 6], require two atrix factorizations and + backsolves per iteration and have Q-order + in the nondegenerate case and + / in the degenerate case These algoriths have been generalized for sufficient LCP in [, 8] The handicap κ 0 of a sufficient LCP easures by how uch the LCP differs fro a onotone LCP see the next section Since there are no polynoial coplexity results for the sufficient LCP, the iteration coplexity of interior point ethods for such probles will depend on the handicap κ However the interior point algoriths that do not depend explicitly on κ can be applied to any sufficient LCP We ention that the algoriths fro [36, 39] depend on κ, while the algorith fro [, 34, 35] and algoriths A and A fro [8] do not depend on κ In a recent paper [7], the first author has analyzed several feasible interior point ethods for the onotone LCP acting in the N neighborhood of the central path that require one atrix factorization and backsolves per iteration and have Q-order + in the nondegenerate case and + / in the degenerate case, the sae as the algorith proposed by the second author in [34] The algoriths fro [7] and [34] share the property that the N neighborhood of the central path is expanded at each iteration The rate of expansion used in [34] ensures the high Q-order results entioned above, but does not guarantee polynoial coplexity However by using the rate of expansion fro [7] it is possible to odify the ethod of [34] in such a way that Q-order results are retained and polynoial coplexity is guaranteed In the present paper we propose a class of polynoial tie infeasible interior point ethods for sufficient LCP that have the sae Q-order and coputational cost per iteration as the ethods fro [34] and [7] ie, the sae asyptotic efficiency index The class depends on a paraeter σ 0 When σ = 0 the algoriths fro this class reduce to the corresponding algoriths fro [7], while for σ > 0 they reduce to a variant of the algorith of [34] The ain contribution of the paper is that it generalizes the results of [7] to sufficient LCP with infeasible starting points and it proposes a variant of the algorith of [34] that has both superlinear convergence and polynoial coplexity Moreover, we hope that our analysis will also contribute to the better understanding of the behavior of interior point ethods in the large neighborhood of the central path The interior point ethods to be presented in this

4 4 FLORIAN POTRA AND JOSEF STOER paper do not depend on the handicap κ of the sufficient LCP Their coputational coplexity depends on n,, and κ More precisely we will show that an approxiate solution having both the copleentarity gap and the nor of the residual less than ε can be obtained in at ost Olog+κ χ+ +χ + n L +ν χ+ + +κ n +χ L + iterations, where L = logε 0 /ε, < ν is a paraeter fro the definition of the algorith, χ = 0 if the starting point is feasible or alost feasible and χ = if the starting point is large enough If we take ax {log n, L, } then the Algorith has O + log+κ +χ+ + κ n +χ L iteration coplexity Since the aount of work per iteration of an th order ethod On 3 + n arithetic operations is coparable, for sall /n, to that of a first-order ethod On 3 arithetic operations, we consider that it is justified to copare all ethods by eans of their iteration coplexity The paper is organized as follows In Section we present soe basic results on the sufficient horizontal linear copleentarity proble and the analyticity of its weighted infeasible central paths In Section 3 we analyze a class of interior point ethods based on the first derivatives of the weighted central paths We prove that the ethods are globally and superlinearly convergent under general assuptions We also prove that they have polynoial coplexity for starting points that are either alost feasible or large enough The paper ends with a very short section of conclusions Conventions We denote by IN the set of all nonnegative integers IR, IR +, IR ++ denote the set of real, nonnegative real, and positive real nubers respectively Given a vector x, the corresponding upper case sybol denotes, as usual, the diagonal atrix X defined by the vector The sybol e represents the vector of all ones, with diension given by the context We denote coponent-wise operations on vectors by the usual notations for real nubers Thus, given two vectors u, v of the sae diension, uv, u/v, etc will denote the vectors with coponents u i v i, u i /v i, etc This notation is consistent as long as coponent-wise operations always have precedence in relation to atrix operations Note that uv Uv and if A is a atrix, then Auv AUv, but in general Auv Auv Also if f is a scalar function and v is a vector, then fv denotes the vector with coponents fv i For exaple if v IR + n and λ IR, then v denotes the vector with coponents v i, and λ v denotes the vector with coponents λ v i Traditionally the vector λ v is written as λe v, where e is the vector of all ones Inequalities are to be understood in a siilar fashion For exaple if v IR n, then v 3 eans that v i 3, i =,, n Traditionally this is written as v 3 e If is a vector nor on IR n and A is a atrix, then the operator nor induced by is defined by A = ax{ Ax ; x = } As a particular case we note that if U is the diagonal atrix defined by the vector u, then U = u If x, s IR n, then the vector z IR n obtained by concatenating x and s is denoted by z = x, s = [ x T, s T ] T, and the ean value of xs is denoted by µz = x T s n The sufficient horizontal linear copleentarity proble Given two atrices Q, R IR n n, and a vector b IR n, the horizontal linear copleentarity proble HLCP consists in finding a pair of vectors z = x, s such that

5 INTERIOR POINT METHODS FOR SUFFICIENT LCP 5 xs = 0 Qx + Rs = b x, s 0 Let us denote by Φ the null space of the atrix [Q R] IR n n Φ := N [Q R] = { u, v : Qu + Rv = 0} and by Φ its orthogonal space 3 Φ = { u, v : u = Q T x, v = R T x, for soe x IR n} We say that the pair Q, R is colun sufficient if and row sufficient if u, v Φ, uv 0 iplies uv = 0, u, v Φ, uv 0 iplies uv = 0 Q, R is called a sufficient pair if it is both colun and row sufficient, and in this case is called a sufficient HLCP The standard sufficient linear copleentarity proble LCP is obtained by taking R = I, and Q a sufficient atrix in the sense of Cottle et al in [6] see also [5] The notion of a sufficient atrix is a far reaching generalization of the notion of a positive seidefinite atrix The notion of a sufficient pair was introduced in [37, 39] It turns out that Q, R is a sufficient pair if and only if for any b, the HLCP has a convex perhaps epty solution set and every KKT point of in x,s st x T s Qx + Rs = b x, s 0 is a solution of By using the result of Väliaho [4] and the equivalence results fro [] see also [35] it follows that Q, R is a sufficient pair if and only if there is a κ 0 such that Qu + Rv = 0 iplies + 4κ 4 u i v i + u i v i 0, u, v IR n, i I + i I where I + = {i : u i v i > 0} and I = {i : u i v i < 0} If the above condition is satisfied we say that Q, R is a P κ pair and we write Q, R P κ In case R = I, Q, I is a P κ pair if and only if Q is a P κ atrix in the sense that: + 4κ x i [Qx] i + x i [Qx] i 0, x IR n, i Î+ i Î where Î+ = {i : x i [Qx] i > 0} and Î = {i : x i [Qx] i < 0} Proble is then called a P κ LCP and it is extensively discussed in [] Thus the class of sufficient pairs coincides with the class P = κ 0 P κ In view of this equivalence the ters P pair, sufficient pair, P HLCP, and sufficient HLCP will be used interchangeably,

6 6 FLORIAN POTRA AND JOSEF STOER although as illustrated by the title of our paper preference will be given to the latter The following result was proved in []: Lea If Q, R IR n n are two atrices such that the pair Q, R is colun sufficient, then the atrix [Q R] has full rank An exaple was also given showing that row sufficiency does not iply the full rank property The full rank property is used in proving the existence of weighted infeasible central paths defined by nonlinear systes of the for 5 xs = τp Qx + Rs = b τb A precise stateent of this fact will be given later For the tie being, let us note that for τ = 0 5 reduces to We denote the set of all feasible points of by F = {z = x, s IR n + : Qx + Rs = b}, and the solution set or the optial face of HLCP by F = {z = x, s F : x s = 0} The structure of F is very iportant in the analysis of interior point ethods Let us define three subsets B, N and J of the index set {,, n} by B = {i =,, n x i > 0 for at least one x, s F }, N = {i =,, n s i > 0 for at least one x, s F }, J = {i =,, n x i = s i = 0 for all x, s F } One can prove that B, N, and J for a partition of {,, n} and that there exists a solution x, s F such that x B > 0 and s N > 0 Such a solution is called a axial copleentarity solution, since one can prove that for any x, s F, x i > 0 i B and s j > 0 j N If the solution of the HLCP is unique, then it is a axial copleentarity solution Otherwise it can be shown that the relative interior of F is coposed of axial copleentarity solutions If the set J is epty then a axial copleentarity solution is called a strictly copleentary solution Let us denote by F c the set of all such solutions, ie, F c = {z = x, s F : x + s > 0} We say that the HLCP is nondegenerate if it has a strictly copleentary solution If the set J is nonepty, then we say that the HLCP is degenerate If HLCP is a reforulation of a linear prograing proble then it is known that it has a strictly copleentary solution In general it is very difficult to establish whether HLCP has a strictly copleentary solution However, if this is the case the algorith should take advantage of this inforation In order to treat the two cases in a unified anner we define {, in general default option 6 ϑ = 0, if HLCP is known to be nondegenerate In the theory of interior point ethods it is often assued that the set F 0 = F IR n ++,

7 INTERIOR POINT METHODS FOR SUFFICIENT LCP 7 called the set of strictly feasible points, is nonepty In the present paper we are not aking this assuption Instead we assue the noneptyness of the set 7 F b,τ := {z = x, s IR n ++ : Qx + Rs = b τb}, which is related to the weighted infeasible central path defined by 5, at τ = τ 0 The distinction between nondegenerate probles ϑ = 0 and perhaps degenerate probles ϑ = plays a role in the following theore: Theore Assue that the sufficient HLCP has a solution and that the set 7 is nonepty for soe τ = τ 0 > 0 and b IR n Then 5 has a unique positive solution zτ, p = xτ, p, sτ, p for any τ, p 0, τ 0 ] IR ++ n and the following holds: The function zq, p := zq +ϑ, p is analytic on 0, q 0 ] IR ++, n 0 < q 0 := τ /+ϑ 0, and can be analytically extended to an open neighborhood of [0, q 0 ] IR ++ n For any copact set K IR ++ n and any integer i IN there are constants ck, i such that i zq, p q i ck, i, q [0, q 0 ], p K, i = 0,,, The above theore was first proved in [38] for τ 0 = under the additional assuption that the atrix [Q R] has full rank According to Lea this assuption is autoatically satisfied For any p IR ++, n τ 0, τ 0 ], and σ IR, we consider the function 8 ẑt, τ, σ, p := zt, P t, τ, σ, p, where P t, τ, σ, p := p + στ te p Since P t, τ, σ, p is obviously analytic in t, τ, σ, p, it follows fro Theore that the function 9 ẑq +ϑ, τ, σ, p, t = q +ϑ, is a well defined analytic function of q whenever P t, τ, σ, p > 0, and ˆx, ŝ := ẑt, τ, σ, p can be coputed as the unique positive solution of the following nonlinear syste: 0 ˆxŝ = q +ϑ P q +ϑ, τ, σ, p Qx + Rs = b q +ϑ b In this paper we will analyze an interior point ethod based on the derivatives of 9 with respect to q up to an arbitrary order The paraeter σ 0 can be interpreted as a centering paraeter: The size of σ deterines how uch the path ẑt, τ, σ, p is tilted towards the central path as t decreases 3 3 Infeasible interior point ethods based on th order derivatives 3 The algorith By setting p = e in 5 we obtain xs = τe Qx + Rs = b τb The curve z = zτ, e defined by this nonlinear syste was called in [4] the infeasible central path pinned on b Such a path can be associated with any infeasible starting point z 0 = x 0, s 0 IR ++, n by taking 3 b := r0, r 0 := Qx 0 + Rs 0 b, τ 0 := µ 0 := µz 0 = x0 T s 0 τ 0 n

8 8 FLORIAN POTRA AND JOSEF STOER If the starting point is feasible, then b = 0 and the curve z = zτ, e coincides with the classical prial-dual central path whose liit point z0, e is the analytic center of the optial face F If b 0 then the liit point z0, e, which exists by virtue of Theore, is the so-called shifted analytic center of the optial face see [4, Section 4] More generally, given any σ 0, any point z := x, s IR n ++, and setting τ := µz = x T s/n, we consider in this paper the weighted infeasible interior point path see 8 ẑt, τ, σ, xs/τ, t τ, starting for t = τ at ẑτ, τ, σ, xs/τ = z, and given by the solution ˆx, ŝ := ẑq +ϑ, τ, σ, xs/τ of 0 for t = q +ϑ The infeasible interior point ethods to be presented in the present paper follow this weighted infeasible interior point path by generating a sequence of points belonging to the following large neighborhood of this central path 33 Dβ = N β = {z = x, s IR n ++ : xs βµze } The initial value for β is β 0 At each iteration we set β k+ = β k α k, where α k is a given onotone sequence of nonnegative nubers satisfying the property 34 α k β 0 β, β > 0 k=0 At the start of a typical iteration of our algorith we have a point z, τ such that z Dβ F b,τ and we consider the apping 8 with p = xs/τ Since our point z = x, s verifies 0 for p = xs/τ and t = τ = µz, it follows that z = ẑτ, τ, σ, xs/τ We wish to approxiate ẑt, τ, σ, xs/τ around t = τ by the th order Taylor polynoial around θ = 0 of the function θ ẑ θτ /+ϑ, τ, σ, xs/τ This Taylor polynoial has the for 35 zθ = xθ, sθ = z + θ i w i, w i = u i, v i, i= where the vectors w i are given by 36 w i = u i, v i := i τ i i! i τ i/ i! i t i ẑ t, τ, σ, xs/τ t=τ, if ϑ = 0, i q i ẑ q, τ, σ, xs/τ q= τ, if ϑ = It is easily seen that the vectors 36 can be obtained by solving the following systes of linear equations: 37 { su + xv = + ϑ στ e + στxs Qu + Rv, = + ϑτb { su + xv = ϑxs + + 4ϑστxs τe u v Qu + Rv = ϑτb { su i + xv i = ϑd i i j= uj v i j Qu i + Rv i = 0 i = 3, 4,,, where d 4 = στxs τe, d 3 = 4d 4, d 5 = d 6 = = 0,,

9 INTERIOR POINT METHODS FOR SUFFICIENT LCP 9 The linear systes above have the sae atrix, so that their nuerical solution requires only one atrix factorization and backsolves This involves On 3 + n arithetic operations For ϑ = 0 we consider the ost general case of arbitrary, but for ϑ = only the case, and we denote by T := {ϑ,, ϑ {0, }, } the set of these pairs We deduce that xθsθ = θ +ϑ xs στθϕ ϑ, θxs τe + µθ = θ +ϑ µ στθϕ ϑ, θµ τ + i=+ i=+ θ i h i, θ i e T h i /n, where ϕ 0, θ =, ϕ 0, θ = θ, =, 3, 30 ϕ, θ = 5θ, ϕ 3, θ = 5θ + 4θ, ϕ, θ = θ θ, 4, h i = u j v i j, i = +, +, j=i For the excluded exceptional case ϑ =, = we only note that the structure of the above forulae changes: xθsθ = θxs θστxs τe + θ h, µθ = θµ θστµ τ + θ e T h /n The quantity µ is a easure of optiality while, as indicated by 7, the quantity τ is a easure of feasibility The stepsize along the direction w is deterined such that two different objectives are achieved First, we would like to keep the iterates in a neighborhood that is expanded in a controlled anner at each iteration Therefore we define [ θ = ax { θ [0, ] : zθ Dβ+, θ 0, θ ]} 3, where β + = β α, and α > 0 is a paraeter that changes fro iteration to iteration the α s will be chosen fro a sequence satisfying 34 Second, we would like to have τ/µ ± bounded We will achieve this by defining 3 θ = ax θ ax θ { θ : γ α } µθ, 0 θ θ θ +ϑ µ γβ0 β+, if τ µ { θ : γ β + β 0 µθ θ +ϑ µ γα, 0 θ θ }, if τ > µ where γ is soe constant satisfying 0 < γ < Finally, we denote, 33 θ = in { θ, θ },

10 0 FLORIAN POTRA AND JOSEF STOER and define the new iterate by 34 z +, τ + = zθ, θ +ϑ τ Note that in the forula for τ + the exceptional case ϑ = = is excluded Since z + Dβ + F b,τ+, the process can be repeated It follows that all the points produced by the following algorith will stay in Dβ The coputation of the exact value of θ for is coplicated since it involves the solution of a syste of polynoial inequalities of order in θ Good lower bounds of the exact solution can be obtained by a line search procedure In the proof of global convergence we will give siple lower bounds for θ In order to siplify the presentation in the following algorith we assue that the exact value of θ is available Algorith Given paraeters 0 < β < β 0 <, 0 < γ <, ϑ {0, },, ϑ,,, a sequence α k satisfying 34, and a starting point z 0 Dβ 0 : Consider the notation fro 3; Set k 0 ; repeat Set z z k, τ τ k, α α k, β β k, β + β α ; Choose a scalar σ [ 0, in { }], γ β0 β /τ 0 ; Copute directions w i = u i, v i fro 37; Copute steplength θ fro 3, 3, 33; Copute z +, τ + fro 34; Set θ k θ, z k+ z +, τ k+ τ +, β k+ β + ; Set k k + continue The algorith is well defined because of Theore 3 If HLCP is sufficient then the Algorith is well defined It generates an iteration sequence satisfying the following properties z k = x k, s k Dβ k Dβ ; Qx k + Rs k = b + τ k τ 0 r 0 ; τ k+ = θ k +ϑ τ k ; r k+ = θ k +ϑ r k, where r k = Qx k + Rs k b ; τ k /γ γ β k β 0 τ k µ k γ β0 β k τ k γ τ k Proof We use induction and prove first r k = τ k /τ 0 r 0 = τ k b, for k = 0,, This property is trivially satisfied for k = 0 Assue it is satisfied for soe k 0 and denote z = z k, r = r k, τ = τ k Since we excluded the case ϑ =, =, we can write rθ := Qxθ + Rsθ b = r + θ Qu + Rv + θ Qu + Rv + θ i Qu i + Rv i i=3 = τb + θ + ϑτb θ ϑτb = θ +ϑ τb =: τθb,

11 INTERIOR POINT METHODS FOR SUFFICIENT LCP which shows that r k+ = τ k+ /τ 0 r 0 = τ k+ b Also the last property is easily proved by induction Indeed, since µ 0 = τ 0 the property is verified for k = 0 Let us assue that it is verified for a given k If τ k µ k then by the induction hypothesis and 3 we have γ β k+ β 0 τ k+ = γ β k+ β 0 θ k +ϑ τ k γ α k θ k +ϑ µ k µ k+ γ β0 β k+ θ k +ϑ µ k γ β0 β k+ θ k +ϑ τ k = γ β0 β k+ τ k+ On the other hand, if τ k > µ k then γ β k+ β 0 τ k+ = γ β k+ β 0 θ k +ϑ τ k > γ β k+ β 0 θ k +ϑ µ k µ k+ γ α k θ k +ϑ µ k γ β0 β k+ θ k +ϑ τ k = γ β0 β k+ τ k+ Reark The algorith is also well defined for an iproper choice of ϑ violating 6, ie, when ϑ = 0 for a degenerate proble Such an iproper choice would not prevent the convergence of the algorith but only its superlinear convergence see Theores 39 and 3 The behavior of the iteration sequence z k strongly depends on the choice of the sequence α k Before specifying different choices for α k, we need to give soe technical results that will be used in the analysis of the Algorith 3 Technical Results We start by finding bounds for the solution of a linear syste of the for { su + xv = a 35 Qu + Rv = 0 By using the notation 36 D = X / S /, u, v z := Du + D v, ã = xs / a, we obtain the following result Lea 3 If HLCP is P κ, then for any z = x, s IR ++ n and any a IR n the linear syste 35 has a unique solution w = u, v for which the following estiate holds w z + κ ã Using this lea and the fact that the functional z is a nor on IR n see [5] for soe properties of this nor we can prove the following leas which establish bounds for the solutions of a linear syste of the for { su + xv = a 37 Qu + Rv = b Lea 33 If HLCP is P κ, then for any z = x, s IR ++ n and any a, b IR n the linear syste 37 has a unique solution w = u, v and the following inequalities are satisfied: w z + κ ã κ ζz, b, uv w z,

12 FLORIAN POTRA AND JOSEF STOER where ã = xs / a and ζz, b { = in ũ, ṽ z } : Qũ + Rṽ = b = b T QD Q T + RD R T b Proof Since Q, R is a sufficient pair, so is QD, RD and therefore, according to Lea, the atrix A = [QD RD] has full rank It follows that the solution of the iniization proble defining ζz, b is given by w = ũ, ṽ = A T AA T b, ζz, b = w z = b T AA T b = bt QD Q T + RD R T b If w = u, v is the solution of 37 then w = u, v = w w satisfies { su + xv = a sũ + xṽ Qu + Rv = 0 By applying Lea 3 we deduce that w z w z + w z + κ ã Dũ + D ṽ + w z + κ ã + Dũ + D ṽ + w z + κ ã + Dũ + D ṽ + Dũ D ṽ + w z + κ ã + w z + w z = + κ ã κ w + κ z Finally, we have uv Du D v Du + D v / = w z / In the reainder of this subsection we will find various bounds on the quantity ζz, b appearing in Lea 33 Lea 34 If HLCP is sufficient then for any z = x, s, z 0 = x 0, s 0 IR ++ n we have ζz, b xsx 0 s 0 / x T s 0 + s T x 0 ζz 0, b, where ζ is defined in Lea 33 Proof Since Q, R is a sufficient pair, we deduce as in the proof of Lea 33 that the atrices B = QD Q T + RD R T, B 0 = QD 0 QT + RD 0R T, where D = X / S /, D 0 = X 0 / S 0 /, are syetric positive definite and ζz, b = b T B b = B / b, ζz0, b = b T B0 b = B / 0 b

13 Therefore, INTERIOR POINT METHODS FOR SUFFICIENT LCP 3 b T B b = b T B / 0 B / 0 BB / 0 B / 0 BB / 0 / B 0 b B / 0 b For any h IR n we denote f = Q T B / 0 h, g = R T B / 0 h and obtain successively: h T B / 0 BB / 0 h = f T D f + g T D g = D f + Dg = D D 0 D0 f + DD 0 D 0g D 0 D D 0 f + D0 D D 0 g { in D0 D, D 0 D } D 0 f + D 0g { = in s 0 x x 0 s, x 0 s s 0 x } h It follows that B / 0 BB / 0 ax { s 0 x x 0 s, x 0 s s 0 x } = ax { x 0 s 0 xs x 0 s, x 0 s 0 xs s 0 x } x 0 s 0 xs ax { x 0 s, s 0 x } = x 0 s 0 xs / ax { x 0 s, s 0 x } x 0 s 0 xs / x T s 0 + s T x 0 The proof is coplete In what follows we show that if HLCP has a solution then x T s 0 + s T x 0 is bounded Lea 35 Assue that HLCP is P κ and that F is nonepty Let x 0, s 0 IR ++ n be a starting point and consider the notations fro 3 and 7 Then for all x, s F and z F b,τ, with 0 < τ < τ 0, we have x T s 0 + s T x 0 + τ 0 τ τ x T s + s T x + 4κ ax{, ζ } + µz/τ nτ 0, where ζ = x 0 T s + s 0 T x /x 0 T s 0 Proof By denoting ψ = τ/τ 0, 0 < ψ <, and using 7, 3 and Qx +Rs = b we obtain Qψx 0 + ψx x + Rψs 0 + ψs s = ψr 0 + b + ψb b τb = 0 Using the inequalities x, s 0, x, s > 0, and the P κ property 4 we have [ψx 0 + ψx x] T [ψs 0 + ψs s] 4κ [ψx 0 + ψx x] i [ψs 0 + ψs s] i i I + 4κ ψ [x 0 ] i [s 0 ] i + ψψ[x ] i [s 0 ] i + [x 0 ] i [s ] i + [x] i [s] i i I + 4κψ x 0 T s 0 + ψψx T s 0 + x 0 T s + x T s,

14 4 FLORIAN POTRA AND JOSEF STOER where I + = {i : [ψx 0 + ψx x] i [ψs 0 + ψs s] i > 0} Since x T s = 0, s T x + x T s 0, s T x 0 + x T s 0 > 0, x T s = nµ, and 0 < ψ <, we deduce that 4κψ nµ 0 + ψψx T s 0 + x 0 T s + nµ [ψx 0 + ψx x] T [ψs 0 + ψs s] = ψ nµ 0 + ψψx 0 T s + s 0 T x ψx T s 0 + s T x 0 + x T s ψs T x + x T s + ψ x T s ψ nµ 0 + ψ ψx 0 T s + s 0 T x ψx T s 0 + s T x 0 + nµ The desired inequality follows by using the fact that ψµ 0 = ψτ 0 = τ We now apply these estiates to bound ζz, τb for a typical iterate x, s, τ found by the Algorith fro a starting point z 0 = x 0, s 0, τ 0 := µz 0 They satisfy for soe τ < τ 0 38 p := xs β µ e, x, s F b,τ, µ = x T s/n, γ µ τ µ/γ In order to avoid the introduction of too any constants in the following estiates and to exhibit the role of the paraeter, which deterines the order of the Algorith, we use a special O r notation order-restricted and denote by π = O r ρ scalar quantities, which ay vary with the iterations of the Algorith, π = π k, ρ = ρ k, for which there is a constant δ 0 not depending on, but eventually depending on the proble instance, so that π k δρ k for all iterations k We write O instead of O r, if δ is order-universal, that is a constant which is independent of the proble, of, and of the starting point z 0 it ay still depend on the other constants β, β 0, γ of the Algorith Hence, eg, β ±, γ ± = O, and κ, n = O r are proble dependent constants not depending on, as are the following constants ζ 0 := ax{, ζ }, ζz 0, r 0 that are connected with the choice of z 0 An iportant property is that q = O r p + + p j, p i = O r p, j = O r, iplies q = O r p, and that q = Of iplies q = O, if f is a bounded function of We now derive bounds for ζz, τb for the case of a general starting point z 0, for the case when z 0 is large enough in the sense that 39 x 0 x, s 0 s, for soe x, s F, and for the case when z 0 is alost feasible in the sense that 30 ζz 0, r 0 ζ 0 + κ n We note that while 30 is difficult to be verified in practice, it is always satisfied for strictly feasible starting points If the starting point is large enough in the sense of 39 then we obtain a siple bound by the following

15 INTERIOR POINT METHODS FOR SUFFICIENT LCP 5 Lea 36 Assue that HLCP is P κ and that F is nonepty Let x 0, s 0 IR ++ n be a starting point satisfying 39 and consider the notations fro 3 and 7 Then for all z, 0 < τ < τ 0 satisfying 38 we have 3 ζz, τb = O + κ n µ Proof Since w = ũ, ṽ = τ τ 0 x 0 x, s 0 s satisfies Qũ + Rṽ = τb we have ζz, τb w z = τ s / x / x 0 x + x / s / s 0 s τ 0 τ τ 0 s / x / x 0 + x / s / s 0 / / / = τ sx / sx 0 τ + sx / xs 0 0 τ sx / sx 0 τ 0 + xs 0 / τ = sx / sx 0, xs 0 τ 0 τ sx / sx 0, xs 0 τ 0 = τ sx / s T x 0 + x T s 0, τ 0 and by 39 ζ = x 0 T s + s 0 T x /x 0 T s 0 x 0 T s 0 + s 0 T x 0 /x 0 T s 0 = Therefore, by virtue of Lea 35, we deduce that ζz, τb τ sx / x T s 0 + s T x 0 + 4κ sx / τ + µzn, τ 0 so that by 38 ζz, τb + κ βµ γ µ + µ n = O + κ n µ The cases of a general starting point z 0 and of an alost feasible starting point 30 are treated in the following Lea 37 Assue that HLCP is P κ and that F is nonepty Let x 0, s 0 IR ++ n be the starting point of the Algorith and z a typical iterate satisfying 38 Then the following holds in general 3 ζz, τb = O ζ 0 ζz 0, r 0 + κn µ If the starting point z 0 satisfies 30, then 33 ζz, τb = O nµ Proof Forula 3 follows fro 38, Leas 34, 35, and the estiate ζz, τb = τ τ 0 ζz, r 0 τ τ 0 xsx 0 s 0 / x T s 0 + s T x 0 ζz 0, r 0

16 6 FLORIAN POTRA AND JOSEF STOER τxt s 0 + s T x 0 τ 0 β µµ 0 ζz 0, r 0 + 4κ β µµ 0 ax{, ζ }τ + µ ζz 0, r 0 n + 4κ β µ0 ax{, ζ }γ + ζz 0, r 0 n µ = O ζ 0 ζz 0, r 0 + κn µ The estiate 33 follows directly fro the general estiates 3 and 30 To prove the ain results of the next subsection we need soe estiates concerning the following vectors appearing in 37 and 37: a = + ϑ[στ e + στxs ], ã = xs / a, b = + ϑτb, where x, s is a current iterate of the Algorith satisfying p := xs βµe β µe, p > 0, e T p = nµ, and στ, στ γ β0 β τ µ Noting that 34 we obtain xs / xs β µ, / n xs = β µ, / = n µ, ã στ xs / + + στxs / 35 n µ + 4 n µ = O n µ = Or µ, β στxs / τxs / nµ + β / = O n µ = Or µ Since a b + c d a + d b + c holds for all a, b, c, d 0 we get for the solutions w i = u i, v i of 37 for all i, j u i v j + u j v i Du i D v j + Du j D v i 36 w i z w j z 33 Global convergence The coputational coplexity of the Algorith is basically the sae for ϑ = 0 and ϑ = One could eventually obtain slightly better constants if ϑ = 0 and/or if HLCP is skew syetric, but in what follows we will obtain bounds that are independent of ϑ We will prove the global convergence of the Algorith under general assuptions, but we will give coputational coplexity results only when the starting point satisfies 39 or 30 In order to treat the two cases together it is convenient to define 37 χ = {, if 39 is satisfied 0, if 30 is satisfied Proposition 38 If HLCP is sufficient and solvable then for any starting point z 0 Dβ, there is a constant 0 < Λ = O r such that for all iterations and all the vectors h i produced at each iteration of the Algorith satisfy h i β Λ i, i = +,, µ + κi

17 INTERIOR POINT METHODS FOR SUFFICIENT LCP 7 Moreover, there are constants 38 0 < Λ χ = O + κ +χ n +χ/, χ = 0,, such that for any starting point z 0 Dβ satisfying 30χ = 0, resp 39 χ = the vectors h i produced at each iteration of the Algorith satisfy for all the following inequalities: h i µ β Λ i χ, i = +,,, + κi Proof In this proof we eploy the O and O r estiates of the previous subsection and use repeatedly the estiates 34, 35, and 36 We start by using 36 to write 39 h i = j=i j=i Du j D v i j + Du i j D v j w j z w i j z and to find constants ω, ω,, ω such that w i 330 z ω i µ, i =,,, Fro 37 and Lea 33 we deduce that w z + κ ã + + ϑ + + 4κ ζz, τb, w z + κ ã + ϑ + + 4κ ζz, τb, w i z + κ ã i 33, i = 3, 4,,, where 33 ã = + ϑ ã = ϑxs / + + 4ϑστ ã 3 = 4ϑστ ã 4 = ϑστ xs / + στ xs / τxs /, xs / τxs / xs / u v, xs / τxs / xs / u v + u v, xs / τxs / xs / u v 3 + u v + u 3 v, i ã i = xs / u j v i j, i = 5, 6,, j= We first consider the general case of an arbitrary starting point z 0 Then by ζt, τb = O r µ Using 35 and applying repeatedly Lea 33 L33 and equations 34, 333, 36 we deduce successively that ã 35 = O r µ 333 w z = O r µ u v L33 05 w z O r µ

18 8 FLORIAN POTRA AND JOSEF STOER 34 xs / u v u v / β µ = O r µ ã = O r µ 333 w z = O r µ 34 xs / u v + u v w z w z / β µ = O r µ ã3 = O r µ 333 w 3 z = O r µ w 4 z = O r µ If 4, it follows that 330 is satisfied with soe constants ω i = O r, i =,, Assue now that 5 According to 34, 33, and 33, we have w i z + κ ã i + κ Du j β µ D v i j, i = 5,,, and proceeding as in 39 we obtain 334 w i z If we define recursively 335 j=i c i w j µ z w i j z, c := j= j= + κ 4β, i = 5,, i ω i := c ω j ω i j, i = in{5, + }, 6,, then by using 334 and the fact that 330 is satisfied for i =,, 3, 4 we can easily prove by induction that 330 holds for i =,,, We show next that 336 where ω i α i c 05Λi Λi, i =,,,, c i { } Λ := 4 ax c ω i /α i /i : i =,, in{4, } = O r and α i is the sequence considered in [47] α i := i i i i 4i, which satisfies the following recurrence i α =, α i = α j α i j Fro construction it follows that 336 holds for i in{4, }, and using the recursions of the α i and the ω i, it is easily proved by induction that it also holds for in{5, + } i By using 39, 330, and 336, we obtain for i = +,, h i j=i µ 05Λi c j= w j z w i j z µ j=i i α j α i j = µα i 05Λ i c j= ω j ω j i µ i ω j ω j i j= µλi c i µ β Λ i + κ i,

19 INTERIOR POINT METHODS FOR SUFFICIENT LCP 9 which ends the proof of the first part of the Proposition The ore explicit order-universal bounds for starting points satisfying 39 or 30 are found by essentially the sae arguents by just using instead of 333 the ore precise O bounds given in the previous subsection So if 30 is satisfied χ = 0 we use 33, and if 39 is satisfied χ = we use 3, that is, 337 ζz, τb = O + κ χ n +χ µ By using Lea 33 L33, equations 35, 34, 337 and the inequality 39 we deduce successively that ã 35 = O nµ 337 w z = O + κ 5+χ n +χ µ 34 xs / u v L33 w β µ z = O + κ +χ n +χ µ w z = O + κ 5+χ n +χ µ ã = O + κ +χ n +χ µ 337 xs / u v + u v 39 w β µ z w z = O + κ +3χ n 3+3χ µ ã3 = O + κ +3χ n 3+3χ 337 µ w 3 z = O + κ 5+3χ n 3+3χ µ xs / u v 3 + u v + u 3 v 39 β µ w z w 3 z + 5 w z = O + κ 3+4χ n +χ µ ã4 = O + κ 3+4χ n +χ µ 337 w 4 z = O + κ 35+4χ n +χ µ We have thus proved that w i z ω χ i µ, ω χ i = O + κ +χi 05 n +χi/, i in{4, } Therefore, we get as in the general case for all i = +,, a bound of the for h i µ 4β Λ i χ + κi, where, c ω χ /i i Λ χ = 4 ax : i =,, in{4, } α i = O + κ +χ n +χ/ This copletes the proof of the Proposition In the next theore we will prove the global convergence of the Algorith under the assuption that the sequence α k satisfies the condition 338 k=0 + α k =

20 0 FLORIAN POTRA AND JOSEF STOER We ention that the conditions 34 and 338 are fairly restrictive, excluding any sequence with sup k α k+ /α k < In subsection 34 we will give an explicit exaple of a sequence α k that satisfies both 34 and 338 plus an additional condition that ensures superlinear convergence Theore 39 If HLCP is sufficient and solvable and 338 is satisfied then for all ϑ, θ T the duality gaps and the residuals of the iteration sequence generated by the Algorith converge to zero, ie, li µ k = li τ k = 0, k k li k rk = 0 Proof We will first find a lower bound for the steplength θ coputed by the Algorith and use repeatedly the following easily verified inequalities 0 ϕ ϑ, θ, 0 θϕ ϑ, θ 4 for all 0 θ /4, ϑ, T Siilarly, one verifies for all 0 στ and all ϑ, T the inequalitiy 339 θ +ϑ στθϕ ϑ, θ 0 for all 0 θ / We note that 339 even holds for all 0 θ for all pairs ϑ, {0,,, 3} In addition, we use the following constants and estiates γ 0 := in{, γ α0 log γ }, γ α γ α αγ α log γ γ 0 α We have zθ Dβ + if and only if xθsθ β αµθe 0 Using the fact that z = z0 Dβ, 38, 39, and assuing θ [0, 05] we get by 339 where, 340 xθsθ β αµθe = θ +ϑ στθϕ ϑ, θ xs + i=+ β α θ +ϑ στθϕ ϑ, θ µe + β θ +ϑ στθϕ ϑ, θ µe + i=+ β α θ +ϑ στθϕ ϑ, θ µe + θ i h i + β + αστ θϕ ϑ, θe i=+ θ i e T h i /n θ i h i + β + αστ θϕ ϑ, θe i=+ θ i e T h i /n = α θ +ϑ στθϕ ϑ, θ µe + β + αστ θϕ ϑ, θe + θ i h i β α et h i n e αµψ α, ϑ, θ e, i=+ ψ α, ϑ, θ := θ +ϑ βτ + ατ µ + στθϕ ϑ, θ αµ αγ 0 µ i=+ θ i h i

21 INTERIOR POINT METHODS FOR SUFFICIENT LCP In the last inequality above we used the fact that h i β αe T h i /n e h i and 0 < γ 0 With the above notation it follows that 34 z Dβ, ψ α, ϑ, θ 0 and 339 = zθ Dβ α We also note that β + α α γ β0 β βτ + ατ µ αµ β + α α γ β β0 and by using στ στ 0 γ β0 β, τ γ β0 β µ, we deduce βτ + ατ µ σττ µ στ > γ β0 β γ β0 β αµ µ 4 Fro Proposition 38 it follows that µ i=+ For any t 0, ] we have i=+ t i i t+ θ i h i i=+ Hence for arbitrary 0 θ θ Λ µ i=+ β + κ i=+ i θλi, 0 θ i < du t+ u = t+ log < 07 θ i h i 4β + κ Λ θ + Since 0 στθϕ ϑ, θ 05 for all 0 θ 05, we get the estiate ψ α, ϑ, θ θ +ϑ 6 4β Λ θ + αγ 0 + κ 4β Λ θ + αγ 0 + κ for all 0 θ θ in{05, Λ } Therefore ψ α, ϑ, θ 0 for all θ θ, where 34 θ = in { 4, Λ, Λ γ0 α + κ 8β } + Hence the quantity θ defined by 3 satisfies θ θ Also we have µθ γ β0 β+ θ +ϑ µ µθ γ α0 θ +ϑ µ µθ γ α θ +ϑ µ = θ +ϑ γ α µ + σττ µθϕ ϑ, θ + µγ 0 α ψ α, ϑ, θ, i=+ θ i e T h i /n

22 FLORIAN POTRA AND JOSEF STOER where 343 ψ α, ϑ, θ = θ +ϑ + σττ µ µγ 0 α θϕ ϑ, θ γ 0 αµ i=+ θ i h i, and µθ γ β+ β0 θ +ϑ µ µθ γ α0 θ +ϑ µ µθ γ α θ +ϑ µ = θ +ϑ γ α µ + σττ µθϕ ϑ, θ + µγ 0 α ψ α, ϑ, θ, where i=+ θ i e T h i /n 344 ψ α, ϑ, θ = θ +ϑ σττ µ µγ 0 α θϕ ϑ, θ γ 0 αµ i=+ θ i h i Therefore the following iplications hold: ψ α 0, ϑ, θ 0 and ψ α, ϑ, θ 0 = γ α µθ θ +ϑ µ, γβ0 β+ ψ α, ϑ, θ 0 and ψ µθ α 0, ϑ, θ 0 = γ β+ β0 θ +ϑ µ γα We will prove now that the quantity defined in 3 satisfies 345 θ θ := in { γ 0 α 0 4, Λ, Λ γ0 α + κ 4β } + For the reainder of this proof we assue that 0 θ θ This iplies 0 θ /4 If τ µ then 0 σττ µ/µ /4, so that by using ϕ ϑ, θ, we have since α α 0, and ψ α 0, ϑ, θ θ +ϑ θ γ 0 α 0 µγ 0 α 0 i=+ θ i h i β Λ θ + > γ 0 α 0 + κ 6 4β Λ θ + 0, γ 0 α 0 + κ ψ α, ϑ, θ θ +ϑ µγ 0 α If τ > µ, then i=+ θ i h i 9 6 4β Λ θ + > 0 γ 0 α + κ 0 σττ µ/µ στ 0 τ µ γ β0 β γ β β0 <

23 INTERIOR POINT METHODS FOR SUFFICIENT LCP 3 so that we find, as before, that for all θ θ we have ψ α, ϑ, θ θ +ϑ µγ 0 α ψ α 0, ϑ, θ θ +ϑ 6 µγ 0 α 0 i=+ θ α 0 γ 0 µγ 0 α 0 i=+ θ i h i > 0, i=+ θ i h i 0 θ i h i This shows that 345 holds Without loss of generality we ay assue that κ and Λ are large enough such that γ 0 + κ 4β, Λ 4 γ0 + + κ 346 γ 0 α 0 4β Since θ θ θ = ω + α, it follows that the steplength coputed by the Algorith satisfies θ θ Hence θ k ω + α k, ω := γ0 + + κ 347, k = 0,, Λ 4β Using Theore 3 we have, 348 k τ k = τ 0 j=0 θ j +ϑ k j=0 +ϑ ωα /+ j, which by virtue of 338 shows that li k τ k = 0 The proof is coplete by noticing that according to Theore 3 we have r k = τ k τ0 r 0 and µ k γ β β0 τ k 34 Higher order convergence To investigate the superlinear convergence of the Algorith we need the further assuption that the sequence 338 diverges slow enough in the sense that 349 li k log α k k i=0 + α i = 0 Then, as we will see, the Algorith with ϑ {0, } satisfying 6,, and ϑ,, is superlinearly convergent for general probles However, if the proble is known to have a strictly copleentary solution it is advantageous to take ϑ = 0 in order to obtain a higher order of convergence In subsection 35 a sequence α k satisfying all requireents 34, 338 and 349 will be given explicitly The proof of superlinear convergence is based on Theore In order to be able to apply this theore we need to show that the range of the function t ẑt, τ, σ, xs/τ = zt, P t, τ, σ, xs/τ, t 0, τ], given by 8 lies in a copact set of IR ++ n This follows fro the fact that for any z = x, s Dβ and p = xs/τ, with γµ τ µ/γ, 0 σ in{, γ β0 β /τ 0 }, and t [0, τ], we have, with λ = σ τ t, P t, τ, σ, p = λ p + λe λ τ µ β e + λβ γ β0 β e β γ β0 β e, P t, τ, σ, p = λ nµ τ + λn γβ β0 n,

24 4 FLORIAN POTRA AND JOSEF STOER see Theore 3 By using 36, Theore, and Theore 3 we obtain the following result: Lea 30 If HLCP is sufficient and solvable then there is a constant c such that the vectors u, v,, u, v coputed at each iteration of the Algorith satisfy u i c τ i +ϑ, v i c τ i +ϑ, i =,,, This Lea is crucial for proving the following theore on the superlinear convergence of the Algorith: Theore 3 Let ϑ {0, } be defined by 6 and let such that ϑ,, If HLCP is sufficient and solvable and α k is a onotone sequence of positive nubers satisfying 34, 349, 338 then the sequence z k produced by the Algorith converges Q-superlinearly to a axial copleentarity solution z F c, Also, the sequences of the corresponding copleentarity gaps µ k, feasibility easures τ k, and residuals r k converge Q-superlinearly to zero Moreover, the Q-orders of convergence of these sequences satisfy Q z k = Q µ k = Q τ k = Q r k + + ϑ Proof Since we are analyzing asyptotic properties we ay assue µ k < and τ k < By using α k <, Theore 3, Theore 39, 349, and 348 we obtain 0 log α k log τ k = log α k logτ k /τ 0 + log τ 0 Hence, since τ k γµ k, there is a sequence ε k so that log α k + ϑ ω 0 k j=0 α/+ j + log τ α k = τ ε k k γµε k k, ε k 0 Fro 30 and Lea 30 it follows that h i j=i u j v i j c µ i +ϑ For k sufficiently large we have µ +ϑ /, so that for all θ [0, ] 35 i=+ θ i h i i=+ h i c µ + +ϑ µ i +ϑ c µ + + +ϑ cµ +ϑ, for any c c In what follows we will show that if µ is sufficiently sall, then the stepsize θ = in{θ, θ } taken by the Algorith deterined by 3 and 3 satisfies i=0 35 θ ˆθ c +ϑ ϑ := µ +ϑ αγ 0 According to 34, in order to prove that θ ˆθ, it is sufficient to show that 339 and ψ α, ϑ, θ 0 where ψ is defined by 340 are satisfied for all ϑ, T,

25 INTERIOR POINT METHODS FOR SUFFICIENT LCP 5 all θ [0, ˆθ ] and sufficiently large k ie sall τ, sall µ, and sall α As to 339 see the note following 339, we have to check only the cases ϑ, = 0, and ϑ, =, 3: If ϑ, = 0,, 339 follows fro φ 0, θ =, σ, τ µ/γ, since for θ ˆθ < θ στθ c αγ 0 µ τ c αγ 0 γ µ 0, for sall α > 0 If ϑ, =, 3, 339 follows siilarly fro φ 3, θ = 5θ + 4θ since, θ στθφ 3, θ c αγ 0 µ /4 µ γ 0, for sall µ > 0 As to ψ, we have to show that for sufficiently large k ψ α, ϑ, θ 0, θ [0, ˆθ ], ϑ, T Using the relations γ β0 β τ/µ γ β β0 we deduce that for α sufficiently sall If 0 β 0 α γ β0 β βτ + ατ µ αµ β + α α γ β β0 γβ β0 α 353 ϑ = 0 and or ϑ = and 3, then ϕ ϑ, θ 0, θ [0, ], so that the following holds ψ α, ϑ, θ θ +ϑ αγ 0 µ i=+ θ i h i c µ ϑ +ϑ > 0, θ [0, αγ ˆθ ], 0 which shows that θ ˆθ for the case 353 On the other hand, if = and ϑ =, then we have ϕ, θ = 5θ, and ˆθ = c/αγ 0 µ 4, so that for all µ satisfying 3 µ /µ 0 c/γ 0 which holds for k sufficiently large we can write ψ α,, θ θ 3στ γβ β0 4 θ i h i α αγ 0 µ θ 3µ c µ αµ i=3 0 αγ 0 c c µ µ c µ = 0 αγ 0 αγ 0 αγ 0 In order to prove θ ˆθ we will show that the functions ψ α, ϑ, θ and ψ α, ϑ, θ see 343, 344 are nonnegative for all θ [0, ˆθ ] and sufficiently large k Because τ µ/γ, the ter σττ µ appearing in their definitions is bounded by σττ µ ττ µ γ γ µ c µ, c := γ Hence, a coon lower bound for ψ α, ϑ, θ and ψ α, ϑ, θ that is valid for all θ [0, ˆθ ] and sall µ is given by θ +ϑ c µ γ 0 α θ ϕ ϑ, θ c γ 0 α µ ϑ c 354 +ϑ µ ϑ +ϑ µ θϕ ϑ, θ, αγ 0

26 6 FLORIAN POTRA AND JOSEF STOER if c := ax{c, c } Fro 30 we deduce that for all θ [0, ˆθ ] and sall µ we have 0 µθϕ 0, θ µ, 0 µθϕ 0, θ µ θ cµ+ αγ 0, =, 3,, µ θϕ, θ 3µ, 0 µθϕ 3, θ µ, 0 µθϕ, θ = µθ θ θ µ θ + cµ, = 4, 5, αγ 0 Fro 350 and 354 it follows that for sufficiently large k we have ψ α, ϑ, θ 0, ψ α, ϑ, θ 0, θ [0, ˆθ ], ϑ, T Hence, the step θ chosen by the Algorith satisfies θ ˆθ, which iplies τ k+ ˆθ +ϑ τ k c µ ϑ +ϑ k τ k α k γ 0 Therefore c α k γ 0 γ ϑ +ϑ Q τ k = li inf log τ k+ log τ k + + ϑ, + c τ k +ϑ γ 0 γ ϑ +ϑ τ k + +ϑ ε k and by using γτ k µ k τ k /γ it follows that Q µ k = Q τ k For k sufficiently large we have cµ +ϑ < /, so that by using Lea 30 we obtain z + z w i i= i= cµ i cµ + +ϑ +ϑ cµ +ϑ = cµ +ϑ cµ +ϑ, and by applying Theore of [3] we deduce the convergence of the sequence z k to a axial copleentarity solution z F and the fact that Q z k = Q µ k 35 Polynoial Coplexity In Theore 39 we have established global convergence for any sequence α k satisfying 34 and 338 We will obtain polynoial coplexity for starting points satisfying 30 or 39 by using the following sequence that was first considered in [7], 355 νβ 0 β α k = exp + k + log +ν, k = 0,,, exp + k + where 0 < ν is a given paraeter Fro [7] it follows that this sequence satisfies 34 and 356 K k=0 α + k > νβ 0 β + log K +ν + K + Using [7, Corollary 35] we obtain the following result Lea 3 If M := νβ 0 β + M > exp, and K + + ν + log + ν +ν + log M + M,

27 then K k=0 α + INTERIOR POINT METHODS FOR SUFFICIENT LCP 7 k > M In the reainder of this subsection we give explicit upper bounds for the nuber of iterations required by the Algorith to obtain a solution of the HLCP with prescribed accuracy in case the starting point satisfies 39 or 30 More precisely given any ɛ > 0 we have to find an upper bound for the nuber 357 K ɛ := in { K : ax { x k T s k, r k } ɛ, k K } The upper bound will depend on n, κ,, { x 0 T s 0 } r L ɛ := log ax,, ɛ ɛ and η χ := Λ χ + ϑ 4β νγ 0 β 0 β + κ +, where Λ χ is the order-universal constant fro Proposition 38 and χ is given by 37 We note that according to 38 we have η χ = O + κ +χ /+ n +χ/ Theore 33 Assue that HLCP is sufficient and solvable and consider the sequence produced by the Algorith, where the sequence α k is given by 355 Assue also that the starting point satisfies 30 or 39 and that L ɛ ax {β 0 β log γ, exp} Then K ɛ 46 log η χ L ɛ +ν η χ L ɛ + +ν = O log + κ χ+ +χ χ+ + n + Lɛ + κ n +χ + Lɛ, with χ given by 37 Proof In what follows we will use Proposition 38, Theore 39, and their proofs Fro 38 it follows that we ay assue without loss of generality that κ and Λ χ are large enough such that 359 γ 0 + κ 4β, Λ χ 4 γ 0 α 0 γ0 + κ 4β As in 347 we deduce that θ k ω χ + α k, k = 0,,, where, 360 ω χ := Λ χ γ0 + κ 4β + +, = O + κ +χ + n +χ/ ω χ By applying Theore 3 we deduce that r k log r 0 = log τ k k + ϑ log ω χ α j + τ 0 j=0 k + ϑ ω χ log µ k β 0 β log γ + log τ k k L ɛ + ϑ ω χ α j + µ 0 τ 0 j=0 j=0 α j +,