A Full Newton Step Infeasible Interior Point Algorithm for Linear Optimization

A Full Newton Step Infeasible Interior Point Algorithm for Linear Optimization Kees Roos e-mail: C.Roos@tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos 37th Annual Iranian Mathematics Conference Tabriz, Iran September 2 5, A.D. 2006

Outline Milestones in the history of linear optimization State-of-the-art in IIPMs Usual search direction Complexity results for feasible IPM Definition of perturbed problems Idea underlying the algorithm Complexity analysis Conjecture Some references Announcement: Today there will be another lecture at 11:30 in Seminar 1, entitled Introduction to Conic Optimization, with some applications Optimization Group 1

Milestones in the history of linear optimization Standard linear optimization problem (P) min { c T x : Ax = b, x 0 }, A is m n 1947: Simplex Method (Dantzig) 1955: Logarithmic Barrier Method (Frisch) 1967: Affine Scaling Method (Dikin) 1967: Center Method (Huard) 1968: Barrier Methods (Fiacco, McCormick) 1972: Exponential example (Klee and Minty) 1979: Ellipsoid Method (Khachiyan) 1984: Projective Method (Karmarkar) Interior Point Methods for Convex Optimization (1989) Semidefinite Optimization (1994) Second Order Cone Optimization (1994) Discrete Optimization (1994) Optimization Group 2

Some keyplayers Optimization Group 3

A is m n and rank(a) = m. State-of-the-art in IIPMs (P) min { c T x : Ax = b, x 0 } (D) max { b T y : A T y + s = c, s 0 } The best know iteration bound for IIPMs is { (x max 0 ) T s 0, b Ax 0 } c, A T y 0 s 0 O n log ǫ x 0 > 0, y 0 and s 0 > 0 denote the starting points, and b Ax 0 and c A T y 0 s 0 are the initial primal and dual residue vectors, respectively. Assumption: the initial iterates are (x 0, y 0, s 0 ) = ζ(e,0, e) where ζ satisfies ζ x, s, for some optimal triple (x, y, s ). The best know iteration bound for solving these problems (with feasible IPMs) is ( n ) n O log. ǫ. Optimization Group 4

Usual search directions A x = b Ax A T y + s = c A T y s s x + x s = µe xs. When moving from x to x + := x + x the new iterate x + satisfies Ax + = b, but not necessarily x + 0. To keep the iterate positive, one uses a damped step x + := x+α x, where α < 1 denotes the step size. Our search directions are designed in such a way that full Newton steps can be used; each iteration reduces the sizes of the residual vectors with exactly the same speed as the duality gap. Our aim is to show that this is a promising strategy. Optimization Group 5

Feasible full-newton step IPMs Central path Ax = b, x 0 A T y + s = c, s 0 xs = µe. If this system has a solution, for some µ > 0, then a solution exists for every µ > 0, and this solution is unique. This happens if and only if (P) and (D) satisfy the interior-point condition (IPC), i.e., if (P) has a feasible solution x > 0 and (D) has a solution (y, s) with s > 0. If the IPC is satisfied, then the solution is denoted as (x(µ), y(µ), s(µ)), and called the µ-center of (P) and (D). The set of all µ-centers is called the central path of (P) and (D). As µ goes to zero, x(µ), y(µ) and s(µ) converge to optimal solution of (P) and (D). The system is hard to solve, but by applying Newton s method one can easily find approximate solutions. Optimization Group 6

Definition of the Newton step Given a primal feasible x > 0, and dual feasible y and s > 0, we want to find displacements x, y and s such that A(x + x) = b, A T (y + y) + s + s = c, (x + x)(s + s) = µe. Neglecting the quadratic term x s in the third equation, and using b Ax = 0 and c A T y s = 0, we obtain the following linear system of equations in the search directions x, y and s. A x = 0, A T y + s = 0, s x + x s = µe xs. Since A has full rank, and the vectors x and s are positive, the coefficient matrix in this linear system is nonsingular. Hence the system uniquely defines the search directions x, y and s. These search directions are used in all existing primal-dual (feasible) IPMs and called after Newton. Optimization Group 7

Properties of the Newton step The new iterates are given by A x = 0, A T y + s = 0, s x + x s = µe xs. x + = x + x, y + = y + y, s + = s + s. An important observation is that x lies in the null space of A, whereas s belongs to the row space of A. This implies that x and s are orthogonal, i.e., ( x) T s = 0. As a consequence we have the important property that after a full Newton step the duality gap assumes the same value as at the µ-centers, namely nµ. Lemma 1 After a (feasible!) primal-dual Newton step one has ( x +) T s + = nµ. Optimization Group 8

Fundamental results We use the quantity δ(x, s; µ) to measure proximity of a feasible triple (x, y, s) to the µ-center (x(µ), y(µ), s(µ)). δ(x, s; µ) := δ(v) := 1 v v 1 xs where v := 2 µ. Lemma 2 Let (x, s) be a positive primal-dual pair and µ > 0 such that x T s = nµ. Moreover, let δ := δ(x, s; µ) and let µ + = (1 θ)µ. Then δ(x, s; µ + ) 2 = (1 θ)δ 2 + θ2 n 4(1 θ). Lemma 3 If δ := δ(x, s; µ) 1, then the primal-dual Newton step is feasible, i.e., x + and s + are nonnegative. Moreover, if δ < 1, then x + and s + are positive and δ(x +, s + ; µ) δ 2. 2(1 δ 2 ) Corollary 1 If δ := δ(x, s; µ) 1 2, then δ(x +, s + ; µ) δ 2. Optimization Group 9

Primal-Dual full-newton step feasible IPM Input: Accuracy parameter ε > 0; barrier update parameter θ, 0 < θ < 1; feasible (x 0, y 0, s 0 ) with ( x 0) T s 0 = nµ 0, δ(x 0, s 0, µ 0 ) 1/2. begin x := x 0 ; y := y 0 ; s := s 0 ; µ := µ 0 ; while x T s ε do begin µ-update: end end µ := (1 θ)µ; centering step: (x, y, s) := (x, y, s) + ( x, y, s); Theorem 1 If θ = 1/ 2n, then the algorithm requires at most 2n log nµ0 ǫ The output is a primal-dual pair (x, s) such that x T s ǫ. iterations. Optimization Group 10

Graphical illustration primal-dual full-newton-step path-following method z 0 µe central path δ(v) τ z k = x k s k (1 θ)µe z 1 One iteration. Optimization Group 11

Definition of perturbed problems We start with choosing arbitrarily x 0 > 0 and y 0, s 0 > 0 such that x 0 s 0 = µ 0 e for some (positive) number µ 0. For any ν with 0 < ν 1 we use the perturbed problem (P ν ), defined by (P ν ) min { (c ν ( c A T y 0 s 0)) T x : Ax = b ν ( b Ax 0 ), x 0 and its dual problem (D ν ), which is given by (D ν ) min { (b ν ( b Ax 0 )) T y : A T y + s = c ν ( c A T y 0 s 0), s 0 }, }. Note that if ν = 1 then x = x 0 yields a strictly feasible solution of (P ν ), and (y, s) = (y 0, s 0 ) a strictly feasible solution of (D ν ). Conclusion: if ν = 1 then (P ν ) and (D ν ) satisfy the IPC. Optimization Group 12

Perturbed problems satisfy IPC Lemma 4 The original problems, (P) and (D), are feasible if and only if for each ν satisfying 0 < ν 1 the perturbed problems (P ν ) and (D ν ) satisfy the IPC. Proof: Let x be feasible for (P) and (y, s ) for (D). Then Ax = b and A T y + s = c, with x 0 and s 0. Now let 0 < ν 1, and consider x = (1 ν) x + ν x 0, y = (1 ν) y + ν y 0, s = (1 ν) s + ν s 0. One has Ax = A ( (1 ν) x + ν x 0) = (1 ν) b + νax 0 = b ν ( b Ax 0), showing that x is feasible for (P ν ). Similarly, A T y + s = (1 ν) ( A T y + s ) + ν ( A T y 0 + s 0) = c ν ( c A T y 0 s 0), showing that (y, s) is feasible for (D ν ). Since ν > 0, x and s are positive, thus proving that (P ν ) and (D ν ) satisfy the IPC. To prove the inverse implication, suppose that (P ν ) and (D ν ) satisfy the IPC for each ν satisfying 0 < ν 1. Obviously, then (P ν ) and (D ν ) are feasible for these values of ν. Letting ν go to zero it follows that (P) and (D) are feasible. Optimization Group 13

Central path of the perturbed problems We assume that (P) and (D) are feasible. Letting 0 < ν 1, the problems (P ν ) and (D ν ) satisfy the IPC, and hence their central paths exist. This means that the system b Ax = ν(b Ax 0 ), x 0 c A T y s = ν(c A T y 0 s 0 ), s 0 xs = µe has a unique solution, for every µ > 0. In the sequel this unique solution is denoted as (x(µ, ν), y(µ, ν), s(µ, ν)). These are the µ-centers of the perturbed problems (P ν ) and (D ν ). Note that since x 0 s 0 = µ 0 e, x 0 is the µ 0 -center of the perturbed problem (P 1 ) and (y 0, s 0 ) the µ 0 -center of (D 1 ). In other words, (x(µ 0,1), y(µ 0,1), s(µ 0,1)) = (x 0, y 0, s 0 ). In the sequel we will always take µ = µ 0 ν. Optimization Group 14

Idea underlying the algorithm We just established that if ν = 1 and µ = µ 0, then x = x 0 is the µ-center of the perturbed problem (P ν ) and (y, s) = (y 0, s 0 ) the µ-center of (D ν ). These are our initial iterates. We measure proximity to the µ-center of the perturbed problems by the quantity δ(x, s; µ). So, initially, δ(x, s; µ) = 0. We assume that at the start of each of the subsequent iterations δ(x, s; µ) τ, where τ > 0. This is certainly true for the first iteration. Now we describe one iteration of our algorithm. Suppose that for some µ (0, µ 0 ] we have x, y and s that are feasible for (P ν ) and (D ν ), with µ = µ 0 ν, and such that x T s = nµ and δ(x, s; µ) τ. Roughly spoken, one iteration of the algorithm does the following. Reduce µ to µ + = (1 θ)µ and ν to ν + = (1 θ)ν, with θ (0,1). Find new iterates x +, y + and s + that are feasible for (P ν +) and (D ν +), and such that x T s = nµ + and δ(x +, s + ; µ + ) τ. Note that µ + = µ 0 ν +. Optimization Group 15

Graphical illustration of one iteration δ(v) 1 2 µe z 1 feasibility step z 0 µe δ(v) τ z 2 first centering step z 3 z 4 central path for ν + central path for ν µ + e µ + e Feasibility step and subsequent centering steps (µ + = (1 θ)µ, ν + = (1 θ)ν). Optimization Group 16

Notations We denote the initial values of the primal and dual residuals as rb 0 and r0 c, respectively, as rb 0 = b Ax0, rc 0 = c AT y 0 s 0. Then the feasibility equations for (P ν ) and (D ν ) are given by b Ax = νrb 0, x 0 c A T y s = νrc 0, s 0. and those of (P ν +) and (D ν +) by b Ax = ν + rb 0, x 0 c A T y s = ν + rc 0, s 0. Optimization Group 17

Feasibility step Let x be feasible for (P ν ) and (y, s) for (D ν ). To get iterates feasible for (P ν +) and (D ν +) we need f x, f y and f s such that b A(x + f x) = ν + r 0 b c A T (y + f y) (s + f s) = ν + r 0 c. Since x is feasible for (P ν ) and (y, s) for (D ν ), it follows that A f x = (b Ax) ν + r 0 b = νr 0 b ν+ r 0 b = θνr 0 b A T f y + f s = (c A T y s) ν + r 0 c = νr 0 c ν+ r 0 c = θνr 0 c. Therefore, the following system is used to define f x, f y and f s: A f x = θνr 0 b A T f y + f s = θνr 0 c s f x + x f s = µe xs. The new iterates are given by ( x f, y f, s f) = ( x + f x, y + f y, s + f s ). Optimization Group 18

Centering steps After the feasibility step the iterates are feasible for (P ν +) and (D ν +) with ν = ν +. The hard part in the analysis will be to guarantee that these iterates lie in the region where Newton s method is quadratically convergent, i.e., δ(x f, s f ; µ + ) 1/ 2. After the feasibility step we perform centering steps in order to get iterates that moreover satisfy x T s = nµ + and δ(x, s; µ + ) τ. By using the quadratic convergence of the Newton step the required number of centering steps can be easily be obtained. Because, assuming δ = δ(x f, s f ; µ + ) 1/ 2, after k centering steps we will have iterates (x, y, s) that are still feasible for (P ν +) and (D ν +) and such that So k must satisfy which certainly holds if δ(x, s; µ + ) ( 1 ( 1 2 ) 2 k 2 τ, ) 2 k ( ) 1 k log 2 log 2 τ 2.. Optimization Group 19

Primal-Dual infeasible full-newton step IPM Input: Accuracy parameter ε > 0; barrier update parameter θ, 0 < θ < 1 threshold parameter τ > 0. begin x := x 0 > 0; y := y 0 ; s := s 0 > 0; x 0 s 0 = µ 0 e; µ = µ 0 ; while max ( x T s, b Ax, c A T y s ) ε do begin feasibility step: (x, y, s) := (x, y, s) + ( f x, f y, f s); µ-update: µ := (1 θ)µ; centering steps: end end while δ(x, s; µ) τ do (x, y, s) := (x, y, s) + ( x, y, s); endwhile Optimization Group 20

Defining we have Hence v = x f s f Analysis of the feasibility step xs µ, d x := v f x x, d s := v f s, s = x + f x = x + xd x v = x v (v + d x) = s + f s = s + sd s v = s v (v + d s). x f s f = xs + ( s f x + x f s ) + f x f s = µe + f x f s = µ(e + d x d s ). Since µv 2 = xs, after division of both sides by µ + we get ( v f ) 2 x f s f = µ + = µ µ + e + 1 xd x µ + v sd s v = e + d xd s 1 θ. Lemma 5 The iterates ( x f, y f, s f) are strictly feasible if and only if e + d x d s > 0. Corollary 2 The iterates ( x f, y f, s f) are certainly strictly feasible if d x d s < 1. Optimization Group 21

In the sequel we denote Proximity after the feasibility step δ(x f, s f ; µ + ) = δ(v f ) = 1 2 ω(v) := 1 2 v f e v f, where ( v f) 2 = e + d x d s 1 θ. d x 2 + d s 2. This implies d x 2ω(v) and d s 2ω(v), and moreover, d T x d s d x d s 1 2 ( dx 2 + d s 2) 2ω(v) 2 d x d s d x d s 2ω(v) 2. Lemma 6 If ω(v) < 1/ 2 then the iterates ( x f, y f, s f) are strictly feasible. Lemma 7 Let ω(v) < 1/ 2. Then one has 4δ(v f ) 2 θ2 n 1 θ + 2ω(v)2 1 θ + (1 θ) 2ω(v) 2 1 2ω(v) 2. Optimization Group 22

Choice of θ In order to have δ(v f ) 1/ 2, it follows from Lemma 7 that it suffices if θ 2 n 1 θ + 2ω(v)2 1 θ + (1 θ) 2ω(v) 2 2. (1) 1 2ω(v) 2 Lemma 8 Let ω(v) 1 2 and θ = α 2n, 0 α Then the iterates ( x f, y f, s f) are strictly feasible and δ(v f ) 1 2. n n + 1. (2) At this stage we decide to choose θ := α 2n, α 1 2. Then we have ω(v) = 1 2 d x 2 + d s 2 1 2 δ(v f ) 1 2. We proceed by considering the vectors d x and d s more in detail. Optimization Group 23

The scaled search directions d x and d s A f x = θνr 0 b A T f y + f s = θνr 0 c s f x + x f s = µe xs. xs v = µ, d x := v f x x, d s := v f s. s The system defining the search directions f x, f y and f s, can be expressed in terms of d x and d s as follows. where Ād x = θνr 0 b, Ā T f y µ + d s = θνvs 1 r 0 c, d x + d s = v 1 v, Ā = AV 1 X. Optimization Group 24

Geometric interpretation of ω(v) d x 0 90 o 2ω(v) { ξ : Āξ = θνr 0 b } q 2δ(v) θνvs 1 r 0 c + { Ā T η : η R m} r d s Optimization Group 25

Upper bound for ω(v) Let us denote the null space of the matrix Ā as L. So, L := { ξ R n : Āξ = 0 }. Obviously, the affine space { ξ R n : Āξ = θνrb 0 } equals dx + L. The row space of Ā equals the orthogonal complement L of L, and d s θνvs 1 rc 0 + L. Lemma 9 Let q be the (unique) point in the intersection of the affine spaces d x + L and d s + L. Then 2ω(v) q 2 + ( q + 2δ(v)) 2. To simplify the presentation we will denote δ(x, s; µ) below simply as δ (δ τ). Recall that we need 2ω(v) 1. Thus we require q to satisfy q 2 + ( q + 2δ) 2 1. Optimization Group 26

Upper bound for q (1) Recall that q is the (unique) solution of the system Āq = θνr 0 b, Ā T ξ + q = θνvs 1 r 0 c. We proceed by deriving an upper bound for q. From the definition of Ā we deduce that Ā = µ AD, where D = diag For the moment, let us write ( xv 1 µ ) = diag ( x s ) = diag ( µ vs 1 ). r b = θνr 0 b, r c = θνr 0 c Then the system defining q is equivalent to µ AD q = rb, µ DA T ξ + q = 1 µ Dr c. Optimization Group 27

Upper bound for q (2) This implies µ q = q 1 + q 2, where q 1 := (I DA T ( AD 2 A T ) ) 1 AD Dr c, q 2 := DA T ( AD 2 A T ) 1 rb. Let (y, s ) be such that A T y + s = c and x such that Ax = b. Then r b = θνr 0 b = θν(b Ax0 ) = θνa(x x 0 ) = θνad ( D 1 ( x x 0)) r c = θνr 0 c = θν ( c A T y 0 s 0) = θν ( A T (y y 0 ) + s s 0). Since DA T (y y 0 ) belongs to the row space of AD, we obtain q 1 θν D ( s s 0), q2 θν D 1 ( x x 0). Since µ q = q 1 + q 2 and q 1 and q 2 are orthogonal, we may conclude that where, as always, µ = µ 0 ν. µ q θν D ( s s 0) 2 + D 1 ( x x 0) 2, We are still free to choose x and s, subject to the constraints Ax = b and A T y +s = c. Optimization Group 28

Upper bound for q (3) µ q θν D ( s s 0) 2 + D 1 ( x x 0) 2 Let x be an optimal solution of (P) and (y, s ) of (D) (assuming that these exist). Then x and s are complementary, i.e., x s = 0. Let ζ be such that x +s ζ. Starting the algorithm with x 0 = s 0 = ζe, y 0 = 0, µ 0 = ζ 2, the entries of the vectors x 0 x are s 0 s satisfy 0 x 0 x ζe, 0 s 0 s ζe. Thus it follows that D ( s s 0) 2 + D 1 ( x x 0) 2 ζ De 2 + D 1 e 2 = ζ e T ( x s + s x ). Substitution gives µ q θν ζ e T ( x s + s x ). Optimization Group 29

Bounds for x and s; choice of τ µ q θν ζ e T ( x s + s x Recall that x is feasible for (P ν ) and (y, s) for (D ν ) and, moreover δ(x, s; µ) τ. Based on his information we need to estimate the sizes of the entries of the vectors x/s and s/x. Since the concerning result does not belong to the mainstream of the analysis we mention it without proof. If ). and δ τ then we have x s 2 x(µ, ν) µ, τ = 1 8 s x 2 s(µ, ν) µ. Optimization Group 30

Substitution gives This implies µ q θν ζ Upper bound for q (4) e T ( x s + s x ), x s x(µ, ν) 2, µ µ q θν ζ 2e T x(µ, ν)2 s(µ, ν)2 +. µ µ µ q θνζ 2 s x s(µ, ν) 2. µ x(µ, ν) 2 + s(µ, ν) 2. Therefore, also using µ = µ 0 ν = ζ 2 ν and θ = α, we obtain the following upper bound 2n for the norm of q: q α ζ x(µ, ν) 2 + s(µ, ν) 2. n Optimization Group 31

We define and κ(ζ, ν) = q α ζ n Choice of α x(µ, ν) 2 + s(µ, ν) 2. x(µ, ν) 2 + s(µ, ν) 2 ζ, 0 < ν 1, µ = µ 0 ν 2n κ(ζ) = max 0<ν 1 Then q can be bounded above as follows. κ(ζ, ν). q α κ(ζ) 2. We found in () that in order to have δ(v f ) 1/ 2, we should have q 2 +( q + 2δ) 2 1. Therefore, since δ τ = 1 8, it suffices if q satisfies q 2 + ( q + 1 4) 2 1. So we certainly have δ(v f ) 1/ 2 if q 1 2. Since q α κ(ζ) 2, the latter inequality is satisfied if we take 1 α = 2 2 κ(ζ). Note that since x(ζ 2,1) = s(ζ 2,1) = ζe, we have κ(ζ,1) = 1. As a consequence we obtain that κ(ζ) 1. We can prove that κ(ζ) 2n. Optimization Group 32

Bound for κ(ζ) (1) We have Ax = b, 0 x ζe A T y + s = c, 0 s ζe x s = 0. To simplify notation, we denote x = x(µ, ν), y = y(µ, ν) and s = s(µ, ν). Then b Ax = ν(b Aζe), x 0 c A T y s = ν(c ζe), s 0 xs = νζ 2 e. This implies Ax Ax = ν(ax Aζe), x 0 A T y + s A T y s = ν(a T y + s ζe), s 0 xs = νζ 2 e. Optimization Group 33

Bound for κ(ζ) (2) Ax Ax = ν(ax Aζe), x 0 A T y + s A T y s = ν(a T y + s ζe), s 0 xs = νζ 2 e. We rewrite this system as A(x x νx + νζe) = 0, x 0 A T (y y νy + νζe) = s s + νs νζe, s 0 xs = νζ 2 e. From this we deduce that ( x x νx + νζe ) T ( s s + νs νζe ) = 0. Define a := (1 ν)x + νζe, b := (1 ν)s + νζe. We then have a νζe, b νζe and (a x) T (b s) = 0. The latter gives a T b + x T s = a T s + b T x. Optimization Group 34

Bound for κ(ζ) (3) Since x T s = 0, x + s ζe and xs = νζ 2 e, we may write a T b + x T s = ((1 ν)x + νζe) T ((1 ν)s + νζe) + νζ 2 n Moreover, also using a νζe, b νζe, we get = ν(1 ν)(x + s ) T ζe + ν 2 ζ 2 n + νζ 2 n ν(1 ν)ζ 2 n + ν 2 ζ 2 n + νζ 2 n = 2νζ 2 n. a T s + b T x = ((1 ν)x + νζe) T s + ((1 ν)s + νζe) T x = (1 ν) ( s T x + x T s ) + νζe T (x + s) νζ ( s 1 + x 1 ). Hence s 1 + x 1 2ζn. Since x 2 + s 2 ( ) s 1 + x 2 1, it follows that x 2 + s 2 ζ 2n thus proving κ(ζ) 2n. s 1 + x 1 ζ 2n 2ζn ζ 2n = 2n, Optimization Group 35

Complexity analysis We have found values for the parameters in the algorithm such that if at the start of an iteration the iterates satisfy δ(x, s; µ) τ = 1 8, then after the feasibility step the iterates satisfy δ(x, s; µ + ) 1/ 2. At most ( ) 1 log 2 log 2 τ 2 = log 2 (log 2 64) 3 centering steps suffice to get iterates that satisfy δ(x, s; µ + ) τ. So each (main) iteration consists of at most 4 so-called inner iterations, in each of which we need to compute a new search direction. In each iteration both the duality gap and the residual vectors are reduced by the factor 1 θ. Hence, using x 0T s 0 = nζ 2 and defining K := max { nζ 2,, }, the total number of main iterations is bounded above by 1 θ log K 2n ǫ = α log K ǫ = 4 κ(ζ) 2n log K ǫ. The total number of inner iterations is therefore bounded above by where κ(ζ) 2n. 16 κ(ζ) 2n log max { nζ 2, r 0 b ǫ, r 0 c }, r 0 b r 0 c Optimization Group 36

Conjecture Conjecture 1 If (P) and (D) are feasible and ζ x + s for some pair of optimal solutions x and (y, s ), then κ(ζ) = 1. Evidence was provided by a simple Matlab implementation of our algorithm. As input we used a primal-dual pair of randomly generated feasible problems with known optimal solutions x and (y, s ), and ran the algorithm with ζ = x + s. This was done for various sizes of the problems and for at least 10 5 instances. No counterexample for the conjecture was found. Typically the graph of κ(ζ, ν), as a function of ν is as depicted below. κ(ζ, ν) 1.0 0.5 0 0 0.5 1.0 Typical behavior of κ(ζ, ν) as a function of ν. The importance of the conjecture is evident. Its trueness would reduce the currently best iteration bound for IIPMs by a factor 2n. ν Optimization Group 37

Related conjecture Conjecture 2 If (P) and (D) are such that (x 0, y 0, s 0 ) = ζ(e,0, e) is a feasible triple, whereas ζ x + s for some optimal triple (x, y, s ), then x(µ) 2 + s(µ) 2 ζ 1, 0 < µ ζ 2. 2n In other words, x(µ) 2 + s(µ) 2 2nζ 2, 0 < µ ζ 2. Optimization Group 38

Concluding remarks The techniques that have been developed in the field of feasible IPMs, and which have now been known for almost twenty years, are sufficient to get an IIPM whose theoretical performance is as good as the currently best known theoretical performance of IIPMs. Following a well-known metaphor of Isaac Newton a, it looks like if a smooth pebble or pretty shell on the sea-shore of IPMs has been overlooked for a surprisingly long time. The full Newton step method presented in this paper is not efficient from a practical point of view. But just as in the case of feasible IPMs one might expect that largeupdate methods for IIPMs can be designed whose complexity is not worse than n times the iteration bound in this paper. Even better results for large-update methods might be obtained by changing the search direction, by using methods that are based on kernel functions. This requires further investigation. Also extensions to secondorder cone optimization, semidefinite optimization, linear complementarity problems, etc. seem to be within reach. a I do not know what I may appear to the world; but to myself I seem to have been only like a boy playing on the sea-shore, and diverting myself in now and then finding a smoother pebble or a prettier shell than ordinary, whilst the great ocean of truth lay all undiscovered before me (Westfall). Optimization Group 39

Some references M. Kojima, N. Megiddo, and S.Mizuno. A primal-dual infeasible-interior-point algorithm for linear programming. Mathematical Programming 61, pages 263 280, 1993. M. Kojima, T. Noma, and A. Yoshise. Global convergence in infeasible-interior-point algorithms. Mathematical Programming, Series A, 65:43 72, 1994. I. J. Lustig. Feasible issues in a primal-dual interior-point method. Mathematical Programming 67, pages 145 162, 1990/91. S. Mizuno. Polynomiality of infeasible-interior-point algorithms for linear programming. Mathematical Programming 67, pages 109 119, 1994. F. A. Potra. A quadratically convergent predictor-corrector method for solving linear programs from infeasible starting points. Mathematical Programming, 67(3):383 404, 1994. F. A. Potra. An infeasible-interior-point predictor-corrector algorithm for linear programming. SIAM Journal on Optimization, 6(1):19 32, 1996. C. Roos. A full-newton step O(n) infeasible interior-point algorithm for linear optimization. SIAM Journal on Optimization, 16(4):1110 1136, 2006. C. Roos, T. Terlaky, and J.-Ph. Vial. Interior-Point Methods for Linear Optimization. Springer, 2005. (2nd edition of Theory and Algorithms for Linear Optimization. An Interior-Point Approach, John Wiley & Sons, Chichester, UK, 1997) M. Salahi, T. Terlaky, and G. Zhang. The complexity of self-regular proximity based infeasible IPMs. Technical Report 2003/3, Advanced Optimization Laboratory, Mc Master University, Hamilton, Ontario, Canada, 2003. R.J. Vanderbei. Linear Programming: Foundations and Extensions. Kluwer Academic Publishers, Boston, USA, 1996. S.J. Wright. Primal-Dual Interior-Point Methods. SIAM, Philadelphia, 1996. Y. Ye. Interior Point Algorithms, Theory and Analysis. John Wiley and Sons, Chichester, UK, 1997. Optimization Group 40