Central path curvature and iteration-complexity for redundant Klee-Minty cubes

Transcription

1 Central path curvature and iteration-complexity for redundant Klee-Minty cubes Antoine Deza, Tamás Terlaky, and Yuriy Zinchenko Advanced Optimization Laboratory, Department of Computing and Software, McMaster University, Hamilton, ON L8S 4K, Canada. Summary. We consider a family of linear optimization problems over the n- dimensional Klee-Minty cube and show that the central path may visit all of its vertices in the same order as simplex methods do. This is achieved by carefully adding an exponential number of redundant constraints that forces the central path to take at least n sharp turns. This fact suggests that any feasible path-following interior-point method will take at least O n iterations to solve this problem, while in practice typically only a few iterations, e.g., 5, suffices to obtain a high quality solution. Thus, the construction potentially exhibits the worst-case iterationcomplexity known to date which almost matches the theoretical iteration-complexity bound for this type of methods. In addition, this construction gives a counterexample to a conjecture that the total central path curvature is On. Key words: linear programming, central path, interior-point methods, total curvature. Introduction Consider the following linear programming problem: minc T x such that Ax b where A R m n, b R m and c, x R n. In theory, the so-called feasible path-following interior-point methods exhibit polynomial iteration-complexity: starting at a point on the central path they take at most O mlnν iterations to attain ν-relative decrease in the duality gap. Moreover, if L is the bit-length of the input data, it takes at most O ml iterations to solve the problem exactly, see for instance []. However, in practice typically only a few iterations, usually less than 5, suffices to obtain a high quality solution. This remarkable difference stands behind the tremendous success of interior-point methods in applications. Let ψ : [α, β] R n be a C map with the non-zero derivative t [α, β]. Denote its arc length by

2 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes t lt := ψτ dτ, α its parametrization by the arc length by ψ arc l : [, lβ] R n, and its curvature at the point l The total curvature K is defined as κl := d dl ψ arc l. K := lβ κl dl. Intuitively, the total curvature is a measure of how far off a certain curve is from being a straight line. Thus, it has been hypothesized that the total curvature of the central path is positively correlated with the number of iterations that any Newton-like path following method will take to traverse this curve, in particular, the number of iterations for feasible path-following interior-point methods, e.g., long-step or predictor-corrector. The worst-case behavior for path-following interior-point methods has been already under investigation, for example, Todd and Ye [] gave a lower iteration-complexity bound of order 3 m necessary to guarantee a fixed decrease in the central path parameter and consequently in the duality gap. At the same time, different notions for the curvature of the central path have been examined. The relationship between the number of approximately straight segments of the central path introduced by Vavasis and Ye [3] and a certain curvature measure of the central path introduced by Sonnevend, Stoer and Zhao [] and further analyzed in [4], was further studied by Monteiro and Tsuchiya in [8]. Dedieu, Malajovich and Shub [] investigated a properly averaged total curvature of the central path. Nesterov and Todd [9] studied the Riemannian curvature of the central path in particular relevant to the so-called short-step methods. We follow a constructive approach originated in [3, 4] which is driven by the geometrical properties of the central path to address these questions. We consider a family of linear optimization problems over the n-dimensional Klee-Minty cube and show that the central path may visit all of its vertices in the same order as simplex methods do. This is achieved by carefully adding an exponential number of redundant constraints that forces the central path to take at least n sharp turns. We derive explicit formulae for the number of the redundant constraints needed. In particular, we give a bound of On 3n on the number of redundant constraints when the distances to those are chosen uniformly. When these distances are chosen to decay geometrically, we give a slightly tighter bound of the same order n 3 n as in [4]. The behavior of the central path suggests that any feasible path-following interior-point method will take at least order n iterations to solve this problem. Thus, the construction potentially exhibits the worst-case iterationcomplexity known to date which almost matches the theoretical iterationcomplexity bound for this type of methods. However, state-of-the art linear

3 Advances in Mechanics and Mathematics, Vol. IV 3 optimization solvers that include preprocessing of the problem as described in [5, 6] are expected to recognize and remove the redundant constraints in no more than two passes. This underlines the importance of the implementation of efficient preprocessing algorithms. We show that the total curvature of the central path for the construction is at least exponential in n and, therefore, provide a counterexample to a conjecture of Dedieu and Shub [] that it can be bounded by On. Also, the construction may serve as an example where one can relate the total curvature and the number of iterations almost exactly. The paper is organized as follows: in Section we introduce a family of linear programming problems studied along with a set of sufficient conditions that ensure the desired behavior for the central path and give a lower bound on the total curvature of the central path, in Section 3 we outline the approach to determine the number of the redundant constraints required, Sections 4 and 5 contain detailed analysis for the two distinct models for the distances to the redundant constraints. We give a brief conclusion in Section 6. Sufficient conditions for bending the central path and the total curvature Let x R n. Consider the following optimization problem min x n x εx k x k εx k k =,...,n d + x repeated h times εx d + x repeated h times. εx n d n + x n repeated h n times. The feasible region is the Klee-Minty n-cube and is denoted by C R n. Denote d := d,..., d n R n + the vector containing the distances to the redundant constraints from C, h := h,...,h n N n the vector containing the number of the redundant constraints. By analogy with the unit cube [, ] n, we denote the vertices of C as follows: for S {,...,n}, a vertex v S of C satisfies { v S if S = otherwise { εv vk S = S k if k S εvk S otherwise k =,...,n. Define δ-neighborhood N δ v S of a vertex v S, with the convention x =, by

4 4 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes { { } N δ v S xk εx := x C : k ε k δ if k S x k εx k ε k k =,...,n. δ otherwise Remark. Observe that S {,..., n} for N δ v S to be pairwise-disjoint it suffices ε + δ < /: given ε, δ >, the shortest amongst all n coordinates distance between the neighborhoods, equal to ε εδ, is attained along the second coordinate and must be positive, which is readily implied. For brevity of the notation we introduce slack variables corresponding to the constraints in the problem above as follows: s = x s k = x k εx k s = x s k = εx k x k s = d + x s k = d k + x n εx n k =,...,n k =,...,n k =,...,n. Recall that the analytic center χ corresponds to the unique maximizer arg max x n lns i + ln s i + h i ln s i. i= Also, recall that the primal central path P can be characterized as the closure of the set of maximizers { } n x R n : x = arg max lns i + ln s i + h i ln s i, for some α, χ n. x:x n=α i= Therefore, setting to the derivatives of n i= lns i + ln s i + h i ln s i with respect to x n, s + h n =, s n n s n and with respect to x k, s k ε s k+ s k ε s k+ + h k s k εh k+ s k+ =, k =,...,n, combined together give us necessary and sufficient conditions for x = χ. Further, combined with x n = α, χ n gives us necessary and sufficient conditions for x P \ {} {χ} where R n denotes the origin. Given ε, δ >, the sufficient conditions for h = hd, ε, δ to guarantee that the central path P visits the disjoint δ-neighborhoods of each vertex of C may be summarized in the following proposition. We write for the vector of all ones in R n.

5 Advances in Mechanics and Mathematics, Vol. IV 5 Proposition. Fix ε, δ >. Denote for k =,...,n and and I k δ := {x C : s k ε k δ, s k ε k δ} B k δ := {x C : s k ε k δ, s k ε k 3 δ,...,s δ}. If h = hd, ε, δ N n satisfies Ah 3 δ 3 h k d k + εk h k+ ε k + 3, k =,...,n, 4 d k+ δ where then A = d + d ε d ε d + ε d ε d k ε k d k + d k ε d n d n + ε n d n ε d n d n+ I k δ P Bk δ. Proof. Fix k and let x I k δ P. Let j k. Summing up all of the i th equations of over i = j,..., k, each multiplied by ε i, and then subtracting the k st equation multiplied by ε k, we have h k ε k s k + h jε j s j + h kε k s k + εj s j + εk s k + εk s k = εk + εj k 3 ε i +. s k s j s i=j i+ Since s k < d k +, s k > d k, s j > d j, and s k ε k δ, s k ε k δ as x Iδ k, from the above we get From 4 it follows that h h k ε k d k + h jε j h kε k d j d k d hjεj d j εj s j + δ., thus we can write

6 6 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes h d + h k ε k d k + h kε k d k that is, as 3 δ h d + h k ε k d k + h kε k d k εj s j + δ, by 3, we have s n j ε j δ, j k. In turn, the k st equation of, implies h k ε k s k h kε k s k and since 3 δ h k ε k d k + h kε k d k = εk + εk + εk εk s k s k s k s k h k ε k d k + h kε k εk + d k s k δ by 4, we have s k ε k δ. Proposition. Fix ε, δ >. If h N n satisfies 3 and 4, then χ N δ v {n}. Proof. Summing up all of the i th equations of over i = k,...,n, each multiplied by ε i, and then subtracting multiplied by ε n, we have implying ε k s k εk s k + h kε k s k h n ε n s n εn n ε i h nε n = s n s i+ s n i=k h kε k s k Since s n d n +, s k d k and by 4, h combined with hnεn d n+ εk s k. d h kε k d k h n ε n d n + h d εk s k, h d 3 δ from 3 this leads to s k εk δ k =,...,n. 3, from the above we get In turn, implies hnεn s n εn s n. And since s n < d n + and by 4, d 3 n+ δ, we have 3 δ εn, that is s n εn 3 δ. h nε n s n Corollary. Fix ε, δ > such that ε + δ < /. If h N n satisfies 3 and 4, then the central path P intersects the disjoint δ-neighborhoods of all the vertices of C. Moreover, P is confined to a polyhedral tube defined by T δ := n k= n j=k+ Ijδ c B k δ with the convention Bδ = C.

7 Advances in Mechanics and Mathematics, Vol. IV 7 Remark. Observe that T is the sequence of connected edges of C starting from v {n} and terminating at v, and is precisely the path followed by the simplex method on the original Klee-Minty problem as it pivots along the edges of C. For simplicity of the notation we will write T instead of T δ when the choice of δ is clear. For a fixed δ, we define a turn of T adjacent to a vertex v S, or corresponding to N δ v S if the δ-neighborhoods are disjoint since in the later case N δ v S determines v S uniquely, to be the angle between the two edges of C that belong to T and connect at this vertex. Intuitively, if a smooth curve is confined to a narrow tube that makes a sharp turn, then the curve itself must at least make a similar turn and thus have a total curvature bounded away from zero. It might be worthwhile to substantiate this intuition with a proposition. Proposition 3. Let Ψ : [, T] R be C, parameterized by its arc length t, such that Ψ[, T] {x, y : x a + b, y b} {x, y : a x a + b, a y b} and Ψ {} [, b], ΨT [a, a+ b] { a}. Then the total curvature K of Ψ satisfies K arcsin b /a. Proof. By Mean-Value Theorem, for any τ such that Ψ τ = a we have τ a; recall that Ψ =. Thus, by the same theorem, t such that Ψ t b/a. Similarly, t such that Ψ t b/a. Now map the values of the derivative of Ψ at t and t onto a sphere and recall that the total curvature K between these two points corresponds to the length of a connecting curve on the sphere, thus bounded below by the length of the geodesic which in this case is the same as the angular distance. Fig.. Total curvature and geodesics. A simple calculation completes the proof.

8 8 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes Remark 3. Note that if b/a, then the corresponding lower bound on the total curvature K approaches π/. Next we construct a simple bound on the total curvature of P by picking suitable d, ε and finally δ small enough, together with h, that results in a narrow polyhedral tube T. For X R n denote its orthogonal projection onto a linear subspace spanned by a subset S {,..., n} of coordinates, with coordinates corresponding to S c suppressed, by X S. For x, z R n we denote x, z the straight line segment connecting the point x and z. Corollary. Fix n. If d i = n n i+, i =,..., n, ε = n n, and h satisfies δ = 3n i 4 5 n, h = + δ n 3 max a ij 3. δ h j= where A h =, then the total curvature of the central path P satisfies K n n 8 5 Proof. That ε, δ, h = hd, ε, δ above satisfies the conditions of Corollary and thus P is confined to the polyhedral tube T is established in Section 5. Instead of analyzing P R n directly we will derive the lower bound on the total curvature of P based on its planar projection P {,}. From I k δ P Bk δ, k =,...,n, it follows that P {,} will traverse the two-dimensional Klee-Minty cube C {,} at least n times, every time originating in either N δ v {,} or N δ v {} {,} and terminating in the other neighborhood, while confined to the polyhedral tube T {,} = {s εδ} { s δ} { s εδ} C {,}. Thus, P {,} will make at least n sharp turns, each corresponding to a turn in N δ v {,} {,} or N δ v {} {,}. In order to understand how the turns of P {,} contribute to the total curvature of P we need the following lemma. Lemma. Let û, v R 3 and u = û {,},, v = v {,},. If the angle ǫ := π arccosarg min bw span{bu,bv}, w span{u,v}, bw = w = ŵ T w

9 Advances in Mechanics and Mathematics, Vol. IV 9 Fig.. Planar projection of the central path for n = 3. between the hyperplane spanned by û, v and the hyperplane spanned by u, v does not exceed arcsinε, then the angle α between û and v satisfies cosα ε cos α + ε cos α cos α cosα + ε +cos α + ε +cos α where α is the angle between u and v. Proof. Without loss of generality we may assume u = v = with

10 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes cos u = sin α = α, v = sin α = cos α, +cos u = cos α = α, v = cos α = +cos α, and, assuming that the angle ǫ is precisely arcsin ε, parameterize span{û, v} by span{u, v} and z = z, z, such that z =, writing x span{û, v} as x = x, x, x T {,} z {,}ε. Introducing β such that z = cosβ and z = sin β we have and, therefore, cos α = cosα + cosα û =, cosα + cosα v =, ût v û v =, ε cos β π + α,, ε cos β π + α, cosα + ε cos β π + α cos β π α + ε cos β π + α + ε cos β π α. Denoting γ := β π and differentiating the above with respect to γ we get cos α γ = + ε 3ε sin γ + 6ε sinγ + α + 6ε sinγ α, D where D = 6 + 8ε cosγ α + 6ε + 8ε cosγ + α + ε 4 cosα +ε 4 cos4γ + 4ε 4 cosγ + α + 4ε 4 cosγ α + 4ε 4 3/. Setting the derivative to and simplifying the enumerator we obtain the necessary condition for the extremum of cos α, 3ε + ε sin γcosα =. That is, γ = k π for k =, ±, ±, and so on. In particular, it follows that the minimum of cos α is attained at β min = and the maximum is attained at β max = π. The bounds are obtained by further substituting the critical values of β into the expression for cos α and observing the monotonicity with respect to ε. Although the full-dimensional tube T might make quite wide turns, the projected tube T {,} is bound to make the same sharp turn equal to π + arcsin ε +ε each time T passes through the δ-neighborhood of a vertex v S, S e.g., consider the turn adjacent to v {,3} for n = 3.

11 Advances in Mechanics and Mathematics, Vol. IV For a moment, equip C and T with a superscript n to indicate the dimension of the cube, i.e., the largest number of linearly independent vectors in span{v S : v S C n }. Recalling the C n defining constraints, namely εx n x n εx n, we note that by construction of the Klee-Minty cube, whenever we increase the dimension from n to n+, C n is affinely transformed into top and bottom n-dimensional faces Ftop n+ n+ and Fbottom of C n+, that is Ftop n+ I =,...,, ε F n+ C n + bottom = I,...,, ε., where I is the identity n n matrix, and C n+ is the convex hull of Ftop n+ and F n+. Consequently, any -dimensional space spanned by two connected bottom F n+ top or T n+ F n+ edges of C n+ from T n+ bottom -dimensional space spanned by the two corresponding edges of C n from T n C n is obtained by tilting the lifted to R n+ by setting the n + st coordinate to zero, by an angle not exceeding arcsin, and moreover, not exceeding arcsinε. Therefore, we ε +ε are in position to apply Lemma to bound how fast cosine of a turn α S of T n adjacent to any v S C n with S may approach its two boundary values of or by induction on the dimension n. Fixing n = 3, S {,, 3} such that S, adding and subtracting to cosα S we get and + cosα S cosα S + cosα S {,} + ε cos α S {,} cosα S {,} + ε +cos α S {,} Furthermore, for any n 3 and v S with S we can write + cosα S + cosαs {,} + ε n. ε + ε n, cosα S cosαs {,} + ε n + ε 5 + ε n, recalling ε 5 cosα S {,} = ε +ε ε since ε. Observe that by construction of a polyhedral tube T, a single linearlyconnected component of T \ S {,...,n} N δv S may be uniquely identified with an edge v R, v S, R, S {,..., n}, of C from T by having,

12 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes a non-empty intersection with this component and thus we denote such a component by L vr,v S and refer to it as a section of T corresponding to v R, v S. Moreover, recalling the definition of N δ v S and T, and noting that δ + εδ + εδ 3 + δ+εδ+ δ since ε /, we get that within a given section of a tube L v R,v S the Euclidean distance from x L v R,v S to the compact closure of v R, v S L v R,v S is bounded from above by δ. Let us consider what happens to the central path in the proximity of a vertex v S C such that S. We will do so by manufacturing a surrogate for a part of T that is easier to analyze. Fix v S C with S and denote the two adjacent vertices to which v S is connected by the two edges from T by v R and v Q. Without loss of generality we may assume that v{,} R =,, =, ε, v S {,} v Q {,} =, ε, and vn R > vs n > vq n, so that the central path P enters the part of the polyhedral tube T sectioned between these three vertices via N δ v R and exits via N δ v Q. Fig. 3. Schematic drawing for the cylindrical tube segments. Define four auxiliary points x, z v R, v S and x, z v S, v Q satisfying

13 Advances in Mechanics and Mathematics, Vol. IV 3 x {,} = 3δ, ε + 3εδ + / ε 3δ +ε, ε, z {,} = 3δ, ε + 3εδ, x {,} =, ε 3δ, z {,} =, /. Since the distance from any point to the part of the identifying edge of L vr,v S or L vs,v Q is no greater than δ and since {,} corresponds to the orthogonal projection from R n onto its first two coordinates, we can define two cylindrical tube segments T := {x R n : min z x,z x z δ} {x R n : x z T x x z T x} {x R n : z x T x z x T z} and T := {x R n : min z x,z x z δ} {x R n : x z T x x z T x} {x R n : z x T x z x T z} such that T L v R,v S {x R n : x z T x x z T x, z x T x z x T z}, T L v S,v Q {x R n : x z T x x z T x, z x T x z x T z}, and T N δ v R N δ v S = T N δ v S N δ v Q =. Therefore, P will traverse T and T, first entering T through its face corresponding to x z T x = x z T x and exiting through the face corresponding to z x T x = z z T z, and then entering T at a point with x z T x = x z T x and exiting through a point with z x T x = z x T z. Now we choose a new system of orthogonal coordinates in R n that will allow us apply the argument similar to that of Proposition 3 as follows. Let the first two coordinates correspond to the linear subspace spanned by x, z and x, z; align the second coordinate axis with the vector z, x, so that the vector x, z forms the same angle equal to α S with the second coordinate axis as with x, z. Choose the rest n coordinates so that they form an orthogonal basis for R n. Consider parametrization of P by its arc length, P arc. Since the shortest distance between the two parallel faces of T that correspond to {x R n : x z T x = x z T x} and {x R n : z x T x = z x T z} is equal to x, z = / ε 3δ, by Mean-Value Theorem it takes at least / ε 3δ change of the arc length parameter for P arc to traverse T. Noting that while traversing the tube T the second coordinate of P arc might change at most by δ sin α S + / ε 3δcosα S, by the same theorem we deduce that t such that

14 4 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes Ṗarc t δ sinαs + / ε 3δcosα S cosα S + / ε 3δ 4δ / ε 3δ. Analogously, considering T along the i th coordinate with i we conclude that i, t i such that 4δ Ṗarc t i i / ε 3δ. We use the points t, t,..., t n to compute a lower bound on the total curvature contribution of a turn of P next to v S : recalling Ṗarc =, the total curvature of the part of P that passes through T and T, i.e., resulting from a turn of T adjacent to v S, may be bounded below by the length of the shortest curve on a unit n-sphere that connects points Ṗarct, Ṗarct,...,Ṗarct n in any order. For simplicity, the later length may be further bounded below by K S := min x i R n, i=,...,n: max j x i =, i, x cos αs 4δ + / ε 3δ, x j j 4δ / ε 3δ, j min max x i R n, i=,...,n: j x =, x cos αs 4δ + / ε 3δ, x j j 4δ / ε 3δ, j distx, x j x x j, where distx, z is the length of the shortest curve on a unit sphere between points x and z, i.e., the geodesic. Clearly, the critical value for the last expression is attained, in particular, at x i R n +, i, when x =, x x j = x x i, i, j, and x = cosαs 4δ + / ε 3δ, xj j = 4δ / ε 3δ, j. It follows that n x j = j= cosα S + and, since cosα S resulting in j= x j x j n n 4δ cosα S / ε 3δ ε +ε n, n x ε j + ε n 4δ / ε 3δ, ε + ε n 4δ / ε 3δ + /4 n n 4δ / ε 3δ 4δ / ε 3δ, j.

15 Advances in Mechanics and Mathematics, Vol. IV 5 Therefore, recalling ε = n n and δ = 3n 4 5 n, we can write K S x x x x n 4 + 4δ n 5 n / ε 3δ n n 4 n 4 n 5 8n n 5 n 3 4 n 3n 5 n n 4 n 4 n 5 8n n n 5 n 4. 4n 5 Finally, recalling that the polyhedral tube T makes n such turns, we conclude that the total curvature of P indeed satisfies K 8 n. n 5 The bound on the total curvature K of P established above is obviously not tight. We expect the true order of K to be n up to a multiplier, rational in n. Remark 4. In R, by combining the optimality conditions and for the analytic center χ with that of the central path P visiting the δ-neighborhoods of the vertices v {} and v {,} one can show that for δ below a certain threshold both d and d are bounded away from by a constant. In turn, this implies that for fixed feasible d, d, the necessary conditions and for h chosen such that the central path visits the δ-neighborhoods of all the vertices of C are asymptotically equivalent as δ to the sufficient conditions 3 and 4, up to a constant multiplier. Here the term asymptotic equivalence refers to the convergence of the normalized extreme rays and the vertices of the unbounded polyhedra given by the set of necessary conditions for a fixed d to those of the polyhedra given by the set of sufficient conditions 3 and 4. This suggests that the following might be true. In R n min d i ˆd >, i=,...,n where ˆd is independent of n, δ, ε. Moreover, the necessary conditions for P to visit the δ-neighborhoods of all the vertices of C for a fixed d are asymptotically equivalent as δ to the sufficient conditions 3 and 4. If, furthermore, we confine ourselves to only bounded subsets of all such feasible d, h corresponding to, say, n h i Hδ i= := min d,h then the conditions 3 and 4 are tight. Tight in a sense that if we denote the set of all d, h satisfying the necessary conditions for P to visit all the n i= h i

16 6 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes δ-neighborhoods intersected with {h : i h H δ } as Necc δ, the set of all d, h satisfying 3 and 4 intersected with {h : i h H δ } as Suff δ, then for some small enough ˆδ there exists M, m > independent of ε such that Necc mδ Suff δ Necc Mδ, < δ ˆδ. 3 Finding h N n satisfying conditions 3 and 4. We write fn gn for fn, gn : N R if c, C > such that cfn gn Cfn, n, the argument n is usually omitted from the notation. Denote b := 3 δ. Let us first concentrate on finding h Nn such that 3 holds. If the integrality condition on h is relaxed, a solution to 3 can be found by simply solving Ah = b. Note that A, = max i n j= a ij is, in fact, small for d large component-wise and ε < /. So to find an integral h we can solve Aĥ = + γb for some small γ >, set h = ĥ. Observe that for h to satisfy 3, it is enough to require max i Aĥ h γb i. In turn, this can be satisfied by choosing γ > such that γ 3 δ max i n a ij. In Subsection 3. we show how to solve this system of linear equations. In Subsection 3. we demonstrate that under some assumption on d, 4 is already implied by 3, and consequently the rounding of ĥ will not cause a problem for 4 either. Remark 5. The choice of rounding down instead of rounding up is arbitrary. j= 3. Solving the linear system Since + γb = + γ 3 δ, we can first solve A h = and then scale h by + γ 3 δ. Our current goal is to find the solution to A h =. For an arbitrary invertible B R n n and y, z R n such that + z T B y, the solution to B + yz T x = b can be written as where x = B b αy,

17 Advances in Mechanics and Mathematics, Vol. IV 7 α = xt B b + z T B y for writing B + yz T x = Bx + yz T x = b, denoting α := z T x, we can express x = B b αy and substitute this x into B + yz T x = b again to compute α. Denoting B := d + ε ε d + d ε d ε k ε k d k + d k ε n d n + ε n d n ε n d n+ y T := d,,,...,, z T :=,,,...,,, we can compute the solution to B + yz T h A h = as h = B + α B d., 5 where α = n j= B j d n j= B j. 6 So to get the explicit formula for h we need to compute B and show that d can be chosen such that α is well defined, i.e., n d j= B j. In order to invert B first note that it satisfies B = Diag d +, ε d +, ε εn..., I + S, d 3 + d n + where a super-diagonal matrix S R n n is such that { εdi+ S ij = d i+ j = i +, i =,...,n otherwise.

18 8 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes Recall that I + Z = I Z + Z Z for any Z R n n such that these matrix-power series converge. In our case, the powers of S are easy to compute for k n, S k ij = { i+k l=i S l,l+ j = i + k, i =,...,n k otherwise, and S m = for all m n, so the inverse of I + S can be computed as above and the inverse of B can be further computed by post-multiplying by the inverse of Diag d, ε + d, ε + d..., εn 3+ d n+. Therefore, B is equal to d + d+d+ d d +d +d 3+ 4d d 3 d + d 4+ 8d d 3d 4 d + ε d +d 3+ 4d 3ε d + d 4+ 8d 3d 4ε d 3+ ε d 3+d 4+ 4d 4ε d 4+ Q n j= dj+ Q n n Q j= dj n j= dj+ n ε Q n Q j=3 d j n j=3 dj+ n ε Q n Q j=4 dj n j=4 dj+ ε Partial implication for sufficient conditions n 3 ε 3 Q n j=5 d j d n+ ε n. Observe that in order for h R n +, we must have α > as in 6. For if not, i.e., if α <, then denoting β := z T B. = n B j j= and writing α = β + d + β d 7 we must have β > d >. So α d = β+d+ β d > and from 5 it follows that if α d <, then h, h 3,..., h n <. From now on we assume α > in Subsections 4., 5. we show how to achieve this by choosing d appropriately. Note that in this case α d >. Suppose h N n is such that 3 holds. If, furthermore, hiεi d i is dominated by h d for i =,...,n, then 3 already implies 4. Therefore, it is left to show that d can be chosen such that h = ĥ satisfies the domination condition above.

19 For this to hold it suffices 3 δ + γ h d Advances in Mechanics and Mathematics, Vol. IV 9 3 δ + γ h i + d i ε i, i =,...,n, where h solves A h =. The above is implied by d + + β + α d β 6 d + α d βi ε i + 6 d i ε i, i =,...,n since γ >, δ < /, where β i := ε i B i. This can be written as + α β α βi + εi, d d 6d d d i 6d i i =,...,n. In particular, if we have then the above inequality holds true if β d > β i d i, i =,...,n 8 ε 6d 5 6d, that is, since ε < /, for d i > for i =,...,n. Finally, observe that if d d i, i and d = O n, then the magnitude of h is primarily determined by α: recalling 5, 7, we write h = α α = α B d B d B d. + B d. + d +. + β d B β B d d +. + β d d +..

20 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes Since d i > for i =,..., n, we have di+ d i h < α α B d.. + for large n.. B d B n d + >.. β and so d <, implying n d + +. n. 9 4 Uniform distances to the redundant hyperplanes Clearly, many different choices for d are possible. In this section we explore the uniform model for d, that is d = d = = d n. 4. Bounding α For α > we need Denoting the above can be written as β < d. ξ := d + d or, equivalently, d + ξ + d + ξ + + d + ξ n < d same as ξ + ξ + ξ + + ξ n < ξ ξ n < ξ. In other words, we want pξ := ξ ξ + ξ n+ >. Note that as d, ξ and pξ ξn+ >, so pξ > for d large enough. Observe that ξ > for any d >, and

21 Advances in Mechanics and Mathematics, Vol. IV so p = 3 + n+ < for n =,,... n p = 4 + n+n n p k = n+! n k n k+! = for n = > for n =, 3, 4 < otherwise > for k 3, ξ p + ξ p + ξ := p + p ξ + p. Thus, to guarantee pξ >, it is enough to require p + ξ letting ξ = / + ξ. Denoting nn + ã := + n n + b := 3 + n c := n and ξ b+ e eb 4eaec := ea, n 4 b e eb 4eaec ea, n > 4 the smallest positive root of p/ + ξ =, we conclude that β < d as long as ξ = d + d + ξ. That is d = ξ ξ. Note that for d = d = = d n, 8 holds, so 4 is readily implied by 3. It is left to demonstrate how to choose d satisfying the above to guarantee a moderate growth in h as n.

22 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes 4. Picking a good d Note that as n, we have ξ 3 n by expanding the square root in as first order Taylor series. Also, and hence β d = pξ pξ β pξ. d In fact, for large n, pξ pξ, for / ξ / + ξ. In turn, pξ is almost linear on this interval since, as n, b 3 c ξ we compare the slope of p/ + ξ at ξ = with its decrement as a function of ξ over [, ξ ]. Recalling that the growth of h is primarily determined by α for large n see 9, our goal becomes to minimize α. From 7,, noting that β d for large n also recall β < d, we get α β + d + pξ d pξ ξ c c ξ ξ and, moreover the right-hand side of this expression approximates α fairly well. So, to approximately minimize h, we maximize ξ c c ξ ξ for ξ ξ, which corresponds to setting thus resulting in and α 6 n ξ = ξ as n n hi = O ε i, i =,..., n.

23 4.3 An explicit formula Advances in Mechanics and Mathematics, Vol. IV 3 For a given n, ε, δ, compute ξ according to, set ξ = ξ, d as in. Compute the solution to A h = using 5, where α is computed according to 6. Set γ = δ 3 max n i j= a ij and, finally, h = + γ 3. δ h From 9 it follows that for large n hi α B d. α ε i, i =,...,n. We are interested in picking ε, δ, to minimize the total number of the redundant constraints, n i= h i. Recalling ε + δ < /, denoting for large n we can write gε := εε n 3ε + ε n i= h i 3 n h δ i= i 3 n δ i= α ε 6 n 6 ε n ε ε = 9 n+ gε i it is natural to pick δ as close to ε as possible, say δ =.999 ε. In order to bound g := min gε <ε< we first bound the minimizer ε of the above, next we bound the derivative of gε, and finally using the Mean-Value Theorem we bound g itself. Observing g ε = 3ε + ε nε n 4ε 3εε n 3ε + ε = ε n εn + ε3 4εε n + ε n + + ε n ε ε 3 4ε n = ε n ε ε + n + εn 3 4ε + εn + ε n + + ε and noting that in the expression above the second and the third summands are monotone-increasing functions of ε for ε, /, n, for n 3 we can write ε ε L, ε U with

24 4 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes since for n we have ε L := n 5/4 n ε n ε n + ε3 4εε n + ε n + + ε=ε L 4n + 5n 5 + 6n 4n + 5 = 4n4n + 5 n 5/4 n n <, where the first summand in brackets has no real roots with respect to n and the second summand is negative for n, so g ε L < and thus g ε < for < ε < ε L, and ε U := n n since for n 3, ε n ε n + ε3 4εε n + ε n + + ε=ε U = n n n + n n nn + n = n n n + 3 n n nn + > n nn + n + 4 n >, n + n n n so g ε U > and thus g ε > for ε U < ε < /. Consequently, for ε ε L, ε U we have n g n ε ε min ε [ε L,ε U ] ε n ε ε = n3 8n n + 4n 5 and, therefore, by the Mean-Value Theorem, i.e., < 4n 5 gε L + 4n 5 4n+5 g 4n 5 min ε [ε L,ε U ] g εε U ε L g gε L n n n 8n 4n 5 4n 5 4n+5 8n 4n 5 4n+ n 8n 4n 5 for n 3. In turn, this results in n i= h i = On 3n, and, in fact, n i= h i has the same order lower bound as well by the above, noting that n 8n = n + 5 n e 5/4 n 4n 5 4n 5 as n, for a suitably chosen small ε >.

25 Advances in Mechanics and Mathematics, Vol. IV 5 5 Geometrically decaying distances to the redundant hyperplanes Next we explore the geometric model for d: d i = ω ε n i+, i =,...,n. 5. Bounding α As in Subsection 4., we need to guarantee β < d. Firstly, we give a lower bound on k k=,...,n recursively defined by with =, and where k+ = d k+ + d k+ k, k =,...,n d k = ω k, k =,...,n ε with some constant ω. We have d i = d n i+ for i =,..., n, and β d = d n + d n n, β i d i = n i+, i =,..., n. Note that to satisfy β < d we necessarily must have k < for k =,...,n. From it follows that and hence k+ = k εk+ ω + k ε k+ ω k εk+, k =,...,n ω k ε k ω k + ε + ε + + ε k, k =,...,n. Observing = ε ω we can write the above inequality as k k = k ε ω ε k ω k ε ω 4 ε ω i= ε i k ε i, k =,...,n. i= Now, for α to be positive, that is for d n + n = d n n + dn n d n >

26 6 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes it suffices n dn > which is implied by n ε ω 4 ε ω If ε =, the above translates into n 3 ε i εn ω >. i= that is n n ω n > ω > n resulting in d i = ω n i+ > n n i+, i =,...,n. It is left to verify that hiεi d i is indeed dominated by h d for i =,...,n, to ensure 4 as in Subsection 3.. In particular, we demonstrate 8, as in the case of uniform d. Recalling β d = d n+ d n and βi n d i = n i+ for i =,...,n, it immediately follows that Also observe β d > β d. β d > β d > β 3 d 3 > > β n d n since, recalling and < k <, k =,...,n, k+ k = k + k + k = k d k+ + k d >. k+ 5. Picking a good ω As in Subsection 4., we would like to minimize α with respect to ω, which, in case of ε =, can be well approximated from above by We look for d n + n dn d n n n ω n.

27 that is or equivalently Advances in Mechanics and Mathematics, Vol. IV 7 min ω ω>n n n n ω n min ω>n Setting the gradient to, we obtain which gives us the minimizer ω ω n + min lnω lnω n +. ω>n ω ω n + ω = = ω n + ωω n + ω = n with the corresponding value of α n n+. This results in d i = n n i+, i =,...,n and n n hi = O ε i, i =,..., n. 5.3 An explicit formula For a given n, ε, δ, set d i = n n i+ for i =,...,n, compute the solution to A h = using 5. Set h = + δ n 3 max a ij 3. δ h i j= From 9 it follows that for large n hi α B i d. α, i =,...,n. ε We choose ε, δ, to minimize the total number of the redundant constraints, n i= h i. Recalling ε + δ < /, for large n we can write

28 8 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes n n h i 3 δ hi 3 n α δ ε i= i= i= i ε n 3n n+ ε ε = 3n n+ ε ε n ε 3n n+ ε n ε. 3 In fact, we would expect ε to be close to / in order for n i= h i to be minimized, so the last inequality also gives us a good approximation, namely n n h i 3n n+ ε ε. 4 Indeed, denoting and introducing i= fζ := ζ := ε we can write the last two lines in 3 as Differentiating ζfζ we get ζ n ζ 3n n+ ζfζ 3n n+ fζ. ζfζ = n + ζn ζ + n ζ n ζ 3 < for < ζ < n n + since n+ ζ n ζ+ n < n+ζ+ n < for such ζ, and therefore, the function is decreasing on this interval. So to maximize ζfζ we must take ζ > n n+, thus justifying 4. Next we demonstrate how to minimize n /. We note that the approximate minimum of n f := min <ζ< fζ. P n i= hi 3n n+ i= h i with respect to < ε < i= h i corresponds to To analyze f we proceed as follows: first we demonstrate that for n large enough fζ is convex on,, then we produce lower and upper bounds on the root of f ζ = in this interval, thus bounding the minimizer, and finally we compute upper and lower bounds on f using Taylor expansions of fζ. Observe f ζ = n + ζ n nζ n ζ 3 and

29 Denoting Advances in Mechanics and Mathematics, Vol. IV 9 f ζ = n + 6n + 6ζ n + 6nζ + n + n 6ζ n+ ζ n+ ζ 4. â := n + 6n + 6 ˆb := n + 6n ĉ := n + n for f ζ to be positive on, it suffices to show that the minimum of âζ + ˆbζ + ĉ is greater or equal than 6 recall < ζ <, so that 6ζ n+ < 6. In turn, min âζ + ˆbζ + ĉ = âζ + ˆbζ ζ= + ĉ = ˆb ζ ˆb 4â + ĉ â so for f ζ > on, it is enough to have The above can be rewritten as and is clearly implied by n + 6n 4n + 6n n + n 6. 4n 3 + n + 4n 4n + 6n + 6 = n n n n + 6n Therefore, fζ is convex on, as long as n 9. Furthermore, at ζ L := n n + we have f ζ L = On the other hand, at we have n n+ n f ζ U = n n+ n 3 <, for n. ζ U := n n n n n n n n n 3 <

30 3 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes and relying on all the derivatives of +z n for n N being positive at z =, n n n expanding n = + n into second-order Taylor series, we get f ζ U + n n + nn n n n n n 3 = n 4n n n n 3 for n 4. Now we are in position to give bounds on f for n 9: by convexity it follows that fζ L + f ζ L ζ U ζ L f fζ L i.e., n n + n + n + < n 4 4n n n + n + f. n 4 n + 4n Consequently n i= h i = On 3 n with the same order lower bound in the best case for ε n n+. Remark 6. Similar analysis can be easily carried out for d k = ω ε k for k =,...,n, with ε = ε. Since α d n + d n+ d n n ω ε n n ε ω 4ε ω = n n 3 ε i εn ω i= ω ε n ω ε + ε n 3 i= εi + ε n ε and is approximately minimized at ω = resulting in α 4 ε n h i 3 δ i= Noting that n ε + ε n 3 i= εi + ε n ε = n n εε n ε + ε n 3 i= εi + ε n ε,, we have n n n 3 hi 3 ε ε 4 + ε ε i + ε n ε n ε i= i= ε + ε n + ε εn. ε

31 εε n Advances in Mechanics and Mathematics, Vol. IV 3 ε + ε n + ε εn n ε εε n for ε, / and thus the later may be bounded from above by the value of the right-hand side at ε =, that is en + n, we get n n+ n h i 4en + n n n = On 3 n. i= The resulting estimate for the number of the redundant constraints is not much different from the case of ε = /, so this model for d is not discussed here in any more details. 6 Conclusions We provide sufficient conditions for the central path to intersect small neighborhoods of all the vertices of the Klee-Minty n-cube, see Propositions, and Corollary. More precisely, we derive explicit formulae for the number of redundant constraints for the Klee-Minty n-cube example given in [3, 4]. We give a smaller number of redundant constraints of order n 3n when the distances to those are chosen uniformly, as opposed to the previously established On 6n. When these distances are chosen to decay geometrically, we give a slightly tighter bound of the same order n 3 n as in [4], that results in a provably smaller number of constraints in practice. We argue that in R the sufficient conditions presented are tight and indicate that the same is likely true in higher dimensions. Our construction potentially gives rise to linear programming instances that exhibit the worst case iteration-complexity for path-following interiorpoint methods, which almost matches its theoretical counterpart. Considering the n-dimensional simplex, Megiddo and Shub [7] demonstrated that the total curvature of the central path can be as large as order n. Combined with Corollary, it follows that the worst case lower bound on the total curvature of the central path is at least exponential in n up to a rational multiplier. We conjecture that the total curvature of the central path is Om and the bound is tight. Acknowledgments We would like to thank Michael Shub for a stimulating and inspiring discussion that triggered the work on the total curvature of the central path. Research supported by an NSERC Discovery grant, by a MITACS grant and by the Canada Research Chair program.

32 3 Deza, Terlaky and Zinchenko: Redundant Klee-Minty cubes References. J-P. Dedieu, G. Malajovich, and M. Shub. On the curvature of the central path of linear programming theory. Foundations of Computational Mathematics, 5:45 7, 5.. J-P. Dedieu and M. Shub. Newton flow and interior point methods in linear programming. International Journal of Bifurcation and Chaos, 5:87 839, A. Deza, E. Nematollahi, R. Peyghami, and T. Terlaky. The central path visits all the vertices of the Klee-Minty cube. Optimization Methods and Software, -5:85 865, A. Deza, E. Nematollahi, and T. Terlaky. How good are interior-point methods? Klee-Minty cubes tighten iteration-complexity bounds. Mathematical Programming, to appear. 5. J. Gondzio. Presolve analysis of linear programs prior to applying an interior point method. INFORMS Journal on Computing, 9-:73 9, I. Maros. Computational Techniques of the Simplex Method. Kluwer Academic Publishers, Boston, MA, N. Megiddo and M. Shub. Boundary behavior of interior point algorithms in linear programming. Mathematics of Operations Research, 4-:97 46, R. Monteiro and T. Tsuchiya. A strong bound on the integral of the central path curvature and its relationship with the iteration complexity of primal-dual path-following LP algorithms. Optimization Online, September Yu. Nesterov and M. Todd. On the Riemannian geometry defined by selfconcordant barriers and interior-point methods. Foundations of Computational Mathematics, :333 36,.. C. Roos, T. Terlaky, and J-Ph. Vial. Theory and Algorithms for Linear Optimization: An Interior Point Approach. Springer, New York, NY, second edition, 5.. G. Sonnevend, J. Stoer, and G. Zhao. On the complexity of following the central path of linear programs by linear extrapolations. ii. Mathematical Programming, 5:57 553, 99.. M. Todd and Y. Ye. A lower bound on the number of iterations of long-step and polynomial interior-point linear programming algorithms. Annals of Operations Research, 6:33 5, S. Vavasis and Y. Ye. A primal-dual interior-point method whose running time depends only on the constraint matrix. Mathematical Programming, 74:79, G. Zhao and J. Stoer. Estimating the complexity of a class of path-following methods for solving linear programs by curvature integrals. Applied Mathematics and Optimization, 7:85 3, 993.