An O(nL) Infeasible-Interior-Point Algorithm for Linear Programming arxiv:1506.06365v [math.oc] 9 Jun 015 Yuagang Yang and Makoto Yamashita September 8, 018 Abstract In this paper, we propose an arc-search infeasible-interior-point algorithm. We show that this algorithm is polynomial and the polynomial bound is O(nL) which is at least as good as the best existing bound for infeasible-interior-point algorithms for linear programming. Keywords: polynomial algorithm, infeasible-interior-point method, linear programming. US NRC, Office of Research, 1 Church Street, Rockville, 0850. Email: yaguang.yang@verizon.net. Department of Mathematical and Computing Sciences, Tokyo Institute of Technology. Email: Makoto.Yamashita@is.titech.ac.jp. 1
1 Introduction Since Klee and Minty [1] showed that a simplex method for linear programming is not a polynomial algorithm, polynomial complexity bound has become a popular metric to measure the efficiency of optimization algorithms. Searching for polynomial algorithms for linear programming was a major research area of optimization between 1980 s and 1990 s after Khachiyan [] announced the first polynomial algorithm for linear programming. Although Khachiyan s algorithm was shown to be much less efficient in practice than the simplex method [3], Karmarkar s interior-point method [4] demonstrated the possibility of existence of efficient polynomial algorithms. For feasible starting point, people quickly established polynomial bounds for various interior-point algorithms [5, 6, 7, 8, 9, 10]. The lowest bound of these algorithms is O( nl) which has not been improved for more than two decades. To have an efficient implementation for interior-point algorithms, Mehrotra [11] and Lustig et. al. [1] realized that higher-order method and infeasible starting point are two necessary improvements. However, algorithms with either one of these features had poorer complexity bounds than O( nl). Monteiro, Adler, and Resende [13] showed that a higher-order algorithm starting from a feasible point has the polynomial bound O(nL). For infeasible-interior-point method, Zhang [14], Mizuno [15], and Miao [16] established polynomiality for several different algorithms(none of them is a higher-order algorithm). The best complexity bound O(nL) for infeasible interior-point methods has not been changed since the eary of 1990 s. Recently, Yang[17, 18] showed that for a higher-order interior-point method starting from a feasible point, the polynomial bound can be improved to O( nl) by using an arc-search method. Very recently, Yang et. al. [19] used the same idea and proposed a polynomial arc-search infeasible-interior-point algorihtm with a complexity bound of O(n 5 4L). In this paper, we show that for higher-order infeasible-interior-point method using arc-search, the polynomial bound can be improved to O(nL), which is a bound at least as good as the best bound of existing infeasible-interior-point algorithms. The remainder of the paper is organized as follows. Section describes the problem. Section 3 provides an infeasible-predictor-corrector algorithm. Section 4 proves its polynomiality. Section 5 summarizes the conclusions. Problem Descriptions The standard form of linear programming in this paper is given as follows: min c T x, subject to Ax = b, x 0, (1) where A R m n, b R m, c R n are given, and x R n is the vector to be optimized. Associated with the linear programming is the dual programming that is also presented in the standard form: max b T y, subject to A T y+s = c, s 0, ()
where dual variable vector y R m, and dual slack vector s R n. We use S to denote the set of all the optimal solutions (x,y,s ) of (1) and (). It is well known that x R n is an optimal solution of (1) if and only if x, y, and s satisfy the following KKT conditions Ax = b, A T y+s = c, (x,s) 0, x i s i = 0, i = 1,...,n. (3a) (3b) (3c) (3d) To simplify the notation, we will denote Hadamard (element-wise) product of two vectors x and s by x s, the element-wise division of the two vectors by s 1 x, or x s 1, or x s if min s i > 0, the Euclidean norm of x by x, the infinite norm of x by x, the identity matrix of any dimension by I, the vector of all ones with appropriate dimension by e, the block column vectors, for example, [x T,s T ] T by (x,s). For x R n, we will denote a related diagonal matrix by X R n n whose diagonal elements are the components of the vector x. Finally, we define an initial vector point of a sequence by x 0, an initial scalar point of a sequence by µ 0, the vector point after the kth iteration by x k, the scalar point after the kth iteration by µ k. Let r k b = Axk b, r k c = AT y k +s k c. (4a) (4b) Given a strictly positive current point (x k,s k ) > 0, the infeasible-predictor-corrector algorithm is to find the solution of (1) approximately along a curve C(t) defined by the following system Ax(t) b = tr k b A T y(t)+s(t) c = tr k c x(t) s(t) = tx k s k (x(t),s(t)) > 0, (5a) (5b) (5c) (5d) where t (0,1]. As t 0, (x(t),y(t),s(t)) appraches the solution of (1). Since C(t) is not easy to obtain, we will use an ellipse E [0] in the n+m dimensional space to approximate the curve defined by (5), where E is given by E = {(x(α),y(α),s(α)) : (x(α),y(α),s(α)) = acos(α)+ bsin(α)+ c}, (6) a R n+m and b R n+m are the axes of the ellipse, and c R n+m is the center of the ellipse. Taking the derivatives of (5) gives A 0 0 ẋ r k 0 A T b I ẏ = r k S k 0 X k c, (7) ṡ x k s k 3
A 0 0 0 A T I S k 0 X k ẍ ÿ s = 0 0 ẋ ṡ. (8) We require the ellipse to pass the same point (x k,y k,s k ) on C(t) and to have the same derivatives given by (7) and (8). The ellipse is given in [17, 18] as Theorem.1 Let (x(α),y(α),s(α)) be an arc defined by (6) passing through a point (x,y,s) E C(t), and its first and second derivatives at (x,y,s) be (ẋ,ẏ,ṡ) and (ẍ,ÿ, s) which are defined by (7) and (8). Then, the ellipse approximation of (5) is given by x(α) = x ẋsin(α)+ẍ(1 cos(α)). (9) y(α) = y ẏsin(α)+ÿ(1 cos(α)). (10) s(α) = s ṡsin(α)+ s(1 cos(α)). (11) 3 Infeasible predictor-corrector algorithm We denote the duality measure by and define the set of neighborhood by µ = xt s n, (1) N(θ) := {(x,s) (x,s) > 0, x s µe θµ}. (13) The proposed algorithm searches an optimizer along the ellipse while staying inside N(θ). Algorithm 3.1 1 Data: A, b, c, θ (0, + ], ǫ > 0, initial point (x0,y 0,s 0 ) N(θ). for iteration k = 1,,... Step 1: If stop. Otherwise continue. r k b = Axk b ǫ, r k c = AT y k +s k c ǫ, µ k ǫ, (14a) (14b) (14c) (x k,s k ) > 0. (14d) Step : Solve the linear systems of equations (7) and (8) to get (ẋ,ẏ,ṡ) and (ẍ,ÿ, s). 4
Step 3: Find the smallest positive ᾱ (0,π/] such that for all α (0,ᾱ], (x(α),s(α)) > 0 and (x(α) s(α)) (1 sin(α))µ k e θ(1 sin(α))µ k. (15) Set (to simplify the notation, we use α in stead of ᾱ in the rest of the paper) (x(α),y(α),s(α)) = (x k,y k,s k ) (ẋ,ẏ,ṡ)sin(α)+(ẍ,ÿ, s)(1 cos(α)). (16) Step 4: Calculate ( x, y, s) by solving A 0 0 x 0 0 A T I y = 0 S(α) 0 X(α) s (1 sin(α))µ k e x(α) s(α) Update and end (for). (17) (x k+1,y k+1,s k+1 ) = (x(α),y(α),s(α))+( x, y, s) (18) Step 5: Set k +1 k. Go back to Step 1. s k+1 µ k+1 = xk+1t. (19) n In the rest of this section, we will show (1) r k b 0, rk c 0, and µ k 0; () there exist α (0,π/] such that (x(α),s(α)) > 0 and (15) holds; (3) (x k,s k ) N(θ). It is easy to show that r k b, rk c, and µ k decrease at the same rate in every iteration. Lemma 3.1 r k+1 b = r k b (1 sin(α)), rk+1 c = r k c (1 sin(α)), µ k+1 = µ k (1 sin(α)). (0) Proof: Using (4), (18), (16), (8), and (7), we have r k+1 b r k b = A(x k+1 x k ) = A(x(α)+ x x k ) = A(x k ẋsin(α) x k ) = Aẋsin(α) = r k b sin(α). This shows the first relation. The second relation follows a similar derivation. From (17), it holds that ( x) T s = ( x) T ( A T y) = (A x) T y = 0. Using (18), we have x k+1t s k+1 = (x(α)+ x) T (s(α)+ s) = x(α) T s(α)+x(α) T s+s(α) T x = x(α) T s(α)+(1 sin(α))µ k n x(α) T s(α) = (1 sin(α))µ k n. Dividing both sides by n proves the last relation. 5
Clearly, if sin(α) = 1 (α = π ), we will find the optimal solution (allowing some x i = 0 and/or s j = 0) in one step, which is rarely the case. Therefore, from now on, we assume α (0, π ). We will use the following lemma of [16]. Lemma 3. Let ( x, s) be given by (17). Then 1 x s 4 (X(α)S(α)) (x(α) s(α) µk+1 e). (1) Theorem 3.1 Let (x(α),y(α),s(α)) and (x k,s k ) N(θ). Then, for all k 0 (i) there is an α > 0, such that (x(α),s(α)) > 0 and (15) holds. (ii) if θ 1 +, then (xk+1,s k+1 ) N(θ) for all the iterations. Proof: Using 1 cos(α) 1 cos (α) = sin (α) and (x k,y k,s k ) N(θ), we have x(α) s(α) (1 sin(α))µ k e = (x k s k µ k e)(1 sin(α))+(ẍ s ẋ ṡ)(1 cos(α)) (ẍ ṡ+ẋ s)sin(α)(1 cos(α)) θµ k (1 sin(α))+( ẍ s + ẋ ṡ )sin 4 (α) +( ẍ ṡ + ẋ s )sin 3 (α). () Clearly, if ( ) ( ) + ẋ ṡ q(α) := ẍ s sin 4 + ẍ ṡ (α)+ ẋ s sin 3 (α)+θµ k sin(α) θµ k 0, (3) then, (15) holds. Indeed, since q(0) = θµ < 0, by continuity, there exist α > 0 such that (3) holds. This shows that (15) holds. From (15), we have x i (α)s i (α) (1 θ)(1 sin(α))µ k > 0, θ [0,0.5) and α [0,π/). This shows (x(α), s(α)) > 0. Therefore, we finish part (i). Furthermore, from Lemma 3.1, (15) is now equivalent to x(α) s(α) µ k+1 e θµ k+1. Using (18), (17), Lemmas 3.1 and 3., and part (i) of this theorem, we have x k+1 s k+1 µ k+1 e = (x(α)+ x) (s(α)+ s) µ k+1 e 1 = x s 4 (X(α)S(α)) (x(α) s(α) µk+1 e) x(α) s(α) µ k+1 e 4 min i x i (α)s i (α) (θ) µ k+1 4(1 θ)µ k+1 θ (1 θ) µ k+1. (4) 6
Itiseasytocheck thatforθ 1 + 0.989, θ 1 θ holds, therefore, forθ (1 θ) +, we have x k+1 s k+1 µ k+1 e θµ k+1. Wenow show that (x k+1,s k+1 ) > 0. Let x k+1 (t) = x(α)+t x ands k+1 (t) = s(α)+t s. Then, x k+1 (0) = x(α) and x k+1 (1) = x k+1. Since x k+1 (t) s k+1 (t) = (x(α)+t x) (s(α)+t s) = x(α) s(α)+t(x(α) s+s(α) x)+t x s, using (17), (15), (4), and the assumption that θ 1 +, we have x k+1 (t) s k+1 (t) µ k+1 e = (1 t)(x(α) s(α) µ k+1 e)+t x s θ (1 t)θµ k+1 +t 1 θ µ k+1 ((1 t)+t )θµ k+1 := f(t)θµ k+1. (5) The function f(t) is a monotonical decreasing function of t [0,1], and f(0) =. This proves x k+1 (t) s k+1 (t) µ k+1 e θµ k+1. Therefore, x k+1 i (t)s k+1 i (t) (1 θ)µ k+1 > 0 for all t [0,1], which means (x k+1 s k+1 ) > 0. This finishes the proof of part (ii). This theorem indicates that the proposed algorithm is well-defined. 4 Polynomiality The analysis follows similar ideas in many existing literatures, such as [16, ]. Let the initial point be selected to satisfy (x 0,s 0 ) N(θ), x ρx 0, s ρs 0, (x,y,s ) S, (6) where ρ 1. Let ω f and ω o be the quality of the initial point which are the distances from feasibility and optimility given by ω f = min x,y,s {max{ (X0 ) 1 (x x 0 ), (S 0 ) 1 (s s 0 ) } Ax = b, A T y+s = c}. (7) and {max{x T ω 0 s 0 s Tx 0 = min x,y,s x 0T s0, x 0T s 0,1} (x,y,s ) S}. (8) Let ωp r and ωr d be the ratios of the feasibility and the total complementarity defined by ωp r = Ax0 b, x 0T s 0 (9a) ωd r = AT y 0 +s 0 c. x 0T s 0 (9b) 7
In view of Lemma 3.1, we have that Ax k b = ω r px kt s k, A T y k +s k c = ω r d xkt s k. (30a) (30b) Invoking Lemma 3.3 of [] for λ p = λ d = ξ = 1 and (7), we have the following two Lemmas [16]. Lemma 4.1 Let (ẋ,ṡ) be defined by (7), and D k = (X k ) 1 (S k ) 1. Then max{ (D k ) 1 ẋ, (D k )ṡ } (x k s k ) 1 +ω f (1+ω o (x k ) T s k ). (31) min i (x k i sk i )1 Lemma 4. Let (x 0,s 0 ) be defined by (6). Then ω f ρ, ω o ρ. (3) This leads to the following lemma. Lemma 4.3 Let (ẋ,ṡ) be defined by (7). Then, there exists a positive constant C 0, independent of n, such that max{ (D k ) 1 ẋ, (D k )ṡ } C 0 n(xk ) T s k. (33) Proof: First, it is easy to see (x k s k ) 1 = i x k i sk i = (x k ) T s k. (34) Since (x k,s k ) N(θ), we have min i (x k is k i) (1 θ)µ k = (1 θ) (xk ) T s k. Therefore, (x k ) T s k min i (x k i sk i )1 n(x k ) T s k n (1 θ). (35) Substituting (34) and (35) into (31) and using Lemma 4. prove (33) with C 0 = 1 + ρ(1+ρ) (1 θ) 1+ ωf (1+ω o ) (1 θ). 8
From Lemma 4.3, we can establish several useful inequalities. The following simple facts will be used several times. Let u and v be two vectors, then u v = ( )( ) (u i v i ) u i vi = u v. (36) i i If u and v satisfy u T v = 0, then, max{ u, v } u + v = u+v, (37) i and (see [3, Lemma 5.3]) u v 3 u+v. (38) Lemma 4.4 Let (ẋ,ṡ) and (ẍ, s) be defined by (7) and (8), respectively. Then, there exist positive constants C 1, C, C 3, and C 4, independent of n, such that ẋ ṡ C 1 n µ k, (39) ẍ s C n 4 µ k, (40) max{ (D k ) 1 ẍ, (D k ) s } C 3 n µ k, (41) max{ ẍ ṡ, ẋ s } C 4 n 3 µ k (4) Proof: First, using (36) and Lemma 4.3, we have ẋ ṡ = (D k ) 1 ẋ (D k )ṡ (D k ) 1 ẋ (D k )ṡ C 0n(x k ) T s k := C 1 n µ k. (43) Second, using (38), (8), (39), and (34), we have ẍ s = (D k ) 1 ẍ (D k ) s 3 (D k ) 1 ẍ+(d k ) s 3 (XS) 1 (ẋ ṡ) n ( ) = 1 ẋi ṡ i n = 1 (ẋ i ṡ i ) xi si x i=1 i=1 i s i n 1 i=1 (ẋ iṡ i ) min i=1,...,n x i s i 1 ẋ ṡ 1 C1n 4 µ k (1 θ)µ k (1 θ)µ k C = 1 1 n4 µ k 1 θ := C n 4 µ k. (44) Third, using (37), (8), and (39), we have = max{ (D k ) 1 ẍ, (D k ) s } (D k ) 1 ẍ+(d k ) s (XS) 1 4C (ẋ ṡ) 1 n4 µ k := C3 1 θ n4 µ k. (45) 9
Taking square root on both sides proves (41). Finally, using (36), (41), and Lemma 4.3, we have Similarly, we can show This finishes the proof. ẍ ṡ = (D k ) 1 ẍ (D k )ṡ (D k ) 1 ẍ (D k )ṡ (C 3 n µ k )(C 0 n µ k ) := C 4 n 3 µ k. (46) ẋ s C 4 n 3 µ k. (47) Now we are ready to estimate a conservative bound for sin(α). Lemma 4.5 Let (x k,y k,s k ) be generated by Algorithm 3.1. Then, sin(α) obtained in Step 3 satisfies the following inequality. where C = max{1,c 1 3 4,(C 1 +C ) 1 4}. sin(α) θ Cn, (48) Proof: Let sin(α) = θ. In view of (3) and Lemma 4.4, we have Cn q(α) µ k ((C 1 +C )n 4 sin 4 (α)+c 4 n 3 sin 3 (α)+θsin(α) θ) := µ k p(α) ( (C1 +C )θ 4 µ k + C ) 4θ 3 + θ 16C 4 8C 3 Cn θ ( ) θ 4 µ k 16 + θ3 4 + θ θ 0. Since p(α) is a monotonic function of sin(α), for all sin(α) θ, the above inequalities Cn hold (the last inequality holds because of θ 1). Therefore, for all sin(α) θ, the Cn inequality (15) holds. This finishes the proof. Remark 4.1 It is worthwhile to point out that the constant C depends on C 0 which depends on ρ, but ρ is an unknown before we find the solution. Also, we can always find a better steplength sin(α) by solving the quartic q(α) = 0 and the calculation of the roots for a quartic polynomial is deterministic, negligible, and independent to n [4, 5]. Following the standard argument developed in [3], we have the main theorem. Theorem 4.1 Let (x k,y k,s k ) be generated by Algorithm 3.1 with an initial point given by (6). For any ǫ > 0, the algorithm will terminate with (x k,y k,s k ) satisfying (14) in at most O(nL) iterations, where L = max{ln((x 0 ) T s 0 /ǫ),ln( r 0 b /ǫ),ln( r 0 c /ǫ)}. Proof: In view of Lemma 3.1, r k b, rk c, and µ k decrease at the same rate (1 sin(α)) in every iteration. Using the Lemma 4.5 and [3, Theorem 3.] proves the claim. 10
5 Conclusions We proposed an infeasible-interior-point algorithm that searches the optimizer along an ellipse that approximates the central-path. We showed that the proposed algorithm is polynomial and that the polynomial bound is at least as good as the best existing bound for infeasible-interior-point algorithms for linear programming. References [1] V. Klee and G. Minty, How good is the simplex algorithm? In: Shisha, O., (eds.) Inequalities, Vol. III, pp. 159-175, Academic Press, (197). [] L. Khachiyan, A polynomial algorithm in linear programming, Doklady Akademiia Nauk SSSR, 4, 1093-1096 (1979). [3] R. Bland, D. Goldfarb, and M. Todd, The ellipsoid method: A survey, Operations Research, 9, 1039-1091 (1981). [4] N. Karmarkar, A new polynomial-time algorithm for linear programming, Combinatorica, 4, 375-395 (1984). [5] J. Renegar, A polynomial-time algorithm based on Newton s method, for linear programming, Mathematical Programming, 40, 59-93 (1988). [6] M. Kojima, S. Mizuno, and A. Yoshise, A polynomial-time algorithm for a class of linear complementarity problem, Mathematical Programming, 44, 1-6 (1989). [7] M. Kojima, S. Mizuno, and A. Yoshise, A primal-dual interior point algorithm for linear programming, in Progress in Mathematical Programming: Interior-point and Related Methods, N. Megiddo, ed., Springer-Verlag, New York, 1989. [8] C. Gonzaga, Polynomial affine algorithms for linear programming, Mathematical Programming, 49, 6-1 (1990). [9] Y. Ye, An O(n 3 L) potential reduction algorithm for linear programming, Mathematical Programming, 50, 39-58 (1991). [10] S. Mizuno, M. Todd, and Y. Ye, On adaptive step primal-dual interior-point algorithms for linear programming, Mathematics of Operations Research, 18, 964-981 (1993). [11] S. Mehrotra, On the implementation of a primal-dual interior point method, SIAM Journal on Optimization,, 575-601 (199). [1] I. Lustig, R. Marsten, and D. Shannon, On implementing Mehrotra s predictorcorrector interior-point method for linear programming, SIAM Journal on Optimization,, 43-449 (199). 11
[13] R. Monteiro, I. Adler, and M. Resende, A polynomial-time primal-dual affine scaling algorithm for linear and convex quadratic programming and its power series extension, Mathematics of Operations Research, 15, 191-14 (1990). [14] Y. Zhang, On the convergence of a class of infeasible-interior-point methods for the horizontal linear Complementarity problem, SIAM Journal on Optimization, 4, 08-7 (1994). [15] S. Mizuno, Polynomiality of the Kojima-Megiddo-Mizuno infeasible interior point algorithm for linear programming, Mathematical Programming, 67, 109-119(1994) [16] J. Miao, Two infeasible interior-point predictor-corrector algorithms for linear programming, SIAM Journal on Optimization, 6, 587-599 (1996). [17] Y. Yang, A Polynomial Arc-Search Interior-Point Algorithm for Linear Programming, Journal of Optimization Theory and Applications, 158, 859-873 (013). [18] Y. Yang, A polynomial arc-search interior-point algorithm for convex quadratic programming, European Journal of Operational Research, 15, 5-38 (011). [19] X. Yang, Y. Zhang, and H. Liu, A wide neighborhood infeasible-interior-point method with arc-search for linear programming, Journal of Applied Mathematics and Computing, 1-17 (015). [0] M. P. Do Carmo, Differential Geometry of Curves and Surfaces, Prentice-Hall, New Jersey, (1976). [1] S. Mizuno, A new polynomial time method for a linear complemenetarity problem, Mathematical Programming 56, 31-43 (199). [] M. Kojima, Basic lemmas in polynomial-time infeasible-interior-point methods for linear programming, Annals of Operations Research, 6, 1-8 (1996). [3] S. Wright, Primal-Dual Interior-Point Methods, SIAM, Philadelphia, (1997). [4] S.L. Shmakov, A universal method of solving quartic equations, International Journal of Pure and Applied Mathematics, 71 (), 51-59 (011). [5] Y. Yang and Z. Zhou, An analytic solution to Wahba s problem, Aerospace Science and Technology, 30, 46-49 (013) 1