Two Lectures on the Ellipsoid Method for Solving Linear Programs

Transcription

1 Two Lectures on the Ellipsoid Method for Solving Linear Programs Lecture : A polynomial-time algorithm for LP Consider the general linear programming problem: δ = max cx S.T. Ax b, x R n, () where A = (a ij ) i,j Z m n is an m n integer matrix, and b = (b i ) i Z m and c = (c j ) j Z n are integer vectors. We denote by a T i the i th row of A. In the unit-cost model of computation, the cost of multiplying two integers x and y is cost(x y) =. However, in the bit-model, it is cost(x y) = l(x)+l(y), where l(x) = log( + x ) + = number of bits used to represent x in binary. For instance, consider the following code for computing x = a 2k :. x a; 2. for i =,..., k, set x x 2. Then in the unit-cost model, the cost of this code is k, while in the bit-model, it is log x 2 k l(a). Let l = max{l(a ij ), l(b i ), l(c j )}, and let L = i,j l(a ij ) + i l(b i ) + j l(c j ) be the total number of bits needed to represent the input. In the bit-model of computation, an algorithm is said to run in polynomial-time, if the number of operations ({+,,, /}) is at most poly(n, m, l) and the bit length of all numbers involved in the computation is at most poly(n, m, l). As we shall see, linear programming is polynomial-time solvable in the bit-model. These notes were taken by Khaled Elbassioni (Max-Planck-Institut für Informatik, Saarbrücken, Germany; (elbassio@mpi-sb.mpg.de)) while attending a graduate course on Linear Programming taught by Leonid Khachiyan in Fall 996 at Rutgers University.

2 The Ellipsoid method Let > 0 be a given constant. We call x an -approximate solution of () if c x δ and a T i x b i +, for i =,...,m. Assumption We know R R +, such that () has an optimal solution x in the Euclidean ball B R = {x / x R}. Let h = max{ a ij, b i, c j }, i.e., h = 2 l. Then under Assumption, the Ellipsoid method computes an -approximate solution of () in O((n + m)n 3 log ( ) ( Rhn ) arithmetic operations over O(log Rhn ) )-bit numbers. Fact Let V be the unit ball V = {x / x }, and let V = V {x / x n 0}. Then there s an ellipsoid E such that V E vol E vol V e 2(n+) 2n. Proof. Let the center of E be (0, 0,...,0, n+ ) = n+ e n, and α = n+ n (see Figure ). The equation of E is ( ) 2 x n n+ α 2 + x2 β 2 + x2 2 β x2 n β 2. At point y in Figure : x n = 0 and x 2 + x x2 n (/n+) 2 (n/n+) + 2 β =, which implies 2 β = + n 2 + 2n 2. =, and we get In the plane, an ellipse E with principal axes lengths α and β, has an area of παβ, and hence the ratio of area of E to that of the unit circle V is αβ. In 3-dimensions the volume of an ellipsoid with principal axes lengths α, β, and γ is 4 3 παβγ, while the volume of the unit ball is 4 3π, giving a ratio of αβγ. (This follows specifically from the fact that the ellipse E [ is obtained ] by α 0 transforming the circle using the linear map x = Ax, where A =, and 0 β thus vol(e) = det(a) vol(v ).) Generalizing, we get that vol(e )/ vol(v ) = αβ n. Using the inequality + x e x, valid for all x R (see Figure 2), we get α e n+ and β e 2(n 2 ), and thus vole volv = αβn e n+ e n 2(n 2 ) = e 2(n+). 2

3 x n V α n+ β y E e x y = + x x, x 2,..., x n x Fig. Fig. 2 Fact 2 Let E be an ellipsoid in R n centered at η. Given a non-zero vector a R n, consider the hyperplane π = {x / a T (x η) = 0}, and let E + and E be the two halves of E obtained by cutting E with π. Then there s an ellipsoid E such that E E vol E vol E e 2(n+) 2n. Proof. This follows by from Fact by applying a linear transformation (rotation) that maps E to the unit ball V. Such a transformation does not change the ratio of volumes or inclusion. Note that an ellipsoid E can be represented as E = {x / x = η + Qz, z }, where η R n is the center, and Q is an n n matrix. Given η, Q and a vector a R n, we can get the center η and matrix Q, corresponding to the ellipsoid E E {x / a T (x η) 0} in Fact as follows. With respect to the z-coordinates, we can write the inequality determined by π as a T (x η) = a T Qz = (Q T a) T z 0. Thus the vector e n if Fact corresponds (by an orthonormal transformation) to the vector Q T a/ Q T a in our case. Hence, the center of E in the z-coordinates is z = Q T a n+ Q T a. Let µ = QT a Q T a (which can be computed with n2 operations), then returning to the x-coordinates we get η = η Qµ (2) n + [ ( ] n Q 2 n = n 2 Q + )Qµµ n + T. (3) Note that the computation in (2), (3) can be done with n 2 operations. Recall that δ = max{cx / a T i x b i, for i =,..., m}. Let X = {x / cx δ, a T i x b i +, for i =,...,m, x R}. Assumption implies that 3

4 0 a T j ˆx b j > 0 X a η 0 t Fig. 3 0 a T j x b j < 0 x Fig. 4 ˆx X because the exact solution x B R belongs to X solution exists). It also implies that ( Fact 3 vol X vol B R h nr) n. (we assume such a Proof. Let x be an exact optimal solution, i.e., δ = cx. Let y R n be such that y x r (we will select r later). Then writing y = x + η, we have η r and hence cy = cx + cη δ c η (by the Cauchy-Schwartz s Inequality). Then, if we set r = h n, we get c η c r h nr. Similarly, a T i x b i implies a T i (x i + η) b i + ǫ. Thus the ball of radius r, centered at x, is contained in the set X and hence vol X vol Br vol B R vol B R ( r n. R) 2 Lecture 2 Recall Facts 2 and 3. Now we prove the following: Fact 4 Suppose we are given an ellipsoid E centered at η such that X E but η X. Then in O((n + m)n) operations, we can compute a new ellipsoid E such that () X E and (2) vole e 2(n+) vole. Proof. This can be done as follows (see Figure 3). Check if a T i η b i +, for i M = {,..., m}. If for i M, a T i η > b i +, then a T i x > a T i η implies x X. Thus, for any x X, we must have a T i x a T i η, i.e., a T i (x η) 0. Thus in this case, we set a = a i as the normal of the cutting hyperplane in Fact 2. Now suppose that η is an -feasible solution. Then we check whether η R. If η > R, then η T (x η) 0 for all x X. Thus we set a = η in this case. Finally, suppose that a T i η b i + for all i M and η R. If η X, then cη < δ. Hence, any x such that cx < cη < δ is not in X. Thus, cx cη, or c(x η) 0, for any -approximate solution. So we set a to c in this case, and apply Fact 2. 4

5 The Ellipsoid method starts with E 0 = B R and generates a sequence of ellipsoids E 0 = B R, E, E 2,..., E K, E K+. As long as η k (the center of E k ) is not in X, we obtain a new ellipsoid E k+ such that X E k+, and vol E k+ vol E k e 2(n+). In particular, volx vole K e 2(n+) volbr. From Fact 3, we get ( ) n volb R h volx vole K e K 2(n+) volbr, nr from which we get an upper bound on K: K N = 2n(n + )ln R nh K = O(n 2 log Rnh ). Termination Criterion: We run the algorithm for K iterations and always maintain the -feasible center with highest objective value: suppose, for some k, a T i η k b i +, for all i M and η k R, then we store η k. Suppose also that, for some l > k, a T i η l b i +, for all i M and η l R. If cη l > cη k, then we replace η k by η l. Thus the total number of operations is O((n+m)n 3 log Rnh ) = O(mn3 log Rnh ) (if m < n, we can find in O(m) time a basis and switch to a smaller problem). It can be shown also that the required accuracy is O(log Rnh ). How to round and -approximate solution to an exact one Recall that we assume the matrices A, b, and c to be integral. Given an integral matrix A, denote by (A) = max{ det B / B is a square submatrix of A}. [ ] 7 3 For e.g. if A =, then (A) = 7. Recall that for a vector v R 3 2 n, v = max i { v i }. Lemma If problem (), with integer matrices A and b, has an optimal solution, then it has an optimal solution x such that x n b (A). Proof. By the fundamental Theorem of Linear Programming, there exists M M = {,...,m} such that a T i x = b i, for all i M. Then by Cramer s rule x j = j, where is a non-zero subdeterminant of A, and j is a subdeterminant of [A b]. Since is an integer, x j j. But j = i b i i, where i is a subdeterminant of A. Using b i b and i (A), we get that x j n b (A). Thus we can set R = hn 3/2 (A) to guarantee that x R (using the fact that n 2 ). 5

6 Lemma 2 Suppose that δ is finite, then δ = t s, where t and s are integers such that s (A). Indeed, δ = c x + c 2 x c n x n = c + + c n n. Lemma 3 Consider the system Ax b, x R n, (4) and let 0 = (n+2) (A). If there is an 0-approximate solution to (4) (i.e. a T i x b i + 0, for i M, is feasible), then (4) has an exact solution. Proof. Consider = min S.T. a T i x b i +, i M. (5) Suppose that Ax b is infeasible. Then ǫ > 0, and hence by Lemma 2, ǫ = t s for some integers t and s, where s ([A e]) (n + ) (A). This gives s (n+) (A). Thus using = 0 = (n+2) (A) will make (4) feasible. Now suppose that a T i x b i + 0, i M. We can compute an exact solution using mn 2 operations as follows. Let I 0 = {i M / a T i x b i 0 }. Then a T j x < b j 0 for j M \I 0. The system a T i x = b i, i I 0, is solvable, by Lemma 3, because x is an 0 -approximate solution ([ for it (the ]) system can be written as a T i x b i, a T i x b A i, i I 0, and I0 (A A I0 ) (A), where I0 A I0 is the submatrix of A with rows indexed by I 0 ). Let ˆx be an exact solution for this system, i.e., a T i ˆx = b i, for i I 0. If x does not satisfy the original system, then we proceed as follows. Define x(t) = ( t) x + tˆx, for 0 t. Then for i I 0, we have a T i x(t) b i = ( t)a T i x + ta T i ˆx ( t)b i tb i = ( t)[a T i x b i] + t[a T i ˆx b i] ( t) a T i x b i + t a T i ˆx b i ( t) 0 0. Thus x(t) satisfies the constraints in I 0 for all t [0, ]. Now we obtain a new 0 -approximate solution with a wider set of tight constraints, by choosing t as follows (see Figure 4): { } t = min j M\I 0 at j x b j a T j ˆx at j x / at j ˆx b j > 0 Let j min be such a minimizer in M \I 0. Then a T j min x(t) b jmin = ( t)(a T j min x b jmin )+t(a T j minˆx b jmin ) = 0. We add j min to I 0, replace x by x(t), and repeat the procedure again. So this way, we do at most n iterations of mn 2 operations. 6

7 each (solving a system of linear equations), but it can be done in O(mn 2 ) in total. To consider the objective function: We use, instead of 0, = 4n 5/2 3 (A) c. We do the same procedure as before without caring about the objective function. We claim that the result is optimal. Indeed, we know that a T i ˆx = b i and a T i x = b i +η i for i I 0, where η. Thus a T i (ˆx x) = η i, for i I 0, and setting ˆx x = y, we get (again from Cramer s rule) that y n η (A), and thus ˆx x n (A). This gives cˆx c x c ˆx x c n 3/2 (A). At the end of the rounding algorithm we obtain x such that Ax b and cx c x c n 5/2 (A) (each iteration of the procedure contributes at most c n 3/2 (A) to this difference). Thus cx c x c n 5/2 (A) δ c n 5/2 (A) δ 2 c n 5/2 (A) = δ 2 2 (A). But cx = t s, where t, s Z and s (A) (as in Lemma 2), and δ = t s, s (A). Since cx δ, we have cx δ 2 2 (A). On the other hand, if cx δ, then cx δ = t s t s = t s ts ss ss 2 (A) > 2 2 (A). Thus we must have cx = δ. Summary Using = 4n 5/2 3 (A) c and R = n3/2 h (A), the Ellipsoid method requires O(mn 3 log Rhn ) = O(mn 3 log(hn (A))) operations. Denote by L = L(A, b, c) the number of bits required to write down the problem, and let L(A) = i,j log( + a ij ). We note that (A) 2 L(A) since (A) i,j (+ a ij ). Also n 2 L(A) and h 2 L(A,b,c). Thus the total number of operations required by the Ellipsoid method is O(mn 3 L). 7