Combinatorial Optimization Spring Term 2015 Rico Zenklusen. 2 a = ( 3 2 ) 1 E(a, A) = E(( 3 2 ), ( 4 0

Transcription

1 3 2 a = ( 3 2 ) 1 E(a, A) = E(( 3 2 ), ( )) Figure 9: An example of an axis parallel ellipsoid E(a, A) in two dimensions. Notice that the eigenvectors of A correspond to the axes of the ellipsoid, and the square roots of the eigenvalues correspond to the radii of the corresponding axes. Additionally to a separation oracle for P, the ellipsoid algorithm also needs an ellipsoid E 0 that contains P. We recall that an ellipsoid is the image of the unit ball under an affine bijection, which can be defined as follows. Definition 23 (Ellipsoid). An ellipsoid in R n is a set E(a, A) := {x R n (x a) T A 1 (x a) 1}, where a R n and A R n n is a positive definite matrix. The point a is called the center of the ellipsoid E(a, A). In particular, an ellipsoid is always full-dimensional. Notice that the above definition of ellipsoid is indeed an affine bijection of the unit ball. This can be seen as follows. A matrix A R n n is positive definite if and only if there is a full rank matrix Q R n n such that A = QQ T. Hence, A 1 = (Q T ) 1 (Q 1 ) = (Q 1 ) T Q 1, and therefore E(a, A) = {x R n Q 1 (x a) 2 1} = {y + a y R n, Q 1 y 2 1} (Substitution with y = x a.) = {Qz + a z R n, z 2 1}. (Substitution with z = Q 1 y.) Figure 9 shows an example of an ellipsoid for n = 2. The separation oracle for P and the ellipsoid E 0 P is all that is needed to run the ellipsoid algorithm and obtain a point x P. However, to make sure that the ellipsoid algorithm runs in polynomial time we need a further( condition. ) One sufficient condition for the ellipsoid algorithm to run in polynomial time is that log vol(e0 ) is polynomially bounded in the input. We will see later that this holds for most cases we are interested in. Description of ellipsoid method The ellipsoid method is described in Algorithm 1. Algorithm 1: Ellipsoid method input : Separation oracle for a polytope P R n with dim(p ) = n, and an ellipsoid E 0 = E(a 0, A 0 ) with P E 0. output: A point y P. 1 i = 0; 2 while a i P (checked with separation oracle) do 3 Get non-zero c R n such that P {x R n c T x c T a i }, using sep. oracle; 4 Find min. volume ellipsoid E i+1 = E(a i+1, A i+1 ) containing E i {x R n c T x c T a i }; 5 i = i + 1; 6 return a i ;

2 P E(a i+1, A i+1 ) a i+1 {x R n c T x = c T a i } a i E(a i, A i ) c Figure 10: Illustration of a single iteration of the ellipsoid method. The polytope P is inside each ellipsoid considered by the ellipsoid method. We will soon give some more details on how to compute E i+1. It turns out that there is relatively simple way to describe E i+1 in terms of E i and c. Unfortunately, as we will see soon, it turns out that this description of E i+1 involves taking a square root, which is an operation that we cannot perform up to arbitrary precision under the typical computational assumptions. Hence, in practice, one only computes an ellipsoid E i+1 that approximates E i+1 and has a polynomial-size description. To simplify the exposition we will not go into these numerical details and assume that we use an exact description of E i+1. Figure 10 illustrates one iteration of the ellipsoid method. Notice that we have P E i for each ellipsoid considered in the ellipsoid method. This can easily be verified by induction. The first ellipsoid E 0 contains P by assumption. Furthermore, for each iteration i, P {x R n c T x c T a i }, and hence P E i {x R n c T x c T a i }, since P E i. We thus obtain P E i+1 because E i {x R n c T x c T a i } E i+1 by definition of E i+1. Getting a bound on the number of iterations The key property of the constructed ellipsoid, besides the fact that they contain P, is that they shrink in terms of volume. More precisely, we have the following. Lemma 24. vol(e i+1 ) vol(e i ) < e 1 2(n+1). Before proving Lemma 24, we observe that it immediately implies an upper bound on the number of iterations that the ellipsoid algorithm performs. ( ) Lemma 25. The ellipsoid method will stop after at most 2(n + 1) ln vol(e0 ) iterations. Proof. Let L Z 0 be the last iteration of the ellipsoid algorithm. Since E L contains P, we must have vol(e L ) which, combined with Lemma 24, leads to vol(e L ) vol(e 0 )e L 2(n+1), and thus L 2(n + 1) ln ( ) vol(e0 ). The rest of this section is devoted to proving Lemma 24. Later, we will show how the bound on the number of iterations given by Lemma 25 can be used to prove that the ellipsoid method runs in polynomial time for any {0, 1}-polytope. For this we will first make the link between checking feasibility of a polytope and optimizing an LP more explicit.

3 Proof of Lemma 24 and explicit description for E i+1 A key simplification for proving Lemma 24 is to observe that it suffices to consider the special case defined by the following properties. Assumption 26. E i is the unit ball, i.e., E i = E(0, I), where 0 R n is the zero vector, and I R n n is the n n identity matrix. The halfspace H i = {x R n c T x c T a i } is equal to H = {x = (x 1,..., x n ) R n x 1 0}. More precisely, a description of a minimum volume ellipsoid under this assumption can be transformed to a description of a minimum volume ellipsoid containing E i H i. Proof that Assumption 26 is without loss of generality. We first sketch the proof plan. We will define an affine bijection ρ : R n R n such that ρ(e(0, I)) = E i and ρ( H) = H i. Furthermore, since ρ is affine, applying it to any measurable set will scale its volume by a factor that only depends on ρ. Using this, we can derive that if Ē is a minimum volume ellipsoid containing E(0, I) H, then ρ(ē) is a minimum volume ellipsoid containing E i H i. We start by defining a bijective affine transformation φ that maps E(0, I) to E i. Let E i = (a i, A i ) and let Q i R n n be such that A i = Q i Q T i. Define As we have already observed previously, φ(x) = Q i x + a i x R n. φ(e(0, I)) = E i. If furthermore we had φ 1 (H i ) = H, then we could chose ρ = φ. However, this is not true in general. Still, a small change to φ (composing φ with an orthonormal matrix) will fix this. To see how we have to change φ to obtain ρ, we start by considering φ 1 (H i ). φ 1 (H i ) = {φ 1 (x) x R n, c T x c T a i } = {y R n c T (Q i y + a i ) c T a i } (y = φ 1 (x) = Q 1 i (x a i )) = {y R n c T Q i y 0}. Notice that c T Q i 0 since c 0 and Q i is full rank. Hence, Q T i c 2 > 0. Moreover, the hyperplane H i does not change when scaling c. Hence, we can assume Q T i c 2 = 1. Ideally, we would have liked to have Q T i c = e 1, where e 1 = (1, 0,..., 0) {0, 1} n is the vector with a single one in the first coordinate and zeros everywhere else. This would have implied φ 1 (H i ) = H. To achieve this we slightly modify φ to get ρ. Since Q T i c 2 = 1, there is a orthonormal matrix R R n n such that R T Q T i c = e 1. (19) We define We now obtain ρ(x) = φ(rx) = Q i Rx + a i x R n. ρ 1 (H i ) = {ρ 1 (x) x R n, c T x c T a i } = {y R n c T (Q i Ry + a i ) c T a i } (y = ρ 1 (x)) = {y R n c T Q i Ry 0} = {y R n y 1 0} (by (19)) = {y R n y 1 0} = H. Thus, ρ is indeed a bijective affine transformation with ρ( H) = H i and ρ(e(0, I)) = E i. Let Ē be a minimum volume ellipsoid that contains E(0, I) H. We will show that ρ(ē) is a minimum volume ellipsoid containing E i H i, which will finish the proof. Notice that all our steps

4 were constructive. In particular one can find efficiently a matrix R as claimed (though, we did not give details for this here). Furthermore, a matrix Q i such that A i = Q i Q T i can be found through a Cholesky decomposition. Since Ē E(0, I) H and ρ is a bijection, we have ρ(ē) ρ(e(0, I)) ρ( H) = E i H i. It remains to show that ρ(ē) has minimum volume among all ellipsoids that contain E i H i. Let Ẽ be an arbitrary ellipsoid in R n such that Ẽ E i H i. We will finish the proof by showing vol(ẽ) vol(ρ(ē)). We recall a well-known property of affine functions x Ax+v, namely that they scale any volume by the same factor det A. For ρ this implies that for any measurable set U R n we have vol(ρ(u)) = vol(u) det(qr) = vol(u) det Q det R = vol(u) det Q. (20) Notice that ρ 1 (Ẽ) is an ellipsoid containing ρ 1 (E i ) ρ 1 (H i ) = E(0, I) H, and hence We thus obtain 1 vol(ρ (Ẽ)) vol(ē). (21) vol(ẽ) = vol(ρ 1 (Ẽ)) det Q (by 20) vol(ē) det Q (by 21) = vol(ρ(ē)) (by 20). Hence, we can accept Assumption 26 without loss of generality. We claim that under Assumption 26, the ellipsoid E i+1 is given by ( ) E i+1 = x Rn n ( x 1 1 ) 2 + n2 1 n n + 1 n 2 x 2 j 1. (22) We will not show formally that the above description is a minimum ellipsoid containing E(0, I) {x R n x 1 0}. However, we will show that E i+1 as described above contains E(0, I) {x R n x 1 0}, and that the ratio of its volume and the volume of E i satisfy the inequality of Lemma 24. Of course, even without accepting that E i+1 is indeed the smallest ellipsoid containing E(0, I) {x R n x 1 0}, this shows Lemma 24. Furthermore, also in the analysis that follows, we never need to prove that E i+1 is the smallest ellipsoid containing E(0, I) {x R n x 1 0}, since we can simply assume that we work with the description of E i+1 given by (22) in the ellipsoid algorithm, without assuming that it has minimum volume. We will later generalize the description of E i+1 given in (22) to the general case when E i is not necessarily the unit ball, and {x R x 1 0} is replaced by a general halfspace going through the center of E i. Lemma 27. Under Assumption 26, we have that E i+1 as described by (22) satisfies E i+1 E i {x R n x 1 0}.

5 Proof. Let x E i {x R n x 1 0}. We have ( n + 1 n ) 2 ( x 1 1 ) 2 + n2 1 n + 1 n 2 = n2 + 2n + 1 n 2 x 2 1 ( n + 1 n x 2 j = 2n + 2 n 2 x 2 1 2n + 2 n 2 x n 2 + n2 1 n 2 ) 2 2x 1 n n 2 + n2 1 n 2 j=1 x 2 j }{{} 1 x 2 i (x E i = E(0, I)) = 2n + 2 n 2 x 1 (x 1 1) +1 (0 x }{{} 1 1) 1, and thus x E i+1. 0 We are now ready to prove Lemma 24.