18.997 Topic in Combinatorial Optimization April 29th, 2004 Lecture 21 Lecturer: Michel X. Goeman Scribe: Mohammad Mahdian 1 The Lovaz plitting-off lemma Lovaz plitting-off lemma tate the following. Theorem 1 Let G = (V {},E) be a graph uch that U V : d(u) k, (1) where d(u) denote the number of edge between U and U, and k 2. Alo, aume that d() (the degree of the vertex ) i even. Then for every (, t) E, there exit (, u) E uch that the graph G =(V, E \{(, t), (, u)} {(t, u)}) alo atifie the condition (1). Proof: Let S denote the et of neighbor of in G (i.e., S = {u V : (, u) E}). Fix a t S. We would like to how that there exit a u S uch that condition (1) hold for the graph G =(V, E \{(, t), (, u)} {(t, u)}). For the ake of contradiction, aume thi doe not hold. Thi mean that for every u S, there exit a et U, = U V, uch that d(u) k + 1 and u, t U (See Figure 1). In other word, the collection of all et U with d(u) k + 1 and t U cover S. Let C be a collection of maximal et U with d(u) k +1 and t U that cover S. t u U Figure 1: A et U with d(u) k +1 and u U For every U C,we have d(u) k +1 and d(u {}) k (the latter inequality hold becaue d(u {}) = d(v \ U) k by (1)). Therefore, 1 d(u) d(u {}) = d(, U) d(, V \ U), and o d(, V \ U) +1 d(, U). On the other hand, d(, V \ U) + d(, U) i equal to the degree of, which i an even number. Thu, d(, V \ U) and d(, U) have the ame parity. Therefore, d(, V \ U) d(, U). In other word, d(, U) 1 d(). Thi, together with the fact that t U for 2 21-1
U 1 U 2 t U 3 Figure 2: Three et U 1,U 2,U 3 atifying propertie (2) every U C, how that two of the et U C are not enough to cover S (i.e., U i U j S for every U i,u j C). Therefore, C mut contain at leat three et U 1,U 2,U 3 uch that t U 1 U 2 U 3 U 1 \ (U 2 U 3 ) U 2 \ (U 1 U 3 ) U 3 \ (U 1 U 2 ). (2) See Figure 2. We now ue the following inequality which i a conequence of the three-way ubmodularity of the function d. d(u 1 )+ d(u 2 )+ d(u 3 ) d(u 1 U 2 U 3 ) + d(u 1 \ (U 2 U 3 )) + d(u 2 \ (U 1 U 3 )) + d(u 3 \ (U 1 U 2 )). (3) It i traightforward to check all cae for an edge e and how that in each cae, e i counted at leat a many time on the left-hand ide a it i counted on the right-hand ide. Thi prove the above inequality. In fact, there i at leat one edge t that i counted three time on the left-hand ide, but only once on the right-hand ide. Therefore, we can trengthen inequality (3) by adding a +2 to it right-hand ide. Since every term on the left-hand ide of (3) i at mot k +1 (by the definition of U i ) and every term on the right-hand ide i at leat k (by aumption (1) on the graph G and propertie (2)), the above inequality implie: 3k +3 4k +2 k 1. Thi give u a contradiction ince K wa aumed to be at leat 2. 2 Submodular function minimization In the ret of thi lecture, we ketch an algorithm for ubmodular function minimization. Thi i from Chapter 45 of Lex Schrijver book. 21-2
0101 Figure 3: The extended polymatroid EP f Problem Statement. Given an oracle for a function f :2 S Z, find a et U S that minimize f(u) over all ubet of S. We aume, without lo of generality, that f( ) = 0, otherwie we can minimize the function f(u) f( ) intead of f. Thi problem ha many application. A an example, conider the matroid interection problem that we dicued in previou lecture. We howed that the convex hull of the of the interection of two matroid i the et of all vector x uch that for every U S, x(u) r 1 (U) and x(u) r 2 (U), where r 1 and r 2 are rank function of the matroid. Therefore, we can optimize over the interection of two matroid by olving a linear program with the above contraint. It i not obviou how to olve thi linear program, ince it i of exponential ize. However, we can get polynomial-time eparation by minimizing r i (U) x(u) over all U S, and checking if the minimum i non-negative (for i = 1, 2). Thi can be done uing an algorithm that olve the ubmodular function minimization, ince r 1 x i a ubmodular function. Notice that it i not obviou that minimizing a ubmodular function given by an oracle i poible in polynomial time. Clearly, without the aumption of ubmodularity, it i not poible to find the minimum of the function before calling the oracle on all 2 n point on which the function i defined. However, in the ret of thi lecture we will ketch an algorithm due to Lex Schrijver that olve thi problem for ubmodular function in polynomial time. We tart by defining two polyhedra related to a ubmodular function f. The firt polyhedron i called the extended polymatroid aociated with f, and i defined a follow: EP f = {x R S : x(u) f(u), U S}. Notice that thi definition doe not require x 0. A an example, if S = {1, 2} and f i defined by f( ) = 0,f({1}) = 1,f({2}) = 1,f({1, 2}) = 0, then the extended polymatroid EP f i the haded area in Figure 3. Prior to the algorithm of Schrijver, there wa a polynomial-time (but not trongly polynomialtime) algorithm for ubmodular function minimization baed on the ellipoid algorithm and the polyhedron EP f. We define the econd polyhedron, which i called the bae polyhedron, a follow: B f = {x R S : x(s) = f(s), x(u) f(u), U S}. 21-3
For example, for the function in the previou example, the polyhedron B f conit of one point that i marked by a cro in Figure 3. Our goal i the following. Goal. Find a et U S and a vector x B f uch that x(v) 0 v U (4) x(v) 0 v U (5) x(u ) = f (U) (6) (7) Claim 2 If we can find a et U and vector x B f atifying propertie (4) (6), then U i the et that minimize f (U). Proof: Thi i becaue for every et W S, f (U ) = x(u ) x(w ) f (W ), where the firt equality follow from property (6), the econd inequality follow from (4) and (5), and the third inequality i a conequence of x B f. It i not clear how one can prove that a vector x belong to B f,ince B f i defined by exponentially many inequalitie. We do thi by expreing x a a convex combination of element that are obviouly in B f. Such element are defined below. In fact, thee element are extreme point (and the only extreme point) of B f, but we do not need thi fact in our proof. Chooe a total order on S. For every v S, we define v = {w S : w v}. The vector b R S i defined by b (v) = f (v {v}) f (v ). Claim 3 For every total order on S, b B f. Proof: By the definition of b, b (S) i a telecopic um that i equal to f (S) f ( ) = f (S). Now, we prove that for every U S, b (U) f (U). We can prove thi by induction on the ize of U. If U = 0, then the tatement i trivial. Otherwie, let v be the maximal element of U (with repect to ), and apply the induction hypothei on U \{v}. Thi give u b (U \{v}) f (U \{v}). Since v i the maximal element of U, we have U v {v}. Therefore, by the ubmodularity of f, we have b (v) = f (v {v}) f (v ) f (U) f (U \{v}). By adding thi inequality with the previou inequality we obtain b (U) = b (U \{v})+ b (v) f (U). We will find a vector x atifying propertie (4) (6) and expre the vector x a a convex combination of b, thereby howing that x B f. We do thi by tarting from an arbitrary x that can be written a a convex combination of b, and modify x (along with the expreion that give x a a convex combination of b ) until there exit a U uch that (x, U ) atifie the deired propertie. Suppoe we have a vector x that can be written a x = k i=1 λ i b i,where λ i > 0 for all i. Since B f R S = R n and all point in B f mut atify an equality, the dimenion of B f i at mot n 1, and hence x can be expreed a a convex combination of n extreme point. So, we aume k n. 21-4
For every and every v S, b (v )= f (v ). We call the et v a prefix (alo known a a lower ideal) of. Therefore, if U S i a prefix of i for every 1 i k, then x(u ) = k i=1 λ i b i (U) = k i=1 λ i f (U) = f (U). Thu, if we can find a et U that i a prefix of i for every 1 i k and atifie x(v) 0for v U and x(v) 0for v U, then we are done. Thi motivate the following definition: Let D =(S, A) be a directed graph on the et of vertice S with the arc et A = {(u, v) : u i v for ome 1 i k}. By thi definition, a et U i a prefix of every i if and only if δ in (U) = in D. Now, let P = {v : x(v) > 0} and N = {v : x(v) < 0}. We conider two cae: Cae 1. D ha no directed path from P to N.In thi cae, let U be the et of vertice v uch that there i a path from v to ome vertex of N. Therefore, U contain N but nothing from P, and i aprefix forevery i. Therefore, (x, U ) atify the propertie (4) (6), and we are done. Cae 2. There i a directed path from P to N. In thi cae, we change either x or the way x i expreed a a convex combination of b. Pick and t on the path from P to N uch that t N, N, and there i an arc from to t in D (detail of the election rule i omitted). We would like to change x or it repreentation to kill the path from P to N.We can do thi either by removing t from N (i.e., increaing x t ), or by removing the arc (, t) from D. The arc (, t) i preent becaue i t for ome i. We focu on one uch i, and will try to get cloer to t in i. t P N Figure 4: Path from P to N and the vertice and t Let χ t denote the unit vector along along the coordinate t (and imilarly for χ ). We how that for ome δ 0, x + δ(χ t χ ) B f,and moreover, we can write x + δ(χ t χ )a a convex combination in which i cloer to t. More preciely, we ue the following lemma a a ubroutine. Lemma 4 Given,, and t, expre the vector b + δ(χ t χ ) for ome δ 0 a a convex combination of b,u for u (, t] = {u : u t}, where,u i the total order that i obtained from by moving u before. Proof: We aume that b = 0. Thi aumption i without lo of generality, becaue we can replace f (U) by f (U) b (U) and apply the argument on thi new function. By the ubmodularity of f, we have b (v) =0 if v or v u b,u (v) = 0 if v u 0 if v = u. The following table how the pattern of non-negative and non-poitive entrie in b,u, for every u (, u]. In thi table, denote a non-poitive entry, + denote a non-negative 21-5
entry, and 0 denote an entry that i zero. u t 0 + 0 0 + 0 0 + 0 0...... t 0... + If the non-negative entry (+) on one row i zero (i.e., b,u (u) = 0 for ome u), then all other entrie of that row mut alo be zero, ince the um of the entrie in each row mut be the ame a the um of the entrie in b (ince they are both equal to f(s)), which i zero. Therefore, in thi cae, we can take δ =0 and ue b,u a the deired convex combination. The other cae i when b,u (u) > 0 for every u (, t]. In thi cae, we can tart from the lat row of the table (i.e., the vector b,t ), and for every row of the table, from the row before t to the row after, add a multiple of the row to the current vector o that the + entry cancel out the correponding entry in the current vector. At the end, we will obtain a vector that ha only one negative entry at poition and one poitive entry at poition t. Furthermore, ince the um of all entrie hould be zero, the abolute value of thee two entrie are equal. Thi mean that for ome δ > 0, we can write δ(χ t χ ) a a convex combination of b,t. We iterate the procedure in the above lemma. Intuitively, every time we get cloer to t in at leat one of i, without changing other i. We might alo increae x t. Therefore, after a finite number of iteration, we will either remove the arc (, t) from D, or will remove t from N.For how and t are choen and the analyi of the running time of thi procedure, ee Lex Schrijver book. 21-6