Iterative Rounding and Relaxation

Iterative Rounding and Relaxation James Davis Department of Computer Science Rutgers University Camden jamesdav@camden.rutgers.edu March 11, 2010 James Davis (Rutgers Camden) Iterative Rounding 1 / 58

Iterative Rounding and Relaxation Ingredients: Linear Program Theorem about individual variable values in LP solution Technique: Solve LP Round some variables Remove variables, relax constraints Iterate James Davis (Rutgers Camden) Iterative Rounding 2 / 58

Brief History Survivable Network Design Jain (1998) MBDST Goemans (2006) Singh and Lau (2007, 2008) Bansal, Khandekar, Nagarajan (2008) James Davis (Rutgers Camden) Iterative Rounding 3 / 58

Outline 1 Introduction: Vertex Cover 2 LP Formulation 3 Algorithm 4 Analysis Bounding Cost Bounding Degrees 5 Main Theorem Laminar Lemma Proof 6 Improvement James Davis (Rutgers Camden) Iterative Rounding 4 / 58

Vertex Cover Input: A graph G = (V, E) Non-negative costs on vertices c v Output: A minimum-cost collection of vertices so that each edge in G is incident on at least one vertex in the collection James Davis (Rutgers Camden) Iterative Rounding 5 / 58

Vertex Cover min v V c v x v x u + x v 1 e = (u, v) x v 0 v V James Davis (Rutgers Camden) Iterative Rounding 6 / 58

Vertex Cover: Main Theorem Theorem (Nemhauser-Trotter) In a basic optimal LP solution, each x v { 1 2, 1, 0} Simple 2-appx algorithm: Solve the Vertex Cover LP Include all vertices with x v 0 in our cover James Davis (Rutgers Camden) Iterative Rounding 7 / 58

MBDST: Problem Statement Input: Output: A graph G = (V, E) Costs c e 0 for all e E A set W V Degree bounds b v for all v W Find a min-cost spanning tree (V, F) that doesn t violate degree bounds. James Davis (Rutgers Camden) Iterative Rounding 8 / 58

Example MST 1 MBDST 1 1 Cost = 3 Cost = 7 James Davis (Rutgers Camden) Iterative Rounding 9 / 58

MBDST Properties Notation: S: any subset of vertices E(S): edges with both endpoints in S F: set of edges in MBDST Properties: Spanning: Acyclic: Degree Bounds: Exactly V 1 edges in F For S 2, at most S 1 edges of F in E(S) At most b v edges of F incident on v James Davis (Rutgers Camden) Iterative Rounding 11 / 58

Integer Program x e = 1 if e F and x e = 0 otherwise min c e x e (Objective) e E x e = V 1 (1) e E e E(S) e δ(v) x e S 1 S V, S 2 (2) x e b v v W (3) x e {0, 1} e E James Davis (Rutgers Camden) Iterative Rounding 12 / 58

Linear Program x e = 1 if e F and x e = 0 otherwise min c e x e (Objective) e E x e = V 1 (1) e E e E(S) e δ(v) x e S 1 S V, S 2 (2) x e b v v W (3) x e 0 e E James Davis (Rutgers Camden) Iterative Rounding 13 / 58

LP Properties There are exponentially many constraints (2) Ellipsoid method Separation oracle (1) and (3) are easy to check (2) requires work Skip Oracle James Davis (Rutgers Camden) Iterative Rounding 14 / 58

Separation Oracle: Flow Network 0 1 1 0 1 James Davis (Rutgers Camden) Iterative Rounding 15 / 58

Separation Oracle: Flow Network s 0 1 1 0 1 t James Davis (Rutgers Camden) Iterative Rounding 15 / 58

Separation Oracle: Flow Network s 1/2 3/2 0 1/2 1 1/2 1 0 1 t James Davis (Rutgers Camden) Iterative Rounding 15 / 58

Separation Oracle: Flow Network s 1/2 3/2 0 1/2 1/2 1/2 1\2 0 1/2 t James Davis (Rutgers Camden) Iterative Rounding 15 / 58

Separation Oracle: Flow Network s 1/2 3/2 0 1/2 1/2 1/2 1\2 0 1/2 1 1 1 1 t James Davis (Rutgers Camden) Iterative Rounding 15 / 58

Separation Oracle:s-t cut Capacity = 2 + 1 2 2 + 1 2 4 James Davis (Rutgers Camden) Iterative Rounding 16 / 58

Separation Oracle The capacity across S is V + ( S 1) The capacity across S is at least V iff x e e E(S) e E(S) The max-flow from s to t is V iff (2) are satisfied x e S 1 James Davis (Rutgers Camden) Iterative Rounding 17 / 58

Main Theorem x =< x 1, x 2,..., x E >: solution to LP Support( x): set of edges s.t. x e > 0 Theorem For any basic solution x to the linear program either: 1 v with exactly one incident edge e Support( x) x e = 1 2 v W with at most 3 edges of Support( x) incident on v Condition 1 identifies a leaf in the tree Condition 2 identifies a vertex with sufficiently small number of nonzero incident edges James Davis (Rutgers Camden) Iterative Rounding 19 / 58

Algorithm F = While V > 1 x LP solution on < G, W > Remove all edges e with x e = 0 If condition 1 is satisfied by x Add (u, v) to F Remove v and (u, v) from G If u W reduce b u by 1 If condition 2 is satisfied by x Remove v from W James Davis (Rutgers Camden) Iterative Rounding 20 / 58

Linear Program min c e x e (Objective) e E x e = V 1 (1) e E e E(S) e δ(v) x e S 1 S V, S 2 (2) x e b v v W (3) x e 0 e E James Davis (Rutgers Camden) Iterative Rounding 21 / 58

LP Relationships In each iteration LP is in the same family Same separation oracle The Main Theorem applies to each LP If condition 1 is satisfied LP is incrementally modified: Delete an x e variable Modify (1) constraint Remove some (2) constraints Modify some (3) constraints If condition 2 is satisfied LP is incrementally modified: Remove a (3) constraint James Davis (Rutgers Camden) Iterative Rounding 22 / 58

Bounding Cost Theorem The tree returned by our algorithm has cost at most LP OPT G' e 1 v IH: cost(f ) LP (G ) cost(f ) + c e LP (G ) + c e x e LP(G ) + c e x e LP: current lin. prog. LP : new lin. prog. F : MBDST in G = LP(G) James Davis (Rutgers Camden) Iterative Rounding 25 / 58

Bounding Cost Lemma LP(G ) is a feasible solution to LP (G ) Changes: 1 1 on RHS, 1 on LHS 2 Remove constraints 3 1 on RHS, 1 on LHS; Remove constraints x e = V 1 (1) e E e E(S) e δ(v) x e S 1 (2) x e b v (3) James Davis (Rutgers Camden) Iterative Rounding 26 / 58

Min-Cost Spanning Trees Recap: Spanning tree has optimal cost Degree bounds? Implications: Theorem For any basic solution x to the linear program either: 1 v with exactly one incident edge e Support( x) xe = 1 2 v W with at most 3 edges of Support( x) incident on v James Davis (Rutgers Camden) Iterative Rounding 27 / 58

Min-Cost Spanning Trees Recap: Spanning tree has optimal cost Degree bounds? Implications: Theorem For any basic solution x to the linear program either: 1 v with exactly one incident edge e Support( x) xe = 1 2 v W with at most 3 edges of Support( x) incident on v When W = we have OPT James Davis (Rutgers Camden) Iterative Rounding 27 / 58

Degree Bounds u 1 b v, b u 1 b v never violated b u adjusted v Algorithm x LP solution on < G, W > Remove e / Support( x) If condition 1 is satisfied by x Add (u, v) to F Remove v and (u, v) from G If u W reduce b u by 1 If condition 2 is satisfied by x Remove v from W James Davis (Rutgers Camden) Iterative Rounding 29 / 58

Degree Bounds 1 u b v, b u 1 b v never violated b u adjusted v Algorithm x LP solution on < G, W > Remove e / Support( x) If condition 1 is satisfied by x Add (u, v) to F Remove v and (u, v) from G If u W reduce b u by 1 If condition 2 is satisfied by x Remove v from W James Davis (Rutgers Camden) Iterative Rounding 29 / 58

Degree Bounds Algorithm x LP solution on < G, W > Remove e / Support( x) If condition 1 is satisfied by x Add (u, v) to F Remove v and (u, v) from G b v 1 All 3 edges may be in F b v violated by at most 2 If u W reduce b u by 1 If condition 2 is satisfied by x Remove v from W James Davis (Rutgers Camden) Iterative Rounding 29 / 58

Analysis Summary Cost: Spanning tree has optimal cost Degree Bounds: No degree bound violated by more than 2 Theorem (Goemans) The algorithm for MBDST produces a spanning tree in which the degree of v is at most b v + 2 for v W and has cost no greater than OPT James Davis (Rutgers Camden) Iterative Rounding 30 / 58

Main Theorem Theorem For any basic solution x to the linear program either: 1 v with exactly one incident edge e Support( x) x e = 1 2 v W with at most 3 edges of Support( x) incident on v James Davis (Rutgers Camden) Iterative Rounding 32 / 58

Linear Program min c e x e (Objective) e E x e = V 1 (1) e E e E(S) e δ(v) x e S 1 S V, S 2 (2) x e b v v W (3) x e 0 e E James Davis (Rutgers Camden) Iterative Rounding 33 / 58

Laminar Lemma Lemma For any basic LP solution x there is a Z W and a collection L of S V where: 1 S L, S is tight; v Z, v is tight 2 The vectors χ E(S) and χ δ(v) are independent 3 L + Z = Support( x) 4 L is laminar James Davis (Rutgers Camden) Iterative Rounding 34 / 58

Characteristic Vector 1 3 χ E(S) =< 1, 1, 1, 0, 0, 0, 0 > χ δ(v) =< 0, 1, 1, 0, 1, 0, 0 > James Davis (Rutgers Camden) Iterative Rounding 35 / 58

Laminar Sets Intersecting Sets A B Laminar Sets No intersecting sets A B A B B A James Davis (Rutgers Camden) Iterative Rounding 36 / 58

Property of L Lemma If all S L contain at least 2 vertices then L V 1 Use induction on V Base case: V = 2 Induction step Shrink smallest set to vertex Generates L and V L is laminar L = L 1 V V 1 L V 1 L V 1 James Davis (Rutgers Camden) Iterative Rounding 38 / 58

Property of Support( x) Lemma Support( x) < V + W Recall property 3 of Laminar Lemma: L + Z = Support( x) Support( x) = L + Z L + W < V + W (Previous Lemma) James Davis (Rutgers Camden) Iterative Rounding 39 / 58

From Laminar Lemma to Main Theorem Theorem For any basic solution x to the linear program either: 1 v with exactly one incident edge e Support( x) x e = 1 2 v W with at most 3 edges of Support( x) incident on v Suppose Main Theorem wasn t true: For every v V there are at least 2 edges incident on it For every v W there are at least 4 edges incident on it Support( x) 1 (2( V W ) + 4( W )) 2 = V + W (Contradiction!) James Davis (Rutgers Camden) Iterative Rounding 40 / 58

From Laminar Lemma to Main Theorem Theorem For any basic solution x to the linear program either: 1 v with exactly one incident edge e Support( x) x e = 1 2 v W with at most 3 edges of Support( x) incident on v S=V-v e E(S) x e V 2 e E x e = V 1 e δ(v) x e 1 x e 1 x e 1 x e = 1 James Davis (Rutgers Camden) Iterative Rounding 41 / 58

From Laminar Lemma to Main Theorem Theorem For any basic solution x to the linear program either: 1 v with exactly one incident edge e Support( x) x e = 1 2 v W with at most 3 edges of Support( x) incident on v v S=V-v e E(S) x e V 2 e E x e = V 1 e δ(v) x e 1 x e 1 x e 1 x e = 1 James Davis (Rutgers Camden) Iterative Rounding 41 / 58

From Laminar Lemma to Main Theorem Theorem For any basic solution x to the linear program either: 1 v with exactly one incident edge e Support( x) x e = 1 2 v W with at most 3 edges of Support( x) incident on v u v S={u,v} e E(S) x e V 2 e E x e = V 1 e δ(v) x e 1 x e 1 x e 1 x e = 1 James Davis (Rutgers Camden) Iterative Rounding 41 / 58

LP Background min c i x i (Objective) a 11 x 1 + a 12 x 2 +... + a 1n x n b 1 (1) a 21 x 1 + a 22 x 2 +... + a 2n x n b 2 (2)... =... a m1 x 1 + a m2 x 2 +... + a mn x n b m (m) x i 0 (Non-Negative) James Davis (Rutgers Camden) Iterative Rounding 44 / 58

LP Background Linear Program Constraints define half-spaces Objective is a hyperplane Solution always a corner n tight constraints Constraints lin. ind. min c i x i a 11 x 1 + a 12 x 2 +... + a 1n x n b 1 (1) a 21 x 1 + a 22 x 2 +... + a 2n x n b 2 (2)... =... a m1 x 1 + a m2 x 2 +... + a mnx n b m (m) x i 0 James Davis (Rutgers Camden) Iterative Rounding 45 / 58

LP Background Constraints define half-spaces Objective is a hyperplane Solution always a corner E tight constraints Constraints lin. ind. MBDST LP min c ex e e E x e = V 1 (1) e E x e S 1 (2) e E(S) x e b v (3) e δ(v) x e 0 James Davis (Rutgers Camden) Iterative Rounding 45 / 58

Laminar Lemma Proof Lemma e E(S) x e is supermodular e E(S) x e + e E(T ) x e e E(S T ) x e + e E(S T ) x e S T James Davis (Rutgers Camden) Iterative Rounding 47 / 58

Laminar Lemma Proof Lemma S, T are tight, S and T cross, then S T, S T are tight and χ E(S) + χ E(T ) = χ E(S T ) + χ E(S T ) ( S 1) + ( T 1) = ( S T 1) + ( S T 1) x e + (feasibility) E(S T ) E(S T ) x e x e + x e (supermodularity) E(S) E(T ) Since S and T are tight, these are all equalities James Davis (Rutgers Camden) Iterative Rounding 48 / 58

Laminar Lemma Proof: Finding L Lemma L that is laminar and Span(T ) Span(L), where T contains all tight sets Let L be a maximal laminar collection of T Recall that χ E(S) + χ E(T ) = χ E(S T ) + χ E(S T ) S T Span(T ) Span(L) S (least int. in L) T (int. S) S T and S T S T and S T S T S T S T S T James Davis (Rutgers Camden) Iterative Rounding 49 / 58

Laminar Lemma Proof: Finding L Lemma L that is laminar and Span(T ) Span(L), where T contains all tight sets Let L be a maximal laminar collection of T Recall that χ E(S) + χ E(T ) = χ E(S T ) + χ E(S T ) T Span(T ) Span(L) S (least int. in L) T (int. S) S T and S T S T and S T S T S T S T S T James Davis (Rutgers Camden) Iterative Rounding 49 / 58

Laminar Lemma Proof: Finding Z Lemma For any basic LP solution x there is a Z W and a collection L of S V where: 1 S L, S is tight; v Z, v is tight 2 The vectors χ E(S) and χ δ(v) are independent 3 L + Z = Support( x) 4 L is laminar (T, Y ) spans R Support( x) (L, Y ) spans R Support( x) To obtain (L, Z ) remove v Y that are dependent James Davis (Rutgers Camden) Iterative Rounding 50 / 58

Recap LP formulation Main Theorem Algorithm Cost no more than OPT Degree bounds violated by at most 2 Main Theorem Proof Laminar Lemma Proof James Davis (Rutgers Camden) Iterative Rounding 51 / 58

Improved Main Theorem Theorem If x is a basic solution to LP where W then there is a v, s.t. δ(v) Support( x) b v + 1 James Davis (Rutgers Camden) Iterative Rounding 53 / 58

Algorithm Phase 1: While W x LP solution on < G, W > For all x e = 0, remove e from E Remove v from W if there are at most b v + 1 edges of δ(v) in Support( x) Phase 2: Run algorithm on < G, > James Davis (Rutgers Camden) Iterative Rounding 54 / 58

Analysis Theorem (Singh and Lau) The improved algorithm for MBDST produces a spanning tree in which the degree of v is at most b v + 1 for v W and has cost no greater than OPT James Davis (Rutgers Camden) Iterative Rounding 55 / 58

References Kamal Jain. A factor 2 approximation algorithm for the generalized Steiner network problem. Combinatorica, 21:39-60, 2001. Michel X. Goemans. Minimum bounded-degree spanning trees. FOCS 06 Mohit Singh and Lap Chi Lau. Approximating minimum bounded degree spanning trees to within one of optimal. STOC 07. James Davis (Rutgers Camden) Iterative Rounding 56 / 58

Acknowledgements Thanks to David Shmoys and David Williamson for letting us use the manuscript of their forthcoming book, The Design of Approximation Algorithms. James Davis (Rutgers Camden) Iterative Rounding 57 / 58

Thank You! Return to Oracle James Davis (Rutgers Camden) Iterative Rounding 58 / 58