Discrete Optimization 2010 Lecture 7 Introduction to Integer Programming Marc Uetz University of Twente m.uetz@utwente.nl Lecture 8: sheet 1 / 32 Marc Uetz Discrete Optimization
Outline 1 Intro: The Matching Polytope 2 Chvátal-Gomory Cuts and Cutting Planes 3 Branch & Bound and Lagrangian Relaxation Lecture 8: sheet 2 / 32 Marc Uetz Discrete Optimization
The Matching Polytope Definition With G = (V, E), E = m, and A as the node-edge incidence matrix of G, the matching polytope is P I = conv{x {0, 1} m x = matching} = conv{x {0, 1} m Ax 1} If we drop the integrality condition on the variables x e, we get Definition For the same G, the fractional matching polytope is P = {x R m Ax 1, x 0} Lecture 8: sheet 3 / 32 Marc Uetz Discrete Optimization
Matching Polytope Theorem Given undirected graph G = (V, E) with node-edge incidence matrix A we have P I = conv{x {0, 1} n Ax 1} {x R n Ax 1, x 0} = P and = holds if (and only if) G is bipartite. P I P is trivial: P I is the convex hull of all matchings, and any matching is a feasible point in P, and P is convex When G is bipartite, we know that all vertices of P are integer. So all vertices of P are matchings, hence P I If all vertices of P are in P I, we conclude P P I (P I convex) Lecture 8: sheet 4 / 32 Marc Uetz Discrete Optimization
Only If : Matching Polytope Non-Bipartite Graphs The matching polytope is more complicated (that is, P) for non-bipartite!"#$%&$%'()%*+,,#-#./)%,(0%1./.023%4'(/5#620)#).7%1026"$ graphs. F F ; ; G G +.+=>?$,@(-% /*+&(@$101#&' -./0#.-& H nonbip. +.+=>?$,@(-% graph /*+&(@$101#&' linear system -./0#.-& polytope H P!"#$#%&$'(#)%*+,$-./0#.-& *12 Matching34).+56789898:97;9898:9789;98:978989;:< polytope is P I = conv{(0,0,0), (1,0,0),!"#$#%&$'(#)%*+,$-./0#.-& (0,1,0), (0,0,1)} *12 34).+56789898:97;9898:9789;98:978989;:< A(+$>&$.>#(*+&B$>0$(BB*+,$ *+&C"(/*#02$D ; ED F ED G! ; A(+$>&$.>#(*+&B$>0$(BB*+,$ Lecture 8: sheet 5 / 32 Marc Uetz Discrete Optimization 3 3
Integer Programming Consider discrete optimization problem for A Z m n, b Z m max c t x s.t. x P = {x R n Ax b} x integer (ILP) This is equivalent to max c t x s.t. x P I (LP) with P I = conv{x R n Ax b, x integer} (why? all integer x P are P I, and linearity of objective) Lecture 8: sheet 6 / 32 Marc Uetz Discrete Optimization
Goal facets of P I, describing the convex hull of integer points of P by linear inequalities Definition A maximal face F of P, F P, is a facet. (For P full dimensional, say dim(p) = n, then dim(f ) = n 1 and F = {x ax = b} P) Lecture 8: sheet 7 / 32 Marc Uetz Discrete Optimization
Chvátal-Gomory Cuts P I = conv{x R n Ax b, x integer} also λ t Ax λ t b is a feasible inequality for P I (λ [0, 1] m ) and if λ t A happens to be an integer vector, a feasible inequality is also λ t Ax λ t b (Chvátal-Gomory Cut) Example: x 1 + 2x 2 4 x 1 1 x 0, integer Take λ = ( 1 2, 1 2 ), then we get x 1 + x 2 5 2, but x integer, so get (stronger) inequality x 1 + x 2 2 Lecture 8: sheet 8 / 32 Marc Uetz Discrete Optimization
Example 2 1 x 1 + 2x 2 = 4 1 4 x 1 + x 2 = 2 = integer solutions of Ax b Chvátal-Gomory cut x 1 + x 2 2 cuts off non-integer vertex (1, 3 2 ) Lecture 8: sheet 9 / 32 Marc Uetz Discrete Optimization
Chvátal-Gomory Cuts P I = conv{x R n Ax b, x integer} Definition Linear Relaxation: P = {x R n Ax b} 1 st Chvatal Closure: P 1 = {x R n λ t Ax λ t b, λ [0, 1] m with λ t A integer} Observations P 1 is a polyhedron (there are only finitely many λ t A integer, if λ [0, 1] m, Exercise) P 1 P (taking as λ all unit vectors gives Ax b) P 1 P I (if λ t A integer, λ t Ax λ t b is valid for P I ) Lecture 8: sheet 10 / 32 Marc Uetz Discrete Optimization
Chvátal-Gomory Cuts By iterating the above construction, we get Theorem (Chvátal-Gomory) P P 1 P 2 P k P I There exists some finite k N such that P I = P k. (Convex hull of integer solutions described by linear inequalities) Definition Chvátal rank of P := min{k P I = P k } For proof of the theorem, see reader [pp.95/96 of Faigle et al.] (show: if P I, each facet of P I is valid for some P k ) Previous example: P I = P 1 Lecture 8: sheet 11 / 32 Marc Uetz Discrete Optimization
Example for P 1 = P I 2 P 1 x 1 + 2x 2 = 4 P I = P 1 1 x 1 + x 2 = 2 = integer solutions of Ax b facets of P I : x 1 = 0, x 2 = 0, x 1 = 1, x 1 + x 2 = 2 4 Lecture 8: sheet 12 / 32 Marc Uetz Discrete Optimization
Intuition about the Chvátal Closure P 1 : Move all supporting hyperplanes of P towards P I, until they touch some integer point Z n Lecture 8: sheet 13 / 32 Marc Uetz Discrete Optimization
Remark P w. Chvátal rank > 1 (Example below, P P 1 P 2 = P I ) P 2 = P I Lecture 8: sheet 14 / 32 Marc Uetz Discrete Optimization
Chvátal-Gomory Cuts for Matching A = node-edge incidence matrix of a graph G = (V, E) max 1 x s.t. Ax 1 x 0, x integer [ ] x e 1 v V e δ(v) Consider any S V, and let λ v = 1 2 for v S, λ v = 0 otherwise, and λ e = 1 2 for e δ(s), then x e 1 2 v S e δ(v) e δ(s) x e 1 ( S 0) 2 x e 1 2 S e S Lecture 8: sheet 15 / 32 Marc Uetz Discrete Optimization
Odd Set Inequalities After rounding down right-hand-side, Chvátal-Gomory cuts for S with S odd: e S x e S 1 2 (odd-set inequalities) S Lecture 8: sheet 16 / 32 Marc Uetz Discrete Optimization
Matching Polytope General Graphs Theorem With A = node-edge incidence matrix of G = (V, E), can show that the matching polytope P I = conv{x R n Ax 1, x 0, integer} equals the following polytope Q = {x R n Ax 1, x 0, e S x e S 1 2 S odd} (odd-set inequalities are sufficient; particularly, P I = P 1 = Q) Proof idea Q P I : Any facet of P I equals {x x e = 0} or {x e δ(v) x e = 1} or {x e S x e = S 1 2 }, so is valid for Q Lecture 8: sheet 17 / 32 Marc Uetz Discrete Optimization
Example!"#$%&$%'()%*+,,#-#./)%,(0%1./.023%4'(/5#620)#).7%1026"$ F F ; ; G G +.+=>?$,@(-% /*+&(@$101#&' -./0#.-& H nonbip. +.+=>?$,@(-% graph /*+&(@$101#&' linear system -./0#.-& linear relaxation H P 3!"#$#%&$'(#)%*+,$-./0#.-& *12 Matching34).+56789898:97;9898:9789;98:978989;:< polytope is P I = conv{(0,0,0), 3 (1,0,0),!"#$#%&$'(#)%*+,$-./0#.-& (0,1,0), (0,0,1)} *12 34).+56789898:97;9898:9789;98:978989;:< A(+$>&$.>#(*+&B$>0$(BB*+,$ *+&C"(/*#02$D ; ED F ED G! ; A(+$>&$.>#(*+&B$>0$(BB*+,$ *+&C"(/*#02$D I+$,&+&@(/2$J.@$(+0$.BB$1&#$.K$+.B&1 ; ED F ED G! ;.BB$1&#$.K$+.B&1 L2$! &46"95<2"95!L D &! 7ML=;M:NF Matching polytope obtained by odd-set inequality x 1 + x 2 + x 3 1. I+$,&+&@(/2$J.@$(+0$.BB$1&#$.K$+.B&1.BB$1&#$.K$+.B&1 L2$! &46"95<2"95!L D &! 7ML=;M:NF Lecture 8: sheet 18 / 32 Marc Uetz Discrete Optimization
Cutting Plane Algorithms Want to solve ILP problem max. cx, s.t. Ax b, x integer equivalently: max cx, s.t. x P I P = {x Ax b} = feasible solutions of LP relaxation 1 solve LP relaxation: x = argmax{cx x P} 2 while (solution x not integer) find valid inequality for P I, violated by x, separating fractional point x from P I (find a Q n, b 0 Q, a t x b 0 for all x P I, but a t x > b 0 ) add new inequality a t x b 0 to LP relaxation P solve new LP relaxation: x = argmax{cx x P} Justification for this approach: Chvátal-Gomory theorem Lecture 8: sheet 19 / 32 Marc Uetz Discrete Optimization
Remarks on Cutting Plane Algorithm Correctness? termination with integer optimal solution of LP relaxation Termination? depends on cuts; Chvátal-Gomory cuts do it in finite time Efficient? it works in practice, as we needn t complete P I, but only at optimal vertex (the art is to find deep cuts efficiently) Lecture 8: sheet 20 / 32 Marc Uetz Discrete Optimization
Outline 1 Intro: The Matching Polytope 2 Chvátal-Gomory Cuts and Cutting Planes 3 Branch & Bound and Lagrangian Relaxation Lecture 8: sheet 21 / 32 Marc Uetz Discrete Optimization
Branch-and-Bound Algorithm (Land & Doig 1960) want to solve ILP: max. c t x s.t. Ax b, x integer now introduce simpler cuts - but need to recurse define recursive algorithm, B&B(LP) current best solution x = 1 Solve relaxation LP: x 0 = argmax{c t x Ax b} 2 If (c t x 0 c t x ) /*bounding: if (c t x 0 < c t x ), forget it */ if x 0 integer if c t x 0 > c t x, update x = x 0 return x else let xi 0 = fractional variable, define two subproblems LP1: add x i xi 0 to LP LP2: add x i xi 0 to LP branch (divide) in two subproblems: return x = argmax{c t B&B(LP1),c t B&B(LP2)} Lecture 8: sheet 22 / 32 Marc Uetz Discrete Optimization
Example: Cuts in B & B add x 2 2 2 add x 2 1 1 x 1 + 2x 2 = 4 1 x 1 + x 2 = 2 4 Lecture 8: sheet 23 / 32 Marc Uetz Discrete Optimization
Example maximize s.t. 17x 10x x x x 1 1 1 1 1 $ $ $ 12x 7x x x x 2 2 2 2 2 # # "! 40 5 0 Z optimal solution LP relaxation: 68 #$%&'()*'+#$*,(*-(./(01'&2&#$*,3(2 1 3 with x 1 = 4 56789(2 5 3 and x 2 = 10 : 54;78 3 in first iteration get two subproblems: =(>&'+1(?@A888 LP1: add x 1 1 "0*C'1%)3 LP2: add x 1 2 Lecture 8: sheet 24 / 32 Marc Uetz Discrete Optimization
Example of Branch & Bound Tree (no bounding) Lecture 8: sheet 25 / 32 Marc Uetz Discrete Optimization
Branch & Bound (fictitious example) Lecture 8: sheet 26 / 32 Marc Uetz Discrete Optimization
LP and Lagrangian Relaxation In Branch&Bound, we solve LP-relaxation for each subproblem represented by a node of the search tree. LP upper bound used for bounding (prune the search tree) used to select at which active node to continue (e.g., depth first search, best upper bound first,... ) Important features are: quality of upper bound (the smaller the better) computation time (the shorter the better) instead of LP relaxation, may also use Lagrangian relaxation Lecture 8: sheet 27 / 32 Marc Uetz Discrete Optimization
LP and Lagrangian Relaxation max c t x s.t. Ax b ( complicating constraints) Bx d (IP) x integer Lagrangian relaxation (λ 0) L(λ) = max x c t x λ t (Ax b) or L(λ) = λ t b + max x (c t λ t A)x s.t. Bx d x integer (LR(λ)) Idea: penalize violation of constraints instead of enforcing them Exercise: L(λ) is an upper bound on optimum IP solution. Lecture 8: sheet 28 / 32 Marc Uetz Discrete Optimization
LP and Lagrangian Relaxation Best possible Lagrangian upper bound is Langragian Dual: LD = min λ 0 L(λ) a a note: λ is unrestricted in sign if we relax equality constraints Ax = b LP upper bound: LP = max x {c t x Ax b, Bx d} denote by ILP the optimum solution value of the integer program Theorem (for maximization problems) ILP LD LP (i.e., best Lagrangian bound is never worse than the LP relaxation) Lecture 8: sheet 29 / 32 Marc Uetz Discrete Optimization
Proof (of 2nd inequality) Use (strong) linear programming duality twice: LD = min λ 0 {b t λ + max x {(c t λ t A)x Bx d, x integer}} = }{{} LP duality min λ 0 {b t λ + max x {(c t λ t A)x Bx d}} min λ 0 {b t λ + min y {d t y B t y = (c A t λ), y 0}} = min λ,y {b t λ + d t y A t λ + B t y = c, y 0, λ 0}} = }{{} LP duality max x {c t x Ax b, Bx d} = LP Note: = holds in 2nd step for example if B is totally unimodular Lecture 8: sheet 30 / 32 Marc Uetz Discrete Optimization
An Example: TSP TSP Given (w.l.o.g. complete) undirected graph G = (V, E), V = n, edge lengths c e 0 e E, find shortest Hamiltonian cycle T (= tour). Example a short but non-optimal tour for all 15,112 cities in D Lecture 8: sheet 31 / 32 Marc Uetz Discrete Optimization
IP Formulation for TSP use variables x e = 1 if e T, 0 otherwise min e E c e x e s.t. x e = n e E x e = 2 e δ(v) x e S 1 e S x {0, 1} n v V S V Subtour elimination constraints (SEC), need to forbid solution: Lecture 8: sheet 32 / 32 Marc Uetz Discrete Optimization