Integer Programming Methods LNMB 2017 2018 Dion Gijswijt homepage.tudelft.nl/64a8q/intpm/ Dion Gijswijt Intro IntPM 2017-2018 1 / 24
Organisation Webpage: homepage.tudelft.nl/64a8q/intpm/ Book: Integer Programming, Conforti, Conuéjols, and Zambelli (download via Springer Link) Prerequisites: Linear programming, Linear algebra, Graphs/Networks. Content course: Theory of integer linear programming. Grades and Homework: Four homework assignments You may work in teams of 2 or 3 students. No collaboration with other teams! Hand in by e-mail as pdf. See webpage for homework sets and deadlines. Dion Gijswijt Intro IntPM 2017-2018 2 / 24
Integer programming Dion Gijswijt Intro IntPM 2017-2018 3 / 24
Integer Program (in canonical form) Given: matrix A Q m n, vector b Q m, vector c Q n. maximize c T x subject to Ax b x 0 x integral We can always scale A and b to be integral. Dion Gijswijt Intro IntPM 2017-2018 4 / 24
Integer Program (in canonical form) Given: matrix A Q m n, vector b Q m, vector c Q n. maximize c T x subject to Ax b x 0 x integral We can always scale A and b to be integral. Special case: feasibility problem Dion Gijswijt Intro IntPM 2017-2018 4 / 24
Integer Program (in canonical form) Given: matrix A Q m n, vector b Q m, vector c Q n. maximize c T x subject to Ax b x 0 x integral We can always scale A and b to be integral. Special case: feasibility problem If the program is feasible and bounded, the maximum is attained. This uses rationality of the data. For example, in { } sup x 1 2 x2 x 1 2 x2 1, x 1, x 2 Z 0 the supremum is not attained. Dion Gijswijt Intro IntPM 2017-2018 4 / 24
Integer Program (in canonical form) Given: matrix A Q m n, vector b Q m, vector c Q n. Mixed Integer Program maximize c T x subject to Ax b x 0 x integral maximize c T x + h T y subject to Ax + Gy b x 0 x integral y 0 We will mostly focus on (pure) Integer Programs. Dion Gijswijt Intro IntPM 2017-2018 5 / 24
Expressive power of IP Dion Gijswijt Intro IntPM 2017-2018 6 / 24
Example: Knapsack problem Problem There are n types of items with weights a i and value c i. We have a knapsack that can carry items of total weight b. Goal: maximize the total value of items in the knapsack. Dion Gijswijt Intro IntPM 2017-2018 7 / 24
Example: Knapsack problem Problem There are n types of items with weights a i and value c i. We have a knapsack that can carry items of total weight b. Goal: maximize the total value of items in the knapsack. IP Formulation maximize c 1 x 1 + + c n x n subject to a 1 x 1 + + a n x n b, x 1,..., x n 0 integer Dion Gijswijt Intro IntPM 2017-2018 7 / 24
Example: Graph Matching Problem Given a graph G = (V, E) with costs c on the edges. Find a minimum cost perfect matching. b 1 1 3 a c 1 2 3 0 h i 5 6 5 3 g j 2 2 f 6 d 9 11 10 10 e Dion Gijswijt Intro IntPM 2017-2018 8 / 24
Example: Graph Matching Problem Given a graph G = (V, E) with costs c on the edges. Find a minimum cost perfect matching. b 1 1 3 a c 1 2 3 0 h i 5 6 5 3 g j 2 2 f 6 d 9 11 10 10 e Dion Gijswijt Intro IntPM 2017-2018 8 / 24
Example: Graph Matching Problem Given a graph G = (V, E) with costs c on the edges. Find a minimum cost perfect matching. b 1 1 3 a c 1 2 3 0 h i 5 6 5 3 g j 2 2 f 6 d 9 11 10 10 e IP Formulation minimize subject to uv E c uv x uv v x uv = 1 for every u V x 0 integral Dion Gijswijt Intro IntPM 2017-2018 8 / 24
Boolean expressions 3-SAT Given a Boolean formula in conjunctive normal form with at most 3 literals per clause. Can we assign TRUE/FALSE to variables x i to make the formula TRUE? For example: f = (x 1 x 2 x 4 ) (x 5 x 3 x 6 ). Dion Gijswijt Intro IntPM 2017-2018 9 / 24
Boolean expressions 3-SAT Given a Boolean formula in conjunctive normal form with at most 3 literals per clause. Can we assign TRUE/FALSE to variables x i to make the formula TRUE? For example: f = (x 1 x 2 x 4 ) (x 5 x 3 x 6 ). IP Formulation The example is satisfyable iff the following IP is feasible: x 1 + (1 x 2 ) + x 4 1 1 x 5 + (1 x 3 ) + (1 x 6 ) 1 x 1,..., x n {0, 1} Dion Gijswijt Intro IntPM 2017-2018 9 / 24
Boolean expressions 3-SAT Given a Boolean formula in conjunctive normal form with at most 3 literals per clause. Can we assign TRUE/FALSE to variables x i to make the formula TRUE? For example: f = (x 1 x 2 x 4 ) (x 5 x 3 x 6 ). IP Formulation The example is satisfyable iff the following IP is feasible: x 1 + (1 x 2 ) + x 4 1 1 x 5 + (1 x 3 ) + (1 x 6 ) 1 x 1,..., x n {0, 1} Theorem Integer programming is NP-hard Dion Gijswijt Intro IntPM 2017-2018 9 / 24
Linear programs and Linear Diophantine systems Two related polynomial-time solvable problems are: Linear programming Given: matrix A Q m n, vector b Q m, vector c Q n. Find: solution to Ax b, x 0 maximizing c T x. simplex method interior point methods ellipsoid method [Fast in practice] [Fast in theory and practice] [Fast in theory] System of linear Diophantine equations Given: matrix A Z m n, vector b Z m. Find: solution x Z n to Ax = b. generalisation of Euclidean algorithm Hermite normal form Dion Gijswijt Intro IntPM 2017-2018 10 / 24
LP-relaxation Dion Gijswijt Intro IntPM 2017-2018 11 / 24
LP-relaxation IP: max{c T x : x P Z n }, where P = {x R n : Ax b, x 0} is a polyhedron. (natural) LP-relaxation: max{c T x : x P} gives upper bound. Dion Gijswijt Intro IntPM 2017-2018 12 / 24
LP-relaxation IP: max{c T x : x P Z n }, where P = {x R n : Ax b, x 0} is a polyhedron. (natural) LP-relaxation: max{c T x : x P} gives upper bound. integer hull: P I := conv.hull(p Z n ). Observation. max{c T x : x P Z n } = max{c T x : x P I } Dion Gijswijt Intro IntPM 2017-2018 12 / 24
LP-relaxation IP: max{c T x : x P Z n }, where P = {x R n : Ax b, x 0} is a polyhedron. (natural) LP-relaxation: max{c T x : x P} gives upper bound. integer hull: P I := conv.hull(p Z n ). Observation. max{c T x : x P Z n } = max{c T x : x P I } Theorem. If P is rational, then P I is again a polyhedron. Corollary. Every IP can (in theory) be written as an LP! Dion Gijswijt Intro IntPM 2017-2018 12 / 24
The integrality gap Example Fix r Z 0. Consider max{x 2 : 2rx 1 + x 2 0, 2rx 1 + x 2 2r, x 0, x Z 2 } The maximum of this IP is 0. The maximum of its LP relaxation is r. Dion Gijswijt Intro IntPM 2017-2018 13 / 24
The integrality gap Example Fix r Z 0. Consider max{x 2 : 2rx 1 + x 2 0, 2rx 1 + x 2 2r, x 0, x Z 2 } The maximum of this IP is 0. The maximum of its LP relaxation is r. So the integrality gap max{c T x : x P} max{c T x : x P, x Z n } may be arbitrarily high. Dion Gijswijt Intro IntPM 2017-2018 13 / 24
Perfect formulations Let P = {x R n : Ax b, x 0} be a polyhedron. Definition If P = P I, then P is an integral polyhedron system Ax b, x 0 is a perfect formulation for P Z n. If we have a perfect formulation, our IP becomes an LP. Dion Gijswijt Intro IntPM 2017-2018 14 / 24
Perfect formulations Let P = {x R n : Ax b, x 0} be a polyhedron. Definition If P = P I, then P is an integral polyhedron system Ax b, x 0 is a perfect formulation for P Z n. If we have a perfect formulation, our IP becomes an LP. Birkhoff polytope Consider the n n doubly stochastic matrices. P = {x R n n 0 : n x ij = 1 (j = 1,..., n), i=1 n x ij = 1 (i = 1,..., n)} j=1 Applications: min cost assignment of n people to n jobs, bipartite matching,... Dion Gijswijt Intro IntPM 2017-2018 14 / 24
Extended formulations Represent polytope as a projection of a simpler polytope in higher dimension. Dion Gijswijt Intro IntPM 2017-2018 15 / 24
Branch and bound Dion Gijswijt Intro IntPM 2017-2018 16 / 24
An example maximize 5.5x 1 + 2.1x 2 subject to x 1 + x 2 2 8x 1 + 2x 2 17 x 1, x 2 0 integer Dion Gijswijt Intro IntPM 2017-2018 17 / 24
Branch & bound Given IP max{c T x : x P Z n }, where P = {x R n : Ax b, x 0}. Dion Gijswijt Intro IntPM 2017-2018 18 / 24
Branch & bound Given IP max{c T x : x P Z n }, where P = {x R n : Ax b, x 0}. The branch & bound algorithm 0. Initialize L = {P}, z =, x =. 1. Terminate if L = and output x. 2. Select node Q in L and remove it from L. 3. Bound by solving LP-relaxation max{c T x x Q}. Infeasible, then go to Step 1. Feasible, then let x LP optimal solution with value z LP. 4. Prune: If z LP z, then go to Step 1. If x is integral, set x = x LP and z = z LP and go to Step 1. 5. Branch. Construct smaller polyhedra Q 1,..., Q k whose integral points cover those of Q. Add Q 1,..., Q k to L. Eg. Q 1 := Q {x R n : x i x LP i } and Q 2 = Q {x R n : x i xi LP }. Dion Gijswijt Intro IntPM 2017-2018 18 / 24
Cutting planes Dion Gijswijt Intro IntPM 2017-2018 19 / 24
Definition A cutting plane for polyhedron P: inequality a T x β satisfied by all points in P Z n (but not all points in P). Dion Gijswijt Intro IntPM 2017-2018 20 / 24
Definition A cutting plane for polyhedron P: inequality a T x β satisfied by all points in P Z n (but not all points in P). Dion Gijswijt Intro IntPM 2017-2018 20 / 24
Definition A cutting plane for polyhedron P: inequality a T x β satisfied by all points in P Z n (but not all points in P). Perfect matching Let G = (V, E) a graph and P = {x R E 0 : x(δ(u)) = 1 for all u V }. Perfect matching polytope P I strictly contained in P. For U V odd size, cutting plane: x(e[u]) U 1 2. Dion Gijswijt Intro IntPM 2017-2018 20 / 24
Branch & cut Given IP max{c T x : x P Z n }, where P = {x R n : Ax b, x 0}. Dion Gijswijt Intro IntPM 2017-2018 21 / 24
Branch & cut Given IP max{c T x : x P Z n }, where P = {x R n : Ax b, x 0}. The branch & cut algorithm 0. Initialize L = {P}, z =, x =. 1. Terminate if L = and output x. 2. Select node Q in L and remove it from L. 3. Bound by solving LP-relaxation max{c T x x Q}. Infeasible, then go to Step 1. Feasible, then let x LP optimal solution with value z LP. 4. Prune: If z LP z, then go to Step 1. If x is integral, set x = x LP and z = z LP and go to Step 1. 5. Add cuts to Q and go to Step 3, or skip this step. 6. Branch. Construct smaller polyhedra Q 1,..., Q k whose integral points cover those of Q. Add Q 1,..., Q k to L. Dion Gijswijt Intro IntPM 2017-2018 21 / 24
Flatness theorem Dion Gijswijt Intro IntPM 2017-2018 22 / 24
Khinchine s Theorem Definition The lattice width of a set P R n is ( w(p) := max{d T x : x P} min{d T x : x P} ) min d Z n \{0} Theorem (Khinchine, 1948) Let n N. There is a constant γ n such that if P R n is any bounded, closed, convex set and P Z n =, then w(p) γ n. Dion Gijswijt Intro IntPM 2017-2018 23 / 24
Khinchine s Theorem Definition The lattice width of a set P R n is ( w(p) := max{d T x : x P} min{d T x : x P} ) min d Z n \{0} Theorem (Khinchine, 1948) Let n N. There is a constant γ n such that if P R n is any bounded, closed, convex set and P Z n =, then w(p) γ n. There is an algorithmic version of Khinchine s Theorem, which yields a polynomial-time algorithm for IP in finite dimension. Dion Gijswijt Intro IntPM 2017-2018 23 / 24
Course outline 4 blocks 1 Basics on linear optimization, polyhedra, lattices. 2 Perfect formulations, extended formulations. 3 Cutting planes, Chvatal-Gomory closure; travelling salesman problem. 4 Khinchine s theorem, Lenstra s algorithm for IP. grade based on 4 sets of homework problems. homework problems, time schedule on: homepage.tudelft.nl/64a8q/intpm/ Dion Gijswijt Intro IntPM 2017-2018 24 / 24