Operations Research Lecture 6: Integer Programming Notes taken by Kaiquan Xu@Business School, Nanjing University May 12th 2016 1 Integer programming (IP) formulations The integer programming (IP) is the same as the linear programming (LP) problem except that some of the variables are restricted to take integer values. IP is a powerful modeling framework that provides great flexibility for expressing discrete optimization problem. But the price for this flexibility is that IP is more difficult to solve than LP. 1.1 Modeling techniques Binary choice An important use of a binary variable is to encode a choice between two alternatives. Example 1. (The zero-one knapsack problem) We are given n items, the jth item has weight w j and its value is c j. Given a bound K on the weight that can be carried in a knapsack, we would like to select items to maximize the total value. In order to model this problem, x j is defined a binary variable, which is 1 if item j is chosen, and 0 otherwise. The problem can be formulated as follows: Forcing constrains max c j x j w j x j K x j {0, 1}, j = 1,..., n. For some discrete problems, certain decisions are dependent. For example, decision A can be make only if decision B has also been made. We can model this by using the constrain x y i.e., if y = 0 (decision B is not made), then x = 0 (decision A cannot be made). Example 2. (Facility location problems) Suppose we are given n potential facility locations and a list of m clients who need to be serviced from these locations. There is a fixed cost c j of opening a facility at location j, while there is a cost d ij of serving client i from facility j. The goal is to select a set of facility locations and assign each client to 1
one facility, while minimizing the total cost. In order to model this problem, we define a binary decision variable y j for each location j, which is equal to 1 if facility j is selected, and 0 otherwise. We define a binary variable x ij, which is equal to 1 if client i is served by facility j, and 0 otherwise. The facility location problem is then formulated as follows: max c j y j + m i=1 x ij = 1, i, d ij x ij x ij y j x ij, y j {0, 1}, i, j. here, the forcing constrain x ij y j captures the fact if there is no facility at location j (y j = 0), then client i cannot be served there, and we must have x ij = 0. Relations between variables A constrain of the form x j 1 where all variables are binary, implies that at most one of the variables x j can be one. If the constrain is of the form x j = 1, then exactly one of the variable x j should be one. Disjunctive constrains Suppose we are given two constrains a T x b and c T x d, in which all of the components of a and c are nonnegative. We want to model a requirement that at least one of the two constrains is satisfied. For achieving this, a binary variable y is defined, and the following constrains are imposed: 2 Integer programming (IP) methods a T x yb c T x (1 y)d, y {0, 1} Unlike LP problems, IP problems are very difficult to solve. No efficient general algorithm is known for their solution. 2.1 Cutting plane methods Given the IP problem min c T x Ax = b x 0 x integer (1) and its LP relaxation 2
min c T x Ax = b x 0 (2) The main idea in cutting plane methods is: first solve the LP relaxation 2 and find an optimal solution x. if x is integer, then it is an optimal solution to 1. If not, find an inequality that all integer solutions to 1 satisfy, but x does not. We add this inequality to the LP problem to obtain a tighter relaxation, and we iterate this step. The performance of a cutting plane method depends critically on the choice of the inequality used to cut x. The Gomory cutting plane algorithm When solving the LP problem 2 with the simplex method, from the optimal tableau, we obtain the coefficients of the constraints x B + B 1 A N X N = B 1 b here, x B are the basic variables, and x N are nonbasic variables. Let ā ij = (B 1 A j ) i and ā i0 = (B 1 b) i, We only consider one equality from the optimal tableau, in which ā i0 is fractional: x i + j N ā ij x j = ā i0 Since x j 0 for all j, we have x i + ā ij x j x i + ā ij x j = ā i0 j N j N Since x j should be integer, we obtain x i + j N ā ij x j ā i0 This inequality is valid for all integer solutions, but not satisfied by x. The reason is x i = ā i0, x j j N, and ā i0 < ā i0 (since ā i0 was assumed fraction). = 0 for all nonbasic Example 3. (Illustration of the Gomory cutting plane algorithm) we consider the IP problem. We transform the problem in standard form min x 1 2x 2 4x 1 + 6x 2 9 x 1 + x 2 4 x 1, x 2 0 x 1, x 2 integer min x 1 2x 2 4x 1 + 6x 2 + x 3 = 9 x 1 + x 2 + x 4 = 4 x 1, x 2, x 3, x 4 = 0 x 1, x 2, x 3, x 4 integer We solve the LP relaxation, and the optimal solution (in terms of the original variable) is x 1 = (15/10, 25/10). One of the equations in the optimal tableau is x 2 + 1 10 x 3 + 4 10 x 4 = 25 10. By applying the Gomory cutting plane algorithm, 3
Figure 1: Gomory Example we find the cut x 2 2 We augment the LP relaxation by adding the constrain x 2 + x 5 = 2, x 5 0, and we find the new optimal solution is x 2 = (3/4, 2). One of the equations is the optimal tableau is x 1 1 4 x 3 + 6 4 x 5 = 3 4 We add a new Gomory cut x 1 x 3 + x 5 0 which, in term of the original variables x 1, x 2, is 3x 1 + 5x 2 7 We add this constrain, and find the new optimal solution x 3 = (1, 2), which is the optimal integer solution. 2.2 Branch and bound Branch and bound uses a divide and conquer approach to explore the set of feasible integer solutions. We partition the original problem into some subproblems, and try to compare the optimal solutions of subproblems. If subproblems are also very difficult to solve, we split them into further subproblems. For making the efficiency of this method, we also need to compute the lower bound of the corresponding subproblems, which usually are easy, for example, solving the corresponding LP relaxation. We also need to maintain an upper bound U, which could be the cost of the best feasible solution encountered so far. If the lower bound of a particular subproblem is larger than or equal to the U, then this subproblem need not to be considered further. A generic branch and bound algorithm 1. Select an active subproblem F i 2. If the subproblem is infeasible, delete it; otherwise, compute its lower bound b(f i ) for the corresponding subproblem. 4
3. If b(f i ) U, delete the subproblem. 4. If b(f i ) < U, either obtain an optimal solution to the subproblem, or break the corresponding subproblem into further subproblems, which are added to the list of active subproblems. If the optimal solution x to the LP relaxation is not integer, we choose a component x i for which x i and create two subproblems, by adding either of the constrains is not integer x i x i or x i x i Example 4. (Illustration of branch and bound) We solve the problem of Example 3 by branch and bound; Figure 2: Branch Bound Example Initially, U =, the optimal solution of the LP relaxation is x 1 = (1.5, 2.5), the b(f ) = 3.5. We create two subproblems by adding the constrains x 2 3 (subproblem F 1 ), or x 2 2 (subproblem F 2 ). The LP relxation of F 1 is infeasible, and we delete it. The optimal solution to the LP relaxation of F 2 is x 2 = (3/4, 2), and thus b(f 2 ) = 3.25. We further decompose subproblem F 2 into two subproblems by adding x 1 1 (subproblem F 3 ) and x 1 0(subproblem F 4 ). The optimal solution to F 3 is x 3 = (1, 2), which is integer and therefore U is updated as -3. We delete subproblem F 3. The optimal solution to the LP relaxation of F 4 is x 4 = (0, 3/2), and thus b(f 4 ) = 3. Since b(f 4 ) U, we do not need to further explore subproblem F 4. Finally, we get the optimal solutionx 3 = (1, 2). 3 References 1. Chapter 3, Operations Research (Third Version), Tsinghua Unversity Press, 2005.6 2. Chapter 10, 11, Dimitris Bertsimas, John N. Tsitsiklis, Introduction to Linear Programming, Athena Scientific, 1997.02 5