Integer Programming ISE 418. Lecture 13. Dr. Ted Ralphs

Integer Programming ISE 418 Lecture 13 Dr. Ted Ralphs

ISE 418 Lecture 13 1 Reading for This Lecture Nemhauser and Wolsey Sections II.1.1-II.1.3, II.1.6 Wolsey Chapter 8 CCZ Chapters 5 and 6 Valid Inequalities for Mixed Integer Linear Programs, G. Cornuejols. Generating Disjunctive Cuts for Mixed Integer Programs, M. Perregaard.

ISE 418 Lecture 13 2 Valid Inequalities from Disjunctions Valid inequalities for conv(s) can also be generated from valid disjunctions. Let X i = {x R n + A i x b i } for i = 1,..., k constitute a disjunction valid for S. Then inequalities valid for k i=1 (P X i are also valid for conv(s).

ISE 418 Lecture 13 3 The Union of Polyhedra The convex hull of the union of polyhedra is not necessarily a polyhedron. Under mild conditions, we can characterize it, however. Consider a finite collection of polyhedra P i = {x R n A i x b i for 1 i k. Let Y be the polyhedron described by the following constraints: A i x i b i y i i = 1,..., k k x i = x i=1 k y i = 1 i=1 y 0 Furthermore, for polyhedron P i, let C i = {x R n A i x 0} and let P i = Q i + C i where Q i is a polytope.

ISE 418 Lecture 13 4 The Convex Hull of the Union of Polyhedra Under the assumptions on the previous slide, we have the following result. Proposition 1. If either k i=1 P i = or C j cone( i:pi C i for all j such that P j =, then the following sets are identical: conv( k i=1 P i} conv( k i=1 Q i) + cone( k i=1 C i) proj x Y. Note that the assumptions of the proposition are necessary, but are automatically satisfied if C i = {0} whenever P i =, or all the polyhedra have the same recession cone.

ISE 418 Lecture 13 5 The Convex Hull of the Union of Polyhedra (cont.) Note also that if all the polyhedra have the same recession cones, then conv( k i=1 P i) = conv( k i=1 P i) and k i=1 P i is the projection of A i x i b i y i i = 1,..., k k x i = x i=1 k y i = 1 i=1 y {0, 1} This is the case when the polyhedra only differ in their right-hand sides, as is the case when branching on variables.

ISE 418 Lecture 13 6 Valid Inequalities from Disjunctions Another viewpoint for constructing valid inequalities based on disjunctions comes from the following result: Proposition 2. S 2 R n +, then for x S 1 S 2. If (π 1, π0) 1 is valid for S 1 R n + and (π 2, π0) 2 is valid for n min(πj 1, πj 2 )x j max(π0, 1 π0) 2 (1) j=1 In fact, all valid inequalities for the union of two polyhedra can be obtained in this way. Proposition 3. If P i = {x R n + A i x b i } for i = 1, 2 are nonempty polyhedra, then (π, π 0 ) is a valid inequality for conv(p 1 P 2 ) if and only if there exist u 1, u 2 R m such π u i A i and π 0 u i b i for i = 1, 2.

ISE 418 Lecture 13 7 Simple Disjunctive Inequalities We want to develop a procedure analogous to C-G for mixed-integer sets. It is straightforward to develop an analog of the rounding principle we used earlier that was geared towards pure integer programs. Proposition 4. Let T = {x Z R + x 1 x 2 b}. Then the inequality x 1 1 x 2 b. 1 f 0 is valid for T. The proof requires exploiting the disjunction x 1 b OR x 1 b + 1

ISE 418 Lecture 13 8 Mixed Integer Rounding Inequalities We can generalize the inequality from the previous slide as follows. Proposition 5. Let T = {x Z p + R n p + n j=1 a jx j b}, where a Q n and b Q. Then the inequality p j=1 ( a j + (f j f 0 ) + 1 f 0 )x j + 1 1 f 0 p+1 j n:a j <0 is valid for T, where f j = a j a j and f 0 = b b. a j x j b. In fact, if a j Z, gcd{a 1,..., a n } = 1, and b Z, then the above inequality is facet-inducing for conv(t ). The above inequality is called a mixed integer rounding (MIR) inequality. Its validity can be proved by aggregating the integer and continuous variables, respectively, and applying Proposition??.

ISE 418 Lecture 13 9 Gomory Mixed Integer Inequalities Let s consider again the set of solutions T to an IP with one equation. This time, we write T equivalently as T = x Zn + f j x j + (f j 1)x j = f 0 + k for some integer k j:f j f 0 j:f j >f 0 Since k 1 or k 0, we have the disjunction f j x j f 0 j:f j f 0 (1 f j ) x j 1 f 0 j:f j >f 0 OR j:f j f 0 f j (1 f 0 ) x j + j:f j >f 0 (1 f j ) (1 f 0 ) x j 1

ISE 418 Lecture 13 10 The Gomory Mixed Integer Cut Applying Proposition??, we get f j x j + f 0 j:f j f 0 (1 f j ) (1 f 0 ) x j 1 j:f j >f 0 This is called a Gomory mixed integer (GMI) inequality. GMI inequalities dominate the associated Gomory cut and can also be obtained easily from the tableau. In the case of the mixed integer set T = x Zp + R n p + the GMI cut is 0 j p f j f 0 f j x j + f 0 0 j p f j >f 0 (1 f j ) (1 f 0 ) x j + p+1 j n a j >0 n a j x j = a 0, j=1 a j f 0 x j p+1 j n a j <0 a j (1 f 0 ) x j 1

ISE 418 Lecture 13 11 GMI vs. MIR Although we derived the GMI inequality using a different logic than that which we used for the MIR inequality, they are equivalent. Beginning with the inequality n a j x j b, (2) j=1 we add a slack variables s = b a x to obtain n a j x j + s = b. j=1 Deriving the GMI inequality from this equation and then substituting out the slack variable s, we obtain the MIR inequality associated with (??).

ISE 418 Lecture 13 12 Gomory Mixed Integer Cuts from the Tableau Let s consider how to generate Gomory mixed integer cuts from the tableau when solving (??). As before, we first introduce a slack variable for each inequality in the formulation. Solving the LP relaxation, we look for a row in the tableau in which an integer variable is basic and has a fractional variable. We apply the GMI procedure to produce a cut. Finally, we substitute out the slack variables in order to express the cut in terms of the original variables only.

ISE 418 Lecture 13 13 Example: GMI Cuts versus Gomory Cuts Recall our example from last time. max 2x 1 + 5x 2 (3) s.t. 4x 1 + x 2 28 (4) The optimal tableau for the LP relaxation is: x 1 + 4x 2 27 (5) x 1 x 2 1 (6) x 1, x 2 0 (7) Basic var. x 1 x 2 s 1 s 2 s 3 RHS x 2 0 1-2/30 8/30 0 16/3 s 3 0 0-1/3 1/3 1 2/3 x 1 1 0 8/30-2/30 0 17/3 Table 1: Optimal tableau of the LP relaxation The associated optimal solution to the LP relaxation is also shown in Figure??.

ISE 418 Lecture 13 14 Example: Gomory Cuts (cont.) Figure 1: Convex hull of S

ISE 418 Lecture 13 15 Example: GMI Cuts versus Gomory Cuts (cont.) The GMI cut from the first row is In terms of x 1 and x 2, we have 1 10 s 1 + 8 10 s 2 1, 12x 1 + 33x 2 234, (GMI-C1)

ISE 418 Lecture 13 16 Example: GMI Cuts versus Gomory Cuts (cont.) The GMI cut from the third row is In terms of x 1 and x 2, we have 4 10 s 1 + 2 10 s 2 1, 3x 1 + 2x 2 26, (GMI-C3)

ISE 418 Lecture 13 17 Geometric Interpretation of GMI Cuts To understand the geometric interpretation of GMI cuts, we consider a relaxation of (??) associated with a basis of the LP relaxation. We simply relax the non-negativity constraints on the basic variables to obtain T = { (x, s) Z n+m Ax + Is = b, x N 0, s N 0 }, where x N and s N are the non-basic variables associated with basis B. This is equivalent to relaxing the non-binding constraints. The convex hull of T is the so-called corner polyhedron associated with the basis B.

ISE 418 Lecture 13 18 Example: Corner Polyhedron Figure 2: The corner polyhedron associated with the optimal basis of the LP relaxation of the earlier example.

ISE 418 Lecture 13 19 Intersection Cuts A simple way to obtain inequalities valid for the corner polyhedron (and hence the original MILP is as follows. Construct a convex set C whose interior contains the solution x associated with the current basis and no other integer points. Determine the points of intersection of each of the extreme rays of the corner polyhedron with the set C. The unique hyperplane determined by these points of intersection then separates x from the corner polyhedron. This is a general paradigm and one can get different classes of valid inequality by choosing the set C in different ways. The GMI cut from row i of the tableau is precisely the intersection cut obtained by setting C = { x R n+m x j x j x j }, where x j is the variable that is basic in row i.

ISE 418 Lecture 13 20 Example: GMI Cut as an Intersection Cut Figure?? shows the GMI cut derived from the second row of the tableau in our example as an intersection cut. The basic variable in this case is s 3. In terms of the original variables, we have C = { x R n+m 0 x 1 x 2 1 }

ISE 418 Lecture 13 21 Example: GMI Cut as an Intersection Cut Figure 3: GMI Cut from row 2 as an intersection cut

ISE 418 Lecture 13 22 GMI Cuts in Practice Here is an example of the slow convergence sometimes seen in practice. min 20x 1 + 15x 2 2x 1 3x 2 5 4x 1 2x 2 15 3x 1 4x 2 20 0 x 1 9 0 x 2 6 x 1, x 2 Z We will solve this using the naive implementation in CuPPy.

ISE 418 Lecture 13 23 The Polyhedra in Example Figure 4: Feasible region of Example MILP

ISE 418 Lecture 13 24 First Iteration The solution to the LP relaxation is (2, 3.5). The tableau row in which x 2 is basic is x 2 + 0.3s 2 0.4s 3 Note that for purposes of illustration, we are explicitly included the bound constraints in the tableau. The GMI is 0.6s 2 + 0.8s 3 1 In terms of the original variables, this is 4.8x 1 4.4x 2 26

ISE 418 Lecture 13 25 Second Iteration Figure 5: Feasible region of Example MILP after adding cut The solution in the second iteration is (1.75, 4) and the cut is 10.4x 1 5.8667x 2 42.6667.

ISE 418 Lecture 13 26 Third Iteration Figure 6: Feasible region of Example MILP after two cuts The solution in the third iteration is (2, 3.7273) and the cut is 14.3x 1 11.7333x 2 73.3333.

ISE 418 Lecture 13 27 Further Iterations Figure 7: Feasible region of Example MILP after 100 cuts

ISE 418 Lecture 13 28 Further Iterations Note the slow convergence rate. Not much progress is being made with each cut. After 100 iteration, the solution is (1.9979, 4), which may be close enough, but would not be considered optimal by most solvers. It is surprising that such a small MILP would have such a high rank. This is at least partly due to numerical errors and the fact that our implementation is naive. We will delve further into these topics later in the course.

ISE 418 Lecture 13 29 Lift and Project In lift and project, we directly write down the separation problem with respect to a given point and disjunction. We will use the variable disjunction on the j th variable. We have that conv(s) conv(p 0 j P1 j ) where P0 j = P {x Rn x j x j } and P1 j = P {x Rn x j x j }. Applying Proposition??, we see that the inequality (π, π 0 ) is valid for P j = conv(p 0 j P1 j ) if there exists ui R m +, v i R n +, and w i R + for i = 0, 1 such that π u 0 A + w 0 e j, π u 1 A w 1 e j, π 0 u 0 b + w 0 x j, π 0 u 1 b w 1 x j, Notice that this is a set of linear constraints, i.e., we could write an LP to generate constraints based on this disjunction.

ISE 418 Lecture 13 30 The Cut Generating LP This leads to the cut generating LP (CGLP), which generates the most violated inequality valid for P j. max πˆx π 0 s.t. π ua + u 0 e j, π va v 0 e j, π 0 ub + u 0 x j, (CGLP) m u i + u 0 + i=1 π 0 vb v 0 x j, m v i + v 0 = 1 i=1 u, u 0, v, v 0 0 The last constraint is for normalization. There are a number of alternatives for normalization and the choice does have an impact (see Perregaard). This shows that the separation problem for P j is polynomially solvable.

ISE 418 Lecture 13 31 Split Inequalities Let (α, β) be a split disjunction and define P 1 = P {x R n α x β} P 2 = P {x R n α x β + 1} Any inequality valid for conv(p 1 P 2 ) is valid for S and is called a split inequality.

ISE 418 Lecture 13 32 Separation Problem for Split Inequalities The LP (??) can be generalized straightforwardly to produce the most violated split cut. max πˆx π 0 s.t. π ua + u 0 α, π va v 0 α, π 0 ub + u 0 β, m u i + u 0 + i=1 π 0 vb v 0 (β + 1), m v i + v 0 = 1 i=1 u, u 0, v, v 0 0 α Z n β Z (SCGLP) The separation problem is a mixed integer nonlinear optimization problem, however, and is not easy to solve.

ISE 418 Lecture 13 33 Strengthening Lift-and-Project Cuts Note that (??) only explicitly accounts for the integrality of a single variable. We can strengthen the generated cuts using the integrality of the other variables (we consider the pure binary case, but this can be generalized). To do this, we simply replace the original coefficients π k = min{ua k, va k } for k j π j = min{ua j + u 0, va j v 0 } for the integer variables indexed 1 k p with π k = max{ua k + u 0 m k, va k v 0 m k }, where m i = va i ua i u 0 + v 0 The proof is to fix the values of u, v, u 0, v 0 obtained by solving (??) and then find an optimal (α, β) in (??).

ISE 418 Lecture 13 34 GMI Cuts vs. Lift-and-Project Cuts There is a correspondence between GMI cuts generated from basic solutions of the LP relaxation and strengthened lift-and-project cuts. We use the normalization π 0 { 1, 0, 1} in (??). Then each of the former can be derived as the latter from some basic solution to (??) (and vice versa, though the relationship is not one-to-one). We may be able to get stronger GMI cuts from tableaus other than the one that is optimal to the current LP relaxation. There are lift-and-project cuts that can only be obtained as GMI cuts from an infeasible tableau. We may also be able to get stronger cuts from a basic solution that is suboptimal for the LP relaxation. By pivoting in the LP relaxation, we can implicitly solve the cut generating LP (see Balas and Perregaard).

ISE 418 Lecture 13 35 Lift-and-Project Cut as GMI from Infeasible Basis Figure 8: A GMI cut arising from an infeasible basis

ISE 418 Lecture 13 36 Lift-and-Project Cut as GMI from Alternative Basis In our earlier example. the inequality x 1 5 dominates 3x 1 + 2x 2 26, but the latter was generated from the current basis. With respect to the basic solution (5.8, 4.8), we obtain the cut x 1 5 as a GMI cut. Figure 9: GMI cut arising from an alternative basis

ISE 418 Lecture 13 37 Another Derivation for Binary Optimization Consider the following procedure: 1: Select j {1,..., n}. 2: Generate the nonlinear system x j (Ax b) 0, (1 x j )(Ax b) 0. 3: Linearize the system by substituting y i for x i x j, i j, and x j for x 2 j. Call this polyhedron M j. 4: Project M j onto the x-space. In this case, the resulting polyhedron is again P j. This procedure can be strengthened in a number of different ways.

ISE 418 Lecture 13 38 The Lift-and-Project Closure The lift-and-project closure is P 1 = n j=1p j We have just shown that optimization over this closure can be accomplished in polynomial time. Let P k be the lift-and-project closure of P k 1 for k > 1. The lift-and-project rank of P is the smallest number k such that P k = conv(s). Surprisingly, the lift-and-project rank is bounded by n in the binary and mixed binary case.

ISE 418 Lecture 13 39 Example: Lift and Project Closure We consider the polyhedron P in two dimensions defined by the constraints 8x 1 + 30x 2 115 2x 1 4x 2 5 14x 1 + 8x 2 1 2x 1 36x 2 5 30x 1 8x 2 191 10x 1 + 10x 2 127

ISE 418 Lecture 13 40 Lift-and-Project Closure for Example Figure 10: Lift-and-project closure for example

ISE 418 Lecture 13 41 The GMI closure A GMI cut with respect to a polyhedron P is any cut that can be derived using the GMI procedure starting from any inequality valid for P. The GMI closure is obtained by adding all GMI cuts to the description of P. The GMI closure is a polyhedron, but in contrast to the lift-and-project closure, optimizing over it is difficult (N P-hard). This seems like a paradox, since we have shown that most-violated GMI cuts are easy to generate. This is only the case, however, for basic solutions to the LP relaxation separating arbitrary points is difficult in general. The GMI rank of both valid inequalities and polyhedra can be defined in a fashion similar to that of the C-G rank (more on this later).

ISE 418 Lecture 13 42 The GMI Closure and the Split Closure The split closure is the set of points satisfying all possible split cuts and is a polyhedron. Every split cut is also a GMI cut and vice versa. The split closure and the GMI closure are therefore identical. As expected, the GMI cut corresponding to a given split cut is not necessarily one that can be derived from a basic solution to the LP relaxation. We can define the split rank of an inequality and of a polyhedron as before. In the pure integer case, the split rank (and GMI rank) of P is finite, but it may not be in the mixed case. In the mixed binary case, the split rank is bounded by n.

ISE 418 Lecture 13 43 Aside: Selection Criteria The criteria by which we select cuts has a big impact on the overall effectiveness. We will see later that we in fact need two different kinds of selection criteria: one for generating cuts and one for choosing which cuts to add. We typically use bound improvement as a rough criteria when selecting disjunctions for branching, but we often use degree of violation with cuts. Why the difference? One simple answer is that degree of violation is a linear objective with respect to the cut generating LP. Generating cuts according to other criteria seems to be more difficult. See http://coral.ie.lehigh.edu/~ted/files/talks/disjunctioninforms12.pdf http://coral.ie.lehigh.edu/~jeff/mip-2006/posters/fukasawa.pdf