Integer Programming Chapter 15

Integer Programming Chapter 15 University of Chicago Booth School of Business Kipp Martin November 9, 2016 1 / 101

Outline Key Concepts Problem Formulation Quality Solver Options Epsilon Optimality Preprocessing Cutting Planes Chvátal-Gomory (C-G) Constraint Aggregation Coefficient Reduction Cuts Separation Branch and Cut COIN-OR Cgl 2 / 101

Key Concepts the quality of the formulation is what counts, not the size quality usually measured by the lower bound critical to generate good formulations problem reformulation preprocess the problem generating cutting planes important to control solver options getting the last bit of optimality can be tough 3 / 101

Problem Formulation Quality Consider the following two sets of integer points. Γ = {(x 1, x 2, y) x 1 y, x 2 y, x 1, x 2, y {0, 1}}. Γ = {(x 1, x 2, y) x 1 + x 2 2y, x 1, x 2, y {0, 1}}. What are the points in each set and how do they differ? 4 / 101

Problem Formulation Quality Consider the following two sets of integer points. Γ = {(x 1, x 2, y) x 1 y, x 2 y, x 1, x 2, y {0, 1}}. y x 1 x 2 0 0 0 1 0 0 1 1 0 1 0 1 1 1 1 Γ has exactly the same set of points! 5 / 101

Problem Formulation Quality Consider the following linear relaxations. Γ = {(x 1, x 2, y) x 1 y, x 2 y, x 1, x 2, y 0, x 1, x 2, y 1}. Γ = {(x 1, x 2, y) x 1 + x 2 2y, x 1, x 2, y 0, x 1, x 2, y 1}. What are the points in each set and how do they differ? What about extreme points? 6 / 101

Problem Formulation Quality Γ is the convex hull of integer points in Γ. Not so for Γ y x 1 x 2 0 0 0 1 0 0 1 1 0 1 0 1 1 1 1 1/2 0 1 1/2 1 0 Important: Γ is a proper subset of Γ. It is a better relaxation. Why? 7 / 101

Problem Formulation Quality 8 / 101

Problem Formulation Quality THE BIG TAKE AWAY: Different polyhedra may contain exactly same set of integer points! THE BIG TAKE AWAY: Different polyhedra may contain exactly same set of integer points! THE BIG TAKE AWAY: Different polyhedra may contain exactly same set of integer points! WHAT IS THE BIG TAKE AWAY? 9 / 101

Problem Formulation Quality Mixed Integer Finite Basis Theorem: If Γ = {(x, y) Ax + By = b, x 0, y Z l +}, (1) where A, B are rational matrices and b is a rational vector, then conv(γ) is a rational polyhedron. Problem: in most real applications conv(γ) P where P = {(x, y) Ax + By = b, x 0, y 0} Objective: Make P smaller and more closely approximate conv(γ). 10 / 101

Problem Formulation Quality Mixed Integer Finite Basis Theorem: If Γ = {(x, y) Ax + By = b, x 0, y Z l +}, (2) where A, B are rational matrices and b is a rational vector, then conv(γ) is a rational polyhedron. Problem: in most real applications conv(γ) P where P = {(x, y) Ax + By = b, x 0, y 0} Objective: Make P smaller and more closely approximate conv(γ). 11 / 101

Problem Formulation Quality Absolutely Critical: Understand the differences between 1. Γ 2. P or Γ 3. conv(γ) 12 / 101

Reformulation (Core Carrier) Core Carrier Revisited: Take the constraints x ij My j, j J i M j and replace with the equivalent set: x ij y j, i M j, j J THE BIG TAKE AWAY: The same set of integer points can have many different representations in terms of linear inequalities. 13 / 101

Reformulation (Core Carrier) Core Carrier Revisited (Continued) : Tight Loose Rows 35152 753 Columns 16300 16300 LP Value 876586 791159 What happens when you try to solve? 14 / 101

Reformulation (Lot Sizing) Dynamic Lot Sizing : Variables: x it units of product i produced in period t I it inventory level of product i at the end of the period t y it is 1 if there is nonzero production of product i during period t, 0 otherwise Parameters: d it demand product i in period t f it fixed cost associated with nonzero production of product i in period t c it marginal production cost for product i in period t h it marginal holding cost charged to product i at the end of period t g t production capacity in period t 15 / 101

Reformulation (Lot Sizing) Objective: Minimize sum of marginal production cost, holding cost, fixed cost N T (c it x it + h it I it + f it y it ) i=1 t=1 Constraint 1: Do not exceed total capacity in each period N x it g t, i=1 t = 1,..., T 16 / 101

Reformulation (Lot Sizing) Constraint 2: Inventory balance equations I i,t 1 + x it I it = d it, i = 1,..., N, t = 1,..., T Constraint 3: Fixed cost forcing constraints x it M it y it 0, i = 1,... N, t = 1,..., T 17 / 101

Reformulation (Lot Sizing) Dynamic Lot Sizing : A standard formulation is: min s.t. N T (c it x it + h it I it + f it y it ) i=1 t=1 N x it g t, t = 1,..., T i=1 I i,t 1 + x it I it = d it, i = 1,..., N, t = 1,..., T x it M it y it 0, i = 1,... N, t = 1,..., T x it, I it 0, i = 1,..., N, t = 1,..., T y it {0, 1}, i = 1,..., N, t = 1,..., T. 18 / 101

Reformulation (Lot Sizing) Dynamic Lot Sizing : An alternate formulation: z itk is 1, if for product i in period t, the decision is to produce enough items to satisfy demand for periods t through k, 0 otherwise. z ij,t 1 + t 1 j=1 T z i1k = 1 k=1 T z itk = 0, i = 1,..., N, t = 2,..., T k=t T z itk y it, i = 1,..., N, t = 1,..., T k=t 19 / 101

Reformulation (Lot Sizing) Dynamic Lot Size (tvw200) : Tight Loose Rows 3208 3208 Columns 7987 4600 LP Value 187503 116880 What happens when you try to solve? The optimal integer value is 187543 (I think the lower bound is 187504). 20 / 101

Solver Options Most solvers take options. This is particularly important in integer programming. In GAMS you can communicate options to solvers through an option text file. First we tell GAMS which solver we want: OPTION MIP = CoinCbc; Next we tell GAMS that we want the first option file (you can have more than one, 2, 3,...) lot_size.optfile = 1; 21 / 101

Solver Options Here is some code we where we put in GAMS options file opt CoinCbc option file /coincbc.opt/; put opt; put optcr 0 / put reslim 100000 / put nodelim 5000000 / put cuts off / put knapsackcuts on /; putclose opt; Make sure to put this code before the solve statement. 22 / 101

Solver Options Here is what the options file does: set the tolerance on integer optimality to zero (optcr 0) set a time limit of 100000 seconds set a node limit of 5000000 turned cutting plane generation off (cuts off) turned knapsack cuts on (knapsackcuts on) 23 / 101

Epsilon Optimality Key Take Away: Actually proving optimality in branch and bound can be tough. The closer you get, the harder it becomes to resolve that list bit of integrality gap. If, for example, you set optcr =.01, then branch and bound will terminate (assume minimization here) when.99*ub LB 24 / 101

Epsilon Optimality Experiment: run the tight version of the lot sizing with ratio 0 and ratio.0005. Here is what happens for me: UB LB Nodes Seconds optcr =.000 187543 187508 112000 4304 optcr =.005 187550 187503 42 21.84 Is it worth it? 25 / 101

Epsilon Optimality Another Take Away: the data may not be that accurate to being with. Standard Oil story. 26 / 101

Preprocessing By preprocessing we mean what is done to a formulation to make it more amenable to solution before solving the linear programming relaxation. Objective: make the linear programming relaxation of the mixed-integer program easy and tight. Try the following eliminate redundant constraints fix variables scale coefficients coefficient reduction rounding improve bounds on variables and constraints probing We work with the following canonical form: a j x j + a j x j a j x j a j x j b (3) j I + j C + j I j C 27 / 101

Preprocessing Rounding: If C + = C =, a j is integer for all j I + I and α = gcd(a j a j I + I ) then the conical form of the constraint is which is equivalent to then a valid rounding is a j x j a j x j b (4) j I + j I (a j /α)x j (a j /α)x j b/α. (5) j I + j I (a j /α)x j (a j /α)x j b/α. (6) j I + j I 28 / 101

Preprocessing Rounding Example: Consider the inequality (assume x 1, x 2 are general integer variables. 2x 1 + 2x 2 3 A feasible solution is x 1 = 1.5 and x 2 = 0. Now let s round, So an equivalent inequality is: α = gcd(a 1, a 2 ) = gcd(2, 2) = 2 (1/2)(2x 1 + 2x 2 ) (1/2)3 x 1 + x 2 1.5 Rounding up the right-hand-side gives: x 1 + x 2 2 Is x 1 = 1.5 and x 2 = 0 feasible? 29 / 101

Preprocessing Rounding Example: Let s look at some geometry. Plot the feasible regions: Γ 1 = {(x 1, x 2 ) 2x 1 + 2x 2 3, x 1, x 2 0} Γ 2 = {(x 1, x 2 ) x 1 + x 2 2, x 1, x 2 0} What is the relationship between? Γ 1 Z 2 and Γ 2 Z 2 Γ 1 and Γ 2 30 / 101

Preprocessing Coefficient Reduction: if C = I = then it is valid to reduce the coefficients on the integer variables to b. That is, (3) is equivalent to j I + min{a j, b}x j + j C + a j x j b. (7) Additionally, if C or I is not empty then upper bounds on the variables in these sets are used as follows. Define: Then (3) is equivalent to λ := b + a j h j + a j h j. (8) j C j I min{a j, λ}x j + a j x j a j x j a j x j b. (9) j I + j C + j I j C 31 / 101

Preprocessing Tightening Bounds: The canonical form a j x j + a j x j a j x j a j x j b j I + j C + j I j C implies for each k C + a k x k b j C + j k a j x j a j x j + a j x j + a j x j. j I + j C j I The smallest the right hand side of can be is b j C + j k a j h j a j h j + a j l j + a j l j. j I + j C j I 32 / 101

Preprocessing Therefore, it is valid to reset l k to (b j C + j k a j h j a j h j + a j l j + a j l j )/a k j I + j C j I Using similar logic the upper bounds of variables indexed by C I are adjusted by ( a j l j + a j l j a j h j a j h j b)/a k j C + j I + j I j C j k 33 / 101

Preprocessing This is an iterative process. The upper and lower bounds on variables are adjusted until there is no improvement in an upper or lower bound. Once the lower and upper bounds are calculated one can apply coefficient reduction on the integer variable coefficients. 34 / 101

Preprocessing Example: Dynamic Lot Sizing min s.t. N T (c it x it + h it I it + f it y it ) i=1 t=1 N x it g t, t = 1,..., T i=1 I i,t 1 + x it I it = d it, i = 1,..., N, t = 1,..., T x it M it y it 0, i = 1,... N, t = 1,..., T x it, I it 0, i = 1,..., N, t = 1,..., T y it {0, 1}, i = 1,..., N, t = 1,..., T. 35 / 101

Preprocessing Example: Dynamic Lot Sizing (Continued) Consider the Big M constraints in canonical form x it My it 0 x it + My it 0 Recall λ := b + a j h j + a j h j. j C j I In this case, what is b, C, I, and λ? What do we get after coefficient reduction? 36 / 101

Preprocessing Example: Dynamic Lot Sizing (Continued) Let s tighten the bounds on the x it variables. Assume 5 time periods (we drop the product subscript all products are treated identically). In canonical form: I 4 x 5 d 5 I 3 x 4 + I 4 d 4 I 2 x 3 + I 3 d 3 I 1 x 2 + I 2 d 2 x 1 + I 1 d 1 What is a valid upper bound on x 5? What about I 4? Work through to period 1. 37 / 101

Preprocessing Feasibility: The canonical form is a j x j + a j x j a j x j a j x j b j I + j C + j I j C if a j h j + a j h j a j l j a j l j < b. j I + j C + j I j C the model instance is not feasible. 38 / 101

Preprocessing Redundancy: The canonical form is if a j x j + a j x j a j x j a j x j b j I + j C + j I j C a j l j + a j l j a j h j a j h j b. j I + j C + j I j C the constraint is redundant and can be deleted. Solve (possibly relaxations) min a j x j + a j x j a j x j a j x j j I + j C + j I j C s.t. Ax b x 0 39 / 101

Preprocessing Probing and Variable Fixing: In a mixed 0/1 linear program probing refers to fixing a binary variable x k to 0 or 1 and then observing any resulting implications. Assume variable x k is binary. If x k = 1, k I and j I + a j h j + j C + a j h j j I \{k} a j l j j C a j l j < b + a k then the model is infeasible which implies it is valid to fix variable x k to 0. If x k = 0, k I + and j I + \{k} a j h j + a j h j a j l j a j l j < b j C + j I j C then the model is infeasible which implies it is valid to fix variable x k to 1. 40 / 101

Preprocessing How to preprocess in Coin-OR Cbc. See (www.coin-or.org) for free optimization solvers. Declare a new solver interface this is what will get preprocessed. OsiSolverInterface *m_osisolverpre = NULL; CglPreProcess process; m_osisolverpre = process.preprocess(*solver->osisolver, false, 10); Build the Cbc model with the preprocessed solver interface and solve CbcModel *model = new CbcModel( *m_osisolverpre); model->branchandbound(); 41 / 101

Preprocessing Unwind to get the original model back process.postprocess( *model->solver() ); Results with p0033.osil Variables Constraints LP Relax Nodes Without 33 16 2520.6 614 With 24 13 2905.7 184 42 / 101

Cutting Planes Motivation Preprocessing is nice, but is usually far from sufficient! The typical integer program requires more than just preprocessing in order to solve. The solution of most real problems requires the use of cutting planes. 43 / 101

Cutting Planes Motivation We can tighten an integer program by adding cutting planes or just cuts. These are sometimes also called valid inequalities. Consider the integer program: min x 1 x 2 x 1 x 2 2 4x 1 + 9x 2 18 2x 1 + 4x 2 4 x 1, x 2 0 x 1, x 2 Z 44 / 101

Cutting Planes Motivation In the figure below the black dots represent Γ (feasible points to the integer program) and the red area represents the linear programming relaxation Γ. 45 / 101

Cutting Planes We are going to generate cuts like the blue line. 46 / 101

Cutting Planes Motivation We have already seen an example of cuts. Consider Γ = {(x 1, x 2, y) x 1 + x 2 2y, x 1, x 2, y {0, 1}}. Γ = {(x 1, x 2, y) x 1 + x 2 2y, 0 x 1, x 2, y 1}. We would say that x 1 y and x 2 y are valid cuts of Γ. These cuts do not cut off any points in Γ but they do cut off points in Γ. 47 / 101

Cutting Planes Motivation The feasible points in are Γ = {(x 1, x 2, y) x 1 + x 2 2y, x 1, x 2, y {0, 1}}. y x 1 x 2 0 0 0 1 0 0 1 1 0 1 0 1 1 1 1 All, yes, every one of these points satisfy the cuts x 1 y and x 2 y. However points (1, 0,.5) and (0, 1,.5) which are in Γ do not satisfy the cuts. 48 / 101

Cutting Planes Theory Big Picture: There is a finite basis theorem for integer programs just like there is for linear programs. That is, Γ = {x R n Bx d, x 0, x j Z, j I } is finitely generated. This means q r conv(γ) = {x R n x = z i x i + z i x i, i=1 i=q+1 q z i = 1, z i 0, i = 1,..., r} (10) i=1 What is the geometry of conv(γ)? What kind of set is it? 49 / 101

Cutting Planes Theory Obviously, conv(γ) is a polyhedron! This implies there exists a system of inequalities (α i ) x α i0, i = 1,..., q such that conv(γ) = {x α i x α i0, i = 1,..., q}. 50 / 101

Cutting Planes Theory We want to add cuts and try and pare down the light grey area to the dark grey area. 3 2.5 2 1.5 1 0.5 x 2-1 1 2 3 4 x 1 51 / 101

Cutting Planes Big Picture: Since conv(γ) is a polyhedron, we could solve the integer program by solving a linear program! min c x s.t. α i x α i0, i = 1,..., q So why don t we just solve the linear program? Why bother with branch and bound? 52 / 101

Cutting Planes Γ = {x R n Bx d, x 0, x j Z, j I } It is critical to understand the difference between CONV (Γ) and Γ (the linear relaxation). Γ = {(x 1, x 2, y) x 1 + x 2 2y, x 1, x 2, y {0, 1}}. What is Γ? CONV (Γ)? Γ? what is the relationship? 53 / 101

CONV (Γ) Versus Γ 54 / 101

Cutting Planes In practice we solve the problem min{c T x x Γ} and get a point x. If life were fair, we would get x CONV (Γ) and be done. Unfortunately, fair is the place where they display animals in summer and has absolutely nothing to do with the way life actually works!!! So we try to find a cut α x α 0 so that α x α 0, x CONV (Γ) and if x Γ but x / CONV (Γ) then α x < α 0 55 / 101

Cutting Planes We now address the problem of finding cutting planes. So far, all we have really done this quarter is add (aggregate) constraints together! So why does anyone think this is a tough course! We keep doing the same thing, only now we do some rounding. That s all just add a few constraints and round a few coefficients. Pretty easy stuff I would say! I am embarrassed! Please do not go around telling the rest of University how trivial the material is that I teach. 56 / 101

Cutting Planes n a ij x j = b i, i = 1,..., m. j=1 Assign a set of multipliers u i, i = 1,..., m to the constraints and generate the aggregate constraint n m ( u i a ij )x j = j=1 i=1 m u i b i. i=1 Assume all of the x j 0 and integer, then n m ( u i a ij ) x j j=1 i=1 m u i b i. i=1 57 / 101

Cutting Planes Since we assume all variables must be integer, the left-hand-side of the aggregate constraint below will always be integer. n m ( u i a ij ) x j j=1 i=1 m u i b i. i=1 Therefore we can round the right-hand-side up and get the cut n m m ( u i a ij ) x j u i b i. j=1 i=1 This cut is a Chvátal-Gomory (C-G) cut and the procedure used to derive the cut is called Chvátal-Gomory rounding. It is very famous in integer programming. i=1 58 / 101

Cutting Planes A slight variation: If we have instead of =, n a ij x j b i, i = 1,..., m. j=1 Assign a set of multipliers u i 0, i = 1,..., m to the constraints and generate the aggregate constraint n m ( u i a ij )x j j=1 i=1 m u i b i. i=1 and we can follow the same derivation. Why is u i 0 required? 59 / 101

Cutting Planes Example One: Consider This is equivalent to 2x 1 + 3x 2 + 4x 3 6 x 3 1 2x 1 + 3x 2 + 4x 3 6 x 3 1 Assign multipliers u 1 = 1 3 and u 2 = 4 3. The aggregate constraint is 2 3 x 1 + x 2 2 3 60 / 101

Cutting Planes Example One (Continued): We have 2 3 x 1 + x 2 2 3 Round the coefficient on x 1, x 1 + x 2 2 3 Now round the left-hand-side, and get the valid cut x 1 + x 2 1 61 / 101

Cutting Planes Example Two: Let s go back to our friend x 1 + x 2 2y x 1 0 x 2 0 y 1 Substitute x 1 = 1 z 1 and x 2 = 1 z 2. This becomes (1 z 1 ) + (1 z 2 ) 2y z 1 1 z 2 1 y 1 62 / 101

Cutting Planes Example Two (continued): This becomes z 1 + z 2 + 2y 2 z 1 1 z 2 1 y 1 Assign multipliers u 1 =.5, u 2 =.5, u 3 = 0, u 4 = 0. This gives the aggregate constraint,.5z 2 + y.5 63 / 101

Cutting Planes Example Two (continued): Round the coefficient of z 2 to 1 Then round the right hand side z 2 + y.5 z 2 + y 1 Substitute back z 2 = 1 x 2, and voila! x 2 y What multipliers give the cut x 1 y? 64 / 101

Cutting Planes Knapsack Cover Cuts: Example 1 and Example 2 are illustrations of the famous knapsack cover cuts. a 1 x 1 + a 2 x 2 + + a n x n b x i {0, 1}, i = 1,..., n Assume all a i 0, for i = 1,..., n. The index set C is a cover if and only if a j b k C a k, b k C a k > 0 for all j C The implied knapsack cover cut is x j 1 j C 65 / 101

Cutting Planes Knapsack Cover Cuts: The knapsack cover cuts have proved exceedingly effective in integer programming. Example 1 Revisited: Define C = {1, 2}. Then 2x 1 + 3x 2 + 4x 3 6 x 1, x 2, x 3 {0, 1} b k C a k = 6 a 3 = 6 4 = 2 Since a 1 = 2 2 and a 2 = 3 2 the corresponding knapsack cover cut is x 1 + x 2 1 66 / 101

Cutting Planes Example 2 Revisited: After the substitution we had z 1 + z 2 + 2y 2 There are two covers (assume 3 in the index of y), C 1 = {1, 3} and C = {2, 3} that lead to the cuts which correspond to z 1 + y 1 and z 2 + y 1 x 1 y and x 2 y 67 / 101

Cutting Planes We now derive a cut using the updated Simplex tableau. The equations corresponding to a basic feasible solution indexed by B are: x Bi + j N a ij x j = b i, i = 1,..., m (11) where B i, i = 1,..., m, indexes the basic variables and N indexes the non basic variables. Applying C-G rounding gives x Bi + j N a ij x j b i, i = 1,..., m. (12) Multiplying (11) by -1 and adding to (12) gives f ij x j f i0 i = 1,..., m (13) j N where f ij = a ij a ij and f i0 = b i b i. 68 / 101

Cutting Planes This tableau based cut is called a Gomory cut. Named after Ralph Gomory a famous computing guru at IBM. Note: You can work with the cut in either the form x Bi + j N a ij x j b i, i = 1,..., m. or f ij x j f i0 i = 1,..., m j N From the viewpoint of re-optimizing the tableau, the second form is preferred. From viewpoint of explanation and derivation the first form is preferred. 69 / 101

Cutting Planes Gomory Cut Example: Start with the integer program min x 1 x 2 x 1 x 2 2 4x 1 + 9x 2 18 2x 1 + 4x 2 4 x 1, x 2 0 x 1, x 2 Z 70 / 101

Cutting Planes Put the linear programming relaxation in standard form. Gomory Cut Example: Start with the integer program min x 1 x 2 x 1 x 2 + x 3 = 2 4x 1 + 9x 2 + x 4 = 18 2x 1 + 4x 2 + x 5 = 4 x 1, x 2 0 Optimize using our good friend basicsimplex.m. See the MATLAB file gomory.m 71 / 101

Cutting Planes If you run gomory.m you get the following tableau. ---------------------------------------------------------------- -0.00-0.00-0.38-0.15-0.00-3.54 1.00 0.00 0.69 0.08 0.00 2.77 0.00 1.00-0.31 0.08 0.00 0.77 0.00 0.00 2.62-0.15 1.00 6.46 Generate a cut based on the fractional variable x 2 = 0.77 Round up x 2.31x 3 + 0.08x 4 = 0.77 x 2 + x 4 1 72 / 101

Cutting Planes We have: x 2 + x 4 1 Since 4x 1 + 9x 2 + x 4 = 18 we have x 4 = 18 4x 1 9x 2 and the cut becomes x 2 + 18 4x 1 9x 2 1 4x 1 + 8x 2 17 73 / 101

Cutting Planes We can even tighten the Gomory cut! implies, 4x 1 + 8x 2 17 which implies x 1 + 2x 2 17/4 = 4 1 4 x 1 + 2x 2 4 I love this stuff! Verify that this is valid. 74 / 101

Cutting Planes Here is the cut x 1 + 2x 2 4 plotted with the feasible region. 75 / 101

Cutting Planes By generating the necessary cutting planes we could solve the integer program as a linear program! The blue region represents the convex hull of integer solutions. 76 / 101

Cutting Planes Questions to Ponder: If there is a fractional solution, will there be a knapsack cover cut, that cuts this solution off? If there is a fractional solution, will there be a Gomory cut, that cuts this solution off? By the way, would you prefer to use Gomory or knapsack cover cuts? Why? 77 / 101

Cutting Planes Test problem P0458 with 177 rows and 548 variables. LP Value IP Value Nodes Iterations No Cuts 591 8691 34826 68343 Gomory + Knapsack 8249 8691 154 7242 78 / 101

Cutting Planes We have developed knapsack cover cuts and Gomory cuts. The generation of valid cuts has been the subject of a huge amount of research. People have developed cuts for very special and for very general structures. See (www.coin-or.org) for free optimization solvers. See the GAMS document coin.pdf (in folder docs/solvers), pages 22-23 for a list of cuts supported by the COIN-OR solver Cbc. Most solvers have options for turning these various cuts on and off. 79 / 101

Cutting Planes For COIN-OR Cbc cuts off turns all cut generation off cuts on turns on all the default cut generators (see coin.pdf for default cut generators) you can set cuts off and then turn on specific cuts cuts off gomorycuts on knapsackcuts on flowcovercuts on 80 / 101

Constraint Aggregation Coefficient Reduction Cuts The C-G cut assumes all integer. However, if continuous variables are present, then it is not valid to conclude that n m u i a ij x j j=1 i=1 m u i b i i=1 n m m u i a ij x j u i b i. j=1 i=1 i=1 In order to treat mixed-integer programs (i.e. some continuous variables present) we use a technique developed by Martin and Schrage. It is very simple and just adds constraints and rounds variable coefficients. Despite the simplicity, it did help me earn tenure! 81 / 101

Constraint Aggregation Coefficient Reduction Cuts Consider the pure integer case first. n a ij x j = b i, i = 1,..., m j=1 Assign a set of multipliers u i, i = 1,..., m to these constraints and generate the aggregate constraint n m ( u i a ij )x j = j=1 i=1 m u i b i. If all of the variables x j are required to be nonnegative, then a valid cut based on the aggregate constraint is i=1 n m max {0, ( u i a ij )}x j j=1 i=1 m u i b i. i=1 82 / 101

Constraint Aggregation Coefficient Reduction Cuts We have: n m max {0, ( u i a ij )}x j j=1 i=1 m u i b i. i=1 If every x j is a nonnegative integer variable and if m i=1 u ib i > 0, then a valid constraint aggregation, coefficient reduction (CACR) cut is n m m min ( u i b i, max {0, ( u i a ij )})x j j=1 i=1 i=1 m u i b i. i=1 83 / 101

Constraint Aggregation Coefficient Reduction Cuts The (CACR) cut has as a nice special case, pure 0-1 programming. Any inequality in a pure 0/1 linear program can be written as a j x j a j x j b (14) j I + j I where a j > 0 for j I + I and x j {0, 1} for all j I + I. Since x j is a binary variable the simple upper bound constraints x j 1 for all j W I + are valid side constraints. Assign a multiplier of a j to all of the side constraints x j 1 for j W and a multiplier of +1 to the constraint (14). 84 / 101

Constraint Aggregation Coefficient Reduction Cuts The aggregate constraint is a j x j a j x j λ := b a j. j I j I + \W j W If λ = b j W a j > 0, then a valid (CACR) cut is j I + \W min{λ, a j }x j λ. (15) 85 / 101

Constraint Aggregation Coefficient Reduction Cuts Example: Generalized Assignment Problem MIN 2 X11 + 11 X12 + 7 X21 + 7 X22 + 20 X31 + 2 X32 + 5 X41 + 5 X42 SUBJECT TO 2) X11 + X12 = 1 3) X21 + X22 = 1 4) X31 + X32 = 1 5) X41 + X42 = 1 6) 3 X11 + 6 X21 + 5 X31 + 7 X41 <= 13 7) 2 X12 + 4 X22 + 10 X32 + 4 X42 <= 10 END 86 / 101

Constraint Aggregation Coefficient Reduction Cuts Example: Generalize Assignment Problem (continued) OBJECTIVE FUNCTION VALUE 1) 20.32000 VARIABLE VALUE REDUCED COST X11 1.000000.000000 X21 1.000000.000000 X31.240000.000000 X32.760000.000000 X41.400000.000000 X42.600000.000000 Generate a cut. 87 / 101

Constraint Aggregation Coefficient Reduction Cuts Example: Generalize Assignment Problem (continued) There are two knapsack constraints. Convert the knapsack constraints to canonical form. Convert to canonical form by complementing the variables z ij = 1 x ij and then multiplying by -1. In canonical the knapsack constraints become. 3z 11 + 6z 21 + 5z 31 + 7z 41 8 2z 12 + 4z 22 + 10z 32 + 4z 42 10 A minimal cover cut is z 11 + z 21 + z 41 1 which is x 11 + x 21 + x 41 2 in the original variables. The current linear programming relaxation violates this cut. Similarly, a minimal cover cut for the second machine is x 32 + x 42 1. 88 / 101

Constraint Aggregation Coefficient Reduction Cuts Example: Generalize Assignment Problem (continued) Add these cuts to the linear program and resolve. OBJECTIVE FUNCTION VALUE 1) 23.20000 VARIABLE VALUE REDUCED COST X11 1.000000.000000 X22 1.000000.000000 X31.400000.000000 X32.600000.000000 X41 1.000000.000000 Are there any CACR cuts? See pages 546-549 of the textbook. 89 / 101

Constraint Aggregation Coefficient Reduction Cuts Example: Lock box model (simple plant location). Consider a constraint of the form x 1 + x 2 2y Write this as 2y x 1 x 2 0 Substitute, x 1 = 1 z 1, x 2 = 1 z 2 and get the system 2y + z 1 + z 2 2 z 1 1 z 2 1 90 / 101

Constraint Aggregation Coefficient Reduction Cuts Example: Lock box model (simple plant location). Aggregating the first and second constraint gives Coefficient reduction gives 2y + z 2 1 y + z 2 1 which gives x 1 y Similarly we get x 2 y. In general, CACR applied to n x i My i=1 gives, x i y, i = 1,..., n 91 / 101

Constraint Aggregation Coefficient Reduction Cuts CACR is trivially extended to the mixed-integer case. If the integer variables are indexed by I and the continuous variables by C, then after aggregating constraints, it follows that a valid CACR cut is m m min ( u i b i, max {0, ( u i a ij )})x j + j I i=1 i=1 m max {0, ( u i a ij )}x j j C i=1 m u i b i. (16) i=1 92 / 101

Constraint Aggregation Coefficient Reduction Cuts Example: Dynamic lot sizing. Consider the constraint set I t 1 + x t I t = d t x t + My t 0 Assign a multiplier of 1 to the first constraint and 1 to the second constraint. The aggregate constraint is I t 1 + My t I t d t Applying coefficient reduction gives I t 1 + d t y t d t This is equivalent to x t I t + d t y t. 93 / 101

Separation Obviously, it is neither efficient nor possible to add all of the CACR cuts. We add them on-the-fly as needed. We discover which CACR cuts to add on-the-fly using separation. Separation Problem: given x R n, is x conv(γ), and if not, find a cut α x α 0 such that α x < α 0 and α x α 0 for all x conv(γ). This is a huge idea in integer programming. It effectively allows real problems to be solved! 94 / 101

Separation Consider a special case finding a CACR cut for the knapsack problem. That is, Assumptions: j I + a j x j b x j 1, j = 1,..., n We apply a multiplier of +1 to the knapsack constraint We apply a multiplier of a j to constraint x j 1 Given x 0, finding the most violated cut implies finding a W I + so that j I + \W min{λ, a j}x j λ is minimized and λ = b j W a j is strictly positive. 95 / 101

Separation This is done by solving a binary knapsack problem. Let r j = 1 if j I + \W and 0 otherwise. Instead of requiring λ = b j W a j, (15) is valid as long as λ b j W a j. Then the separation knapsack problem is min j I + min{λ, a j }x j r j λ (KSEP) s.t. b j I + a j + j I + a j r j λ r j {0, 1}, j I +. Note: b j I + a j + j I + a j r j = b j W a j 96 / 101

Separation Example: (Generalized Assignment Part Deux) VARIABLE VALUE REDUCED COST X11 1.000000.000000 X21 1.000000.000000 X31.240000.000000 X32.760000.000000 X41.400000.000000 X42.600000.000000 The constraint 3 X11 + 6 X21 + 5 X31 + 7 X41 <= 13 becomes 3 Z11 + 6 Z21 + 5 Z31 + 7 Z41 >= 8 97 / 101

Separation Example: (Generalized Assignment Part Deux) The separation knapsack with λ = 1 is min 0r 11 + 0r 21 +.76r 31 +.6r 41 1 (KSEP) s.t. 3r 11 + 6r 21 + 5r 31 + 7r 41 14 r 11, r 21, r 31, r 41 {0, 1} The optimal solution is r 11 = 1, r 21 = 1, r 31 = 0 and r 41 = 1. The implied minimal cover cut is z 11 + z 21 + z 41 1 which is x 11 + x 21 + x 41 2 in the original variables. 98 / 101

Branch and Cut 99 / 101

COIN-OR Cgl COIN-OR Cgl (Cut generation library) Add cuts of various types to Cbc Basic idea each type of cut is a class that we define in our code. CglKnapsackCover cover; CglSimpleRounding round; CglGomory gomory; add each cut to the model and solve. model->addcutgenerator(&cover, 1, "Cover"); model->addcutgenerator(&round, 1, "Round"); model->addcutgenerator(&gomory, 1, "Gomory"); 100 / 101

COIN-OR COIN-OR: No size limitations. No license issue. Two ways to build a model. 1. Use Modeling Language to call a solver. 2. Write a matrix generator and interact directly with solver. 101 / 101