Integer Programming Chapter 15

Transcription

1 Integer Programming Chapter 15 University of Chicago Booth School of Business Kipp Martin November 9, / 101

2 Outline Key Concepts Problem Formulation Quality Solver Options Epsilon Optimality Preprocessing Cutting Planes Chvátal-Gomory (C-G) Constraint Aggregation Coefficient Reduction Cuts Separation Branch and Cut COIN-OR Cgl 2 / 101

3 Key Concepts the quality of the formulation is what counts, not the size quality usually measured by the lower bound critical to generate good formulations problem reformulation preprocess the problem generating cutting planes important to control solver options getting the last bit of optimality can be tough 3 / 101

4 Problem Formulation Quality Consider the following two sets of integer points. Γ = {(x 1, x 2, y) x 1 y, x 2 y, x 1, x 2, y {0, 1}}. Γ = {(x 1, x 2, y) x 1 + x 2 2y, x 1, x 2, y {0, 1}}. What are the points in each set and how do they differ? 4 / 101

5 Problem Formulation Quality Consider the following two sets of integer points. Γ = {(x 1, x 2, y) x 1 y, x 2 y, x 1, x 2, y {0, 1}}. y x 1 x Γ has exactly the same set of points! 5 / 101

6 Problem Formulation Quality Consider the following linear relaxations. Γ = {(x 1, x 2, y) x 1 y, x 2 y, x 1, x 2, y 0, x 1, x 2, y 1}. Γ = {(x 1, x 2, y) x 1 + x 2 2y, x 1, x 2, y 0, x 1, x 2, y 1}. What are the points in each set and how do they differ? What about extreme points? 6 / 101

7 Problem Formulation Quality Γ is the convex hull of integer points in Γ. Not so for Γ y x 1 x / /2 1 0 Important: Γ is a proper subset of Γ. It is a better relaxation. Why? 7 / 101

8 Problem Formulation Quality 8 / 101

9 Problem Formulation Quality THE BIG TAKE AWAY: Different polyhedra may contain exactly same set of integer points! THE BIG TAKE AWAY: Different polyhedra may contain exactly same set of integer points! THE BIG TAKE AWAY: Different polyhedra may contain exactly same set of integer points! WHAT IS THE BIG TAKE AWAY? 9 / 101

10 Problem Formulation Quality Mixed Integer Finite Basis Theorem: If Γ = {(x, y) Ax + By = b, x 0, y Z l +}, (1) where A, B are rational matrices and b is a rational vector, then conv(γ) is a rational polyhedron. Problem: in most real applications conv(γ) P where P = {(x, y) Ax + By = b, x 0, y 0} Objective: Make P smaller and more closely approximate conv(γ). 10 / 101

11 Problem Formulation Quality Mixed Integer Finite Basis Theorem: If Γ = {(x, y) Ax + By = b, x 0, y Z l +}, (2) where A, B are rational matrices and b is a rational vector, then conv(γ) is a rational polyhedron. Problem: in most real applications conv(γ) P where P = {(x, y) Ax + By = b, x 0, y 0} Objective: Make P smaller and more closely approximate conv(γ). 11 / 101

12 Problem Formulation Quality Absolutely Critical: Understand the differences between 1. Γ 2. P or Γ 3. conv(γ) 12 / 101

13 Reformulation (Core Carrier) Core Carrier Revisited: Take the constraints x ij My j, j J i M j and replace with the equivalent set: x ij y j, i M j, j J THE BIG TAKE AWAY: The same set of integer points can have many different representations in terms of linear inequalities. 13 / 101

14 Reformulation (Core Carrier) Core Carrier Revisited (Continued) : Tight Loose Rows Columns LP Value What happens when you try to solve? 14 / 101

15 Reformulation (Lot Sizing) Dynamic Lot Sizing : Variables: x it units of product i produced in period t I it inventory level of product i at the end of the period t y it is 1 if there is nonzero production of product i during period t, 0 otherwise Parameters: d it demand product i in period t f it fixed cost associated with nonzero production of product i in period t c it marginal production cost for product i in period t h it marginal holding cost charged to product i at the end of period t g t production capacity in period t 15 / 101

16 Reformulation (Lot Sizing) Objective: Minimize sum of marginal production cost, holding cost, fixed cost N T (c it x it + h it I it + f it y it ) i=1 t=1 Constraint 1: Do not exceed total capacity in each period N x it g t, i=1 t = 1,..., T 16 / 101

17 Reformulation (Lot Sizing) Constraint 2: Inventory balance equations I i,t 1 + x it I it = d it, i = 1,..., N, t = 1,..., T Constraint 3: Fixed cost forcing constraints x it M it y it 0, i = 1,... N, t = 1,..., T 17 / 101

18 Reformulation (Lot Sizing) Dynamic Lot Sizing : A standard formulation is: min s.t. N T (c it x it + h it I it + f it y it ) i=1 t=1 N x it g t, t = 1,..., T i=1 I i,t 1 + x it I it = d it, i = 1,..., N, t = 1,..., T x it M it y it 0, i = 1,... N, t = 1,..., T x it, I it 0, i = 1,..., N, t = 1,..., T y it {0, 1}, i = 1,..., N, t = 1,..., T. 18 / 101

19 Reformulation (Lot Sizing) Dynamic Lot Sizing : An alternate formulation: z itk is 1, if for product i in period t, the decision is to produce enough items to satisfy demand for periods t through k, 0 otherwise. z ij,t 1 + t 1 j=1 T z i1k = 1 k=1 T z itk = 0, i = 1,..., N, t = 2,..., T k=t T z itk y it, i = 1,..., N, t = 1,..., T k=t 19 / 101

20 Reformulation (Lot Sizing) Dynamic Lot Size (tvw200) : Tight Loose Rows Columns LP Value What happens when you try to solve? The optimal integer value is (I think the lower bound is ). 20 / 101

21 Solver Options Most solvers take options. This is particularly important in integer programming. In GAMS you can communicate options to solvers through an option text file. First we tell GAMS which solver we want: OPTION MIP = CoinCbc; Next we tell GAMS that we want the first option file (you can have more than one, 2, 3,...) lot_size.optfile = 1; 21 / 101

22 Solver Options Here is some code we where we put in GAMS options file opt CoinCbc option file /coincbc.opt/; put opt; put optcr 0 / put reslim / put nodelim / put cuts off / put knapsackcuts on /; putclose opt; Make sure to put this code before the solve statement. 22 / 101

23 Solver Options Here is what the options file does: set the tolerance on integer optimality to zero (optcr 0) set a time limit of seconds set a node limit of turned cutting plane generation off (cuts off) turned knapsack cuts on (knapsackcuts on) 23 / 101

24 Epsilon Optimality Key Take Away: Actually proving optimality in branch and bound can be tough. The closer you get, the harder it becomes to resolve that list bit of integrality gap. If, for example, you set optcr =.01, then branch and bound will terminate (assume minimization here) when.99*ub LB 24 / 101

25 Epsilon Optimality Experiment: run the tight version of the lot sizing with ratio 0 and ratio Here is what happens for me: UB LB Nodes Seconds optcr = optcr = Is it worth it? 25 / 101

26 Epsilon Optimality Another Take Away: the data may not be that accurate to being with. Standard Oil story. 26 / 101

27 Preprocessing By preprocessing we mean what is done to a formulation to make it more amenable to solution before solving the linear programming relaxation. Objective: make the linear programming relaxation of the mixed-integer program easy and tight. Try the following eliminate redundant constraints fix variables scale coefficients coefficient reduction rounding improve bounds on variables and constraints probing We work with the following canonical form: a j x j + a j x j a j x j a j x j b (3) j I + j C + j I j C 27 / 101

28 Preprocessing Rounding: If C + = C =, a j is integer for all j I + I and α = gcd(a j a j I + I ) then the conical form of the constraint is which is equivalent to then a valid rounding is a j x j a j x j b (4) j I + j I (a j /α)x j (a j /α)x j b/α. (5) j I + j I (a j /α)x j (a j /α)x j b/α. (6) j I + j I 28 / 101

29 Preprocessing Rounding Example: Consider the inequality (assume x 1, x 2 are general integer variables. 2x 1 + 2x 2 3 A feasible solution is x 1 = 1.5 and x 2 = 0. Now let s round, So an equivalent inequality is: α = gcd(a 1, a 2 ) = gcd(2, 2) = 2 (1/2)(2x 1 + 2x 2 ) (1/2)3 x 1 + x Rounding up the right-hand-side gives: x 1 + x 2 2 Is x 1 = 1.5 and x 2 = 0 feasible? 29 / 101

30 Preprocessing Rounding Example: Let s look at some geometry. Plot the feasible regions: Γ 1 = {(x 1, x 2 ) 2x 1 + 2x 2 3, x 1, x 2 0} Γ 2 = {(x 1, x 2 ) x 1 + x 2 2, x 1, x 2 0} What is the relationship between? Γ 1 Z 2 and Γ 2 Z 2 Γ 1 and Γ 2 30 / 101

31 Preprocessing Coefficient Reduction: if C = I = then it is valid to reduce the coefficients on the integer variables to b. That is, (3) is equivalent to j I + min{a j, b}x j + j C + a j x j b. (7) Additionally, if C or I is not empty then upper bounds on the variables in these sets are used as follows. Define: Then (3) is equivalent to λ := b + a j h j + a j h j. (8) j C j I min{a j, λ}x j + a j x j a j x j a j x j b. (9) j I + j C + j I j C 31 / 101

32 Preprocessing Tightening Bounds: The canonical form a j x j + a j x j a j x j a j x j b j I + j C + j I j C implies for each k C + a k x k b j C + j k a j x j a j x j + a j x j + a j x j. j I + j C j I The smallest the right hand side of can be is b j C + j k a j h j a j h j + a j l j + a j l j. j I + j C j I 32 / 101

33 Preprocessing Therefore, it is valid to reset l k to (b j C + j k a j h j a j h j + a j l j + a j l j )/a k j I + j C j I Using similar logic the upper bounds of variables indexed by C I are adjusted by ( a j l j + a j l j a j h j a j h j b)/a k j C + j I + j I j C j k 33 / 101

34 Preprocessing This is an iterative process. The upper and lower bounds on variables are adjusted until there is no improvement in an upper or lower bound. Once the lower and upper bounds are calculated one can apply coefficient reduction on the integer variable coefficients. 34 / 101

35 Preprocessing Example: Dynamic Lot Sizing min s.t. N T (c it x it + h it I it + f it y it ) i=1 t=1 N x it g t, t = 1,..., T i=1 I i,t 1 + x it I it = d it, i = 1,..., N, t = 1,..., T x it M it y it 0, i = 1,... N, t = 1,..., T x it, I it 0, i = 1,..., N, t = 1,..., T y it {0, 1}, i = 1,..., N, t = 1,..., T. 35 / 101

36 Preprocessing Example: Dynamic Lot Sizing (Continued) Consider the Big M constraints in canonical form x it My it 0 x it + My it 0 Recall λ := b + a j h j + a j h j. j C j I In this case, what is b, C, I, and λ? What do we get after coefficient reduction? 36 / 101

37 Preprocessing Example: Dynamic Lot Sizing (Continued) Let s tighten the bounds on the x it variables. Assume 5 time periods (we drop the product subscript all products are treated identically). In canonical form: I 4 x 5 d 5 I 3 x 4 + I 4 d 4 I 2 x 3 + I 3 d 3 I 1 x 2 + I 2 d 2 x 1 + I 1 d 1 What is a valid upper bound on x 5? What about I 4? Work through to period / 101

38 Preprocessing Feasibility: The canonical form is a j x j + a j x j a j x j a j x j b j I + j C + j I j C if a j h j + a j h j a j l j a j l j < b. j I + j C + j I j C the model instance is not feasible. 38 / 101

39 Preprocessing Redundancy: The canonical form is if a j x j + a j x j a j x j a j x j b j I + j C + j I j C a j l j + a j l j a j h j a j h j b. j I + j C + j I j C the constraint is redundant and can be deleted. Solve (possibly relaxations) min a j x j + a j x j a j x j a j x j j I + j C + j I j C s.t. Ax b x 0 39 / 101

40 Preprocessing Probing and Variable Fixing: In a mixed 0/1 linear program probing refers to fixing a binary variable x k to 0 or 1 and then observing any resulting implications. Assume variable x k is binary. If x k = 1, k I and j I + a j h j + j C + a j h j j I \{k} a j l j j C a j l j < b + a k then the model is infeasible which implies it is valid to fix variable x k to 0. If x k = 0, k I + and j I + \{k} a j h j + a j h j a j l j a j l j < b j C + j I j C then the model is infeasible which implies it is valid to fix variable x k to / 101

41 Preprocessing How to preprocess in Coin-OR Cbc. See ( for free optimization solvers. Declare a new solver interface this is what will get preprocessed. OsiSolverInterface *m_osisolverpre = NULL; CglPreProcess process; m_osisolverpre = process.preprocess(*solver->osisolver, false, 10); Build the Cbc model with the preprocessed solver interface and solve CbcModel *model = new CbcModel( *m_osisolverpre); model->branchandbound(); 41 / 101

42 Preprocessing Unwind to get the original model back process.postprocess( *model->solver() ); Results with p0033.osil Variables Constraints LP Relax Nodes Without With / 101

43 Cutting Planes Motivation Preprocessing is nice, but is usually far from sufficient! The typical integer program requires more than just preprocessing in order to solve. The solution of most real problems requires the use of cutting planes. 43 / 101

44 Cutting Planes Motivation We can tighten an integer program by adding cutting planes or just cuts. These are sometimes also called valid inequalities. Consider the integer program: min x 1 x 2 x 1 x 2 2 4x 1 + 9x x 1 + 4x 2 4 x 1, x 2 0 x 1, x 2 Z 44 / 101

45 Cutting Planes Motivation In the figure below the black dots represent Γ (feasible points to the integer program) and the red area represents the linear programming relaxation Γ. 45 / 101

46 Cutting Planes We are going to generate cuts like the blue line. 46 / 101

47 Cutting Planes Motivation We have already seen an example of cuts. Consider Γ = {(x 1, x 2, y) x 1 + x 2 2y, x 1, x 2, y {0, 1}}. Γ = {(x 1, x 2, y) x 1 + x 2 2y, 0 x 1, x 2, y 1}. We would say that x 1 y and x 2 y are valid cuts of Γ. These cuts do not cut off any points in Γ but they do cut off points in Γ. 47 / 101

48 Cutting Planes Motivation The feasible points in are Γ = {(x 1, x 2, y) x 1 + x 2 2y, x 1, x 2, y {0, 1}}. y x 1 x All, yes, every one of these points satisfy the cuts x 1 y and x 2 y. However points (1, 0,.5) and (0, 1,.5) which are in Γ do not satisfy the cuts. 48 / 101

49 Cutting Planes Theory Big Picture: There is a finite basis theorem for integer programs just like there is for linear programs. That is, Γ = {x R n Bx d, x 0, x j Z, j I } is finitely generated. This means q r conv(γ) = {x R n x = z i x i + z i x i, i=1 i=q+1 q z i = 1, z i 0, i = 1,..., r} (10) i=1 What is the geometry of conv(γ)? What kind of set is it? 49 / 101

50 Cutting Planes Theory Obviously, conv(γ) is a polyhedron! This implies there exists a system of inequalities (α i ) x α i0, i = 1,..., q such that conv(γ) = {x α i x α i0, i = 1,..., q}. 50 / 101

51 Cutting Planes Theory We want to add cuts and try and pare down the light grey area to the dark grey area x x 1 51 / 101

52 Cutting Planes Big Picture: Since conv(γ) is a polyhedron, we could solve the integer program by solving a linear program! min c x s.t. α i x α i0, i = 1,..., q So why don t we just solve the linear program? Why bother with branch and bound? 52 / 101

53 Cutting Planes Γ = {x R n Bx d, x 0, x j Z, j I } It is critical to understand the difference between CONV (Γ) and Γ (the linear relaxation). Γ = {(x 1, x 2, y) x 1 + x 2 2y, x 1, x 2, y {0, 1}}. What is Γ? CONV (Γ)? Γ? what is the relationship? 53 / 101

54 CONV (Γ) Versus Γ 54 / 101

55 Cutting Planes In practice we solve the problem min{c T x x Γ} and get a point x. If life were fair, we would get x CONV (Γ) and be done. Unfortunately, fair is the place where they display animals in summer and has absolutely nothing to do with the way life actually works!!! So we try to find a cut α x α 0 so that α x α 0, x CONV (Γ) and if x Γ but x / CONV (Γ) then α x < α 0 55 / 101

56 Cutting Planes We now address the problem of finding cutting planes. So far, all we have really done this quarter is add (aggregate) constraints together! So why does anyone think this is a tough course! We keep doing the same thing, only now we do some rounding. That s all just add a few constraints and round a few coefficients. Pretty easy stuff I would say! I am embarrassed! Please do not go around telling the rest of University how trivial the material is that I teach. 56 / 101

57 Cutting Planes n a ij x j = b i, i = 1,..., m. j=1 Assign a set of multipliers u i, i = 1,..., m to the constraints and generate the aggregate constraint n m ( u i a ij )x j = j=1 i=1 m u i b i. i=1 Assume all of the x j 0 and integer, then n m ( u i a ij ) x j j=1 i=1 m u i b i. i=1 57 / 101

58 Cutting Planes Since we assume all variables must be integer, the left-hand-side of the aggregate constraint below will always be integer. n m ( u i a ij ) x j j=1 i=1 m u i b i. i=1 Therefore we can round the right-hand-side up and get the cut n m m ( u i a ij ) x j u i b i. j=1 i=1 This cut is a Chvátal-Gomory (C-G) cut and the procedure used to derive the cut is called Chvátal-Gomory rounding. It is very famous in integer programming. i=1 58 / 101

59 Cutting Planes A slight variation: If we have instead of =, n a ij x j b i, i = 1,..., m. j=1 Assign a set of multipliers u i 0, i = 1,..., m to the constraints and generate the aggregate constraint n m ( u i a ij )x j j=1 i=1 m u i b i. i=1 and we can follow the same derivation. Why is u i 0 required? 59 / 101

60 Cutting Planes Example One: Consider This is equivalent to 2x 1 + 3x 2 + 4x 3 6 x 3 1 2x 1 + 3x 2 + 4x 3 6 x 3 1 Assign multipliers u 1 = 1 3 and u 2 = 4 3. The aggregate constraint is 2 3 x 1 + x / 101

61 Cutting Planes Example One (Continued): We have 2 3 x 1 + x Round the coefficient on x 1, x 1 + x Now round the left-hand-side, and get the valid cut x 1 + x / 101

62 Cutting Planes Example Two: Let s go back to our friend x 1 + x 2 2y x 1 0 x 2 0 y 1 Substitute x 1 = 1 z 1 and x 2 = 1 z 2. This becomes (1 z 1 ) + (1 z 2 ) 2y z 1 1 z 2 1 y 1 62 / 101

63 Cutting Planes Example Two (continued): This becomes z 1 + z 2 + 2y 2 z 1 1 z 2 1 y 1 Assign multipliers u 1 =.5, u 2 =.5, u 3 = 0, u 4 = 0. This gives the aggregate constraint,.5z 2 + y.5 63 / 101

64 Cutting Planes Example Two (continued): Round the coefficient of z 2 to 1 Then round the right hand side z 2 + y.5 z 2 + y 1 Substitute back z 2 = 1 x 2, and voila! x 2 y What multipliers give the cut x 1 y? 64 / 101

65 Cutting Planes Knapsack Cover Cuts: Example 1 and Example 2 are illustrations of the famous knapsack cover cuts. a 1 x 1 + a 2 x a n x n b x i {0, 1}, i = 1,..., n Assume all a i 0, for i = 1,..., n. The index set C is a cover if and only if a j b k C a k, b k C a k > 0 for all j C The implied knapsack cover cut is x j 1 j C 65 / 101

66 Cutting Planes Knapsack Cover Cuts: The knapsack cover cuts have proved exceedingly effective in integer programming. Example 1 Revisited: Define C = {1, 2}. Then 2x 1 + 3x 2 + 4x 3 6 x 1, x 2, x 3 {0, 1} b k C a k = 6 a 3 = 6 4 = 2 Since a 1 = 2 2 and a 2 = 3 2 the corresponding knapsack cover cut is x 1 + x / 101

67 Cutting Planes Example 2 Revisited: After the substitution we had z 1 + z 2 + 2y 2 There are two covers (assume 3 in the index of y), C 1 = {1, 3} and C = {2, 3} that lead to the cuts which correspond to z 1 + y 1 and z 2 + y 1 x 1 y and x 2 y 67 / 101

68 Cutting Planes We now derive a cut using the updated Simplex tableau. The equations corresponding to a basic feasible solution indexed by B are: x Bi + j N a ij x j = b i, i = 1,..., m (11) where B i, i = 1,..., m, indexes the basic variables and N indexes the non basic variables. Applying C-G rounding gives x Bi + j N a ij x j b i, i = 1,..., m. (12) Multiplying (11) by -1 and adding to (12) gives f ij x j f i0 i = 1,..., m (13) j N where f ij = a ij a ij and f i0 = b i b i. 68 / 101

69 Cutting Planes This tableau based cut is called a Gomory cut. Named after Ralph Gomory a famous computing guru at IBM. Note: You can work with the cut in either the form x Bi + j N a ij x j b i, i = 1,..., m. or f ij x j f i0 i = 1,..., m j N From the viewpoint of re-optimizing the tableau, the second form is preferred. From viewpoint of explanation and derivation the first form is preferred. 69 / 101

70 Cutting Planes Gomory Cut Example: Start with the integer program min x 1 x 2 x 1 x 2 2 4x 1 + 9x x 1 + 4x 2 4 x 1, x 2 0 x 1, x 2 Z 70 / 101

71 Cutting Planes Put the linear programming relaxation in standard form. Gomory Cut Example: Start with the integer program min x 1 x 2 x 1 x 2 + x 3 = 2 4x 1 + 9x 2 + x 4 = 18 2x 1 + 4x 2 + x 5 = 4 x 1, x 2 0 Optimize using our good friend basicsimplex.m. See the MATLAB file gomory.m 71 / 101

72 Cutting Planes If you run gomory.m you get the following tableau Generate a cut based on the fractional variable x 2 = 0.77 Round up x 2.31x x 4 = 0.77 x 2 + x / 101

73 Cutting Planes We have: x 2 + x 4 1 Since 4x 1 + 9x 2 + x 4 = 18 we have x 4 = 18 4x 1 9x 2 and the cut becomes x x 1 9x 2 1 4x 1 + 8x / 101

74 Cutting Planes We can even tighten the Gomory cut! implies, 4x 1 + 8x 2 17 which implies x 1 + 2x 2 17/4 = x 1 + 2x 2 4 I love this stuff! Verify that this is valid. 74 / 101

75 Cutting Planes Here is the cut x 1 + 2x 2 4 plotted with the feasible region. 75 / 101

76 Cutting Planes By generating the necessary cutting planes we could solve the integer program as a linear program! The blue region represents the convex hull of integer solutions. 76 / 101

77 Cutting Planes Questions to Ponder: If there is a fractional solution, will there be a knapsack cover cut, that cuts this solution off? If there is a fractional solution, will there be a Gomory cut, that cuts this solution off? By the way, would you prefer to use Gomory or knapsack cover cuts? Why? 77 / 101

78 Cutting Planes Test problem P0458 with 177 rows and 548 variables. LP Value IP Value Nodes Iterations No Cuts Gomory + Knapsack / 101

79 Cutting Planes We have developed knapsack cover cuts and Gomory cuts. The generation of valid cuts has been the subject of a huge amount of research. People have developed cuts for very special and for very general structures. See ( for free optimization solvers. See the GAMS document coin.pdf (in folder docs/solvers), pages for a list of cuts supported by the COIN-OR solver Cbc. Most solvers have options for turning these various cuts on and off. 79 / 101

80 Cutting Planes For COIN-OR Cbc cuts off turns all cut generation off cuts on turns on all the default cut generators (see coin.pdf for default cut generators) you can set cuts off and then turn on specific cuts cuts off gomorycuts on knapsackcuts on flowcovercuts on 80 / 101

81 Constraint Aggregation Coefficient Reduction Cuts The C-G cut assumes all integer. However, if continuous variables are present, then it is not valid to conclude that n m u i a ij x j j=1 i=1 m u i b i i=1 n m m u i a ij x j u i b i. j=1 i=1 i=1 In order to treat mixed-integer programs (i.e. some continuous variables present) we use a technique developed by Martin and Schrage. It is very simple and just adds constraints and rounds variable coefficients. Despite the simplicity, it did help me earn tenure! 81 / 101

82 Constraint Aggregation Coefficient Reduction Cuts Consider the pure integer case first. n a ij x j = b i, i = 1,..., m j=1 Assign a set of multipliers u i, i = 1,..., m to these constraints and generate the aggregate constraint n m ( u i a ij )x j = j=1 i=1 m u i b i. If all of the variables x j are required to be nonnegative, then a valid cut based on the aggregate constraint is i=1 n m max {0, ( u i a ij )}x j j=1 i=1 m u i b i. i=1 82 / 101

83 Constraint Aggregation Coefficient Reduction Cuts We have: n m max {0, ( u i a ij )}x j j=1 i=1 m u i b i. i=1 If every x j is a nonnegative integer variable and if m i=1 u ib i > 0, then a valid constraint aggregation, coefficient reduction (CACR) cut is n m m min ( u i b i, max {0, ( u i a ij )})x j j=1 i=1 i=1 m u i b i. i=1 83 / 101

84 Constraint Aggregation Coefficient Reduction Cuts The (CACR) cut has as a nice special case, pure 0-1 programming. Any inequality in a pure 0/1 linear program can be written as a j x j a j x j b (14) j I + j I where a j > 0 for j I + I and x j {0, 1} for all j I + I. Since x j is a binary variable the simple upper bound constraints x j 1 for all j W I + are valid side constraints. Assign a multiplier of a j to all of the side constraints x j 1 for j W and a multiplier of +1 to the constraint (14). 84 / 101

85 Constraint Aggregation Coefficient Reduction Cuts The aggregate constraint is a j x j a j x j λ := b a j. j I j I + \W j W If λ = b j W a j > 0, then a valid (CACR) cut is j I + \W min{λ, a j }x j λ. (15) 85 / 101

86 Constraint Aggregation Coefficient Reduction Cuts Example: Generalized Assignment Problem MIN 2 X X X X X X X X42 SUBJECT TO 2) X11 + X12 = 1 3) X21 + X22 = 1 4) X31 + X32 = 1 5) X41 + X42 = 1 6) 3 X X X X41 <= 13 7) 2 X X X X42 <= 10 END 86 / 101

87 Constraint Aggregation Coefficient Reduction Cuts Example: Generalize Assignment Problem (continued) OBJECTIVE FUNCTION VALUE 1) VARIABLE VALUE REDUCED COST X X X X X X Generate a cut. 87 / 101

88 Constraint Aggregation Coefficient Reduction Cuts Example: Generalize Assignment Problem (continued) There are two knapsack constraints. Convert the knapsack constraints to canonical form. Convert to canonical form by complementing the variables z ij = 1 x ij and then multiplying by -1. In canonical the knapsack constraints become. 3z z z z z z z z A minimal cover cut is z 11 + z 21 + z 41 1 which is x 11 + x 21 + x 41 2 in the original variables. The current linear programming relaxation violates this cut. Similarly, a minimal cover cut for the second machine is x 32 + x / 101

89 Constraint Aggregation Coefficient Reduction Cuts Example: Generalize Assignment Problem (continued) Add these cuts to the linear program and resolve. OBJECTIVE FUNCTION VALUE 1) VARIABLE VALUE REDUCED COST X X X X X Are there any CACR cuts? See pages of the textbook. 89 / 101

90 Constraint Aggregation Coefficient Reduction Cuts Example: Lock box model (simple plant location). Consider a constraint of the form x 1 + x 2 2y Write this as 2y x 1 x 2 0 Substitute, x 1 = 1 z 1, x 2 = 1 z 2 and get the system 2y + z 1 + z 2 2 z 1 1 z / 101

91 Constraint Aggregation Coefficient Reduction Cuts Example: Lock box model (simple plant location). Aggregating the first and second constraint gives Coefficient reduction gives 2y + z 2 1 y + z 2 1 which gives x 1 y Similarly we get x 2 y. In general, CACR applied to n x i My i=1 gives, x i y, i = 1,..., n 91 / 101

92 Constraint Aggregation Coefficient Reduction Cuts CACR is trivially extended to the mixed-integer case. If the integer variables are indexed by I and the continuous variables by C, then after aggregating constraints, it follows that a valid CACR cut is m m min ( u i b i, max {0, ( u i a ij )})x j + j I i=1 i=1 m max {0, ( u i a ij )}x j j C i=1 m u i b i. (16) i=1 92 / 101

93 Constraint Aggregation Coefficient Reduction Cuts Example: Dynamic lot sizing. Consider the constraint set I t 1 + x t I t = d t x t + My t 0 Assign a multiplier of 1 to the first constraint and 1 to the second constraint. The aggregate constraint is I t 1 + My t I t d t Applying coefficient reduction gives I t 1 + d t y t d t This is equivalent to x t I t + d t y t. 93 / 101

94 Separation Obviously, it is neither efficient nor possible to add all of the CACR cuts. We add them on-the-fly as needed. We discover which CACR cuts to add on-the-fly using separation. Separation Problem: given x R n, is x conv(γ), and if not, find a cut α x α 0 such that α x < α 0 and α x α 0 for all x conv(γ). This is a huge idea in integer programming. It effectively allows real problems to be solved! 94 / 101

95 Separation Consider a special case finding a CACR cut for the knapsack problem. That is, Assumptions: j I + a j x j b x j 1, j = 1,..., n We apply a multiplier of +1 to the knapsack constraint We apply a multiplier of a j to constraint x j 1 Given x 0, finding the most violated cut implies finding a W I + so that j I + \W min{λ, a j}x j λ is minimized and λ = b j W a j is strictly positive. 95 / 101

96 Separation This is done by solving a binary knapsack problem. Let r j = 1 if j I + \W and 0 otherwise. Instead of requiring λ = b j W a j, (15) is valid as long as λ b j W a j. Then the separation knapsack problem is min j I + min{λ, a j }x j r j λ (KSEP) s.t. b j I + a j + j I + a j r j λ r j {0, 1}, j I +. Note: b j I + a j + j I + a j r j = b j W a j 96 / 101

97 Separation Example: (Generalized Assignment Part Deux) VARIABLE VALUE REDUCED COST X X X X X X The constraint 3 X X X X41 <= 13 becomes 3 Z Z Z Z41 >= 8 97 / 101

98 Separation Example: (Generalized Assignment Part Deux) The separation knapsack with λ = 1 is min 0r r r r 41 1 (KSEP) s.t. 3r r r r r 11, r 21, r 31, r 41 {0, 1} The optimal solution is r 11 = 1, r 21 = 1, r 31 = 0 and r 41 = 1. The implied minimal cover cut is z 11 + z 21 + z 41 1 which is x 11 + x 21 + x 41 2 in the original variables. 98 / 101

99 Branch and Cut 99 / 101

100 COIN-OR Cgl COIN-OR Cgl (Cut generation library) Add cuts of various types to Cbc Basic idea each type of cut is a class that we define in our code. CglKnapsackCover cover; CglSimpleRounding round; CglGomory gomory; add each cut to the model and solve. model->addcutgenerator(&cover, 1, "Cover"); model->addcutgenerator(&round, 1, "Round"); model->addcutgenerator(&gomory, 1, "Gomory"); 100 / 101

101 COIN-OR COIN-OR: No size limitations. No license issue. Two ways to build a model. 1. Use Modeling Language to call a solver. 2. Write a matrix generator and interact directly with solver. 101 / 101