A computational study of Gomory cut generators

Transcription

1 A computational study of Gomory cut generators Gerard Cornuéjols 1, François Margot 1, Giacomo Nannicini 2 1. CMU Tepper School of Business, Pittsburgh, PA. 2. Singapore University of Technology and Design, Singapore, and MIT Sloan School of Management, Cambridge, MA. January 12, 2012 G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

2 Summary of Talk 1 Motivation 2 Cut generation parameters 3 Failures and Feasibility 4 Dive-and-Cut 5 Computational framework 6 Optimizing GMI cut generators G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

3 1 Motivation 2 Cut generation parameters 3 Failures and Feasibility 4 Dive-and-Cut 5 Computational framework 6 Optimizing GMI cut generators G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

4 The problem We consider the following problem: min c x Ax b x R n + j N I x j Z, (MILP) where c Q n, b Q m, A Q m n and N I {1,...,n} We denote by (LP) the Linear Programming relaxation of (MILP) G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

5 Introduction General-purpose solvers for (MILP) rely on Branch-and-Cut Gomory Mixed-Integer (GMI) [Gomory, 1960] cuts are one of the most important cutting plane families [Balas et al., 1996, Bixby and Rothberg, 2007] used by state-of-the-art software Surprisingly, there is no rigorous study of GMI cut generation in the literature G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

6 1 Motivation 2 Cut generation parameters 3 Failures and Feasibility 4 Dive-and-Cut 5 Computational framework 6 Optimizing GMI cut generators G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

7 The GMI formula G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

8 GMI cuts for dummies Cut generation loop: 1 Solve LP relaxation 2 For each basic integer variable that has a fractional value, generate a GMI cut 3 Add round of cuts to the LP relaxation 4 Repeat G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

9 GMI cuts for dummies Cut generation loop: 1 Solve LP relaxation 2 For each basic integer variable that has a fractional value, generate a GMI cut 3 Add round of cuts to the LP relaxation 4 Repeat That s it! G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

10 GMI cuts for dummies Cut generation loop: 1 Solve LP relaxation 2 For each basic integer variable that has a fractional value, generate a GMI cut 3 Add round of cuts to the LP relaxation 4 Repeat That s it!...is it? G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

11 How I wish things were that simple This scheme is not a good idea Because of floating point (finite precision) arithmetic, not all rational numbers can be represented exactly: round-off error For q Q, let R(q) be its representation as a 64-bit floating point number We can have q1 +q 2 = q 3 but R(q 1 )+R(q 2 ) R(q 3 )! Operations on numbers can inflate errors In rational arithmetic, computations can be checked exactly [Espinoza, 2006, Cook et al., 2011] Typically, we use floating point because of its rapidity G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

12 GMI cuts in practice Several operations are performed on generated cutting planes to reduce occurrence of numerical problems A number of empirical rules are applied in open-source codes (COIN-OR Cgl, SCIP) Cut modification Cut rejection G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

13 GMI cuts in practice Several operations are performed on generated cutting planes to reduce occurrence of numerical problems A number of empirical rules are applied in open-source codes (COIN-OR Cgl, SCIP) Cut modification Cut rejection Analysis applies to all tableau cut generators G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

14 Cut modification and rejection routines Cut modification (and its parameters) Coefficient Removal (EPS ELIM for surplus variables, EPS COEFF for original variables) Rhs Relaxation (EPS RELAX ABS, EPS RELAX REL) Cut rejection (and its parameters) Fractionality Check (AWAY) Dynamism Check (MAX DYN) Support Check (MAX SUPP ABS, MAX SUPP REL) Violation Check (MIN VIOL) Scaling ignored here Some cut generators distinguish variables with large bounds ( LUB) and treat them separately (e.g. EPS COEFF LUB, MAX DYN LUB) G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

15 Cut generation parameters 12 parameters involved in the generation of each cutting plane Their value is chosen after tests on a small number of instances Are all the parameters necessary? What value should they take? To answer these questions, we need a framework for testing cut generators G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

16 Strength vs Safety Properties of a good cut generator: G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

17 Strength vs Safety Properties of a good cut generator: Decreases solution time with Branch-and-Cut G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

18 Strength vs Safety Properties of a good cut generator: Decreases solution time with Branch-and-Cut Strength G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

19 Strength vs Safety Properties of a good cut generator: Decreases solution time with Branch-and-Cut Strength Does not cut off feasible solutions or lead to numerical failures of LP solver G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

20 Strength vs Safety Properties of a good cut generator: Decreases solution time with Branch-and-Cut Strength Does not cut off feasible solutions or lead to numerical failures of LP solver Safety G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

21 Strength vs Safety Properties of a good cut generator: Decreases solution time with Branch-and-Cut Strength Does not cut off feasible solutions or lead to numerical failures of LP solver Safety Safety must be tested before strength G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

22 Strength vs Safety Properties of a good cut generator: Decreases solution time with Branch-and-Cut Strength Does not cut off feasible solutions or lead to numerical failures of LP solver Safety Safety must be tested before strength Goal: develop a framework for testing safety of cut generators in a Branch-and-Cut setting, so that strength of generators with similar safety can be compared Related work: [Margot, 2009] G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

23 Strength vs Safety Properties of a good cut generator: Decreases solution time with Branch-and-Cut Strength Does not cut off feasible solutions or lead to numerical failures of LP solver Safety Safety must be tested before strength Goal: develop a framework for testing safety of cut generators in a Branch-and-Cut setting, so that strength of generators with similar safety can be compared Related work: [Margot, 2009] Define safe and unsafe G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

24 1 Motivation 2 Cut generation parameters 3 Failures and Feasibility 4 Dive-and-Cut 5 Computational framework 6 Optimizing GMI cut generators G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

25 Failures Assume that we employ a cut generator in the typical cut generation loop A failure of a given cut generator on (MILP) is the occurrence of one of the following events: 1 A cutting plane that cuts off a known integral feasible solution to (MILP) is generated 2 (LP) becomes infeasible after the addition of cutting planes, but an integral feasible solution for the original problem is known 3 A time limit for cut generation and (LP) resolve is hit G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

26 Feasibility In floating point arithmetic, we have to accept slight violations of the constraints Notation: Rows of A: a i,i = 1,...,m For k > 0, [k] is the set {1,...,k} A point x is (ǫ abs,ǫ rel,ǫ int )-feasible for (MILP) if: 1 i N I,x i x i ǫ int 2 min i [m] {a i x b i } ǫ abs 3 min i [m] {(a i x b i )/ a i 2 } ǫ rel 4 x : Ax b,x 0, x x ǫ rel G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

27 Why relative tolerance? 1 Invariant to rescaling (condition 3) 2 Sanity check (condition 4): x does not satisfy condition 4 and would mark the cut as invalid x satisfies condition 4 and the cut is valid (up to tolerances) Valid cut: (λ 1 a 1 +λ 2 a 2 )x (λ 1 b 1 +λ 2 b 2 ) x ǫ rel ǫ x x rel ǫ rel a 1 x b 1 a 2 x b 2 G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

28 Generation of feasible solutions We picked ǫ abs = 10 9, ǫ rel = 10 9, ǫ int = 0 We generated solution for all instances in the union of miplib3, miplib2003, and the Benchmark Set of miplib2010: 170 instances For each problem instance, we form a set S of (ǫ abs,ǫ rel,ǫ int )-feasible solutions We used Cplex 12.2 to guide the Branch-and-Cut search and QSopt ex [Applegate et al., 2007] as rational LP solver to check feasibility and distances For 160 instances out of 170, we found at least a feasible solution: Data Set G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

29 Dataset Histogram of num_solutions Frequency num_solutions G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

30 1 Motivation 2 Cut generation parameters 3 Failures and Feasibility 4 Dive-and-Cut 5 Computational framework 6 Optimizing GMI cut generators G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

31 The Dive-and-Cut algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

32 The Dive-and-Cut algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

33 The Dive-and-Cut algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

34 The Dive-and-Cut algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

35 The Dive-and-Cut algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

36 The Dive-and-Cut algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

37 The Dive-and-Cut algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

38 The Dive-and-Cut algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

39 The Dive-and-Cut algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

40 The Dive-and-Cut algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

41 1 Motivation 2 Cut generation parameters 3 Failures and Feasibility 4 Dive-and-Cut 5 Computational framework 6 Optimizing GMI cut generators G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

42 Comparing cut generators We identified a set of 51 difficult instances (out of the 160 in Data Set): Failure Set We can compare safety of two cut generators by applying statistical tests to verify whether one yields more failures than the other We assume that the result of an experiment (e.g. failure/not failure) is a random variable but we do not assume an a priori distribution The comparison are based on ranking and are performed at a given significance level Examples: Friedman, Quade, Cochran Q test Detection power and number of failures G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

43 Failures and number of dives Type1 Type2 Type # of failures # of dives G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

44 1 Motivation 2 Cut generation parameters 3 Failures and Feasibility 4 Dive-and-Cut 5 Computational framework 6 Optimizing GMI cut generators G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

45 Our goal Now that we have a framework for testing cut generators, we can try to optimize over the set of all GMI cut generators Each point in the parameter space corresponds to a cut generator Objective? Constraints? G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

46 Our goal Now that we have a framework for testing cut generators, we can try to optimize over the set of all GMI cut generators Each point in the parameter space corresponds to a cut generator Objective? Constraints? GMI cuts Cut modification and rejection clean GMI cuts dangerous GMI cuts G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

47 The problem Constraints: maximum failure rate (i.e. (# failures)/(# dives)), at least as safe as a reference generator (= Cplex) Vector-valued objective function: average cut rejection rate per instance, Friedman test to perform comparisons at the 95% significance level Several cut generators are indistinguishable: multiple local minima G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

48 Don t try this at home! Black-box optimization over a discretized parameter space Costly evaluation of objective function and constraints Not well suited for traditional response surface methods How many function evaluations do we need? G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

49 Don t try this at home! Black-box optimization over a discretized parameter space Costly evaluation of objective function and constraints Not well suited for traditional response surface methods How many function evaluations do we need? Many thanks to Jeff Linderoth and the Condor team at UW-Madison! G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

50 Shrinking the parameter space Even with a huge computational effort, optimizing over 12-dimensional parameter space is too difficult We use standard regression techniques to find the most important parameters 6 parameters suffice to build a good model of the cut rejection rate and the failure rate: AWAY EPS RELAX REL EPS COEFF EPS RELAX ABS We perform: RhsRelaxation CoefficientRemoval MAX DYN MIN VIOL Fractionality Check Dynamism Check Violation Check G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

51 The optimization algorithm Good news: rejection rate is convex! G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

52 The optimization algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

53 The optimization algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

54 The optimization algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

55 The optimization algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

56 The optimization algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

57 The optimization algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

58 The optimization algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

59 The optimization algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

60 The optimization algorithm G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

61 Results 6 iterations Final parameter ranges: AWAY 10 2 EPS RELAX REL [0,10 11 ] EPS COEFF [10 13,10 10 ] MAX DYN [10 7,10 9 ] EPS RELAX ABS [0,10 11 ] MIN VIOL [0,10 11 ] One optimal generator BestGen: AWAY 10 2 EPS RELAX REL 0 EPS COEFF MAX DYN 10 7 EPS RELAX ABS 0 MIN VIOL G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

62 Does this work? We compare BestGen over 300 dives on Failure Set against: Cplex CglGomory CglLandP (used to generate GMI cuts) CglGomoryClone ( clone of CglGomory using our own code) CglLandPClone ( clone of CglLandP using our own code) G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

63 Either it works or we got lucky We use a Cochran Q test to compare the number of dives that end with a failure at the 95% significance level, and obtain the following ranking (failure rate in round brackets): 1 BestGen (0.27%) = CglGomoryClone (0.22%) = Cplex (0.19%) 2 CglLandP (1.41%) 3 CglLandPClone (1.73%) 4 CglGomory (3.35%) We use a Friedman test to compare the gap closed per dive at the 95% significance level by the 3 equally safe generators: 1 BestGen 2 CglGomoryClone 3 Cplex G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

64 Conclusions Framework for testing safety cut generators Dataset of instances and feasible solutions Algorithms and their implementation Attempt to find an optimal parameterization of a GMI cut generator some parameters are not needed Future research: Fine tuning of the parameters Testing strength... G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

65 ...and that s all Thank you! G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36

66 Bibliography Applegate, D. L., Cook, W., Dash, S., and Espinoza, D. G. (2007). Exact solutions to linear programming problems. Operations Research Letters, 35(6): Balas, E., Ceria, S., Cornuéjols, G., and Natraj, N. (1996). Gomory cuts revisited. Operations Research Letters, 19(1):1 9. Bixby, R. and Rothberg, E. (2007). Progress in computational mixed integer programming a look back from the other side of the tipping point. Annals of Operations Research, 149(1): Cook, W., Koch, T., Steffy, D. E., and Wolter, K. (2011). An exact rational mixed-integer programming solver. In Günlük, O. and Woeginger, G. J., editors, Proceedings of IPCO 2011, volume 6655 of Lecture Notes in Computer Science, pages , Berlin Heidelberg. Springer-Verlag. Espinoza, D. G. (2006). On Linear Programming, Integer Programming and Cutting Planes. PhD thesis, School of Industrial and Systems Engineering, Georgia Institute of Technology. Gomory, R. E. (1960). An algorithm for the mixed-integer problem. Technical Report RM-2597, RAND Corporation. Margot, F. (2009). Testing cut generators for mixed-integer linear programming. Mathematical Programming Computation, 1(1): G. Nannicini (SUTD/MIT) Study of Gomory cut generators January 12, / 36