From structures to heuristics to global solvers

Transcription

1 From structures to heuristics to global solvers Timo Berthold Zuse Institute Berlin DFG Research Center MATHEON Mathematics for key technologies OR2013, 04/Sep/13, Rotterdam

2 Outline From structures to heuristics to global solvers 1 Undercover: the largest sub-mip [B., Gleixner. Math. Prog., 2013] 2 RENS: the optimal rounding [B. Math. Prog. C, under review] 3 Measuring the impact of primal heuristics [B. OR Letters, accepted] Timo Berthold: From structures to heuristics to global solvers 2 / 34

3 MIP & MINLP Mixed-Integer Linear Programming (MIP): min s.t. c T x Ax b x i Z for all i I Mixed-Integer Nonlinear Programming (MINLP): min c T x s.t. g(x) 0, g C 1 (R n, R m ) x i Z for all i I Timo Berthold: From structures to heuristics to global solvers 3 / 34

4 Gradient cuts MINLP solving techniques Underestimators Spatial branching g x g c x Reformulation Bound tightening Primal heuristics Timo Berthold: From structures to heuristics to global solvers 4 / 34

5 Gradient cuts MINLP solving techniques Underestimators Spatial branching g x g c x Reformulation Bound tightening Primal heuristics Timo Berthold: From structures to heuristics to global solvers 4 / 34

6 Primal heuristics inside a global solver Primal heuristics... are incomplete methods which often find good solutions within a reasonable time without any warranty! Why use primal heuristics inside a global solver? to prove feasibility of the model often nearly optimal solutions suffice in practice feasible solutions guide remaining search process Timo Berthold: From structures to heuristics to global solvers 5 / 34

7 Active field of research Timo Berthold: From structures to heuristics to global solvers 6 / 34

8 Outline From structures to heuristics to global solvers 1 Undercover: the largest sub-mip [B., Gleixner. Math. Prog., 2013] 2 RENS: the optimal rounding [B. Math. Prog. C, under review] 3 Measuring the impact of primal heuristics [B. OR Letters, accepted] Timo Berthold: From structures to heuristics to global solvers 7 / 34

9 The motivation Large Neighborhood Search: common paradigm in MIP heuristics fix a subset of variables easy subproblem solve MIP: easy = few integralities MINLP: easy = few nonlinearities observation: any MINLP can be reduced to a MIP by fixing (sufficiently many) variables. idea: fix a small subset of variables to obtain a linear subproblem use solution of an LP or NLP relaxation to determine fixing values Timo Berthold: From structures to heuristics to global solvers 8 / 34

10 The structure Definition (cover of an MINLP) Let a domain box [L, U] = i [L i, U i ], a function g j : [L, U] R, x g j (x) on [L, U], and a set C N := {1,..., n} of variable indices be given. We call C a cover of g if and only if for all x [L, U] the set {(x, g j (x)) x [L, U], x k = x k for all k C} is an affine set intersected with [L, U] R. We call C a cover of P if and only if C is a cover for g 1,..., g m. Timo Berthold: From structures to heuristics to global solvers 9 / 34

11 The heuristic Input: MINLP P begin compute a solution x of an approximation of P; round x k for all k I; determine a cover C of P; solve the sub-mip of P given by fixing x k = x k for all k C; Timo Berthold: From structures to heuristics to global solvers 10 / 34

12 The heuristic Input: MINLP P begin compute a solution x of an approximation of P; round x k for all k I; determine a cover C of P; solve the sub-mip of P given by fixing x k = x k for all k C; Timo Berthold: From structures to heuristics to global solvers 10 / 34

13 The heuristic Input: MINLP P begin compute a solution x of an approximation of P; round x k for all k I; determine a cover C of P; solve the sub-mip of P given by fixing x k = x k for all k C; Timo Berthold: From structures to heuristics to global solvers 10 / 34

14 The heuristic Input: MINLP P begin compute a solution x of an approximation of P; round x k for all k I; determine a cover C of P; solve the sub-mip of P given by fixing x k = x k for all k C; Timo Berthold: From structures to heuristics to global solvers 10 / 34

15 The heuristic Input: MINLP P begin compute a solution x of an approximation of P; round x k for all k I; determine a cover C of P; solve the sub-mip of P given by fixing x k = x k for all k C; Timo Berthold: From structures to heuristics to global solvers 10 / 34

16 The heuristic Input: MINLP P begin compute a solution x of an approximation of P; round x k for all k I; determine a cover C of P; solve the sub-mip of P given by fixing x k = x k for all k C; Remark: MIP heuristics: trade-off fixing many vs. few variables here: eliminate nonlinearities by fixing as few as possible variables Timo Berthold: From structures to heuristics to global solvers 10 / 34

17 The heuristic Input: MINLP P begin compute a solution x of an approximation of P; round x k for all k I; determine a cover C of P; solve the sub-mip of P given by fixing x k = x k for all k C; Remark: MIP heuristics: trade-off fixing many vs. few variables here: eliminate nonlinearities by fixing as few as possible variables How to find minimum cover? Timo Berthold: From structures to heuristics to global solvers 10 / 34

18 Covering quadratic functions Let g j : R n R, x x T Qx + qx + c, Q R n n symmetric for all j. Auxiliary binary variables: α k = 1 : x k is fixed in P Set covering constraints: C(α) := {k α k = 1} is a cover of P if and only if α k = 1 for all square nonzeros: Qkk i 0, (SN) α k + α j 1 for all bilinear nonzeros: Qkj i 0, k j. (BN) Solve the covering problem { n min α k : (SN), (BN), α {0, 1} }. n k=1 Timo Berthold: From structures to heuristics to global solvers 11 / 34

19 Covering quadratic functions Let g j : R n R, x x T Qx + qx + c, Q R n n symmetric for all j. Auxiliary binary variables: α k = 1 : x k is fixed in P Set covering constraints: C(α) := {k α k = 1} is a cover of P if and only if α k = 1 for all square nonzeros: Qkk i 0, (SN) α k + α j 1 for all bilinear nonzeros: Qkj i 0, k j. (BN) Solve the covering problem { n min α k : (SN), (BN), α {0, 1} }. n k=1 Timo Berthold: From structures to heuristics to global solvers 11 / 34

20 Covering quadratic functions Let g j : R n R, x x T Qx + qx + c, Q R n n symmetric for all j. Auxiliary binary variables: α k = 1 : x k is fixed in P Set covering constraints: C(α) := {k α k = 1} is a cover of P if and only if α k = 1 for all square nonzeros: Qkk i 0, (SN) α k + α j 1 for all bilinear nonzeros: Qkj i 0, k j. (BN) Solve the covering problem { n min α k : (SN), (BN), α {0, 1} }. n k=1 Timo Berthold: From structures to heuristics to global solvers 11 / 34

21 General covering problems Covering problem is N P-hard, but branch-and-cut (empirically) very fast For general MINLPs, the covering problem becomes more difficult, e.g. for a global cover of a monomial x p 1 1 x n pn, p 1,..., p n N 0 : k:p k =1 α k = 1 for all p k 2 (1 α k ) 1. For general MINLPs, global covers become larger and larger. Timo Berthold: From structures to heuristics to global solvers 12 / 34

22 Implementation within SCIP one of the fastest non-commercial MIP solvers time in seconds x 3.94x non-commercial commercial GLPK 4.47 lpsolve CBC SCIP CLP x SCIP SoPlex SCIP Cplex x x 0.77x Xpress x Cplex x 0.17x 0 Gurobi solved (of 87 instances) results by H. Mittelmann (10/Jan/2013) MINLP solver benchmark time in seconds x 1.82x 2.05x 2.40x solved (of 208 instances) SCIP CPLEX BARON 11.3 Couenne 0.4 LindoAPI results on MINLPLIB (August 2012) (all instances that can be handled by all solvers) Timo Berthold: From structures to heuristics to global solvers 13 / 34

23 Empirical cover sizes How big are minimum covers? SCIP 3.0 on 255 instances from MINLPLib 14% on 85, 36% on 170 instances similar on MIQCPs and general MINLPs Timo Berthold: From structures to heuristics to global solvers 14 / 34

24 The results Test set 149 MIQCPs from GloMIQO test set Comparison to other heuristics Undercover: solution for 76 instances (typically less than 0.1 sec) root heuristics: Baron 65, Couenne 55, SCIP 98 lower success rate on general MINLPs Extensions cover structure can be further exploited Undercover branching [B., Gleixner. Proc. of SEA 2013] Timo Berthold: From structures to heuristics to global solvers 15 / 34

25 Outline From structures to heuristics to global solvers 1 Undercover: the largest sub-mip [B., Gleixner. Math. Prog., 2013] 2 RENS: the optimal rounding [B. Math. Prog. C, under review] 3 Measuring the impact of primal heuristics [B. OR Letters, accepted] Timo Berthold: From structures to heuristics to global solvers 16 / 34

26 The motivation(s) First motivation: many MIP heuristics based on rounding How likely to succeed? How good can they get? Timo Berthold: From structures to heuristics to global solvers 17 / 34

27 The motivation(s) First motivation: many MIP heuristics based on rounding How likely to succeed? How good can they get? Second motivation: Large Neighborhood Search e.g., RINS, Local Branching typically requires incumbent solution Timo Berthold: From structures to heuristics to global solvers 17 / 34

28 The motivation(s) First motivation: many MIP heuristics based on rounding How likely to succeed? How good can they get? Second motivation: Large Neighborhood Search e.g., RINS, Local Branching typically requires incumbent solution Timo Berthold: From structures to heuristics to global solvers 17 / 34

29 The structure Definition (rounding) Let x [L, U]. The set R( x) := {x [L, U] x j { x j, x j } j I} is called the set of roundings of x. Definition (optimal rounding) Let x [L, U] and x R( x). 1. We call x a feasible rounding of x, if g i ( x) 0 i M. 2. We call x an optimal rounding of x, if x argmin{d T x x R(x), g i (x) 0 i M}. 3. We call x roundable if it has a feasible rounding. Timo Berthold: From structures to heuristics to global solvers 18 / 34

30 The heuristic Input: MINLP P begin x relaxation optimum; Fix all integral variables x i := x i for all i : x i Z; Reduce domain of fractional variables x i { x i ; x i }; Solve the resulting sub-minlp; Timo Berthold: From structures to heuristics to global solvers 19 / 34

31 The heuristic Input: MINLP P begin x relaxation optimum; Fix all integral variables x i := x i for all i : x i Z; Reduce domain of fractional variables x i { x i ; x i }; Solve the resulting sub-minlp; Timo Berthold: From structures to heuristics to global solvers 19 / 34

32 The heuristic Input: MINLP P begin x relaxation optimum; Fix all integral variables x i := x i for all i : x i Z; Reduce domain of fractional variables x i { x i ; x i }; Solve the resulting sub-minlp; Timo Berthold: From structures to heuristics to global solvers 19 / 34

33 The heuristic Input: MINLP P begin x relaxation optimum; Fix all integral variables x i := x i for all i : x i Z; Reduce domain of fractional variables x i { x i ; x i }; Solve the resulting sub-minlp; Timo Berthold: From structures to heuristics to global solvers 19 / 34

34 The heuristic Input: MINLP P begin x relaxation optimum; Fix all integral variables x i := x i for all i : x i Z; Reduce domain of fractional variables x i { x i ; x i }; Solve the resulting sub-minlp; Timo Berthold: From structures to heuristics to global solvers 19 / 34

35 Observations Lemma Let the starting point x be feasible for a relaxation. 1. If the sub-minlp is infeasible, then no feasible rounding of x exists. 2. If not, the optimum of the sub-minlp is the optimal rounding of x. Corollary x either has an optimal rounding or is not roundable. Timo Berthold: From structures to heuristics to global solvers 20 / 34

36 Computational results Results when using an LP relaxation: MIPLIB MIQCP MINLPLib integrality (% vars) roundability (% inst) Further findings Timo Berthold: From structures to heuristics to global solvers 21 / 34

37 A closer look at roundability 100 MIP: % success rate MIQCP: % success rate moving average (10 inst.) moving average (30 inst.) % integral variables moving average (10 inst.) moving average (30 inst.) % integral variables Timo Berthold: From structures to heuristics to global solvers 22 / 34

38 Computational results Results when using an LP relaxation: MIPLIB MIQCP MINLPLib integrality (% vars) roundability (% inst) Further findings cuts: no effect on integrality, bad for roundability NLP solutions much less integral, similar roundability Timo Berthold: From structures to heuristics to global solvers 23 / 34

39 Computational results Results when using an LP relaxation: MIPLIB MIQCP MINLPLib integrality (% vars) roundability (% inst) Further findings cuts: no effect on integrality, bad for roundability NLP solutions much less integral, similar roundability RENS as primal heuristic defensive version succeeds in 50% (MIP and MIQCP) inferior to portfolio, superior to single best mixed runtimes: mostly 5 sec, for MIP sometimes 1 min Timo Berthold: From structures to heuristics to global solvers 23 / 34

40 Outline From structures to heuristics to global solvers 1 Undercover: the largest sub-mip [B., Gleixner. Math. Prog., 2013] 2 RENS: the optimal rounding [B. Math. Prog. C, under review] 3 Measuring the impact of primal heuristics [B. OR Letters, accepted] Timo Berthold: From structures to heuristics to global solvers 24 / 34

41 And what about the global solvers? How important are primal heuristics in global solvers? A MIP software vendor says: Our advanced MIP heuristics for quickly finding feasible solutions often produce good quality solutions where other solvers fall flat, leading to some of our biggest wins vs. the competition. Timo Berthold: From structures to heuristics to global solvers 25 / 34

42 And what about the global solvers? How important are primal heuristics in global solvers? A MIP software vendor says: Our advanced MIP heuristics for quickly finding feasible solutions often produce good quality solutions where other solvers fall flat, leading to some of our biggest wins vs. the competition. Which heuristics? trade secret few parameters to influence behavior of heuristics output only tells you that solution was found by some heuristic Timo Berthold: From structures to heuristics to global solvers 25 / 34

43 And what about the global solvers? How important are primal heuristics in global solvers? A MIP software vendor says: Our advanced MIP heuristics for quickly finding feasible solutions often produce good quality solutions where other solvers fall flat, leading to some of our biggest wins vs. the competition. Which heuristics? trade secret few parameters to influence behavior of heuristics output only tells you that solution was found by some heuristic very important Timo Berthold: From structures to heuristics to global solvers 25 / 34

44 And what about the global solvers? How important are primal heuristics? Typical measure: running time to prove optimality Timo Berthold: From structures to heuristics to global solvers 26 / 34

45 And what about the global solvers? How important are primal heuristics? Typical measure: running time to prove optimality Timo Berthold: From structures to heuristics to global solvers 26 / 34

46 And what about the global solvers? How important are primal heuristics? Typical measure: running time to prove optimality one vendor: 6% improvement other vendor: 9% improvement non-commercial solver: 15 % improvement not important at all Timo Berthold: From structures to heuristics to global solvers 26 / 34

47 And what about the global solvers? How important are primal heuristics? Typical measure: running time to prove optimality one vendor: 6% improvement other vendor: 9% improvement non-commercial solver: 15 % improvement not important at all So, what is wrong here? Timo Berthold: From structures to heuristics to global solvers 26 / 34

48 Comparing performance How to measure the added value of a primal heuristic? time to optimality t solved, number of branch-and-bound nodes very much depends on dual bound time to best solution t opt nearly optimal solution might be found long before time to first solution t 1 disregards solution quality performance profiles depend on tsolved, hence on dual bound not an absolute number primal integral Timo Berthold: From structures to heuristics to global solvers 27 / 34

49 Primal gap function Let x be a solution, x opt be an optimum, t max R 0 be a timelimit. Primal gap γ [0, 1] of x: 0, if c T x opt = c T x = 0, γ( x) := 1, if c T x opt c T x < 0, c T x opt c T x else. max{ c T x opt, c T x }, Primal gap function p : [0, t max ] [0, 1]: { 1, if no incumbent until point t, p(t) := γ( x(t)), with x(t) incumbent at point t. Timo Berthold: From structures to heuristics to global solvers 28 / 34

50 Primal integral step function, changes at points t i when new incumbent found p(0) = 1, p(t) = 0 for all t t opt monotonically decreasing Primal integral P(T ) of T [0, t max ]: P(T ) := T t=0 p(t) dt = I p(t i 1 ) (t i t i 1 ), i=1 Timo Berthold: From structures to heuristics to global solvers 29 / 34

51 Comparing performance How to measure the added value of a primal heuristic? time to optimality t solved, number of branch-and-bound nodes very much depends on dual bound time to best solution t opt nearly optimal solution might be found long before time to first solution t 1 disregards solution quality performance profiles depend on tsolved, hence on dual bound not an absolute number primal integral P(t max ) favors finding good solutions early considers each update of incumbent P(t max)/t max average solution quality expected quality of the incumbent, if stopped arbitrarily Timo Berthold: From structures to heuristics to global solvers 30 / 34

52 Average primal integral: SCIP / MIP no heuristics (16.18%) default ( 9.05%) Timo Berthold: From structures to heuristics to global solvers 31 / 34

53 Average primal integral: SCIP / MINLP no heuristics (23.97%) default (15.99%) Timo Berthold: From structures to heuristics to global solvers 32 / 34

54 Conclusion Undercover & RENS: LNS start heuristics for MINLP exploit generic problem structures successful application as root node heuristics Primal integral: new performance measure, captures overall solution process more robust, though not immune, against randomness overall impact of primal heuristics (w.r.t. P(t max )) significant slides and technical reports of presented papers: SCIP Optimization Suite: Timo Berthold: From structures to heuristics to global solvers 33 / 34

55 From structures to heuristics to global solvers Timo Berthold Zuse Institute Berlin DFG Research Center MATHEON Mathematics for key technologies OR2013, 04/Sep/13, Rotterdam