Heuristics for nonconvex MINLP

Transcription

1 Heuristics for nonconvex MINLP Pietro Belotti, Timo Berthold FICO, Xpress Optimization Team, Birmingham, UK 18th Combinatorial Optimization Workshop, Aussois, 9 Jan 2014 ======This presentation is provided for the recipient only and ======cannot be reproduced or shared without Fair Isaac Corporation s express consent. ====== c 2014 Fair Isaac Corporation

2 Mixed-Integer Non-Linear Programming min f(x) s.t. g(x) 0 x (Z p R n p ) [l,u]

3 Mixed-Integer Non-Linear Programming min f(x) s.t. g(x) 0 x (Z p R n p ) [l,u] Assume f : R n R and g : R n R m are factorable

4 Feasibility Pump heuristics 1 Feasibility Pump heuristics are suited for problems where integrality constraints: x I = Z p R n p and continuous constraints: x C = {x R n : g(x) 0} coexist, such as MILP and MINLP. 1 M. Fischetti, F. Glover, A. Lodi, The feasibility pump, Math. Prog. 104(1):91-104, 2005.

5 Feasibility Pump heuristics 1 Feasibility Pump heuristics are suited for problems where integrality constraints: x I = Z p R n p and continuous constraints: x C = {x R n : g(x) 0} coexist, such as MILP and MINLP. Idea: generate two sequences of (possibly infeasible) solutions: (ˆx k ) k N : satisfies integrality, i.e. ˆx k I ( x k ) k N : satisfies continuous constraints, i.e., x k C The next point of sequence ( x k ) is the closest to the previous point of sequence (ˆx k ) and viceversa. 1 M. Fischetti, F. Glover, A. Lodi, The feasibility pump, Math. Prog. 104(1):91-104, 2005.

6 Feasibility Pump heuristics Initialization: x 0 C. k 1 repeat ˆx k argmin x I x x k 1 x k argmin x C x ˆx k k k +1 until ˆx k or x k is feasible

7 MILP: The Feasibility Pump Consider (MILP) min{c x : Ax b,x Z p R n p }. Set x 0 argmin{c x : Ax b} and k = 1. Then ˆx k = x k x k argmin{ x ˆx k 1 1 : Ax b}

8 MILP variants The original FP finds a feasible, but often bad, initial solution. The objective function c x is ignored Objective FP 2 : Feasibility Pump ˆx k = x k x k argmin{α x ˆx k 1 1 +βc x : Ax b} better rounding heuristics than x perturbation in the event of cycling 2 T. Achterberg, T. Berthold, Improving the feasibility pump. Discrete Optimization 4(1):77-86, M. Fischetti, D. Salvagnin, Feasibility pump 2.0, Math. Prog. Comp. 1(2-3): , 2009.

9 Convex MINLP: The enhanced Feasibility Pump 4 The first attempt to port the FP to the MINLP realm. The two sequences obtained by solving two simpler problems: an NLP problem a MILP approximation of the MINLP MILP provided by Outer Approximation: min z s.t. f( x)+ f( x)(x x) z g i ( x)+ g i ( x)(x x) 0 i = 1,2,...,m, 4 P. Bonami, G. Cornuéjols, A. Lodi, F. Margot, A Feasibility Pump for Mixed Integer Nonlinear Programs. Math. Prog. 119(2): , 2009.

10 Nonconvex MINLP integrality constraints: x I = Z p R n p and.. nonconvex continuous constraints: x C = {x R n : g(x) 0} Previous attempts try to solve (even heuristically) the continuous nonconvex problem through a global solver or multi-start 5 5 C. D Ambrosio, A. Frangioni, L. Liberti, A. Lodi, Experiments with a feasibility pump approach for nonconvex MINLPs, Proceedings of SEA 2010.

11 Nonconvex Feasibility Pump Take advantage of features of both the solver and the problem. convexification cuts: use a valid LP relaxation rather than Outer Approximation nonlinearity: use second-order information

12 Nonconvex problems: lower bounds (P 0 ) min f(x) s.t. g(x) 0 x (Z p R n p ) [l,u]

13 Nonconvex problems: lower bounds (P 0 ) min f(x) s.t. g(x) 0 x (Z p R n p ) [l,u] is reformulated as an equivalent problem (P) min x n+q s.t. x k = ϑ k (x) x (Z s R n+q s ) [l,u ] n+1 k n+q

14 Nonconvex problems: lower bounds (P 0 ) min f(x) s.t. g(x) 0 x (Z p R n p ) [l,u] is reformulated as an equivalent problem (P) min x n+q s.t. x k = ϑ k (x) x (Z s R n+q s ) [l,u ] n+1 k n+q and each x k = ϑ k (x) is linearized: (MILP) min x n+q s.t. a k x k +A k x b k x (Z s R n+q s ) [l,u ] n+1 k n+q

15 NLP points and MILP points (P) min x n+q s.t. x k = ϑ k (x) x (Z s R n+q s ) [l,u ] n+1 k n+q

16 NLP points and MILP points (P) min x n+q s.t. x k = ϑ k (x) x (Z s R n+q s ) [l,u ] n+1 k n+q (MILP) min x n+q s.t. a k x k +A k x b k x (Z s R n+q s ) [l,u ] n+1 k n+q

17 NLP points and MILP points (NLP) min x n+q s.t. x k = ϑ k (x) x (R s R n+q s ) [l,u ] n+1 k n+q (MILP) min x n+q s.t. a k x k +A k x b k x (Z s R n+q s ) [l,u ] n+1 k n+q C = {x R n+q [l,u ] : x k = ϑ k (x), k} I = {x Z s R n+q s [l,u ] : a k x k +A k x b k, k}

18 Nonconvex Feasibility Pump Initialization: x 0 argmin{f(x) : g(x) 0} C. k 1 repeat ˆx k argmin x I (x, x k 1 ) x k argmin x C (x,ˆx k ) k k +1 until ˆx k or x k is feasible and combine l p distance, objective function, and second order information from the problem

19 Distance measures: second order information Consider the (locally!) optimal solution x 0 of the continuous relaxation of the MINLP. If no active constraints, the Hessian of the objective is PSD: 2 f( x 0 ) 0 In general, the Hessian of the Lagrangian provides useful info We consider it only in the null space of the gradient: ( f) x = x n+q = f( x 0 )

20 Distance measures: second order information Construct PSD matrix P from the Hessian of the Lagrangian by Obtaining a projection on the cone of PSD matrices... (by eliminating negative eigenvalues) Then use: NLP: (x, x) := α x x 2 +β (x x) P(x x)+γx n+q MILP: (x, x) := α x x 1 +β P 2(x x) 1 1 +γx n+q

21 Distance measures: unconstrained case

22 Distance measures: unconstrained case

23 Distance measures: solving NLP

24 Distance measures: solving NLP

25 Distance measures: solving MILP

26 Distance measures: solving MILP

27 (Not) solving the MILP exactly (ˆx k ) obtained by solving an MILP. Optimality takes time and is unnecessary Impose node limit (10k) or Use heuristics Optional: adaptive MILP method selection If succeeded five times in a row, switch to cheaper method If current method returns no solution, switch to more expensive one and rerun

28 Computational tests Implemented in Couenne, an Open-Source MINLP solver 6. Available in stable/0.4 Run at most once at every node until depth 5... and with decreasing frequency afterward Time limit: 2 hours MILP (+heuristics): SCIP 7 NLP solver: Ipopt

29 Computational tests: full Couenne run Tested on 266 MINLP instances from minlplib 9, macminlp were solved by all variants in less than a minute ignored, 86 instances remaining

30 Compared four variants No heuristic (MINLP solutions found after LP solver) Greedy rounding heuristic (default in Couenne) FP1: Feasibility pump where, : resp. l 1 and l 2 distance FP2: Feasibility pump with NLP:.25 distance,.75 Hessian MILP:.75 distance,.25 objective

31 Summary None GreedyR FP1 FP2 Instances solved (Strictly) better LB Best time before time limit 12 for the 35 instances not solved by any 13 for the 24 instances solved by all

32 Standalone FP on 200 instances from MinlpLib MILP solver best obj worst obj #sol t(geo) t(sh.geo) adaptive MILP MILP, emph: feas MILP, 5k nodes RENS ObjFP Partial overlap with 250 instances from the storm 14 paper... Storm paper: solution found in 200 instances out of Some of our solutions (< 10) were better 14 C. D Ambrosio, A. Frangioni, L. Liberti, A. Lodi, A storm of feasibility pumps for nonconvex MINLP, Math Prog B 136(2): , 2012.

33 Tests on Distance/Hessian/Objective weights Weights differ for NLP and MILP (full MILP solve) For distance, Hessian, and objective term in and : : 1 (exclusive) : decreasing as.95 iter, + : increasing as 1.95 iter

34 Tests on Distance/Hessian/Objective weights NLP MILP dist Hess obj dist Hess obj best obj worst obj #feas

35 Thank you

36 Some results CPU times (in seconds) or, if above time limit (2h), the lower bound in brackets Name #var #int #con None GR FP1 FP2 pump (78303) (74524) synheat (81794) (87461) (154997) 1665 synh-mod (81990) (85248) armin (24.6) parallel (-1755) (-21) multist (-39525) (-24751) (-7681) (-7801) product (-2238) (-2238) oil (-0.9) 1084

37 Some results (cont.) CPU times (in seconds) or, if above 2h, lower bound in brackets Name #var #int #con None GR FP1 FP2 st e (54416) (55302) tln (7.4) (6.75) trimlon (7.4) (6.77) tln (9.0) (8.89) (12.2) (11.3) tln (5.4) (5.37) (10.5) (9.8) tln (20.6) (22.25) (23) (24.1) trimlon (20.5) (22.33) (24.0) (23.9) tls (1.7) (1.76) (6.6) (8.5) tls (9.9) (9.30) (12.1) (12.5) space (105.1) (105.1) (71.8) 59

38 minlplib/ex None RH FP2 FP1

39 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x

40 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

41 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

42 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

43 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

44 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

45 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

46 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

47 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

48 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

49 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

50 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

51 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

52 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

53 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

54 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

55 Spatial B&B with convexification Apply a branch and bound where, at each node: (BT) Tighten variables bounds repeat k times: add cuts of linear env. if no cuts found, break get lower bd., soln. (ˆx,ŷ) (Ub) Look for a feasible solution with NLP solver Ipopt (Int Br.) If a variable ŷ i is fractional, branch on y i (NL Br.) If w i = h i (x) infeasible, i.e. ŵ i h i (ˆx), branch on x x 2 i l i x j u i x i

56 Reformulation-Linearization Create linear envelope for each operator in the objective/constraints. Ex.: y 2 log(x 3 +5) 7

57 Reformulation-Linearization Create linear envelope for each operator in the objective/constraints. Ex.: y 2 log(x 3 +5) 7 w 1 = x 3 w 2 = w 1 +5 w 3 = logw 2 w 4 = y 2 w 5 = w 4 w 3 w 5 7

58 Reformulation-Linearization Create linear envelope for each operator in the objective/constraints. Ex.: y 2 log(x 3 +5) 7 w 1 = x 3 w 2 = w 1 +5 w 3 = logw 2 w 4 = y 2 w 5 = w 4 w 3 w 5 7

59 Reformulation-Linearization Create linear envelope for each operator in the objective/constraints. Ex.: y 2 log(x 3 +5) 7 x 2 (x 1 ) 3 w 1 = x 3 a 1 w 1 +a 2 x b 1 a 3 w 1 +a 4 x b 2 w 2 = w 1 +5 l 1 u 1 x 1 w 3 = logw 2 w 4 = y 2 w 5 = w 4 w 3 w 5 7

60 Reformulation-Linearization Create linear envelope for each operator in the objective/constraints. Ex.: y 2 log(x 3 +5) 7 w 1 = x 3 a 1 w 1 +a 2 x b 1 a 3 w 1 +a 4 x b 2 w 2 = w 1 +5 w 2 = w 1 +5 w 3 = logw 2 w 4 = y 2 w 5 = w 4 w 3 w 5 7

61 Reformulation-Linearization Create linear envelope for each operator in the objective/constraints. Ex.: y 2 log(x 3 +5) 7 x 2 l 1 u 1 x 1 w 1 = x 3 a 1 w 1 +a 2 x b 1 a 3 w 1 +a 4 x b 2 w 2 = w 1 +5 w 2 = w 1 +5 w 3 = logw 2 a 5 w 3 +a 6 w 2 b 3 a 7 w 3 +a 8 w 2 b 4 w 4 = y 2 w 5 = w 4 w 3 w 5 7

62 Reformulation-Linearization Create linear envelope for each operator in the objective/constraints. Ex.: y 2 log(x 3 +5) 7 x 2 l 1 u 1 x 1 w 1 = x 3 a 1 w 1 +a 2 x b 1 a 3 w 1 +a 4 x b 2 w 2 = w 1 +5 w 2 = w 1 +5 w 3 = logw 2 a 5 w 3 +a 6 w 2 b 3 a 7 w 3 +a 8 w 2 b 4 w 4 = y 2 a 9 w 4 +a 10 y b 5 a 11 w 4 +a 12 y b 6 w 5 = w 4 w 3 w 5 7

63 Reformulation-Linearization Create linear envelope for each operator in the objective/constraints. Ex.: y 2 log(x 3 +5) 7 w 1 = x 3 a 1 w 1 +a 2 x b 1 a 3 w 1 +a 4 x b 2 w 2 = w 1 +5 w 2 = w 1 +5 w 3 = logw 2 a 5 w 3 +a 6 w 2 b 3 a 7 w 3 +a 8 w 2 b 4 w 4 = y 2 a 9 w 4 +a 10 y b 5 a 11 w 4 +a 12 y b 6 w 5 = w 4 w 3 a 13 w 5 +a 14 w 4 +a 15 w 3 b 7 a 16 w 5 +a 17 w 4 +a 18 w 3 b 8 w 5 7

64 Reformulation-Linearization Create linear envelope for each operator in the objective/constraints. Ex.: y 2 log(x 3 +5) 7 w 1 = x 3 a 1 w 1 +a 2 x b 1 a 3 w 1 +a 4 x b 2 w 2 = w 1 +5 w 2 = w 1 +5 w 3 = logw 2 a 5 w 3 +a 6 w 2 b 3 a 7 w 3 +a 8 w 2 b 4 w 4 = y 2 a 9 w 4 +a 10 y b 5 a 11 w 4 +a 12 y b 6 w 5 = w 4 w 3 a 13 w 5 +a 14 w 4 +a 15 w 3 b 7 a 16 w 5 +a 17 w 4 +a 18 w 3 b 8 w 5 7 w 5 7

65 Reformulation-Linearization Create linear envelope for each operator in the objective/constraints. Ex.: y 2 log(x 3 +5) 7 w 1 = x 3 a 1 w 1 +a 2 x b 1 a 3 w 1 +a 4 x b 2 w 2 = w 1 +5 w 2 = w 1 +5 a w 3 = logw 5 w 3 +a 6 w 2 b 3 2 a 7 w 3 +a 8 w 2 b 4 w 4 = y 2 a 9 w 4 +a 10 y b 5 a 11 w 4 +a 12 y b 6 w 5 = w 4 w 3 a 13 w 5 +a 14 w 4 +a 15 w 3 b 7 a 16 w 5 +a 17 w 4 +a 18 w 3 b 8 w 5 7 w 5 7