Advances in CPLEX for Mixed Integer Nonlinear Optimization

Transcription

1 Pierre Bonami and Andrea Tramontani IBM ILOG CPLEX ISMP Pittsburgh - July Advances in CPLEX for Mixed Integer Nonlinear Optimization IBM Corporation

2 CPLEX Optimization Studio is available now! DoCloud services: OPL on the cloud. Performance improvements to CPO CPLEX Optimizer performance: IBM Corporation

3 CPLEX Optimization Studio is available now! DoCloud services: OPL on the cloud. Performance improvements to CPO CPLEX Optimizer performance: Exploit symmetry for LP Cuts for non-convex (MI)QP Improvements for SOCP Improvement to convex MIQCP/MISOCP IBM Corporation

4 CPLEX Optimization Studio is available now! DoCloud services: OPL on the cloud. Performance improvements to CPO CPLEX Optimizer performance: Exploit symmetry for LP Cuts for non-convex (MI)QP Improvements for SOCP Improvement to convex MIQCP/MISOCP Nonconvex (MI)QP > 0 sec 395 Models > 1 sec 134 Models IBM Corporation

5 CPLEX Optimization Studio is available now! DoCloud services: OPL on the cloud. Performance improvements to CPO CPLEX Optimizer performance: Exploit symmetry for LP Cuts for non-convex (MI)QP Improvements for SOCP Improvement to convex MIQCP/MISOCP Nonconvex (MI)QP > 0 sec 395 Models > 1 sec 134 Models IBM Corporation

6 Convex MIQCP and MISOCP min c T x x T Q k x +ak Tx a0 k Ax = b, x j Z k = 1,...,m, j = 1,...,p. (MIQCP) Where all quadratic constraints can be represented as second order cones: L d := {(x,x 0 ) R d+1 : d i=1 x 2 i x 2 0,x 0 0}. (L d defines the (d + 1)-dimensional second order cone.) IBM Corporation

7 Second order cone representability Through basic transformations can model: Convex quadratic constraints: x T Qx +a T x a 0, with Q 0 Rotated second order cones: d xi 2 x 0 x 1, with x 0,x 1 0 or more complicated... i=2 x T Qx +a T x c T x +b, with Q 0 Second Order Cone Programming (p = 0) Solved efficiently by interior point methods Supported by many commercial solvers (CPLEX in particular). CPLEX automatically recognizes and reformulates convex quadratics, rotated SOC, but not every form IBM Corporation

8 MISOCP min c T x (x Ji,x hi ) L d i i = 1,...,m Ax = b, x j Z j = 1,...,p. (MISOCP) Algorithms are based on SOCP relaxation. Convex MINLP algorithms work with some added technicality due to non-differentiability [Drewes, 2009, Drewes and Ulbrich, 2012]. Supported by most commercial MIP solvers (CPLEX in particular) IBM Corporation

9 Agenda The basic algorithms in CPLEX SOCP-BB OA-BB Comparisons in CPLEX Novelties of CPLEX Cone disaggregation Lift-and-project for MISOCP Comparisons of CPLEX vs IBM Corporation

10 SOCP based branch-and-bound Straightforward generalization of main MILP algorithm: solve an SOCP at each node of the tree IBM Corporation

11 SOCP based branch-and-bound Straightforward generalization of main MILP algorithm: solve an SOCP at each node of the tree. Branch on variables with fractional value. integer feasible infeasible fathomed by bound IBM Corporation

12 SOCP based branch-and-bound Straightforward generalization of main MILP algorithm: solve an SOCP at each node of the tree. Branch on variables with fractional value. Prune by infeasibility, bounds and integer feasibility. integer feasible infeasible fathomed by bound IBM Corporation

13 Outer Approximation [Duran and Grossmann, 1986] min c T x s.t. g i (x) 0 Ax = b x j Z, i = 1,...,m, j = 1,...,p. Idea: Take first-order approximations of constraints at different points and build an equivalent MILP IBM Corporation

14 Outer Approximation [Duran and Grossmann, 1986] min c T x s.t. g i (x) 0 Ax = b x j Z, i = 1,...,m, j = 1,...,p. Idea: Take first-order approximations of constraints at different points and build an equivalent MILP. min c T x s.t. g i (x k )+ g i (x k ) T (x x k ) 0 x j Z, i = 1,...,m, k = 1,...,K j = 1,...,p IBM Corporation

15 OA Branch-and-cut [Quesada and Grossmann, 1992] Initialize by solving SOCP relaxation and taking OA s at the optimum. At each integer feasible node: integer feasible IBM Corporation

16 OA Branch-and-cut [Quesada and Grossmann, 1992] Initialize by solving SOCP relaxation and taking OA s at the optimum. At each integer feasible node: 1 solve SOCP with integers fixed, and enrich the set of linearizations. 2 Resolve the LP relaxation of the node with the new cuts. 3 Repeat as long as node is integer feasible. integer feasible IBM Corporation

17 OA Branch-and-cut [Quesada and Grossmann, 1992] Initialize by solving SOCP relaxation and taking OA s at the optimum. At each integer feasible node: 1 solve SOCP with integers fixed, and enrich the set of linearizations. 2 Resolve the LP relaxation of the node with the new cuts. 3 Repeat as long as node is integer feasible. Never prune by integer feasibility. integer feasible IBM Corporation

18 The MISOCP solver in CPLEX Implements two main algorithms; choice controled by parameter CPXPARAM_MIP_Strategy_MIQCPStrat SOCP based Branch-and-bound (miqcpstrat 1). LP based branch-and-cut (miqcpstrat 2). Default (miqcpstrat 0) is deciding which of the two to run in a clever way. History of MISOCP with CPLEX class p algorithm V. (Year) SOCP 0 barrier 9.0 (2003) MISOCP > 0 SOCP based B&B 9.0 (2003) LP based B&C 11.0 (2007) Cone propagations (2014) IBM Corporation

19 Comparing OA and SOCP BB s in CPLEX % model solved 225 models solved by at least one method and failed by none. Default strategy picked _miqcpstrat_0 1261_miqcpstrat_1 1261_miqcpstrat_ time factor OA 113 times SOCP-BB 46 times 56 models identical with both To be perfect should have picked 14 more models with OA 36 more models with SOCP-BB IBM Corporation

20 How bad can outer approximation be? Consider the following convex MIQCP: min s.t. c T x n ( ) i=1 xi n 1 (1) 4 x Z n z (1) is infeasible: The ball is too small to contain integer points. It is large enough to touch every edge of the hypercube. x y IBM Corporation

21 Solving (1) with OA cuts z No OA constraint can cut 2 vertices of the hypercube. If an inequality cuts two vertices, it cuts the segment joining them. This can not be: the ball has non-empty intersection with any such segment. x y IBM Corporation

22 Solving (1) with OA cuts z No OA constraint can cut 2 vertices of the hypercube. If an inequality cuts two vertices, it cuts the segment joining them. This can not be: the ball has non-empty intersection with any such segment. An OA would need at least 2 n OA cuts to converge. x y IBM Corporation

23 Solving (1) with OA cuts z No OA constraint can cut 2 vertices of the hypercube. If an inequality cuts two vertices, it cuts the segment joining them. This can not be: the ball has non-empty intersection with any such segment. An OA would need at least 2 n OA cuts to converge. x Note: A basic SOCP branch-and-bound also enumerates at least 2 n integer sols. y IBM Corporation

24 Experimental illustration 2 CPLEX SCIP 2.1 Bonmin B-Hyb n 2 n nodes nodes nodes 10 1,024 2, , ,768 65,535 31, , ,048,576 2,097,151 1,216, CPLEX ran in single threaded mode 2 Three years ago I liked this slide IBM Corporation

25 Experimental illustration 2 Remark CPLEX SCIP 2.1 Bonmin B-Hyb n 2 n nodes nodes nodes 10 1,024 2, , ,768 65,535 31, , ,048,576 2,097,151 1,216, Problem is simple for CPLEX/SCIP if variables are 0 1: replace xi 2 by x i, the contradiction n 4 n 1 4 follows. 1 CPLEX ran in single threaded mode 2 Three years ago I liked this slide IBM Corporation

26 Experimental illustration 2 Remark CPLEX SCIP 2.1 Bonmin B-Hyb n 2 n nodes nodes nodes 10 1,024 2, , ,768 65,535 31, , ,048,576 2,097,151 1,216, Problem is simple for CPLEX/SCIP if variables are 0 1: replace xi 2 by x i, the contradiction n 4 n 1 4 follows. SCIP 3.0 and CPLEX solve it in a blink. 1 CPLEX ran in single threaded mode 2 Three years ago I liked this slide IBM Corporation

27 New in CPLEX Cone disaggregation for MISOCP Lift-and-project cuts for MISOCP Redesigned heuristic choice of most promising algorithm Updated history of MIQCP with CPLEX class p algorithm V. (Year) SOCP 0 barrier 9.0 (2003) MISOCP > 0 barrier based B&B 9.0 (2003) Outer approximation B&C 11.0 (2007) Cone propagations (2014) Major improvements (June 2015) IBM Corporation

28 Separable mixed integer convex programs min c T x s.t. g i (x) 0 i = 1,...,m Ax = b x j Z j = 1,...,p l x u (sminlp) For i = 1,...,m, g i are convex separable: g i (x) = with g ij : [l j,u j ] R convex. n g ij (x j ) j= IBM Corporation

29 Disaggregated formulation Introduce one variable y ij for each elementary function: min s.t. c T x n y ij 0 j=1 g ij (x j ) y ij Ax = b, x i Z l x u. i = 1,...,m, i = 1,...,m, j = 1,...,n, i = 1,...,p, (sminlp ) IBM Corporation

30 Cone disaggregation for MISOCP In standard form the nonlinear constraint describing the second order cone is not convex separable: n xi 2 x0 2 i=1 [Vielma et al., 2015] (go to FC24!), divide the constraint by x 0 0 to get a convex separable constraint: n i=1 Now introduce y 1,...,y n and rewrite as: x 2 i x 0 x 0. n y i x 0 i=1 x 2 i x 0 y i IBM Corporation

31 Extended formulation of (1) Application to (1) [Hijazi et al., 14] z min c T x n s.t. y i (n 1)/4 i=1 (x i 0.5) 2 y i i = 1,...,n x Z n. Its outer approximation min c T x n s.t. y i (n 1)/4 i=1 2 ( x k i 0.5 ) (x i x k i )+ ( x k i 0.5 ) 2 yi i = 1,...,n k = 1,...,K x Z n x y IBM Corporation

32 Extended formulation of (1) min c T x n s.t. y i (n 1)/4 i=1 Application to (1) [Hijazi et al., 14] (x i 0.5) 2 y i i = 1,...,n x Z n. Its outer approximation min c T x n s.t. y i (n 1)/4 i=1 2 ( x k i 0.5 ) (x i x k i )+ ( x k i 0.5 ) 2 yi i = 1,...,n k = 1,...,K x Z n IBM Corporation

33 Extended formulation of (1) Application to (1) [Hijazi et al., 14] min c T x 2 points suffice to make the mixed-integer set infeasible n x 1 = 0 and x 2 = 1: s.t. y i (n 1)/4 x i y i i = 1,...,n i=1 (x i 0.5) 2 x i y y i i = 1,...,n i i = 1,...,n y i 0.25 x Z n x. i Z Its outer approximation min c T x n s.t. y i (n 1)/4 i=1 2 ( x k i 0.5 ) (x i x k i )+ ( x k i 0.5 ) 2 yi i = 1,...,n k = 1,...,K x Z n IBM Corporation

34 Lift-and-project cuts for MISOCP Cuts are an essential component of MILP solvers Can always apply MILP cuts to a linear OA of MISOCP (and we do it) Can we generate better cuts by looking directly at nonlinear functions? A partial answer: as long as the cut generated is linear it could also have been obtained from a linear outer approximation In the past three years, tremendous activity towards conic cuts for conic programming [Andersen and Jensen, 2013, Belotti et al., 2013, Kılınç-Karzan and Yıldız, 2015, Modaresi et al., 2015] (among others) Our goal Derive linear cutting planes Fast Find an appropriate OA from which to derive a cut IBM Corporation

35 Split Relaxation Consider C and M := C (Z p R n p ). Let π Z p {0} n p, π 0 Z and ( C (π,π0) := conv C ({ x : π T } x π 0 { x : π T x π })). α T x = β ˆx C (π,π0) (clearly M C (π,π 0) C). π T x π 0 π T x π IBM Corporation

36 Two approaches from Mixed Integer Convex Programming Goal: build a linear OA from which a "best" cut can be deduced. Use Disjunctive Programming for MILP to derive the cuts. Using only LP [Kılınc et al., 2011]. Using nonlinear programming [Bonami, 2011] 1 Start with any linear OA of C 1 Solve a single NLP that tells if ˆx is in the split relaxation. 2 Solve a Cut Generation LP, if a cut if found STOP. 3 Deduce from dual of CGLP two points proving that ˆx is in the split relaxation. 4 If point(s) not in C generate new OA and goto 2, otherwise there is no cut, STOP. 2 If not, deduce from solution two points such that ˆx = λx 1 +(1 λ)x 0 and closest to the disjunction. 3 Build OA around these two points. 4 Solve MLP and get the cut IBM Corporation

37 Lift-and-project for MISOCP Hybridizing the approaches of [Kılınc et al., 2011] and [Bonami, 2011] From [Kılınc et al., 2011] Pure LP approach, Dynamic generation of additional OA constraints. ˆx From [Bonami, 2011] compact formulation using MLP, penalization of the non-linear constraints. π T x π0 π T x π IBM Corporation

38 Lift-and-project for MISOCP Hybridizing the approaches of [Kılınc et al., 2011] and [Bonami, 2011] From [Kılınc et al., 2011] Pure LP approach, Dynamic generation of additional OA constraints. ˆx From [Bonami, 2011] compact formulation using MLP, penalization of the non-linear constraints. π T x π0 π T x π IBM Corporation

39 Lift-and-project for MISOCP Hybridizing the approaches of [Kılınc et al., 2011] and [Bonami, 2011] From [Kılınc et al., 2011] Pure LP approach, Dynamic generation of additional OA constraints. ˆx x 1 x 0 From [Bonami, 2011] compact formulation using MLP, penalization of the non-linear constraints. π T x π0 π T x π IBM Corporation

40 Lift-and-project for MISOCP Hybridizing the approaches of [Kılınc et al., 2011] and [Bonami, 2011] From [Kılınc et al., 2011] Pure LP approach, Dynamic generation of additional OA constraints. ˆx x 1 x 0 From [Bonami, 2011] compact formulation using MLP, penalization of the non-linear constraints. π T x π0 π T x π IBM Corporation

41 Lift-and-project for MISOCP Hybridizing the approaches of [Kılınc et al., 2011] and [Bonami, 2011] From [Kılınc et al., 2011] Pure LP approach, Dynamic generation of additional OA constraints. ˆx x 1 x 0 From [Bonami, 2011] compact formulation using MLP, penalization of the non-linear constraints. π T x π0 π T x π IBM Corporation

42 How good can this be? z [Cornuéjols and Li, 2001] showed that the empty ball in dimension n has split rank n (also holds for any outer approximation in this space) Practically unsolvable using any form of split cuts (even conic ones). x y Instead the OA of the disaggregated formulation has (simple) split rank IBM Corporation

43 How good can this be? z [Cornuéjols and Li, 2001] showed that the empty ball in dimension n has split rank n (also holds xfor i + any 0.25outer y i i = 1,...,n approximation inx i this space) y i i = 1,...,n Practically unsolvable using any form y i 0.25 x of split cuts (even i 0 OR x conic ones). i 1 x y Instead the OA of the disaggregated formulation has (simple) split rank 1. y i 1 4 x i 0 x i IBM Corporation

44 A more complicated empty ellipse n (100x2i x2i 1 2 4x 2i x 2i 1 98x 2i 98x 2i 1 ) 1 i=1 x Z 2n Results on 12 threads with , and aggressive cuts 3, 3 hours time limit n nodes nodes nodes 5 2,261 2,045 1, ,97, > 23,125,426 7,769 1 (Largest model solved in 2.2 sec by , in 5.5 sec by ) 3 set mip limit cutpasses set mip cuts liftproj 3 set mip strategy miqcpstrat IBM Corporation

45 A more complicated empty ellipse Similar results previously observed by [Kılınç, 2011] Original Extended n root gap sol time root gap sol time Batch n Markowitz (100x2i x2i x> 2i x10 2i x 2i x 2i 1 1) i=1 SLay uflquad x Z 2n SeeResults also related on 12theoretical threads with results , for MILP case and [Boduraggressive et al., 2015] cuts and, 3go hours to FD01. time limit n nodes nodes nodes 5 2,261 2,045 1, ,97, > 23,125,426 7,769 1 (Largest model solved in 2.2 sec by , in 5.5 sec by ) IBM Corporation

46 Remarks Cone disaggregation Automatically applied by default during presolve, Only interesting (and done): 1 If using the OA-BB, 2 For cones that are long enough. Lift-and-project Also only done in OA based algorithm. Still an expensive algorithm (may not speed up every easy model). Also... Redesigned heuristic to choose algorithm to apply in view of these changes IBM Corporation

47 Computational experiments MISOCP Test bed Cplex internal MIQCP test bed: 296 models Benchmarks test set of CBLIB ( 80 models Compare the new release Cplex against Cplex : geo. mean of branch-and-bound computing times. Compare the two algorithmic strategies SOCP-BB and OA-BB in Cplex All tests are carried on Linux machines: Intel 2.67 GHz, 24 GB RAM, 12 threads, deterministic IBM Corporation

48 CPLEX vs > 0 sec 231 Models > 1 sec 149 Models > 0 sec 66 Models > 1 sec 46 Models CPLEX test bed CPLEX : 62 time limits CPLEX : 38 time limits CBLIB CPLEX : 17 time limits CPLEX : 8 time limits IBM Corporation

49 Comparing OA and SOCP BB s in CPLEX % model solved 245 models solved by at least one method and failed by none. Default strategy picked _miqcpstrat_0 1262_miqcpstrat_1 1262_miqcpstrat_ time factor OA-BB 186 times SOCP-BB 4 times 55 models identical with both To be perfect should have picked 2 more models with OA-BB 9 more models with SOCP-BB IBM Corporation

50 Reminder of CPLEX % model solved 225 models solved by at least one method and failed by none. Default strategy picked _miqcpstrat_0 1261_miqcpstrat_1 1261_miqcpstrat_ time factor OA 113 times SOCP-BB 46 times 56 models identical with both To be perfect should have picked 14 more models with OA 36 more models with SOCP-BB IBM Corporation

51 Other CPLEX and CPO related Failure-directed Search for Constraint-based Scheduling P. Vilím, Monday 11:20 (MB25) Symmetry in Linear Programming R. Wunderling, Tuesday 10:20 (TB07) Accelerating the Development of Efficient CP Optimizer Models P. Laborie, Tuesday 14:45 (TD19) Advances in the CPLEX Distributed Solver L. Ladanyi, Thursday 13:10 (ThC15) On Mathematical Programming with Indicator Constraints A. Lodi, Thursday 16:35 (ThE01) Max Clique Cuts for Standard Quadratic Programs J. Schweiger, Friday 10:50 (FB29) Zero-Half Cuts for Solving Nonconvex Quadratic Programs with Box Constraints J. Linderoth, Friday 11:20 (FB29) IBM Corporation

52

53 Legal Disclaimer IBM Corporation All Rights Reserved. The information contained in this publication is provided for informational purposes only. While efforts were made to verify the completeness and accuracy of the information contained in this publication, it is provided AS IS without warranty of any kind, express or implied. In addition, this information is based on IBM s current product plans and strategy, which are subject to change by IBM without notice. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this publication or any other materials. Nothing contained in this publication is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in this presentation may change at any time at IBM s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results. Performance is based on measurements and projections using standard IBM benchmarks in a controlled environment. The actual throughput or performance that any user will experience will vary depending upon many factors, including considerations such as the amount of multiprogramming in the user s job stream, the I/O configuration, the storage configuration, and the workload processed. Therefore, no assurance can be given that an individual user will achieve results similar to those stated here. IBM, the IBM logo, CPLEX and SPSS are trademarks of International Business Machines Corporation in the United States, other countries, or both. Intel, Intel Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other countries. Linux is a registered trademark of Linus Torvalds in the United States, other countries, or both. Other company, product, or service names may be trademarks or service marks of others IBM Corporation

54 References I K. Andersen and A. Jensen. Intersection cuts for mixed integer conic quadratic sets. In M. Goemans and J. Correa, editors, Integer Programming and Combinatorial Optimization, volume 7801 of Lecture Notes in Computer Science, pages Springer Berlin Heidelberg, ISBN doi: / _4. URL P. Belotti, J. C. Góez, I. Pólik, T. K. Ralphs, and T. Terlaky. On families of quadratic surfaces having fixed intersections with two hyperplanes. Discrete Applied Mathematics, 161(16 17): , ISSN X. doi: URL M. Bodur, S. Dash, and O. Günlük. Cutting planes from extended lp formulations. Technical report, P. Bonami. Lift-and-project cuts for mixed integer convex programs. In IPCO, pages 52 64, G. Cornuéjols and Y. Li. On the rank of mixed 0,1 polyhedra. In K. Aardal and B. Gerards, editors, Integer Programming and Combinatorial Optimization, volume 2081 of Lecture Notes in Computer Science, pages Springer Berlin Heidelberg, ISBN doi: / _6. URL S. Drewes. Mixed Integer Second Order Cone Programming. PhD thesis, Technische Universität Darmstadt, S. Drewes and S. Ulbrich. Subgradient based outer approximation for mixed integer second order cone programming. In J. Lee and S. Leyffer, editors, Mixed Integer Nonlinear Programming, volume 154 of The IMA Volumes in Mathematics and its Applications, pages Springer New York, ISBN doi: / _2. URL M. A. Duran and I. Grossmann. An outer-approximation algorithm for a class of mixed-integer nonlinear programs. Mathematical Programming, 36: , H. Hijazi, P. Bonami, and A. Ouorou. An outer-inner approximation for separable mixed-integer nonlinear programs. INFORMS Journal on Computing, 26(1):null, 14. doi: /ijoc M. Kılınc, J. Linderoth, and J. Luedtke. Effective separation of disjunctive cuts for convex mixed integer nonlinear programs. Technical Report 1681, M. R. Kılınç. Disjunctive Cutting Planes ann Algorithms for Convex Mixed Integer Nonlinear Programming. PhD thesis, University of Wisconsin-Madison, IBM Corporation

55 References II F. Kılınç-Karzan and S. Yıldız. Two term disjunctions on the second-order cone. Mathematical Programming, April S. Modaresi, M. Kılınç, and J. Vielma. Intersection cuts for nonlinear integer programming: convexification techniques for structured sets. Mathematical Programming, pages 1 37, ISSN doi: /s URL I. Quesada and I. E. Grossmann. An LP/NLP based branch and bound algorithm for convex MINLP optimization problems. Computers and Chemical Engineering, 16: , J. P. Vielma, I. Dunning, J. Huchette, and M. Lubin. Extended Formulations in Mixed Integer Conic Quadratic Programming. Research Report, IBM Corporation

56