Mixed Integer Non Linear Programming

Transcription

1 Mixed Integer Non Linear Programming Claudia D Ambrosio CNRS Research Scientist CNRS & LIX, École Polytechnique MPRO PMA

2 Outline What is a MINLP? Dealing with nonconvexities Global Optimization methods Spatial Branch-and-Bound Expression trees Convex relaxation Variable ranges Bounds tightening Practical Tools MINLP Solvers Neos MINLP Libraries

3 Recap: What is a MINLP? Mixed Integer NonLinear Programming (MINLP). min f (x) g(x) 0 x j N j J {1,..., n} with f (x) : R n R and g(x) : R n R m are * continuous * twice differentiable functions. Local optima are not always global optima.

4 Recap: Continuous (NLP) relaxation min f (x) g(x) 0 Hard to solve in general (might be undecidable in theory)!

5 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs.

6 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. 0 LB = 30

7 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. y 1 = 1 LB = 30 0 y 1 = 0

8 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. LB = 35 LB = 30 0 y 1 = 1 1 y 1 = 0

9 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. LB = 30 0 y 1 = 1 1 LB = 35 y 2 = 1 y 2 = 0 y 1 = 0

10 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. LB = 30 0 y 1 = 1 LB = 35 1 y 2 = 1 y 2 = 0 2 z = 26!!!!!! y 1 = 0

11 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. LB = 30 0 y 1 = 1 LB = 35 1 y 2 = 1 y 2 = z = 26!!!!!! y 1 = 0

12 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. LB = 30 0 y 1 = 1 y 1 = 0 LB = y 2 = 1 y 2 = z = 26!!!!!!

13 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. LB = 30 0 y 1 = 1 y 1 = 0 LB = y 2 = 1 y 2 = z = 26!!!!!! 0 LB = 30

14 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. LB = 30 0 y 1 = 1 y 1 = 0 LB = y 2 = 1 y 2 = z = 26!!!!!! y 1 = 0 LB = 30 0 y 1 = 1

15 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. LB = 30 0 y 1 = 1 y 1 = 0 LB = y 2 = 1 y 2 = z = 26!!!!!! LB = 30 0 y 1 = 0 1 z = 31 y 1 = 1

16 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. LB = 30 0 y 1 = 1 y 1 = 0 LB = y 2 = 1 y 2 = z = 26!!!!!! LB = 30 0 y 1 = 0 y 1 = z = 31 LB = 35

17 MINLP branch-and-bound with local NLP solver Branch-and-bound algorithm: solve continuous (NLP) relaxation at each node of the search tree and branch on variables. NLP solver used: Local NLP solvers local optimum No valid bound for nonconvex MINLPs. LB = 30 0 y 1 = 1 y 1 = 0 LB = y 2 = 1 y 2 = z = 26!!!!!! LB = 30 0 y 1 = 0 y 1 = z = 31 LB = 35

18 Outer Approximation and nonconvex MINLPs Several methods for convex MINLPs use Outer Approximation cuts (Duran and Grossman, 1986) which are not exact for nonconvex MINLPs. g i (x) 0 g i (x k ) + g i (x k ) T ( x x k ) 0 where g(x k ) is the Jacobian of g(x) evaluated at point (x k ).

19 Outline What is a MINLP? Dealing with nonconvexities Global Optimization methods Spatial Branch-and-Bound Expression trees Convex relaxation Variable ranges Bounds tightening Practical Tools MINLP Solvers Neos MINLP Libraries

20 Global Optimization methods Exact Exact in continuous space: ε-approximate (find solution within pre-determined ε distance from optimum in obj. fun. value) Heuristic Find solution with probability 1 in infinite time For some problems, finite convergence to optimum (ε = 0)

21 Outline What is a MINLP? Dealing with nonconvexities Global Optimization methods Spatial Branch-and-Bound Expression trees Convex relaxation Variable ranges Bounds tightening Practical Tools MINLP Solvers Neos MINLP Libraries

22 Spatial Branch-and-Bound Falk and Soland (1969) An algorithm for separable nonconvex programming problems. 20 years ago: first general-purpose exact algorithms for nonconvex MINLP. Tree-like search Explores search space exhaustively but implicitly Builds a sequence of decreasing upper bounds and increasing lower bounds to the global optimum Exponential worst-case Only general-purpose exact algorithm for MINLP Since continuous vars are involved, should say ε-approximate Like BB for MILP, but may branch on continuous vars Done whenever one is involved in a nonconvex term

23 Spatial B&B: Example Original problem P

24 Spatial B&B: Example Starting point x

25 Spatial B&B: Example Local (upper bounding) solution x

26 Spatial B&B: Example Convex relaxation (lower) bound f with f f > ε

27 Spatial B&B: Example Branch at x = x into C 1, C 2

28 Spatial B&B: Example Convex relaxation on C 1 : lower bounding solution x

29 Spatial B&B: Example localsolve. from x: new upper bounding solution x

30 Spatial B&B: Example f f > ε: branch at x = x

31 Spatial B&B: Example Repeat on C 3 : get x = x and f f < ε, no more branching

32 Spatial B&B: Example Repeat on C 2 : f > f (can t improve x in C 2 )

33 Spatial B&B: Example Repeat on C 4 : f > f (can t improve x in C 4 )

34 Spatial B&B: Example No more subproblems left, return x and terminate

35 Spatial B&B: Pruning 1. P was branched into C 1, C 2 2. C 1 was branched into C 3, C 4 3. C 3 was pruned by optimality (x G(C 3 ) was found) 4. C 2, C 4 were pruned by bound (lower bound for C 2 worse than f ) 5. No more nodes: whole space explored, x G(P) Search generates a tree Suproblems are nodes Nodes can be pruned by optimality, bound or infeasibility (when subproblem is infeasible) Otherwise, they are branched

36 Spatial B&B: A recursive version processsubproblem ε(c): 1: if isfeasible(c) then 2: x = globalopt(c) 3: if x then 4: if f P (x ) < f P (x ) then 5: update x x // improvement 6: end if 7: else 8: if lowerbound(c) < f P (x ) ε then 9: Split [x L, x U ] into two hyperrectangles [x L, x], [x, x U ] 10: processsubproblem ε(c[x L, x]) 11: processsubproblem ε(c[x, x U ]) 12: end if 13: end if 14: end if

37 Spatial B&B: Bad news 1. If globalopt(c) works on any problem, why not call globalopt(p) and be done with it? 2. For arbitrary C, isfeasible(c) is undecidable 3. How do we compute lowerbound(c)?

38 Spatial B&B: Upper bounds Upper bounds: x can only decrease Computing the global optima for each subproblem yields candidates for updating x As long as we only update x when x improves it, we don t need x to be a global optimum Any good feasible point will do Specifically, use feasible local optima Replace globalopt() by localsolve()

39 Spatial B&B: Lower bound Lower bounds: increase over -chains Let R P be a relaxation of P such that: 1. R P also involves the decision variables of P (and perhaps some others) 2. for any range I = [x L, x U ], R P [I] is a relaxation of P[I] 3. if I, I are two ranges I I min R P [I] min R P [I ] 4. For any subproblem C of P, finding x G(R C ) or showing F(R C ) = is efficient Specifically, x = localsolve(r C ) G(R C ) Define lowerbound(c) = f RC ( x)

40 Spatial B&B: A decidable feasibility test Processing C when it s infeasible will make sbb slower but not incorrect sbb still works if we simply never discard a potentially feasible C Use a partial feasibility test isevidentlyinfeasible(p) If isevidentlyinfeasible(c) is true, then C is guaranteed to be infeasible, and we can discard it Otherwise, we simply don t know, and we shall process it Thm: If R C is infeasible then C is infeasible Proof: = F(R C ) F(C) = { true if localsolve(rc ) = isevidentlyinfeasible(c) = false otherwise

41 Tricks To make an sbb work efficiently, you need further tricks

42 Expression trees Representation of objective f and constraints g Encode mathematical expressions in trees or DAGs E.g. x x 1x 2 : x 1 + ˆ 2 x 1 x 2

43 Expression trees Representation of objective f and constraints g Encode mathematical expressions in trees or DAGs E.g. x x 1x 2 : + ˆ x 1 2 x 2

44 Standard form Identify all nonlinear terms x i x j, replace them with a linearizing variable w ij Add a defining constraint w ij = x i x j to the formulation Standard form: x1 2 + x 1x 2 min c (x, w) s.t. A(x, w) b w ij = x i ij x j for suitable i, j bounds & integrality constraints w 11 + w 12 w 11 = x1 2 : w 12 = x 1 x 2

45 Convex relaxation Standard form: all nonlinearities in defining constraints Each defining constraint w ij = x i x j is replaced by two convex inequalities: w ij overestimator(x i x j ) w ij underestimator(x i x j ) Convex relaxation is not the tightest possible, but it can be constructed automatically

46 Summary ORIGINAL MINLP min x f (x) g(x) 0 x L x x U STANDARD FORM min w 1 Aw = b w i = φ i (x, w) i w L w w U CONVEX RELAXATION min w 1 Aw = b relax def constr w i i w L w w U Some variables may be integral Easier to perform symbolic algorithms Linearizes nonlinear terms Adds linearizing variables and defining constraints Each defining constraint replaced by convex under- and concave over-estimators

47 Variable ranges Crucial property for sbb convergence: convex relaxation tightens as variable range widths decrease convex/concave under/over-estimator constraints are (convex) functions of x L, x U it makes sense to tighten x L, x U at the sbb root node (trading off speed for efficiency) and at each other node (trading off efficiency for speed)

48 Bounds Tightening In sbb we need to tighten variable bounds at each node Two methods: Optimization Based Bounds Tightening (OBBT) Feasibility Based Bounds Tightening (FBBT) OBBT: for each variable x in P compute x = min{x conv. rel. constr.} x = max{x conv. rel. constr.} Set x x x

49 Bounds Tightening In sbb we need to tighten variable bounds at each node Two methods: Optimization Based Bounds Tightening (OBBT) Feasibility Based Bounds Tightening (FBBT) FBBT: propagation of intervals up and down constraint expression trees, with tightening at the root node Example: 5x 1 x 2 = 0.

50 Bounds Tightening In sbb we need to tighten variable bounds at each node Two methods: Optimization Based Bounds Tightening (OBBT) Feasibility Based Bounds Tightening (FBBT) FBBT: propagation of intervals up and down constraint expression trees, with tightening at the root node Example: 5x 1 x 2 = 0. Up: :[5, 5] [0, 1] = [0, 5]; :[0, 5] [0, 1] = [ 1, 5]. Root node tightening: [ 1, 5] [0, 0] = [0, 0]. Downwards: :[0, 0]+[0, 1] = [0, 1]; x 1 :[0, 1]/[5, 5] = [0, 1 5 ]

51 RLT: Quadratic problems All nonlinear terms are quadratic monomials Aim to reduce gap betwen the problem and its convex relaxation replace quadratic terms with suitable linear constraints (fewer nonlinear terms to relax) Can be obtained by considering linear relations (called reduced RLT constraints) between original and linearizing variables

52 RLT: Quadratic problems For each k n, let w k = (w k1,..., w kn ) Multiply Ax = b by each x k, substitute linearizing variables w k, get reduced RLT constraint system (RRCS) k n (Aw k = bx k ) i, k n define z ki = w ki x i x k, let z k = (z k1,..., z kn ) Substitute b = Ax in RRCS, get k n(a(w k x k x) = 0), i.e. k n(az k = 0). Let B, N be the sets of basic and nonbasic variables of this system Setting z ki = 0 for each nonbasic variable implies that the RRCS is satisfied It suffices to enforce quadratic constraints w ki = x i x k for (i, k) N (replace those for (i, k) B with the linear RRCS)

53 Example: the pooling problem Q-formulation x ij + x il C i, j J i l L i x il C l, i I l x ij + x lj C j, i I j l L j x il q il x lj = 0 j J l q il = 1 i I l ( ) λ ki x ij + x lj λ ki q il α kj x ij, i I j l L j i I l i I j L j i I l L j J i I, l L i l L k K, j J

54 Example: the pooling problem PQ-formulation by Sahinidis and Tawarmalani (2005). Like Q-formulation but with extra (redundant) constraints: x lj i I l q il = x lj l L, j J l q il j J l x lj C l q il i I, l L i

55 Example: the pooling problem PQ-formulation by Sahinidis and Tawarmalani (2005). Like Q-formulation but with extra (redundant) constraints: x lj i I l q il = x lj l L, j J l q il j J l x lj C l q il i I, l L i Strongest known formulation!

56 Citations Sherali, Alameddine, A new reformulation-linearization technique for bilinear programming problems, JOGO, 1991 Falk, Soland, An algorithm for separable nonconvex programming problems, Manag. Sci Horst, Tuy, Global Optimization, Springer Ryoo, Sahinidis, Global optimization of nonconvex NLPs and MINLPs with applications in process design, Comp. Chem. Eng Adjiman, Floudas et al., A global optimization method, αbb, for general twice-differentiable nonconvex NLPs, Comp. Chem. Eng Smith, Pantelides, A symbolic reformulation/spatial branch-and-bound algorithm for the global optimisation of nonconvex MINLPs, Comp. Chem. Eng Nowak, Relaxation and decomposition methods for Mixed Integer Nonlinear Programming, Birkhäuser, Belotti, et al., Branching and bounds tightening techniques for nonconvex MINLP, Opt. Meth. Softw., Vigerske, PhD Thesis: Decomposition of Multistage Stochastic Programs and a Constraint Integer Programming Approach to Mixed-Integer Nonlinear Programming, Humboldt-University Berlin, 2013.

57 Methods for convex MINLPs Simple BB [Dakin, 1965] Outer approximation [Grossmann, 1986] Generalized Benders Decomposition [Geoffrion, 1972] Quesada&Grossmann, 1992 Extended Cutting Plane [Westerlund&Pettersson, 1995] Hybrid algorithms Software: ALPHA-ECP, BONMIN, DICOPT, FILMINT, LAGO, MINLPBB, MINOPT, MINOTAUR, SBB, SCIP,... D Ambrosio & Lodi, Mixed integer nonlinear programming tools: an updated practical overview, Annals of Operations Research, 2013.

58 Outline What is a MINLP? Dealing with nonconvexities Global Optimization methods Spatial Branch-and-Bound Expression trees Convex relaxation Variable ranges Bounds tightening Practical Tools MINLP Solvers Neos MINLP Libraries

59 MINLP Solvers Convex MINLP ALPHA-ECP: BONMIN: DICOPT: FilMINT: LAGO: MINLPBB: www-unix.mcs.anl.gov/~leyffer/solvers.htm SBB: Non-convex MINLP ANTIGONE: BARON: COUENNE: LINGO/LINDOGlobal: MINOPT: MINOTAUR: SCIP: Need to evaluate function, its first and its second derivative at (x, y ). Possible source of errors! Solution? Modeling Languages!

60 Neos NEOS:

61 MINLP Libraries CMU/IBM: 23 different kind of MINLP problems MacMINLP: 51 instances MINLPlib: 270 instances