On mathematical programming with indicator constraints

Transcription

1 On mathematical programming with indicator constraints Andrea Lodi joint work with P. Bonami & A. Tramontani (IBM), S. Wiese (Unibo) University of Bologna, Italy École Polytechnique de Montréal, Québec, Canada International Symposium on Mathematical Programming (ISMP 2015) Pittsburgh, July 16, 2015 Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

2 Introduction & Motivation We consider Mathematical Optimization problems of the form min f (x, z) subject to h(x, z) 0 [z k = 0] [x S0 k] [z k = 1] [x S1 k] x R m z {0, 1} K. k = 1,..., K (P 0 ) The variable z k indicates whether or not the constraint sets S k 0 and Sk 1 are activated. Usually, these sets are defined as the intersection of level sets of given functions S = {x R m g i (x) 0, i = 1,..., l}. We deal with indicator constraints, allowed in Mathematical Programming software since the release of IBM-ILOG CPLEX 10. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

3 Introduction & Motivation (cont.d) Written as a Mathematical Programming problem (and slightly more general), (P 0 ) becomes min f (x, z) subject to h(x, z) 0 (x, z k ) Γ k j k = 1,..., K (P 1 ) j J x R m z J K, where Γ k j = S k j {j} and S k j R m j, k. This highlights the connection to the paradigm of Generalized Disjunctive Programming. In Mixed-Integer Programming there are (at least) two ways of dealing with the disjunctions associated with indicator constraints. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

4 The so-called bigm approach A very straightforward way to model an indicator constraint is g i (x) M i (1 z i ), where M i is a (very) large positive constant. By defining with F the feasible set for x, then M i sup x F g i (x). Although this quantity might not be easily (or not even at all) computable, setting a reasonably high value of M i will usually do the job in practice. However, two issues of this modeling trick are very well known: Weak continuous relaxations, and Severe numerical difficulties for both MILP & MINLP solvers. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

5 A change of perspective! In order to describe (x, z k ) Γ k j j J k = 1,..., K of (P 1 ), so as to be able of using the disjunctive programming approach for indicator constraints, we need the definition of the so-called perspective function. Definition 1 For a given closed convex function g R m R { }, the perspective function g R m+1 R { } is defined as λ g ( g(x, x λ) = λ ), λ > 0,, λ 0. (1) Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

6 Disjunctive convex optimization The perspective function is instrumental to extend the seminal work of Balas on the union of polyhedra to the study of the union of convex bodies. Namely, this extension has been studied by Ceria & Soares, whose main theorem is Theorem 1 (Ceria & Soares, 1999) Let C j = {x R m g j,i (x) 0, i = 1,..., l j }, for j J, and assume each g j,i R m R is a closed convex function. Then, conv ( j J C j ) = proj x cl(c), where (x, x,..., x J, λ 1,..., λ J ) R ( J +1)m+ J, x = j J x j, C = g j,i (x j, λ j ) 0, i = 1,..., l j, j J,. j J λ j = 1, λ j 0, j J Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

7 Disjunctive convex optimization (cont.d) The construction of Theorem 1 begins in a straightforward manner by creating copies x j of the initial variable x, each one constrained to lie in one of the disjunctive sets C j. Then, the original x is a convex combination of these copies with weights λ j. In this way, the resulting bilinear terms λ j x j lead to a set description of the convex hull that is, however, nonconvex. A simple transformation is then used to obtain the convex set C containing the perspective function. Although now the weakness of the continuous relaxation of the bigm approach seems solved by the strength of taking the convex hull in Theorem 1, Ceria & Soares result leaves a couple of highly nontrivial computational issues on the table. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

8 Disjunctive convex optimization: computational issues First, the closure of the perspective function has no algebraic representation in general. In other words, the perspective function becomes non-differentiable for λ j 0. The resulting numerical difficulties were already noticed in the process of designing a branch-and-cut algorithm for general 0 1 mixed convex programming problems, based on lift-and-project cuts separated through an alternative version of Theorem 1. [Stubbs & Mehrotra (1999)] It is worth noting that for more than ten years there was no significant progress in the separation of lift-and-project cuts for general mixed-integer convex programming problems. Recently, two major steps have been made to circumvent non-differentiability algorithmically through the solution of some (potentially many) easier optimization problems. [Bonami (2011) and Kılınç, Linderoth & Luedtke (2011)] Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

9 Issue #2: Lifting up and projecting down again The second difficulty associated with the practical use of Theorem 1 is the fact that the initial system is lifted into a space with a multiple dimension, i.e., C is defined in R ( J +1)m+ J, which makes the optimization over it computationally challenging. Although the representation of the convex hull is theoretically compact in the number of variables, on the practical side the size of the resulting NLPs had a significant impact on limiting the use of this disjunctive representation both in the separation of L&P cuts for convex MINLPs and, in general, for formulating MP problems with indicator constraints. However, in the context of indicator constraints, projecting out additional variables is possible in several special cases, as we will show in the following. (It is worth noting that working on the original space of variables had a huge impact also on L&P cuts in the linear case.) [Balas & Perregaard (2002) and Bonami (2012)] Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

10 Working on the original space of variables As mentioned working in the original space of variables is of fundamental importance in practice (and of mathematical fun). In the following we review some existing results in this area and we discuss new ones. Literature results are for the case of single disjunctions. Constraint vs nucleus On/off constraint On/off linear constraint The new results are instead for the case of pairs of related disjunctions. Complementary disjunctions Almost complementary disjunctions Ternary disjunctions Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

11 Single disjunctions: constraint vs. nucleus In the notation of (P 1 ) this special case can be written (dropping index k) as S 0 = {x R n x = 0}, and S 1 = {x R n l 1 x u 1, g 1,i (x) 0, i = 1,..., l}, and translates to Theorem 2 (Günlük & Linderoth, 2012) Let J = {0, 1} and for j J define Γ j = S j {j}, with S j specified as above and non-empty. We then have that if Γ 1 is a convex set, conv(γ 0 Γ 1 ) = cl(γ), where (x, z) R n+1, g 1,i (x, z) 0, i = 1,..., l, Γ =. zl 1 x zu 1, 0 < z 1 Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

12 Constraint vs. nucleus: example Example S 0 = {x R n x = 0}, and S 1 = {x R n l 1 x u 1, g 1,i (x) 0, i = 1,..., l}, let l 0 = l 1 = 0 R 2 and u 0 = u 1 = 1 R 2, and g 1 (x 1, x 2 ) = x 2 + 2(x ) 3 5. x 2 x 2 z x 1 x 1 Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

13 Single disjunctions: constraint vs. nucleus (cont.d) Note that in Theorem 2 the indicator variable z of the initial implication (or disjunction) takes over the role of the weight or multiplier inside the perspective function g i. Theorem 2 can also be formulated for the situation in which on the zero side, the variables are not forced to zero but to an arbitrary single point. Theorem 2 is also extendable to the situation in which on the zero side the set S 0 is not a single point, but a ray. Moreover, if the form of the functions g 1,i is known in more detail, in some cases one can avoid the need to take the closure and thus the differentiability issue, for example for polynomial functions. [Günlük & Linderoth (2012)] The remarkable computational advantages of Theorem 2 are, for example, for uncapacitated facility location with quadratic costs and unit commitment problems. [Günlük & Linderoth (2012) and Frangioni & Gentile (2006)] Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

14 Single disjunctions: on/off constraint Consider the slightly more general case where the set S 0 is not a point but a box. Namely, S 0 = {x R n l 0 x u 0 }, and S 1 = {x R n l 1 x u 1, g 1 (x) 0}. Theorem 3 (Hijazi et al., 2012) Let J = {0, 1} and for j J define Γ j = S j {j}, with S j specified as above and non-empty. We then have that if g 1 is isotone, conv(γ 0 Γ 1 ) = cl(γ ), where (x, z) R n+1, Γ zq 1,0 = I ( x, z) 0, I N, z zl 1 + (1 z)l 0 x zu 1 + (1 z)u 0, 0 < z 1. The statement of the theorem and the definition of Γ require the definition of an isotone function and its perspective q j,j I = g j h j,j I. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

15 A variant of the perspective for isotone functions Definition 2 A function g R n R is called independently non-decreasing (resp. non-increasing) in the i-th coordinate, if x dom(g) and λ > 0, we have g(x + λe i ) g(x) (resp. g(x + λe i ) g(x)). We say that g is independently monotone in the i-th coordinate, if it is either independently non-decreasing or non-increasing. Finally, g is called isotone, if it is independently monotone in every coordinate. For an isotone function g let J 1 (g) (resp. J 2 (g)) be the set of indices in which g is independently non-decreasing (resp. non-increasing), I N = {1,..., n}, x R n, z > 0 and j, j J (l j ) i i I J 1 (g j ), (u j ) i i I J 2 (g j ), I ) (x, z) = i x i (1 z)(u j ) i i z Ī J 1(g j ), x i (1 z)(l j ) i i z Ī J 2(g j ), (h j,j Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

16 On/off constraint: example Let l 0 = l 1 = 0 R 2 and u 0 = u 1 = 1 R 2, and consider the function g 0 (x 1, x 2 ) = exp (2x ) + x 2 1. x 2 x 2 z x 1 x 1 Formulating the delay-constrained routing problem by means of Theorem 3 leads to a significant computational advantage over a straightforward bigm formulation. [Hijazi et al. (2012)] Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

17 Theorem 3 vs Theorem 1: Trade-off On the one side, the advantage of Theorem 3 is the fact that we project back onto the original space of variables. In fact, we can express conv(γ 0 Γ 1 ) = cl(γ ) as a subset of a (n + 1)-dimensional space. A direct application of Theorem 1 would lead to expressing the convex hull as the projection of a subset of a (3n + 5)-dimensional space. On the other side, this gain does not come for free. In Theorem 3, we need exponentially many constraints. In particular, including all the simple bound constraints, we have 2 n + 2n + 2 constraints opposed to 4n + 9 that would result from a direct application of Theorem 1. In essence, an upper bound on the number of constraints needed to describe the convex hull is always exponential, although only in the number of variables involved in the constraint. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

18 Single disjunctions: on/off linear constraint In the special case that g 1 is an affine function there is no need of the closure. Corollary 1 (Hijazi, 2010) Consider the situation of Theorem 3. If g 1 is affine, conv(γ 0 Γ 1 ) = Γ, where (x, z) R n+1, Γ H 1,0 = I (x, z) 0, I N, zl 1 + (1 z)l 0 x zu 1 + (1 z)u 0, 0 z 1. where H j,j I (x, z) = i Ī a j,i x i + z a j,0 + (1 z) i I,a j,i >0 i Ī,a j,i >0 for any g j (x) = a j,0 + n i=1 a j,i x i, j J and I N. a j,i (l j ) i + a j,i (u j ) i + i I,a j,i <0 i Ī,a j,i <0 a j,i (u j ) i a j,i (l j ) i. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

19 On/off linear constraint: computation Supervised classification: consider a set Ω of m objects, where each object i Ω is characterized by the vector x i R d and associated with one of two classes, say, y i { 1, 1}. The task is to find a hyperplane ω T x b in R d that separates the two classes by maximizing a confidence margin. z i = 1 y i (ω T x i + b) 1 + ξ i 0 i = 1,..., m prep instance bigm lp CH lp P e bigm P e CH P e bigm P e CH weak... mean strong 2nl nl nl nl mean Trade-off: we have m vs m 2 d+2 constraints. However, many are redundant. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

20 Pairs of related disjunctions: complementary case We move on to two indicator constraints and therefore consider the sets S 0 = {x R n l 0 x u 0 }, S 1 = {x R n l 1 x u 1, g 1 (x) 0}, S 0 = {x R n l 0 x u 0, g 0 (x) 0}, S 1 = {x R n l 1 x u 1 }. We require that the two indicator constraints are complementary, i.e., the activation of one implies the deactivation of the other and vice versa. Because the two disjunctions above are complementary, i.e., z = 1 z, the whole situation could be described by a single disjunction with a single indicator variable, that is, as a subset of R n+1. For example, (x, z) ( S 0 {0}) (S 1 {1}). Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

21 Complementary disjunctions If the index sets of the two functions decompose in an opposite way, i.e., J 2 (g 1 ) = J 1 (g 0 ), then the above inequalities still give the convex hull. Theorem 4 (Bonami, Lodi, Tramontani & Wiese, 2015) Let Γ 0 = S 0 {0} and Γ 1 = S 1 {1}, with S 0 and S 1 specified as above and non-empty. Thus, if J 2 (g 1 ) = J 1 (g 0 ), then conv( Γ 0 Γ 1 ) = cl( Γ), where (x, z) R n+1, zq 1,0 I ( x, z) 0, z Γ = (1 z)q 0,1 I N, I ( x, 1 z) 0, 1 z zl 1 + (1 z)l 0 x zu 1 + (1 z)u 0, 0 < z < 1. In a sense this is a surprising result: conv ((Γ 0 Γ 1 ) ( Γ0 Γ 1 ) L = ) = conv (Γ 0 Γ 1 ) conv ( Γ0 Γ 1 ) L =, where L = = R n H = = {(z, z) z + z = 1}. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

22 Complementary disjunctions: example To the previous example add the function g 1 (x 1, x 2 ) = max { 1 2 x 1, 1 2 x 2}. x 2 x 2 z x 1 x 1 x 2 x 2 x 1 x 1 Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

23 Is J 2 (g 1 ) = J 1 (g 0 ) needed? Counterexample Let l 0 = l 1 = 0 and u 0 = u 1 = 1, H 0 (x) = x 1 2 and H1 (x) = x 3, both of which 4 are non-decreasing in their first and only coordinate. x x x z z z Open question: How to compute the convex hull in this case? Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

24 Complementary disjunctions: computation Job shop scheduling: we are given n jobs, J 1,..., J n, that have to be scheduled without preemption on m machines, M 1,..., M m. Because two jobs, say i, j, cannot overlap on a machine, say k, then x k ij = 1 s kj s ki + p ki x k ij = 0 s ki s kj + p kj (i, j, k) i < j instance bigm lp CH lp... P e bigm P e CH Pe bigm Pe CH la01-10x la03-10x la08-15x mean Trade-off: for each triple (i, j, k) we have 2 vs 8 constraints (reduced to 6). Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

25 Pairs of related disjunctions: almost complementary Instead of z + z = 1 as for complementary indicators, we could also consider z + z 1, leading to the three sets S 0 = {x R n l 0 x u 0 }, S 1 = {x R n l 1 x u 1, g 1 (x) 0}, and S 1 = {x R n l 1 x u 1, g 1 (x) 0}. In this case, we can compute a superset of the convex hull of the disjunction. We need a bit of notation: and for j J = {0, 1} and H = {(z, z) z + z 1} and L = R n H, Γ j = S j {j} [0, 1] and Γ j = S j [0, 1] {j}, (x, z, z) (Γ 0 Γ 1 ) ( Γ0 Γ 1 ) L. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

26 Almost complementary disjunctions With that we get the following Corollary: Corollary 2 (Bonami, Lodi, Tramontani & Wiese 2015) Let S 0, S 1 and S 1 be specified as above and non-empty. Thus, we have that if g 1 and g 1 are isotone functions, then conv ((Γ 0 Γ 1 ) ( Γ0 Γ 1 ) L ) conv (Γ 0 Γ 1 ) conv ( Γ0 Γ 1 ) L = cl( Γ), where Γ = (x, z, z) R n+2, zq 1,0 I ( x z, z) 0 zq 1,0 I ( x z, z) 0 0 < z, z < 1, z + z 1 I N, zl 1 + (1 z)l 0 x zu 1 + (1 z)u 0, zl 1 + (1 z)l 0 x zu 1 + (1 z)u 0,. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

27 Ternary disjunctions Almost complementary disjunctions could also be modeled by a single indicator variable assuming the values { 1, 0, 1}, i.e., ternary disjunctions. Natural questions are 1 is it possible to write the counterpart of Theorem 3 for this case? 2 which is the relationship between these two ways of representing essentially the same disjunction? Question 1 has a positive answer but only in the special case of affine functions because we are unable to define the perspective function for negative values of z. (The convexity-preserving property is otherwise lost.) The answer to Question 2 is also somehow surprising. We can show that, at least for affine functions, the convex hull of the almost complementary disjunction is contained in the convex hull of the ternary disjunction (when transforming everything into a common space). Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

28 Almost complementary disjunctions: computation Traveling salesman problem with time windows: find a minimum-cost tour visiting a set of cities exactly once, where each city i must be visited within a given time window [a i, b i ]. Binary variable x ij denotes if city i is visited in a tour immediately before city j, and activates the temporal constraint x ij = 1 s j s i + t ij (i, j) A instance bigmz lp CHz lp bigmx lp CHx lp P e bigmx P e CHx... rbg016a rbg040a rbg rbg201a *1.45 rbg *1.77 * mean Trade-off: for each pair (i, j) we have 2 versus 8 constraints (reduced to 6). Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29

29 Computational facts, hopes and conclusions The computation is mildly promising although not conclusive because there is no instance that the general-purpose solver IBM-ILOG CPLEX can solve with the perspective formulation and cannot with the bigm one. However, by taking a closer look at the inequalities in the previous slides one realizes that, in the linear case, they have the structure of bigm constraints, and, in any case, they depend on the bounds on the involved variables. This opens the possibility of extensively using bound tightening in the nodes of the branch-and-bound tree where the perspective constraints can be replaced by locally-valid strengthened versions. This is what has been proposed to effectively solve supervised classification models and is now part of IBM-ILOG CPLEX algorithmic arsenal. [Belotti et al. (2014) and IBM-ILOG CPLEX (v )] We believe/hope that combining disjunctive formulations with aggressive cut generation and bound reductions within the tree could lead to effective algorithms for hard MINLPs. Andrea Lodi (Bologna U & Polytechnique Montréal) On MP with indicator constraints ISMP / 29