Integer Programming, Constraint Programming, and their Combination Alexander Bockmayr Freie Universität Berlin & DFG Research Center Matheon Eindhoven, 27 January 2006
Discrete Optimization General framework Discrete variables x 1,..., x n, with x i D i Z Constraints c 1,..., c m D 1 D n constraint satisfaction Objective function f : D 1 D n R optimization Application areas Operations Research : Planning, scheduling, transportation,... Computational biology : Alignment, folding,... Generic solution approaches Integer linear programming IP Finite domain constraint programming CP 1
Outline 1. IP vs. CP Model building language Model solving algorithms 2. An illustrating example Discrete tomography IP vs. CP models 3. Combining IP and CP Cut generation for IP using CP SCIL, SCIP,... 2
1. Integer vs. Constraint Programming 3
IP vs. CP : Language IP CP Variables 0-1 Finite domain Constraints Linear equations Arithmetic constraints and inequalities Symbolic/global constraints Example Variables : x 1,..., x n {0,..., m 1} Constraint : Pairwise different values 4
Integer programming: Only linear equations and inequalities x i x j x i < x j x i > x j x i x j 1 x i x j + 1 Eliminating disjunction x i x j + 1 my i, x j x i + 1 my j, y i + y j = 1, y i, y j {0, 1}, 0 x i, x j m 1, New variables: z ik = 1 iff x i = k, i = 1,..., n, k = 0,..., m 1 z i0 + + z im 1 = 1, z 1k + + z nk 1, Constraint programming symbolic constraint alldifferent(x 1,..., x n ) 5
Symbolic Constraints alldifferent([x 1,..., x n ]) cumulative([s 1,..., s n ], [d 1,..., d n ], [r 1,..., r n ], c, e). n tasks with starting time s i, duration d i, resource demand r i resource capacity c, completion time e. 3 3 3 2 B C 2 C 2 1 A 1 A B 1 A B C 1 2 3 4 5 cumulative([1,2,4], [4,2,2], [1,2,2],3,5) 1 2 3 4 5 cumulative([1,2,2], [1,1,1], [2,1,2],3,2) 1 2 3 4 5 cumulative([1,3,5], [2,1,1], [1,1,1],1,4) 6
Diffn Constraint Beldiceanu/Contejean 94 Nonoverlapping of n-dimensional rectangles [O 1,..., O n, L 1,..., L n ], where O i resp. L i denotes the origin resp. length in dimension i diffn([[o 11,..., O 1n, L 11,..., L 1n ],..., [O m1,..., O mn, L m1,..., L mn ]]) 4 3 2 1 1 2 3 4 5 diffn([[1,2,2,2],[3,1,2,1],[4,2,3,3]]) General form: diffn(rectangles,min_vol,max_vol,end,distances,regions) 7
IP vs. CP : Algorithms IP CP Inference Linear programming Domain filtering Cutting planes Constraint propagation Search Branch-and-relax Branch-and-bound Branch-and-cut Bounds on the Two-sided One-sided objective function 8
Local vs. Global Reasoning Linear arithmetic constraints 3 x + y 7, 3 y + x 7, x + y = z, x, y {0,..., 3} CP (Filtering) : x, y 2, z 4 LP (Linear programming) : x, y 2, z 3.5 IP (Cutting plane) : x, y 2, z 3 Global reasoning in CP? global constraints y 3 2 1 0 0 1 2 3 x 9
Global Reasoning in CP Example x 1, x 2, x 3 {0, 1} pairwise different values Local consistency : 3 disequalities : x 1 x 2, x 1 x 3, x 2 x 3 x 1, x 2, x 3 {0, 1}, i.e. no domain reduction is possible Global constraint : alldifferent(x 1, x 2, x 3 ) detects infeasibility (uses bipartite matching) Global reasoning in CP : inside global constraints Use domain-specific algorithms within a general solver 10
2. IP vs. CP : An Illustrating Example 11
Discrete Tomography Bockmayr/Kasper/Zajac 98 Binary matrix with m rows and n columns Horizontal projection numbers (h 1,..., h m ) Vertical projection numbers (v 1,..., v n ) Properties Horizontal convexity (h) Vertical convexity (v) Connectivity (polyomino) (p) Complexity (Woeginger 01) H 1 3 4 2 V 1 2 3 2 1 1 polynomial: ( ), (p,v,h) NP-complete: (p,v), (p,h), (v,h), (v), (h), (p) 12
IP Model Variables x ij = { 0 cell(i,j) is labeled white 1 cell(i,j) is labeled black Constraints I: Projections n x ij = h i, j=1 m x ij = v j i=1 Constraints II: Convexity hi= 3 x ik x i(k+3) h i x ik + n l=k+h i x il h i, v j x kj + m l=k+v j x lj v j, 13
IP Model (contd) Constraints III: Connectivity j+h i 1 k=j x ik j+h i 1 k=j x (i+1)k h i 1 hi= 4 h i+1 =3 x i2 Various linear arithmetic models possible, e.g. convexity Enormous differences in size and running time, e.g. 1 day vs. < 1 sec Large number of constraints ( 3mn in the above model) 14
Finite Domain Model Variables x i start of the horizontal convex block in row i, for 1 i m y j start of the vertical convex block in column j, for 1 j n Domain H 2 Y 2 1 1 3 3 3 V 1 3 5 3 1 1 x i [1,..., n h i + 1], for 1 i m y j [1,..., m v j + 1], for 1 j n X 1 2 3 3 2 3 5 2 2 15
Conditional Propagation Projection/Convexity modelled by FD variables Compatibility of x i and y j x i j < x i + h i y j i < y j + v j for 1 i m and 1 j n row i Conditional propagation if x i j then (if j < x i + h i then (y j i, i < y j + v j )) 16
Finite Domain Model (contd) Connectivity row i row i+1 1 x i x i + h i -1 n 1 n x i+1 + h i+1-1 x i+1 Block i must start before the end of block i + 1 x i x i+1 + h i+1 1, for 1 i m 1 Block i + 1 must start before the end of block i x i+1 x i + h i 1, for 1 i m 1 17
Cumulative 3 4 3 7 6 4 4 3 4 2 3 4 3 7 6 4 4 3 4 2 1 4 5 5 4 4 7 7 2 1 18
2d and 3d Diffn Model 3 4 3 7 6 4 4 3 4 2 3 4 3 7 6 4 4 3 4 2 1 4 5 5 4 4 7 7 2 1 1 4 5 5 4 4 7 7 2 1 19
3. Combining IP and CP 20
Generic MIP/CP Model Bockmayr/Pisaruk 03 max cx + dz, (1) Ax + Gz b, (2) F i (x i ) = 0, i = 1,..., m, (3) l x u, x Z n, l z u, z R p, (4) where x i = (x i1,..., x ik(i) ) consists of some components of x = (x 1,..., x n ), and F i : Z k(i) {0, 1} is monotone, i.e., x y F (x) F (y). 21
Monotone constraints Let x, y [l, u] with F (x) = 0 and F (y) = 1. For F : [l, u] {0, 1} monotone, we cannot have y x. Therefore (x 1 y 1 1) (x 2 y 2 1)... (x n y n 1) (5) If y i = l i y i = u i, for i = 1,..., n, then (5) is equivalent to x i ( i:y i >l i i:y i >l i y i ) 1 (6) In general, inequality (6) is stronger than the disjunction (5). 22
Binary Case If [l, u] = {0, 1} n, then (6) and (5) are equivalent. (6) can be rewritten as the cardinality inequality x i S(y) 1, (7) i S(y) where S(y) = {j : y j 0} denotes the support of the vector y. It follows F 1 (0) = {x {0, 1} n : F (x) = 0} (8) = {x {0, 1} n : x i S(y) 1, y F 1 (1)}, (9) i S(y) 23
Heuristic Separation Separate x [0, 1] n from conv(f 1 (0)) 1. Sort components of x in nondecreasing order: x π(1)... x π(n). 2. Let r {1,..., n} be the largest index such that r { j=1 xi π(j) > r 1. 3. Obtain a 0-1 vector y {0, 1} n 1, if 1 j r, by y π(j) = 0, if r + 1 j n. 4. If F (y) = 1, the inequality r j=1 x π(j) r 1 separates x from conv(f 1 (0)). The procedure may fail to separate x from conv(f 1 (0)) even if x conv(f 1 (0)). However, it always separates infeasible integral points. 24
IP/CP Branch and Cut Start with the IP program (1), (2), and (4). Assume x has to be tested for feasibility (when solving some LP subproblem). For i = 1,..., m, use the heuristic together with CP algorithms to separate x i from conv(f 1 i (0)). Since the heuristic always separates infeasible integer points, the output is an optimal feasible solution to problem (1) (4) (if it exists). 25
Multiple Machine Scheduling Problem Jain/Grossmann 01 n tasks, m dissimilar machines. any task can be processed on any machine. processing cost c ij and processing time p ij for task i on machine j r i release date, d i due date of task i. carry out all the tasks at the least possible cost. Main decisions 1. Assignment of tasks to machines. 2. Sequencing of tasks on each machine, and starting time for each task. 26
Combined IP/CP Model x ij = 1 if task i is assigned to machine j, x ij = 0 otherwise. F j (x j ) = 0 if the tasks in S(x j ) can be done on machine j respecting release and due dates. min n i=1 m j=1 c ijx ij m j=1 x ij = 1, i = 1,..., n, m j=1 p ijx ij d i r i, i = 1,..., n, n i=1 p ijx ij max 1 i n d i min 1 i n r i, j = 1,..., m, F j (x 1j,..., x nj ) = 0, j = 1,..., m, x ij {0, 1}, i = 1,..., n; j = 1,..., m. 27
Testing Feasibility by CP Handle F j (x j ) = 0 by solving a CP program with a single global constraint. Let D j (x j ) = max i S(x j ) d i, and assume S(x j ) = {s 1,..., s k }, k = S(x j ). Then F j (x j ) = 0 iff the CP problem t si [r si, d si p si,j], i = 1,..., k, cumulative([[t s1, p s1,j, 1],..., [t sk, p sk,j, 1]], 1, D j (x j )) has a solution. Computational results: Bockmayr/Pisaruk 03 Further applications: Supply chain optimization (LISCOS) 28
Conclusion IP and CP Model building: Language Model solving: Algorithms Case study: Discrete tomography Combining IP and CP: Cut generation for IP using CP 29
References A. Bockmayr and J. N. Hooker: Constraint programming. In Handbooks in Operations Research and Management Science. Vol. 12: Discrete Optimization (Eds. K. Aardal, G. Nemhauser, and R. Weismantel), Chapter 10, 559-600, Elsevier, 2005. A. Bockmayr and N. Pisaruk: Detecting Infeasibility and Generating Cuts for MIP using CP. 5th International Workshop on Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems, CPAIOR 03, Montreal, 2003. Revised version to appear in Computers & Operations Research 30