the alldifferent constraint. Bockayr and Kasper propose an interesting fraework in (Bockayr & Kasper 1998) for cobining CLP and IP, in which several a

Transcription

1 On Integrating Constraint Propagation and Linear Prograing for Cobinatorial Optiization John N. Hooker, y Greger Ottosson, z Erlendur S. Thorsteinsson y and Hak-Jin Kiy y Graduate School of Industrial Adinistration Carnegie Mellon University, Pittsburgh, PA 15213, U.S.A. z Coputing Science Dept., Uppsala University PO Box 311, S Uppsala, Sweden Abstract Linear prograing and constraint propagation are copleentary techniques with the potential for integration to benet the solution of cobinatorial optiization probles. Attepts to cobine the have ainly focused on incorporating either technique into the fraework of the other traditional odels have been left intact. We argue that a rethinking of our odeling traditions is necessary to achieve the greatest benet of such anintegration. We propose a declarative odeling fraework in which the structure of the constraints indicates how LPand CP can interact to solve the proble. Introduction Linear prograing (LP) and constraint propagation (CP) are techniques fro dierent elds that tend to be used separately in integer prograing (IP) and constraint (logic) prograing (CLP), respectively. They have the potential for integration to benet the solution of cobinatorial optiization probles. Yet only recently have attepts been ade at cobining the. IP has been successfully applied to a wide range of probles, such as capital budgeting, bin packing, crew scheduling and traveling salesan probles. CLP has in the last decade been shown to be a exible, ecient and coercially successful technique for scheduling, planning and allocation. These probles usually involve perutations, discretization or syetries that ay result in large and intractable IP odels. Both CLP and IP rely on branching to enuerate regions of the search space. But within this fraework they use dual approaches to proble solving: inference and search. CLP ephasizes inference in the for of constraint propagation, which reoves infeasible values fro the variable doains. It is not a search ethod, i.e., an algorith that exaines a series of coplete labellings until it nds a solution. IP, bycontrast, does exactly this. It obtains coplete labellings by solving linear prograing relaxations of the proble in the branching tree. IP has the advantage that it can generate cutting planes (inequalities iplied by the constraint set) that strengthen the linear relaxation. This can be a powerful technique when the proble is aenable to polyhedral analysis. But IP has the disadvantage that its constraints ust be expressed as inequalities (or equations) involving integer-valued variables. Otherwise the linear prograing relaxation is not available. This places a severe restriction on IP's odeling language. In this paper we argue that the key to eective integration of CP and LP lies in the design of the odeling language. We propose a language in which conditional constraints indicate how CP and LP can work together to solve the proble. We begin, however, by reviewing eorts that have hitherto been ade toward integration. Previous Work Several articles copare CLP and IP (Sith et al Darby-Dowan & Little 1998). They report experiental results that illustrate soe key properties of the techniques: IP is very ecient for probles with good relaxations, but it suers when the relaxation is weak or when its restricted odeling fraework results in large odels. CLP, with its ore expressive constraints, has saller odels that are closer to the proble description and behaves well for highly constrained probles, but it lacks the global perspective of relaxations. Soe attepts have been ade at integration. In (Rodosek, Wallace, & Hajian1997), CPis used along with LP relaxations in a single search tree to prune doains and establish bounds. A node can fail either because propagation produces an epty doain, or the LP relaxation is infeasible or has an optial value that is worse than the value of the optial solution (discussed below). A systeatic procedure is used to create a shadow MIP odel for the original CLP odel. It includes reied arithetic constraints (which produce big-m constraints, illustrated below) and alldifferent constraints. The odeler ay annotate constraints to indicate which solver should handle the CP, LP or both. Soe research has been aied at incorporating better support for sybolic constraints in IP. (Hajian, Rodosek, & Richards 1996 Hajian 1996) show how disequalities ( i 6= j ) can be handled (ore) eciently in IP solvers. Further, they give a linear odeling of

2 the alldifferent constraint. Bockayr and Kasper propose an interesting fraework in (Bockayr & Kasper 1998) for cobining CLP and IP, in which several approaches to integration or synergy are possible. They investigate how sybolic constraints can be incorporated into IP uch as cutting planes are. They also show how a linear syste of inequalities can be used in CLP by incorporating it as a sybolic constraint. They also discuss a closer integration in which both linear inequalities and doains appear in the sae constraint store. Characterization We start with a basic characterization of CLP and IP. Constraint (Logic) Prograing In Finite Doain CLP each integer variable x i has an associated doain D i, which is the set of possible values this variable can take on in the (optial) solution. The cartesian product of the doains, D 1 ::: D n, fors the solution space of the proble. This space is nite and can be searched exhaustively for a feasible or optial solution, but to liit this search CP is used to infer infeasible solutions and prune the corresponding doains. Fro this viewpoint, CP operates on the set of possible solutions and narrows it down. Integer Prograing In contrast to CLP, IP does not aintain and reduce a set of solutions dened by variable doains. It generates a series of coplete labellings, each obtained at a node of the branching tree by solving a relaxation of the proble at that node. The relaxation is usually constructed by dropping soe of the constraints, notably the integrality constraints on the variables, and perhaps by adding valid constraints (cutting planes) that ake the relaxation tighter. In a typical application the relaxation is rapidly solved to optiality with a linear prograing algorith. If the ai is to nd a feasible solution, branchingcontinues until the solution of the relaxation happens to be feasible in the original proble (in particular, until it is integral). Relaxations therefore provide a heuristic ethod for identifying solutions. In an optiization proble, relaxation also provides bounds for a branchand-bound search. At each node of the branching tree, one can check whether the optial value of the relaxation is better than the value of the best feasible found so far. If not, there is no need to branch further at that node. The dual of the LP relaxation can also provide useful inforation, perhaps by xing soe integer variables or generating additional constraints (nogoods) in the for of Benders cuts. (Benders 1962 Georion 1974). We will see how the latter can be exploited in an integrated fraework. Coparison of CP and LP CP can accelerate the search for a solution by reducing variable doains (and in particular by proving infeasibility), tightening the linear relaxation by adding bounds and cuts in addition to classical cutting planes, and eliinating search of syetric solutions, which are often ore easily excluded by using sybolic constraints. LP can enhance the solver by nding feasible solutions early in the searchby global reasoning, i.e., solution of an LP relaxation, siilarly providing stronger bounds that accelerate the proof of optiality, and providing reasons for failure or a poor solution, so as to produce nogoods. Modeling for Hybrid Solvers The approaches taken so far to integration of CP and LP are (a) to use both odels in parallel, and (b) to try to incorporate one within the other. The ore fundaental question of whether a new odeling fraework should be used has not yet been explored in any depth. The success of (a) depends on the strength of the links between the odels and to what degree the overhead of having two odels can be avoided. Option (b) is liited in what it can achieve. The high-level sybolic constraints of CLP cannot directly be applied in an IP odel, and the sae holds for attepts to use IP's cutting planes and relaxations in CLP. We will use a siple ultiple-achine scheduling proble to illustrate the advantages of a new odeling fraework. Assue that we wish to schedule n tasks for processing on as any asn identical achines. The objective is to iniize the total xed cost of the achines we use. Let R j be the release tie, P j the processing tie and D j the deadline for task j. Let C be the xed cost of using achine and t j the start tie of task j. We rst state an IP odel, which uses 01 variables x ij to indicate the sequence in which the jobs are processed. Let x ij =1if task i precedes task j, i 6= j, on the sae achine, with x ij =0otherwise. Also let 01 variable y j =1if task j is assigned to achine, and 01 variable z =1if achine is used. Then an IP

3 forulation of this proble is, in C z s.t. z y j 8 j (1) y j =1 8j: (2) t j + P j D j R j t j 8j 8j t i + P i t j + M (1 ; x ij ) 8i 6= j (3) x ij + x ji y i + y j ; 1 8 i < j (4) z y j x ij 2f0 1g t j 0 8 i j: Constraint (3) is a big-m constraint. If M is a suf- ciently large nuber, the constraint forces task i to precede task j if x ij =1and has no eect otherwise. A CLP odel for the sae proble is in s.t. C z if i = j then (t i + P i t j ) _ (t j + P j t i ) 8i j (5) t j + P j D j R j t j if atleast([ 1 ::: n ] 1) 8j 8j then z =1else z =0 8 (6) j 2f1::: ng t j 0 where j is the achine assigned to task j. The CLP odel has the advantage of dispensing with the doublysubscripted 01 variables x ij and y i, which are necessary in IP to represent perutations and assignents. This advantage can be pronounced in larger probles. A notorious exaple is the progressive party proble (Sith et al., 1995), whose IP odel requires an enorous nuber of ultiply-subscripted variables. The IP odel has the advantage of a useful linear prograing relaxation, consisting of the objective function, constraints (1)(2), and bounds 0 x j 1. The 01 variables y j enlarge the odel but copensate by aking this relaxation possible. However, the IP constraints involving the perutation variables x ij yield a very weak relaxation and needlessly enlarge the LP relaxation. Soehow we ust cobine the succinctness of the CLP odel with an ability to create a relaxation fro that portion of the IP odel that has a useful relaxation. To dothiswe propose taking a step back toinvestigate how one can design a odel to suit the solvers rather than adjust the solvers to suit the traditional odels. 8j Mixed Logical/Linear Prograing We begin with the fraework of Mixed Logical/Linear Prograing (MLLP) proposed in (Hooker 1994 Hooker & Osorio 1997 Hooker, Ki, & Ottosson 1998). It writes constraints in the for of conditionals that link the discrete and continuous eleents of the proble. A odel has the for in cx (7) s.t. h i (y)! A i x b i i 2 I x 2 R n y 2 D where y is a vector of discrete variables and x avector of continuous variables. The antecedents h i (y) of the conditionals are constraints that can be treated with CP techniques. The consequents are linear inequality systes that can be inserted into an LP relaxation. A linear constraint set Ax b which is enforced unconditionally ay beso written for convenience, with the understanding that it can always be put in the conditional for (0 = 0)! Ax b. Siilarly, an unconditional discrete constraint h can be forally represented with the conditional :h! (1 = 0). A useful odeling device is a variable subscript, i.e., a subscript that contains one or ore discrete variables. For exaple, if c jk is the cost of assigning worker k to job P j, the total cost of an assignent can be written j c jy j, where y j is a discrete variable that indicates the worker assigned job j. The value of c jyj is in eect a function of y =(y 1 ::: :y n ) and can be written g j (y), where function g j happens to depend only on y j. The MLLP odel can incorporate this device as follows: where in cx s.t. h i (y)! L i (x y) i 2 I L i (x y) = x 2 R n y2 D k2k i (y) a ik (y)x jik (y) b i (y): Note that the odel also allows for a suation taken over a variable index set K i (y), which is a set-valued function of y, aswell as real-valued variable constants b i (y). Models of this sort can in principle be written in the ore priitive for (7) by adding suciently any conditional P constraints. For exaple, P the constraint z j c jy j can be written z j z j, if the following constraints are added to the odel (y j = k)! (z j = c jk ) all j k 2f1:::ng where each y j 2f1::: ng. It is preferable, however, for the solver to process variables subscripts and index sets directly. The Solution Algorith An MLLP proble is solved by branching on the discrete variables. The conditionals assign roles to CP and

4 LP: CP is applied to the discrete constraints to reduce the search and help deterine when partial assignents satisfy the antecedents. At each node of the branching tree, an LP solver iniizes cx subject to the inequalities A i x b i for which h i (y) is deterined to be true. This delayed posting of inequalities leads to sall and lean LP probles that can be solved eciently. A feasible solution is obtained when the truth value of every antecedent is deterined, and the LP solver nds an optial solution subject to the enforced inequalities. Coputational tests reported in (Hooker & Osorio 1997) suggest that an MLLP fraework not only has odeling advantages but can often perit ore rapid solution of the proble than traditional MILP solvers. However, a nuber of issues are not addressed in this work, including: (a) systeatic ipleentation of variable subscripts and index sets, (b) taking full advantage of the LP solution at each node, and (c) branching on continuous variables and propagation of continuous constraints. An Exaple We can now forulate the ultiple achine scheduling proble discussed earlier in an MLLP fraework. Let k index a sequence of events, each of which is the start of soe task. Let t k be the start tie of event k, s k the task that starts, and k the achine to which itis assigned. The foral MLLP odel is, in s.t. f ( k = l )! (t k + P sk t l ) 8k <l t k + P sk D sk R sk t k alldifferentfs 1 ::: s n g 8k 8k f k = C k f 0 8k : The odel shares CLP's succinctness by dispensing with doubly-subscripted variables. To obtain the relaxation aorded by IP, we can siply add the objective function and constraints (1)(2) to the odel, and link the variables y i to the other variables logically in (10). in s.t. C z ( k = l )! (t k + P sk t l ) 8k <l t k + P sk D sk R sk t k alldifferentfs 1 ::: s n g 8k 8k z y j 8 j (8) y j =1 8j (9) y k s k =1 8k: (10) P The relaxation now iniizes C z subject to (8), (9), 0 y j 1, and y j = 1 for all y j xed to 1by (10). One can of course use relaxations other than the traditional ones, including discrete relaxations. A Perspective on MLLP The fraework for integration described in (Bockayr & Kasper 1998) provides an interesting perspective on MLLP. The CLP literature distinguishes between priitive and nonpriitive constraints. Priitive constraints are easy constraints for which there are ecient (polynoial) satisfaction and optiization procedures. They are aintained in a constraint store, which in nite-doain CLP consists siply of variable doains. Propagation algoriths for nonpriitive constraints retrieve current doains fro the store and add the resulting saller doains to the store. In IP, linear inequalities over continuous variables are priitive because they can be solved by linear prograing. The integrality conditions are (the only) nonpriitive constraints. Bockayr and Kasper propose two ways of integrating LP and CP. The rst is to incorporate the LP part of the proble into a CLP fraework as a nonpriitive constraint. Thus LP becoes a constraint propagation technique. It accesses doains in the for of bounds fro the constraint store and add new bounds obtained by iniizing and axiizing single variable. A second approach is to ake linear inequalities priitive constraints. The constraint store contains continuous inequality relaxations of the constraints but excludes integrality conditions. For exaple, discrete constraints x 1 _ x 2 and :x 1 _ x 2 couldberepresented in the constraint store as inequalities x 1 + x 2 1 and (1 ; x 1 )+x 2 1 and bounds 0 x j 1. If constraint propagation deduced that x 2 is true, the inequality x 2 1 would be added to the store. This is an instance of what has long been known as preprocessing in MILP, which can therefore be viewed as a special case of this second kind of integration. MLLP is a third typeofintegration in whichtwoconstraint stores are aintained (see Figure 1). A classical Nonpriitive ::: Nonpriitive Constraint 1 Constraint n FD Store LP Store Figure 1: Constraint stores and nonpriitive constraints in MLLP nite doain constraint store, S FD,contains doains, and the LP constraint store, S LP, contains linear inequalities and bounds. The nonpriitive constraints can access and add to both constraint stores. Since only doain constraints x i 2 D i exist in the FD store, integrality constraints can reain therein as priitive

5 constraints. There are no continuous variables in the CP store and no discrete variables in the LP store. The conditional constraints of MLLP act as the prie inference agents connecting the two stores, reading doains of the CP store and adding inequalities to the LP store (Figure 2). In the original MLLP schee, the conditionals are unidirectional, in the sense that they infer fro S FD and post to S LP and not vice-versa. This is because the solution algorith branches on discrete variables. As the discrete doains are reduced by branching, the truth value of ore antecedents is inferred by constraint propagation, and ore inequality constraints are posted. However, conditionals in the opposite direction could also be used if one branched on continuous variables by splitting intervals. The antecedents would contain continuous nuerical constraints (not necessarily linear inequalities), and the consequents would contain priitive discrete constraints, i.e., restrictions on discrete variable doains. The truth value of the antecedents ight be inferred using interval propagation. We will next give two ore exaples of nonpriitive constraints in MLLP a generalized version of the eleent constraint for handling variable subscripts, and a constraint which derives nogoods fro S LP. Variable Subscripts As seen before, MLLP provides variable subscripts as a odeling coponent, but an expansion to conditional constraints is in ost cases not tractable. Instead a nonpriitive constraint, or inference agent, can be designed to handle variable subscripts ore eciently. There are basically two cases in which avariable subscript can occur in a discrete constraint orinacontinuous inequality. In the forer case it can either be avector of constants or a vector of discrete variables both of these correspond to the traditional use of the eleent/3 (Marriott & Stuckey 1998) constraint found in all ajor CP systes and libraries (e.g. (Dincbas et al Carlsson 1995)). This constraint takes the for eleent FD (I, [ 1 ::: n ], Y ), where I is an integer variable with doain D I = f1::: ng, indexing the list, and Y = I. Here we will consider the second case, eleent LP (I, [ 1 ::: n ], Y ), wherei is still a discrete, indexing variable, but x i and Y are continuous variables or constants. Propagating this constraint can be done alost as before. Let the interval [in(x i ) ax(x i )] be the doain of x i. Then upon change of the doain of I, we can copute in = fin(x i )ji 2 D I g ax =fax(x i )ji 2 D I g reading D I fro the S CP and the adding new bounds in Y ax to S LP. Siilarly, bounds of Y can be used to prune D I. (Stronger bounds for variables in LP can be obtained by iniizing and axiizing the variable subject to the linear inequalities in S LP, which for soe cases ight be benecial.) The iportant point here is not the details of how we can propagate this constraint, but rather to exeplify how an inference agent can naturally access both constraint stores. Infeasible LP When the LP is infeasible, any dual solution species an infeasible linear cobination of the constraint set. For each conditional constraint y i! A i x b i i 2 I where soe ax b 2 A i x b i has a corresponding nonzero dual ultiplier, we can for the logical constraint _ i2i :y i This nogood (Tsang 1993) ust be satised by any solution of the proble, because its corresponding set of linear inequalities fors an infeasible cobination. This schee can naturally be encapsulated within a nonpriitive constraint, nogood, reading fro S LP and writing to S FD. This agent can collect, erge and aintain no-goods for any cobination of nonzero dual values in any infeasible LP node, and can infer priitive and nonpriitive constraints which will strengthen S FD. Figure 2 shows this and the other basic nonpriitive constraints of MLLP. Eleent FD FD Store Eleent LP Conditional y i! Ax b Nogood W :yi LP Store Figure 2: Nonpriitive constraints in MLLP Feasible LP In IP, the solution of the relaxation provides a coplete labelling of the variables. This sort of labelling is not iediately available in MLLP, because the relaxation involves only continuous variables. However, the solution x of a feasible relaxation can be heuristically extended to a coplete labelling (x y) that ay satisfy the constraints. (Because y does not occur in the objective function, its value will not aect the optial value of the proble.) Given any conditional h i (y)! A i x b i, h i (y) ust be false if A i x 6 b i, but it can be true or false if A i x b i. One can therefore eploy a heuristic (or even an exhaustive search)

6 that tries to assign values y j to the y j 's fro their current doains so as to falsify the antecedents that ust be false. Conclusion LP and CP have long been used separately, but they have thepotential to be integrated as copleentary techniques in future optiization fraeworks. To do this fully and in general, the odeling traditions of atheatical prograing and constraint prograing also ust be integrated. We propose a unifying odeling and solution fraework that ais to do so. Continuous and discrete constraints are naturally cobined using conditional constraints, allowing a clean separation and a natural link between constraints aenable to CP and continuous inequalities eciently handled bylp. References Benders, J. F Partitioning procedures for solving ixed-variables prograing probles. Nuer. Math. 4: Bockayr, A., and Kasper, T Branch-andinfer: A unifying fraework for integer and nite doain constraint prograing. INFORMS J. Coputing 10(3): Carlsson, M SICStus Prolog User's Manual. SICS research report, Swedish Institute of Coputer Science. URL: Darby-Dowan, K., and Little, J Properties of soe cobinatorial optiization probles and their eect on the perforance of integer prograing and constraint logic prograing. INFORMS Journal on Coputing 10(3): Dincbas, M. Van Hentenryck, P. Sionis, H. Aggoun, A. Graf, T. and Berthier, F The Constraint Logic Prograing Language CHIP. In FGCS-88: Proceedings International Conference on Fifth Generation Coputer Systes, Tokyo: ICOT. Georion, A Lagrangian relaxation for integer prograing. Matheatical Prograing Study 2: Hajian, M. Rodosek, R. and Richards, B Introduction of a new class of variables to discrete and integer prograing probles. Baltzer Journals. Hajian, M. T Dis-equality constraints in linear/integer prograing. Technical report, IC-Parc. Hooker, J. N., and Osorio, M. A Mixed logical/linear prograing. Discrete Applied Matheatics, to appear. Hooker, J. N. Ki, H.-J. and Ottosson, G A declarative odeling fraework that integrates solution ethods. Annals of Operations Research, Special IssueonModeling Languages and Approaches, subitted. Hooker, J. N Logic-based ethods for optiization. In Borning, A., ed., Principles and Practice of Constraint Prograing, volue 874 of Lecture Notes in Coputer Science, Marriott, K., and Stuckey, P. J Prograing with Constraints: An Introduction. MIT Press. Rodosek, R. Wallace, M. and Hajian, M A new approach to integrating ixed integer prograing and constraint logic prograing. Baltzer Journals. Sith, B. Brailsford, S. Hubbard, P. and Willias, H. P The Progressive Party Proble: Integer Linear Prograing and Constraint Prograing Copared. In CP95: Proceedings 1st International Conference on Principles and Practice of Constraint Prograing). Tsang, E Foundations of Constraint Satisfaction. Acadeic Press.