Consistency as Projection - PDF Free Download

Consistency as Projection John Hooker Carnegie Mellon University INFORMS 2015, Philadelphia USA

Consistency as Projection Reconceive consistency in constraint programming as a form of projection. For eample, domain consistency, k-consistency, bounds consistency. 2

Consistency as Projection Define a stronger form of consistency, based on projection, that propagates through relaed decision diagrams. rather than through a domain store. J-consistency is achieved by projection onto a set J of variables. 3

Consistency as Projection Find J-consistency algorithms for some popular global constraints. Among, sequence, regular, SAT, alldiff. 4

The Big Picture Decision diagrams and optimization Relaed decision diagrams For constraint propagation, bounding, and discrete optimization Propagating J-consistency through relaed decision diagrams Achieving J-consistency 5

Decision Diagrams Graphical encoding of a function of Boolean (or multivalued) variables Historically used for circuit design & verification Lee (1959), Bryant (1986) Adapt to optimization and constraint programming (CP) Hadžić and JH (2006, 2007) 6

Decision Diagrams Eample: Stable set problem 1 Find ma-weight subset of nonadjacent vertices 1 2 4 3 5 2 3 4 5 j = 0 j = 1 7

Decision Diagrams Eample: Stable set problem 1 Find ma-weight subset of nonadjacent vertices 1 2 4 3 5 2 3 4 5 j = 0 j = 1 Each path corresponds to a feasible solution. 8

Decision Diagrams Eample: Stable set problem 20 1 Find ma-weight subset of nonadjacent vertices 10 2 4 10 10 10 10 2 3 j = 0 j = 1 4 20 1 3 5 10 15 15 5 Longest path (35) corresponds to an optimal solution. 9

Optimization in Decision Diagams We can optimize by finding a longest path in the decision diagram, but The decision diagram can grow eponentially with the problem size. 10

Relaed Decision Diagrams Key idea: Use relaed decision diagrams Represent a superset of the feasible set with a limited-width diagram. First used to strengthen propagation in CP solvers. Reduced 1-million-node search trees to 1 node for graph coloring (multiple alldiff) problem Andersen, Hadžić, JH, Tiedemann (2007) 11

Relaed Decision Diagrams 1 2 3 4 5 Eact diagram Width = 3 9 solutions Relaed diagram Width = 2 11 solutions 12

Propagation Standard CP solvers use domain propagation. Domain consistency = each value in a variable s domain is part some feasible solution. Constraint 1 Reduce domains by achieving (or approimating) domain consistency Propagate domains Constraint 2 Constraint 3 Reduce domains further Reduce domains further Propagate domains 13

Propagation Eample with 2 constraints on 0-1 variables: 2 1 + 2 + 3 1 nonadjacency constraint in stable set problem Reduce domains to: 1 {0} 2 {0,1} 3 {0,1} Longest path over reduced domains is 25, which is optimal for both constraints. 14

Propagation Another eample: 1 2 nonadjacency constraint in stable set problem Constraints have no effect on domains. No information transmitted from one constraint to the other 16

Propagation Another eample: 1 2 nonadjacency constraint in stable set problem Constraints have no effect on domains. No information transmitted from one constraint to the other We can improve propagation by propagating through a relaed decision diagram. 17

Relaed Decision Diagrams Relaed diagram for nonadjacency constraint Diagram after propagating 1 2 Information is now transmitted between constraints. 18

Relaed Decision Diagrams Relaed decision diagrams also provide optimization bounds. Longest path in relaed diagram provides a bound on optimal value. Discrete relaation replaces LP relaation. Bergman, Ciré, van Hoeve, JH (2014) 19

Relaed Decision Diagrams 20 1 20 10 2 10 10 10 3 4 10 10 15 5 15 Eact diagram Longest path = 35 Relaed diagram Longest path = 35 Bound is sharp in this case 20

Relaed Decision Diagrams Wider diagrams yield tighter bounds But take longer to build. Adjust width dynamically. 21

Discrete Optimization Relaed decision diagrams provide a general method for discrete optimization Solve a recursive (dynamic programming) model using a branch-and-bound algorithm. Branch on nodes in last eact layer of relaed decision diagram. rather than branching on variables. New approach to curse of dimensionality. Bergman, Ciré, van Hoeve, JH (2015) 22

Optimization 1 Branching in a relaed decision diagram 2 3 Diagram is eact down to here 4 5 6 Relaed decision diagram 23

Optimization 1 Branching in a relaed decision diagram 2 3 Branch on nodes in this layer 4 5 6 Relaed decision diagram 24

Optimization 1 Branching in a relaed decision diagram 2 3 First branch 4 5 6 Relaed decision diagram 25

Optimization 1 Branching in a relaed decision diagram 2 3 Second branch 4 5 6 Relaed decision diagram 26

Optimization 1 Branching in a relaed decision diagram 2 3 Third branch 4 Continue recursively 5 6 Relaed decision diagram 27

Average solution time (sec) Ma cut on a graph Avg. solution time vs graph density 30 vertices 80 70 60 50 40 30 Optimization CPLEX MDDs 20 10 0 0 0.2 0.4 0.6 0.8 1 Density of graph

Number of instances solved Ma 2-SAT Optimization Performance profile 30 variables 100 90 80 70 60 50 40 MDDs CPLEX 30 20 10 0 0.1 1 10 100 1000 Computation time (sec)

Number of instances solved Ma 2-SAT Optimization Performance profile 40 variables 100 90 80 70 60 50 40 MDDs CPLEX 30 20 10 0 0.1 1 10 100 1000 Computation time (sec)

Consistency Net step: Relaed decision diagrams transmit more than domain information. So let s define a stronger form of consistency than domain consistency. And propagate the results of achieving it. 31

Consistency Key observation: Eisting types of consistency are forms of projection. So define a stronger form of consistency based on projection. 32

Consistency as Projection Domain consistency Project onto each individual variable j. 33

Consistency as Projection Domain consistency Project onto each individual variable j. Eample: Constraint set alldiff,, 1 2 3 1 2 3 a, b a, b b, c 34

Consistency as Projection Domain consistency Project onto each individual variable j. Eample: Constraint set 1 2 3 alldiff,, 1 2 3 a, b a, b b, c Solutions,, 1 2 3 ( a, b, c) ( bac,, ) 35

Consistency as Projection Domain consistency Project onto each individual variable j. Eample: Constraint set Solutions Projection onto 1 1 2 3 alldiff,, 1 2 3 a, b a, b b, c,, 1 2 3 ( a, b, c) ( bac,, ) a b 1, Projection onto 2 a b 2, Projection onto 3 3 c 36

Consistency as Projection Domain consistency Project onto each individual variable j. Eample: Constraint set Solutions Projection onto 1 1 2 3 alldiff,, 1 2 3 a, b a, b b, c,, 1 2 3 ( a, b, c) ( bac,, ) This achieves domain consistency. a b 1, Projection onto 2 a b 2, Projection onto 3 3 c 37

Consistency as Projection Domain consistency Project onto each individual variable j. Eample: Constraint set Solutions Projection onto 1 1 2 3 alldiff,, 1 2 3 a, b a, b b, c,, 1 2 3 ( a, b, c) ( bac,, ) This achieves domain consistency. We will regard a projection as a constraint set. a b 1, Projection onto 2 a b 2, Projection onto 3 3 c 38

Consistency as Projection k-consistency J = ( j j J) Can be defined: A constraint set S is k-consistent if: for every J {1,, n} with J = k 1, every assignment J = v J D j for which ( J, j ) does not violate S, and every variable j J, there is an assignment j = v j D j for which ( J, j ) = (v J,v j ) does not violate S. 39

Consistency as Projection Consistency and backtracking: Strong k-consistency for entire constraint set avoids backtracking if the primal graph has width < k with respect to branching order. Freuder (1982) But strong k-consistency is very hard to achieve. 41

J-Consistency A type of consistency more directly related to projection. Constraint set S is J-consistent if it contains the projection of S onto J = ( j j J) S is domain consistent if it is { j }-consistent for each j. JH (2015) 42

J-Consistency J-consistency and backtracking: If we branch on variables 1, 2,, a natural strategy is to project out n, n1, until computational burden is ecessive. 43

Propagating J-Consistency Eample: among (, ),{ c, d},1,2 ( c) ( d), a, b, c, d 1 2 1 2 alldiff,,, 1 2 3 4 a, b c, d 1 2 3 4 Already domain consistent for individual constraints. If we branch on 1 first, must consider all 4 branches 1 = a,b,c,d 44

Eample: ( c) ( d), a, b, c, d Propagating J-Consistency among (, ),{ c, d},1,2 1 2 1 2 alldiff,,, 1 2 3 4 a, b c, d 1 2 3 4 a,b,d a,b,c,d Suppose we propagate through a relaed decision diagram of width 2 for these constraints c d 52 paths from top to bottom represent assignments to 1, 2, 3, 4 36 of these are the feasible assignments. a,b c,d 45

Eample: ( c) ( d), a, b, c, d Propagating J-Consistency among (, ),{ c, d},1,2 1 2 1 2 alldiff,,, 1 2 3 4 a, b c, d 1 2 3 4 a,b,d a,b,c,d 52 paths from top to bottom represent assignments to 1, 2, 3, 4 36 of these are the feasible assignments. Suppose we propagate through a relaed decision diagram of width 2 for these constraints a,b c,d c d Projection of alldiff onto 1, 2 is alldiff, 1 2 atmost (, ),{ a, b},1 1 2 atmost (, ),{c,d},1 1 2 46

Propagating J-Consistency Let s propagate the 2 nd atmost constraint in the projected alldiff through the relaed decision diagram. Let the length of a path be number of arcs with labels in {c,d}. a,b,d c For each arc, indicate length of shortest path from top to that arc. a,b,c,d a,b c,d d Projection of alldiff onto 1, 2 is alldiff, 1 2 atmost (, ),{ a, b},1 1 2 atmost (, ),{c,d},1 1 2 47

Propagating J-Consistency Let s propagate the 2 nd atmost constraint in the projected alldiff through the relaed decision diagram. Let the length of a path be number of arcs with labels in {c,d}. a,b,d 0,0,1 c 1 For each arc, indicate length of shortest path from top to that arc. a,b,c,d 0,0,1,1 a,b c,d d 2 Projection of alldiff onto 1, 2 is alldiff, 1 2 atmost (, ),{ a, b},1 1 2 atmost (, ),{c,d},1 1 2 48

Propagating J-Consistency Let s propagate the 2 nd atmost constraint in the projected alldiff through the relaed decision diagram. Let the length of a path be number of arcs with labels in {c,d}. a,b,d 0,0,1 c 1 Remove arcs with label > 1 For each arc, indicate length of shortest path from top to that arc. a,b,c,d 0,0,1,1 a,b c,d d 2 Projection of alldiff onto 1, 2 is alldiff, 1 2 atmost (, ),{ a, b},1 1 2 atmost (, ),{c,d},1 1 2 49

Propagating J-Consistency Let s propagate the 2 nd atmost constraint in the projected alldiff through the relaed decision diagram. Let the length of a path be number of arcs with labels in {c,d}. a,b,d 0,0,1 c 1 Remove arcs with label > 1 For each arc, indicate length of shortest path from top to that arc. a,b,c,d 0,0,1,1 a,b c,d Projection of alldiff onto 1, 2 is alldiff, 1 2 atmost (, ),{ a, b},1 1 2 atmost (, ),{c,d},1 1 2 50

Propagating J-Consistency Let s propagate the 2 nd atmost constraint in the projected alldiff through the relaed decision diagram. Let the length of a path be number of arcs with labels in {c,d}. a,b,d 0,0,1 c 1 Remove arcs with label > 1 Clean up. For each arc, indicate length of shortest path from top to that arc. a,b,c,d 0,0,1,1 a,b c,d Projection of alldiff onto 1, 2 is alldiff, 1 2 atmost (, ),{ a, b},1 1 2 atmost (, ),{c,d},1 1 2 51

Propagating J-Consistency Let s propagate the 2 nd atmost constraint in the projected alldiff through the relaed decision diagram. Let the length of a path be number of arcs with labels in {c,d}. For each arc, indicate length of shortest path from top to that arc. a,b,d 0,0,1 a,b,c,d 0,0,1,1 a,b c,d Remove arcs with label > 1 Clean up. Projection of alldiff onto 1, 2 is alldiff, 1 2 atmost (, ),{ a, b},1 1 2 atmost (, ),{c,d},1 1 2 52

Propagating J-Consistency Let s propagate the 2 nd atmost constraint in the projected alldiff through the relaed decision diagram. We need only branch on a,b,d rather than a,b,c,d a,b,d Remove arcs with label > 1 Clean up. a,b,c,d a,b c,d Projection of alldiff onto 1, 2 is alldiff, 1 2 atmost (, ),{ a, b},1 1 2 atmost (, ),{c,d},1 1 2 53

Achieving J-consistency Constraint How hard to project? among sequence regular Easy and fast. More complicated but fast. Easy and basically same labor as domain consistency. SAT alldiff Efficient if conflict clauses are used as logic-based Benders cuts. Quite complicated but practical for small domains. 54

Projection of J-consistency for Among among (,, ), V, t, u where 1 among ( 1,, n 1), V, t, u onto 1,, n1 is ( t 1), u 1 if D n V ( t, u) t,min{ u, n 1} if Dn V ( t 1),min{ u, n 1} otherwise n 55

Eample Projection of J-consistency for Among among (,, ), V, t, u where D D D D D 1 2 3 4 5 1 onto 1,, n1 is ( t 1), u 1 if D n V ( t, u) t,min{ u, n 1} if Dn V ( t 1),min{ u, n 1} otherwise a, b a, b, c a,d c, d d among ( 1,, n 1), V, t, u n among (,,,, ),{ c, d}, t, u 1 2 3 4 5 among ( 1, 2, 3, 4 ),{ c, d},( t 1), u 1 57

J-consistency for Among Eample Projection of among (,, ), V, t, u where D D D D D 1 2 3 4 5 1 onto 1,, n1 is ( t 1), u 1 if D n V ( t, u) t,min{ u, n 1} if Dn V ( t 1),min{ u, n 1} otherwise a, b a, b, c a,d c, d d among ( 1,, n 1), V, t, u n among (,,,, ),{ c, d}, t, u 1 2 3 4 5 among ( 1, 2, 3, 4 ),{ c, d},( t 1), u 1 among ( 1, 2, 3 ),{ c, d},( t 2), u 2 among ( 1, 2),{ c, d},( t 3),min{ u 2,2} among ( 1),{ c, d},( t 4),min{ u 2,1} among (),{ c, d},( t 4),min{ u 2,0} 61

J-Consistency for Sequence Projection is based on an integrality property. The coefficient matri of the inequality formulation has consecutive ones property. So projection of the conve hull of the feasible set is an integral polyhedron. Polyhedral projection therefore suffices. Straightforward (but tedious) application of Fourier elimination yields the projection. Projection onto any subset of variables is a generalized sequence constraint. Compleity of projecting out k is O(kq), where q = length of the overlapping sequences. 64

J-Consistency for Sequence Eample t3 t t,,, 0,1,, 1 among (,, ),{1},2,2, 4,5,6 1 3 4 6 2 5 To project out 6, add constraint among (,, ),{1},1,1 3 4 5 66

J-Consistency for Sequence Eample t3 t t,,, 0,1,, 1 among (,, ),{1},2,2, 4,5,6 1 3 4 6 2 5 To project out 6, add constraint among (,, ),{1},1,1 3 4 5 To project out 5, add constraints among (,, ),{1},1,1 among (, ),{1},0,0 2 3 4 3 4 To project out 4, add constraints 1 1 2 3 among ( ),{1},1,1 among (,, ),{1},1,2 among (, ),{1},0,1 among ( ),{1},0,0 2 3 3 68

J-Consistency for Regular Constraint Projection can be read from state transition graph. Compleity of projecting onto 1,, k for all k is O(nm 2 ), where n = number of variables, m = ma number of states per stage. Shift scheduling eample Assign each worker to shift i {a,b,c} on each day i = 1,,7. Must work any given shift 2 or 3 days in a row. No direct transition between shifts a and c. Variable domains: D 1 = D 5 = {a,c}, D 2 = {a,b,c}, D 3 = D 6 = D 7 = {a,b}, D 4 = {b,c} 71

J-Consistency for Regular Constraint Deterministic finite automaton for this problem instance: = absorbing state Regular language epression: (((aa aaa)(bb bbb))* ((cc ccc)(bb bbb))*)*(c (aa aaa) (cc ccc)) 72

J-Consistency for Regular Constraint State transition graph for 7 stages Dashed lines lead to unreachable states. 73

J-Consistency for Regular Constraint State transition graph for 7 stages Dashed lines lead to unreachable states. Filtered domains 74

J-Consistency for Regular Constraint To project onto 1, 2, 3, truncate the graph at stage 4. 75

J-Consistency for Regular Constraint To project onto 1, 2, 3, truncate the graph at stage 4. 76

J-Consistency for Regular Constraint To project onto 1, 2, 3, truncate the graph at stage 4. Resulting graph can be viewed as a constraint that describes the projection. Constraint is easily propagated through a relaed decision diagram. 77

J-Consistency for SAT Project clause set onto J = { 1, 2, 3 }. Projection is 1 2 1 3 78

J-Consistency for SAT Use logic-based Benders decomposition Benders cuts describe projection onto master problem variables. Current Master problem 1 2 Benders cut from previous iteration 79

J-Consistency for SAT Use logic based Benders decomposition Benders cuts describe projection onto master problem variables. Current Master problem 1 2 solution of master ( 1, 2, 3 ) = (0,1,0) Resulting subproblem 80

J-Consistency for SAT Use logic based Benders decomposition Benders cuts describe projection onto master problem variables. Current Master problem 1 2 1 3 solution of master ( 1, 2, 3 ) = (0,1,0) Benders cut (nogood) Resulting subproblem Subproblem is infeasible. ( 1, 3 )=(0,0) creates infeasibility 82

J-Consistency for SAT Use logic based Benders decomposition Benders cuts describe projection onto master problem variables. Current Master problem 1 2 1 3 1 2 3 solution of master ( 1, 2, 3 ) = (0,1,1) Enumerative Benders cut Subproblem is feasible Resulting subproblem 85

J-Consistency for SAT Benders cuts = conflict clauses in a SAT algorithm Branch on 1, 2, 3 first. 87

J-Consistency for SAT Benders cuts = conflict clauses in a SAT algorithm Branch on 1, 2, 3 first. Conflict clauses 88

J-Consistency for SAT Benders cuts = conflict clauses in a SAT algorithm Branch on 1, 2, 3 first. Conflict clauses Backtrack to 3 at feasible leaf nodes 89

J-Consistency for SAT Benders cuts = conflict clauses in a SAT algorithm Branch on 1, 2, 3 first. Conflict clauses containing 1, 2, 3 describe projection 90

J-Consistency for Alldiff Constraint Projection is inherently complicated. But it can simplify for small domains. The result is a disjunction of constraint sets, each of which contains an alldiff constraint and some atmost constraints. 91

J-Consistency for Alldiff Constraint Eample alldiff,,,, 1 2 3 4 5,,,,,,,,,,,,,, D a b c D c d e D d e f D e f g D a f g 1 2 3 4 5 92

J-Consistency for Alldiff Constraint Eample alldiff,,,, Projecting out 5, we get 1 2 3 4 5,,,,,,,,,,,,,, D a b c D c d e D d e f D e f g D a f g 1 2 3 4 5 a f g alldiff,,,, atmost (,,, ),{,, },2 1 2 3 4 1 2 3 4 because 5 must take one of the values in {a,f,g}, leaving 2 for other i s. Projecting out 4, we note that 4 {a,f,g} or 4 {a,f,g}. If 4 {a,f,g}, we get a f g alldiff,,, atmost (,, ),{,, },1 1 2 3 1 2 3 If 4 {a,f,g}, we get 4 = e, and we remove e from other domains. So the projection is D1 a, b, c alldiff 1, 2, 3 D2 c, d atmost ( 1, 2, 3 ),{ a, f, g},1 D 3 d, f 94

J-Consistency for Alldiff Constraint Eample alldiff,,,, 1 2 3 4 5,,,,,,,,,,,,,, D a b c D c d e D d e f D e f g D a f g 1 2 3 4 5 Projecting out 3, we get simply alldiff, 1 2 Projecting out 2, we get the original domain for 1 D1 a, b, c 95

Research Program Investigate projection for other global constraints. Design J-consistency maintenance algorithms (complete or incomplete) Add J-consistency maintenance to a CP solver. Along with propagation through decision diagrams. 97