Symmetry Breaking Ordering Constraints

Symmetry Breaking Ordering Constraints PhD dissertation Uppsala University, Sweden, March 2004 Zeynep KIZILTAN Università di Bologna, Italy URL: http://lia.deis.unibo.it/~zk/

Overview Brief Background Thesis Motivation Goals Vector Orderings Global Ordering Constraints Experimental Results Conclusions Ongoing Research 2

Improving Search Efficiency in CP Use Global Constraints: More powerful inference (GAC) Constraint semantics allows to perform local consistency more efficiently Alldifferent, global cardinality, cumulative etc Exploit Problem Symmetries 3

Symmetry in CSPs Symmetry: permutation in variables/values that preserves satisfiability Symmetry Θ : If A is a solution, Θ(A) is also a solution If A is not a solution, neither is Θ(A) Symmetry exploitation: If subspace S searched, avoid subspace Θ(S) Subspace Θ(S): symmetric states to S Symmetry breaking 4

How to Break Symmetry? 1. Symmetry Breaking Constraints: Add extra constraints, imposing an ordering on the symmetric objects (e.g. variables) Only assignments satisfying the order are considered 2. Modification of the Search Algorithm: SBDS: each explored subtree generates a new constraint to rule out symmetric equivalents in the future SBDD: each node is checked for dominance by subtrees explored in the past 5

Thesis Row and column symmetry is common in constraint programming. Ordering constraints can effectively break this symmetry. Efficient global constraints can be designed for propagating such ordering constraints. 6

Motivations An important class of symmetries in CP matrices of decision variables rows/columns represent indistinguishable objects, hence symmetric Rows and columns can be permuted without affecting satisfiability Encountered frequently 7

Example: Sports Scheduling Schedule games between n teams over n-1 weeks Each week is divided into n/2 periods Each period has 2 slots: home and away Find a schedule such that 1. every team plays exactly once a week 2. every team plays against every other team 3. every team plays at most twice in the same period over the tournament 8

Example: Sport Scheduling We need a table of meetings! Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Period 1 0 vs 1 0 vs 2 4 vs 7 3 vs 6 3 vs 7 1 vs 5 2 vs 4 Period 2 2 vs 3 1 vs 7 0 vs 3 5 vs 7 1 vs 4 0 vs 6 5 vs 6 Period 3 4 vs 5 3 vs 5 1 vs 6 0 vs 4 2 vs 6 2 vs 7 0 vs 7 Period 4 6 vs 7 4 vs 6 2 vs 5 1 vs 2 0 vs 5 3 vs 4 1 vs 3 Weeks are indistinguishable Periods are indistinguishable 9

Example: Sport Scheduling Weeks are indistinguishable Periods are indistinguishable Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Period 1 0 vs 1 0 vs 2 4 vs 7 3 vs 6 3 vs 7 1 vs 5 2 vs 4 Period 2 2 vs 3 1 vs 7 0 vs 3 5 vs 7 1 vs 4 0 vs 6 5 vs 6 Period 3 4 vs 5 3 vs 5 1 vs 6 0 vs 4 2 vs 6 2 vs 7 0 vs 7 Period 4 6 vs 7 4 vs 6 2 vs 5 1 vs 2 0 vs 5 3 vs 4 1 vs 3 10

Example: Sport Scheduling Weeks are indistinguishable Periods are indistinguishable Week 1 Week 2 Week 3 Week 4 Week 5 Week 6 Week 7 Period 1 0 vs 1 3 vs 7 4 vs 7 3 vs 6 0 vs 2 1 vs 5 2 vs 4 Period 2 2 vs 3 1 vs 4 0 vs 3 5 vs 7 1 vs 7 0 vs 6 5 vs 6 Period 3 4 vs 5 2 vs 6 1 vs 6 0 vs 4 3 vs 5 2 vs 7 0 vs 7 Period 4 6 vs 7 0 vs 5 2 vs 5 1 vs 2 4 vs 6 3 vs 4 1 vs 3 12

Many more examples Combinatorics BIBD generation, ternary Steiner problem Design Steel mill slab design Configuration Rack configuration Scheduling Social golfers, sport scheduling Timetabling Progressive party problem Bioinformatics Word design 14

Motivations (cont d) An n X m matrix with row and column symmetry has n! X m! symmetries grows super-exponentially Too many symmetric search states It can be very expensive to visit all the symmetric branches of the search tree first solution all solutions (e.g. proving optimality) Breaking symmetries could be very beneficial 15

Motivations (cont d) SBDS and SBDD do not explore the parts of the search tree symmetric to those already visited SBDS treat each symmetry individually many duplicate symmetry constraints impractical when #symmetries is large using a subset of the symmetries requires problem-specific search methods SBDD dominance checks can be expensive in the presence of many symmetries dominance checks are problem-specific 16

Objective and Approach Eliminate row and column symmetries easily effectively efficiently How?: Posting ordering constraints on the rows and columns (symmetry breaking ordering constraints) 17

Goals Investigate the types of ordering constraints that can be posted on a matrix to break row and column symmetries Devise global constraints to post and propagate the ordering constraints effectively and efficiently Show the effectiveness of the ordering constraints in breaking row and column symmetries 18

Lexicographic Ordering GOAL 1 X = [x 1,.,x n ] Y = [y 1,.,y n ] X < lex Y iff 0 k < n x 1,.,x k-1,x k,.,x n = < [1, 2, 5, 3] [1, 3, 1, 5] <lex y 1,.,y k-1, y k,.,y n X lex Y iff X < lex Y or X =Y Constraint posting: on rows on columns Simultaneously? ( lex RC) (Total ordering on vectors) lex lex 19

Lexicographic Ordering Attention with anti-lexicographic ordering! lex lex lex lex lex R lex C lex R lex C lexrc and lex RC effectively remove many symmetries, but not every symmetry 20

Multiset Ordering GOAL 1 Lex is focused on positions Multiset: set with repetitions {{ 1, 2, 1, 1, 2 }} X, Y multisets, X < m Y: X = {{ }} and Y {{ }} or max(x) < max(y) or max(x) = max(y) and X - max(x) < m Y - max(y) {{1, 2, 5, 3}} >m {{1, 3, 1, 5}} X m Y iff X < m Y or X = Y Treat each row/column as a multiset 21

Multiset Ordering Constraint posting: on rows on columns Simultaneously! ( m RC) m m Total ordering on multisets but preordering on vectors: not antisymmetric X = [1,2,3] Y = [2,3,1] We have {{X}} m {{Y}} and {{Y}} m {{X}} but not X = Y 22

Multiset Ordering Multiset and anti-multiset ordering m m m m m m m R m C m R m C m R m C m RC, m RC, m R m C, m R m C effectively remove many symmetries, but not every symmetry 23

Lex vs Multiset Ordering GOAL 1 Incomparable: example, matrix with row symmetry 0 2 1 0 lex, not multi 1 0 0 2 not lex, multi lex RC m R m C m R m C m RC m RC 24

Lex vs Multiset Ordering Incomparable: example, matrix with row symmetry 0 2 1 0 lex, not multi 1 0 0 2 not lex, multi lex RC m R m C m R m C m RC m RC 25

Combining Lex and Multiset Non-compatible on the same dimension Compatible on different dimensions: m R lex C lex m lex R m C m lex m R lex C m R lex C m R lex C m R lex C lex R m C lex R m C lex R m C lex R m C 26

Combining Lex and Multiset Incomparability: lex RC lex R m C m R lex C m RC Many symmetries are broken, but not every symmetry 27

Immediate Questions How much symmetry is broken? size of an equivalence class? #assignments satisfying the ordering constraints? How effective are the ordering constraints? no reduction in the size of the search space variable order is important when looking for 1 solution problem constraints imply the orderig constraints (e.g. Permutation -> multiset) increase in the size of the search space bad variable ordering Are the incomparable constraints incomparable also in practice? 28

Lex Ordering Constraint GOAL 2 Global constraint: LexLeq(X,Y), LexLess(X,Y) Filtering algorithm: GAC in O(n), amortised constant Optimal complexity! Alternatives: Arithmetic constraints (n, d small): equivalent to LexLeq AND decomposition, OR decomp., AND OR decomp. strictly weaker than LexLeq to LexLeq too many constraints to propagate: extra overhead equivalent Experimental results: LexLeq is generally faster than alternatives Warning: interaction with dynamic variable ordering 29

How LexLeq Works Consider the following example where we have two vectors of variables: 30

How LexLeq Works Consider the following example where we have two vectors of variables: X {2} {1,3,4} {1,2,3,4,5} {1,2} {3,4,5} Y {0,1,2} {1} {0,1,2,3,4} {0,1} {0,1,2} 31

How LexLeq Works Consider the following example where we have two vectors of variables: X {2} {1,3,4} {1,2,3,4,5} {1,2} {3,4,5} Y {0,1,2} {1} {0,1,2,3,4} {0,1} {0,1,2} We want to enforce GAC on X lex Y 32

A Tale of Two Pointers We use two pointers, α and β, to avoid repeatedly traversing the vectors. 33

A Tale of Two Pointers We use two pointers, α and β, to avoid repeatedly traversing the vectors We index the vectors as follows: X 0 {2} 1 {1,3,4} 2 {1,2,3,4,5} 3 {1,2} 4 {3,4,5} Y {0,1,2} {1} {0,1,2,3,4} {0,1} {0,1,2} 34

A Tale of Two Pointers We use two pointers, α and β, to avoid repeatedly traversing the vectors X 0 {2} 1 {1,3,4} 2 {1,2,3,4,5} 3 {1,2} 4 {3,4,5} Y {0,1,2} {1} {0,1,2,3,4} {0,1} {0,1,2} α: index such that all variables at more significant indices are ground and equal 36

Pointer Initialisation Needs one traversal of the vectors (linear) α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint 39

Pointer Initialisation Needs one traversal of the vectors (linear) X 0 {2} 1 {1,3,4} 2 {1,2,3,4,5} 3 {1,2} 4 {3,4,5} Y {0,1,2} {1} {0,1,2,3,4} {0,1} {0,1,2} α α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint 40

Pointer Initialisation Needs one traversal of the vectors (linear). X 0 {2} 1 {1,3,4} 2 {1,2,3,4,5} 3 {1,2} 4 {3,4,5} Y {0,1,2} {1} {0,1,2,3,4} {0,1} {0,1,2} α β α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint 41

Failure Inconsistent if β α α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint 42

How LexLeq Works We maintain α and β as assignments made α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint 43

How LexLeq Works We maintain α and β as assignments made. When β = α + 1 we enforce bounds consistency on X α < Y α α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint 44

How LexLeq Works We maintain α and β as assignments made When β = α + 1 we enforce bounds consistency on X α < Y α When β > α + 1 we enforce bounds consistency on X α Y α α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint 45

How LexLeq Works We maintain α and β as assignments made Key: we reduce GAC on vectors to BC on binary constraints α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint 46

How LexLeq Works BC on X α Y α removes 0 and 1 from Y α X 0 {2} 1 {1,3,4} 2 {1,2,3,4,5} 3 {1,2} 4 {3,4,5} Y {0,1,2} {1} {0,1,2,3,4} {0,1} {0,1,2} α α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint β 47

Update α How LexLeq Works X 0 {2} 1 {1,3,4} 2 {1,2,3,4,5} 3 {1,2} 4 {3,4,5} Y {2} {1} {0,1,2,3,4} {0,1} {0,1,2} α α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint β 48

How LexLeq Works BC on X α Y α removes 3 and 4 from X α X 0 {2} 1 {1,3,4} 2 {1,2,3,4,5} 3 {1,2} 4 {3,4,5} Y {2} {1} {0,1,2,3,4} {0,1} {0,1,2} α α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint β 50

Update α How LexLeq Works X 0 {2} 1 {1} 2 {1,2,3,4,5} 3 {1,2} 4 {3,4,5} Y {2} {1} {0,1,2,3,4} {0,1} {0,1,2} α α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint β 51

How LexLeq Works BC on X α < Y α removes 4 and 5 from X α and 0, 1 from from Y α X Y 0 {2} {2} 1 {1} {1} 2 {1,2,3,4,5} {0,1,2,3,4} 3 {1,2} {0,1} 4 {3,4,5} {0,1,2} α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint α β 52

How LexLeq Works The constraint is now GAC X 0 {2} 1 {1} 2 {1,2,3} 3 {1,2} 4 {3,4,5} Y {2} {1} {2,3,4} {0,1} {0,1,2} α: index such that all variables at more significant indices are ground and equal β: most significant index from which the two vectors tails necessarily violate the constraint α β 53

LexLeq with Repeated Variables We have assumed that all the variables in the vectors are distinct What if some variables are shared? Crawford et al symmetry breaking constraint: [X 1, X 2, X 3, X 4, X 5 ] lex [X 2, X 1, X 5, X 4, X 3 ] LexLeq algorithm may not maintain GAC: The fact that X α = Y α is a consistent assignmment when β > α + 1 may not be valid anymore 54

LexLeq with Repeated Variables: Example Let s assume Y 0 = Y 1 X 0 {0} 1 {0} {1} 2 Y {0,1} {0,1} {0} α β 55

LexLeq with Repeated Variables: Example Seeking support for X α = Y α = 0 X 0 {0} 1 {0} {1} 2 Y {0,1} {0,1} {0} α β Need to enforce AC(X α < Y α ) even if β > α + 1 56

LexLeq with Repeated Variables LexLeq(X,Y) can be modified to take into account such cases without increasing the worst case complexity When max(x α ) = max(y α ), we need to propagate the support of X α = Y α = max(y α ) over the shared variables to check the support is still valid. If not, post X α < max(x α ) Similarly, when min(x α ) = min(y α ), we propagate the support of X α = Y α = min(y α ) over the shared variables If support is not valid, post Y α > min(y α ) 57

Lex with Sum Constraint GOAL 2 For a 0/1 matrix: LexLessAndSum(X,Y, Sx, Sy) = X < lex Y, Σ x i = Sx, Σ y i = Sy LexEqAndSum (X,Y, Sx, Sy) = X lex Y, Σ x i = Sx, Σ y i = Sy Often appear in demand, capacity or partitioning problems. Global constraints: LexEqAndSum(X,Y, Sx, Sy) LexLessAndSum(X,Y, Sx, Sy) Filtering algorithm: GAC in O(n) Optimal complexity Experimental results: Comparing with 58

Multiset Ordering Constraint Global constraints: MsetLeq(X,Y), MsetLess(X,Y) Filtering: if d <<n, GAC in O(n) if d >>n, GAC in O(n log(n)) Optimal comlexity when d << n Alternatives: Arithmetic: equivalent to MsetLeq Decomposition using gcc or sorted weaker than sorted GOAL 2 weaker than MsetLeq Experimental results: MsetLeq produces smaller trees than alternatives MsetLeq is faster than alternatives 59

How MsetLeq Works 1 2 4 Algorithm exploits: Multiset order = Lex order on occurrence vectors 4 1 1 {{1,1,1,2,4,4}} >m {{1,1,1,1,2,3,4}} as > 1 2 3 1 4 3 2 1 4 3 2 1 [2, 0, 1, 3] >lex [1, 1, 1, 4] 4 1 1 two 4s one 4 60

How MsetLeq Works Algorithm exploits transitivity of ordering [X 1,..,X i,..,x n ] <m [Y 1,..,Y j,..,y n ] To find support for values for X i : Just need support for max(x i ) Best support comes from [min(x 1 ),..,max(x i ),..,min(x n )] [max(y 1,..,max(Y j ),..,max(y n )] 61

Experimental Results GOAL 3 Ordering constraints are often no redundant and bring additional pruning Ordering constraints are often very effective in breaking row and column symmetry: significantly smaller search trees substantial decrease in solving time orders of magnitude gain (few seconds vs timeout) Ordering constraints that are incomparable in theory are also incomparable in practice 62

Example: BIBDs BIBD Lexicographic Ordering Constraints Basic Symmetry Breaking 1000 Time (sec.) 100 10 1 0,1 0,01 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 BIBD Lexicographic Ordering Constraints Basic Symmetry Breaking Fails 10000000 1000000 100000 10000 1000 100 10 1 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 63

More Experimental Results GOAL 3 Social Golfers: 4 models ordering constraints: improve in 75% cases best results: anti-lex ordering Progressive Party: 8 models ordering constraints: improve in (5,13,29),(6,13,29) best results: lex on rows, multiset on columns Rack Configuration: 3 models ordering constraints: improve in the 6 instances tested best results: lex on rows 64

More Experimental Results (II) GOAL 3 Hamming Code Generation: 8 models ordering constraints: improve in (10,15,9),(10,10,5) best results: multiset on rows, lex on columns (1) strict lex on rows, lex on columns (2) Word Design: 3 models, 3 static variable orders ordering constraints: improve in 1 case, get worse in 8 systematic clash with the static variable ordering heuristic 65

Conclusions Row and column symmetry is common in constraint programming. Ordering constraints can effectively break this symmetry. Efficient global constraints can be designed for propagating such ordering constraints. 66

Ongoing Research Some of the first work on dealing with row and column symmetry effectively and efficiently Row and column symmetry is now an active research area recognised as being critical in many applications GAP-SBDS, GAP-SBDD, STAB,... Ordering constraints gained attention constraint encodings of lex, lex_chain, all-perms See CP conferences and the Symmetry Workshops (SymCon) since 2001 67

And finally... The ordering constraints and the propagators proposed here are central to some of the mechanisms proposed for dealing with row and column symmetries e.g. Puget CP 02 (SBDD) and CP 03 (STAB) Dissertation available via my web page: http://lia.deis.unibo.it/~zk/ 68