Alternative Methods for Obtaining. Optimization Bounds. AFOSR Program Review, April Carnegie Mellon University. Grant FA

Transcription

1 Alternative Methods for Obtaining Optimization Bounds J. N. Hooker Carnegie Mellon University AFOSR Program Review, April 2012 Grant FA

2 Integrating OR and CP/AI Early support by AFOSR First conference (1995) Now an annual conference (CPAIOR)

3 Integrating OR and CP/AI Early support by AFOSR First conference (1995) Now an annual conference (CPAIOR) Today Growing literature (4 books, many papers) Moving into optimization software OPL Studio, SIMPL, SCIP (constraint integer programming), Eclipse, Mosel, BARON, G12 Regular sessions at major conferences INFORMS, ISMP, INFORMS Computing Society, INFORMS Optimization Society

4 Some Current Projects Bounds from finite domain cuts (OR + CP)* Joint work with David Bergman Bounds from binary decision diagrams (OR + CS)* Joint work with David Bergman, Andre Cire, & Willem van Hoeve BDD-based branching methods (OR + CS) Joint work with D. Bergman, A. Cire, W. van Hoeve, & T. Yunes Semantic typing for optimization models (OR + AI) Joint work with Andre Cire & Tallys Yunes *Presented today

5 Bounds from Finite-Domain Cuts Joint work with David Bergman

6 Finite Domain Formulations 0-1 variables often encode choices that can be represented with finite domain variables. x i = finite domain variable Job assigned to worker i Start time of job i City visited after city i y ij = corresponding 0-1 variable y ij = 1 if x i = j

7 Finite Domain Cuts Finite-domain variables are common in constraint programming formulations. If the variables are numeric, the problem has polyhedral structure. Finite-domain cuts can be mapped into the 0-1 model. This may yield new and stronger cuts in the 0-1 model.

8 Finite Domain Cuts Finite-domain variables are common in constraint programming formulations. If the variables are numeric, the problem has polyhedral structure. Finite-domain cuts can be mapped into the 0-1 model. This may yield new and stronger cuts in the 0-1 model. We apply this idea to graph coloring. Has a natural CP formulation.

9 Motivation We obtain two kinds of results: If you find a structure (e.g., odd hole) that yields a known valid inequality in 0-1 space We will give you a stronger cut for free. Use whatever separation algorithm you want.

10 Motivation We obtain two kinds of results: If you find a structure (e.g., odd hole) that yields a known valid inequality in 0-1 space We will give you a stronger cut for free. Use whatever separation algorithm you want. We identify additional structures that yield valid inequalities. They are much stronger than known cuts. Many fewer are required. We have separation algorithms (if needed)

11 Graph Coloring We focus on the vertex coloring problem. Given a graph, assign colors to vertices so that no two adjacent vertices receive the same color. Minimize the number of colors

12 0-1 model 1 Graph Coloring = 1 if color j is used 2 3 min j j y = 1, all vertices i ij y + y w, all colors j 1 j 2 j j y + y w, all colors j 1 j 5 j j y + y w, all colors j 2 j 3 j j y + y + y w, all colors j y 3 j 4 j 5 j j ij w { 0,1 } j 4 = 1 if vertex i receives color j 5

13 Alldiff Systems Use an all-different constraint for each clique. 2 3 min z x, all vertices i i ( 1 2 ) ( 1 5 ) ( 2 3 ) ( ) { 0,...,4 } alldiff x, x, all colors j alldiff x, x, all colors j alldiff x, x, all colors j alldiff x, x, x, all colors j x i z 1 4 = color assigned to vertex i 5 Objective reduces symmetry

14 Alldiff Systems Applications: Scheduling, timetabling. Employee scheduling. Course timetabling. Latin squares. Alldiff for each row, column. Experimental design: orthogonal Latin squares. Sudoku puzzles. Graph coloring. Many applications.

15 Related Work Convex hull of single alldiff. Hooker (2000), Williams and Yan (2001). Convex hull of 2 alldiffs. Appa, Magos and Mourtos (2004) Convex hull of alldiff systems with inclusion property. Appa, Magos and Mourtos (2011). Same facets as individual alldiffs. Some facets of systems without inclusion property. Magos and Mourtos (2011).

16 Variable Mapping To write finite domain cuts in terms of 0-1 variables y ij : Substitute x = i jy ij j

17 Variable Mapping To write finite domain cuts in terms of 0-1 variables y ij : Substitute x = i jy ij j In general, facet-defining finite-domain cuts don t map to facet-defining 0-1 cuts. They can nonetheless be more effective than known cuts.

18 Choice of Domain We will assume each x i has domain {0,, n 1}. To simplify exposition. Most results can be generalized to an arbitrary numeric domain {v 0,, v n-1 } with each v i 0. Some results are valid for domain D = {0,δ,, (n 1)δ} with δ > 0.

19 Odd Cycles A q-cycle consists of q alldiff constraints that look like this: alldiff x 17 x 10 x 9 x 1 x 2 x 11 alldiff x 3 x 4 x 12 alldiff x 16 x 8 x x x x 6 x 5 x 13 alldiff alldiff

20 Odd Cycles Select any subset of s vertices in each overlap: s = 2 S 5 x 17 x 10 x 9 x 16 S 1 x 1 x 2 x 11 x 13 x 3 x 4 S 2 x 12 x 8 x x x S 4 x 6 x 5 S 3

21 Odd Cycles i S We get a valid x-cut: q 1 x i sq L ( L 1) = 20 4 sq where L = ( q 1) / x 10 x x 8 x x 1 x 2 x 6 x 5 0 x 3 x

22 Odd Cycles We get a valid x-cut: i S q 1 x i sq L ( L 1) = 20 4 sq where L = ( q 1) / 2 The inequality is facetdefining if q is odd. 4 3 x 10 x x 8 x x 1 x 2 x 6 x 5 0 x 3 x

23 Odd Cycles We get a valid x-cut: i S q 1 x i sq L ( L 1) = 20 4 sq where L = ( q 1) / 2 The inequality is facetdefining if q is odd. For s = 1 we get the odd hole cut i S x i q x 10 x x 8 x x 1 x 2 x 6 x 5 0 x 3 x

24 Odd Cycles We also get a valid z-cut (bound on number of colors z): 1 q 1 z x i + 1 L ( L 1) qs i S 4 qs 1 = x i i S 4 3 This is facet defining. x 10 x x 8 x x 1 x 2 x 6 x 5 0 x 3 x

25 z-cuts in general In fact, facet-defining x-cuts for a graph coloring problem always give rise to facet-defining z-cuts: Theorem: if ax b is facet defining for a coloring problem with domain D = {0, δ, 2δ,, (n 1)δ} for δ > 0, then aez ax + b is also facet defining, where e = (1,, 1).

26 Mapping into 0-1 Space The finite-domain cuts map to i S j q 1 jy ij sq L ( L 1) 4 1 q 1 z jy ij + 1 L ( L 1) qs i S j 4 qs How do they compare with classical odd hole cuts?

27 Computed Bounds Lower bound on number of colors in 0-1 model of 5-cycle s = All odd hole cuts* x-cut only z-cut only x and z-cut only Optimal No. odd hole cuts ,480 78,125 * And clique inequalities

28 Computed Bounds Lower bound on number of colors in 0-1 model of 7-cycle s = All odd hole cuts* x-cut only z-cut only x and z-cut only Optimal No. odd hole cuts , ,752 * And clique inequalities

29 Computed Bounds Lower bound on number of colors in 0-1 model of 9-cycle s = All odd hole cuts* x-cut only z-cut only x and z-cut only Optimal No. odd hole cuts ,441 * And clique inequalities

30 Cuts in x-space Finite domain cuts can also be used in their original form. This results in a much more compact relaxation. O(n) variables rather than O(n 2 ) variables. Is the bound in the x-space as tight as in the 0-1 space?

31 Cuts in x-space Finite domain cuts can also be used in their original form. This results in a much more compact relaxation. O(n) variables rather than O(n 2 ) variables. Is the bound in the x-space as tight as in the 0-1 space? Yes.

32 Computed Bounds Lower bound on number of colors in x-model of 5-cycle s = Clique cuts only Plus x-cut Plus z-cut Plus x and z-cut Optimal

33 Computed Bounds Lower bound on number of colors in x-model of 7-cycle s = Clique cuts only Plus x-cut Plus z-cut Plus x and z-cut Optimal

34 Computed Bounds Lower bound on number of colors in x-model of 9-cycle s = Clique cuts only Plus x-cut Plus z-cut Plus x and z-cut Optimal 3 5 7

35 Odd Paths A q-path looks like alldiff alldiff alldiff alldiff alldiff x a x 1 x 2 x 3 x 4 x b q = 5 x 5 x 6 x 7 x 7

36 Odd Paths Select q + 1 variables: x a x 1 x 2 x 3 x 4 x b q = 5 x 5 x 6 x 7 x 7

37 Odd Paths This yields a valid inequality (x-cut) x a x 1 x 2 x 3 x 4 x b q = q 1 q ( x + x ) + x = 4 2 a b i i = 1

38 Odd Paths This yields a valid inequality (x-cut): x a x 1 x 2 x 3 x 4 x b q = q 1 q ( x + x ) + x = 4 2 a b i i = 1 The inequality is facet-defining if q is odd.

39 Odd Paths We also have a z-cut x a x 1 x 2 x 3 x 4 x b q = q z 2 ( x a x b ) x i q i = 1 2 This is also facet defining.

40 Mapping into 0-1 Space When mapped into 0-1 space, the finite domain cuts are redundant of the 0-1 model. They don t change the bound. However, the finite domain cuts provide a compact relaxation.

41 Webs A web W(q,k) is a cycle of q vertices in which edges connect all vertices separated by distance at least k. W(q,2) is an anti-hole. x 0 = 0 x 6 = 3 x 1 = 0 x 5 = 2 x 2 = 1 W(7,2) x 4 = 2 x 3 = 1

42 Webs If q and k are mutually prime, i where 1 x i rq ( r + 1 ) rk 2 r q = k is facet-defining. x 6 = 3 x 5 = 2 x 0 = 0 x 4 = 2 x 3 = 1 x i i 9 x 1 = 0 x 2 = 1

43 Webs If q and k are mutually prime, 1 k z x i + 1 ( r 1 ) r q + i 2 q x q 6 = 0 where r = k is facet-defining. x 5 = 1 x 0 = 3 x 4 = 1 x 3 = 2 x i i 9 x 1 = 3 x 2 = 2

44 Mapping into 0-1 Space Finite-domain web cuts compare similarly with finite-domain odd hole cuts.

45 Intersecting Systems An intersecting system looks something like V 2 S 3 S 1 T 2 T 3 U S 2 V 3 S = V \ V k l k l k T = V \ V k k U = V k k l k l T 1 q = 3 V 1

46 Intersecting Systems Facet-defining inequality. Let S = S k T = T u = U k k k V 2 q = 3 S 3 S 1 T 2 T 3 U T 1 V 1 S 2 V 3 q ( q 1) qs + u x + x b ( ) where 1 A valid inequality is: i i 2 i T i S U ( 1 )( )( 1 ) b = q q qs + u qs + u + 2

47 Intersecting Systems Facet-defining inequality. Let S = S k T = T u = U k k k V 2 q = 3 S 3 S 1 T 2 T 3 U T 1 V 1 S 2 V 3 2 q 1 z x + x + c q ( q + 1) i ( q + 1)( qs + u ) i where A valid bound is: i T i S U 1 q 1 c = qs + u + 2 q + 1 ( 1 )

48 Benchmark Instances with < 100 variables Instance Lower bound on number of colors in 0-1 model. Odd cycle cuts for s = 1,2,3 Odd hole Odd cycle Opt Odd hole time Odd cycle time 1-Fulllns_ Fulllns_ insertions_ Fulllns_ insertions_

49 Benchmark Instances Lower bound on number of colors in 0-1 model. Odd cycle cuts for s = 1,2,3 Instance Odd hole Odd cycle Opt Odd hole time Odd cycle time 3-Fulllns_ insertions_ insertions david huck jean

50 Benchmark Instances Lower bound on number of colors in 0-1 model. Odd cycle cuts for s = 1,2,3 Instance Odd hole Odd cycle Opt Odd hole time Odd cycle time mug88_ Mug88_

51 Benchmark Instances Lower bound on number of colors in 0-1 model. Odd cycle cuts for s = 1,2,3 Instance Odd hole Odd cycle Opt Odd hole time Odd cycle time myciel myciel myciel myciel

52 Benchmark Instances Lower bound on number of colors in 0-1 model. Odd cycle cuts for s = 1,2,3 Instance Odd hole Odd cycle Opt Odd hole time Odd cycle time queen5_ queen6_ queen7_ queen8_8? ? 3.4 queen8_ queen9_

53 Future Work Conduct polyhedral for other finite-domain formulations. Cumulative scheduling. Circuit constraint (TSP). Etc.

54 Bounds from Binary Decision Diagrams Joint work with David Bergman, Andre Cire, Willem van Hoeve

55 Binary Decision Diagrams BDDs historically used for circuit design and verification. Lee 1959, Akers 1978, Bryant 1986.

56 Binary Decision Diagrams BDDs historically used for circuit design and verification. Lee 1959, Akers 1978, Bryant Compact graphical representation of boolean function. Can also represent feasible set of problem with binary variables. Slight generalization (MDDs) represents finite domain variables.

57 Binary Decision Diagrams BDDs historically used for circuit design and verification. Lee 1959, Akers 1978, Bryant Compact graphical representation of boolean function. Can also represent feasible set of problem with binary variables. Slight generalization (MDDs) represents finite domain variables. BDD is result of superimposing isomorphic subtrees in a search tree. Unique reduced BDD for given variable ordering.

58 The 0-1 inequality 300 x x x x x x x x x x x x x x x x x x x has 117,520 minimal solutions

59 The 0-1 inequality 300 x x x x x x x x x x x x x x x x x x x has 117,520 minimal solutions The BDD has only 152 nodes. Paths from top to bottom right correspond to feasible solutions

60 Binary Decision Diagrams BDD can grow exponentially with problem size. So we use a smaller, relaxed BDD that represents superset of feasible set. Andersen, Hadzic, Hooker, Tiedemann For alldiff systems, reduced search tree from >1 million nodes to 1 node. Subsequent papers with Hadzic, Hoda, van Hoeve, O Sullivan. We focus on independent set problem on a graph

61 Independent Set Problem Let each vertex have weight w i Select nonadjacent vertices to maximize w i x i i

62 Exact BDD for independent set problem 4 x 1 x 2 x 3 x 4 x 5 x 6

63 Exact BDD for independent set problem 4 x 2 = 0 x 1 = 1 x 1 x 2 x 3 x 4 x 5 x 6

64 Paths from top to bottom correspond to the 11 feasible solutions 4 x 1 x 2 x 3 x 4 x 5 x 6

65 Paths from top to bottom correspond to the 11 feasible solutions 4 x 1 x 2 x 3 x 4 x 5 x 6

66 Paths from top to bottom correspond to the 11 feasible solutions 4 x 1 x 2 x 3 x 4 x 5 x 6

67 Paths from top to bottom correspond to the 11 feasible solutions 4 x 1 x 2 x 3 x 4 x 5 x 6

68 Paths from top to bottom correspond to the 11 feasible solutions and so forth 4 x 1 x 2 x 3 x 4 x 5 x 6

69 For objective function, associate weights with arcs 4 w w 2 w 4 w w 5 w 6 0 w 1 w w 5 x 1 x 2 x 3 x 4 x 5 x 6

70 For objective function, associate weights with arcs Optimal solution is longest path w 2 w 3 w4 w w 5 w 6 0 w 1 w w 5 x 1 x 2 x 3 x 4 x 5 x 6

71 Objective Function In general, objective function can be any separable function. Linear or nonlinear, convex or nonconvex.

72 To build BDD, associate state with each node 4 {123456} {23456} {3456} {35} {456} {4} {56} {5} {6} x 1 x 2 x 3 x 4 x 5 x 6

73 To build BDD, associate state with each node 4 {123456} x 1 x 2 x 3 x 4 x 5 x 6

74 To build BDD, associate state with each node 4 {123456} {23456} {35} x 1 x 2 x 3 x 4 x 5 x 6

75 To build BDD, associate state with each node 4 {3456} {123456} {23456} {4} {35} x 1 x 2 x 3 x 4 x 5 x 6

76 To build BDD, associate state with each node 4 {123456} {23456} {3456} {456} {4} {35} {5} x 1 x 2 x 3 x 4 x 5 x 6

77 To build BDD, associate state with each node 4 {123456} {23456} {3456} {35} {456} {4} {56} {5} {6} x 1 x 2 x 3 x 4 x 5 x 6

78 To build BDD, associate state with each node 4 {123456} {23456} {3456} {35} {456} {4} {56} {5} {6} x 1 x 2 x 3 x 4 x 5 x 6

79 Width = 2 To build BDD, associate state with each node 4 {123456} {23456} {3456} {35} {456} {4} {56} {5} {6} x 1 x 2 x 3 x 4 x 5 x 6

80 To build relaxed BDD, merge some nodes as we go along 4 {123456} x 1 x 2 x 3 x 4 x 5 x 6

81 To build relaxed BDD, merge some nodes as we go along 4 {123456} {23456} {35} x 1 x 2 x 3 x 4 x 5 x 6

82 To build relaxed BDD, merge some nodes as we go along 4 {3456} {123456} {23456} {4} {35} x 1 x 2 x 3 x 4 x 5 x 6

83 To build relaxed BDD, merge some nodes as we go along 4 {123456} {23456} {4} {3456} x 1 x 2 x 3 x 4 x 5 x 6

84 To build relaxed BDD, merge some nodes as we go along 4 {123456} {23456} {456} {4} {3456} x 1 x 2 x 3 x 4 x 5 x 6

85 To build relaxed BDD, merge some nodes as we go along 4 {123456} {23456} {456} {3456} x 1 x 2 x 3 x 4 x 5 x 6

86 To build relaxed BDD, merge some nodes as we go along 4 {123456} {23456} {456} {6} {3456} {56} x 1 x 2 x 3 x 4 x 5 x 6

87 To build relaxed BDD, merge some nodes as we go along 4 {123456} {23456} {456} {6} {3456} {56} x 1 x 2 x 3 x 4 x 5 x 6

88 Width = 1 To build relaxed BDD, merge some nodes as we go along 4 {123456} {23456} {456} {6} {3456} {56} x 1 x 2 x 3 x 4 x 5 x 6

89 Width = 1 Represents 18 solutions, including 11 feasible solutions 4 {123456} {23456} {456} {6} {3456} {56} x 1 x 2 x 3 x 4 x 5 x 6

90 Width = 1 Longest path gives bound of 3 on optimal value of 2 4 {123456} {23456} {456} {6} {3456} {56} x 1 x 2 x 3 x 4 x 5 x 6

91 Comparison with LP Bound Random and benchmark instances Compare with LP bound at root node in CPLEX Turn off presolve We don t use it. It makes CPLEX bound worse or at most 1 better. Use only clique cuts Other cuts improve bound at most 1 And require orders of magnitude more time S 2

92 Random instances Relative bound vs. density 100 vertices Max BDD width = 100 Time vs. density CPLEX BDDs CPLEX BDDs

93 Benchmark instances (DIMACS) Relative bound Max BDD width = 100 Time BDDs CPLEX CPLEX BDDs

94 Future Work More problems Assembly line sequencing Vehicle routing with time windows General BDD-based solver Branch in the BDD Combine with BDD-based propagation BDD-based bounds, primal heuristic No LP relaxation, cutting planes Linearity, convexity irrelevant But we need separable objective function