Lessons from MIP Search. John Hooker Carnegie Mellon University November 2009

Transcription

1 Lessons from MIP Search John Hooker Carnegie Mellon University November 2009

2 Outline MIP search The main ideas Duality and nogoods From MIP to AI (and back) Binary decision diagrams From MIP to constraint programming Banff workshop. Slide 2

3 Background reading MIP search General Survey J. T. Linderoth and M. W. P. Savelsbergh, A computational study of search strategies for mied integer programming, INFORMS Journal on Computing 11 (1999) Survey of Cutting Planes H. Marchand, A. Martin, R. Weismantel, and L. Wolsey. Cutting planes in integer and mied integer programming. Discrete Applied Mathematics, 123 (2002) Some Recent Developments M. Fischetti, F. Glover, A. Lodi, The feasibility pump, Mathematical Programming 104 (2005) M. Fischetti and A. Lodi, Repairing MIP infeasibility through local branching, Computers and Operations Research 35 (2008) , T. Sandholm, R. Shields, Nogood learning for mied integer programming, CMU, November 2006 Banff workshop. Slide (#)

4 Background reading BDDs Basic Ideas H. R. Andersen, T. Hadzic, J. N. Hooker, and P. Tiedemann, A constraint store based on multivalued decision diagrams, in C. Bessiere, ed., CP 2007, Refinements Tarik Hadzic and J. N. Hooker, Cost-bounded binary decision diagrams for 0-1 programming, in E. Loute and L. Wolsey, eds., CPAIOR 2007, T. Hadzic, J. N. Hooker, and P. Tiedemann, Approimate compilation of constraints into multivalued decision diagrams, in P. J. Stuckey, ed., CP 2007, Banff workshop. Slide (#)

5 MIP search MIP and the LP relaation Branch and bound Variable selection Node selection Feasibility heuristics Banff workshop. Slide 5

6 Mied integer programming We focus on mied integer/linear programming (MILP) min c A b, 0 Z, j J j integers Banff workshop. Slide 6

7 LP relaation An advantage of MIP: ready-made global relaation Linear programming (LP) problem min A j c b, 0 R, all j Drop integrality condition Banff workshop. Slide 7

8 LP relaation An advantage of MIP: ready-made global relaation Linear programming (LP) problem min c A b, 0 j R, all j Cutting planes can strengthen the relaation Drop integrality condition Banff workshop. Slide 8

9 LP relaation An advantage of MIP: ready-made global relaation Linear programming (LP) problem min A j c b, 0 R, all j Feasible set of LP relaation Banff workshop. Slide 9

10 LP relaation An advantage of MIP: ready-made global relaation Linear programming (LP) problem min A j c b, 0 R, all j Feasible set of MIP Banff workshop. Slide 10

11 LP relaation An advantage of MIP: ready-made global relaation Linear programming (LP) problem min c A b, 0 j R, all j Simple method finds a verte solution of the LP relaation Banff workshop. Slide 11

12 Branch and bound Used for 50 years Land and Doig Creates search tree by branching on variables Solves continuous relaation (LP) at each node Basic search framework for all general-purpose MIP codes Branch and cut adds cutting planes at some nodes. Banff workshop. Slide 12

13 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Initially, there is one open node (the original problem)

14 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Initially, there is one open node (the original problem) Optimal value of LP relaation is lower bound on optimal value of original problem

15 Branch and bound - illustration Because solution is fractional, branch on a fractional variable Root node = (2, 3.5, 4.6) LB = Remember: we are minimizing

16 Branch and bound - illustration Because solution is fractional, branch on a fractional variable Root node = (2, 3.5, 4.6) LB = Close after branching Two new open nodes

17 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Integer solution = (2, 3, 4) UB = 170 Select open node and solve LP

18 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Integer solution = (2, 3, 4) UB = 170 Select open node and solve LP Close node, because solution is integer

19 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Integer solution = (2, 3, 4) UB = 170 Select open node Close node, and solve LP because solution is integer Objective function value is upper bound on optimal value

20 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Integer solution = (2, 3, 4) UB = 170 = (2.3, 4, 4.7) LB = 165 Incumbent solution Select open node and solve LP

21 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Incumbent solution Integer solution = (2, 3, 4) UB = 170 = (2.3, 4, 4.7) LB = 165 Must branch again

22 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Incumbent solution Integer solution = (2, 3, 4) UB = 170 = (2.3, 4, 4.7) LB = Select open node = (2, 4.1, 4.7) LB = 171

23 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Incumbent solution Integer solution = (2, 3, 4) UB = 170 = (2.3, 4, 4.7) LB = 165 Close node, because relaation value is no better than incumbent = (2, 4.1, 4.7) LB = 171

24 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Incumbent solution Integer solution = (2, 3, 4) UB = 170 = (2.3, 4, 4.7) LB = 165 Select open node = (2, 4.1, 4.7) LB = 171 Integer solution = (3, 4, 5) LB = UB = 169

25 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Incumbent solution Integer solution = (2, 3, 4) UB = 170 = (2.3, 4, 4.7) LB = Integer solution is better than incumbent, becomes new incumbent = (2, 4.1, 4.7) LB = 171 Integer solution = (3, 4, 5) LB = UB = 169

26 Branch and bound - illustration Root node = (2, 3.5, 4.6) LB = 160 Remember: we are minimizing Integer solution = (2, 3, 4) UB = 170 = (2.3, 4, 4.7) LB = 165 No remaining open nodes. Incumbent is optimal = (2, 4.1, 4.7) LB = 171 Integer solution = (3, 4, 5) LB = UB = 169

27 Branch and bound procedure Let original problem be an open node Let UB = While open nodes remain Select an open node N and solve its continuous relaation R Let LB = value of R and close N If solution of R is integer then If LB < UB then let UB = LB Else if LB < UB then Branch on fractional variable and create 2 new open nodes Banff workshop. Slide 27

28 Why it works Luck Most variables tend to be integer in solution of relaation. Banff workshop. Slide 28

29 Why it works Luck Most variables tend to be integer in solution of relaation. Global continuous relaation Cutting planes, good formulations Banff workshop. Slide 29

30 Why it works Luck Most variables tend to be integer in solution of relaation. Global continuous relaation Cutting planes, good formulations Presolve Bag of tricks to simplify problem Banff workshop. Slide 30

31 Why it works Luck Most variables tend to be integer in solution of relaation. Global continuous relaation Cutting planes, good formulations Presolve Bag of tricks to simplify problem Feasibility heuristics Find feasible solutions before branching Banff workshop. Slide 31

32 Why it works Luck Most variables tend to be integer in solution of relaation. Global continuous relaation Cutting planes, good formulations Presolve Bag of tricks to simplify problem Feasibility heuristics Find feasible solutions before branching Good engineering 50 years of eperience Banff workshop. Slide 32

33 Basic issues Variable selection Branch on which variable at current node? Node selection Process which node net? Feasibility heuristics How to find integer solutions early? to get good upper bounds Banff workshop. Slide 33

34 Variable selection Goal is to tighten lower bound after branching We don t know how to increase probability of finding integer solutions. Pseudocosts Estimate change in objective function after branching Strong branching Put bound on change in objective function after branching Banff workshop. Slide 34

35 Pseudocosts Pseudocost = rate of increase of LP lower bound when fractional variable j is rounded up or down Current LP bound j = LB j f LB j + LB + j j 1 f j = LB Fractional part of j Banff workshop. Slide 35

36 Pseudocosts Pseudocost = rate of increase of LP lower bound when fractional variable j is rounded up or down Bound after branching down Current LP bound Bound after branching up j = LB j f LB j + LB + j j 1 f j = LB Fractional part of j Banff workshop. Slide 36

37 Pseudocosts Using pseudocosts One method: branch on variable j with largest + j f j + j f j (1 ) j = LB j f LB j + LB + j j 1 f j = LB Banff workshop. Slide 37

38 Pseudocosts Computing pseudocosts at root node + One method: Eplicitly compute LB for fractional j, LBj variables by solving 2 LPs. Perhaps with limited number of simple iterations. j = LB j f LB j + + j j 1 f j = LB LB Banff workshop. Slide 38

39 Pseudocosts Computing pseudocosts lower in the tree For variables not yet branched on or evaluated: + Eplicitly compute LBj, LBj For variables previously branched on:, + Use average of values obtained after previous j j branches. j = LB j f LB j + LB + j j 1 f j = LB Banff workshop. Slide 39

40 Strong branching We need the concept of LP duality Will eplain origin of dual later ma ua u i ub c, u 0 R, all i min c A j b, 0 R, all j Dual problem Primal problem Banff workshop. Slide 40

41 LP dual ma ua u i Dual ub c, u 0 R, all i min c A j Primal b, 0 R, all j Objective function value Dual feasible solutions Primal feasible solutions Weak duality Banff workshop. Slide 41

42 LP dual ma ua u i Dual ub c, u 0 R, all i min c A j Primal b, 0 R, all j Objective function value When primal and dual feasible solutions have the same value, both are optimal Banff workshop. Slide 42 Strong duality

43 LP dual ma ua u i Dual ub c, u 0 R, all i min c A j Primal b, 0 R, all j Objective function value Simple method maintains primal feasibility and strives for dual feasibility Banff workshop. Slide 43

44 LP dual ma ua u i Dual ub c, u 0 R, all i min c A j Primal b, 0 R, all j Objective function value Dual simple method maintains dual feasibility and strives for primal feasibility Simple method maintains primal feasibility and strives for dual feasibility Banff workshop. Slide 44

45 Dual simple Dual simple is used to reoptimize after adding a constraint such as new bound on variable after branching. Solution remains dual feasible after adding constraint. Because this adds a variable in the dual. Dual simple restores primal feasibility. Premature termination of dual simple provides lower bound on optimal value. Due to weak duality. Banff workshop. Slide 45

46 Strong branching For fractional variables j : Compute lower bounds j j on increase in LP value after branching down or up. Branch on j with largest L, L + j j Obtain by running a few iterations of dual simple. Perhaps only one. L, L + L + L + j j To save time, evaluate only a few fractional variables. Banff workshop. Slide 46

47 Node selection Select which of the open nodes to process. Goal is to find good feasible solution, or prove that there is no solution better than incumbent. In either case, it is reasonable to look at quality of LP bounds. Two common methods: Depth first Best bound Banff workshop. Slide 47

48 Depth first Advantages: Easy to implement. Fast reoptimization of LP. Minimal memory requirements. Disadvantages: Blind to what is going on. Must search entire subtree before moving to a more promising subtree. Tends to eplode in hard problems. Banff workshop. Slide 48

49 Best bound Select open node with best LP value. For a given branching rule, this minimizes number of nodes processed. Banff workshop. Slide 49

50 Best bound LB = 10 Complete search tree for a given variable selection strategy LB = 13 LB = 11 LB = 24 LB = 15 LB = 22 LB = 12 LB = 20 LB = 16 LB = 18 LB = 14 Banff workshop. Slide 50

51 Best bound LB = 10 Complete search tree for a given variable selection strategy LB = 13 LB = 11 LB = 24 LB = 15 LB = 22 LB = 12 LB = 20 LB = 16 LB = 18 LB = 14 Banff workshop. Slide 51 Leaf nodes represent integer solutions

52 Best bound Black = open node LB = 10 Select open node with best bound. LB = 13 LB = 11 LB = 24 LB = 15 LB = 22 LB = 12 LB = 20 LB = 16 LB = 18 LB = 14 Banff workshop. Slide 52 Leaf nodes represent integer solutions

53 Best bound LB = 10 Select node with best bound. We are in the right subtree. LB = 13 LB = 11 LB = 24 LB = 15 LB = 22 LB = 12 LB = 20 LB = 16 LB = 18 LB = 14 Optimal solution Banff workshop. Slide 53

54 Best bound LB = 10 Select node with best bound. We are still in the right subtree. LB = 13 LB = 11 LB = 24 LB = 15 LB = 22 LB = 12 LB = 20 LB = 16 LB = 18 LB = 14 Optimal solution Banff workshop. Slide 54

55 Best bound LB = 10 Wrong subtree, but we would have to select this node in any method LB = 13 LB = 11 LB = 24 LB = 15 LB = 22 LB = 12 LB = 20 LB = 16 LB = 18 LB = 14 Optimal solution Banff workshop. Slide 55

56 Best bound LB = 10 Net choice is optimal solution. LB = 13 LB = 11 LB = 24 LB = 15 LB = 22 LB = 12 LB = 20 LB = 16 LB = 18 LB = 14 Optimal solution Banff workshop. Slide 56

57 Best bound LB = 10 Net choice is optimal solution. LB = 13 LB = 11 No need to consider other open nodes. LB = 24 LB = 15 LB = 22 LB = 12 LB = 20 LB = 16 LB = 18 LB = 14 Optimal solution Banff workshop. Slide 57

58 Feasibility heuristics We wish find good integer solutions as early as possible. To get useful upper bounds. Two approaches (among others): Local branching. Feasibility pump. Banff workshop. Slide 58

59 Local branching Must begin with known integer solution Two level branching procedure Strategic branches define neighborhoods of known integer solutions. Low-level branching searches each neighborhood using MIP Banff workshop. Slide 59

60 Local branching Assume integer variables are binary. + (1 ) k j j: v = 0 j: v = 1 j j j Current node + (1 ) > k j j: v = 0 j: v = 1 j j j Solutions within k-neighborhood of known integer solution v Strategic branch Other solutions Banff workshop. Slide 60

61 Local branching Assume integer variables are binary. + (1 ) k j j: v = 0 j: v = 1 j j j Current node + (1 ) > k j j: v = 0 j: v = 1 j j j Solutions within k-neighborhood of known integer solution v Strategic branch Other solutions Solve this MIP to optimality. Perhaps obtain better integer solutions. Banff workshop. Slide 61

62 Local branching Assume integer variables are binary. + (1 ) k j j: v = 0 j: v = 1 j j j Current node + (1 ) > k j j: v = 0 j: v = 1 j j j Solutions within k-neighborhood of known integer solution v Strategic branch Other solutions Solve this MIP to optimality. Perhaps obtain better integer solutions. Repeat process with another integer solution v Banff workshop. Slide 62

63 Feasibility pump Can find initial integer solution Alternate between integrality and linear feasibility Solve LP Round to integer Find closest LP solution Banff workshop. Slide 63

64 Feasibility pump Solve LP relaation Banff workshop. Slide 64

65 Feasibility pump Round to nearest integer Banff workshop. Slide 65

66 Feasibility pump Round to nearest integer Banff workshop. Slide 66

67 Feasibility pump Find closest LP-feasible point using L 1 norm Banff workshop. Slide 67

68 Feasibility pump Find closest LP-feasible point using L 1 norm Banff workshop. Slide 68 This problem is itself an LP

69 Feasibility pump Round to nearest integer Banff workshop. Slide 69

70 Feasibility pump Round to nearest integer Procedure terminates Banff workshop. Slide 70

71 Feasibility pump Round to nearest integer Use random perturbations to break out of loops Procedure terminates Banff workshop. Slide 71

72 Duality and Nogoods Inference duality Nogood-based search Eample: SAT Eample: MIP Banff workshop. Slide 72

73 Inference duality Duality in optimization is closely related to nogood-based search in AI. Nogood = set of solutions to avoid in the rest of the search. All optimization duals are inference duals LP, Lagrangean, surrogate, subadditive, etc. Solution of inference dual is proof of optimality Dual solution provides nogoods Including Benders cuts, conflict clauses for SAT Nogood = conditions under which proof is still valid Slide 73

74 Inference duality Primal min f ( ) S Feasible set ma v P S v f ( ) P Dual P Family of proofs (inference method) Follows using proof P Dual is defined relative to an inference method. Strong duality applies if inference method is complete. Slide 74

75 Eample: LP dual Primal min c A b 0 ma Dual v A b P c v 0 P P = ma λb λa c λ 0 Nonnegative linear combination + domination 0 λa λb dominates c v A b c v iff for some λ 0 λa c and λb v Slide 75 Inference method is complete (assuming feasibility) due to Farkas Lemma. So we have strong duality (assuming feasibility).

76 Duals for MIP Lagrangean dual Inference = same as LP Inference method incomplete duality gap Surrogate dual Inference = same as LP, ecept domination is weaker Therefore smaller duality gap than Lagrangean dual Subadditive dual Inference = subadditive, homogeneous function + domination Inference method complete (cf. Gomory s method) no gap Slide 76

77 Nogood-based search All search is nogood-based search Each solution eamined generates a nogood. Net solution must satisfy current nogood set. Nogoods are derived by solving inference dual of the subproblem. Subproblem is normally defined by fiing variables to current values in the search. Slide 77

78 Eample: Logic-based Benders Partition variables,y and search over values of Subproblem results from fiing min f (, y ) min f (, y ) (, y ) S (, y ) S Let proof P be solution of subproblem dual for = Let B(P,) be lower bound obtained by P for given. Add Benders cut v B(P,) to master problem: Solve master problem for net min v Benders cuts Slide 78

79 Eample: Logic-based Benders Master and subproblem may be solved by different techniques MIP for master problem CP for subproblem Benders cuts are problem-specific Based on type of inference in dual Some applications Planning & scheduling (e.g., assembly lines) Computer processor allocation and scheduling Chemical batch sizing and scheduling Sports scheduling Location/allocation Slide 79

80 Eample: Classical Benders Partition variables,y and search over values of min f ( ) + cy g( ) + Ay b D, 0 y Subproblem results from fiing k min f ( ) + cy k Ay b g( ) y 0 ( λ) Let proof λ be solution of subproblem dual for Let B(λ,) = f() + λ(b g()) be bound obtained by λ for given. Add Benders cut v B(λ,) to master problem: Solve master problem for net = min v Benders cuts Slide 80

81 Eample: SAT Solve SAT by chronological backtracking + unit clause rule = DPL. Chronological = fied branching order. To get nogood, solve subproblem at current node. Solve with unit clause rule Nogood identifies branches that create infeasibility. Simplest scheme: nogood rules out path to current leaf node. Process nogood set with parallel resolution Nogood set is a relaation of the problem. Solve relaation without branching Select solution with preference for 0 Slide 81

82 DPL with chronological backtracking 1 = 0 2 = 0 3 = 0 5 = 0 4 = 0 Branch to here. Solve subproblem with unit clause, which proves infeasibility. ( 1, 5 ) = (0,, 0) creates the infeasibility. Banff workshop. Slide 82

83 DPL with chronological backtracking 1 = 0 2 = 0 3 = 0 5 = 0 4 = Branch to here. Solve subproblem with unit clause, which proves infeasibility. ( 1, 5 ) = (0,, 0) creates the infeasibility. Generate nogood. Banff workshop. Slide 83

84 DPL with chronological backtracking 1 = 0 Consists of processed nogoods 3 = 0 2 = 0 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) k k = 0 5 = 0 Conflict clause appears as nogood induced by solution of R k Banff workshop. Slide 84

85 DPL with chronological backtracking 1 = 0 Consists of processed nogoods 3 = 0 2 = 0 k Relaation R Solution of R Nogoods k 0 (0,0,0,0,0, ) (0,0,0,0,1, ) k 5 = 0 4 = 0 5 = 1 Go to solution that solves relaation, with priority to 0 Banff workshop. Slide 85

86 DPL with chronological backtracking 1 = 0 Consists of processed nogoods 3 = 0 2 = 0 k Relaation R Solution of R Nogoods k 0 (0,0,0,0,0, ) (0,0,0,0,1, ) k 5 = 0 4 = 0 5 = Process nogood set with parallel resolution parallel-absorbs Banff workshop. Slide 86

87 DPL with chronological backtracking 1 = 0 Consists of processed nogoods 5 = 0 4 = 0 3 = 0 2 = 0 5 = 1 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) k k Process nogood set with parallel resolution parallel-absorbs Banff workshop. Slide 87

88 DPL with chronological backtracking 1 = 0 Consists of processed nogoods 5 = 0 4 = 0 3 = 0 2 = 0 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) 2 (0,0,0,1,0, ) k Solve relaation again, continue. So backtracking is nogood-based search with parallel resolution k Banff workshop. Slide 88

89 Eample: SAT and conflict clauses Nogoods = conflict clauses. Nogoods rule out only branches that play a role in unit clause refutation. Slide 89

90 DPL with conflict clauses 1 = 0 2 = 0 3 = 0 4 = 0 = 0 5 Branch to here. Unit clause rule proves infeasibility. ( 1, 5 ) = (0,0) is only premise of unit clause proof. Banff workshop. Slide 90

91 DPL with conflict clauses 1 = 0 3 = 0 2 = 0 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) k k = 0 5 = 0 Conflict clause appears as nogood induced by solution of R k. 1 5 Banff workshop. Slide 91

92 DPL with conflict clauses 1 = 0 Consists of processed nogoods 3 = 0 2 = 0 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) k 1 (0,0,0,0,1, ) k 4 = 0 5 = 0 5 = Banff workshop. Slide 92

93 DPL with conflict clauses 1 = 0 Consists of processed nogoods 4 = 0 3 = 0 2 = 0 5 = 0 5 = k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) k parallel-resolve to yield 1 2 parallel-absorbs k Banff workshop. Slide 93

94 DPL with conflict clauses 1 = 0 2 = 0 2 = 1 4 = 0 3 = 0 5 = 0 = Banff workshop. Slide k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) 2 (0,1,,,, ) k parallel-resolve to yield k

95 DPL with conflict clauses 1 = 0 1 = 1 4 = 0 3 = 0 2 = 0 5 = 0 5 = Banff workshop. Slide k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) 2 (0,1,,,, ) 3 (1,,,,, ) 4 k Search terminates k

96 Eample: SAT and partial-order dynamic backtracking Solve relaation by selecting a solution that conforms to nogoods. Conform = takes opposite sign than in nogoods. More freedom than in branching. Slide 96

97 Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) k k 5 1 Arbitrarily choose one variable to be last Banff workshop. Slide 97

98 Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) k k 5 1 Other variables are penultimate Arbitrarily choose one variable to be last Banff workshop. Slide 98

99 Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) k k 5 1 Since 5 is penultimate in at least one nogood, it must conform to nogoods. It must take value opposite its sign in the nogoods. 5 will have the same sign in all nogoods where it is penultimate. Banff workshop. Slide 99 This allows more freedom than chronological backtracking.

100 Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) 2 k k Choice of last variable is arbitrary but must be consistent with partial order implied by previous choices. Since 5 is penultimate in at least one nogood, it must conform to nogoods. It must take value opposite its sign in the nogoods. 5 will have the same sign in all nogoods where it is penultimate. Banff workshop. Slide 100 This allows more freedom than chronological backtracking.

101 Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) k k Choice of last variable is arbitrary but must be consistent with partial order implied by previous choices Parallel-resolve to yield 5 Banff workshop. Slide 101 Since 5 is penultimate in at least one nogood, it must conform to nogoods. It must take value opposite its sign in the nogoods. 5 will have the same sign in all nogoods where it is penultimate. This allows more freedom than chronological backtracking.

102 Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) 2 (,0,,,1, ) k k 5 does not parallel-resolve with because 5 is not last in both clauses 5 2 Banff workshop. Slide 102

103 Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) 2 (,0,,,1, ) (,1,,,1, ) 4 k k 2 Search terminates Must conform Banff workshop. Slide 103

104 Eample: MIP Nogoods now being used in MIP For eample, in SIMPL and SCIP. So we come full circle MIP AI MIP. Slide 104

105 Binary Decision Diagrams Motivating idea Among constraint BDDs Propagation through a BDD Computational results Banff workshop. Slide 105

106 Motivating idea Domain store is key element of constraint programming (CP) Consists of current variable domains. Domain store propagates results of filtering domains. Constraint-specific filtering algorithms remove infeasible values from domains. Reduced domains are handed over to net constraint. Slide 106

107 Motivating idea Problem: Domain store is a weak relaation It encodes limited information. Not worth investing time to process nodes of search tree. LP is a strong relaation for MIP It pays to invest time at nodes (solve LP, add cutting planes) Needed: A strong, discrete relaation for CP Adaptable to any constraint Domain store should be a degenerate case Slide 107

108 Among constraint We illustrate the idea using the among constraint. Important in sequencing and scheduling. ( X Sl u) among,,, Set of variables Nonnegative integers Set of values Slide 108

109 Among constraint We illustrate the idea using the among constraint. Important in sequencing and scheduling. ( X Sl u) among,,, Set of variables Nonnegative integers Set of values The constraint requires that at least l and at most u variables in X take a value in S Slide 109

110 Binary decision diagrams A BDD is a graphical representation of a boolean function It can also represent constraint on binary variables. The boolean function is 1 when the constraint is satisfied. A (reduced) BDD is basically a compact representation of a branching tree. Superimpose isomorphic subtrees. Remove unnecessary nodes. There is a unique reduced BDD for a given branching order. Generalizes to multivalued decision diagrams (MDDs) for arbitrary discrete variables. Slide 110

111 Search tree for ( ) among {,, },{1},2, Infeasible Feasible

112 ( ) among {,, },{1},2, Banff workshop. Slide 112 Remove infeasible paths

113 ( ) among {,, },{1},2, Banff workshop. Slide 113 Remove infeasible paths

114 ( ) among {,, },{1},2, Banff workshop. Slide 114 Merge isomorphic subtrees

115 ( ) among {,, },{1},2, {0,1} {0,1} Banff workshop. Slide 115 Merge isomorphic subtrees

116 ( ) among {,, },{1},2, {0,1} {0,1} Banff workshop. Slide 116 Can also remove redundant nodes

117 ( ) among {,, },{1},2, Banff workshop. Slide 117 Can also remove redundant nodes

118 ( ) among {,, },{1},2, {0,1} {0,1} Banff workshop. Slide 118 But this is unnecessary.

119 ( ) among {,, },{1},2, {0} {1} 2 {0,1} {0,1} 3 {0} {1} {0} 4 {1} 1 {0} Each path corresponds to a cartesian product of solutions. {0} {0,1} {0} {1} Banff workshop. Slide 119

120 New constraint: ( ) among {,,, },{1},2, {0} u 1 {1} 2 {0} u 2 u 3 {1} {0} {1} 3 u 4 u 5 u 6 4 {1} {0} {1} {0} u 7 u 8 {1} {0} 1 Banff workshop. Slide 120 Eact BDD representation: 6 solutions

121 ( ) among {,,, },{1},2, {0,1} u 1 2 u 2 {0} {1} 3 u 3 u 4 {0,1} {0} {1} 4 u 5 u 6 {0,1} {0} 1 Banff workshop. Slide 121 Relaed BDD of width 2: 14 solutions

122 ( ) among {,,, },{1},2, {0,1} u 1 2 {0,1} u 2 3 {0,1} u 3 4 {0,1} u 4 Relaed BDD of width 1 is just the domain store 1

123 Propagation through a BDD At each node of the search tree: Apply each constraint to current BDD. Refine BDD to incorporate relaation of the constraint. Refine by splitting nodes, subject to ma width. Width limit keeps BDD size within bounds. Greater width yields tighter relaation. Slide 123

124 Propagation of among through a BDD 1 {0,1} u 1 2 {0,1} u 2 3 {0,1} u 3 4 {0,1} u 4 Arbitrarily start with BDD of width 1 and refine it 1

125 ( ) among {,,, },{1},2, u 1 {0,1} 2 {0,1} u 2 Two incoming edges are not equivalent. So split u 2 3 {0,1} u 3 4 {0,1} u 4 1 Banff workshop. Slide 125 Use a maimum width of 3

126 ( ) among {,,, },{1},2, u 1 {0} {1} 2 u 2 {0,1} u 2 Two incoming paths are not equivalent. So split u 2 3 {0,1} u 3 4 {0,1} u 4 1 Banff workshop. Slide 126 Use a maimum width of 3

127 ( ) among {,,, },{1},2, u 1 {0} {1} 2 u 2 {0,1} u 2 {0,1} Duplicate outgoing edges 3 u 3 {0,1} 4 u 4 {0,1} 1 Banff workshop. Slide 127 Use a maimum width of 3

128 ( ) among {,,, },{1},2, {0} u 1 {1} 2 u 2 u 2 3 {0,1} {0,1} u 3 {0,1} 3 incoming paths are nonequivalent 4 {0,1} u 4 1 Banff workshop. Slide 128 Use a maimum width of 3

129 ( ) among {,,, },{1},2, u 1 {0} {1} 2 {0} u 2 {1} {0} u 2 {1} 4 3 u 3 u 3 u 3 {0,1} {0,1} {0,1} {0,1} u 4 Split into 3 Banff workshop. Slide 129 Use a maimum width of 3 1

130 ( ) among {,,, },{1},2, u 1 {0} {1} 2 {0} u 2 {1} {0} u 2 {1} 4 3 u 3 u 3 u 3 {0,1} {0,1} {0,1} {0,1} u 4 Filter edge domains Banff workshop. Slide 130 Use a maimum width of 3 1

131 ( ) among {,,, },{1},2, u 1 {0} {1} 2 {0} u 2 {1} {0} u 2 {1} 4 3 u 3 u 3 u 3 {0,1} {1} {0} {0,1} u 4 Filter edge domains Banff workshop. Slide 131 Use a maimum width of 3 1

132 ( ) among {,,, },{1},2, u 1 {0} {1} 2 {0} u 2 u 3 {1} {0} {1} 3 u 4 u 5 u 6 4 {1} {0} {0} {1} u 7 u 8 {1} {0} Another split generates eact BDD, which has width 3 1 Banff workshop. Slide 132 Use a maimum width of 3

133 Computational results Multiple all-different constraints Multivalued variables encoded as binary variables Using domain propagation only: Some instances has search tree of > 1 million nodes Using BDD relaation: All instances solved at the root node. Slide 133

134 Number of branches + number of splits Domain store BDD-based Banff workshop. Slide 134 Ma width of BDD

135 Computation time (milliseconds) Domain store MDD-based Banff workshop. Slide 135 Ma width of MDD

136 Computational results Multiple among constraints Nurse scheduling problems When checking for equivalence of incoming paths Paths are viewed as nonequivalent if distance measure eceeds a threshold Results shown for finding first feasible solution Results are very similar for finding all feasible solutions Slide 136

137 Domain store only Search tree nodes 1.E+05 Class 2 (n=80) Search Tree Nodes 1.E+04 1.E+03 1.E+02 Threshold 1 Threshold 2 Threshold 3 Threshold 4 Threshold 5 1.E Banff workshop. Slide 137 Maimum Width

138 Domain store only Computation time Class 2 (n=80) Time (sec) Threshold 1 Threshold 2 Threshold 3 Threshold 4 Threshold Banff workshop. Slide 138 Maimum Width