Integrating Solution Methods. through Duality. US-Mexico Workshop on Optimization. Oaxaca, January John Hooker

Transcription

1 Integrating Solution Methods through Duality John Hooker US-Meico Workshop on Optimization Oaaca, January 2011

2 Zapotec Duality mask Late classical period Represents life/death, day/night, heat/cold See a similar mask in Centro Cultural Santo Domingo Slide 2

3 Duality Two perspectives on the same phenomenon Lalita Mahatripurasundari representing Brahman-Atman consciousness (mind/body) Slide 3 Complementarity Mingling of opposites Yin/yang Daoism Alharamazda Zoroastrian god Good/evil

4 Goal Design a general-purpose solver that eploits problem-specific structure. Slide 4

5 Goal Design a general-purpose solver that eploits problem-specific structure. Approach: Find common algorithmic structure across solution methods Adjust specific elements of this structure to suit the problem. Slide 5

6 Goal Design a general-purpose solver that eploits problem-specific structure. Approach: Find common algorithmic structure across solution methods Adjust specific elements of this structure to suit the problem. Thesis: Successful combinatorial optimization methods use a primal-dual-dual solution strategy. The duals can be designed to eploit problem structure. This perspective can unify methods and lead to new ones. Slide 6

7 Outline Two dualities Eample: Integrated approach to IP Focus on inference duality Eample: SAT Eamples of integrated solving Piecewise linear optimization Planning and disjunctive scheduling Planning and cumulative scheduling Single-facility scheduling Truss structure design (nonlinear) Conclusion Slide 7

8 Surviving this talk We must cover a wide variety of problems to make the point. Focus on the structural parallels Rather than the details of each problem Slide 8

9 Two dualities Inference and relaation can eploit problem structure. They work together with search in most combinatorial optimization methods. Search, inference, and relaation are linked by two underlying dualities: Duality of search and inference Duality of search and relaation Slide 9

10 Two ways to eploit structure Inference Use knowledge of problem structure to reveal hidden information. This can eclude unpromising areas of the search space.. Search space Reduced search space Solution Inferred constraint Slide 10

11 Two ways to eploit structure Relaation Use knowledge of problem structure to design a larger but simpler search space. Solution of relaation may be near solution of original problem and provides a bound. Search space Relaed search space Solution Solution of relaation Slide 11

12 Two dualities Duality of search and inference. Search looks for a certificate of feasibility. The inference dual looks for a certificate (proof) of infeasibility (or optimality). Duality of search and relaation. Search enumerates restrictions of the problem. The relaation dual enumerates (parameterized) relaations of the problem. Slide 12

13 Primal-dual-dual methods Primal methods. Enumerate possible solutions (in general, problem restrictions). Dual methods. Enumerate proofs or relaations. Primal-dual methods. Solve the primal and a dual simultaneously. Primal-dual-dual methods. Solve the primal and both duals simultaneously. Strategy of most successful combinatorial optimization methods. Slide 13

14 Classical solution methods CP solver Search: Branching Inference: Flltering Relaation: Domain store MILP solver Search: Branching Inference: Cutting planes, presolve, reduced cost variable fiing Relaation: LP, Lagrangean Benders Search: Enumerate subproblems. Inference: Benders cuts Relaation: Master problem Slide 14

15 Classical solution methods Global optimization Search: Enumerate boes Inference: Domain reduction, dual-based variable bounding Relaation: Conveification SAT Search: Branching, dynamic backtracking, etc. Inference: Conflict clauses Relaation: Same as restriction Local search Search: Enumerate neighborhoods. Inference: Tabu list, etc. Relaation: Same as restriction Slide 15

16 Eample: Integrated approach to IP min i {0,1,2,3} 8 Slide 16

17 Inference: Bounds propagation Simple constraint programming technique min i {0,1,2,3} = 7 1 Slide 17

18 Bounds propagation Simple constraint programming technique Reduced domain min {1,2,3},,, {0,1,2,3} = 7 1 Slide 18

19 Bounds propagation Simple constraint programming technique Reduced domain min {1,2,3},,, {0,1,2,3} = 7 1 In general: Bounds propagation aims for bounds consistency Slide 19 Domain filtering aims for hyperarc consistency (GAC)

20 Relaation: Linear programming min , 1 i 1 8 Replace domains with bounds Slide 20

21 Inference: cutting planes (valid inequalities) min , 1 i 1 8 We can create a tighter relaation with the addition of cutting planes. Slide 21

22 Inference: cutting planes (valid inequalities) min , 1 i 1 8 {1,2} is a packing because alone cannot satisfy the inequality, even with 1 = 2 = 3. Slide 22

23 Inference: cutting planes (valid inequalities) min , 1 i 1 8 {1,2} is a packing So, ( ) which implies 42 ( ) = ma { 4,3 } General integer knapsack cut 2 Slide 23

24 Slide 24 Cutting planes (valid inequalities) min , 1 i 1 8 Maimal Packings Knapsack cuts {1,2} {1,3} {1,4} Knapsack cuts corresponding to nonmaimal packings can be nonredundant.

25 Slide 25 Continuous relaation with cuts min , i Knapsack cuts Optimal value of is a lower bound on optimal value of original problem.

26 Primal-dual-dual method Search: branching Inference: Bounds propagation, cutting planes Relaation: LP, domain store Slide 26

27 Primal-dual-dual method 1 { 123} 2 {0123} 3 {0123} 4 {0123} = (2⅓,3,2⅔,0) value = 523⅓ Propagate bounds and solve relaation. Since solution of relaation is infeasible, branch. Slide 27

28 Primal-dual-dual method Branch on a variable with nonintegral value in the relaation. Slide 28 1 { 123} 2 {0123} 3 {0123} 4 {0123} = (2⅓,3,2⅔,0) value = 523⅓ 1 {1,2} 1 = 3

29 Primal-dual-dual method Propagate bounds and solve relaation. Since relaation is infeasible, backtrack. 1 { 12 } 2 { 23} 3 { 123} 4 { 123} infeasible relaation Slide 29 1 { 123} 2 {0123} 3 {0123} 4 {0123} = (2⅓,3,2⅔,0) value = 523⅓ 1 {1,2} 1 = 3

30 Primal-dual-dual method Propagate bounds and solve relaation. Branch on nonintegral variable. 1 { 12 } 2 { 23} 3 { 123} 4 { 123} infeasible relaation Slide 30 1 { 123} 2 {0123} 3 {0123} 4 {0123} = (2⅓,3,2⅔,0) value = 523⅓ 1 {1,2} 1 = 3 1 { 3} 2 {0123} 3 {0123} 4 {0123} = (3,2.6,2,0) value = = 3 2 {0,1,2}

31 Primal-dual-dual method Branch again. 1 { 12 } 2 { 23} 3 { 123} 4 { 123} infeasible relaation Slide 31 1 { 123} 2 {0123} 3 {0123} 4 {0123} = (2⅓,3,2⅔,0) value = 523⅓ 1 {1,2} 1 = 3 1 { 3} 2 {012 } 3 { 123} 4 {0123} = (3,2,2¾,0) value = 527½ 3 {1,2} 3 = 3 1 { 3} 2 {0123} 3 {0123} 4 {0123} = (3,2.6,2,0) value = = 3 2 {0,1,2}

32 Primal-dual-dual method Solution of relaation is integral. This becomes the incumbent solution. 1 { 12 } 2 { 23} 3 { 123} 4 { 123} infeasible relaation 1 { 123} 2 {0123} 3 {0123} 4 {0123} = (2⅓,3,2⅔,0) value = 523⅓ 1 {1,2} 1 = 3 1 { 3} 2 {012 } 3 { 123} 4 {0123} = (3,2,2¾,0) value = 527½ Slide 32 1 { 3} 2 { 12 } 3 { 12 } 4 { 123} = (3,2,2,1) value = 530 feasible solution 3 {1,2} 3 = 3 1 { 3} 2 {0123} 3 {0123} 4 {0123} = (3,2.6,2,0) value = = 3 2 {0,1,2}

33 Primal-dual-dual method Solution is nonintegral, but use bound to backtrack, 1 { 12 } 2 { 23} 3 { 123} 4 { 123} infeasible relaation 1 { 123} 2 {0123} 3 {0123} 4 {0123} = (2⅓,3,2⅔,0) value = 523⅓ 1 {1,2} 1 = 3 1 { 3} 2 {012 } 3 { 123} 4 {0123} = (3,2,2¾,0) value = 527½ Slide 33 1 { 3} 2 { 12 } 3 { 12 } 4 { 123} = (3,2,2,1) value = 530 feasible solution 3 {1,2} 3 = 3 1 { 3} 2 {0123} 3 {0123} 4 {0123} = (3,2.6,2,0) value = = 3 2 {0,1,2} 1 { 3} 2 {012 } 3 { 3} 4 {012 } = (3,1½,3,½) value = 530 backtrack due to bound

34 Primal-dual-dual method Another feasible solution found. Search terminates with incumbent as optimum. 1 { 12 } 2 { 23} 3 { 123} 4 { 123} infeasible relaation 1 { 123} 2 {0123} 3 {0123} 4 {0123} = (2⅓,3,2⅔,0) value = 523⅓ 1 {1,2} 1 = 3 1 { 3} 2 {012 } 3 { 123} 4 {0123} = (3,2,2¾,0) value = 527½ Slide 34 1 { 3} 2 { 12 } 3 { 12 } 4 { 123} = (3,2,2,1) value = 530 feasible solution 3 {1,2} 3 = 3 1 { 3} 2 {0123} 3 {0123} 4 {0123} = (3,2.6,2,0) value = = 3 2 {0,1,2} 1 { 3} 2 {012 } 3 { 3} 4 {012 } = (3,1½,3,½) value = 530 backtrack due to bound 1 { 3} 2 { 3} 3 {012 } 4 {012 } = (3,3,0,2) value = 530 feasible solution

35 Eample of search/inference duality We solved the inference dual by solving the primal (search) problem. From top down, the tree is a search. From bottom up, the tree is a proof of optimality. Slide 35

36 Eample of search/inference duality We solved the inference dual by solving the primal (search) problem. From top down, the tree is a search. From bottom up, the tree is a proof of optimality. Cutting planes, domain filtering are steps in the proof. Slide 36

37 Eample of search/inference duality We solved the inference dual by solving the primal (search) problem. From top down, the tree is a search. From bottom up, the tree is a proof of optimality. Cutting planes, domain filtering are steps in the proof. The tree also encodes a solution of a relaation dual (subadditive dual). Slide 37

38 Eample of search/inference duality We solved the inference dual by solving the primal (search) problem. From top down, the tree is a search. From bottom up, the tree is a proof of optimality. Cutting planes, domain filtering are steps in the proof. The tree also encodes a solution of a relaation dual (subadditive dual). We can also solve the primal (search) problem by solving the inference dual Inference dual generates nogoods for nogood-based search. Slide 38

39 Relaation and inference duality All(?) optimization duals are instances of both. Outline: Relaation dual Eample: Surrogate dual Eample: Lagrangean dual Inference dual Eample: LP dual Eample: Lagrangean dual Eample: Surrogate dual Slide 39

40 Relaation dual Primal Dual min f ( ) S Feasible set ma θ ( u ) u U Problem with relaed epigraph, parameterized by u where θ ( u ) = min { f (, u ) S ( u ) } S ( u ) S, f (, u ) f ( ), all S Slide 40

41 Eample: Surrogate dual Primal Dual min f ( ) g ( ) 0 S ma θ ( u ) u 0 where θ ( u ) = min { f ( ) ug ( ) 0, S } Replace constraints g() 0 with a surrogate ug() 0, objective function unchanged. LP dual is a special case. Slide 41

42 Eample: Lagrangean dual Primal Dual min f ( ) g ( ) 0 S ma θ ( u ) u 0 where θ ( u ) = min { f ( ) ug ( ) S } Replace objective function f() with a lower bound f() ug(), inequality constraints dropped. LP dual is a special case. Slide 42

43 Inference dual Primal Dual min f ( ) S ma S v f ( ) P P v P Follows using proof P Feasible set Family of proofs (inference method) Dual is defined relative to an inference method. Strong duality applies if inference method is complete. Slide 43

44 Eample: LP dual Primal Dual min A b c 0 ma v A b P c v 0 P P Nonnegative linear combination + domination Slide 44

45 Eample: LP dual Primal Dual min A b c 0 ma v A b P c v 0 P P Nonnegative linear combination + domination 0 ua ub dominates c v A b c v iff for some u 0 ua c and ub v Slide 45

46 Eample: LP dual Primal Dual min A b c 0 ma v A b P c v 0 P P = ma ua c Nonnegative linear combination + domination u 0 ub 0 ua ub dominates c v A b c v iff for some u 0 ua c and ub v Inference method is complete (assuming feasibility) due to Farkas Lemma. So we have strong duality (assuming feasibility). Slide 46

47 Eample: Lagrangean dual Primal Dual min f ( ) g ( ) 0 S ma P g ( ) b f ( ) v P P v Nonnegative linear combination + domination Slide 47

48 Eample: Lagrangean dual Primal Dual min f ( ) g ( ) 0 S ma P g ( ) b f ( ) v P P v Nonnegative linear combination + domination S ug ( ) 0 dominates f ( ) v 0 g ( ) 0 f ( ) v iff for some u 0 ug() f() v for all S That is, v f() ug() for all S Or v min { f ( ) ug ( ) } S Slide 48

49 Eample: Lagrangean dual Primal Dual min f ( ) g ( ) 0 S ma P g ( ) b f ( ) v P P v Nonnegative linear combination + domination 0 { f ug } = ma min ( ) ( ) u S S ug ( ) 0 dominates f ( ) v 0 g ( ) 0 f ( ) v iff for some u 0 ug() f() v for all S That is, v f() ug() for all S Or v min { f ( ) ug ( ) } S Slide 49 Inference method is incomplete

50 Eample: Surrogate dual Primal Dual min f ( ) g ( ) 0 S ma P g ( ) b f ( ) v P P v Nonnegative linear combination + implication Slide 50

51 Eample: Surrogate dual Primal Dual min f ( ) g ( ) 0 S ma P g ( ) b f ( ) v P P v Nonnegative linear combination + implication S ug ( ) 0 implies f ( ) v g ( ) 0 f ( ) v iff for some u 0 Any S with ug() 0 satisfies f() v So, min {f() ug() 0, S} v Slide 51

52 Eample: Surrogate dual Primal Dual min f ( ) g ( ) 0 S ma P g ( ) b f ( ) v P P v { f ug } = ma min ( ) ( ) 0 u 0 S Nonnegative linear combination + implication S ug ( ) 0 implies f ( ) v g ( ) 0 f ( ) v iff for some u 0 Any S with ug() 0 satisfies f() v So, min {f() ug() 0, S} v Slide 52 Inference method is incomplete

53 Focus on inference duality Outline: Nogood-based search Solution of inference dual provides nogoods to guide search. Benders cuts and conflict clauses in SAT are eamples. Slide 53

54 Focus on inference duality Outline: Nogood-based search Solution of inference dual provides nogoods to guide search. Benders cuts and conflict clauses in SAT are eamples. Eample: SAT DPL + conflict clauses Partial order dynamic backtracking Slide 54

55 Focus on inference duality Outline: Nogood-based search Solution of inference dual provides nogoods to guide search. Benders cuts and conflict clauses in SAT are eamples. Eample: SAT DPL + conflict clauses Partial order dynamic backtracking Eamples: Integrated problem solving Including more eamples of logic-based Benders Slide 55

56 Nogood-based search Nogoods record search eperience Each solution eamined generates a nogood. Net solution must satisfy current nogood set. Branching search is a special case. Nogoods can be derived by solving inference dual of a subproblem. Subproblem can be defined by fiing some variables to current values in the search. Slide 56

57 Eample: Logic-based Benders Partition variables,y and search over values of Subproblem results from fiing min f (, y ) min f (, y ) (, y ) S (, y ) S Slide 57 Hooker 1994 Hooker & Ottosson 2003

58 Eample: Logic-based Benders Partition variables,y and search over values of Subproblem results from fiing min f (, y ) min f (, y ) (, y ) S (, y ) S Let proof P be solution of subproblem inference dual for Slide 58 Hooker 1994 Hooker & Ottosson 2003 =

59 Eample: Logic-based Benders Partition variables,y and search over values of Subproblem results from fiing min f (, y ) min f (, y ) (, y ) S (, y ) S Let proof P be solution of subproblem inference dual for Then P derives a lower bound (, ) B P Slide 59 Hooker 1994 Hooker & Ottosson 2003 =

60 Eample: Logic-based Benders Partition variables,y and search over values of Subproblem results from fiing min f (, y ) min f (, y ) (, y ) S (, y ) S Let proof P be solution of subproblem inference dual for = Then P derives a lower bound (, ) B P The same proof P provides a lower bound B(P,) for any Slide 60 Hooker 1994 Hooker & Ottosson 2003

61 Eample: Logic-based Benders Partition variables,y and search over values of Subproblem results from fiing min f (, y ) min f (, y ) (, y ) S (, y ) S Let proof P be solution of subproblem inference dual for = Then P derives a lower bound (, ) B P The same proof P provides a lower bound B(P,) for any Add the Benders cut v B(P,) to the master problem min v Benders cuts Slide 61 Hooker 1994 Hooker & Ottosson 2003

62 Eample: Logic-based Benders Partition variables,y and search over values of Subproblem results from fiing min f (, y ) min f (, y ) (, y ) S (, y ) S Let proof P be solution of subproblem inference dual for = Then P derives a lower bound (, ) B P The same proof P provides a lower bound B(P,) for any Add the Benders cut v B(P,) to the master problem Solve master problem for net min v Benders cuts Slide 62 Hooker 1994 Hooker & Ottosson 2003

63 Classical Benders Partition variables,y and search over values of min f ( ) + cy g ( ) + Ay b D, 0 y Slide 63 Subproblem results from fiing min f ( ) + cy Ay b g ( ) y 0 ( u )

64 Classical Benders Partition variables,y and search over values of Subproblem results from fiing min f ( ) + cy g ( ) + Ay b D, 0 y y 0 min f ( ) + cy Ay b g ( ) Let u be a dual solution of subproblem for = Slide 64 ( u )

65 Classical Benders Partition variables,y and search over values of Subproblem results from fiing min f ( ) + cy g ( ) + Ay b D, 0 y y 0 min f ( ) + cy Ay b g ( ) Let u be a dual solution of subproblem for Then u is proof of bound = B ( u, ) = f ( ) + u ( b g ( )) Slide 65 ( u )

66 Classical Benders Partition variables,y and search over values of Subproblem results from fiing min f ( ) + cy g ( ) + Ay b D, 0 y y 0 min f ( ) + cy Ay b g ( ) ( u ) Let u be a dual solution of subproblem for Then u is proof of bound = B ( u, ) = f ( ) + u ( b g ( )) Also u is proof of bound B(u,) = f() + u(b g()) for any Slide 66

67 Classical Benders Partition variables,y and search over values of Subproblem results from fiing min f ( ) + cy g ( ) + Ay b D, 0 y y 0 min f ( ) + cy Ay b g ( ) ( u ) Let u be a dual solution of subproblem for Then u is proof of bound = B ( u, ) = f ( ) + u ( b g ( )) Also u is proof of bound B(u,) = f() + u(b g()) for any Add Benders cut v = f() + u(b g()) to master problem: min v Benders cuts Slide 67

68 Classical Benders Partition variables,y and search over values of Subproblem results from fiing min f ( ) + cy g ( ) + Ay b D, 0 y y 0 min f ( ) + cy Ay b g ( ) ( u ) Let u be a dual solution of subproblem for Then u is proof of bound = B ( u, ) = f ( ) + u ( b g ( )) Also u is proof of bound B(u,) = f() + u(b g()) for any Add Benders cut v = f() + u(b g()) to master problem: Solve master problem for net min v Benders cuts Slide 68

69 Eample: SAT Solve SAT by chronological backtracking + unit clause rule = DPL To get nogood, solve inference dual at current node. Solve dual with unit clause rule Process nogood set (master problem) with parallel resolution Nogood set is a relaation of the problem. Slide 69

70 Slide Find a satisfying solution. Eample: SAT

71 Slide 71 DPL with chronological backtracking 1 = 0 2 = 0 3 = 0 5 = 0 4 = 0 Branch to here. Solve subproblem with unit clause rule, which proves infeasibility. ( 1, 5 ) = (0,, 0) creates the infeasibility.

72 Slide 72 DPL with chronological backtracking 1 = 0 2 = 0 3 = 0 5 = 0 4 = 0 Branch to here. Solve subproblem with unit clause rule, which proves infeasibility. ( 1, 5 ) = (0,, 0) creates the infeasibility Generate nogood.

73 1 = 0 2 = 0 3 = 0 4 = 0 5 = Slide 73 DPL with chronological backtracking Master problem k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) Conflict clause appears as nogood induced by solution of R k.

74 1 = 0 2 = 0 3 = 0 4 = 0 5 = 0 5 = 1 Slide 74 DPL with chronological backtracking Master problem k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) (0,0,0,0,1, ) Go to solution that solves relaation, with priority to 0

75 1 = 0 2 = 0 3 = 0 4 = 0 5 = 0 5 = 1 Slide 75 DPL with chronological backtracking Master problem k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) (0,0,0,0,1, ) Process nogood set with parallel resolution parallel-absorbs

76 1 = 0 2 = 0 3 = 0 4 = 0 5 = 0 5 = 1 Slide 76 DPL with chronological backtracking Master problem k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) Process nogood set with parallel resolution parallel-absorbs

77 1 = 0 2 = 0 3 = 0 4 = 0 5 = 0 Slide 77 DPL with chronological backtracking Master problem k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) 2 (0,0,0,1,0, ) Solve relaation again, continue. So backtracking is nogood-based search with parallel resolution

78 Eample: SAT + conflict clauses Use stronger nogoods = conflict clauses. Nogoods rule out only branches that play a role in unit clause refutation. Slide 78

79 Slide 79 DPL with conflict clauses 1 = 0 2 = 0 3 = 0 4 = 0 = 0 Branch to here. Unit clause rule 5 proves infeasibility. ( 1, 5 ) = (0,0) is only premise of unit clause proof.

80 Slide 80 DPL with conflict clauses 1 = 0 3 = 0 2 = 0 k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) = 0 5 = 0 Conflict clause appears as nogood induced by solution of R k. 1 5

81 Slide 81 DPL with conflict clauses 1 = 0 Master problem 3 = 0 2 = 0 k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) (0,0,0,0,1, ) 4 = 0 5 = 0 5 =

82 DPL with conflict clauses 1 = 0 Master problem 4 = 0 3 = 0 2 = 0 5 = 0 5 = k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) parallel-resolve to yield parallel-absorbs Slide 82

83 Slide 83 DPL with conflict clauses 1 = 0 2 = 0 2 = 1 4 = 0 3 = 0 5 = 0 = k k k R R Relaation Solution of Nogoods 0 (0,0,0,0,0, ) (0,0,0,0,1, ) (0,1,,,, ) parallel-resolve to yield 1

84 Slide 84 DPL with conflict clauses 1 = 0 1 = 1 2 = = 0 3 = 0 5 = 0 5 = k k k R R Relaation Solution of Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) 2 (0,1,,,, ) 3 (1,,,,, ) Search terminates

85 SAT + partial order dynamic backtracking Forget about the branching tree Let nogoods direct the search in a more general way. Similar method used in recent SAT solvers Solve relaation by selecting a solution that conforms to nogoods. Conform = takes opposite sign than in nogoods. More freedom than in branching. Ginsberg & McAllister 1993, 1994 Slide 85

86 Slide 86 Partial Order Dynamic Backtracking k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) Arbitrarily choose one variable to be last

87 Slide 87 Partial Order Dynamic Backtracking k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) Other variables are penultimate Arbitrarily choose one variable to be last

88 Slide 88 Partial Order Dynamic Backtracking k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) Since 5 is penultimate in at least one nogood, it must conform to nogoods. It must take value opposite its sign in the nogoods. 5 will have the same sign in all nogoods where it is penultimate. This allows more freedom than chronological backtracking.

89 Slide 89 Partial Order Dynamic Backtracking k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) (1,,,,0, ) 2 Choice of last variable is arbitrary but must be consistent with partial order implied by previous choices. Since 5 is penultimate in at least one nogood, it must conform to nogoods. It must take value opposite its sign in the nogoods. 5 will have the same sign in all nogoods where it is penultimate. This allows more freedom than chronological backtracking.

90 Partial Order Dynamic Backtracking k Relaation R k Solution of R k Nogoods 0 (0,0,0,0,0, ) (1,,,,0, ) 2 5 Choice of last variable is arbitrary but must be consistent with partial order implied by previous choices. 5 1 Parallel-resolve 5 1 to yield 5 Since 5 is penultimate in at least one nogood, it must conform to nogoods. It must take value opposite its sign in the nogoods. 5 will have the same sign in all nogoods where it is penultimate. Slide 90 This allows more freedom than chronological backtracking.

91 Partial Order Dynamic Backtracking k k k R R Relaation Solution of Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) 2 (,0,,,1, ) does not parallel-resolve with 5 because 5 is not last in both clauses 5 2 Slide 91

92 Slide 92 Partial Order Dynamic Backtracking k k k R R Relaation Solution of Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) 2 (,0,,,1, ) 5 3 (,1,,,1, ) Search terminates Must conform

93 Eamples of integrated solving Piecewise linear optimization. Illustrates modeling with metaconstraint Planning & scheduling - Logic-based Benders Planning and disjunctive scheduling Planning and cumulative scheduling Single-facility scheduling Truss structure design - Global optimization. Solved by SIMPL, a prototype integrated solver. Aron, Hooker, & Yunes 2004, 2010 Slide 93

94 Piecewise linear optimization Maimize a piecewise linear separable function subject to budget constraint. ma f ( ) i i i C i i Each f i is a piecewise linear semicontinuous function Slide 94

95 Piecewise linear optimization Semicontinuous piecewise linear function f() Slide 95

96 Piecewise linear optimization Integrated approach Search: Branch on variables Relaation: Use a specialized conve hull relaation for piecewise linear functions (no need for 0-1 model or SOS2). Inference: Bounds propagation. Ottosson, Thorsteinsson & Hooker 1999 Slide 96

97 Piecewise linear optimization Integrated model ma i i i u C i ( ) piecewise, u, L, U, c, d, all i i i i i i i Metaconstraint (global constraint in CP) Slide 97

98 Piecewise linear optimization Semicontinuous piecewise linear function f() Tight linear relaation after propagation Slide 98

99 Piecewise linear optimization Semicontinuous piecewise linear function f() Tighter relaation after branching Value of in solution of current linear relaation Slide 99

100 Piecewise linear optimization SIMPL model Slide 100

101 Piecewise linear optimization SIMPL model Slide 101 Piecewise linear metaconstraint.

102 Piecewise linear optimization Computation time (sec) vs. number of jobs Slide 102

103 Planning and disjunctive scheduling Assign jobs to machines, and schedule the machines assigned to each machine within time windows. The objective is to minimize processing cost, makespan, or total tardiness. Slide 103

104 Planning and disjunctive scheduling Job Data Eample Assign 5 jobs to 2 machines. Schedule jobs assigned to each machine without overlap (disjunctive scheduling) Machine A Machine B Slide 104

105 Planning and disjunctive scheduling Integrated model min j c j r s d p, all j j j j j Start time of job j ( j j = ij j = ) Time windows nooverlap ( s i ),( p i ), all i Jobs cannot overlap Machine assigned to job j Here we minimize processing cost. Slide 105 j j

106 Planning and disjunctive scheduling Integrated approach Search: Enumerate subproblems (defined by assigning jobs to machines) Relaation: Enumerate master problems (which assign jobs to machines) Inference: Generate nogoods (logic-based Benders cuts), which are added to master problem. Slide 106

107 Planning and disjunctive scheduling Integrated approach Assign the jobs in the master problem, to be solved by MILP. Schedule the jobs in the subproblem, to be solved by CP. Jain & Grossmann 2001 Slide 107

108 Planning and disjunctive scheduling Integrated approach Assign the jobs in the master problem, to be solved by MILP. Schedule the jobs in the subproblem, to be solved by CP. The subproblem decouples into a separate scheduling problem on each machine. In this problem, the subproblem is a feasibility problem. Jain & Grossmann 2001 Slide 108

109 Planning and disjunctive scheduling Integrated approach Assign the jobs in the master problem, to be solved by MILP. Schedule the jobs in the subproblem, to be solved by CP. The subproblem decouples into a separate scheduling problem on each machine. In this problem, the subproblem is a feasibility problem. Solve inference dual of subproblem to generate nogoods (logic-based Benders cuts), which are added to master problem. Slide 109

110 Planning and disjunctive scheduling Integrated model min j c j r s d p, all j j j j j Start time of job j ( j j = ij j = ) Time windows nooverlap ( s i ),( p i ), all i Jobs cannot overlap For a fied assignment the subproblem on each machine i is r s d p, all j with = i j j j j j j ( s j j = i p ij j = i ) nooverlap ( ),( ) Slide 110 j j

111 Planning and disjunctive scheduling Logic-based Benders approach Suppose we assign jobs 1,2,3,5 to machine A in iteration k. We can prove that there is no feasible schedule. Edge finding derives infeasibility by reasoning only with jobs 2,3,5. So these jobs alone create infeasibility. 2 = 3 = 5 = A So we have a Benders cut ( ) Slide 111

112 Planning and disjunctive scheduling Logic-based Benders approach We want the master problem to be an MILP, which is good for assignment problems. So we write the Benders cut ( ) A = = = A + A + A Using 0-1 variables: = 1 if job 5 is assigned to machine A Slide 112

113 Planning and disjunctive scheduling The master problem is a relaation, formulated as an MILP: min ij A 2 A 3 A 5 { 0,1 } ij ij 2 relaation of subproblem ij c + + Benders cut from machine A Slide 113

114 Planning and disjunctive scheduling Keys to success: Strengthen Benders cuts by solving subproblems for a neighborhood of current master solution. Use relaation of subproblem in the master. Slide 114

115 Planning and disjunctive scheduling SIMPL model Slide 115

116 Planning and disjunctive scheduling Yunes, Aron & Hooker 2010 Computational results Long processing times Jobs Machines MILP (CPLEX 11) Nodes Sec. SIMPL Benders Iter. Cuts Sec , , , ,309 10, SIMPL results are similar to original hand-coded results. Slide 116

117 Planning and disjunctive scheduling Yunes, Aron & Hooker 2010 Computational results Short processing times Jobs Machines MILP (CPLEX 11) Nodes Sec. SIMPL Benders Iter. Cuts Sec , > 27.5 mil. > 48 hr > 5.4 mil. > 19 hr* Slide 117 *out of memory

118 Planning and cumulative scheduling Consider a cumulative scheduling constraint: ( ) cumulative ( s 1, s 2, s 3 ),( p 1, p 2, p 3 ),( c 1, c 2, c 3 ), C A feasible solution: Slide 118

119 Edge finding for cumulative scheduling We can deduce that job 3 must finish after the others finish: 3 > { 1,2 } Because the total energy required eceeds the area between the earliest release time and the later deadline of jobs 1,2: ( ) e 3 + e {1,2} > C L {1,2} E {1,2,3} Slide 119

120 Edge finding for cumulative scheduling We can deduce that job 3 must finish after the others finish: 3 > { 1,2 } Because the total energy required eceeds the area between the earliest release time and the later deadline of jobs 1,2: ( ) e 3 + e {1,2} > C L {1,2} E {1,2,3} Total energy required = Slide 120

121 Edge finding for cumulative scheduling We can deduce that job 3 must finish after the others finish: 3 > { 1,2 } Because the total energy required eceeds the area between the earliest release time and the later deadline of jobs 1,2: ( ) e 3 + e {1,2} > C L {1,2} E {1,2,3} Total energy required = 22 Area available = Slide 121

122 Edge finding for cumulative scheduling We can deduce that job 3 must finish after the others finish: 3 > { 1,2 } We can update the release time of job 3 to E {1,2} + e ( C c )( L E ) J 3 {1,2} {1,2} c 3 Energy available for jobs 1,2 if space is left for job 3 to start anytime = Slide 122

123 Edge finding for cumulative scheduling We can deduce that job 3 must finish after the others finish: 3 > { 1,2 } We can update the release time of job 3 to E {1,2} + e ( C c )( L E ) J 3 {1,2} {1,2} c 3 Energy available for jobs 1,2 if space is left for job 3 to start anytime = 10 Ecess energy required by jobs 1,2 = Slide 123

124 Edge finding for cumulative scheduling We can deduce that job 3 must finish after the others finish: 3 > { 1,2 } We can update the release time of job 3 to E {1,2} + e ( C c )( L E ) J 3 {1,2} {1,2} c 3 Energy available for jobs 1,2 if space is left for job 3 to start anytime = 10 Ecess energy required by jobs 1,2 = 4 4 Move up job 3 release time 4/2 = 2 units beyond E 10 {1,2} E 3 Slide 124

125 Edge finding for cumulative scheduling In general, if e ( ) J { k } > C L J E J { k } then k > J, and update E k to ma J J e ( C c )( L E ) > 0 J k J J e J ( C c k )( L J E J ) E J + c k In general, if e ( ) J { k } > C L J { k } E J then k < J, and update L k to min J J e ( C c )( L E ) > 0 J k J J e J ( C c k )( L J E J ) L J c k Slide 125

126 Edge finding for cumulative scheduling Key result: There is an O(n 2 ) algorithm that finds all applications of the edge finding rules. Nuitjen & Aarts 1994, 1996 Baptiste, Le Pape & Nuitjen 2001 Slide 126

127 Other propagation rules for cumulative scheduling Etended edge finding. Timetabling. Not-first/not-last rules. Energetic reasoning. Slide 127

128 Planning and cumulative scheduling Assign jobs to facilities (e.g. factories), with cumulative scheduling in each facility. Slide 128

129 Planning and cumulative scheduling Assign jobs to facilities (e.g. factories), with cumulative scheduling in each facility. Benders cut for minimum makespan, with a cumulative scheduling subproblem: (1 ) ma { } min { } i i * M M i p ij ij + d j d j j J i j J j J Jobs currently assigned to machine i Minimum makespan on machine i for jobs currently assigned Slide 129 Hooker 2004

130 Planning and cumulative scheduling Minimum Cost Computation time (sec) vs. number of jobs 4 facilities (each point = 5 instances) Ran out of memory Computation terminated after 7200 sec Slide 130

131 Planning and cumulative scheduling Minimum Makespan Computation time (sec) vs. number of jobs 4 facilities (each point = 5 instances) Slide 131 Computation terminated after 7200 sec

132 Other applications of logic-based Benders Steel production scheduling Batch scheduling in a chemical plant (BASF) Assembly line scheduling (Citroën/Pugeot) Location-allocation Dispatching of automated guided vehicles Queuing design & control Real-time scheduling of computer processes Traffic diversion Home health care delivery* Sports scheduling *Cire & Hooker 2010 Slide 132

133 Single-facility Scheduling Can we apply decomposition to pure scheduling problems? Consider single-machine scheduling with a long time horizon. Slide 133

134 Single-facility Scheduling Can we apply decomposition to pure scheduling problems? Consider single-machine scheduling with a long time horizon. Segmented problem: jobs must finish within a segment. Segment boundaries are weekends or shutdown times. Master problem assigns jobs to segments. z 0 z 1 z 2 z 3 z 4 Coban & Hooker 2010 Slide 134

135 Single-facility scheduling Benders cut from segment i (min makespan problem): ( ) ( ) * * M M p 1 y w M 1 y i j ij i i ij q 1 y, j J i ij i j J j J { } { } ( ) w ma d min d 1 y i j j ij j J i j J i j J { } { } w ma d min d i i i j j j J j J i i i.. = 1 when job j is assigned to segment i Slide 135 J i = { jobs with release times before start of segment i } J i = { jobs with release times after start of segment i }

136 Single-facility scheduling Segmented Problem Computation time (sec) vs. number of jobs Time horizon proportional to no. of jobs Computation terminated after 600 sec Slide 136

137 Single-facility scheduling Unsegmented problem: jobs can be scheduled anytime within their time windows. Segment boundaries are only for decomposition purposes. z 0 z 1 z 2 z 3 z 4 Coban & Hooker 2011 Slide 137

138 Single-facility scheduling Master problem must encode whether a job is split between segments. Slide 138 min M y 1, all j ij y = y + y + y + y, all i, j ij ij ij ij ij y 1, y 1, y 1, ij 1 ij 2 ij 3 j j j y y + y, all i > 1, all j ij 1 i 1, j,2 i 1, j,3 y y + y, all i < n, all j ij 2 i 1, j,1 i 1, j,3 y y + y, all i > 1, all ij 3 i 1, j,3 i 1, j,2 y y + y, all i < n, all j ij 3 i + 1, j,3 i + 1, j,1 y 1, y 1, y 1, y = y = y = y = 0, all j ij 0 ij 1 ij 2 i i i ij 1 ij 3 nj 2 nj 3 p y, all j ij 3 i z i + 1 z i Benders cuts, relaation { } y, y 0,1 ij ijk j i j

139 Single-facility scheduling Benders cuts are generated for several cases. Benders cut when a job j 1 completes at start of segment and some job starts at its release time: * * M M 1 η M 1 y j J i Slide 139 { 0,1 } ( ) ( ) i i i ij ( ) + p + + p η ε ij ij i min ij i min i i ( )( 1 η ) + p p + p η ij ij i min j ij min i i = 0 when reoptimizing schedule on segment doesn t change job order 0 δ Release time 0 { s } j r ij s j s k = min < * * * * j J i Start time of first job k for which s k * = r k Start time of job j in solution of subproblem { * * * } j j k j j k j i p min = min p s > s, r + p r, p + δ j J i

140 Single-facility scheduling Unsegmented Problem Computation time (sec) vs. number of jobs Time horizon proportional to no. of jobs Computation terminated after 600 sec Slide 140

141 Truss Structure Design Select size of each bar (possibly zero) to support the load while minimizing weight. 10-bar cantilever truss 0 deg. freedom 2 deg. freedom Slide 141 Total 8 degrees of freedom Load

142 Truss Structure Design Notation v i = elongation of bar s i = force along bar h = length of bar i i d j = node displacement A i = cross-sectional area of bar p j = load along d.f. j Slide 142

143 Truss Structure Design nonlinear min i } Minimize total weight s. t. cos θ s = p, all j i j cos θ d = v, all i E i Av i i = s i, all i h i v v v, all i d d d, all j \/ h A i i ij i j ij j i L U i i i L U j j j ( A = A ) k i ik } Equilibrium } Compatibility } Hooke s law } Elongation bounds } Displacement bounds } Logical disjunction Area must be one of several discrete values A ik Slide 143 Constraints can be imposed for multiple loading conditions

144 Truss Structure Design Introducing new variables linearizes the problem but makes it much larger. MILP model Slide 144 min i ik ik i k s. t. cos θ s = p, all j i E h ij i j cos θ d = v, all i ij j ik j k i A v = s, all i ik ik i i k L U v v v, all i i i i L U d d d, all j k h A y j j j y = 1, all i ik 0-1 variables indicating size of bar i Elongation variable disaggregated by bar size Hooke s law becomes linear

145 Truss Structure Design Integrated approach Search: Branch by splitting the range of areas A i (no need for 0-1 variables). Relaation: Generate a quasi-relaation, which is linear and much smaller than MILP model. Inference: Use logic cuts. Bollapragada, Ghattas & Hooker 2001 Slide 145

146 Truss Structure Design Theorem Suppose we minimize c subject to g(,y) 0. If g(,y) is semihomogeneous in R n and concave in scalar y, then the following is a quasi-relaation of g(,y) 0: Hooker 2005 g y g y 1 2 (, L ) + (, U ) 0 L 1 U α α L 2 U (1 α ) (1 α ) = Slide 146

147 Truss Structure Design Theorem Suppose we minimize c subject to g(,y) 0. If g(,y) is semihomogeneous in R n and concave in scalar y, then the following is a quasi-relaation of g(,y) 0: Hooker 2005 g y g y 1 2 (, L ) + (, U ) 0 L 1 U α α L 2 U (1 α ) (1 α ) = Not a valid relaation, and it may contain different variables, But its optimal value is a valid lower bound on the optimal value of the original problem. Slide 147

148 Truss Structure Design Theorem Suppose we minimize c subject to g(,y) 0. If g(,y) is semihomogeneous in R n and concave in scalar y, then the following is a quasi-relaation of g(,y) 0: Hooker 2005 g y g y 1 2 (, L ) + (, U ) 0 L 1 U α α L 2 U (1 ) (1 ) α α = g ( α, y ) α g (, y ) for all, y and α [0,1] g (0, y ) = 0 for all y Slide 148

149 Truss Structure Design Theorem Suppose we minimize c subject to g(,y) 0. If g(,y) is semihomogeneous in R n and concave in scalar y, then the following is a quasi-relaation of g(,y) 0: Hooker 2005 g y g y 1 2 (, L ) + (, U ) 0 L 1 U α α L 2 U (1 α ) (1 α ) = Bounds on y Slide 149

150 Truss Structure Design Theorem (JNH 2005) Suppose we minimize c subject to g(,y) 0. If g(,y) is semihomogeneous in R n and concave in scalar y, then the following is a quasi-relaation of g(,y) 0: Hooker 2005 g y g y 1 2 (, L ) + (, U ) 0 L 1 U α α L 2 U (1 α ) (1 α ) = Bounds on Slide 150

151 Truss Structure Design Theorem (JNH 2005) Suppose we minimize c subject to g(,y) 0. If g(,y) is semihomogeneous in R n and concave in scalar y, then the following is a quasi-relaation of g(,y) 0: g y g y 1 2 (, L ) + (, U ) 0 L 1 U α α L 2 U (1 α ) (1 α ) = E i Av i i = s i h i Slide 151 has the form g(,y) = 0 with g semihomogenous in because we can write it with = (A i,s i ), y = v i. E i Av i i s i h i = 0

152 Truss Structure Design So we have a quasi-relaation of the truss problem: Slide 152 min h [ A y + A (1 y )] i L U i i i i i s. t. cos θ s = p, all j i j ij i j cos θ d = v + v, all i ij j i 0 i 1 E i ( A L U i v i 0 + A i v i 1 ) = s i, all i h i L U v y v v y, all i i i i 0 i i L U i i i 1 i i v (1 y ) v v (1 y ), all i d d d, all j L U j j j 0 y 1, all i i Hooke s law is linearized Elongation bounds split into 2 sets of bounds

153 Truss Structure Design Logic cuts v i0 and v i1 must have same sign in a feasible solution. If not, we branch by adding logic cuts v i 0, v i 1 0, v i 0, v i 1 0 Slide 153

154 Truss Structure Design SIMPL model Slide 154

155 Truss Structure Design 10-bar cantilever truss Slide 155 Load

156 Truss Structure Design Yunes, Aron & Hooker 2010 Computational results (seconds) No. bars Loads BARON CPLEX 11 Hand coded SIMPL * * Slide 156 *plus displacement bounds

157 Truss Structure Design 25-bar problem Slide 157

158 Truss Structure Design 72-bar problem Slide 158

159 Truss Structure Design Computational results (seconds) No. bars Loads BARON CPLEX 11 Hand coded SIMPL , , , > 24 hr* > 24 hr* > 24 hr* > 24 hr** > 24 hr*** * no feasible solution found ** best feasible solution has cost 32,748 Slide 159 *** best feasible solution has cost 32,700

160 Summary Combinatorial optimizations methods tend to use a common primal/dual/dual approach. This can be a basis for a general-purpose solver if the problem is modeled with metaconstraints. Slide 160

161 Summary Combinatorial optimizations methods tend to use a common primal/dual/dual approach. This can be a basis for a general-purpose solver. if the problem is modeled with metaconstraints. This more general point of view can suggest new methods. Methods based on inference duality illustrated here. So far, new methods based on relaation duality are primarily discrete and/or trees, mini-buckets & nonserial DP, multivalued decision diagrams, etc. Slide 161

162 Slide 162 Gracias. Hay preguntas?