A Scheme for Integrated Optimization

A Scheme for Integrated Optimization John Hooker ZIB November 2009 Slide 1

Outline Overview of integrated methods A taste of the underlying theory Eamples, with results from SIMPL Proposal for SCIP/SIMPL cooperation Slide 2

Overview of integrated methods Integration principles. Search-infer-rela Classical solution methods. Evolution of SIMPL Slide 3

Integration Principles Integrate MILP, constraint programming, global optimization in a unified approach. Slide 4

Integration Principles Integrate MILP, constraint programming, global optimization in a unified approach. Low-level integration with high-level modeling. Slide 5

Integration Principles Integrate MILP, constraint programming, global optimization in a unified approach. Low-level integration with high-level modeling. Succinct modeling with meta-constraints. Model communicates problem structure to the solver. General search-infer-rela solution algorithm. Enumerate problem restrictions. Branching or logic-based Benders. Underlying search/inference and search/relaation dualities. Slide 7

Search-Infer-Rela Search: Enumerate problem restrictions. Branching tree nodes, Benders subproblems, local search neighborhoods, etc. Infer: Deduce constraints from current restriction Nogoods, cutting planes, filtering, etc. Rela: Solve relaation of current restriction LP, Lagrangean, domain store, Benders master, etc. Slide 9

Classical solution methods CP solver Search: Branching Inference: Flltering Relaation: Domain store MILP solver Search: Branching Inference: Cutting planes, presolve, reduced cost variable fiing Relaation: LP Benders Search: Enumerate subproblems. Inference: Benders cuts Relaation: Master problem Slide 10

Classical solution methods Global optimization Search: Enumerate boes Inference: Domain reduction, dual-based variable bounding Relaation: Conveification SAT Search: Branching Inference: Conflict clauses Relaation: Same as restriction Local search Search: Enumerate neighborhoods. Inference: Tabu list, etc. Relaation: Same as restriction Slide 11

Interaction Inference Strengthens Fies variables Reduces domains Adds IP cuts Adds conflict clause Adds Benders cuts Shrinks bo Creates neighborhood Guides Separating cut Relaation Slide 12 Activates IP cut Fied variable Filtering/propagation Reduced domain Benders cut Subproblem dual Restriction Defines Identifies net branch Fractional variable Nonsingleton domain Violated constraint Defines subproblem Solution of master Defines neighborhood Center on previous solution

Evolution of SIMPL Basic Framework JNH. Logic-based methods for optimization, CP 1994 JNH, M. A. Osorio. Mied logical/linear programming, Discrete Applied Mathematics 1999. JHH, G. Ottosson, E. S. Thorsteinsson, H.-J. Kim. On integrating CP and LP for combinatorial optimization. AAAI 1999. G. Ottosson, E. Thorsteinsson, JNH. Mied global constraints and inference in hybrid IP-CLP solvers. CP 1999 JNH, G. Ottosson, E. S. Thorsteinsson, H.-J. Kim. A scheme for unifying optimization and constraint satisfaction methods. Knowledge Engineering Review, 2000. Slide 13

Evolution of SIMPL JNH. A framework for integrating solution methods. ICS 2003. I. Aron, JNH, T. H. Yunes. SIMPL: A system for integrating optimization techniques. CPAIOR 2004. T. Yunes, I. Aron, JNH. An integrated solver for optimization problems. Operations Research, to appear. Slide 14

Evolution of SIMPL Search JNH. Unifying local and ehaustive search. ENC 2005. JNH, A search-infer-and-rela framework for integrating solution methods. CPAIOR 2005 Modeling JNH, H.-J. Kim, G. Ottosson. A declarative modeling framework that integrates solution methods. Annals of OR 2001. JNH. Hybrid modeling. Ten Years of CPAIOR, to appear. Slide 15

Evolution of SIMPL Theory and background JNH. Logic-Based Methods for Optimization. 2000. JNH. Logic, optimization and constraint programming. INFORMS Journal on Computing, 2002. A. Bockmayr, JNH. Constraint programming. Handbook of Discrete Optimization, 2005. JNH. Operations research methods in constraint programming, Handbook of Constraint Programming, 2006. J. N. Hooker. Duality in optimization and constraint satisfaction. CPAIOR 2006 JNH. Integrated Methods for Optimization. 2007. Slide 16

Evolution of SIMPL Nogood-based search (Logic-based Benders) JNH, H. Yan. Logic circuit verification by Benders decomposition. CP 1995. JNH. Logic-Based Methods for Optimization. 2000. E. Thorsteinsson. Branch and check: A hybrid framework for integrating MIP and CLP. CP 2001. JNH, G. Ottosson. Logic-based Benders decomposition. Mathematical Programming, 2003. JNH. A hybrid method for planning and scheduling. Constraints 2005. JNH. Planning and scheduling to minimize tardiness. CP 2005. JNH. Planning and scheduling by logic-based Benders decomposition. Operations Research, 2007. Slide 17

Evolution of SIMPL Relaation methods I. E. Grossmann, JNH, R. Raman, H. Yan. Logic cuts for processing networks with fied charges. Computers and OR, 1994. E. Thorsteinsson and G. Ottosson. Linear relaations and reduced-cost based propagation of continuous variable subscripts. Annals of OR, 2001. S. Bollapragada, O. Ghattas, JNH. Optimal design of truss structures by mied logical and linear programming. Operations Research 2001. JNH. Conve programming methods for global optimization, COCOS 2003 L. Genc-Kaya and JNH. The circuit polytope. 2008. Slide 18

A taste of the underlying theory Inference duality. LP, Lagrangean, surrogate duals Nogood-based search Logic-based and classical Benders Eample: SAT DPL DPL + conflict clauses Partial order dynamic backtracking Slide 19

Inference duality All optimization duals are inference duals Also relaation duals Solution of inference dual is proof of optimality Primal in co-np when dual is in NP Provides nogoods Including Benders cuts, conflict clauses Nogood = conditions under which proof is still valid Directs the search All search is nogood-based search Net restriction defined by nogood Postoptimality analysis Result of altering premises of proof Slide 20

Inference dual Primal min f ( ) S Feasible set ma Dual v P S v f ( ) P P Family of proofs (inference method) Follows using proof P Dual is defined relative to an inference method. Strong duality applies if inference method is complete. Slide 21

Eample: LP dual Primal min c A b 0 ma Dual v A b P c v 0 P P = ma λb λa c λ 0 Nonnegative linear combination + domination 0 λa λb dominates c v A b c v iff for some λ 0 λa c and λb v Slide 22 Inference method is complete (assuming feasibility) due to Farkas Lemma. So we have strong duality (assuming feasibility).

Eample: Lagrangean dual Primal min f ( ) g( ) 0 S ma v P g( ) b f ( ) v P P Dual λ 0 Nonnegative linear combination + domination { λ } = ma min ( ) ( ) f g S S λg( ) 0 dominates f ( ) v 0 g( ) 0 f ( ) v iff for some λ 0 λg() f() v for all S That is, v f() λg() for all S Or v min { f ( ) λg( ) } S Slide 23 Inference method is incomplete

Eample: Surrogate dual Primal min f ( ) g( ) 0 S ma v P g( ) b f ( ) v P P Dual λ 0 S { f λg } = ma min ( ) ( ) 0 Nonnegative linear combination + implication S λg( ) 0 implies f ( ) v g( ) 0 f ( ) v iff for some λ 0 Any S with λg() 0 satisfies f() v So, min {f() λg() 0, S} v Slide 24 Inference method is incomplete

Nogood-based search All search is nogood-based search Each solution eamined generates a nogood. Net solution must satisfy current nogood set. Nogoods are derived by solving inference dual of the subproblem. Subproblem is normally defined by fiing variables to current values in the search. Slide 25

Eample: Logic-based Benders Partition variables,y and search over values of Subproblem results from fiing min f (, y ) min f (, y ) (, y ) S (, y ) S Let proof P be solution of subproblem dual for = Let B(P,) be lower bound obtained by P for given. Add Benders cut v B(P,) to master problem: Solve master problem for net min v Benders cuts Slide 26

Eample: Classical Benders Partition variables,y and search over values of min f ( ) + cy g( ) + Ay b D, 0 y Subproblem results from fiing k min f ( ) + cy k Ay b g( ) y 0 ( λ) Let proof λ be solution of subproblem dual for Let B(λ,) = f() + λ(b g()) be bound obtained by λ for given. Add Benders cut v B(λ,) to master problem: Solve master problem for net = min v Benders cuts Slide 27

Eample: SAT Solve SAT by chronological backtracking + unit clause rule = DPL. Chronological = fied branching order. To get nogood, solve subproblem at current node. Solve with unit clause rule Nogood identifies branches that create infeasibility. Simplest scheme: nogood rules out path to current leaf node. Process nogood set with parallel resolution Nogood set is a relaation of the problem. Solve relaation without branching Select solution with preference for 0 Slide 28

DPL with chronological backtracking 1 = 0 2 = 0 3 = 0 5 = 0 4 = 0 Branch to here. Solve subproblem with unit clause, which proves infeasibility. ( 1, 5 ) = (0,, 0) creates the infeasibility. SARA 07 Slide 29

DPL with chronological backtracking 1 = 0 Consists of processed nogoods 3 = 0 2 = 0 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 1 2 3 4 5 k k 1 2 3 4 5 4 = 0 5 = 0 Conflict clause appears as nogood induced by solution of R k. 1 2 3 4 5 SARA 07 Slide 31

DPL with chronological backtracking 1 = 0 Consists of processed nogoods 3 = 0 2 = 0 k Relaation R Solution of R Nogoods k 0 (0,0,0,0,0, ) 1 2 3 4 5 1 (0,0,0,0,1, ) 1 2 3 4 5 1 2 3 4 5 k 5 = 0 4 = 0 5 = 1 Go to solution that solves relaation, with priority to 0 SARA 07 Slide 32

DPL with chronological backtracking 1 = 0 Consists of processed nogoods 3 = 0 2 = 0 k Relaation R Solution of R Nogoods k 0 (0,0,0,0,0, ) 1 2 3 4 5 1 (0,0,0,0,1, ) 1 2 3 4 5 1 2 3 4 5 k 5 = 0 4 = 0 5 = 1 1 2 3 4 5 1 2 3 4 5 Process nogood set with parallel resolution 1 2 3 4 parallel-absorbs 1 2 3 4 5 1 2 3 4 5 SARA 07 Slide 33

DPL with chronological backtracking 1 = 0 Consists of processed nogoods 5 = 0 4 = 0 3 = 0 2 = 0 5 = 1 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) 2 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 1 2 3 4 5 1 2 3 4 5 k k Process nogood set with parallel resolution 1 2 3 4 parallel-absorbs 1 2 3 4 5 1 2 3 4 5 SARA 07 Slide 34

DPL with chronological backtracking 1 = 0 Consists of processed nogoods 5 = 0 4 = 0 3 = 0 2 = 0 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) 2 (0,0,0,1,0, ) 1 2 3 4 5 1 2 3 4 5 1 2 3 4 5 1 2 3 4 k Solve relaation again, continue. So backtracking is nogood-based search with parallel resolution k SARA 07 Slide 35

Eample: SAT + conflict clauses Nogoods = conflict clauses. Nogoods rule out only branches that play a role in unit clause refutation. Slide 36

DPL with conflict clauses 1 = 0 2 = 0 3 = 0 4 = 0 = 0 5 Branch to here. Unit clause rule proves infeasibility. ( 1, 5 ) = (0,0) is only premise of unit clause proof. SARA 07 Slide 37

DPL with conflict clauses 1 = 0 3 = 0 2 = 0 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 1 5 k k 1 5 4 = 0 5 = 0 Conflict clause appears as nogood induced by solution of R k. 1 5 SARA 07 Slide 38

DPL with conflict clauses 1 = 0 Consists of processed nogoods 3 = 0 2 = 0 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) k 1 (0,0,0,0,1, ) 1 5 1 5 2 5 k 4 = 0 5 = 0 5 = 1 1 5 2 5 SARA 07 Slide 39

DPL with conflict clauses 1 = 0 Consists of processed nogoods 4 = 0 3 = 0 2 = 0 5 = 0 5 = 1 1 5 2 5 1 2 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) 2 1 5 2 5 1 5 1 5 2 5 1 2 k parallel-resolve to yield 1 2 parallel-absorbs k 1 5 2 5 1 2 SARA 07 Slide 40

DPL with conflict clauses 1 = 0 2 = 0 2 = 1 4 = 0 3 = 0 5 = 0 = 5 1 1 5 SARA 07 Slide 41 2 5 1 2 1 2 1 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) 2 (0,1,,,, ) 3 1 2 1 5 1 5 2 5 1 2 1 2 1 k parallel-resolve to yield 1 1 2 k

DPL with conflict clauses 1 = 0 1 = 1 4 = 0 3 = 0 SARA 07 Slide 42 2 = 0 5 = 0 5 = 1 1 5 2 5 1 2 1 1 2 1 k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (0,0,0,0,1, ) 2 (0,1,,,, ) 3 (1,,,,, ) 4 k 1 5 1 5 2 5 1 2 1 2 1 1 Search terminates k

Eample: SAT + partial order dynamic backtracking Solve relaation by selecting a solution that conforms to nogoods. Conform = takes opposite sign than in nogoods. More freedom than in branching. Slide 43

Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 1 5 k k 5 1 Arbitrarily choose one variable to be last SARA 07 Slide 44

Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 1 5 k k 5 1 Other variables are penultimate Arbitrarily choose one variable to be last SARA 07 Slide 45

Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) 2 1 5 k k 5 1 Since 5 is penultimate in at least one nogood, it must conform to nogoods. It must take value opposite its sign in the nogoods. 5 will have the same sign in all nogoods where it is penultimate. SARA 07 Slide 46 This allows more freedom than chronological backtracking.

Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) 2 k 5 1 1 5 5 1 k Choice of last variable is arbitrary but must be consistent with partial order implied by previous choices. Since 5 is penultimate in at least one nogood, it must conform to nogoods. It must take value opposite its sign in the nogoods. 5 will have the same sign in all nogoods where it is penultimate. SARA 07 Slide 47 This allows more freedom than chronological backtracking.

Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) 2 5 1 1 5 5 1 5 k k Choice of last variable is arbitrary but must be consistent with partial order implied by previous choices. 5 1 5 1 Parallel-resolve to yield 5 SARA 07 Slide 48 Since 5 is penultimate in at least one nogood, it must conform to nogoods. It must take value opposite its sign in the nogoods. 5 will have the same sign in all nogoods where it is penultimate. This allows more freedom than chronological backtracking.

Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) 2 (,0,,,1, ) 3 5 1 1 5 5 1 5 5 2 5 5 2 k k 5 does not parallel-resolve with because 5 is not last in both clauses 5 2 SARA 07 Slide 49

Partial Order Dynamic Backtracking k Relaation R Solution of R Nogoods 0 (0,0,0,0,0, ) 1 (1,,,,0, ) 2 (,0,,,1, ) 3 5 5 2 (,1,,,1, ) 4 k 5 1 1 5 5 1 5 5 2 k 2 Search terminates Must conform SARA 07 Slide 50

Eamples, with results from SIMPL Production planning. Semicontinuous piecewise linear functions Product configuration. Variable indices Machine scheduling. Logic-based Benders Truss structure design. Global optimization. Slide 51

Production Planning Maimize profit, which is a piecewise linear function of output. ma f ( ) i i i i C i Each f i is a piecewise linear semicontinuous function Slide 52

Production Planning Semicontinuous piecewise linear function f() Slide 53

Production Planning MILP model SOS2 branching not useful min i i ik = λ L + µ U, all i λ + µ = 1, all i 0 λ y, all i, k 0 µ y, all i, k { } ik ik ik ik i ik ik ik ik k k k y ik y ik ik ik ik λ c + µ d C ik ik ik = 1, all i 0,1, all i, k Slide 54

Production Planning MILP model SOS2 branching not useful = 1 if i is in interval k min i i ik = λ L + µ U, all i λ + µ = 1, all i 0 λ y, all i, k 0 µ y, all i, k { } ik ik ik ik i ik ik ik ik k k k y ik y ik ik ik ik λ c + µ d C ik ik ik = 1, all i 0,1, all i, k Slide 55

Production Planning Integrated model ma i i i u C i ( u L U c d ) piecewise,,,,,, all i i i i i i i Metaconstraint (global constraint in CP) Original hand-coded method: Ottosson, Thorsteinsson and JNH 1999. Slide 56

Production Planning Semicontinuous piecewise linear function f() Tight linear relaation Slide 57

Production Planning Semicontinuous piecewise linear function f() Tighter relaation after branching Slide 58 Value of in solution of current linear relaation

Production Planning SIMPL model Slide 59

Production Planning SIMPL model Recognized as a linear system. Slide 60

Production Planning SIMPL model Is its own LP relaation. CP relaation propagates bounds. Slide 61

Production Planning SIMPL model Piecewise linear metaconstraint. Slide 62

Production Planning SIMPL model LP relaation is conve hull. CP relaation propagates bounds. Slide 63

Production Planning SIMPL model Branch-and-bound search. Dive to leaf node from node with best lower bound. Slide 64

Production Planning SIMPL model Branch on piecewise constraint with greatest violation. Slide 65

Production Planning Computational Results (seconds) Hand-coded integrated method was comparable to CPLEX 9 No. Products MILP CPLEX 9 MILP CPLEX 11 SIMPL 50 9.5 2.4 0.43 60 55 1.8 1.1 70 99 2.6 0.82 80 61 4.6 1.25 90 422 6.2 1.7 100 4458 4.4 2.8 Slide 66

Production Planning CPLEX has become orders of magnitude faster, but still slower than SIMPL No. Products MILP CPLEX 9 MILP CPLEX 11 SIMPL 50 9.5 2.4 0.43 60 55 1.8 1.1 70 99 2.6 0.82 80 61 4.6 1.25 90 422 6.2 1.7 100 4458 4.4 2.8 Slide 67

SIMPL s advantage grows with the problem size No. Products MILP CPLEX 9 Seconds MILP CPLEX 11 SIMPL 300 82 376 19 300 701 372 19 600 3515 4509 39 600 214 9416 131 Slide 68

SIMPL s advantage grows with the problem size No. Products MILP CPLEX 9 Seconds Nodes MILP CPLEX 11 SIMPL 300 82 376 19 300 701 372 19 600 3515 4509 39 600 214 9416 131 Slide 69 No. Products MILP CPLEX 9 MILP CPLEX 11 SIMPL 300 10,164 101,756 73 300 43,242 128,333 58 600 363,740 646,907 74 600 7,732 1,297,071 214

Product configuration Choose what type of each component, and how many Memory Memory Memory Memory Memory Memory Personal computer Power supply Disk drive Disk drive Disk drive Disk drive Disk drive Power supply Power supply Power supply Slide 70

Product configuration Unit cost of producing attribute Amount of attribute produced (< 0 if consumed): memory, heat, power, weight, etc. MILP model min v = A q, all L v U, all q M, all i, k k ik ik tk ik i ik ik c v =1, all i Quantity of component i installed Slide 71

Product configuration Unit cost of producing attribute Amount of attribute produced (< 0 if consumed): memory, heat, power, weight, etc. min c v v = A q, all iti i ik L v U, all Integrated model Quantity of component i installed Original hand-coded method: Thorsteinsson and Ottosson 2001. Slide 74

Product configuration Unit cost of producing attribute Amount of attribute produced by type t i of component i Amount of attribute produced (< 0 if consumed): memory, heat, power, weight, etc. min c v v = A q, all iti i ik L v U, all Integrated model Quantity of component i installed Original hand-coded method: Thorsteinsson and Ottosson 2001. Slide 75

Product configuration Unit cost of producing attribute Amount of attribute produced by type t i of component i Amount of attribute produced (< 0 if consumed): memory, heat, power, weight, etc. min c v v = A q, all iti i ik L v U, all t i is a variable inde Integrated model Original hand-coded method: Thorsteinsson and Ottosson 2001. Slide 76

Product configuration Linear inequality metaconstraint min c v v = q A, all i it ik L v U, all i Original hand-coded method: Thorsteinsson and Ottosson 2001. Slide 77

Product configuration Indeed linear metaconstraint min c v v = q A, all i it ik L v U, all i Original hand-coded method: Thorsteinsson and Ottosson 2001. Slide 78

Product configuration Propagation min c v v = q A, all i it ik L v U, all i This is propagated in the usual way Slide 79

Product configuration Propagation v z, all = i i ( ) element t,( q, A,, q A ), z, all i, i i i1 i in i min c v v = q A, all i it ik L v U, all i This is rewritten as This is propagated in the usual way Slide 80

Product configuration Propagation v z, all = i i ( ) element t,( q, A,, q A ), z, all i, i i i1 i in i This is propagated by (a) using specialized filters for element constraints of this form Slide 81

Product configuration Propagation v z, all = i i ( ) element t,( q, A,, q A ), z, all i, i i i1 i in i Slide 82 This is propagated by (a) using specialized filters for element constraints of this form, (b) adding knapsack cuts for the valid inequalities: i i ma k D t i min k D t i { } A q v, all ik { } i A q v, all ik i and (c) propagating the knapsack cuts. [ v, v ] is current domain of v

Product configuration Relaation min c v v = q A, all i it ik L v U, all i This is relaed as v v v Slide 83

Product configuration Relaation v z, all = i i ( ) element t,( q, A,, q A ), z, all i, i i i1 i in i min c v v = q A, all i it ik L v U, all i This is relaed by relaing this and adding the knapsack cuts. This is relaed as v v v Slide 84

Product configuration Relaation v z, all = i i ( ) element t,( q, A,, q A ), z, all i, i i i1 i in i This is relaed by writing each element constraint as a disunction of linear systems and writing a conve hull relaation of the disunction: z = A q, q = q i ik ik i ik k D k D ti ti Slide 85

Product configuration SIMPL model Recognized as indeed linear system Slide 86

Product configuration SIMPL model LP relaation is conve hull of disunction. CP relaation propagates bounds. Slide 87

Product configuration SIMPL model Generate knapsack cuts from associated valid inequalities. Slide 88

Product configuration SIMPL model Logical constraint on quantities Slide 89

Product configuration SIMPL model First branch on violated logical constraint on q i variables Slide 90

Product configuration SIMPL model Then branch on most violated t i in-domain constraint. Violated when domain of t i is not a singleton, or two or more associated q ik s are positive. Slide 91

Product configuration SIMPL model Then branch on least violated q i in-domain constraint. Create three branches: q i = nearest integer q i, q i < q i, q i > q i Slide 92

Product configuration SIMPL model Then branch on most violated logical constraint on t i variables (omitted) Slide 93

Product configuration SIMPL model Reduced-cost variable fiing for q i s Slide 94

Product configuration MILP (CPLEX 11) SIMPL Computational results SIMPL matches hand-coded integrated method, which was orders of magnitude faster than CPLEX. Again, CPLEX has become much faster, now somewhat faster than SIMPL. Slide 95 Nodes Sec. Nodes Sec. 1 0.07 56 0.49 1 0.10 32 0.25 31 0.68 186 1.67 1 0.02 28 0.24 1 0.12 32 0.33 1 0.07 9 0.09 10 0.18 35 0.30 1 0.10 32 0.25 1 0.05 28 0.22 1 0.12 14 0.13

Machine scheduling Assign obs to machines, and schedule the machines assigned to each machine within time windows. The obective is to minimize processing cost. Slide 96

Machine scheduling Job Data Eample Assign 5 obs to 2 machines. Schedule obs assigned to each machine without overlap. Machine A Machine B Slide 97

Machine scheduling MILP continuoustime model from Jain & Grossmann min i i i r s d p, all i i i i i = 1, all y + y 1, all > y + y + + 2, all >, i i i i y + y + 1, all >, i i i s s + p M(1 y ), all i i i i c { } { } p ma d min r, all i i i { } { } 0,1, y 0,1, all Slide 98

Machine scheduling MILP continuoustime model from Jain & Grossmann = 1 of ob precedes ob start time min i i i r s d p, all i i i i i y + y 1, all > y + y + + 2, all >, i i i i y + y + 1, all >, i i i s s + p M(1 y ), all i i i i c = 1, all { } { } p ma d min r, all i i i { } { } = 1 if ob assigned to machine i 0,1, y 0,1, all Slide 99

Machine scheduling Integrated model min c r s d p, all Start time of ob ( = i = ) Time windows disunctive ( s i),( p i), all i Jobs cannot overlap Machine assigned to ob Original hand-coded method: Jain and Grossman 2001. Slide 100

Machine scheduling Integrated approach Assign the obs in the master problem, to be solved by MILP. Schedule the obs in the subproblem, to be solved by CP. Original hand-coded method: Jain and Grossman 2001. Slide 101

Machine scheduling Integrated approach Assign the obs in the master problem, to be solved by MILP. Schedule the obs in the subproblem, to be solved by CP. The subproblem decouples into a separate scheduling problem on each machine. In this problem, the subproblem is a feasibility problem. Original hand-coded method: Jain and Grossman 2001. Slide 102

Machine scheduling Integrated model min c r s d p, all ( = i = ) Indeed linear metaconstraint disunctive ( s i ),( p i ), all i Slide 103

Machine scheduling Integrated model min c r s d p, all ( = i = ) Indeed linear metaconstraint disunctive ( s i ),( p i ), all i Disunctive scheduling metaconstraint Slide 104

Machine scheduling Integrated model min M M s + p, all r s d p, all Start time of ob ( = i = ) Time windows disunctive ( s i ),( p i ), all i Jobs cannot overlap For a fied assignment the subproblem on each machine i is Slide 105 min M M s + p, all with = i r s d p, all with = i ( s = i pi = i ) disunctive ( ),( )

Machine scheduling Logic-based Benders approach Suppose we assign obs 1,2,3,5 to machine A in iteration k. We can prove that there is no feasible schedule. Edge finding derives infeasibility by reasoning only with obs 2,3,5. So these obs alone create infeasibility. So we have a Benders cut ( = = = A) 2 3 5 Slide 106

Machine scheduling Logic-based Benders approach We want the master problem to be an MILP, which is good for assignment problems. So we write the Benders cut ( A) = = = 2 3 5 Using 0-1 variables: 2 + 3 + 5 2 A A A = 1 if ob 5 is assigned to machine A Slide 107

Machine scheduling The master problem is a relaation, formulated as an MILP: min 5 = 1 5 = 1 i A B A2 A3 A5 i p p c i A B { 0,1} i 9, etc. 9, etc. + + 2 Constraints derived from time windows Benders cut from machine A Slide 108

Machine scheduling The master problem is a relaation, formulated as an MILP: min 5 = 1 5 = 1 A2 A3 A5 i i p A p B c i A B { 0,1} i 9, etc. 9, etc. + + 2 Constraints derived from time windows Benders cut from machine A Benders cuts have been developed for min makespan and min tardiness (subproblem is an optimization problem) Slide 109 Also for cumulative scheduling.

Machine scheduling SIMPL model Machine assignment constraint Slide 110

Machine scheduling SIMPL model MILP relaation of the constraint (which is the constraint itself) goes into master problem Slide 111

Machine scheduling SIMPL model Definition of i variables for MILP master problem Slide 112

Machine scheduling SIMPL model CP-based propagation Slide 113

Machine scheduling SIMPL model Time window constraints recognized as indeed linear system Slide 114

Machine scheduling SIMPL model MILP formulation goes into master problem CP-based propagation Slide 115

Machine scheduling SIMPL model Disunctive scheduling constraint written as special case of cumulative scheduling constraint (resource consumption = 1, capacity = 1) Slide 116

Machine scheduling SIMPL model The CP problem goes into the Benders subproblem. A relaation of the constraint goes into the master Slide 117

Machine scheduling SIMPL model Generate logic-based Benders cuts when the subproblem is infeasible Slide 118

Machine scheduling SIMPL model Benders-based search, where problem restrictions are Benders subproblems and problem relaations are master problems. Slide 119

Machine scheduling Computational results Long processing times Jobs Machines MILP (CPLEX 11) Nodes Sec. SIMPL Benders Iter. Cuts Sec. Slide 120 3 2 1 0.00 2 1 0.00 7 3 1 0.00 13 16 0.12 12 3 3,351 6.6 26 35 0.73 15 5 2,779 8.8 20 29 0.83 20 5 33,321 882 13 82 5.4 22 5 352,309 10,563 69 98 9.6 SIMPL results are similar to original hand-coded results.

Machine scheduling Computational results Short processing times Jobs Machines MILP (CPLEX 11) Nodes Sec. SIMPL Benders Iter. Cuts Sec. 3 2 1 0.01 1 0 0.00 7 3 1 0.02 1 0 0.00 12 3 499 0.98 1 0 0.01 15 5 529 2.6 2 1 0.06 20 5 250,047 369 6 5 0.28 22 5 > 27.5 mil. > 48 hr 9 12 0.42 25 5 > 5.4 mil. > 19 hr* 17 21 1.09 Slide 121 *out of memory

Machine scheduling Benders cut for minimum makespan, with a cumulative scheduling subproblem M M p (1 ) + ma d min d i { } { } * i i i Ji Ji J Jobs currently assigned to machine i Minimum makespan on machine i for obs currently assigned Slide 122

Logic-based Benders Decomposition In general, Benders cuts are obtained by solving the inference dual of the subproblem. The dual solution is a proof of optimality. LP dual is a special case, where the proof is encoded by dual multipliers. The Benders cut states conditions on the master problem variables under which the proof remains valid. Classical Benders cut is a special case. LP, surrogate, Lagrangean, and superadditive duals are special cases of inference duality and relaation duality. Whence the prevalence of relaation and inference dualities in problem solving. Slide 125

Truss Structure Design Select size of each bar (possibly zero) to support the load while minimizing weight. 10-bar cantilever truss 0 deg. freedom 2 deg. freedom Slide 126 Total 8 degrees of freedom Load

Truss Structure Design Notation v i = elongation of bar s i = force along bar h = length of bar i i d = node displacement p = load along d.f. A i = cross-sectional area of bar Slide 127

Truss Structure Design min i i i } Minimize total weight s. t. cos θ s = p, all i h A i i cos θ d = v, alli i i } Equilibrium } Compatibility nonlinear Ei Av i i = s i, all i h i L U v v v, alli L U d d d, all \/ i i i ( A = A ) k i ik } Hooke s law } Elongation bounds } Displacement bounds } Logical disunction Area must be one of several discrete values A ik Slide 128 Constraints can be imposed for multiple loading conditions

Truss Structure Design Introducing new variables linearizes the problem but makes it much larger. MILP model Slide 129 min h A y i i ik ik k s. t. cos θ s = p, all i E h i i i i cos θ d = v, alli k i ik k A v = s, alli ik ik i L U v v v, alli i i i L U d d d, all k y ik = 1, all i 0-1 variables indicating size of bar i Elongation variable disaggregated by bar size Hooke s law becomes linear

Truss Structure Design Integrated approach Use the original model (don t introduce new variables) Branch by splitting the range of areas A i Generate a quasi-relaation, which is linear and much smaller than MILP model. Original hand-coded method: Bollapragada, Ghattas, and JNH 2001. Slide 130

Truss Structure Design Theorem (JNH 2005) If g(,y) is semihomogeneous in R n and concave in scalar y, then the following is a quasi-relaation of g(,y) 0: g y g y 1 2 (, L ) + (, U ) 0 L 1 U α α L 2 (1 α ) (1 α ) = + 1 2 U Its optimal value is a lower bound on the optimal value of the original problem, if cost is a function of alone. Slide 132

Truss Structure Design Theorem (JNH 2005) If g(,y) is semihomogeneous in R n and concave in scalar y, then the following is a quasi-relaation of g(,y) 0: g y g y 1 2 (, L ) + (, U ) 0 L 1 U α α L 2 (1 α ) (1 α ) = + 1 2 U g( α, y ) αg(, y ) for all, y and α [0,1] g(0, y ) = 0 for all y Slide 133

Truss Structure Design Theorem (JNH 2005) If g(,y) is semihomogeneous in R n and concave in scalar y, then the following is a quasi-relaation of g(,y) 0: g y g y 1 2 (, L ) + (, U ) 0 L 1 U α α L 2 (1 α ) (1 α ) = + 1 2 U Ei Av i i = s i hi Slide 136 has the form g(,y) = 0 with g semihomogenous in because we can write it with = (A i,s i ), y = v i. Ei Av i i s i hi = 0

Truss Structure Design So we have a quasi-relaation of the truss problem: Slide 137 L U min h [ A y + A (1 y )] i i i i i i s. t. cos θ s = p, all i i i cos θ d = v + v, alli i i 0 i1 E ( A v + A v ) = s, all i h i L U i i 0 i i1 i i L U i i i 0 i i, all L U i i i1 i i L U, all v y v v y i v (1 y ) v v (1 y ), alli d d d 0 y 1, all i i Hooke s law is linearized Elongation bounds split into 2 sets of bounds

Truss Structure Design Logic cuts v i0 and v i1 must have same sign in a feasible solution. If not, we branch by adding logic cuts v, v 0, v, v 0 i 0 i1 i 0 i1 Slide 138

Truss Structure Design In general, we can have a metaconstraint to represent the semihomogeneous constraint g(,y) 0 and generate a quasirelaation. Since a bilinear constraint y = α is always semihomogeneous, we will use a bilinear metaconstraint with a quasi-relaation option. Slide 139

Truss Structure Design SIMPL model Recognized as linear systems Slide 140

Truss Structure Design SIMPL model Recognized as bilinear system Slide 141

Truss Structure Design SIMPL model Generate quasirelaation for semihomogenous function Slide 142

Truss Structure Design SIMPL model Branch first on violated logic cuts for quasirelaation Slide 143

Truss Structure Design SIMPL model Then branch on A i in-domain constraint. Violated when A i is not one of the discrete bar sizes. Take upper branch first. Slide 144

Truss Structure Design 10-bar cantilever truss Load Slide 145

Truss Structure Design Computational results (seconds) Hand-coded integrated method No. bars Loads BARON CPLEX 11 Hand coded SIMPL 10 1 5.3 0.40 0.03 0.08 10 1 3.8 0.26 0.02 0.07 10 1 8.1 0.83 0.16 0.49 10 1 8.8 1.2 0.22 0.63 10 2 24 4.9 0.64 1.84 10 2* 327 146 145 65 10 2* 2067 1087 600 651 Slide 146 *plus displacement bounds

Truss Structure Design 25-bar problem Slide 147

Truss Structure Design 72-bar problem Slide 148

Truss Structure Design Computational results (seconds) Hand-coded integrated method No. bars Loads BARON CPLEX 11 Hand coded SIMPL 25 2 3,302 44 44 20 72 2 3,376 208 33 28 90 2 21,011 570 131 92 108 2 > 24 hr* 3208 1907 1720 200 2 > 24 hr* > 24 hr* > 24 hr** > 24 hr*** Slide 149 * no feasible solution found ** best feasible solution has cost 32,748 *** best feasible solution has cost 32,700

Current Version of SIMPL To download: Click the link to SIMPL on John Hooker s website. See readme file for complete instructions. Download eecutable and associated files Operational on GNU/Linu only Requires subsidiary solvers CPLEX (version 9, 10, or 11) Eclipse (any version 5.8.80 or later), free download Download problem instances Including all reported in this talk. Slide 150

Proposal for SCIP/SIMPL cooperation A unique opportunity What SCIP/SIMPL can offer Primary tasks Slide 151

A unique opportunity Net generation of general-purpose solvers High-level modeling that integrates CP, MIP, GO, LS. Efficient code Eploits program micro-structure through meta-constraints Non-commercial SCIP brings efficient algorithmic tools Advanced MIP technology. SIMPL brings eperience in integrated problem-solving High-level modeling Slide 152

SCIP offers highly efficient noncommercial LP solver SIMPL currently uses CPLEX. well-developed MIP code. SIMPL has rudimentary MIP features. many cutting planes SIMPL implements few. constraint handles For user-supplied constraints Slide 153

SIMPL offers high-level modeling language SCIP uses ZIMPL language. built-in integration of CP, MIP, GO. SCIP leaves this largely to user. some CP filters (needs more) SCIP leaves it to user to write constraint handles. nogood-based search (logic-based Benders) Not available in SCIP Slide 154

Primary tasks Implement library of metaconstraints Design high-level modeling language or user interface Filters Relaations / conveification / rules for pooling relaations Cutting planes / disunctive MIP formulations Constraint-based branching choices Implement single branching scheme for CP & MIP Combine propagation and relaation Implement other search schemes Nogood-based search Continuous global optimization Local search? Perhaps integrate these with CP/MIP branching Slide 155