A Search-Infer-and-Relax Framework for. Integrating Solution Methods. Carnegie Mellon University CPAIOR, May John Hooker

Transcription

1 A Search-Infer-and-Rela Framework for Integratng Soluton Methods John Hooker Carnege Mellon Unversty CPAIOR, May 005 CPAIOR 005

2 Why ntegrate soluton methods? One-stop shoppng. One solver does t all. CPAIOR 005

3 Why ntegrate soluton methods? One-stop shoppng. One solver does t all. More modelng optons. Natural models, less debuggng & development tme. CPAIOR 005

4 Why ntegrate soluton methods? One-stop shoppng. One solver does t all. More modelng optons. Natural models, less debuggng & development tme. Computatonal speedup. A selecton of results CPAIOR 005

5 Computatonal Advantage of Integratng CP and MILP Usng CP + relaaton from MILP Problem Speedup Focacc, Lod, Mlano (999) Lesson tmetablng to 50 tmes faster than CP Refalo (999) Pecewse lnear costs to 00 tmes faster than MILP Hooker & Osoro (999) Flow shop schedulng, etc. 4 to 50 tmes faster than MILP. Thorstensson & Ottosson (00)* Product confguraton 30 to 40 tmes faster than CP, MILP CPAIOR 005 *Wll dscuss

6 Computatonal Advantage of Integratng CP and MILP Usng CP + relaaton from MILP Problem Speedup Sellmann & Fahle (00) Automatc recordng to 0 tmes faster than CP, MILP Van Hoeve (00) Stable set problem Better than CP n less tme Bollapragada, Ghattas & Hooker (00) Beck & Refalo (003) Structural desgn (nonlnear) Schedulng wth earlness & tardness costs Up to 600 tmes faster than MILP Solved 67 of 90, CP solved only CPAIOR 005

7 Computatonal Advantage of Integratng CP and MILP Usng CP-based Branch and Prce Problem Speedup Yunes, Moura & de Souza (999) Easton, Nemhauser & Trck (00) Urban transt crew schedulng Travelng tournament schedulng Optmal schedule for 0 trps, vs. 0 for tradtonal branch and prce Frst to solve 8-team nstance CPAIOR 005 *Wll dscuss

8 Computatonal Advantage of Integratng CP and MILP Usng CP/MILP Benders methods Problem Speedup Jan & Grossmann (00)* Thorstensson (00) Mn-cost plannng & schedung Mn-cost plannng & schedulng Tmpe (00) Polypropylene batch schedulng at BASF 0 to 000 tmes faster than CP, MILP 0 tmes faster than Jan & Grossmann Solved prevously nsoluble problem n 0 mn CPAIOR 005 *Wll dscuss *Wll dscuss

9 Computatonal Advantage of Integratng CP and MILP Usng CP/MILP Benders methods Problem Speedup Benost, Gaudn, Rottembourg (00) Call center schedulng Solved twce as many nstances as tradtonal Benders Hooker (004) Mn-cost, mn-makespan plannng & cumulatve schedulng Hooker (005) Mn tardness plannng & cumulatve schedulng tmes faster than CP, MILP tmes faster than CP, MILP CPAIOR 005

10 Proposal: Vew soluton methods as specal cases of the same basc algorthm. CPAIOR 005

11 Proposal: Vew soluton methods as specal cases of the same basc algorthm. Hybrd methods can then be vewed as other specal cases of the same basc algorthm. CPAIOR 005

12 The basc algorthm: Search: Enumerate problem restrctons e.g., search tree nodes, Benders subproblems, neghborhoods CPAIOR 005

13 The basc algorthm: Search: Enumerate problem restrctons e.g., search tree nodes, Benders subproblems, neghborhoods Infer: Deduce constrants from current restrcton e.g., Reduced domans, cuttng planes, nogoods, Benders cut CPAIOR 005

14 The basc algorthm: Search: Enumerate problem restrctons e.g., search tree nodes, Benders subproblems, neghborhoods Infer: Deduce constrants from current restrcton e.g., Reduced domans, cuttng planes, nogoods, Benders cuts Rela: Solve relaaton of current restrcton e.g., doman store, lnear programmng relaaton, Lagrangean relaaton, Benders master problem. CPAIOR 005

15 The basc algorthm: Search: Enumerate problem restrctons e.g., search tree nodes, Benders subproblems, neghborhoods Infer: Deduce constrants from current restrcton e.g., Reduced domans, cuttng planes, nogoods, Benders cuts Rela: Solve relaaton of current restrcton e.g., doman store, lnear programmng relaaton, Lagrangean relaaton, Benders master problem Selecton functon determnes whch soluton of relaaton to use. CPAIOR 005

16 The basc algorthm: Search: Enumerate problem restrctons e.g., search tree nodes, Benders subproblems, neghborhoods Infer: Deduce constrants from current restrcton e.g., Reduced domans, cuttng planes, nogoods, Benders cuts Rela: Solve relaaton of current restrcton e.g., doman store, lnear programmng relaaton, Lagrangean relaaton, Benders master problem Selecton functon determnes whch soluton of relaaton to use. Use post-relaaton nference f desred. Soluton of relaaton gudes choce of net restrcton. CPAIOR 005

17 The basc algorthm: Search: Enumerate problem restrctons P,, P k CPAIOR 005

18 The basc algorthm: Search: Enumerate problem restrctons P,, P k Infer: Deduce constrants from each P and add them to P. CPAIOR 005

19 The basc algorthm: Search: Enumerate problem restrctons P,, P k Infer: Deduce constrants from each P and add them to P Rela: Formulate a relaaton R of each P. CPAIOR 005

20 The basc algorthm: Search: Enumerate problem restrctons P,, P k Infer: Deduce constrants from each P and add them to P Rela: Formulate a relaaton R of each P. Select a soluton s(r ) of R. CPAIOR 005

21 The basc algorthm: Search: Enumerate problem restrctons P,, P k Infer: Deduce constrants from each P and add them to P Rela: Formulate a relaaton R of each P. Select a soluton s(r ) of R. Let s(r ) gude deducton of further constrants from P. CPAIOR 005

22 The basc algorthm: Search: Enumerate problem restrctons P,, P k Infer: Deduce constrants from each P and add them to P Rela: Formulate a relaaton R of each P. Select a soluton s(r ) of R. Let s(r ) gude deducton of further constrants from P. Let s(r ) gude choce of P +. CPAIOR 005

23 CPAIOR 005 Underlyng dea: Prmal-dual algorthm eplots dualty of problem restrcton and relaaton.

24 The algorthm s nterestng only when you see how t works out n eamples. CPAIOR 005

25 The algorthm s nterestng only when you see how t works out n eamples. Don t read the paper. It contans no eamples and sn t nterestng. CPAIOR 005

26 The algorthm s nterestng only when you see how t works out n eamples. Don t read the paper. It contans no eamples and sn t nterestng. If you must read t, read the revsed verson on my webste. CPAIOR 005

27 The algorthm s nterestng only when you see how t works out n eamples. Don t read the paper. It contans no eamples and sn t nterestng. If you must read t, read the revsed verson on my webste. After lookng at the eamples, you can udge whether ths framework s helpful or artfcal. CPAIOR 005

28 Soluton Method Branchng CP, MILP, global optmzaton Constrant drected DPL, dynamc backtrackng, Benders Heurstcs Local search, GRASP CPAIOR 005 General Framework Restrcton Relaaton R k Selecton P k Functon s(r k ) Node of search tree LP, NLP, domans Any optmal (feasble) soluton of R k Add nogoods generated so far Processed nogoods generated so far Soluton that results n easy R k Neghborhood of current soluton Research topc Random, best, etc. Inference Doman flterng, cuttng planes Nogood generaton?

29 CP+MILP CPAIOR 005 Branchng methods Contnuous global optmzaton Structure of Talk Depth-frst traversal of tree: General framework Constrant -drected search DPL Logc-based Benders decomposton Partal order dynamc backtrackng Heurstc methods Tabu search Smulated annealng GRASP

30 Soluton Method Branchng CP, MILP, global optmzaton Constrant drected DPL, dynamc backtrackng, Benders Heurstcs Local search, GRASP CPAIOR 005 General Framework Restrcton Relaaton R k Selecton P k Functon s(r k ) Node of search tree Add nogoods generated so far LP, NLP, domans Processed nogoods generated so far Any optmal (feasble) soluton of R k Soluton that results n easy R k Neghborhood of current soluton Research topc Random, best, etc. Inference Doman flterng, cuttng planes Nogood generaton?

31 Branchng Methods Soluton Method Restrcton Relaaton R k Selecton P k Functon s(r k ) Inference CP Created by splttng doman, etc. Current domans Any feasble Doman soluton of R k flterng, propagaton MILP Branch and cut Contnuous global optmzaton Created by branchng on fractonal varables Created by splttng ntervals LP relaaton + cuttng planes LP or conve NLP relaaton Optmal Cuttng planes, soluton of R k preprocessng Optmal soluton of R k. Interval propagaton, Lagrangean boundng CPAIOR 005

32 Branchng Methods Soluton Method Restrcton Relaaton R k Selecton P k Functon s(r k ) CP Created by splttng doman, etc. Current domans Any feasble soluton of R k MILP Branch and cut Contnuous global optmzaton Created by branchng on fractonal varables Created by splttng ntervals LP relaaton + cuttng planes LP or conve NLP relaaton Optmal soluton of R k Optmal soluton of R k. CPAIOR 005 Inference Doman flterng, propagaton Cuttng planes, preprocessng Interval propagaton, Lagrangean boundng

33 Memory Memory Memory CPAIOR 005 Branchng Search: Product Confguraton by CP/MILP Choose what type of each component, and how many Personal computer Memory Memory Memory Dsk drve Dsk drve Power supply Power supply Power supply Power supply Dsk drve Dsk drve Dsk drve

34 CPAIOR 005 Ths eample llustrates how a hybrd method s a specal case of the general algorthm.

35 Amount of attrbute produced (< 0 f consumed): memory, heat, power, weght, etc. CPAIOR 005 Model of the problem mn v L = k v c q v A U t, all, all Quantty of component nstalled

36 Amount of attrbute produced (< 0 f consumed): memory, heat, power, weght, etc. CPAIOR 005 mn v L = k v c q v A U t, Quantty of component nstalled Amount of attrbute produced by type t of component, all all

37 Amount of attrbute produced (< 0 f consumed): memory, heat, power, weght, etc. CPAIOR 005 mn v L = k v c q v A U t, Quantty of component nstalled, Amount of attrbute produced by type t of component all all t s a varable nde

38 Unt cost of producng attrbute Amount of attrbute produced (< 0 f consumed): memory, heat, power, weght, etc. mn v L = k v c q v A U t, Quantty of component nstalled CPAIOR 005, Amount of attrbute produced by type t of component all all t s a varable nde

39 To solve t: Search: branch on domans of t and q. Each node of search tree s a problem restrcton. CPAIOR 005

40 To solve t: Search: branch on domans of t and q. Each node of search tree s a problem restrcton. Infer: propagate ndeed lnear constrant and bounds on v. CPAIOR 005

41 To solve t: Search: branch on domans of t and q. Each node of search tree s a problem restrcton. Infer: propagate ndeed lnear constrant and bounds on v. Varable nde s converted to specally structured element constrant, whch s fltered. CPAIOR 005

42 To solve t: Search: branch on domans of t and q. Each node of search tree s a problem restrcton. Infer: propagate ndeed lnear constrant and bounds on v. Varable nde s converted to specally structured element constrant, whch s fltered. Vald knapsack cuts are derved and propagated. CPAIOR 005

43 To solve t: Search: branch on domans of t and q. Each node of search tree s a problem restrcton. Infer: propagate ndeed lnear constrant and bounds on v. Varable nde s converted to specally structured element constrant, whch s fltered. Vald knapsack cuts are derved and propagated. Rela: use lnear contnuous relaatons. CPAIOR 005

44 To solve t: Search: branch on domans of t and q. Each node of search tree s a problem restrcton. Infer: propagate ndeed lnear constrant and bounds on v. Varable nde s converted to specally structured element constrant, whch s fltered. Vald knapsack cuts are derved and propagated. Rela: use lnear contnuous relaatons. Use specal purpose relaaton for element CPAIOR 005

45 To solve t: Search: branch on domans of t and q. Each node of search tree s a problem restrcton. Infer: propagate ndeed lnear constrant and bounds on v. Varable nde s converted to specally structured element constrant, whch s fltered. Vald knapsack cuts are derved and propagated. Rela: use lnear contnuous relaatons. Use specal purpose relaaton for element. Selecton functon: Any optmal soluton of relaaton. CPAIOR 005

46 mn v L = k v c q v A U CPAIOR 005 t, Infer (propagate), all all Ths s propagated n the usual way

47 v L k v CPAIOR 005 c q v A U t mn = Infer (propagate) v = element z, all ( t,( q A,, q A ), z ), all, n,, all all Ths s automatcally rewrtten as Ths s propagated n the usual way

48 Infer (propagate) v = element z, all ( t,( q A,, q A ), z ), all, n Ths s propagated by (a) usng specalzed flters for element constrants of ths form CPAIOR 005

49 Infer (propagate) v = element z ( t,( q A,, q A ), z ), all,, all n Ths s propagated by (a) usng specalzed flters for element constrants of ths form, (b) addng knapsack cuts for the vald nequaltes: ma k k D D t mn t { A } k { A } k q q v v,, all all [ v, v ] s current doman of v CPAIOR 005

50 Infer (propagate) v = element z ( t,( q A,, q A ), z ), all,, all n Ths s propagated by (a) usng specalzed flters for element constrants of ths form, (b) addng knapsack cuts for the vald nequaltes: ma k k D D { A } { A } and (c) propagatng the knapsack cuts. t mn t k k q q v v,, all all [ v, v ] s current doman of v CPAIOR 005

51 mn v L = k v c q v A U CPAIOR 005 t,, all all Rela Ths s relaed as v v v

52 mn v L = k v c q v A U CPAIOR 005 t,, Rela v = element z all all, all ( t,( q A,, q A ), z ), all, n Ths s relaed by relang ths and addng the knapsack cuts. Ths s relaed as v v v

53 CPAIOR 005 Rela v = element z, all ( t,( q A,, q A ), z ), all, n Ths s relaed by replacng each element constrant wth a conve hull relaaton of a dsunctve programmng constrant: A = k q k q z, = q k D k D t t k

54 Rela So the followng LP relaaton s solved at each node of the search tree to obtan a lower bound: CPAIOR 005 mn v q v q q k = = k v q c v k D D t q k v q A 0, all, k t k q all all all k knapsack cuts for knapsack cuts for,,,, all ma k k D D t mn t { A } k { A } k q q v v, all, all

55 Branchng Methods Soluton Method Restrcton Relaaton R k Selecton P k Functon s(r k ) Inference CP Created by splttng doman, etc. Current domans Any feasble Doman soluton of R k flterng, propagaton MILP Branch and cut Contnuous global optmzaton Created by branchng on fractonal varables Created by splttng ntervals LP relaaton + cuttng planes LP or conve NLP relaaton Optmal Cuttng planes, soluton of R k preprocessng Optmal soluton of R k. Interval propagaton, Lagrangean boundng CPAIOR 005

56 ma 4 CPAIOR 005 Branchng Search: Contnuous Global Optmzaton + = + [0,], [0,] Global optmum Local optmum Feasble set

57 To solve t: Search: splt nterval domans of,. Each node of search tree s a problem restrcton. CPAIOR 005

58 To solve t: Search: splt nterval domans of,. Each node of search tree s a problem restrcton. Infer: Interval propagaton, post-relaaton nference. CPAIOR 005

59 To solve t: Search: splt nterval domans of,. Each node of search tree s a problem restrcton. Infer: Interval propagaton, post-relaaton nference. Post-relaaton nference: Use Lagrange multplers to nfer vald nequalty for propagaton. Reduced-cost varable fng s a specal case. CPAIOR 005

60 To solve t: Search: splt nterval domans of,. Each node of search tree s a problem restrcton. Infer: Interval propagaton, post-relaaton nference. Post-relaaton nference: Use Lagrange multplers to nfer vald nequalty for propagaton. Reduced-cost varable fng s a specal case. Rela: Use functon factorzaton to obtan lnear contnuous relaaton. Selecton functon: Any optmal soluton of relaaton. CPAIOR 005

61 Infer (nterval propagaton) Propagate ntervals [0,], [0,] through constrants to obtan [/8,7/8], [/4,7/4] CPAIOR 005

62 Rela (functon factorzaton) Factor comple functons nto elementary functons that have known lnear relaatons. CPAIOR 005

63 Rela (functon factorzaton) Factor comple functons nto elementary functons that have known lnear relaatons. Wrte 4 = as 4y = where y =. Ths factors 4 nto lnear functon 4y and blnear functon. CPAIOR 005

64 Rela (functon factorzaton) Factor comple functons nto elementary functons that have known lnear relaatons. Wrte 4 = as 4y = where y =. Ths factors 4 nto lnear functon 4y and blnear functon. Lnear functon 4y s ts own lnear relaaton. CPAIOR 005

65 CPAIOR 005 Rela (functon factorzaton) Factor comple functons nto elementary functons that have known lnear relaatons. Wrte 4 = as 4y = where y =. Ths factors 4 nto lnear functon 4y and blnear functon. Lnear functon 4y s ts own lnear relaaton. Blnear functon y = has relaaton: y y Where doman of s ], [

66 CPAIOR 005 Rela The lnear relaaton becomes:,, 4 mn = = + y y y

67 Solve lnear relaaton. CPAIOR 005 Rela

68 Rela [,.75] Solve lnear relaaton. Snce soluton s nfeasble, splt an nterval and branch. [0.5,] CPAIOR 005

69 CPAIOR 005 [,.75] [0.5, ]

70 CPAIOR 005 [,.75] [0.5, ] Soluton of relaaton s feasble, value =.5 Ths becomes ncumbent soluton

71 CPAIOR 005 [,.75] [0.5, ] Soluton of relaaton s feasble, value =.5 Ths becomes ncumbent soluton Soluton of relaaton s not qute feasble, value =.854 Also use postrelaaton nference

72 CPAIOR 005 Post-Relaaton Inference,, 4 mn = = + y y y Assocated Lagrange multpler n soluton of relaaton s.

73 CPAIOR 005 Post-Relaaton Inference,, 4 mn = = + y y y Assocated Lagrange multpler n soluton of relaaton s = + Ths yelds a vald nequalty for propagaton:

74 CPAIOR 005 Post-Relaaton Inference,, 4 mn = = + y y y Assocated Lagrange multpler n soluton of relaaton s = + Ths yelds a vald nequalty for propagaton: Value of relaaton

75 CPAIOR 005 Post-Relaaton Inference,, 4 mn = = + y y y Assocated Lagrange multpler n soluton of relaaton s = + Ths yelds a vald nequalty for propagaton: Value of relaaton Lagrange multpler

76 CPAIOR 005 Post-Relaaton Inference,, 4 mn = = + y y y Assocated Lagrange multpler n soluton of relaaton s = + Ths yelds a vald nequalty for propagaton: Value of relaaton Value of ncumbent soluton Lagrange multpler

77 Post-Relaaton Inference Reduced-cost varable fng s a specal case. CPAIOR 005

78 Post-Relaaton Inference Reduced-cost varable fng s a specal case. Separatng cuts represent another form of post-relaaton nference. CPAIOR 005

79 Soluton Method Branchng CP, MILP, global optmzaton Constrant drected DPL, dynamc backtrackng, Benders Heurstcs Local search, GRASP CPAIOR 005 General Framework Restrcton Relaaton R k Selecton P k Functon s(r k ) Node of search tree LP, NLP, domans Any optmal (feasble) soluton of R k Add nogoods generated so far Processed nogoods generated so far Soluton that results n easy R k Neghborhood of current soluton Research topc Random, best, etc. Inference Doman flterng, cuttng planes Nogood generaton?

80 Constrant-Drected Search Soluton Method Restrcton P k Relaaton R k Selecton Functon s(r k ) DPL for SAT Add conflct clauses Processed conflct clauses Unt clause rule + greedy soluton of R k Partal order dynamc backtrackng Add nogoods Processed nogoods Greedy, consstent wth partal order Logc-based Benders Subproblem defned by soluton of master Master problem (Benders cuts) Optmal soluton of master CPAIOR 005 Inference Parallel resoluton & absorpton Parallel resoluton & absorpton Benders cuts (nogoods)

81 Constrant-Drected Search Soluton Method Restrcton P k Relaaton R k Selecton Functon s(r k ) DPL for SAT Add conflct clauses Processed conflct clauses Unt clause rule + greedy soluton of R k Partal order dynamc backtrackng Add nogoods Processed nogoods Greedy, consstent wth partal order Logc-based Benders Subproblem defned by soluton of master Master problem (Benders cuts) Optmal soluton of master CPAIOR 005 Inference Parallel resoluton & absorpton Parallel resoluton & absorpton Benders cuts (nogoods)

82 CPAIOR 005 Constrant-Drected Search: DPL for Propostonal Satsfablty DPL (Davs-Putnam-Loveland) wth clause learnng can be nterpreted as constrant-drected search

83 To solve t by branchng: Search: branch on. Each node of search tree s a problem restrcton. CPAIOR 005

84 To solve t by branchng: Search: branch on. Each node of search tree s a problem restrcton. Infer: clause learnng, unt clause rule. CPAIOR 005

85 To solve t by branchng: Search: branch on. Each node of search tree s a problem restrcton. Infer: clause learnng, unt clause rule. Rela: not used. CPAIOR 005

86 CPAIOR 005 Branchng = 0 = 0 3 = 0 4 = 0 5 = 0 Branch to here. Unt clause rule proves nfeasblty. (, 5 ) = (0,0) creates the conflct.

87 CPAIOR 005 Branchng = 0 = 0 3 = 0 4 = 0 5 = 0 Branch to here. Unt clause rule proves nfeasblty. 5 (, 5 ) = (0,0) creates the conflct. Add conflct clause to constrant set.

88 CPAIOR 005 Branchng = 0 = 0 3 = 0 4 = 0 = 0 = Backtrack and branch to here. (, 5 ) = (0,) creates the conflct. Generate conflct clause

89 CPAIOR 005 Branchng = 0 = 0 4 = 0 3 = 0 = 0 = 5 5 Backtrack to here and note that (, ) = (0,0) s enough to create conflct. Generate new conflct clause 5 5 Backtrack and branch to here. (, 5 ) = (0,) creates the conflct. Generate conflct clause

90 CPAIOR 005 Branchng = 0 = 0 = 3 = 0 4 = 0 = 0 = Backtrack and branch to here and generate conflct clause

91 CPAIOR 005 Branchng = 0 = 0 = 4 = 0 3 = 0 = 0 = Backtrack to here and generate new conflct clause Backtrack and branch to here and generate conflct clause

92 CPAIOR 005 Branchng = 0 = = 0 4 = 0 3 = 0 = 0 = Backtrack and branch to here and generate fnal conflct clause

93 To solve t by constrant-drected search: Search: generate problem restrctons. Each leaf node of search tree s a problem restrcton. CPAIOR 005

94 To solve t by constrant-drected search: Search: generate problem restrctons. Each leaf node of search tree s a problem restrcton. Infer: generate nogoods. Process nogoods n current relaaton wth parallel resoluton. CPAIOR 005

95 To solve t by constrant-drected search: Search: generate problem restrctons. Each leaf node of search tree s a problem restrcton. Infer: generate nogoods. Process nogoods n current relaaton wth parallel resoluton. Rela: Relaaton R k conssts of current processed nogoods. CPAIOR 005

96 To solve t by constrant-drected search: Search: generate problem restrctons. Each leaf node of search tree s a problem restrcton. Infer: generate nogoods. Process nogoods n current relaaton wth parallel resoluton. Rela: Relaaton R k conssts of current processed nogoods. Selecton functon: Mmc chronologcal backtrackng; apply unt clause rule. CPAIOR 005

97 Constrant-Drected Search = 0 = 0 = 0 3 k Relaaton R k Soluton of R k Nogoods 0 (0,0,0,0,0, ) = 0 5 = 0 Conflct clause appears as nogood nduced by soluton of R k. 5 CPAIOR 005

98 Constrant-Drected Search = 0 Conssts of processed nogoods = 0 = 0 3 k Relaaton R k Soluton of R k Nogoods 0 (0,0,0,0,0, ) (0,0,0,0,, ) = 0 = 0 = CPAIOR 005

99 Constrant-Drected Search = 0 Conssts of processed nogoods 4 = 0 3 = 0 = 0 = 0 = k Relaaton R k Soluton of R k 0 (0,0,0,0,0, ) 5 (0,0,0,0,, ) Nogoods 5 parallel-resolve to yeld 5 parallel-absorbs CPAIOR 005

100 Constrant-Drected Search = 0 = 0 = CPAIOR = 0 3 = 0 = 0 = k Relaaton R k Soluton of R k 0 (0,0,0,0,0, ) 3 5 (0,0,0,0,, (0,, parallel-resolve to yeld,,, ) ) Nogoods 5 5

101 Constrant-Drected Search = 0 = = 0 4 = 0 3 = 0 = 0 = k Relaaton R k Soluton of R k 0 (0,0,0,0,0, ) (0,0,0,0,, (0,, (,,,,,,,, ) ) ) CPAIOR 005 Search termnates Nogoods 5 5

102 Constrant-Drected Search Soluton Method Restrcton P k Relaaton R k Selecton Functon s(r k ) DPL for SAT Add conflct clauses Processed conflct clauses Unt clause rule + greedy soluton of R k Partal order dynamc backtrackng Logc-based Benders Add nogoods Processed nogoods Subproblem defned by soluton of master Master problem (Benders cuts) Greedy, consstent wth partal order Optmal soluton of master CPAIOR 005 Inference Parallel resoluton & absorpton Parallel resoluton & absorpton Benders cuts (nogoods)

103 CPAIOR 005 Constrant-Drected Search: Partal Order Dynamc Backtrackng Solve same problem as before

104 To solve t: Search: generate problem restrctons. CPAIOR 005

105 To solve t: Search: generate problem restrctons. Infer: generate nogoods. Process nogoods n current relaaton wth parallel resoluton. CPAIOR 005

106 To solve t: Search: generate problem restrctons. Infer: generate nogoods. Process nogoods n current relaaton wth parallel resoluton. Rela: Relaaton R k conssts of current processed nogoods. Selecton functon: Soluton of R k must conform to current nogoods. Also apply unt clause rule. CPAIOR 005

107 Partal Order Dynamc Backtrackng k Relaaton R k Soluton of R k Nogoods 0 (0,0,0,0,0, ) 5 5 Arbtrarly choose one varable to be last CPAIOR 005

108 Partal Order Dynamc Backtrackng k Relaaton R k Soluton of R k Nogoods 0 (0,0,0,0,0, ) 5 5 Other varables are penultmate Arbtrarly choose one varable to be last CPAIOR 005

109 Partal Order Dynamc Backtrackng k Relaaton R k Soluton of R k Nogoods 0 (0,0,0,0,0, ) 5 5 (,,,,0, ) Snce 5 s penultmate n at least one nogood, t must conform to nogoods. It must take value opposte ts sgn n the nogoods. 5 wll have the same sgn n all nogoods where t s penultmate. CPAIOR 005 Ths allows more freedom than chronologcal backtrackng.

110 Partal Order Dynamc Backtrackng k Relaaton R k Soluton of R k 0 (0,0,0,0,0, ) 5 (,,,,0, ) Nogoods 5 5 Choce of last varable s arbtrary but must be consstent wth partal order mpled by prevous choces. Snce 5 s penultmate n at least one nogood, t must conform to nogoods. It must take value opposte ts sgn n the nogoods. 5 wll have the same sgn n all nogoods where t s penultmate. CPAIOR 005 Ths allows more freedom than chronologcal backtrackng.

111 Partal Order Dynamc Backtrackng k Relaaton R k Soluton of R k 0 (0,0,0,0,0, ) 5 5 (,,,,0, ) Nogoods 5 5 Choce of last varable s arbtrary but must be consstent wth partal order mpled by prevous choces. 5 5 Parallel-resolve to yeld 5 Snce 5 s penultmate n at least one nogood, t must conform to nogoods. It must take value opposte ts sgn n the nogoods. 5 wll have the same sgn n all nogoods where t s penultmate. CPAIOR 005 Ths allows more freedom than chronologcal backtrackng.

112 Partal Order Dynamc Backtrackng k Relaaton R k Soluton of R k 0 (0,0,0,0,0, ) (, (,,0,,,,0,,, ) ) Nogoods does not parallel-resolve wth 5 because 5 s not last n both clauses CPAIOR 005

113 CPAIOR ),,,,, ( 3 ),,,,0, ( ),0,,, (, ) (0,0,0,0,0, 0 Nogoods Soluton of Relaaton R R k k k Partal Order Dynamc Backtrackng Must conform Search termnates

114 Constrant-Drected Search Soluton Method Restrcton P k Relaaton R k Selecton Functon s(r k ) DPL for SAT Add conflct clauses Processed conflct clauses Unt clause rule + greedy soluton of R k Partal order dynamc backtrackng Add nogoods Processed nogoods Greedy, consstent wth partal order Logc-based Benders Subproblem defned by soluton of master Master problem (Benders cuts) Optmal soluton of master CPAIOR 005 Inference Parallel resoluton & absorpton Parallel resoluton & absorpton Benders cuts (nogoods)

115 Constrant-Drected Search: Logc-Based Benders Decomposton Plannng and schedulng problem: Allocate tasks to facltes. Schedule tasks assgned to each faclty. Subect to deadlnes. Facltes may run at dfferent speeds and ncur dfferent costs. Cumulatve schedulng Several tasks may run smultaneously on a faclty. But total resource consumpton must never eceed lmt. CPAIOR 005

116 Plannng and Schedulng p = processng tme of task on faclty c = resource consumpton of task on faclty C = resources avalable on faclty C task c Faclty Faclty task 4 task 5 C c task 3 task p p CPAIOR 005 Total resource consumpton C at all tmes.

117 CPAIOR 005 p d t C y c y p y t c y y all, 0 all, ) ( ) ( ) ( cumulatve mn = = = y = faclty assgned to task Plannng and Schedulng

118 CPAIOR 005 p d t C y c y p y t c y y all, 0 all, ) ( ) ( ) ( cumulatve mn = = = y = faclty assgned to task Cost of processng task on faclty Plannng and Schedulng

119 CPAIOR 005 p d t C y c y p y t c y y all, 0 all, ) ( ) ( ) ( cumulatve mn = = = Observe resource lmt on each faclty y = faclty assgned to task start tmes of tasks assgned to faclty Cost of processng task on faclty Plannng and Schedulng

120 Plannng and Schedulng mn c y cumulatve 0 t d Observe tme wndows CPAIOR 005 p t p c y Cost of processng task on faclty y = faclty assgned to task start tmes of tasks assgned to faclty y y y C, = all ) ) ( ( = = ), all ( Observe resource lmt on each faclty

121 To solve t: Search: enumerate assgnments of tasks to facltes. Each assgnment defnes a problem restrcton (schedulng subproblem). CPAIOR 005

122 To solve t: Search: enumerate assgnments of tasks to facltes. Each assgnment defnes a problem restrcton (schedulng subproblem). Infer: generate nogoods. Nogoods (Benders cuts) eclude assgnments that have no feasble schedule. CPAIOR 005

123 To solve t: Search: enumerate assgnments of tasks to facltes. Each assgnment defnes a problem restrcton (schedulng subproblem). Infer: generate nogoods. Nogoods (Benders cuts) eclude assgnments that have no feasble schedule. Rela: Relaaton R k conssts of Benders cuts generated so far. CPAIOR 005

124 To solve t: Search: enumerate assgnments of tasks to facltes. Each assgnment defnes a problem restrcton (schedulng subproblem). Infer: generate nogoods. Nogoods (Benders cuts) eclude assgnments that have no feasble schedule. Rela: Relaaton R k conssts of Benders cuts generated so far. Selecton functon: Any optmal soluton of R k (= master problem). CPAIOR 005

125 CPAIOR 005 d t C y c y p y t all, 0 ) ( ) ( ) ( cumulatve = = = For a gven assgnment of tasks to facltes, fnd a feasble schedule Solve by constrant programmng Restrct (Subproblem) Gven assgnment

126 CPAIOR 005 d t C y c y p y t all, 0 ) ( ) ( ) ( cumulatve = = = Infer (Benders cut = nogood) Let J h = {tasks assgned to faclty n teraton h}. If there s no feasble schedule, create Benders cut: J h y for some

127 CPAIOR 005 mn J h ( Rela (Master problem) Solve by MILP. = c, {0,} ) all, all h, Let = when y = Task s assgned to one faclty Benders cuts: = 0 for some J h Stop when subproblem s feasble (orgnal problem s feasble) or when master problem s nfeasble (orgnal problem s nfeasble).

128 Soluton Method Branchng CP, MILP, global optmzaton Constrant drected DPL, dynamc backtrackng, Benders Heurstcs Local search, GRASP CPAIOR 005 General Framework Restrcton Relaaton R k Selecton P k Functon s(r k ) Node of search tree LP, NLP, domans Any optmal (feasble) soluton of R k Add nogoods generated so far Processed nogoods generated so far Soluton that results n easy R k Neghborhood of current soluton Research topc Random, best, etc. Inference Doman flterng, cuttng planes Nogood generaton?

129 Heurstc Methods Soluton Method Restrcton Relaaton R k Selecton P k Functon s(r k ) Smulated annealng Neghborhood of current soluton P k Random soluton from nbhd Tabu search Neghborhood mnus tabu lst P k Best soluton n nbhd GRASP Neghborhood of partal soluton Problem specfc Solve R k only at leaf node CPAIOR 005 Inference None Items n tabu lst None

130 Heurstc Methods Soluton Method Restrcton Relaaton R k Selecton P k Functon s(r k ) Smulated annealng Neghborhood of current soluton P k Random soluton from nbhd Tabu search Neghborhood mnus tabu lst P k Best soluton n nbhd GRASP Neghborhood of partal soluton Problem specfc Solve R k only at leaf node CPAIOR 005 Inference None Items n tabu lst None

131 Heurstcs: Smulated Annealng To solve t: Search: enumerate neghborhoods (restrctons). CPAIOR 005

132 Heurstcs: Smulated Annealng To solve t: Search: enumerate neghborhoods (restrctons). Infer: none CPAIOR 005

133 Heurstcs: Smulated Annealng To solve t: Search: enumerate neghborhoods (restrctons). Infer: none Rela: Same as restrcton. CPAIOR 005

134 Heurstcs: Smulated Annealng To solve t: Search: enumerate neghborhoods (restrctons). Infer: none Rela: Same as restrcton. Selecton functon: Random soluton n neghborhood. CPAIOR 005

135 Heurstcs: Smulated Annealng To solve t: Search: enumerate neghborhoods (restrctons). Infer: none Rela: Same as restrcton. Selecton functon: Random soluton n neghborhood. Net restrcton: neghborhood of f s better than prevous soluton; CPAIOR 005

136 Heurstcs: Smulated Annealng To solve t: Search: enumerate neghborhoods (restrctons). Infer: none Rela: Same as restrcton. Selecton functon: Random soluton n neghborhood. Net restrcton: neghborhood of f s better than prevous soluton; otherwse neghborhood of wth probablty p, current neghborhood wth probablty p. CPAIOR 005

137 Heurstc Methods Soluton Method Restrcton Relaaton R k Selecton P k Functon s(r k ) Smulated annealng Neghborhood of current soluton P k Random soluton from nbhd Tabu search Neghborhood mnus tabu lst P k Best soluton n nbhd GRASP Neghborhood of partal soluton Problem specfc Solve R k only at leaf node CPAIOR 005 Inference None Items n tabu lst None

138 Heurstcs: Tabu Search To solve t: Search: enumerate neghborhoods (restrctons). CPAIOR 005

139 Heurstcs: Tabu Search To solve t: Search: enumerate neghborhoods (restrctons). Infer: tem n tabu lst (functons as nogood) CPAIOR 005

140 Heurstcs: Tabu Search To solve t: Search: enumerate neghborhoods (restrctons). Infer: tem n tabu lst (functons as nogood) Rela: Same as restrcton. CPAIOR 005

141 Heurstcs: Tabu Search To solve t: Search: enumerate neghborhoods (restrctons). Infer: tem n tabu lst (functons as nogood) Rela: Same as restrcton. Selecton functon: Best soluton n neghborhood that s consstent wth tabu lst. CPAIOR 005

142 Heurstcs: Tabu Search To solve t: Search: enumerate neghborhoods (restrctons). Infer: tem n tabu lst (functons as nogood) Rela: Same as restrcton. Selecton functon: Best soluton n neghborhood that s consstent wth tabu lst. Net restrcton: neghborhood of. CPAIOR 005

143 Heurstc Methods Soluton Method Restrcton Relaaton R k Selecton P k Functon s(r k ) Smulated annealng Neghborhood of current soluton P k Random soluton from nbhd Tabu search Neghborhood mnus tabu lst P k Best soluton n nbhd GRASP Neghborhood of partal soluton Problem specfc Solve R k only at leaf node CPAIOR 005 Inference None Items n tabu lst None

144 CPAIOR 005 Heurstcs: Generalzed GRASP Greedy Randomzed Adaptve Search Procedure TSP wth Tme Wndows Home base 5 A B [0,35] [5,35] 3 5 E C 6 [5,5] 5 7 D [0,30]

145 Heurstcs: Generalzed GRASP To solve t: Search: enumerate neghborhoods of partal solutons Neghborng soluton s created by selectng customer to vst net. CPAIOR 005

146 Heurstcs: Generalzed GRASP To solve t: Search: enumerate neghborhoods of partal solutons Neghborng soluton s created by selectng customer to vst net. Infer: none CPAIOR 005

147 Heurstcs: Generalzed GRASP To solve t: Search: enumerate neghborhoods of partal solutons Neghborng soluton s created by selectng customer to vst net. Infer: none Rela: Same as restrcton. CPAIOR 005

148 Heurstcs: Generalzed GRASP To solve t: Search: enumerate neghborhoods of partal solutons Neghborng soluton s created by selectng customer to vst net. Infer: none Rela: Same as restrcton. Selecton functon: Greedy phase: Select net customer to vst n greedy fashon. CPAIOR 005

149 Heurstcs: Generalzed GRASP To solve t: Search: enumerate neghborhoods of partal solutons Neghborng soluton s created by selectng customer to vst net. Infer: none Rela: Same as restrcton. Selecton functon: Greedy phase: Select net customer to vst n greedy fashon. Local search phase: Randomly backtrack and select net customer n random fashon. CPAIOR 005

150 Sequence of customers vsted CPAIOR 005 Generalzed GRASP A A Bascally, GRASP = greedy soluton + local search Begn wth greedy assgnments that can be vewed as creatng branches

151 Greedy phase CPAIOR 005 Generalzed GRASP A A AD A Vst customer than can be served earlest from A Bascally, GRASP = greedy soluton + local search Begn wth greedy assgnments that can be vewed as creatng branches

152 Generalzed GRASP Greedy phase A A AD A ADC A Net, vst customer than can be served earlest from D CPAIOR 005 Bascally, GRASP = greedy soluton + local search Begn wth greedy assgnments that can be vewed as creatng branches

153 Greedy phase CPAIOR 005 ADC A ADCBEA Feasble Value = 34 Generalzed GRASP A A AD A Contnue untl all customers are vsted. Ths soluton s feasble. Save t. Bascally, GRASP = greedy soluton + local search Begn wth greedy assgnments that can be vewed as creatng branches

154 Local search phase CPAIOR 005 ADC A ADCBEA Feasble Value = 34 Generalzed GRASP A A AD A Backtrack randomly

155 Local search phase CPAIOR 005 ADC A ADCBEA Feasble Value = 34 Generalzed GRASP A A AD A Delete subtree already traversed

156 Local search phase CPAIOR 005 Generalzed GRASP A A AD A ADC A ADE A ADCBEA Feasble Value = 34 Randomly select partal soluton n neghborhood of current node

157 Greedy phase CPAIOR 005 Generalzed GRASP A A AD A ADC A ADE A ADCBEA Feasble Value = 34 ADEBCA Infeasble Complete soluton n greedy fashon

158 Local search phase CPAIOR 005 Generalzed GRASP A A Randomly backtrack AD A ADC A ADE A ADCBEA Feasble Value = 34 ADEBCA Infeasble

159 CPAIOR 005 Generalzed GRASP A A AD A AB A ADC A ADE A ABD A ADCBEA Feasble Value = 34 ADEBCA Infeasble ABDECA Infeasble Contnue n smlar fashon

160 CPAIOR 005 Local search s generalzed GRASP n whch the branchng tree has levels

161 To add a relaaton to generalzed GRASP: Suppose that customers 0,,, k have been vsted so far. Let t = travel tme from customer to. Then total travel tme of completed route s bounded below by T + {,, } 0 k { t, mn { t } mn + mn { 0 t k {, 0,, k } { 0,, k } } Earlest tme vehcle can leave customer k Mn tme from customer s predecessor to Mn tme from last customer back to home CPAIOR 005

162 Heurstcs: Generalzed GRASP wth relaaton To solve t: Search: enumerate neghborhoods of partal solutons Neghborng soluton s created by selectng customer to vst net. Infer: None. Rela: As ust descrbed. Selecton functon: Greedy phase: Select net customer to vst n greedy fashon. Local search phase: Randomly backtrack and select net customer n random fashon. CPAIOR 005

163 Greedy phase CPAIOR 005 Generalzed GRASP wth relaaton A A AD A

164 Greedy phase ADC A CPAIOR 005 Generalzed GRASP A A AD A

165 Greedy phase ADC A ADCBEA Feasble Value = 34 CPAIOR 005 Generalzed GRASP A A AD A

166 Local search phase CPAIOR 005 ADC A ADCBEA Feasble Value = 34 Generalzed GRASP A A AD A Backtrack randomly

167 Local search phase CPAIOR 005 Generalzed GRASP A A AD A ADC A ADE A Relaaton value = 40 Prune ADCBEA Feasble Value = 34

168 Local search phase CPAIOR 005 ADC A ADCBEA Feasble Value = 34 Generalzed GRASP A A Randomly backtrack AD A ADE A Relaaton value = 40 Prune

169 Local search phase CPAIOR 005 ADC A ADCBEA Feasble Value = 34 Generalzed GRASP A A AD A AB A Relaaton value = 38 Prune ADE A Relaaton value = 40 Prune

170 CPAIOR 005 Product endorsement: SIMPL s a partal mplementaton of ths approach (CP-AI-OR 004) We epect to post on the web ths fall.