DM545 Linear and Integer Programming. Lecture 13 Branch and Bound. Marco Chiarandini

Transcription

1 DM545 Linear and Integer Programming Lecture 13 Marco Chiarandini Department of Mathematics & Computer Science University of Southern Denmark

2 Outline

3 Exam Tilladt Håndscanner/digital pen og ordbogsprogrammet fra ordbogen.com Ikke tilladt at anvende digitalt kamera eller webcam o. lign. metoder for at digitalisere sin besvarelse Du afleverer efter fristen og kun en gang Exam Monitor er et lille program, som logger, hvilke programmer du afvikler på din computer under eksamen, samtidig med at din skærm optages. Internet Internet er ikke tilladt ved eksamener på NAT, men undtagelsesvis til denne eksamen er det tilladt, at benytte følgende webside http: // www. imada. sdu. dk/ ~marco/ DM545/ og siderne linket derfra. Det er ikke tilladt at benytte andre sider Vejledning og templates snart tilgænglig fra kurset web siden ved afsnittet Assessment Kom vel forberedet, bring noget at drikke og spise 3

4 To come Two weeks left This week: two lectures + joint training class on Wednesday Next week: two exercise classes + one lecture. Question time? Thursday 31st at 9:00? 4

5 Outline

6 Consider the problem z = max{c T x : x S} Divide and conquer: let S = S 1... S k be a decomposition of S into smaller sets, and let z k = max{c T x : x S k } for k = 1,..., K. Then z = max k z k 6

7 Consider the problem z = max{c T x : x S} Divide and conquer: let S = S 1... S k be a decomposition of S into smaller sets, and let z k = max{c T x : x S k } for k = 1,..., K. Then z = max k z k For instance if S {0, 1} 3 the enumeration tree is: x 1 = 0 S x 1 = 1 x 2 = 0 S 0 S 1 x 3 = 0 S 00 S 01 S 10 S 11 S 000 S 001 S 010 S 011 S 100 S 101 S 110 S 111 6

8 Bounding Let s consider a maximization problem (gurobi s default is minimization) Let z k be an upper bound on z k (dual bound) Let z k be a lower bound on z k (primal bound) (z k z k z k ) 7

9 Bounding Let s consider a maximization problem (gurobi s default is minimization) Let z k be an upper bound on z k (dual bound) Let z k be a lower bound on z k (primal bound) (z k z k z k ) z = 7

10 Bounding Let s consider a maximization problem (gurobi s default is minimization) Let z k be an upper bound on z k (dual bound) Let z k be a lower bound on z k (primal bound) (z k z k z k ) z = max k z k is a lower bound on z z = 7

11 Bounding Let s consider a maximization problem (gurobi s default is minimization) Let z k be an upper bound on z k (dual bound) Let z k be a lower bound on z k (primal bound) (z k z k z k ) z = max k z k is a lower bound on z z = max k z k is an upper bound on z 7

12 Pruning z = z =

13 Pruning z = 25 z = 20 pruned by optimality 8

14 Pruning z = 25 z = 20 pruned by optimality z = z =

15 Pruning z = 25 z = 20 pruned by optimality z = 26 z = 21 pruned by bounding 8

16 Pruning z = 25 z = 20 pruned by optimality z = 26 z = 21 pruned by bounding 40 z = z =

17 Pruning z = 25 z = 20 pruned by optimality z = 26 z = 21 pruned by bounding z = 37 z = 13 nothing to prune 8

18 Pruning infeas. z = 26 z = 14 pruned by infeasibility 9

19 Example max x 1 + 2x 2 x 1 + 4x 2 8 4x 1 + x 2 8 x 1, x 2 0, integer Solve LP x1 x2 x3 x4 -z b x 2 x 1 + 4x 2 = 8 x 1 x 1 + 2x 2 = 1 4x 1 + x 2 = 8 10

20 Example max x 1 + 2x 2 x 1 + 4x 2 8 4x 1 + x 2 8 x 1, x 2 0, integer Solve LP x1 x2 x3 x4 -z b x1 x2 x3 x4 -z b I =I-II 0 15/4 1-1/4 0 6 II =1/4II 1 1/4 0 1/ III =III-II 0 7/4 0-1/4 0-2 x 2 x 1 + 4x 2 = 8 x 1 x 1 + 2x 2 = 1 4x 1 + x 2 = 8 10

21 continuing x1 x2 x3 x4 -z b I =4/15I 0 1 4/15-1/ /15 II =II-1/4I 1 0-1/15 4/ / III =III-7/4I 0 0-7/15-3/ /5 x 2 = 1 + 3/5 = 1.6 x 1 = 8/5 The optimal solution will not be more than /5 =

22 continuing x1 x2 x3 x4 -z b I =4/15I 0 1 4/15-1/ /15 II =II-1/4I 1 0-1/15 4/ / III =III-7/4I 0 0-7/15-3/ /5 x 2 = 1 + 3/5 = 1.6 x 1 = 8/5 The optimal solution will not be more than /5 = 4.8 Both variables are fractional, we pick one of the two: 4.8 x 1 1 x

23 continuing x1 x2 x3 x4 -z b I =4/15I 0 1 4/15-1/ /15 II =II-1/4I 1 0-1/15 4/ / III =III-7/4I 0 0-7/15-3/ /5 x 2 = 1 + 3/5 = 1.6 x 1 = 8/5 The optimal solution will not be more than /5 = 4.8 Both variables are fractional, we pick one of the two: 4.8 x 1 1 x 1 2 x 2 x 1 = 1 x 1 + 4x 2 = 8 x 1 x 1 + 2x 2 = 1 4x 1 + x 2 = 8 11

24 Let s consider first the left branch: 12

25 Let s consider first the left branch: x1 x2 x3 x4 x5 -z b /15-1/ / /15 4/ / /15-3/ /5 12

26 Let s consider first the left branch: x1 x2 x3 x4 x5 -z b /15-1/ / /15 4/ / /15-3/ /5 x1 x2 x3 x4 x5 b -z I =I-III 0 0 1/15-4/ / /15-1/ / /15 4/ / /15-3/ /5 12

27 Let s consider first the left branch: x1 x2 x3 x4 x5 -z b /15-1/ / /15 4/ / /15-3/ /5 x1 x2 x3 x4 x5 b -z I =I-III 0 0 1/15-4/ / /15-1/ / /15 4/ / /15-3/ /5 x1 x2 x3 x4 x5 b -z I =-15/4I 0 0-1/4 1-15/4 0 9/4 II =II-1/4I /60 0-1/4 0 7/4 III =III+I /60 0-9/4 1-90/20 always a b term negative after branching: b 1 = b 3 b 1 = b 3 b 3 < 0 Dual simplex: min j { c j a ij : a ij < 0} 12

28 Let s branch again 4.8 x 1 1 x x 2 1 x

29 Let s branch again 4.8 x 1 1 x 1 2 x x 2 1 x 2 2 x 1 + 4x 2 = 8 x 1 x 1 + 2x 2 = 1 4x 1 + x 2 = 8 13

30 Let s branch again x x 1 2 x 2 x 2 1 B 4.5 x 2 2 A C x 1 + 4x 2 = 8 x 1 x 1 + 2x 2 = 1 4x 1 + x 2 = 8 We have three open problems. Which one we choose next? Let s take A. 13

31 x1 x2 x3 x4 x5 x6 b -z /4 1-15/4 0 9/ /60 0-1/4 0 7/ /60 0-9/4 1-9/2 14

32 x1 x2 x3 x4 x5 x6 b -z /4 1-15/4 0 9/ /60 0-1/4 0 7/ /60 0-9/4 1-9/2 x1 x2 x3 x4 x5 x6 b -z III+I 0 0 1/4 0-1/ / /4 1-15/4 0 9/ /60 0-1/4 0 7/ /60 0-9/4 1-9/2 continuing we find: x 1 = 0 x 2 = 2 OPT = 4 14

33 The final tree: x x 1 2 x x x 1 =2 x 2 =0 x 1 =1 x 2 = x 1 =0 x 2 =2 The optimal solution is 4. 15

34 Pruning Pruning: 1. by optimality: z k = max{c T x : x S k } 2. by bound z k z Example: by infeasibility S k = 16

35 B&B Components Bounding: 1. LP relaxation 2. Lagrangian relaxation 3. Combinatorial relaxation 4. Duality 17

36 B&B Components Bounding: 1. LP relaxation 2. Lagrangian relaxation 3. Combinatorial relaxation 4. Duality Branching: S 1 = S {x : x j x j } S 2 = S {x : x j x j } thus the current optimum is not feasible either in S 1 or in S 2. 17

37 B&B Components Bounding: 1. LP relaxation 2. Lagrangian relaxation 3. Combinatorial relaxation 4. Duality Branching: S 1 = S {x : x j x j } S 2 = S {x : x j x j } thus the current optimum is not feasible either in S 1 or in S 2. Which variable to choose? Eg: Most fractional variable arg max j C min{f j, 1 f j } 17

38 B&B Components Bounding: 1. LP relaxation 2. Lagrangian relaxation 3. Combinatorial relaxation 4. Duality Branching: S 1 = S {x : x j x j } S 2 = S {x : x j x j } thus the current optimum is not feasible either in S 1 or in S 2. Which variable to choose? Eg: Most fractional variable arg max j C min{f j, 1 f j } Choosing Node for Examination from the list of active (or open): Depth First Search (a good primal sol. is good for pruning + easier to reoptimize by just adding a new constraint) Best Bound First: (eg. largest upper: z s = max k z k or largest lower - to die fast) Mixed strategies 17

39 Reoptimizing: dual simplex Updating the Incumbent: when new best feasible solution is found: z = max{z, 4} Store the active nodes: bounds + optimal basis (remember the revised simplex!) 18

40 Enhancements Preprocessor: constraint/problem/structure specific tightening bounds redundant constraints variable fixing: eg: max{c T x : Ax b, l x u} fix x j = l j if c j < 0 and a ij > 0 for all i fix x j = u j if c j > 0 and a ij < 0 for all i 19

41 Enhancements Preprocessor: constraint/problem/structure specific tightening bounds redundant constraints variable fixing: eg: max{c T x : Ax b, l x u} fix x j = l j if c j < 0 and a ij > 0 for all i fix x j = u j if c j > 0 and a ij < 0 for all i Priorities: establish the next variable to branch 19

42 Enhancements Preprocessor: constraint/problem/structure specific tightening bounds redundant constraints variable fixing: eg: max{c T x : Ax b, l x u} fix x j = l j if c j < 0 and a ij > 0 for all i fix x j = u j if c j > 0 and a ij < 0 for all i Priorities: establish the next variable to branch Special ordered sets SOS (or generalized upper bound GUB) k x j = 1 x j {0, 1} j=1 instead of: S 0 = S {x : x j = 0} and S 1 = S {x : x j = 1} {x : x j = 0} leaves k 1 possibilities {x : x j = 1} leaves only 1 possibility hence tree unbalanced here: S 1 = S {x : x ji = 0, i = 1..r} and S 2 = S {x : x ji = 0, i = r + 1,.., k}, r = min{t : t i=1 x j i 1 2 } 19

43 Cutoff value: a user-defined primal bound to pass to the system. Simplex strategies: simplex is good for reoptimizing but for large models interior points methods may work best. 20

44 Cutoff value: a user-defined primal bound to pass to the system. Simplex strategies: simplex is good for reoptimizing but for large models interior points methods may work best. Strong branching: extra work to decide more accurately on which variable to branch: 1. choose a set C of fractional variables 2. reoptimize for each of them (in case for limited iterations) 3. z j, z j (dual bound of down and up branch) j = arg min j C max{z j, z j } ie, choose variable with largest decrease of dual bound, eg UB for max 20

45 There are four common reasons because integer programs can require a significant amount of solution time: 1. There is lack of node throughput due to troublesome linear programming node solves. 2. There is lack of progress in the best integer solution, i.e., the upper bound. 3. There is lack of progress in the best lower bound. 4. There is insufficient node throughput due to numerical instability in the problem data or excessive memory usage. 21

46 There are four common reasons because integer programs can require a significant amount of solution time: 1. There is lack of node throughput due to troublesome linear programming node solves. 2. There is lack of progress in the best integer solution, i.e., the upper bound. 3. There is lack of progress in the best lower bound. 4. There is insufficient node throughput due to numerical instability in the problem data or excessive memory usage. For 2) or 3) the gap best feasible-dual bound is large: gap = Primal bound Dual bound Primal bound + ɛ

47 heuristics for finding feasible solutions (generally NP-complete problem) find better lower bounds if they are weak: addition of cuts, stronger formulation, branch and cut Branch and cut: a B&B algorithm with cut generation at all nodes of the tree. (instead of reoptimizing, do as much work as possible to tighten) Cut pool: stores all cuts centrally Store for active node: bounds, basis, pointers to constraints in the cut pool that apply at the node 22

48 Relative Optimality Gap In CPLEX: gap = best dual bound best integer best integer In SCIP and MIPLIB standard: gap = pb db inf{ z, z [db, pb]} 100 for a minimization problem (if pb 0 and db 0 then pb db db ) if db = pb = 0 then gap = 0 if no feasible sol found or db 0 pb then the gap is not computed. 23

49 Last standard avoids problem of non decreasing gap if we go through zero % % % Elapsed real time = sec. (tree size = MB, solutions = 2) * % % % % 24

50 Advanced Techniques We did not treat: LP: Dantzig Wolfe decomposition LP: Column generation LP: Delayed column generation IP: Branch and Price LP: Benders decompositions LP: Lagrangian relaxation 25

51 Outline

52 rules Consider S = {x : a 0 x 0 + n j=1 a jx j b, l j x j u j, j = 0..n} Bounds on variables. If a 0 > 0 then: x 0 b a j l j a j u j /a 0 j:a j >0 j:a j >0 j:a j <0 and if a 0 < 0 then x 0 b a j l j a j u j /a 0 j:a j <0 28

53 rules Consider S = {x : a 0 x 0 + n j=1 a jx j b, l j x j u j, j = 0..n} Bounds on variables. If a 0 > 0 then: x 0 b a j l j a j u j /a 0 j:a j >0 j:a j >0 j:a j <0 and if a 0 < 0 then x 0 b a j l j a j u j /a 0 j:a j <0 Redundancy. The constraint n j=0 a jx j b is redundant if a j u j + a j l j b j:a j >0 j:a j <0 28

54 Infeasibility: S = if (swapping lower and upper bounds from previous case) a j l j + a j u j > b j:a j >0 j:a j <0 29

55 Infeasibility: S = if (swapping lower and upper bounds from previous case) a j l j + a j u j > b j:a j >0 j:a j <0 Variable fixing. For a max problem in the form max{c T x : Ax b, l x u} if i = 1..m : a ij 0, c j < 0 then fix x j = l j if i = 1..m : a ij < 0, c j > 0 then fix x j = u j 29

56 Infeasibility: S = if (swapping lower and upper bounds from previous case) a j l j + a j u j > b j:a j >0 j:a j <0 Variable fixing. For a max problem in the form max{c T x : Ax b, l x u} if i = 1..m : a ij 0, c j < 0 then fix x j = l j if i = 1..m : a ij < 0, c j > 0 then fix x j = u j Integer variables: l j x j u j 29

57 Infeasibility: S = if (swapping lower and upper bounds from previous case) a j l j + a j u j > b j:a j >0 j:a j <0 Variable fixing. For a max problem in the form max{c T x : Ax b, l x u} if i = 1..m : a ij 0, c j < 0 then fix x j = l j if i = 1..m : a ij < 0, c j > 0 then fix x j = u j Integer variables: l j x j u j Binary variables. Probing: add a constraint, eg, x 2 = 0 and check what happens 29

58 Example max 2x 1 + x 2 x 3 R1 : 5x 1 2x 2 + 8x 3 15 R2 : 8x 1 + 3x 2 x 3 9 R3 : x 1 + x 2 + x x x 2 1 x

59 Example max 2x 1 + x 2 x 3 R1 : 5x 1 2x 2 + 8x 3 15 R2 : 8x 1 + 3x 2 x 3 9 R3 : x 1 + x 2 + x x x 2 1 x 3 1 {}}{ R1 :5x x 2 8x u 2 l 3 {}}{ 1 = 9 x 1 9/5 8x x 2 5x = 17 x 3 17/8 2x 2 5x 1 + 8x = 7 x 2 7/2, x 2 0 R2 :8x 1 9 3x 2 + x = 7 x 1 7/8 R1 :8x x 2 5x /8 = 101/8 x 3 101/64 R3 : x 1 + x 2 + x 3 9/ /64 < 6 Hence R3 is redundant

60 Example max 2x 1 + x 2 x 3 R1 : 5x 1 2x 2 + 8x 3 15 R2 : 8x 1 + 3x 2 x 3 9 R3 : x 1 + x 2 + x x x 2 1 x 3 1 {}}{ R1 :5x x 2 8x u 2 l 3 {}}{ 1 = 9 x 1 9/5 8x x 2 5x = 17 x 3 17/8 2x 2 5x 1 + 8x = 7 x 2 7/2, x

61 Example max 2x 1 + x 2 x 3 R1 : 5x 1 2x 2 + 8x 3 15 R2 : 8x 1 + 3x 2 x 3 9 R3 : x 1 + x 2 + x x x 2 1 x 3 1 {}}{ R1 :5x x 2 8x u 2 l 3 {}}{ 1 = 9 x 1 9/5 8x x 2 5x = 17 x 3 17/8 2x 2 5x 1 + 8x = 7 x 2 7/2, x 2 0 R2 :8x 1 9 3x 2 + x = 7 x 1 7/8 R1 :8x x 2 5x /8 = 101/8 x 3 101/64 30

62 Example max 2x 1 + x 2 x 3 R1 : 5x 1 2x 2 + 8x 3 15 R2 : 8x 1 + 3x 2 x 3 9 R3 : x 1 + x 2 + x x x 2 1 x 3 1 {}}{ R1 :5x x 2 8x u 2 l 3 {}}{ 1 = 9 x 1 9/5 8x x 2 5x = 17 x 3 17/8 2x 2 5x 1 + 8x = 7 x 2 7/2, x 2 0 R2 :8x 1 9 3x 2 + x = 7 x 1 7/8 R1 :8x x 2 5x /8 = 101/8 x 3 101/64 R3 : x 1 + x 2 + x 3 9/ /64 < 6 Hence R3 is redundant 30

63 Example max 2x 1 + x 2 x 3 R1 : 5x 1 2x 2 + 8x 3 15 R2 : 8x 1 + 3x 2 x 3 9 7/8 x 1 9/5 0 x x 3 101/64 31

64 Example max 2x 1 + x 2 x 3 R1 : 5x 1 2x 2 + 8x 3 15 R2 : 8x 1 + 3x 2 x 3 9 7/8 x 1 9/5 0 x x 3 101/64 Increasing x 2 makes constraints satisfied x 2 = 1 Decreasing x 3 makes constraints satisfied x 3 = 1 31

65 Example max 2x 1 + x 2 x 3 R1 : 5x 1 2x 2 + 8x 3 15 R2 : 8x 1 + 3x 2 x 3 9 7/8 x 1 9/5 0 x x 3 101/64 Increasing x 2 makes constraints satisfied x 2 = 1 Decreasing x 3 makes constraints satisfied x 3 = 1 We are left with: max{2x 1 : 7/8 x 1 9/5} 31

66 for Set Covering/Partitioning 1. if e T i A = 0 then the ith row can never be satisfied [ ] =

67 for Set Covering/Partitioning 1. if e T i A = 0 then the ith row can never be satisfied [ ] = if e T i A = e k then x k = 1 in every feasible solution [ ] =

68 for Set Covering/Partitioning 1. if e T i A = 0 then the ith row can never be satisfied [ ] = if e T i A = e k then x k = 1 in every feasible solution [ ] = In SPP can remove all rows t with a tk = 1 and set x j = 0 (ie, remove cols) for all cols that cover t 32

69 3. if e T t A e T p A then we can remove row t, row p dominates row t (by covering p we cover t) t p 1 1

70 3. if e T t A e T p A then we can remove row t, row p dominates row t (by covering p we cover t) t p 1 1 In SPP we can remove all cols j: a tj = 1, a pj = 0 4. if j S Ae j = Ae k and j S c j c k then we can cover the rows by Ae k more cheaply with S and set x k = 0 (Note, we cannot remove S if j S c j c k )

71 Summary