SOLVING INTEGER LINEAR PROGRAMS. 1. Solving the LP relaxation. 2. How to deal with fractional solutions?

Transcription

1 SOLVING INTEGER LINEAR PROGRAMS 1. Solving the LP relaxation. 2. How to deal with fractional solutions?

2 Integer Linear Program: Example max x 1 2x 2 0.5x 3 0.2x 4 x x 6 s.t. x 1 +2x 2 1 x 1 + x 2 +3x 6 1 x 1 + x 2 + x 6 1 x 3 3x 4 1 x 3 2x 4 5x 5 1 x 4 +3x 5 4x 6 1 x 2 + x 5 + x apple x 1,x 2,,x 6 apple 10 x 1,...,x 6 2 Z

3 Solving ILPs var x1 integer >= 0, <= 10; var x2 integer >= 0, <= 10; var x3 integer >= 0, <= 10; var x4 integer >= 0, <= 10; var x5 integer >= 0, <= 10; var x6 integer >= 0, <= 10; maximize obj: -x1-2* x2-0.5*x3-0.2* x4 - x5 +0.5* x6; c1: x1 + 2 * x2 >= 1; c2: x1 + x2 + 3* x6 >= 1; c3: x1 + x2 + x6 >= 1; c4: x3-3* x4 >= 1; c5: x3-2* x4-5* x5 >= 1; c6: x4 + 3* x5-4 *x6 >= 1; c7: x2 + x5 + x6 >= 1; solve; display x1, x2,x3, x4, x5, x6; end;

4 Solution Original ILP x1.val = 0 x2.val = 1 x3.val = 4 x4.val = 1 x5.val = 0 x6.val = 0 Optimal Value: -4.2 LP Relaxation x1.val = x2.val = x3.val = x4.val = 0 x5.val = x6.val = 0 Optimal Value:

5 Case-1: Both LP and ILP are feasible. Opt. Solution of LP relaxation Opt. Solution of ILP

6 Solving ILPs Solve LP relaxation of the ILP. Case-1: LP relaxation solution satisfies integrality constraint. Case-2: LP relaxation solution does not satisfy the integrality constraint.

7 Dealing with Case-2 Branch and Bound Cutting Plane Method. 1 2

8 BRANCH-AND-BOUND: BASIC IDEA

9 Integer Linear Program: Example max x 1 2x 2 0.5x 3 0.2x 4 x x 6 s.t. x 1 +2x 2 1 x 1 + x 2 +3x 6 1 x 1 + x 2 + x 6 1 x 3 3x 4 1 x 3 2x 4 5x 5 1 x 4 +3x 5 4x 6 1 x 2 + x 5 + x apple x 1,x 2,,x 6 apple 10 x 1,...,x 6 2 Z

10 Step #1: Solve the LP relaxation Solving the LP relaxation. x1.val = x2.val = x3.val = x4.val = 0 x5.val = x6.val = 0 Optimal Value (LP relaxation):

11 Step #2: Choose a branch variable x1.val = x2.val = x3.val = x4.val = 0 x5.val = x6.val = 0 Choose a variable that should be integer Optimal Value (LP relaxation):

12 Step #3: Branching Constraints b1 max x 1 2x 2 0.5x 3 0.2x 4 x x 6 s.t. x 1 +2x 2 1 x 1 + x 2 +3x 6 1 x 1 + x 2 + x 6 1 x 3 3x 4 1 x 3 2x 4 5x 5 1 x 4 +3x 5 4x 6 1 x 2 + x 5 + x apple x 1,x 2,,x 6 apple 10 Original Problem b2 Original Problem Constr. + ( x 3 <= 2 ) Original Problem Constr. + ( x 3 >= 3 )

13 Visualization x 3 <= 2 x 3 >= 3

14 Branch-and-Bound P0 Original Problem (LP Relaxation) x j : s j P1 Orig. Problem + x j applebs j c x k : s k P3 P4 Orig. Problem x j applebs j c x k applebt k c Orig. Problem x j applebs j c x k dt k e Non Integral Solution P2 Orig. Problem + x j ds j e

15 Branch-And-Bound: Operations Branch-And-Bound Tree: Nodes are ILP instances. P0 P Regular Node P1 P2 P3 P4 P5 P6 Leaves P7 P8

16 Branch-And-Bound Tree 7 Regular Node (to be explored) P0 P0 P3 Leaf node P1 P2 P4 Leaf node P3 P4 P5 P6 m P5 Leaf node Leaves P7 P8 Expand Node

17 Expanding a Node 1. Solve the LP relaxation for the node ILP. Regular Node (unexplored) LP relaxation solution 2. Three cases: 1. Infeasible. 2. Integral Solution. 3. Fractional Solution.

18 Case-1: Infeasible LP relaxation. LP relaxation is infeasible. Regular Node (unexplored) Infeasible Leaf Node

19 Case-2: LP relaxation yields integral solution LP relaxation solution is integral: ILP solution = LP solution. Regular Node (unexplored) Integral Leaf Node bestobjective := max (lpoptimum, bestobjective)

20 Case-3: LP relaxation yields a fractional solution LP relaxation yields a fractional solution. Regular Node (unexplored) Regular Node Regular (unexplored) Node (explored) x j applebs j c x j ds j e Regular Node (unexplored) LP relaxation solution: x1: x2: x3: Opt. Solution: optsolution

21 Optimal Pruning Regular Node (explored) optsolution <= bestobjective? LP relaxation solution: x1: x2: x3: Opt. Solution: optsolution

22 Optimal Pruning P0 bestobjective P1 P2 P3 P4 P5 P6 P7 P8 optsolution <= bestobjective

23 Branch-And-Bound Initial Node Original ILP (unexplored node) bestobjective := -Infinity

24 BRANCH-AND-BOUND (EXAMPLE)

25 Original Problem var x1 integer >= 0, <= 10; var x2 integer >= 0, <= 10; var x3 integer >= 0, <= 10; var x4 integer >= 0, <= 10; var x5 integer >= 0, <= 10; var x6 integer >= 0, <= 10; maximize obj: -x1-2* x2-0.5 * x3-0.2* x4 - x5 +0.5* x6; c1: x1 + 2 * x2 >= 1; c2: x1 + x2 + 3* x6 >= 1; c3: x1 + x2 + x6 >= 1; c4: x3-3* x4 >= 1; c5: x3-2* x4-5* x5 >= 1; c6: x4 + 3* x5-4 *x6 >= 1; c7: x2 + x5 + x6 >= 1; solve; display x1, x2,x3, x4, x5, x6; end;

26 Original ILP integer solution x1.val = 0 x2.val = 1 x3.val = 4 x4.val = 1 x5.val = 0 x6.val = 0 Optimal Value: LP relaxation x1.val = x2.val = x3.val = x4.val = 0 x5.val = x6.val = 0 Optimal Value:

27 Initial Node bestobjective := - Infinity x1.val = x2.val = Original Problem (p0.ampl) x3 <= 2 x3 >= 3 p1.ampl p2.ampl x3.val = x4.val = 0 x5.val = x6.val = 0 Optimal Value:

28 Tree #1 bestobjective := - Infinity Infeasible! p1.ampl Original Problem (p0.ampl) x3 <= 2 x3 >= 3 p2.ampl p2.ampl x1.val = 0.4 x2.val = 0.55 x3.val = 3 x4.val = 0 x5.val = 0.4 x6.val = 0.05

29 p3.ampl BnB Tree x1.val = 1 x2.val = 0 x3.val = x4.val = 0 x5.val = x6.val = Optimal Value: p1 p0 p3 p2 p4 p5 p6 p6.ampl p7 p8 p8.ampl x1.val = 1 x2.val = 0 x3.val = 6 x4.val = 0 x5.val = 1 x6.val = 0.5 Optimal Value: x1.val = 1 x2.val = 0 x3.val = 5 x4.val = x5.val = x6.val = Optimal Value:

30 BnB Tree p0 bestobjective = -4.2 p9.ampl x1.val = 0 p4.ampl x1.val = 0 x2.val x2.val = 1 = 1 x3.val = 3 = 4 x4.val = = 1 x5.val = x5.val = 0 x6.val = 0 Optimal x6.val Value: = Optimal Value: -4.2 p1 p3 p5 p8.ampl x1.val = 1 x2.val = 0 p7 x3.val = 6 x4.val = 0 x5.val = 1 x6.val = 0.5 Optimal Value: p2 p6 p8 p9 p4 p10

31 BnB Tree p0 bestobjective = -4.2 p1 p2 p10.ampl p12.ampl x1.val = 0 p3 x2.val = 1 x3.val = 36 p4 x4.val = 0 p5 p6 p9 p10 x5.val = x6.val = 0.5 Optimal Value: p7 p8 p11 p12

32 BnB Final Tree p0 bestobjective = -4.2 x1.val = 0 x2.val = 1 x3.val = 4 x4.val = 1 x5.val = 0 x6.val = 0 Optimal Value: p1 p5 p3 p7 p2 p6 p4 p9 p10 p8 p11 p12

33 BRANCH-AND-BOUND METHOD Heuristics

34 BnB choices to be made Which unexplored regular leaf node should I expand? How to choose the branching variables? Other tricks: Randomized rounding to find an integer solution? Carrying out BnB at the dictionary level.

35 BRANCH-AND-BOUND Choice of unexplored node to expand

36 General Situation Which node should I examine first? Unexplored frontier

37 Branch-and-bound p0 Q: Which one should we examine first? p1 p5 p3 p2 p6 p4 Unexplored Nodes p7 p8

38 Goal Minimize the number of nodes explored in the tree. Which node should I expand first: Yield integral solutions. Improve the value of bestobjective.

39 Unexplored Node Selection Heuristic Deepest node first. Select the node that has largest depth in the tree to explore first. Best LP relaxation solution. Select the node whose LP relaxation has the best answer. Breadth First Search breadth-first.

40 In Practice ILPs come from some problem domain: eg., We are converting a minimum cost vertex cover computation to ILP. Selection heuristics are best found by experimenting with a large set of problems from the domain. There is no general rule, unfortunately.

41 BRANCH-AND-BOUND Choosing the branch variable

42 p3.ampl BnB Tree x1.val = 1 x2.val = 0 x3.val = x4.val = 0 x5.val = x6.val = Optimal Value: p1 p0 p3 p2 p4 p5 p6 p6.ampl p7 p8 p8.ampl x1.val = 1 x2.val = 0 x3.val = 6 x4.val = 0 x5.val = 1 x6.val = 0.5 Optimal Value: x1.val = 1 x2.val = 0 x3.val = 5 x4.val = x5.val = x6.val = Optimal Value:

43 Original Problem var x1 integer >= 0, <= 10; var x2 integer >= 0, <= 10; var x3 integer >= 0, <= 10; var x4 integer >= 0, <= 10; var x5 integer >= 0, <= 10; var x6 integer >= 0, <= 10; maximize obj: -x1-2* x2-0.5 * x3-0.2* x4 - x5 +0.5* x6; c1: x1 + 2 * x2 >= 1; c2: x1 + x2 + 3* x6 >= 1; c3: x1 + x2 + x6 >= 1; c4: x3-3* x4 >= 1; c5: x3-2* x4-5* x5 >= 1; c6: x4 + 3* x5-4 *x6 >= 1; c7: x2 + x5 + x6 >= 1; solve; display x1, x2,x3, x4, x5, x6; end;

44 BnB Final Tree p0 bestobjective = -4.2 x1.val = 0 x2.val = 1 x3.val = 4 x4.val = 1 x5.val = 0 x6.val = 0 Optimal Value: p1 p5 p3 p7 p2 p6 p4 p9 p10 p8 p11 p12

45 Initial Node bestobjective := - Infinity x1.val = x2.val = Original Problem (p0.ampl) x3 <= 2 x3 >= 3 p1.ampl p2.ampl x3.val = x4.val = 0 x5.val = x6.val = 0 Optimal Value:

46 What if we chose a different branch variable..? bestobjective := - Infinity Original Problem (p0.ampl) x5 <= 0 x5 >= 1 q1.ampl (integer feasible) q2.ampl x1.val = x1.val = 0 x2.val x2.val = = x3.val = = x4.val = = 00 x5.val = = x6.val = = 00.5 Optimal Value: bestobjective := -4.2

47 How to select a branch variable? There are heuristics The But choice it is a hard of a problem, branch variable in general. is very important. User specifies a variable ordering or priority. There is no single best heuristic. Solver heuristics: Often, Choose some fractional expertise solution with closest the to problem an integer domain value? informs the heuristic. Choose binary (0-1) variables preferentially? Random choice?

48 BRANCH-AND-BOUND At the dictionary level

49 Thus far Use LP solver as a black box. This lecture: Consider how to fold branch-and-bound in a dictionary setup. Final dictionary BnB Node (Solve LP relaxation) Can I reuse the final dictionary? Child # 1 Child # 2

50 View from the dictionary x B b z c 0 +c N x N Final dictionary BnB Node (Solve LP relaxation) x j applebs j c Claim: Dictionary D is always infeasible. Child # 1 x B b w z c 0 +c N x N

51 Proof (Pictorial) Solution represented by dictionary D and D x 3 <= 2 x 3 >= 3

52 Example (ILP) var x1 >= 0, <= 10; var x2 >= 0, <= 10; var x3 >= 0, <= 10; maximize obj: x1 + x2-5* x3 ; c1: -2* x1 + 7 *x2 <= 1; c2: x1-2 *x2 + 5* x3 <= 3; c3: x1 + x2-3 * x3 <= 7; solve; display x1, x2, x3; end; x 4 : 10 x 1 x 5 : 10 x 2 x 6 : 10 x 3 x 7 : 1+2x 1 7x 2 x 8 : 3 x 1 +2x 2 5x 3 x 9 : 7 x 1 x 2 +3x 3 Slack variables are also integers.

53 Final Dictionary x x x x 3 x x x x 3 x 6 10 x 3 x 7 3 3x 8 +x 9 18x 3 x x x x 3 x x x x 3 z x 8 x 9 2x 3 x 1 6 x 10 : 6 + x 1

54 Dictionary After Branch x 10 : 6 + x 1 x x x x 3 x x x x 3 x 6 10 x 3 x 7 3 3x 8 +x 9 18x 3 x x x x 3 x x x x 3 x x x x 3 z x 8 x 9 2x 3 Dictionary becomes primal infeasible. Back to initialization phase Simplex?

55 Dictionary After Branch x x x x 3 x x x x 3 x 6 10 x 3 x 7 3 3x 8 +x 9 18x 3 x x x x 3 x x x x 3 x x x x 3 z x 8 x 9 2x 3 Dictionary becomes primal infeasible. Dual Feasible Dictionary!! But not dual final.

56 How to update after branch? add branch constraint Parent Node (Final Dictionary) Child Node: 1. Add new row. 2. Primal Infeasible but Dual Feasible Consider dual complement dictionary. (Feasible + but non-final) Final dual dictionary (also final primal) Opt. Phase on dual dictionaries LP relaxation solved for child node!

57 General Form Simplex add branch constraint Parent Node (Final Dictionary) Child Node: 1. Add new row. 2. Primal Infeasible but Dual Feasible Consider dual complement dictionary. (Feasible + but non-final) Final dual dictionary (also final primal) Opt. Phase on dual dictionaries LP relaxation solved for child node!

58 General Form Dictionary Give special treatment to bounds on variables. l apple x apple u Modify the Simplex algorithm in two ways: Dictionaries now have special ways to track bounds. Pivoting is modified.

59 Branch-and-Bound with General Form Parent Node (Final Dictionary) Simply update Child Node: variable bound. 1. Add new row. 2. Primal Infeasible but Dual Feasible

60 CUTTING PLANE METHOD

61 LP Relaxation for ILP Opt. Solution of LP relaxation Opt. Solution of ILP

62 Removing a Fractional Solution Branch and Bound Cutting Plane Method. 1 2

63 Cutting Plane How do we find a cutting plane?

64 GOMORY-CHVATAL CUTTING PLANE Part 1: Setting up the problem.

65 ILP in Standard Form max c x s.t. Ax apple b x 0 x 2 Z A, b, c are all assumed to be integers.

66 Conversion to standard from Recap from LP formulations lectures: Equality constraints into two inequalities. Rewrite in case x i 0 is missing: x i 7! x + i x i Convert to by negating both sides. How do we make sure A, b, c are integers? Reasonable assumption: original problem coefficients are rationals.

67 Integer Coefficients. Scale constraints + Objective max 2x x 2 0.1x 3 s.t. 0.1x 1 2x 2 x 3 apple x 1 2.6x x 3 apple 0.15 x 1, x 2, x 3 0 x 1, x 2, x 43 2 Z max 2x x 2 0.1x 3 10 s.t. 0.1x 1 2x 2 x 3 apple x 1 2.6x x 3 apple x 1, x 2, x 3 0 x 1, x 2, x 4 2 Z

68 Conversion to Integer Coefficients. max 2x x 2 0.1x 3 s.t. 0.1x 1 2x 2 x 3 apple x 1 2.6x x 3 apple 0.15 x 1, x 2, x 3 0 x 1, x 2, x 4 2 Z Divide result by 10 max 20x 1 +3x 2 x 3 s.t. 2x 1 40x 2 20x 3 apple 5 50x 1 260x x 3 apple 15 x 1, x 2, x 3 0 x 1, x 2, x 4 2 Z

69 Adding Slack Variables max c x s.t. Ax apple b x 0 x 2 Z max c x s.t. Ax + x s = b x, x s 0 x, x s 2 Z

70 Summary We assume problem is in LP standard form. Assume coefficients are rational. ILP standard form: coefficients (A,b,c) are integers. Scale the original rational problem. Make sure that result is divided by scale factor for objective. Advantage of ILP standard form: Slack variables are naturally integers. No need for separate treatment of decision and slack variables.

71 CUTTING PLANE METHOD Part 2: Gomory-Chvatal Cuts

72 Overall method Solve LP relaxation of the problem. Is the solution Integral? Yes Done. Guarantee: No integer point is lost. Add a new cutting plane constraint. No

73 Cutting Plane Visualization First optimum Second Optimum 2 1 Cutting Plane Third 2 Optimum Idea: cutting plane attempts to successively expose an integer optimal vertex.

74 Cutting Plane Method Deriving the Cutting Plane Gomory-Chvatal Cuts Integrating the method inside Simplex. Using the dual dictionary to avoid initialization phase Simplex.

75 GOMORY-CHVATAL CUTS

76 Overall Idea max c x s.t. Ax + x s = b x, x s 0 x, x s 2 Z 1. Solve the LP relaxation using Simplex algorithm. LP Relax Initialization Phase Optimization Phase Final Dictionary

77 Final Dictionary Optimal solution to ILP found!! x B b + Ãx N z z 0 + c N x N fractional entries Derive a cutting plane

78 Example: Final Dictionary x x x 3 0.5x 5 x 4 1 x 2 +x 3 x 5 x x 2 2.1x 3 +x 5 z x 2 2.3x 3 2.1x 5 x 1 =1.2, x 2 =0, x 3 =0, x 4 =1, x 5 =0, x 6 =2.5

79 Cutting Plane Derivation: Step 1 Step 1 Identify row with fractional constant. x x x 3 0.5x 5 x 4 1 x 2 +x 3 x 5 x x 2 2.1x 3 +x 5 z x 2 2.3x 3 2.1x 5 x 1 = x x 3 0.5x x 2 4.3x x 5 =1.2 5

80 Cutting Plane Derivation: Step 2 x x 2 4.3x x 5 =1.2 (x 1 +3x 2 5x 3 +0x 5 ) {z } A +(0.1x x x 5 ) {z } B A: integer B: integer + fraction B 0 A + B = 1.2 =1+0.2

81 Claim (x 1 +3x 2 5x 3 +0x 5 ) Conclusion: {z } A +(0.1x x x 5 ) {z } B =1+0.2 A is 1. integer Fractional part of B is B 0.2 B has a fractional part. B 0 In other words, A + B = B 0.2 is an integer. 2. B A B A+B Possible YES No No No Yes Yes

82 Cutting Plane x x x 3 0.5x 5 x 4 1 x 2 +x 3 x 5 x x 2 2.1x 3 +x 5 z x 2 2.3x 3 2.1x 5 Cutting Plane: 0.1x x x 5 0.2

83 Cutting Plane: Definition. Final Dictionary (LP Relax) x B1 = b 1 +a 11 x I1 + +a 1j x Ij + +a 1n x In.. x Bk = b k +a k1 x I1 + +a kj x Ij + +a kn x In. x Bm = b m +a m1 x I1 + +a mj x Ij + +a mn x In z = c 0 +c 1 x I1 + +c j x Ij + +c n x In b k 62 Z Cutting Plane frac( a k1 )x I1 + + frac( a kn )x In frac(b k ) frac(x) =x bxc

84 b k 62 Z Cutting Plane x B1 = b 1 +a 11 x I1 + +a 1j x Ij + +a 1n x In.. x Bk = b k +a k1 x I1 + +a kj x Ij + +a kn x In. x Bm = b m +a m1 x I1 + +a mj x Ij + +a mn x In z = c 0 +c 1 x I1 + +c j x Ij + +c n x In frac( a k1 )x I1 + + frac( a kn )x In frac(b k ) Claim: Every (integer) feasible point of the ILP satisfies the cutting plane constraint.

85 We have nx x Bk + ( a kj )x Ij = b k (1) j=1 Since a kj b a kj c and x Ij 0, we obtain nx X n ( a kj )x Ij b a kj cx Ij j=1 j=1 x B1 = b 1 +a 11 x I1 + +a 1j x Ij + +a 1n x In.. x Bk = b k +a k1 x I1 + +a kj x Ij + +a kn x In. x Bm = b m +a m1 x I1 + +a mj x Ij + +a mn x In z = c 0 +c 1 x I1 + +c j x Ij + +c n x In b k 62 Z Applying this to Eq.(1) we obtain: Therefore, we get x Bk + nx ( a kj )x Ij x Bk + j=1 b k x Bk + nx b a kj cx Ij (2) j=1 nx b a kj cx Ij (3) j=1 The RHS of (3) is an integer, therefore, we conclude: bb k c x Bk + Combining (4) with (1), we have, b k nx b a kj cx Ij (4) j=1 bb k capple P n j=1 ( a kj b a kj c)x Ij In other words, P n j=1 frac( a kj)x Ij frac(b k )

86 Example x x x 3 0.5x 5 x 4 1 x 2 +x 3 x 5 x x 2 2.1x 3 +x 5 z x 2 2.3x 3 2.1x 5 0.7x x 3 +0x x x x 5 0.7

87 CUTTING PLANE METHOD Updating the dictionary Setup for subsequent iterations.

88 Overall method Solve LP relaxation of the problem. Is the solution Integral? Yes Done. Guarantee: No integer point is lost. Add a new cutting plane constraint. No

89 Cutting Plane Example x x x 3 0.5x 5 x 4 1 x 2 +x 3 x 5 x x 2 2.1x 3 +x 5 z x 2 2.3x 3 2.1x 5 Cutting Plane: 0.1x x x 5 0.2

90 Adding cutting plane to dictionary x x x 3 0.5x 5 x 4 1 x 2 +x 3 x 5 x x 2 2.1x 3 +x 5 w x x x 5 z x 2 2.3x 3 2.1x 5 x x x 3 0.5x 5 x 4 1 x 2 +x 3 x 5 x x 2 2.1x 3 +x 5 z x 2 2.3x 3 2.1x 5 0.1x x x 5 0.2

91 Cutting Plane Method Claim: Resulting dictionary after adding cutting plane is primal infeasible. Cutting Plane Dictionary Solution

92 Cutting Plane: Example x x x 3 0.5x 5 x 4 1 x 2 +x 3 x 5 x x 2 2.1x 3 +x 5 w x x x 5 z x 2 2.3x 3 2.1x 5 Dual dictionary is feasible but non-final. Naïve Approach: Re-solve initialization phase Simplex.

93 Cutting Plane: Solving again after cut. Also final for primal Final Dictionary LP relaxation. Cut Primal Infeasible Dual Feasible Final dictionary for Dual Optimize the dual problem dictionary.