Interior-Point versus Simplex methods for Integer Programming Branch-and-Bound

Transcription

1 Interior-Point versus Simplex methods for Integer Programming Branch-and-Bound Samir Elhedhli Department of Management Sciences, University of Waterloo, Canada Page of 4 McMaster University, March 22, 24

2 . Motivation Interior-point methods are quite useful for solving largescale LPs, but they are not widely employed in branchand-bound algorithms because it is cumbersome to reoptimize an LP with an interior-point code after rows or columns have been added. E.L. Johnson, G. Nemhauser and M.W.P. Savelsbergh, Progress in linear programmingbased algorithms for integer programming: An exposition, Informs JOC,2:, 2. the simplex method has a significant advantage over interiorpoint methods within a branch and bound algorithm. In the branch-and-bound algorithm we can warm start the solution of each new subproblem with the optimal solution from the parent subproblem. Using the simplex method, a new optimal solution can easily be found after a few simplex iterations. With interior-point methods, warm starts are not as effective. J. E. Mitchell and B. Borchers Using an interior point method in a branch-and-bound Page 2 of 4 algorithm for integer programming, 99, revised 992, Rensselaer Polytechnic Institute, Troy, N.Y.

3 2. Branch-and-bound: LP bound Simplex Optimal Primal Optima Dual Basic NB branching constraints (Primal) Page 3 of 4 First Primal First Dual Basic N B Dual simplex

4 Replacing the simplex by an IPM is not going to work. (you may sacrifice warm starting). crossover is a prtial solution (Cplex) How should we apply IPMs? Save an approximate solution before termination (Gondzio et al Math. Prog.) Cutting plane/column generation context (Elhedhli and Goffin. 23. Math. Prog.) Page 4 of 4

5 3. A cutting plane context: Lagrangean Relaxation [MIP ] : min c T x s.t. Ax b (complicating) λ Bx d (easy) x x j integer for j J Lagrangean subproblem: [MIP λ ] : b T λ + min (c Ax) T λ s.t. Bx d x x j integer for j J Dual Lagrangean problem The dual master problem { max b T λ + v(ip λ ) } λ Page 5 of 4 max b T λ + λ s.t. (c Ax i ) T λ λ i I

6 max b T λ + λ s.t. λ + (Ax i ) T λ c T x i i I Where x i are the integer feasible points to the bounded set: {x : Bx d, x j integer for j J, x } The Dantzig-Wolfe master problem min s.t. i I (c T x i )α i α i (Ax i ) b i I α i = i I i I α i, i I. Page 6 of 4

7 4. Facts: The LR (DW) bound is at least as good as the LP bound. The subproblems are usually decomposable: solved much faster & allow the addition of multiple cuts (columns). Columns can be used as a basis for generating good feasible solutions (better incumbents). Exponential number of cuts (columns): need a cutting plane (column generation) approach May suffer from tailing effects. selected with care. Cuts (columns) should be Page 7 of 4

8 5. Using the LR bound within branch and bound: branchand-price Columns satisfying left branching Optimal Bound : zl Analytic center : a a a ( x, y, s ) Columns satisfying right branching Page 8 of 4 zl is valid Analytic center : a a ( y, s ) is dual feasible Dual IPM

9 6. Issues Careful branching: you may make the subproblerms more difficult to solve (considerably distort their structure) Early termination of the column gneeration/cutting plane scheme Warm starting Primal and dual heuristics Page 9 of 4

10 7. Example: The (binary) cutting stock problem Given: Rolls of length V and demand d l for items of lengths D l l =,..., L Find: the minimum number of rolls necessary to meet the demand D Page of 4 V D l If d l is either zero or one, we get the bin packing problem

11 8. MIP model for the bin packing problem { zk = when bin k is used Binary variables : y kl = when item l is assigned to bin k [BP P ] min s.t. K z k k= K y kl = l =,.., L Single sourcing constraints k= L D l y kl V z k k =,..., K Capacity constraints z k {, } k =,..., K; y kl {, } k =,..., K; l =,..., L; Page of 4 Question: Which decomposition to use? Answer: Depends on the quality of the bound.

12 9. The LP bound: LP The dual: [LP ] min s.t. K z k k= Dual variable K y kl = l ν l k= L D l y kl V z k k µ k z k k ρ k y kl k, l max s.t. L ν l K ρ k k= ν l D l µ k k, l V µ k ρ k = k µ k, ρ k k Solution: ρ k = ; ν l = D l V ; µ k = V ; L LP bound: D l V. Page 2 of 4

13 . Lagrangean relaxation of capacity constraints: LR2 min s.t. K z k k= K y kl = k= l L D l y kl V z k k :µ k z k {, } k y kl {, } k l min s.t. K k= L µ k D l y kl + K ( µ k V )z k K y kl = k= k= z k {, } k y kl {, } k l Optimal: µ k = V K L (Geoffrion): LR bound: V D ly kl = L k= V D l l K y kl = L k= D l V = LP Page 3 of 4

14 . Lagrangean relaxation of the single sourcing constraints: LR min s.t. K z k k= K y kl = l :λ l k= L D l y kl V z k k z k {, } k y kl {, } k l = The dual Lagrangean problem is : max λ Master problem: max Kθ + L s.t. λ l [KP λ ] min y,z ( λ l )y l + z l s.t. D l y l V z l y l, z =, l K identical - knapsacks λ l + Kv(KP λ ) L ( λ l )yl h + z h θ h =,..., H H is the index set of the integer solution to the sets l Page 4 of 4

15 { (y l, z) : L } D l y l V z; y l, z =, ; l =,.., L Dual: the Dantzig-Wolfe master problem Bound: min s.t. H h= ( z h ) α h H α h = K h= H yl hα h = l =,.., L h= α h h =,..., H. v(lp ) = v(lr 2 ) v(lr ) Page 5 of 4

16 2. Lagrangean decomposition: LD min z k k s.t. x kl = k l Two sets of subproblems [SP k y ] min s.t. L x kl = y kl k l :π kl D l y kl V k k l y kl, z k =, k, l π kl y kl + z k L D l y kl V z k y kl, z k =, l The dual Lagrangean problem: K max v(spy k ) + λ k= \qquad L v(spx) l [SP l x] min s.t. K k= π kl x kl K x kl = k= x kl =, k Page 6 of 4

17 The full master problem max K k= s.t. θ k + l θ k + L θ l (π kl )y h kl zh k h =,..., H k y. θ l k π kl x h kl h =,..., Hl x. where H k y and H l x are as defined previously. Easy structure of [SPx] l = ( K L ) max π kl ykl h + zk h + π = max π k= K k= min h H k y min h H k y ( L ) π kl ykl h + zk h + Which yields a smaller master problem max K k= s.t. θ k + l θ k + L θ l L L ( ) min π kl x h h Hx l kl ( ) min π kl k=,..,k (π kl )y h kl zh k h =,..., H k y ; k =,..., K Page 7 of 4 θ l π kl k =,.., K; l =,.., L

18 3. Lagrangean decomposition: LD2 Instead of copying variable y, let us copy D l y kl, leading to [BP P ] min z k k s.t. y kl = k l x kl = D l y kl k; l :ω kl x kl V z k k l y kl, z k =, k; l x kl D l k; l Two sets of subproblems Page 8 of 4 [SPy k ] min s.t. K D l y kl k= y kl = k y kl =, l \qquad [SP l x] min s.t. L ω kl x kl + z k x kl V z k l z k =, k x kl D l k

19 The dual Lagrangean problem is max λ K v(spy k ) + k= The full master problem max K k= s.t. θ k + l θ l k θ k + L Ranking of the bounds θ l L v(spx) l (ω kl )y h kl zh k h =,..., H k y. ω kl x h kl h =,..., Hl x. v(lr2) = v(lp ) v(ld2) v(lr) = V (LD) Page 9 of 4

20 4. Using v(lr) in an interior-point branch-and-price framework Master problem max Kθ + L s.t. λ l L ( λ l )yl h + z h θ h =,..., H H { is the index set of the integer points: } L (y l, z) : D l y l V z; y l, z =, ; l =,.., L Dual: the Dantzig-Wolfe master problem min s.t. H h= ( z h ) α h H α h K h= H yl hα h = l =,.., L h= α h h =,..., H. Page 2 of 4

21 As H α h h= = K is redundant, ( we assume that we have a sufficient number of available bins), We get min s.t. H h= ( z h ) α h H yl hα h = l =,.., L h= α h h =,..., H. The Gilmore-Gomory formulation for the cutting stock problem Page 2 of 4

22 5. Cutting plane methods Relaxed master problem: max Kθ + L s.t. λ l L ( λ l )yl h + z h θ h H H Solution: Cutting Plane Method Get a query point λ from the relaxed master problem Check for optimality at the full master problem. If not optimal, add a new set of constraints Solve the K subproblems [SP λ ] m Get lower bound (z l ) : λ l + Kv(RMP ) Form a new relaxed master problem Page 22 of 4

23 6. ACCPM The choice of the query point gives the different variants of the cutting plane algorithms. Kelley s cutting plane method (Kelly, 96): the query point is the optimal solution of [RMP ]. Analytic Centre Cutting Plane Method (ACCPM): (Goffin, Haurie & Vial, 992) the query point is the analytic centre of the localization set: (θ, λ) : Kθ + m λ l z l ; F D (z l ) = θ + m yl hλ l z h ; h H, Page 23 of 4

24 7. ACCPM Kelley s (usually Simplex methods) ACCPM (interior point methods) Cut Localization Set ac New ac Lower bound New LB Page 24 of 4 Extreme point

25 8. Computing the Analytic Centre The analytic centre is the point that maximizes the distance from the boundaries of the localization set F q D (z l). is the unique point maximizing The weighted dual potential. The weighted primal potential The weighted primal dual potential satisfies the first order conditions Page 25 of 4 S x = Ne ν () Ã x =, x > (2) Ã T y + s = c, s > (3)

26 9. Waram starting Computing the analytic centre after adding cuts Primal IPM. Primal-dual IPM. Computing the analytic cemnte after branching Dual IPM. Page 26 of 4

27 2. The IP-B&P algorithm Columns satisfying left branching Optimal Bound : zl Analytic center : a a a ( x, y, s ) Columns satisfying right branching Page 27 of 4 zl is valid Analytic center : a a ( y, s ) is dual feasible Dual IPM

28 Initilaization: Intial upper bound: UB =. Initial set of nodes to explore: S = {}. Initial lower bound for node : LB =. Initial matrix for node : A is empty. Iteration: While there are nodes to explore (S is not empty) End while. Pick a node, say node n 2. Get a lower bound LB n from the full master problem. Use the matrix A n as a starting matrix. Use the lower bound LBn as an intial lower bound Get a lower bound LB n by applying a cutting plane/column generation method (ACCPM, Kelley s method or Bundle methods ). 3. If possible, generate a feasible solution. This gives an upper bound UB n 4. Update upper bound: UB = min (UB n, UB) 5. If LB n UB 6. Else Fathom node n: S = S\{n}. Branch: Create two new nodes n and n 2 : S = S {n, n 2 } 7. Save warm starting information: Use the branching rule to split the columns of A n into two matrices A n and A n 2. Use them as initial matrices for the child nodes n and n 2. The lower bound LB n is valid for the child nodes. Use it to initialize the lower bounds at child nodes n and n 2. LBn = LB n and LBn 2 = LB n. Page 28 of 4 Figure : The main steps of the branch-and-price algorithm

29 2. Branching rule Pick two items l and l 2 that are once put in the same bin k and once put in diffeent bins (search for an other bin k 2 that has one but not the other.) This corresponds to identifying the pattern x k l = x k l 2 = x k2 l = ; x k2 l 2 = Branching constraints x kl = x kl2 x kl + x kl2 Page 29 of 4

30 Page 3 of 4 l h

31 22. Generating feasible solutions Dual heuristics: Rounding α kh Rounding the original variables y kl = h y h kl α kh min δ s.t. n k= h H k Primal heuristics ( m ) c kl ykl h δ kh δ kh = h H k n ykl h δ kh = k= h H k δ kh Use the set of generated columns. k =,..., n l =,..., m h H k, k =,..., n Page 3 of 4

32 23. Numerical Testing Matlab 6. Cplex 7.5 Problems Triplets Random On a Sun Ultra-/44 workstation. Page 32 of 4

33 24. ACCPM: x 5 The progress of the bounds Bound upper bound lower bound best lower bound Newton iters to compute the Analytic centre Newton iters Newton iters to find a primal feasible point Newton iters 5 Page 33 of Phase I Phase II Figure 2: The progress of the lower and upper bounds in ACCPM

34 25. Comparsion between AC-BP, K-BP and Cplex-MIP 7.5 LP Lag. AC- BP Cplex-MIP 7.5 K-BP bound bound Nodes CPU b Nodes Gap CPU b Nodes CPU b BinG BinG BinG %() a BinG BinG %() BinG BinG BinG %() BinG %() BinG BinG BinG %(7) BinG %(6) BinG BinG Min Max Average (.) a : Difference between lower and upper bound. (.) b : All CPU s in minutes. : Failed to find a feasible solution within the allowed time. # : Did not get past node. (.) d : found lower bound as a percentage of best lower bound. : Gap =. Page 34 of 4

35 26. Comparsion between AC-BP, K-BP and Cplex-MIP 7.5 LP Lag. AC- BP Cplex-MIP 7.5 K-BP bound bound Nodes CPU b Nodes Gap CPU b Nodes CPU b BinT BinT BinT %() a > 2.89 BinT %() > BinT %() > BinT %() > BinT %() > BinT %() > BinT %() > BinT %() > 2 #(99.72%) d > 2 BinT %() > 2 #(99.9%) > 2 BinT %() > 2 #(99.5%) > 2 BinT > 2 #(99.52%) > 2 BinT > 2 #(82.75%) > 2 Min.2 c 2.2 c.6 c Max c 276 c c Average c c c (.) a : Difference between lower and upper bound. (.) b : All CPU s in minutes. (.) c : Taken over the first 9 instances (those solved successfully by AC-BP and K-BP). : Failed to find a feasible solution within the allowed time. # : Did not get past node. (.) d : found lower bound as a percentage of best lower bound. : Gap =. Page 35 of 4

36 27. Comparsion between AC-BP and K-BP: Warm starting Problem A-BP K-BP SP SPrest CPU CPUrest SP SPrest CPU CPUrest BinG BinG BinG BinG BinG BinG BinG BinG BinG BinG BinG BinG BinG BinG BinG Min Max Average # : Did not get past node. a : SPrest as a percent of SP b : CPUrest as a percent of CPU : No branching was done. Page 36 of 4

37 28. Comparsion between AC-BP and K-BP: Warm starting Problem A-BP K-BP SP SPrest CPU CPUrest SP SPrest CPU CPUrest BinT BinT BinT BinT BinT BinT BinT BinT BinT BinT # BinT # BinT # BinT # BinT # Min Max Average # : Did not get past node. a : SPrest as a percent of SP b : CPUrest as a percent of CPU : No branching was done. Page 37 of 4

38 29. IP B&P vs Classical B&P: Nodes solved and CPU 35 CPU (mins) Classical B&B (Kelley) IP B&B (ACCPM) Problem number Number of nodes Classical B&B (Kelley) IP B&B (ACCPM) Page 38 of Problem number

39 3. IP B&P vs Classical B&P: Subproblems called. Subporoblems called at node IP B&P (ACCPM) Classical B&P (Kelley) Problem number Subporoblems called at rest of nodes IP B&P (ACCPM) Classical B&P (Kelley) Problem number Page 39 of 4

40 3. Conclusion Simpelx: LP bounding scheme IPMs: Lagrangean bounding scheme: Dantzig-Wolfe formulation Decomp. approaches Branch & Price ACCPM IP B&P IP B&P Branch & Bound IP B&B Page 4 of 4 Interior point methods

41 Warm starting can be done efficciently with IPMs. IPM-based Branch-and-Bound is promising to solve large MIPs Still to do: An efficient implementation (framework??) Branching Dual heuristics Preprocessing, valid cuts, etc Testing... Page 4 of 4