Column Generation I. Teo Chung-Piaw (NUS)

Transcription

1 Column Generation I Teo Chung-Piaw (NUS) 21 st February 2002

2 1 Outline Cutting Stock Problem Slide 1 Classical Integer Programming Formulation Set Covering Formulation Column Generation Approach Connection with Lagrangian Relaxation 2 Cutting Stock Problem A paper company has a supply of large rolls of paper,each of width W. Slide 2 Slide 3 Customers demand n i rolls of width w i (i=1,...,m). (w i W ) Example: Quantity Ordered Order Width (inches) The demand can be met by slicing a large roll in a certain way,called a pattern. Slide 4 For example,a large roll of width 100 can be cut into 4 rolls of width 25,or 2 rolls of width 35,with a waste of 30. 1

3 2.1 Complexity Why is the cutting stock problem difficult? Slide 5 Given integers a 1,a 2,...,a m Is it possible to partition the numbers into two sets S and T such that a i = a i? i S i T Check: m i=1 a i even? The above problem (PARTITION) is NP-complete. We can turn (PARTITION)into a cutting stock problem! W = m i=1 a i/2. m i=1 a i even. n i =1foralli =1,...,m. Slide 6 w i = a i, i =1,...,m. By solving the cutting stock problem,we know the answer to (PARTITION) If the cutting stock problem has a minimum solution of 2,then the (PAR- TITION) problem is feasible. If the minimum solution to the cutting stock problem is more than 2,then the (PARTITION) problem is NOT feasible. 3 IP Formulation 3.1 Kantorovich K: Set of available rolls. Slide 7 y k : 1 if roll k is cut,0 otherwise. 2

4 x k i : number of times item i is cut on roll k. Objective: To minimize the number of rolls used to meet all the demand min k K y k Constraints Slide 8 Total number of times item i is cut is not less than the demand. x k i n i k K The width of a roll is at most W w i x k i Wyk i Slide 9 min k K yk k K xk i n i for a fixed item i, i w ix k i Wyk for fixed roll k, x k i 0, 0 yk 1 Integrality constraints on all variabes How good is the IP code? N1 Slide 10 Demand of item i : uniform,between 1 and 100 (rand(100)+1); Width of item i : uniform,between 1 and 30 (rand(30)+1); Width of Roll, W = 3000; Rolls Items constr variables CPU (s) Note 1 Code 3

5 Slide 11 Demand of item i : uniform,between 1 and 100 (rand(100)+1); Width of item i : uniform,between 1 and 30 (rand(30)+1); Width of Roll = 150; Rolls Items constr variables CPU (s) never never Out of memory The performance has deteriorated. Why? Quality of formulation How good is the LP relaxation? i Observation: Z LP = wini W. N2 This bound is trivial! In the optimal solution, w i x k i = Wy k i x k i = n i k Slide 12 Note 2 Proof 4

6 We have the constraints w i x k i Wy k. i In the LP,since the objective is to minimize k yk,the optimal LP solution will be such that i w ix k i W = yk. y k will be small if the x k i to make x k i values are small. At the same time,because x k i n i, k K small,at the optimal solution,we must have x k i = n i. k K So the objective function,for the optimal LP solution,reduces to y k = i w ix k i W k K k K = w i W xk i i k K = w i x k i W i k K = w i n i W i Another Example: W = 273 Quantity Ordered Order Width (inches) Slide 13 LP: solved in 0.27s. Solution IP: halted after 12 hours of computational time! 4 Approach II: Set Covering Observation: many of the rolls are cut using the same pattern Slide 14 Use number of times each pattern is used as the decision variable 5

7 Example: An instance: W = 100, m =3. pattern w i n i Another way to formulate the cutting stock problem: Slide 15 minimize 6 j=1 x j x 1 x 2 x 3 x 4 x 5 x 6 RHS 4x 1 + 2x 2 + 2x 3 + 1x 4 + 0x 5 + 0x x 1 + 1x 2 + 0x 3 + 2x 4 + 1x 5 + 0x x 1 + 0x 2 + 1x 3 + 0x 4 + 1x 5 + 2x x j = number of rolls to be cut using pattern j. Slide 16 The following IP minimizes the number of rolls required: n min j=1 x j n j=1 a ijx j n i, i =1,...,m, x j Z +, j =1,...,n N3 (a 1j,a 2j,...,a mj ) : representing a possible pattern e.g.,in the above instance,(a 11,a 21,a 31 )=(4,0,0),... x j : the number of rolls cut according to pattern j Number of variables huge! 5 Computational Issues: Cutting Stock 5.1 LP relaxation Linear relaxation: (LP) n min j=1 x j n j=1 a ijx j n i, i =1,...m, x j 0, j =1,...,n LP solution provides a lower bound to IP. How to solve (LP)? Feasible patterns: all nonnegative integer vectors (z 1,...,z m ) satisfying m w i z i W i=1 Slide 17 Slide 18 6

8 Not all feasible patterns are needed in the above formulation. We only need to focus on the non-dominated feasible patterns. N4 It is impractical to maintain a full matrix A explicitly as the number of non-dominated feasible solution is usually rather large. Note 4 Number of Solutions Non-dominated solutions: The cutting pattern on a roll is non-dominated if the waste (left-over) cannot be use to satisfy demand of other items E.g. If (1,1,0) and (2,1,0) are both valid cutting patterns, then pattern (1,1,0) is dominated since the waste can be use to generate another unit of item 1. Example: The number of non-dominated solutions can be obtained via OPL: enum pattern a,b,c,d,e; int width[pattern]=[5,15,25,35,45]; int rollwidth = 200; var int+ x[pattern] in 0..rollwidth; solve{ forall( i in pattern) { sum(j in pattern)width[j]*x[j] <= rollwidth; sum(j in pattern)width[j]*x[j]+width[i] > rollwidth; }; }; The number of non-dominated patterns = 325 The total number of possible patterns: Revised Simplex Method Consider LP in standard form min c x Ax = b x 0 assuming that rows of matrix A are linearly independent. Slide 19 Slide 20 Revised simplex method 1. Start with the columns corresponding to the basic variables of a basic feasible solution (i.e. store only the basis matrix and not the full matrix A). 2. Let x =(x B, x N ), A =[B, N], c =(c B, c N ). 7

9 3. Identify a column from N with a negative reduced cost (reduced cost = c j c B B 1 N j to enter the basis. 4. Update the basis matrix and repeat the steps until all reduced costs are non-negative. For linear programming problems with large number of decision variables (or columns),it is computationally impossible to generate and store the entire coefficient matrix A. Revised simplex method provides a means to overcome this problem provided there is an efficient way to identify a nonbasic column with negative reduced cost c j. 5.2 Identifying a column with a negative reduced cost The problem of identifying a negative reduced cost column can be resolved by solving the problem Slide 21 Slide 22 where the minimization is over all j. min c j c j c B B 1 N j Very often,this optimization problem has a special structure and the smallest c j can be found without examining every c j. Slide 23 Column Generation for cutting stock problem: 1. Start with a basic feasible solution. For example,use the simple pattern to cut a roll into W/w i rolls of width w i. (The basis matrix is a diagonal matrix.) 2. For any pattern j,reduced cost is 1 m i=1 π ia ij (c j =1),where(π 1,...,π m ) (=c B B 1 ) is the simplex multipliers vector associated with the current set of columns. Slide Identify a pattern with negative reduced cost to be used for updating the basis min 1 m i=1 π ix i subject to i w ix i W. x i integral. Equivalently,solve the Knapsack problem Z m =max i=1 π ix i subject to i w ix i W, x i integral. Slide 25 8

10 5.2.1 Knapsack problem Z =max subject to m i=1 πixi i wixi W, x i integral. Slide 26 The knapsack problem is NP-complete. Why? Fortunately,there exist practical algorithms for this problem. Slide 27 Computational Result (CPLEXMIP): N5 n CPU (s) 1, , , Note 5 int+ n = 1000; Code for Knapsack float+ w[1..n]; initialize{ forall(i in 1..n) w[i] = (rand(3000)+1)/100; }; float+ p[1..n]; initialize{ forall(i in 1..n) p[i] = ((rand(10000)+1)/100; }; var int+ x[1..n] in ; maximize sum(i in 1..n)p[i]*x[i] subject to{ sum(i in 1..n)w[i]*x[i] <= 150; }; Slide 28 9

11 Solution time depends on parameters w i =(rand(3000) + 1)/100 gives rise to easy instances most of the time. Change to w i =20+(rand(30) + 1)/100 in the code. n CPU (s) 1, , How Good is the LP relaxation from CG? Number of items = 5. Tested on several instances: Kantorovich (IP) Col Gen (LP) Slide 29 Conjecture: Unfortunately,this is not true: W = 273 w 1 =18 n 1 = 233 w 2 =91 n 2 = 310 w 3 =21 n 3 = 122 w 4 = 136 n 4 = 157 w 5 =51 n 5 = 120 Z IP Z LP? Slide 30 Z LP (CG) = Z IP = 230. N6 Note 6 Round Up Conjecture In 1985 the theoretical result of Marcotte (The cutting stock problem and integer rounding,math. Programming,33 (1985),pp ) shed light on the relationship between solutions of the LP relaxation and the cutting stock problem itself. She proved that for some practical instances of the problem a so-called round-up property is valid. It means that to find the optimal value of the cutting stock problem,it is sufficient just to solve the LP relaxation and to round up the value of the objective function. Unfortunately,this conjecture is not true for all instances of the cutting stock problem. Fieldhouse (The duality gap in trim problems,sicup-bulletin No. 5,1990) presents an example of the cutting stock problem with a gap of This gives rise to the modified round up conjecture. 10

12 Modified Round Up Conjecture: Slide 31 Z IP Z LP +1? This conjecture has not been answered. 6 Getting Intergal Solution The solution obtained from solving the column generation problem is fractional. How to obtain integral solution from the fractional solution? - round up Rounding Up Let x j be the (fractional) LP solution obtained from the column generation method. Let x j = x j. x j integral. j a i,jx j n i implies j a i,jx j n i. So x j defined in this way is a feasible integral solution. How good is this heuristic? Z LP (CG) = W = 273 w 1 =18 n 1 = 233 w 2 =91 n 2 = 310 w 3 =21 n 3 = 122 w 4 = 136 n 4 = 157 w 5 =51 n 5 = 120 Round-Up produces a solution of 231 Round-Up Fractional cut cut cut cut cut cut cut What are some of the disadvantages of the round-up heuristic? Slide 32 Slide 33 Slide 34 Slide 35 11

13 7 Implementation in OPL Script There are four files: CuttingStock.osc, cut.dat, chooseconfigs.mod, and generateconfigs.mod. Slide 36 Master Problem solved using chooseconfigs.mod. Pricing problem solved using generateconfigs.mod Flow between Master and Pricing problems controlled by the OPL script CuttingStock.osc. Script starts by generating an initial set of pure patterns for the master LP. Read: OPL Script: Composing and Controlling Models by Pascal Van Hentenryck. 8 Issues with Column Generation How to select the columns to enter the Master Problem? For problems with degeneracy,the column selected may not be useful in improving the quality of the solution. Slide 37 How to speed up the convergence of the column generation algorithm? The Column Generation solves the LP relaxation. What if we want to obtain the optimal integral solution? Branch-And-Bound,Branch-And-Cut,Branch-And-Price etc. How to ensure that a column,fixed at 0 in certain node of the branching tree,will not be selected by the pricing algorithm? 9 Connection with Lagrangian Relaxation How would you use LR to solve the cutting stock problem? Slide 38 min K k=1 yk K k=1 xk i n i i =1, 2,...,m, m i=1 xk i w i Wy k k =1,...,K, y k {0, 1} k, x k i 0,xk i integral Which constraints would you relax? Slide 39 Suppose you relax the first class of constraints: 12

14 L(u) =min K k=1 yk + ( m i=1 u i n i ) K k=1 xk i m i=1 xk i w i Wy k k =1,...,K, y k {0, 1} k, x k i 0,xk i integral where u i 0 for all i. Slide 40 where K m L(u) = L k (u)+ u i n i k=1 i=1 L k (u) =min y k m i=1 u ix k i m i=1 xk i w i Wy k y k {0, 1} x k i 0,xk i integral L k (u) is the minimum of the two values: zero (when y k = 0),or 1-Z (when Slide 41 y k =1),whereZ is obtained by solving the knapsack problem Z =max m i=1 u ix k i m i=1 xk i w i W x k i 0,xk i integral The subproblem in Lagragian Relaxation reduces again to a Knapsack problem! Proposition: Slide 42 max u 0 L(u) =Z LP (CG). The optimal Lanagrangian multiplers is the LP dual multiplers to the constraints N j=1 a i,jkλ j n i in the column formulation. Lagrangian relaxation solves the dual of the column formulation! Slide 43 The bounds obtained by both methods are identical. Whichmethodisbetter? Column Generation Primal,dual optimal solution Bounds monotone LP solver needed Langrangean Relaxation Dual but not primal solution Bounds zig-zag Easy to implement 13

15 10 Derivation of proposition (Recitation) K m L(u) = L k (u)+ u i n i k=1 i=1 The value L k (u) does not depend on the index k. Slide 44 m L(u) =KL(u)+ u i n i i=1 where L(u) =min y m i=1 u ix i m i=1 x iw i Wy y {0, 1} x i 0,x i integral Slide 45 Let z j =(a 1,j,...,a m,j ), j =1,...,N be the extreme points of the polytope m x i w i W, x i 0, x i integral. i=1 m L(u) =min(0, 1 max j=1,...,n i=1 a i,ju i )=min j=1,...,n min(0, 1 m i=1 a i,ju i ) Slide 46 m L(u) =KL(u)+ u i n i The Lagrangian Dual reduces to max u 0 L(u) = max u 0 + min j=1,...,n m ) u i n i i=1 i=1 ( K min(0, 1 m a i,j u i ) i=1 max y y m i=1 u in i for the zero extreme point y K(1 m i=1 u ia i,j )+ m i=1 u in i for the jth extreme point u i 0 i =1,...,m. Equivalently, Slide 47 14

16 max y (λ 0 ) y m i=1 u in i 0 (λ j ) y + K( m i=1 u ia i,j ) m i=1 u in i K u i 0 i =1,...,m. λ 0,λ j are the associated dual variables. The dual of this problem: min N j=1 Kλ j λ 0 + N j=1 λ j =1 n i (λ 0 + N j=1 λ j)+ N j=1 a i,jkλ j 0. λ j 0 j =1,...,N. This is just Slide 48 ) min (Kλ j N j=1 λ 0 + N j=1 λ j =1 N j=1 a i,jkλ j n i λ j 0 j =1,...,N. For K large,the constraint λ 0 + N j=1 λ j = 1 is redundant as long as there is a feasible solution λ j with N j=1 λ j 1 Let x j = Kλ j. We obtain the column formulation of the cutting stock problem! 15