Column Generation The idea of the column generation can be motivated by the trim-loss problem: We receive an order to cut 50 pieces of.5-meter (pipe) segments, 250 pieces of 2-meter segments, and 200 pieces of 4-meter segments from some 7-meter steel pipes of a given diameter. Assume that the cutting operation does not consume any length of a steel pipe, and any cut pipes and segments left after satisfying the demands are regarded as loss. Determine how to cut the steel pipes so as to minimize the loss. Let X be the total number of steel pipes used to cut the segments. Then the trim loss = 7X (50)(.5) (250)(2) (200)(4) = 7X-525. Thus, the minimal solution is given by minimizing X, the number of steel pipes used. There are different formulations. For ease of reference, let (l, k, m) be a cut pattern that produce l pieces of.5-meter segments, k pieces of 2-meter segments, and m pieces of 4- meter segments. Formulation. The order can be satisfied by 255 steel pipes: 200 steel pipes with the (0,, ) cut pattern, 38 steel pipes with the (4, 0, 0) cut pattern, and 7 steel pieces with the (0, 3, 0) cut pattern. Thus, the minimum number should be no more than 255. Let, if the ith pipe is used, y i = i =,, 255; 0, ow.., x ij = the number of the jth type of segments cut from the ith pipe, where j = for the.5-meter segments, j = 2, for the 2-meter segments, and j = 3 for the 4- meter segments; e.g., the cut pattern x i = (2, 2, 0) gives two.5-meter and two 2-meter segments. Let M be a large positive number, and Z + be the set of non-negative integers. min 255 y, i s.t. 255 x 50; i i i 255 i xi 2 250; 255 i xi 3 200;.5x 2x 4x 7 y, i =,, 255; i i2 i3 i x ij Z + {0}, y i {0, } i, j.
The formulation is a correct one (where 255 can be replaced by other upper bounds, possibly tighter, of the minimum number of steel pipes). However, the formulation is hard to solve in the sense that it contains many alternate minimum; e.g., for any minimum solution, one get an alternate minimum by swapping the values of * * * i, i 2, i 3 x x x with * * * i, i 2, i 3 x x x (i.e., by 2 2 2 swapping the cut patterns of the i th and the i 2 th steel pipe.) Finding the solution by branch and bound will face an enormous search tree. Formulation 2. A more efficient formulation is to define variables on the cut patterns. Let x i be the number of steel pipes cut in the ith pattern, where the cut patterns are given by the following table. Let a ij be the number of j type segments produced by the ith cut pattern. Thus, the ith cut pattern is defined by (a i, a i2, a i3 ) T. As an example, a = (a, a 2, a 3 ) T = (0, 0, ). Further let t i be the trim loss of the ith cut pattern; e.g., t = 3. Cut Patterns x x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 x 0 x x 2 x 3 x 4 x 5.5-meter segment 0 0 2 0 2 0 2 0 4 3 2 2-meter segment 0 0 0 3 2 2 2 0 0 0 0 4-meter segment 0 0 0 0 0 0 0 0 0 0 0 trim loss (meter) 3.5 0 0.5 3 2 3.5 5 2.5 4 5.5 min 5 tx, i i i s.t. 5 a x 50; i i i 5 i ai 2 xi 250; 5 i ai 3 xi 200; x i Z + {0}, i. Here is a question on Formulation 2: Is it necessary to include inefficient cut patterns such as x in which a 3-meter segment is wasted? Example. It is necessary to include the inefficient cut patterns. Otherwise, there may not be a feasible solution for the above formulation. Consider cutting 3-meter and 7-meter segments from 0-meter steel pipes. If we only consider the efficient segments, Formulation 2 gives no feasible solution for an order of 3 pieces of 3-meter segments and piece of 7-meter segment. Is there any way to skip the inefficient cut patterns? If so, we can reduce the number of variables. 2
Formulation 3. This formulation ignores inefficient cutting patterns. min 5 i i x, s.t. 5 a x 50; i i i 5 a 2 x 250; i i i 5 a 3 x 200; i i i x i Z + {0} i. Cut Patterns x x 2 x 3 x 4 x 5.5-meter segment 0 2 0 2 4 2-meter segment 0 3 2 0 4-meter segment 0 0 0 trim loss (meter) 0 0 Formulation 3 is the simplest one for the trim-loss problem. It is an integer program, which is hard to solve. The formulation has another drawback: the number of variables (i.e., cut patterns) increases rapidly with the length of steel pipes and the number of pipe segments. Try to, for example, find out the number of efficient cut patterns for steel pipes of 0 meters long, or for the 6-meter steel pipes with an additional demand of 0.8-meter pipe segments. In the following we introduce the idea of column generation to tackle the explosive number of variables. It is basically an elegant revised simplex method that at each iteration identifies the best entering variable by solving an optimization problem. Note that in the introduction of the idea, we work with continuous rather than discrete variables; the techniques to handle integer variables is beyond the scope of this course. Observe that the formulation is in the form of: min i x i, s.t. i a ij x i b j, j =, 2, 3, x i 0, i =, 2, 3. (Recall we ignore the integral constraints.) The reduced cost of the ith variable is given by c i - T cbb a i = c i - T a i, where T is the vector of dual variables. Note that all c i =. Thus, the most negative reduced cost is given by min i - T a i, where a i = (a ij ) be a desirable cut pattern. Let l j be the length of the jth type segment required; L be the length of a steel pipe. Then the most negative reduced cost can be found by solving: Subproblem max j j a j, s.t. j l j a j L; 3
a j Z + {0}, i. This is the standard knapsack problem, which is NP-hard. Fortunately, for a given L the problem can be solved by algorithms polynomial in L; e.g., dynamic programming. From this knapsack problem, we generate a new column to enter the problem. The problem with the columns selected so far is called the master problem. Generic Column Generation Algorithm (for the trim loss problem): Select columns for the initial basis. 2 Solve the master problem (by revised simplex) and find the dual variables. 3 Solve the knapsack subproblem max z sub = j j a j s.t. j l j a j L. Stop if the maximal value ; else go to 2. To use column generation, we may start with inefficient cut patterns. For example, consider a = (, 0, 0) T, a 2 = (0,, 0) T, and a 3 = (0, 0, ) T. Iteration. Master Program min x + x 2 + x 3, s.t. x 50; x 2 250; x 3 200; x, x 2, x 3 0. Obviously, x B = (x, x 2, x 3 ) T ; the minimum solution is x * = 50, * x 2 = 250, and x 3 * = 200, 0 0 with z * = 600; B = 0 0 = B -. With 0 0 Sub-problem: max a + a 2 + a 3, s.t..5a + 2a 2 + 4a 3 7, a, a 2, a 3 Z + {0}. T c B = (,, ), T B c B = (,, ). The maximum of this knapsack problem, (a, a 2, a 3 ) * = (3,, 0), with z sub * = 4 >. Thus, it is good to include the cut pattern a 4 = (3,, 0) T into the master program. Iteration 2. Master Program min x + x 2 + x 3 + x 4, s.t. x + 3x 4 50; x 2 + x 4 250; x 3 + 200; x, x 2, x 3, x 4 0. 4
Clearly x 3 remains in the optimal basis. Similarly, by comparing x and x 2, x should be the leaving variable. x B = (x 2, x 3, x 4 ) T ; The minimum solution is x 2 * * * = 200, x 3 = 200, and x 4 = 0 0 3 /3 0 50, with z * = 450; B 0 and B - = T 0 0. cbb = (0,, ). 0 0 /3 0 0 Remark. The new basis inverse can be found from (B new ) - = E(B old ) - with the basic variables arranged in the order of the equations as in the revised simplex method. Here, to avoid being sidetracked by such procedure, we simply arrange basic variables in increasing indices and assume that B - can be found as B is known. Sub-problem: max a 2 + a 3, s.t..5a + 2a 2 + 4a 3 7, a, a 2, a 3 Z + {0}. The maximum of this knapsack problem, (a, a 2, a 3 ) * = (0, 3, 0), with z sub * = 3 >. Let us include a 5 = (0, 3, 0) T into the problem. Iteration 3. Master Program min x + x 2 + x 3 + x 4 + x 5, s.t. x + 3x 4 50; x 2 + x 4 + 3x 5 250; x 3 + 200; x, x 2, x 3, x 4, x 5 0. x 3 remains in the basis. The entering of x 5 replaces x 2 (agree?). x B = (x 3, x 4, x 5 ) T ; the 0 3 0 minimum solution is x 3 * * = 200, x 4 = 50, and x 5 * = 200 3,with z* = 36 2 ; B 0 3 3 and 0 0 0 0 B - = T /3 0 0. cbb = 2,,. 9 3 /9 /3 0 Sub-problem: Max 2 9 a + 3 a 2 + a 3, s.t..5a + 2a 2 + 4a 3 7, a, a 2, a 3 Z + {0}. The maximum of this knapsack problem, (a, a 2, a 3 ) * = (2, 0, ), with z sub * = 3 9 include a 6 = (2, 0, ) T into the problem. >. Let us 5
Iteration 4. Master Program min x + x 2 + x 3 + x 4 + x 5 + x 6, s.t. x + 3x 4 + 2x 6 50; x 2 + x 4 + 3x 5 250; x 3 + + x 6 200; x, x 2, x 3, x 4, x 5, x 6 0. x B = (x 3, x 5, x 6 ) T ; the minimum solution is 0 0 2 /2 0 B 0 3 0 and B - = 0 /3 0. 0 /2 0 0 Sub-problem: Max 2 3 a + a 3, s.t..5a + 2a 2 + 4a 3 7, a, a 2, a 3 Z + {0}. * x 3 = 25, * x 5 = 250 3, and * 6 T cbb = 3 0,,. x = 75, with z * = 283 ; The maximum of this knapsack problem, (a, a 2, a 3 ) * = (0,, ), with z sub * = 4 3 include a 7 = (0,, ) T into the problem. Iteration 5. Master Program min x + x 2 + x 3 + x 4 + x 5 + x 6 + x 7, s.t. x + 3x 4 + 2x 6 50; x 2 + x 4 + 3x 5 + x 7 250; x 3 + + x 6 + x 7 200; x, x 2, x 3, x 4, x 5, x 6, x 7 0. 3 >. Let us x B = (x 5, x 6, x 7 ) T * ; the minimum solution is x 5 = 25 3, * * x 6 = 75, and x 7 = 25, with z * = 24 2 ; 3 0 2 0 /6 /3 /3 B 3 0 and B - = T /2 0 0. cbb =,, 2. 6 3 3 0 /2 0 Sub-problem: Max 6 a + 3 a 2 + 2 a 3 3, s.t..5a + 2a 2 + 4a 3 7, a, a 2, a 3 Z + {0}. 6
The maximum of this knapsack problem cannot be larger than (why?), and the minimum of the master problem has been reached. The optimal cutting of the pipes are: cut pattern (0, 3, pieces; cut pattern (2, 0, ): 75 pieces; cut pattern (0,, ): 25 pieces. 0): 25 3 2. A shortest-path Sub-problem in Column Generation In solving network-based optimization problems by column generation, we frequently encounter a shortest-path sub-problem in looking for new columns. In this section we give an example to provide the intuition. Our example is from [], which is in fact originated from our textbook. Figure shows a network G where the pair (c ij, t ij ) beside arc (i, j) is the cost spent and time taken in passing through the arc. The objective is to find a shortest path such that the total time should be no longer than 4 units. Let A be the set of arcs in G; x ij = if the arc is taken, and x ij = 0 otherwise; (i, j) A. (, ) 2 4 (, 0) (2, 3) (, 7) (, 2) (5, 7) (0, ) 6 Problem P. (0, 3) 3 (2, 3) 5 Figure. Time-Constrained Network (2, 2) min (, i j) A c x s.t. x (, i j) A ij ij j,, x ij {:(,) j i j A} {:(,) j i j A} (, ij) A x i6 t x ij ij (, i j) A, 4, x ji 0, x ij {0, }, (i, j) A. (6) This is a constrained shortest-path problem, which is NP-hard. We will solve this problem by column generation. In doing so, we will use the path formulation of the problem. Table. The set of Paths in the Problem path cost time -2-4-6 3 8-2-4-5-6 4 4-2-5-6 5 5 () (2) (3) (4) (5) 7
-3-2-4-6 3 3-3-2-4-5-6 24 9-3-2-5-6 5 0-3-4-6 6 7-3-4-5-6 27 3-3-5-6 24 8 There are 9 paths from node to node 6. Only three of them are feasible and the cheapest one is path -3-2-4-6. For general problems, there are so many paths that it is impossible to list out all the paths. We only use this small example to illustrate the idea of running into a short-path subproblem in column generation. Observe that P is easy to solve if there is no constraint (5). Moreover, the optimal solution of ()-(4) and (6) is a path from node to node 6. We will manipulate this fact. To do so, we first introduce the path formulation of the problem., if arc ( i, j) is in path p, Let a pij = For example, for path -2-4-6 to be the first 0, ow.. path, a 2 =, a 24 =, a 46 =, and other a ij = 0. Note that a pij are parameters, not variables. Define variable p to be the probability that path p is taken. Let P be the set of all paths. x ij apijp, pp pp p, (i, j) A, (7) p 0, p P. (9) We can substitute x ij by p and a ijp to form a new optimization problem. Problem P2. t x t a t a A A P P A cij xij cij apij p cijapij p ; (, ij) A (, ij) A pp pp (, ij) A ij ij ij pij p ij pij p. (, i j) (, i j) p p (, i j) (8) min cij xij cij apij p cijapij p, (, i j) A (, i j) A pp pp (, i j) A (0) s.t. ta ij pij p 4, (, i j) A, pp pp p () (2) p 0, p P, (3) x a (i, j) A, (4) ij pij p, pp x ij {0, }, (i, j) A. (5) 8
We solve P2 to find p, where the paths p are found from the optimal problem with ()-(4) and (6), and we need to ensure that x ij being binary. P2 is a hard integer program. To solve the problem, our first step is to relax the integer constraints (5). Note that we can also drop (4), because the relationship has been used to transform decision from the arc variables x ij to the path variables p. The resulted optimization problem (0)-(3) is called the master problem. Here the main idea of column generation: We start with a restricted master problem with a small set of paths and incorporate a path in the set for consideration only if it is beneficial to do so. Let and 0 be the dual variable of constraint () and (2), respectively. Our steps are: (a). (b). Solve the restricted master problem with the current set of paths to obtain, 0, and the minimum value. Search for another path that reduces objective function value. In step (b), we look for variable p with negative reduced cost, i.e., cp cijapij tijapij0 0. (, ij) A (, ij) A To do so, we solve the following shortest-path problem P3: min c t x, (6) (, i j) A s.t. (2)-(4), (6). ij ij ij 0 If the minimum value of P3 is non-negative, the current set of paths is already optimal; otherwise, the new path is added into P2 to solve for, 0, and minimum value. Remarks. (). The procedure discussed only solves the LP relaxation but not the original P. To really solve the problem, we need to adopt branch and bound search in case there is any fractional x ij. (2). Note the tricks of our solution method. There is no simple efficient algorithm for an NP-problem. Thus, we consider the LP relaxation of the problem, which gives a network problem plus some hard constraints. Then we use the path-based formulation to express the problem in terms of the path variables. This path-based optimization problem is solved by column generation. The upper level is a master problem that contains paths considered up to the current iteration, and the lower-level sub-problem is to select a new path for the next iteration of the master problem. As usual, the objective function of the sub-problem is the reduced cost of the master program, which requires the dual variables of the master problem. Since the constraints of the sub-problem usually form a network optimization problem, the minimal reduced cost is often found from a shortest-path problem. 9
Reference [] Desrosiers, J., and M.E. Lubbecke [2005] A primer in column generation. In Column Generation, edited by G. Desaulniers, J. Desrosiers, and M.M. Solomon, Springer, Yew York, pp -32. 0