3.7 Cutting plane methods

Transcription

1 3.7 Cutting plane methods Generic ILP problem min{ c t x : x X = {x Z n + : Ax b} } with m n matrix A and n 1 vector b of rationals. According to Meyer s theorem: There exists an ideal formulation: conv(x ) is a rational polyhedron, i.e, conv(x ) = {x R n + : Ax b} for appropriate A and b with rational entries. The ideal formulation can contain a potentially very large number of constraints. The same result also holds for mixed integer sets X = {(x, y) R n 1 + Z n 2 + : A 1x + A 2y b} provided the coefficients of A 1, A 2 and b are rational. For NP-hard problems the ideal formulation is unknown or cannot be directly used. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

2 Idea: Try to improve an initial formulation, to obtain a better approximation of conv(x ), by adding valid inequalities. Illustration: Definition: π t x π 0 is a valid inequality for X R n if π t x π 0 for each x X. Examples: 1) Uncapacitated Facility Location (UFL) problem Weak original ILP formulation: min m n i=1 j=1 c ijx ij + n j=1 f jy j s.t. n j=1 x ij = 1 i M m i=1 x ij my j j N (1) y j {0, 1} with the n aggregated coherence constraints (1). j N 0 x ij 1 i M, j N For each pair i M and j N, the disaggregated coherence constraint x ij y j is a valid inequality for UFL not implied by constraints (1). Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

3 2) Asymmetric TSP problem ATSP: Given directed G = (V, A) and a cost c ij for each arc (i, j) A, find a Hamiltonian circuit of minimum total cost. ILP formulation: min s.t. (i,j) A c ijx ij (2) (i,j) δ (j) x ij = 1 j (3) (i,j) δ + (i) x ij = 1 i (4) (i,j) δ + (S) x ij 1 S V : 1 S (5) x ij {0, 1} (i, j) A (6) with an exponential number of valid cut-set inequalities (5). Two main ways to use valid inequalities: add them a priori to strengthen the initial formulation generate them as needed via a cutting plane method Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

4 1) Addition a priori of valid inequalities Advantage: Branch and Bound method based on a stronger formulation is in general more efficient. Disadvantage: If the number of valid inequalities is huge, it makes extremely heavy the linear programming relaxation and/or impossibile the use of standard Branch and Bound. Example: Uncapacitated Facility Location (UFL) problem Given the weak UFL formulation with n aggregated coherence constraints x ij my j j N, i M add the stronger mn disaggregated coherence constraints x ij y j with i M and j N, and delete the former ones. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

5 2) Cutting plane methods Given a generic ILP problem: min{c t x : x X = P Z n } where P = {x R n + : Ax b} is the feasible region of the linear relaxation. Consider a family F of inequalities πx π 0 valid for X, (π, π 0) F. Often the number of inequalities in F is too large to be added a priori. Example: cut-set (subtour-elimination) inequalities for TSP Definition: Given x P with x X, a cutting plane is an inequality π t x π 0 such that π t x π 0 is valid for X = P Z n π t x > π 0 Illustration: Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

6 Idea cutting plane methods: The ideal formulation of X (namely conv(x )) is not necessary, it is enough to add a subset of cutting planes which provide a good description of conv(x ) around the optimal solution x ILP, i.e., which bring out x ILP as optimal extreme point of the linear relaxation polyhedron for c under consideration. Illustration: Separation problem: Given a vector x / X and a family of valid inequalities F for X, find a valid inequality in F which separates x from conv(x ) or establish that such a cutting plane does not exist. Illustration: Examples: Separation of cut-set inequalities for ATSP and of Gomory fractional cutting planes for general ILPs see below. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

7 Cutting plane algorithm Initialization P := P = {x R n + : Ax b} 1 Solve the current linear relaxation min{c t x : x P } and let x LP optimal solution. denote an 2 IF x LP Zn THEN terminate because x LP is also an optimal solution of the ILP ELSE Solve the separation problem for x LP, F and X = P Z n IF π t x π 0 is found THEN P := P {x R n back to 1. : π t x π 0} and go ELSE stop N.B.: If we fail to find an optimal integer solution x LP, P is anyway stronger than P. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

8 Separation of cut-set inequalities for the ATSP ILP formulation: min s.t. (i,j) A c ijx ij (7) (i,j) δ (j) x ij = 1 j (8) (i,j) δ + (i) x ij = 1 i (9) (i,j) δ + (S) x ij 1 S V : 1 S (10) x ij {0, 1} (i, j) A (11) Due to the exponential number of constraints (10), we adopt a cutting plane approach: start solving the linear relaxation of (7)-(11) without cut-set inequalities (10), namely min (i,j) A c ijx ij (12) s.t. (i,j) δ (j) x ij = 1 j (13) (i,j) δ + (i) x ij = 1 i (14) x ij 0 (i, j) A, (15) and iteratively add some of those which substantially violate the current optimal fractional solution. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

9 Proposition: Given an optimal solution x LP of the current linear relaxation ( (12)-(15) with the constraints (10) generated so far), a cut-set inequality (10) violated by x LP can be obtained (if it exists) by solving a sequence of instances of the maximum flow problem. Separation algorithm: (see 2nd computer lab) Given x LP, we look for a subset S V with 1 S such that (i,j) δ + (S ) x ij < 1. Consider the directed graph G = (V, A ) where A := {(i, j) A : xij source s = 1 and capacity xij for each arc (i, j) A. > 0} with For every choice of sink t V \ {1}, we look for a cut δ(s ), separating 1 S from t V \ S, of minimum capacity. This can be done in polynomial time by finding a Maximum Flow from s to t in the capacitated network G (strong LP duality). If the minimum cut capacity < 1, S induces a cut-set inequality that is violated by x LP, otherwise no such violated inequality separating node 1 from node t exists. N.B.: The procedure yields a number of violated cut-set inequalities (potentially one for each t). Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

10 3.7.1 Simple valid inequalities 1) Binary set X = {x {0, 1} 5 : 3x 1 4x 2 + 2x 3 3x 4 + x 5 2} Since the constraint cannot be satisfied when x 2 = x 4 = 0, x 2 + x 4 1 is a valid inequality. If x 1 = 1 and x 2 = 0, the constraint cannot be satisfied. Thus x 1 x 2 is another valid inequality. 2) Mixed 0-1 set Consider X = {(x, y) : x cy, 0 x b, y {0, 1}} with c > b. Example: c = 5 and b = 2 Since X = {(0, 0), (x, 1) con 0 x b}, the inequality x by is valid and describes, together with x 0 e y 1, the convex hull of X. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

11 3) Combinatorial set (matching problem) Maximum Matching problem: Given an undirected graph G = (V, E) with a profit p e R for each edge e = {i, j} E, determine a matching, i.e., a subset of edges without common nodes, of maximum total profit. Illustration: Let X = {x {0, 1} E : x e 1, i V } with δ(i) = {e E : e = {i, j} for any j V }, e δ(i) be the set of all incidence vectors of matchings in G. Consider any subset S V of odd cardinality, with S 3. Since the number of edges in any matching having both end-points (nodes) in S is at most S 1, 2 x e S 1 2 e E(S) is a valid inequality for X, where E(S) = {e E : e = {i, j}, i, j S}. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

12 3.7.2 Chvátal cutting planes for ILP Generate valid inequalities by linearly combining inequalities and rounding coefficients. Integer rounding principle: Given X = {x Z : x b} where b Q \ Z, then x b is a valid inequality for X. Example 1: Consider the integer set X = {(x 1, x 2) t Z 2 + : x 1 + 2x 2 4, x 1 2x 2 3, 1 x 1 3}. By multiplying x 1 1 and x 1 + 2x 2 4 by 1/2, adding them we obtain x 1 + x 2 3/2. Rounding down the right-hand-side term, we have: x 1 + x 2 1, which is valid for X and needed to describe conv(x ). Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

13 Example 2: z ILP = max x 1 + x 2 + x 3 s.t. x X = {x Z 3 + : x 1 + x 2 1, x 2 + x 3 1, x 1 + x 3 1} z LP = max x 1 + x 2 + x 3 s.t. x P = {x R 3 + : x 1 + x 2 1, x 2 + x 3 1, x 1 + x 3 1} Optimal solutions: x ILP = (1, 0, 0) with z ILP = 1, x LP = (0.5, 0.5, 0.5) with z LP = 1.5 Adding the first constraint multiplied by 0.75 and the other two constraints multiplied by 0.5, we obtain 5 4 x x2 + x3 7 4 which is a valid inequality for P, and hence also for X. By rounding down the coefficients, we obtain the inequality x 1 + x 2 + x which is valid for P because x 0, and hence also for X. Since the variables x i must be integer, the inequality 7 x 1 + x 2 + x 3 = 1 4 is valid for X but not for P. For instance, it is violated by x LP. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

14 Chvátal-Gomory procedure for generating valid inequalities: Consider X = P Z n with P = {x R n + : Ax b} Choose u R m + and consider u t Ax u t b Setting π t = u t A, we obtain π t x u t b with m π j = u i a ij which is valid for P and for X. i=1 j = 1,..., n Setting π 0 = u t b we obtain π t x π 0 which is valid for X (but not necessarily for P). Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

15 Example 3: Matching polytope Consider a generic maximum matching problem instance with an undirected graph G = (V, E). For any subset S V of odd cardinality, with S 3, the valid inequality e E(S) x e S 1 2 where E(S) = {e E : e = {i, j}, i, j S}, is a Chvátal-Gomory inequality w.r.t. the linear description x e 1 i V (16) e δ(i) x e 0 e E. (17) Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

16 Proof: Consider any S V with S 3. Taking the linear combination of constraints (16) with u i = 0.5 for i S and u i = 0 for i S, we obtain x e + 1 x e S 2 2 which is valid for X. e E(S) Since x e 0 and x e Z for each e E, also is valid for X. e E(S) e δ(s) x e S 2 (18) If S is even, the inequality (18) is clearly implied by the constraints (16) associated to the nodes i S and constraints (17). If S is odd, S 2 = S 1 2 and the valid inequality (18) is not implied. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

17 Chvátal-Gomory procedure allows to generate all valid inequalities of any ILP problem. Theorem (Chvátal) Any valid inequality for X can be obtained by applying Chvátal-Gomory procedure a finite number of times. Proof for case X {0, 1} n cf. L. Wolsey, Integer Programming, Wiley, p Given any vertex (extreme point) x LP of P with some fractional coordinates, there always exits u 0 such that the corresponding Chvátal-Gomory inequality π t x π 0, with π t = u t A and π 0 = u t b, is not only valid for X but also violated by x LP. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

18 Definition: Denote by A 1 x b 1 all the inequalities that can be obtained by varying the vector u in R m +. P 1 = {x R n + : Ax b, A 1 x b 1 } is the first Chvátal closure of P. Obviously P 1 P, and P 1 = P if and only if P has no fractional vertices, that is P = conv(x ). If fractional vertices are left (P 1 conv(x )), we can iterate the procedure to obtain a Chvátal closure P k of higher rank, with k 2. Definition: The smallest integer k such that P k = conv(x ) is the Chvátal rank of conv(x ) with respect to P. Clearly P k = conv(x )... P 2 P 1 P. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

19 3.7.3 Gomory fractional/integer cutting planes Consider a generic ILP problem in standard form: min{ c t x : Ax = b, x 0, x Z n } where the m n matrix A and the n 1 vector b have integer coefficients, with n > m. Assumption: A is of full rank m (there are no redundant constraints) Idea: At each iteration of the cutting plane method, generate Chvátal-Gomory cut exploiting the information associated with the optimal basic feasible solution x LP of the current linear relaxation. Let B denote the basis (m m non singular submatrix) of A associated with x LP. Partitionning the columns of A in basic and non basic variables, Ax = b, x 0 can be written as Bx B + Nx N = b with x B 0 and x N 0, and can be expressed in canonical form as x B = B 1 b B 1 Nx N with x B 0 and x N 0, which emphasizes the optimal basic (feasible) solution x LP = (x B, x N ) = (B 1 b, 0). Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

20 If all components of B 1 b are integer, x LP is also optimal for the ILP. If at least one component of B 1 b is fractional, we generate a Chvátal-Gomory inequality (belonging to the first Chvátal closure and violated by x LP ). Let x h be a fractional basic variable corresponding to row t of the optimal tableau x h + j N a tj x j = b t (= x h ) (19) where N is the set of the indices of the non basic variables. N.B.: Equation (19) amounts to take u equal to the t-th row of B 1 in u t Ax = u t b, namely u t = e t t B 1 where e t is the t-th m-dimensional unit vector. Applying Chvátal rounding procedure to (19), we obtain: Definition: The integer form of the Gomory cut generated by row t of the optimal tableau of the linear relaxation is: x h + j N a tj x j b t. (20) Clearly (20) is valid for X but violated by x LP (since x j fractional). = 0 for j N and x h = b t is Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

21 Substracting (20) from (19) we obtain: Definition: The fractional form of the Gomory cut generated from row t of the optimal tableau of the linear relaxation: (a tj a tj ) x j b t b t. (21) j N If {a} := a a 0 denotes the fractional part of a R, (21) is equivalent to {a tj } x j {b t}. j N Recall: {4/3} = 1/3 but { 4/3} = 4/3 ( 2) = 2/3 The fractional and integer forms of a Gomory cut are clearly equivalent. Observation: The difference (slack) between the left-hand and right-hand sides of (20) and hence of (21) is always integer when x is integer. Note that the generation of a fractional/integer Gomory cut has minimal computational requirements. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

22 Example: Consider the ILP max x 1 + x 2 s.t. x 1 + x 2 5 2x 1 + x 2 0 5x 1 + 2x 2 18 x 1, x 2 Z + 1. Graphical solution of the linear relaxation: Two optimal basic solutions: x = (5/3, 10/3) and x = (8/3, 7/3) of value 5. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

23 2. Linear relaxation in standard form: max x 1 + x 2 s.t. x 1 + x 2 + x 3 = 5 2x 1 + x 2 + x 4 = 0 5x 1 + 2x 2 + x 5 = 18 x 1,..., x Canonical form w.r.t. the optimal basic solution x = (8/3, 7/3, 0, 3, 0): x x x5 = 8 3 x x3 1 3 x5 = 7 3 3x 3 + x 4 + x 5 = 3 Gomory cut derived from the x 1 row: - integer form: x 1 x fractional form: 3 x x5 2 3 Gomory cut derived from the x 2 row: - integer form: x 2 + x 3 x fractional form: 3 x x5 1 3 Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

24 4. Express the integer form of the Gomory cut associated with the fractional basic variable x 1 as a function of only x 1 and x 2. Substituting x 3 = 5 x 1 x 2 in x 1 x 3 2, we obtain the cut: 2x 1 + x Add this Gomory cut to the linear relaxation and find the optimal solution. Adding the Gomory cut 2x 1 + x 2 7 to the original problem, we obtain an optimal solution of the new linear relaxation x LP = (2, 3) with z LP5. Since x LP is integer, it is also optimal for ILP. Very unusual! Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

25 Theorem (Gomory): A lexicographic cutting plane method based on Gomory fractional/integer cuts terminates after a finite number of iterations. Finite termination is guaranteed by a careful choice of (i) the basis defining the optimal solution we intend to cut off and (ii) the row of the tableau used to generate the cut. In practice: Huge number of iterations and fractional/integer Gomory cuts tend to become weaker after a few iterations. Common strategy: Introduce several cuts at each iterations, e.g., all those with {b t} > ε = 0.01 Observation: Gomory fractional/integer cuts are generated through simple integer rounding: Let X = {x Z : x b} then the inequality x b is valid for X. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

26 3.7.4 Other methods to generate valid inequalities 1) Disjunctions Idea: Partition the set X R n + of feasible integer solutions into two (more) sets X 1 and X 2, derive inequalities for each X i and turn them into valid inequalities for X = X 1 X 2. Proposition 1: If n j=1 πi j x j π i 0 is valid for X i with i = 1, 2, then the inequality is valid for X = X 1 X 2. n min{πj 1, πj 2 }x j max{π0, 1 π0} 2 (22) j=1 Proof: Let x X 1 X 2. W.l.o.g. we can assume that x X 1, and hence n j=1 π1 j x j π0 1 which implies that n n min{πj 1, πj 2 }x j πj 1 x j π0 1 max{π0, 1 π0} 2 since x 0. j=1 j=1 Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

27 2) Mixed integer rounding Consider the mixed integer set X = {(x, y) t Z R + : x y b} where b Q \ Z. Illustration for the special case b = 3/2: Proposition 2: The mixed-integer rounding (MIR) inequality is valid for conv(x ). x 1 y b (23) 1 {b} Proof: Disjunction X 1 = X {(x, y) t : x b } and X 2 = X {(x, y) t : x b + 1}. By adding 1 {b} times x b 0 and inequality 0 y, we see that is valid for X 1. (x b )(1 {b}) y By adding (x b ) 1 and x y b with multipliers {b} and 1, we see that is valid for X 2. (x b )(1 {b}) y According to Proposition 1, (23) is valid for conv(x 1 X 2) = conv(x ). Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

28 Observation: The convex hull of X = {(x, y) t Z R + x y b, y 0 and the simple MIR inequality: x : x y b} is defined by 1 1 {b} y b. Illustration: Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

29 3.7.5 Gomory mixed integer cutting planes Consider a generic mixed integer program in standard form: min c t 1 x + ct 2 y s.t. A 1x + A 2y = b (24) x 0, y 0 (25) x integer. (26) Let (x LP, y ) be an optimal basic feasible solution of the linear relaxation (associated LP with the basis B and non basis N). Denote by N 1/N 2 the indices in N corresponding to the integer/continuous variables. Since x LP is not integer (otherwise (x LP, y ) would be optimal), an index h B such LP that xh Z. The simplex method yields the linear system in canonical form w.r.t. an optimal basis with t-th row: x h + j N 1 a tj x j + j N 2 a tj y j = b t (27) for appropriate coefficients a tj and b t, with b t Z. Notation: For any real number a let a + = max{a, 0} and a = max{ a, 0}. Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

30 Proposition 3: The Gomory mixed integer (GMI) inequality x h + ( a tj + ({a + tj } {b t}) )x j b t + (a tj ) j N 1 {b 1 t} j N 1 {b y j (28) 2 t} is valid for the feasible region (24)-(26) and is violated by (x LP, y LP ). Proof: Due to the nonnegativity of all variables, (27) implies that x h + a tj x j + a tj x j b t + (a tj ) y j j N 2 j N 1 : {a tj } {bt } is valid for the feasible region (24)-(26). j N 1 : {a tj }>{bt } Since a tj = ( a tj + 1) (1 {a tj }) for each j, by splitting the second summation we obtain x h + a tj x j + ( a tj +1)x j b t + (a tj ) y j + (1 {a tj })x j. j N 2 j N 1 : {a tj } {bt } j N 1 : {a tj }>{bt } j N 1 : {a tj }>{bt } Denoting by x the left hand side term and by y the right hand side one except b t, we have x b t + y. Since x Z and y 0, Proposition 2 implies that x b t + algebraic manipulations we then obtain the GMI cut (28). y is valid. With simple 1 {b t } Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31

31 Clearly (x LP, y ) does not satisfy (28) because x LP j j N 2, and xh = b t > b t. = 0 for each j N 1, y j = 0 for each Observations: For pure integer sets (ILP) i) the GMI cut (28) is potentially stronger than the corresponding Gomory fractional/integer cut since ({a tj } {b t }) + 0 and y j = 0 for all j N 2, 1 {b t } ii) the coefficients are not integer anymore. Unlike for Gomory fractional/integer cuts in the pure integer case, we have no finite termination guarantee for GMI cuts but they are very effective in practice (see later). Edoardo Amaldi (PoliMI) Ottimizzazione A.A / 31