Discrete (and Continuous) Optimization Solutions of Exercises 2 WI4 131 Kees Roos Technische Universiteit Delft Faculteit Electrotechniek, Wiskunde en Informatica Afdeling Informatie, Systemen en Algoritmiek e-mail: C.Roos@ewi.tudelft.nl URL: http://www.isa.ewi.tudelft.nl/ roos November December, A.D. 2003
Course Schedule 1. Formulations (18 pages) 2. Optimality, Relaxation, and Bounds (10 pages) 3. Well-solved Problems (13 pages) 4. Matching and Assigments (10 pages) 5. Dynamic Programming (11 pages) 6. Complexity and Problem Reduction (8 pages) 7. Branch and Bound (17 pages) 8. Cutting Plane Algorithms (21 pages) 9. Strong Valid Inequalities (22 pages) 10. Lagrangian Duality (14 pages) 11. Column Generation Algorithms (16 pages). Heuristic Algorithms (15 pages) 13. From Theory to Solutions (20 pages) Optimization Group 1
Exercise 2.8.1 We are asked to find a maximum cardinality matching in the graph left below. Solution: The edges colored red in the right figure form a matching of cardinality 5. The green nodes form a covering by nodes of cardinality 5. Since the cardinalities are the same, the matching has maximal cardinality. Optimization Group 2
Exercise 2.8.2 Let G = (V, E) be a graph. A stable set in G is a set of nodes S V such that there are no edges between any two nodes in S. A clique in G is a set of nodes C V such that there is an edge between any two nodes in C. Show that the problem of finding a maximal cardinality stable set is dual to the problem of finding a minimum cover by cliques of the edges. Use this observation to find bounds on the maximum size of a stable set in the graph shown below. Solution: Let C = {C 1, C 2,..., C k } be a set of 5 6 7 10 cliques such that all edges of G belong to one the cliques 1 4 8 9 C i. Then, if S is a stable set, none of the cliques contains more than 1 point of S. Hence, S k. Thus every 2 3 15 14 13 11 covering of the edges by cliques provides an upper bound for the size of a stable set in G. In the above graph, the largest cliques have size 3; there are 4 of such cliques and they contain together 10 edges. These 10 edges are colored red. 9 other edges are covered by these 4 cliques, these edges are colored magenta. The remaining 4 edges are covered by 2 cliques if size 2 (with the green edges). So we have found a covering by 6 cliques of the edges. The set S = {2,4,8,9,11,15} is a stable set of size 6. The bound is tight! Optimization Group 3
Exercise 2.8.3 Find primal and dual bounds for the integer knapsack problem: z = max 42x 1 +26x 2 +35x 3 +71x 4 +53x 5 14x 1 +10x 2 +x 3 +25x 4 +20x 5 69 x {0,1} 5 Solution: We obtain an dual (i.e., upper) bound by solving the linear relaxation. The solution of the linear relaxation takes x 1 = 69 14 and x i = 0 for i = 2,...,,5, because the quotient c i a is maximal for i = 1. This yields the upper bound 42 69 i 14 = 207. A primal (i.e, lower) bound is provided by any feasible solution. One may easily verify that x = (4, 0, 1, 0, 0) is feasible, which gives the lower bound 42.4 + 35 = 203. Optimization Group 4
Consider the 0-1 IP problem (P 1 ) max ct x : n j=1 and the 0-1 equality knapsack problem (P 2 ) max ct x : n j=1 Exercise 2.8.4 a ij x j = b i, i = 1,..., m, x {0,1} n n i=1 u i a ij x j = where u R m. Show that (P 2 ) is a relaxation of (P 1 ). n i=1 u i b i, x {0,1} n,, Solution: Suppose that x is feasible for (P 1 ). Then we have n j=1 a ij x j = b i for i = 1,..., m. Hence, for each u i R we have u i nj=1 a ij x j = u i b i for i = 1,..., m. Taking the sum over all i, we obtain n j=1 ( ni=1 u i a ij ) xj = n i=1 u i b i, showing that x is feasible for (P 2 ). Since (P 1 ) and (P 2 ) have the same objective function, this proves that (P 2 ) is a relaxation of (P 1 ). Optimization Group 5
Exercise 2.8.5 Consider the equality knapsack problem { (P 1 ) max c T x : 7 4 x 1 2 3 x 2 + 5 2 x 3 5 x 4 + 19 6 x 5 = 8 } 3, x Z5 +. Show that (P 2 ) max { is a relaxation of (P 1 ) and that { (P 3 ) max is a relaxation of (P 2 ). c T x : 3 4 x 1 + 1 3 x 2 + 1 2 x 3 + 7 x 4 + 1 6 x 5 = 2 } 3 + ω, x Z5, ω Z + c T x : 3 4 x 1 + 1 3 x 2 + 1 2 x 3 + 7 x 4 + 1 6 x 5 2 3, x R5 Solution: Suppose that x is feasible for (P 1 ). Since x is integral we then have 7 x 1 + 2 5 x 2 + x 3 + 5 19 8 x 4 + x 5. 4 3 2 6 3 This gives x 1 x 2 + 2x 3 x 4 + 3x 5 2, or, equivalently, x 1 x 2 + 2x 3 x 4 + 3x 5 = 2 ω, ω Z +. Subtracting this from the constraint in (P 1 ) it follows that x satisfies the constraint in (P 2 ). So (P 2 ) is a relaxation of (P 1 ). On the other hand, if x is feasible for (P 2 ) it is obvious that x satisfies the constraint in (P 3 ). So (P 3 ) is a relaxation of (P 2 ). }.. Optimization Group 6
Exercise 2.8.6 Consider a digraph G = (V, A) with arc lengths c e 0 for e A. Taking two distinct nodes s, t in V, consider the problem of finding a shortest path from s to t. Show that z = max { π t : π j π i c ij for e = (i, j) A, π R V, π s = 0 } is a strong dual problem. Solution: If x R A is the incidence vector of a path P = s = v 0 v 1... v k = t from s to t, and π is feasible for the above problem, we may write l(p) = x ij c ij = c ij ( ) πj π i = πt π s = π t, (i,j) A (i,j) P (i,j) P proving that both problems form a weakly dual pair. As a consequence we have l(p) z. On the other hand, for each v V, let p v denote the length of a shortest path from s to v. Obviously, p s = 0, and p j p i + c ij for each arc (i, j) A. Hence p is feasible for the above problem, whence p t z. Thus we have p t z l(p) for any s-t path P. If P is a shortest s-t path then p t = l(p) and hence z = l(p), proving that both problems form a strongly dual pair. Optimization Group 7
Exercise 2.8.7 Apply a greedy heuristic to the instance of the UFL problem with m = 6 clients and n = 4 depots, and costs as shown below: (c ij ) = 6 2 3 4 1 9 4 11 15 2 6 3 9 11 4 8 7 23 2 9 4 3 1 5 and (f j ) = (21, 16, 11, 24). Solution: Let N = {1,2,3,4} denote the set of depots. Let us assign each client to each cheapest depot. Then we need to open depot 1 to serve client 2 (cost 1+21 = 22), depot 2 to serve clients 1 and 3 (cost 2 + 2 + 16 = 20), and depot 3 to serve clients 4, 5 and 6 (cost 4 + 2 + 1 + 11 = 18). Total cost 22 + 20 + 18 = 60. We proceed by applying simple interchanges. If we close depot 1, client 2 is most cheaply served by the already open depot 3, which decreases the total cost to 60 22 + 4 = 42. Closing also depot 2, clients 1 and 3 can also be served by depot 3: the serving costs increase with 1 + 4 = 5, but the facility costs decrease with 16, yielding the total cost 42 16 + 5 = 31. This is our greedy solution. Optimization Group 8
Exercise 2.8.8 Define greedy and local search heuristics for the maximum cardinality stable set problem. Solution: Starting with S =, we add to S a node v with smallest degree. Then we remove from G node v and all of its neighbors (and the edges connected to these nodes). In the remaining graph we look for a node with smallest degree, add this node to S, remove this node and all of it neighbors, and so on until the remaining graph is empty. By way of example we apply this heuristic to the graph of Exercise 2.8.2: 5 6 7 10 5 6 7 10 5 6 7 10 1 4 8 9 8 9 8 9 2 3 15 14 13 11 3 15 14 13 11 3 15 S = {1} S = {1,14} S = {1,11,14} 5 6 5 11 8 9 8 3 15 3 15 3 15 S = {1,9,11,14} S = {1,8,9,11,14} S = {1,3,8,9,11,14} In this example the greedy heuristic yields a stable set of maximum size! A local search heuristic is the following: Given a stable set that cannot be extended, remove one of its elements and find out if the resulting stable set can be extended to a stable set by adding two nonadjacent elements not yet in this set. Optimization Group 9
Exercise 2.8.9 Formulate the maximum cardinality matching problem of Exercise 2.8.1 as an integer problem and solve its linear programming relaxation. Find a maximum weight matching for the weights shown. Check that the linear progamming relaxation has an integral solution. 1 9 4 2 6 7 1 2 Solution: We use decision variables x ij, 1 i, j 6. An IP formulation is n n max ct x : x ij 1, x ij 1, x ij {0,1} i=1 j=1 3 4 5 6 3 1 2 7 6 3 5 3 4 5 6 For the maximum cardinality problem we take c ij = 1 for all arcs, and for the maximum weight matching the indicated weights. The model is solved by using the LO solver MOSEK. The result is as follows: Edge i j : 1 1 2 2 2 3 3 4 5 6 6 3 5 2 4 5 1 6 3 5 5 6 Cardinality matching 1 0 1 0 0 1 0 0 1 0 1 Weighed matching 1 0 1 0 0 1 0 0 1 0 1 It turns out that the solution of the linear relaxation is the same in both cases, and also that it is integral. Optimization Group 10