Discrete (and Continuous) Optimization WI4 131

Transcription

1 Discrete (and Continuous) Optimization WI4 131 Kees Roos Technische Universiteit Delft Faculteit Electrotechniek, Wiskunde en Informatica Afdeling Informatie, Systemen en Algoritmiek URL: roos November December, A.D. 2004

2 Course Schedule 1. Formulations (18 pages) 2. Optimality, Relaxation, and Bounds (10 pages) 3. Well-solved Problems (13 pages) 4. Matching and Assigments (10 pages) 5. Dynamic Programming (11 pages) 6. Complexity and Problem Reduction (8 pages) 7. Branch and Bound (17 pages) 8. Cutting Plane Algorithms (21 pages) 9. Strong Valid Inequalities (22 pages) 10. Lagrangian Duality (14 pages) 11. Column Generation Algorithms (16 pages) 12. Heuristic Algorithms (15 pages) 13. From Theory to Solutions (20 pages) Optimization Group 1

3 Capter 5 Dynamic Programming Optimization Group 2

4 Motivation: Shortest Paths in a Network Given a directed graph D = (V, A) with nonnegative arc lengths c e for e A, and an initial node s V, the problem is to find the shortest path from s to every other node t V. s p 7 8 Observation 1 If the shortest path from s to t passes by node p, the subpaths (s, p) and (p, t) are shortest paths from s to p, and from p to t respectively. Observation 2 Let d(j) denote the length of the shortest path from s to j. Then 4 6 t d(j) = min i V (j) ( d(i) + cij ). Observation 3 If D = (V, A) is acyclic, then the nodes can be ordered such that i < j for all arcs (i, j) A (Why? N.B. Not every node can have an incoming arc!). Then the above recursion yields an O( E )-algorithm for finding the shortest paths from node 1 to all other nodes. (Apply the recursion to compute d(j) in the natural order, starting with d(1) = 0.) In a general (not acyclic) network an O( E V )-algorithm is obtained by using the recursion d k (j) = min { d k 1 (j), min i V (j) [ dk 1 (i) + c ij ] }, where d k (i) denotes the length of a shortest path form s to i containing at most k arcs. Optimization Group 3

5 Uncapacitated Lot-Sizing The problem is to find a minimum cost production plan for an n-period horizon that satisfies all the nonnegative demands d t, given the costs for production p t, storage h t and set-up f t, for t = 1,..., n. Earlier we derived the following MIP formulation: min t=1 p t x t + t=1 h t s t + s t 1 + x t = d t + s t, t=1 f t y t t = 1,..., n x t My t, t = 1,..., n s 0 = s n = 0, s R n+1, x R n, y {0,1} n. Every feasible solution corresponds to a feasible flow in the following network: 0 x 1 x 2 x 3 x 4 x 5 1 s 1 2 s 2 3 s 3 4 s 4 5 d 1 d 2 d 3 d 4 d 5 Optimization Group 4

6 Uncapacitated Lot-Sizing (cont.) 0 x 1 x 2 x 3 x 4 x 5 1 s 1 2 s 2 3 s 3 4 s 4 5 d 1 d 2 d 3 d 4 d 5 Proposition 1 (i) There exists an optimal solution with s t 1 x t = 0 for all t. (ii) There exists an optimal solution such that if x t > 0, then x t = t+k i=t d t for some k 0. Proof: The arcs used by an optimal flow form a tree. This implies (i). The second statement follows from (i). From now we use d it = t j=i d j, the total demand in periods i up to t. We can eliminate the stock variables s t from the objective function. With c t = p t + n i=t h i one has t=1 p t x t + t=1 h t s t = t=1 p t x t + t h t t=1 i=1 x i d 1t = t=1 c t x t t=1 h t d 1t, Let H(k) be the minimum cost for meeting the demands in periods 1 to k. Then we have the recursion H(k) = min 1 t k {H(t 1) + f t + c t d tk }, H(0) = 0. O(n 2 ) operations suffice to compute H(n) as well as an optimal solution. Optimization Group 5

7 Example Consider an instance of ULS with n = 4, d = (2,4,5,1), p = (3,3,3,3), h = (1,2,1,1) and f = (12,20,16,8). We then have c = (8,7,5,4), (d tk ) = , h t d 1t = t=1 Using the recursion H(k) = min 1 t k {H(t 1) + f t + c t d tk } we compute the values of H(k), starting with H(0) = 0: H(1) = H(0) + f 1 + c 1 d 11 = 28. H(2) = min[h(0) + f 1 + c 1 d 12, H(1) + f 2 + c 2 d 22 ] = min[ , ] = min[60,76] = 60. H(3) = min[h(0) + f 1 + c 1 d 13, H(1) + f 2 + c 2 d 23, H(2) + f 3 + c 3 d 33 ] = min[ , , ] = min[100,111,101] = 100. H(4) = min[h(0) + f 1 + c 1 d 14, H(1) + f 2 + c 2 d 24, H(2) + f 3 + c 3 d 34, H(2) + f 3 + c 3 d 44 ] = min[ , , , ] = min[108,118,106,112] = 106. The solution can be found by working backwards: H(4) = 106 = H(2)+f 3 +c 3 d 34, so y 3 = 1, x 3 = 6, y 4 = x 4 = 0. Furthermore, H(2) = 60 = f 1 + c 1 d 12, so y 1 = 1, x 1 = 6, y 2 = x 2 = 0. Thus we find as optimal solution: x = (6, 0, 6, 0), y = (1, 0, 1, 0), s = (4, 0, 1, 0), whose value in the original costs is = 69. Optimization Group 6

8 Example: alternative solution We can the ULS also formulate as a shortest path problem. We use nodes (0,1,..., n) and arcs (i, j) for all i < j. The cost f i+1 + c i+1 d i+1,j of arc (i, j) represent the cost of starting production in i + 1 and satisfying the demands in the periods i + 1 to j Using the data of the previous example the above network arises. In this network H(k) is equal to the length of a shortest path from node 0 tot node k, and as the graph is acyclic, the corresponding shortest path algorithm is O(m) = O(n 2 ). 90 Optimization Group 7

9 An Optimal Rooted Subtree of a Tree Given are a tree T = (V, E) and weights c v for v V. One specific node of the tree is called its root r. The problem is to find a (connected) subtree of T rooted at r with maximal positive weight. If no positive weight rooted subtree exists, the empty tree is the solution, with weight 0. For each node v( r) V its predecessor p(v) is the node immediately preceding v on the unique path form r to v; its immediate successors form the set S(v) = {w : p(w) = v}. Furthermore, the subtree T(v) is the subtree of T whose nodes are the nodes w for which the path from r to w contains v. For each node v let H(v) denote the optimal value of the optimal subtree problem for the subtree T(v) rooted at v. If the optimal subtree is empty, then H(v) = 0. Otherwise it contains v, and possibly other subtrees of T(w) for w S(v). By the principle of optimality these subtrees must themselves be optimal rooted subtrees. Thus we have the recursion: H(v) = max 0, c v + w S(v) H(w). Optimization Group 8

10 Example of an Optimal Rooted Subtree r We start with the end nodes: H(4) = H(6) = H(7) = H(11) = 0, H(9) = 5, H(10) = H(12) = H(13) = 3. Climbing one level higher we obtain: H(5) = max {0, } = 2, H(8) = max {0, } = 8. Proceeding further in this way: and finally, H(2) = max {0,2 + 2} = 4, H(3) = max {0, 8 + 8} = 0, H(1) = max {0, } = 2. Working back, we find the colored subtree being optimal, with value 2. Optimization Group 9

11 First we consider the 0-1 knapsack problem: z = max 0-1 Knapsack Problem c j x j : a j x j b, x {0,1} n Thinking of the right hand side λ as the state, with λ = 0, 1..., b, and the subset of variables x 1,..., x k as the stage, represented by k, we define the subproblems f k (λ) = max k c j x j : k The optimal value is given by f n (b). We have the recursion which can be initialized by. a j x j λ, x {0,1} k f k (λ) = max { f k 1 (λ), c k + f k 1 (λ a k ) }, f k (λ) =, k 0, λ < 0, f k (0) = 0, k 0, f 0 (λ) = 0, λ 0. The complexity of the algorithm is O(nb).. Optimization Group 10

12 Example of 0-1 Knapsack Problems Solution: λ f 0 f 1 f 2 f 3 f z = max10x 1 + 7x x x 4 2x 1 + 1x 2 + 6x 3 + 5x 4 7 x {0,1} 7. f k (λ) = max {f k 1 (λ), c k + f k 1 (λ a k )}. We first construct the table at the left, column by column, starting at the top. We find that the highest value is f 4 (7) = 34. This was obtained from f 4 (7) = c 4 + f 3 (2) = So, to obtain this value, we should have x 4 = 1 From f 3 (2) = c 1 + f 2 (2) = we deduce that x 1 = 1. The other values should be taken 0. So the optimal solution is x = (1,0,0,1). Optimization Group 11

13 Integer Knapsack Problem Now we consider the integer knapsack problem: z = max c j x j : Just as in the previous case we define the subproblems f k (λ) = max k c j x j : a j x j b, x Z n + k. a j x j λ, x Z k + The optimal value is given by f n (b). We now may use the recursion which can be initialized as before: f k (λ) = max { f k 1 (λ), c k + f k (λ a k ) }, f k (λ) =, k 0, λ < 0, f k (0) = 0, k 0, f 0 (λ) = 0, λ 0. The complexity of the algorithm is O(nb).. Optimization Group 12

14 Example of Integer Knapsack Problems z = max7x 1 + 9x 2 + 2x x 4 3x 1 + 4x 2 + 1x 3 + 7x 4 10 Solution: f 1 f 2 f 3 f 4 λ = x Z 7 +. f k (λ) = max {f k 1 (λ), c k + f k (λ a k )}, Using the recursive formula we construct the table at the left, column by column, starting at the top. We find that the highest value is f 4 (10) = 23. This was obtained from f 4 (10) = f 3 (10) = f 2 (10) = c 2 +f 2 (6) = So, to obtain this value, we should have x 2 = 1 From f 2 (6) = f 1 (6) = c 1 + f 1 (3) = 2c 1 = 14 we deduce that x 1 = 2. The other values should be taken 0. So the optimal solution is x = (2, 1, 0, 0). Optimization Group 13

15 Imrpoved Recursion for the Integer Knapsack Problem λ f 1 f 2 f 3 f Looking at the table left we see that all the important information is contained in its 4-th column, which contains the values of f n (λ). Can we write down a recursive formula for f n (λ)? The answer is yes. Again the principle of optimality works: if x is an optimal solution of h(λ) = max k c j x j : k a j x j λ, x Z k + with x j 1 then h(λ) = c j + h(λ a j ), for each j. Thus we obtain the recursion: h(λ) = max { 0, max j:a j λ. { h(λ) = cj + h(λ a j } }. The optimal value is given by h(b), and we can initialize by h(0) = 0. This also leads to an algorithm with complexity O(nb). Optimization Group 14

16 Example: Longest Path form of Integer Knapsack Problem z = max10x 1 + 7x x x 4 2x 1 + 1x 2 + 6x 3 + 5x 4 7 x Z 7 +. Construct acyclic digraph D = (V, A) with nodes 0, 1..., b, arcs (λ, λ+a j ) for λ Z +, λ b a j with weight c j for j = 1,... n, and arcs (λ, λ+1) for λ b 1 with weight 0. Then h(λ) is precisely the length of a longest path from node 0 to node λ. For the above example problem the digraph is shown below. The zero weight arcs are omitted because of dominance by the arcs with weight Optimization Group 15