Find: a multiset M { 1,..., n } so that. i M w i W and. i M v i is maximized. Find: a set S { 1,..., n } so that. i S w i W and. i S v i is maximized.

Transcription

1 Knapsack gain Slides for IS 675 PV hapter 6: ynamic Programming, Part 2 Jim Royer EES October 28, 2009 The Knapsack Problem (KP) knapsack with weight capacity W. Items 1,..., n where item i has weight w i and value v i.... with repetition Find: a multiset M 1,..., n } so that... without repetition Find: a set S 1,..., n } so that i S w i W and i S v i is maximized. Image from: IS 675 (EES) ynamic Programming, part 2 October 28, / 21 IS 675 (EES) ynamic Programming, part 2 October 28, / 21 Knapsack with repetition Knapsack without repetition, 1 Knapsack with repetition knapsack with capacity W. Items 1,..., n Item i has weight w i & value v i. Find: a multiset M 1,..., n } K(w) = max. value gained from a knapsack with cap. w = max i:w i w K(w w i) + v i (Why?) array K[0..W] K[0] 0 for w 1 to W do K[w] 0 for i 1 to n do if w i w then K[w] max(k[w], K[w w i ] + v i ) This runs in Θ(n W) time. Since we usually measure the size of W as W = the number of bits in the binary rep. of W. Hence, Θ(n W) = Θ(n 2 W ). So this is only useful for small values of W. Knapsack without repetition knapsack with capacity W. Items 1,..., n Item i has weight w i & value v i. Find: a set M 1,..., n } Problem K[w w n ] is not useful since it does not tell you whether item n was used in an optimal solution. Therefore, we refine things to: K[w, j] the best value obtainable with = capacity w using items from 1,..., j K[w, j 1], if w j > w; = max(k[w, j 1], K[w w j, j 1] + v j ), otherwise. (Why?) IS 675 (EES) ynamic Programming, part 2 October 28, / 21 IS 675 (EES) ynamic Programming, part 2 October 28, / 21

2 Knapsack without repetition, 2 Knapsack w/o repetition knapsack with capacity W. Items 1,..., n Item i has weight w i & value v i. Find: a set M 1,..., n } Our recursive relation is: K[w, j] the best value obtainable with = capacity w using items from 1,..., j K[w, j 1], if w j > w; = max(k[w, j 1], K[w w j, j 1] + v j ), otherwise. array K[0..W, 0..n] // This is also Θ(n W) time. for w 0 to W do K[w, 0] 0 // Hence only useful K[0, j] 0 // when W is small. for w 1 to W do if w i > w then K[w, j] K[w, j 1] else K[w, j] max(k[w, j 1], K[w w i, j 1] + v i ) IS 675 (EES) ynamic Programming, part 2 October 28, / 21 hain matrix multiplication, 1 Recall Multiplying an d 0 d 1 matrix by an d 1 d 2 matrix results in a d 0 d 2 matrix and takes (d 0 d 1 d 2 )-many scalar multiplies. The hain Matrix Multiplication Problem (MMP) d 0,..., d n N + and matrices 1,..., n where dim( i ) = d i 1 d i. Find: The cheapest way to order the multiplications. Example: Suppose that is an matrix. is an 20 1 matrix. is an 1 10 matrix. is an matrix. Then: Parenthesization ost computation ost ( ( )) ,000 (( ) ) ,200 ( ) ( ) ,000 ( ( )) ,200 (( ) ) ,000 IS 675 (EES) ynamic Programming, part 2 October 28, / 21 hain matrix multiplication, 2 hain matrix multiplication, 3 Parenthesization ost computation ost ( ( )) ,000 (( ) ) ,200 ( ) ( ) ,000 ( ( )) ,200 (( ) ) ,000 The hain Matrix Multiplication Problem (MMP) d 0,..., d n N + and matrices 1,..., n where dim( i ) = d i 1 d i. Find: The cheapest way to order the multiplications. The (i, k) subproblem, 1 i k n Find: (i, k) = the min. cost of the i k. (i, i) = 0. (i, k) = min j=i,...,k 1 ((i, j) + (j + 1, k) + d i 1 d j d k ), where i < k. (1, n) is the minimal cost of the MMP for 1,..., n. Question: Why does optimal substructure hold? IS 675 (EES) ynamic Programming, part 2 October 28, / 21 IS 675 (EES) ynamic Programming, part 2 October 28, / 21

3 hain matrix multiplication, 4 The (i, k) subproblem, 1 i k n Find: (i, k) = the min. cost of the i k. (i, i) = 0. (i, k) = min j=i,...,k 1 (1, n) is the minimal cost of the MMP for 1,..., n. ( (i, j) + (j + 1, k) + di 1 d j d k ), where i < k. for i 1 to n do [i, i] 0 for s 1 to n 1 do // ompute [1, 1 + s], [2, 2 + s],..., [n s, n] for i 1 to n s do k i + s; [i, k] + for j i to k 1 do [i, k] min([i, k], [i, j] + [j + 1, k] + d i 1 d j d k return [1, n] Run Time: Θ(n 3 ). Question: How do you reconstruct the order of multiplication from the [, ] table? IS 675 (EES) ynamic Programming, part 2 October 28, / 21 Shortest paths: ll-pairs, 1 Vaughn Truth or onseq. Tucumcari lbuquerque lamogordo Taos Socorro Silver ity Santa Rosa Santa Fe Ruidoso Roswell Reserve Red River Raton Los lamos Portales Lordsburg Las Vegas IS 675 (EES) ynamic Programming, part 2 October 28, / 21 Las ruces Hobbs Farmington Gallup eming lovis layton hama arlsbad rtesia Shortest paths: ll-pairs, 2 Shortest paths: ll-pairs, 3 ll-pairs Shortest Paths Problem (PSP) G = ( 1,..., n }, E) an undirected graph and len:e R +. onstruct: S[1..n, 1..n] so that S[i, j] = the length of a shortest G-path from i to j. ssumption: G is initially given by a matrix [1..n, 1..n] so that len(i, j), if (i, j) E; [i, j] = +, if (i, j) / E. PSP, restated: Given [1..n, 1..n], compute S[1..n, 1..n]. Question: What are good subproblems? is the approximation to S in which that paths have no vertices: ❶ ❸ ❶ ❷ ❸ llowing more and more vertices gives subproblems... vertices from 1,..., k } len(i, j), if (i, j) E; dist[i, j, 0] = [i, j] = +, if (i, j) / E. dist[i, j, n] = S[i, j] = the length of a shortest G-path from i to j IS 675 (EES) ynamic Programming, part 2 October 28, / 21 IS 675 (EES) ynamic Programming, part 2 October 28, / 21

4 length of the shortest path from i to j in which only nodes 1, 2,..., k} can be used as interme diates. Initially, dist(i, j, 0) is the length of the direct edge between i and j, if it exists, and is Shortest paths: ll-pairs, 3 otherwise. a,.h. Papadimitriou, and U.V. Vazirani 187 Shortest paths: ll-pairs, 4 What happens when we expand the set to include an extra node k? We mus reexamine all pairs Question: i, j and check How do whether we go using from dist[, k as, an k 1] to dist[,, k], point where: gives us a shorter shortest paths path from i to j. ut this is easy: a shortest path from i to j that uses k along with possibly want to find the shortest path not just between s and t but between other all lower-numbered pairs nodes goes through k just once (why? because we assume One approach would be to execute our general shortest-path algorithm that there from are no negative cycles). nd we have already vertices from calculated 1,... the, k } length of the shortes.1 (since there may be the negative length of edges) a shortest V times, G-path once from for i to each j using starting path node. from The i to k and from k to j using only lower-numbered vertices: ng time would then be O( V 2 E ). vertices We ll now from see 1, a.. better., k } alternative, the O( V 3 ) k ogramming-based Floyd-Warshall len(i, algorithm. j), if (i, j) E; dist(i, k, k 1) is a good dist[i, subproblem j, 0] = [i, for j] = computing distances between all pairs of vertices in a +, if (i, j) / E. i dist(k, j, k 1) ply solving the problem for more and more pairs or starting points is unhelpful, leads right dist[i, back j, n] to= the S[i, O( V j] = 2 the E ) length algorithm. of a shortest G-path from i to j a comes to mind: the shortest path u w 1 w l v between u and v dist(i, j, k 1) j number of nodes possibly none. Suppose we disallow ether. Then Question: we can How solve do we all-pairs go from shortest dist[,, k paths 1] to at dist[, once:, k]? the shortest Thus, pathusing from k gives us a shorter path from i to j if and only if mply the direct edge (u, v), if it exists. What if we now gradually expand the set dist[i, j, k 1] = dist[i, j, k] means: there is a shortest path from i to j using dist(i, k, k vertices 1) + dist(k, from j, 1, k..., k 1) } that < dist(i, does not j, use k k. 1), ble nodes? We can do this one node at a time, updating the shortest s at each stage. Eventually this set grows to all of V, at which pointin allwhich vertices case dist(i, j, k) should be updated accordingly. to be on all paths, and we have found the true shortest paths between vertices Here isofthe Floyd-Warshall algorithm and as you can see, it takes O( V 3 ) time. IS 675 (EES) ynamic Programming, part 2 October 28, / 21 IS 675 (EES) ynamic Programming, part 2 October 28, / 21 ncretely, number the vertices in V as 1, 2,..., n}, and let dist(i, j, k) denote forthe i = 1 to n: e shortest path from i to j in which only nodes 1, 2,..., k} can be used as intermeially, for j = 1 to n: dist(i, j, 0) is the length of the direct edge between i and j, if it exists, and is dist(i, j, 0) = e. Shortest paths: ll-pairs, 5 appens when we expand the set to include an extra node k? We must Shortest paths: ll-pairs, 6 all pairs Question: i, j and check How do whether we go using from dist[, k as, an k 1] to dist[,, k], point where: gives us a shorter i to j. ut this is easy: a shortest path from i to j that uses k along with possibly Question: How do we go from dist[,, k 1] to dist[,, k]? -numbered nodes goes through k just once (why? because we assume are no negative dist[i, cycles). j, k] = nd we have already vertices from calculated 1,... the, k } length of the shortest vertices from 1,..., k } to k and from k to j using only lower-numbered vertices: = min(dist[i, j, k 1], dist[i, k, k 1] + dist[k, j, k 1]). dist(i, k, k 1) k for i 1 to n do // Initialization i dist(k, j, k 1) dist[i, j, 0] [i, j] for i 1 to n do // Main iteration dist(i, j, k 1) j for k 1 to n do g k gives us a shorter path from i to j if and only if dist[i, j, k] min(dist[i, j, k 1], dist[i, j, k 1] = dist[i, j, k] means: shortest paths from i to j using dist[i, k, k 1] + dist[k, j, k 1]) for i dist(i, k, k vertices 1) + dist(k, from j, 1, k..., k 1) } must < dist(i, use k. j, k 1), 1 to n do // Output dist[i, k, k 1] + dist[k, j, k 1]. (Why?) S[i, j] dist[i, j, n] se dist(i, j, k) should be updated accordingly. the Floyd-Warshall algorithm and as you can see, it takes O( V 3 return S ) time. i = 1 to n: IS 675 (EES) ynamic Programming, part 2 October 28, / 21 IS 675 (EES) ynamic Programming, part 2 October 28, / 21

5 Shortest paths: ll-pairs, 7 The traveling salesman problem for i 1 to n do // Initialization dist[i, j, 0] [i, j] for i 1 to n do // Main iteration for k 1 to n do dist[i, j, k] min(dist[i, j, k 1], dist[i, k, k 1] + dist[k, j, k 1]) for i 1 to n do // Output S[i, j] dist[i, j, n] return S Time complexity: Θ( V 3 ). (Why?) Space complexity: lso Θ( V 3 ), but this is easy to improve to Θ( V 2 ). The traveling salesman problem G a complete graph on verts 1,..., n and d(i, j) = (the distance of i to j) < Find: minimal cost of a complete tour of G. Subproblems For S 1,..., n } with 1 S, and j S: the minimal cost of a path from [S, j] = 1 to j using just nodes in S. [S, 1] =, when S > 1. [S, j] = min [S j }, i] + dist(i, j). i (S j } Note: This is a set up for hapter There are at most 2 n n-many subproblems, and each one takes O(n) time to solve.!!! This takes O(n 2 2 n ) time!!! 2 IS 675 (EES) ynamic Programming, part 2 October 28, / 21 IS 675 (EES) ynamic Programming, part 2 October 28, / 21 Independent sets in trees, 1 Independent sets in trees, 2 efinition Suppose G = (V, E) is an undirected graph. (a) u and v V are independent when (u, v) / E. (b) U V is an independent set when every pair of elements from U are independent. The Independent Set Problem (ISP) G an undirected graph. Find: max-sized independent set for G. The Independent Set Problem for Trees T a tree. Find: max-sized independent set for T. Example In the graph below: 1, 4, 5 } is not an independent set. 1, 5 } and 1, 6 } are independent sets 2, 3, 6 } and 1, 4, 6 } are max-sized independent sets efinition Suppose G = (V, E) is an undirected graph. (a) u and v V are independent when (u, v) / E. (b) U V is an independent set when every pair of elements from U are independent. The Indep. Set Problem for Trees T a tree. Find: max-sized independent set for T. Strategy Pick some vertex of T as the root, r. Now each vertex of T is the root of a subtree. For each v in T, define I(v) = the size of a largest independent set in v s subtree. I(v) = 1 when v is a leaf. I(r) = the size of a largest indep. set in T. So, what is the recursion? IS 675 (EES) ynamic Programming, part 2 October 28, / 21 IS 675 (EES) ynamic Programming, part 2 October 28, / 21

6 Independent sets in trees, 3 Strategy Pick some vertex of T as the root, r. For each v in T: I(v) = the size of a largest indep. set in v s subtree. I(v) = 1 when v is a leaf and I(r) = the size of a largest indep. set in T. Wwhat is the recursion for I(u)? ase: u is in some maximal sized indep. set for u s subtree Then I(u) = 1 + I(v) : v is a grandchild of u }. ase: u is in no maximal sized indep. set for u s subtree Then I(u) = I(v) : v is a child of u }. Independent sets in trees, 4 Strategy Pick some vertex of T as the root, r. For each u in T compute: I(u) = the size of a largest indep. set in u s subtree = max ( 1 + v is a grandchild of u I(v), v is a child of u I(v) ) This can be done in Θ( V ) time. For the general graph case, O(2 V ) is the best known time. Note: This is another set up for hapter 8. I(u) = max ( 1 + v is a grandchild of u I(v), v is a child of u I(v) ) IS 675 (EES) ynamic Programming, part 2 October 28, / 21 IS 675 (EES) ynamic Programming, part 2 October 28, / 21