1 The Knapsack Problem

Transcription

1 Comp 260: Advanced Algorithms Prof. Lenore Cowen Tufts University, Spring 2018 Scribe: Tom Magerlein 1 Lecture 4: The Knapsack Problem 1 The Knapsack Problem Suppose we are trying to burgle someone s house. In this house, we find a collection of objects S = a 1, a 2,..., a n 1, a n s 1, s 2,..., s n 1, s n p 1, p 2,..., p n 1, p n the objects names their sizes their profits We have a knapsack with capacity B, into which we would like to put these objects. Our goal is to find a subset of these objects whose total size is no larger than B, and whose total profit is maximized. We can view this in terms of indicator variables, where { 1 if a i is chosen x i = 0 otherwise We then need to find a setting of these x i s to maximize x i p i subject to the constraint that x i s i B. This would be a very simple linear program, but we have an integer constraint, since no fence will pay you one third of the price if you bring them one third of a piccolo. Thus, the knapsack problem is an integer program that is NP-Hard. We would like to find a polynomial time approximation to the optimal solution, so that we can leave the house we are burgling before the residents get home. 1 Based on previous version scribed by Kristen McNeely-Shaw 1

2 2 Running example We will refer to this example throughout the remainder of the notes. Object A B C D E Size Profit Knapsack size: B 3 A First Guess at Approximation For each i, we consider the profit-to-size ratio p i, and reassign the objects s i indices such that p 1 p 2 p n s 1 s 2 s n Referring back to the running example, this yields Object A B C D E Profit-Size Ratio 3/7 1 1/3 1/3 2 Order a 3 a 2 a 4 a 5 a 1 Intuitively, this organizes the objects by profit density, or profit per unit size. Algorithm 1 (Greedy): pick the first k objects greedily in this order until the next object, a k+1, will not fit in the knapsack. We can construct a simple example to show that this will not always choose the optimal collection of objects: Chosen by Algorithm 1 ((100 B) 1) B More profitable overall 2

3 Algorithm 1 chooses the first object, because it has a slightly higher profitto-size ratio, but choosing it leaves much of the knapsack empty. Note that by setting the profit appropriately, we can generalize this example to make the approximation ratio arbitrarily bad. 4 Improving the Algorithm We can modify the greedy algorithm to compensate for this issue: Algorithm 2 (Smart Greedy): pick the more profitable of {a 1,..., a k } and {a k+1 }. This algorithm is still not optimal. Consider the example below: 5/50 5/51 9/100 Chosen by Algorithm 1 Chosen by Algorithm 2 Most profitable overall However, we are now going to be able to prove a bound on the approximation ratio. Claim 4.1. Algorithm 2 always gives a profit at least 0.5 OP T, i.e. if the profit of the optimal solution is P, then the profit of the solution found by Algorithm 2 is at least P 2. If we fill a knapsack to capacity, the most profit we can get is in order of profit density: {a 1 a k } a k+1 Figure 1: A knapsack filled in order of profit ratio. Thus if we increased the size of the knapsack just enough to also allow space for the next most profit-dense object, this would be an optimal profit for the 3

4 slightly bigger knapsack! So it s certainly an upper bound on the profit of the original knapsack, and thus: Lemma 4.2. The optimal profit P p 1 + p p k + p k+1. From this lemma, it follows that ( P 2 max p k+1, ) k p i i=1 5 How close can we get to OPT? We now show that no algorithm gets within an additive factor of OPT unless P = NP. In the next section, we then show that we can get arbitrarily close to optimal by a multiplicative factor. Theorem 5.1. If P N P, no polynomial time approximation algorithm can solve knapsack with value P k for any fixed constant k. Proof. By contradiction. Assume there exists a polynomial time algorithm A with performance guarantee k (an integer) for all instances of knapsack. That is, for any knapsack instance I, A(I) = P k. We show that A can be used to construct a solution to knapsack with value P in polynomial time. Suppose we have an instance of knapsack: I = { a i, p i, s i } for i = 1... n Let I = { a i, p i, s i } where a i = a i, p 1 = p i (k + 1) and s i = s i. Note: a feasible solution for I corresponds to a feasible solution for I, and the value of a solution for I is exactly (k+1) times the value of the corresponding solution for I. 4

5 Let M = A(I). If we run A on I, the solution satisfies the following: Since A(I ) P (I ) k M (k + 1) P (I) (k + 1) k M P (I) k k + 1 k k + 1 < 1 and both M and P (I) must be integers, this gives that M P (I) 0 This in turn implies that M is an optimal solution for I, and thus that A is a polynomial time algorithm to determine an exact solution to knapsack, which we above demonstrated was impossible. Thus, no such algorithm A exists. 6 Approximation Schemes Definition 6.1. Let π be an optimization problem with objective function f π and optimal solution S. We say that A is an approximation scheme for π if on input (I, ɛ) where I is an instance of π and ɛ > 0 is a fixed error parameter, it outputs a solution S such that f π (I, S) (1 + ɛ) S if π is a minimization problem f π (I, S) (1 ɛ) S if π is a maximization problem Definition 6.2. A is a Polynomial Time Approximation Scheme (PTAS) if it is an approximation scheme where for each fixed ɛ > 0, its running time is polynomial in the size of instance I. Note that an PTAS is polynomial time in the size of I its time complexity can depend arbitrarily badly on ɛ, for example n /ɛ or n 3 21/ɛ 5

6 Definition 6.3. A is an Fully Polynomial Time Approximation Scheme (FPTAS) if it is a PTAS, and its running time is also polynomial in 1/ɛ. Remark: If P NP, an FPTAS is the best you can do for an NP-Hard problem. 7 An FPTAS for Knapsack using Dynamic Programming We now show that knapsack has an FPTAS, meaning that, at least in the approximate sense, it is an easy NP-hard problem. Recall our running example Objects A B C D E Sizes Profits Knapsack size: B Let P max be the profit of the most profitable object i.e. P max = max i S (p i ). Trivially, n P max P. Use the original arbitrary ordering. For each count i = 1,..., n and profit p = 1,..., n P max, let S i,p denote a subset of objects a 1,..., a i whose total profit is exactly p, and whose total space is minimized. Let A(i, p) denote the space that set S i,p fills if it exists, or if no such set exists. It is clear that P = max(p A(n, p) B) We now show how to compute A(i, p) for all i = 1,..., n and p = 1,..., n P max in time O(n 2 P max ) using dynamic programming. Note: This is not a contradiction to the NP-hardness of knapsack, because P max isn t necessarily bounded by a polynomial in n (although it would be if we required profits to be represented in unary). 6

7 All of the values of A(i, p) are computed as follows: A(1, p) is known for every value of p, so begin by initializing the table so that A(1, p 1 ) = s 1 and A(1, p i 1 ) =. Count i Profit p Further cells in the table can be filled in using the recurrence relation A(i + 1, p) = min{a(i, p), A(i, p p i+1 ) + s i+1 } and treating A(i, 0) = 0 for all i. This recurrence relation comes from the fact that, given the lowest-space ways to attain each value of profit using the first k items, there are two possibilities for the minimum-space way to attain a particular profit when the next item is added. First, it may be that the new item does not allow a lower-space way of reaching the desired profit; in this case, the new item is not taken, so the optimal collection of items is the same as in the previous row of the table. Alternatively, the new item may be taken, along with the lowest-space collection of additional items needed to reach the appropriate profit; we already know what this collection is since we have filled in the entire previous line of the table. The recurrence relation then just takes whichever of these two uses the least total space. Evaluating the relation is constant time in each cell, and thus it costs O(n 2 P max ) to fill the table. Count i Profit p

8 8 What to Do When Profits are Too Large When P max is bounded by a polynomial in the size of the input, then the above is a polynomial time algorithm for knapsack. When it is not, we now present a (1 ɛ) approximation algorithm, by rounding the profit values. Algorithm 3 (FPTAS): 1. Given an ɛ > 0, let k = ɛ P max. n 2. For each a i, define p pi i =. k 3. Let I = ( a i, s i, p i ) where a i = a i, s i = s i, and p i as above. 4. Using the dynamic programming method from the previous section, find the most profitable set S. 5. Output max(s max, S ), where S max is the smallest object of profit P max if S max B, and otherwise is 0. Lemma 8.1. Let A denote the set output by Algorithm 3. Then Profit(A) (1 ɛ) P Proof. Let O denote the set with profit P. For any a i, k p i can be smaller than p i, but not by more than k. Thus, Profit(O) k Profit (O) n k Under the Profit scheme, Profit (S ) is OPTIMAL, i.e. Profit (S ) Profit (Y ) for all Y, and in particular Profit (S ) Profit (O). Thus: Profit(S ) k Profit (S ) k Profit (O) Profit(O) n k = P ɛ P max 8

9 Since the algorithm also considers the most profitable element Profit(A) Profit(P max ) Profit(A) Profit(S ) P ɛ Profit(A)s Theorem 8.2. Algorithm 3 is an FPTAS for knapsack. Proof. By lemma, the solution P is within (1 ɛ) of OPT. ( ) The running time is O n 2 Pmax ( n ) = O n 2, which is polynomial in k ɛ n and 1/ɛ. 9 Strong NP-Hardness Many NP-hard problems are more difficult to approximate than knapsack, and do not admit an FPTAS: Definition 9.1. A problem π is strongly NP-hard if every problem in NP can be polynomial-time reduced to π so that all numbers in the reduced instance can be written in unary. Theorem 9.2. Let p be a polynomial and π be a strongly NP-hard minimization problem such that the objective function f π is integer valued, and on any instance I, the size of OP T (I) < p( I ). Then π does not admit an FPTAS, assuming P NP. We don t prove this theorem here, although the proof is not that difficult. Note that this implies that knapsack is not strongly NP-hard, as we above constructed an FPTAS to approximate it. 9