COSC 341: Lecture 25 Coping with NP-hardness (2)

Transcription

1 1 Introduction Figure 1: Famous cartoon by Garey and Johnson, 1979 We have seen the definition of a constant factor approximation algorithm. The following is something even better. 2 Approximation Schemes Definition 2.1 (approximation scheme). An algorithm A is an approximation scheme if for every ɛ > 0, A runs in polynomial time (which may depend on ɛ) and returns a solution: 1) SOL (1 + ɛ) OPT for a minimization problem 2) SOL (1 ɛ) OPT for a maximization problem That is, the approximation scheme produces a solution that is within a factor 1+ɛ of being optimal (or 1 ɛ for maximization problems). For example, A may run in time n 100/ɛ. So, there is a time-accuracy tradeoff. We can either achieve a more accurate solution by spending more running time, or speed up the algorithm by relaxing the error rate requirement. 1

2 3 Example: The knapsack problem In the knapsack problem, you are given a knapsack of size B N and a set S = {a 1,..., a n } of objects with corresponding sizes and profits s(a i ) N and p(a i ) N. The goal is to find the optimal subset of objects whose total size is bounded by B and has the maximum possible total profit. This problem is also sometimes called the 0/1 knapsack problem because each object must be either in the knapsack completely or not at all. There are other variations as well, notably the multiple knapsack problem, in which you have more than one knapsack to fill. The knapsack problem has many applications. One example is room renting. Assume we have one room for 24 hours, some guys would like to rent the room for different hours with different price (they do not care the starting time). The goal is to maximize the overall profit with the time constraint. Then we can solve it as a knapsack problem. The decision version of knapsack problem is NP-complete. It is very easy to reduce an instance of Subset Sum problem to an instance of Knapsack problem. (Subset Sum is a special case of the Knapsack problem.) The obvious greedy algorithm would sort the objects in decreasing order using the objects ratio of profit to size, or profit density, and then pick objects in that order until no more objects will fit into the knapsack. The problem with this is that we can make this algorithm perform arbitrarily bad. Greedy Algorithm 1. Sort items in non-increasing order of p(a i )/s(a i ) 2. Greedily pick items in above order. Intuitively, we want the items with the most bang for the buck. Unfortunately, this algorithm is arbitrarily bad. Consider the following input as a counterexample: 1. An item with size 1 and profit 2 2. An item with size B and profit B. 2

3 Our greedy algorithm will only pick the small item, making this a pretty bad approximation algorithm. Therefore, we make the following small adjustment to our greedy algorithm: Greedy Algorithm Redux 1. Sort items in non-increasing order of p(a i )/s(a i ) 2. Greedily add items until we hit an item a i that is too big. ( i k=1 s i > B) 3. Pick the better of {a 1, a 2,..., a i 1 } and a i. Theorem 3.1. Greedy Algorithm Redux is a 2-approximation for the knapsack problem. Proof: We employed a greedy algorithm. Therefore we can say that if our solution is suboptimal, we must have some leftover space B S at the end. Imagine for a second that our algorithm was able to take a fraction of an item. Then, by adding B S s k+1 p k+1 to our knapsack value, we would either match or exceed OP T (remember that OP T is unable to take fractional items). Therefore, either k i=1 p i 1 2 OP T or p k+1 B S s k+1 p k OP T. Definition 3.2 (PTAS). A polynomial time approximation scheme (PTAS) for problem X is an approximation scheme whose time complexity is polynomial in the input size. A smarter approach to the knapsack problem involves brute-forcing part of the solution and then using the greedy algorithm to finish up the rest. PTAS for knapsack problem 1. Choose k = 1 ɛ 1 (and obviously k n) 2. For all subsets L N with L k 3. if j L s j B then 4. apply Greedy Algorithm to remaining problem 5. choose best solution 3

4 In particular, consider all O(kn k ) possible subsets of objects that have up to k objects, where k is some fixed constant. Then for each subset, use the greedy algorithm to fill up the rest of the knapsack in O(n) time. Pick the most profitable subset A. The total running time of this algorithm is thus O(kn k+1 ). Let P (A) denote the profit achieved by this algorithm and P (O) be the profit achieved by the optimal set O, then the resulting approximation P (A) achieves P (O) P (A)(1 + 1 k ) (1) So, we have a (1 ɛ) approximation where 1/ɛ = k + 1. The resulting running time is O(kn k+1 ), so the approximation scheme is polynomial in n but not 1/ɛ. Definition 3.3 (FPTAS). A fully polynomial time approximation scheme (FPTAS) for problem X is an approximation scheme whose time complexity is polynomial in the input size and also polynomial in 1/ɛ INTRODUCTION 5 Knapsack actually has something but a time complexitystronger proportional to known I 8 /ε 3 would be asfine. a With fully respect polynomial to time approximation scheme or FPTAS. derive(the for an NP-hard running problem. time Figure 0.1isillustrates O(nthe 2 /ɛ) relationships by dynamic between programming.) In a worst case approximation, an FPTAS is the strongest possibleresultthatwecan the classes NP, APX, P, the class of problems that are pseudo-polynomially very technical sense, ansolvable, FPTAS and the isclasses theof best problemsone that have can a PTAS hope and FPTAS. for for an NP-hard optimization problem, assuming P NP. NP APX PTAS FPTAS Pseudo-Polynomial Time P Figure 0.1: Containment relations between some of the complexity classes discussed in this chapter. Figure 2: Figure from gwoegi/papers/ptas.pdf Alittlebitofhistory. The first paper with a polynomial time approximation algorithm for an NP-hard problem is probably 4 the paper [26] by Grahamfrom It studies simple heuristics for scheduling on identical parallel machines. In 1969, Graham [27] extended his approach to a PTAS. However, atthattime these were isolated results. The concept of an approximationalgorithmwas formalized in the beginning of the 1970s by Garey, Graham & Ullman [21]. The paper [45] by Johnson may be regarded as the real starting point of the field; it raises the right questions on the approximability ofawiderangeof optimization problems. In the mid-1970s, a number of PTAS s was developed in the work of Horowitz & Sahni [42, 43], Sahni [69, 70], and Ibarra & Kim [44].

5 4 Tutorial problems 1. Makespan Scheduling problem: We are given m machines for scheduling, indexed by the set M = {1,..., m}. There are furthermore given n jobs, indexed by the set J = {1,..., n}, where job j takes p i,j units of time if scheduled on machine i. Let J i be the set of jobs scheduled on machine i. Then l i = j J i p i,j is the load of machine i. The maximum load l max = c max = max i M l i is called the makespan of the schedule. The problem is NP-hard, even if there are only two identical machines. List Scheduling algorithm works as follows: Determine any ordering of the job set J, stored in a list L. Starting with all machines empty, determine the machine i with the currently least load and schedule the respective next job j in L on i. The load of i before the assignment of j is called the starting time s j of job j and the load of i after the assignment is called the completion time c j of job j. Prove that List Scheduling algorithm is a 2-approximation for Makespan Scheduling on identical machines. 2. This is the end of the second part of this paper. Think about some questions: Which problems are efficiently solvable and which problems are efficiently verifiable? 5

6 How can we prove that a problem B is at least as hard as some problem A? How do we show the evidence that no efficient algorithm exists for hard problems? What are the hard problems you have known? Is there some way to show that you find near-optimal solutions in polynomial time for hard problems? If you don t know how to answer the above questions. Please check if you have understood the following knowledge: P, NP, NP-complete, NP-hard, Polynomialtime reduction, SAT problem, Cook s Theorem, Proving NP-completeness, Graphcoloring Problem, Vertex Cover problem, Subset-sum problem, Optimisation problems vs. Decision problems, Approximation factor. Please feel free to ask me if you have any questions. ( yawen@cs.otago.ac.nz) 6