Problem Complexity Classes

Transcription

1 Problem Complexity Classes P, NP, NP-Completeness and Complexity of Approximation Joshua Knowles School of Computer Science The University of Manchester COMP Week , March 20th 2015

2 In This Lecture The Classes P and NP The most difficult problems: NP-Complete and NP-hard Strongly and Weakly NP-hard An Approximation Complexity Class: APX Complexity / NP-Completeness , March 20th 2015

3 Quick Overview of Main Complexity Classes P The set of problems for which a worst-case polynomial-time algorithm is known (in short: Easy problems). NP The set of problems for which we know how to check a proposed solution in worst-case polynomial time (in short: this set might include some harder problems to find solutions for, but we can at least check the solutions quickly if we are given them). NP-complete The set of problems in NP that we can efficiently reduce any other problem in NP to. (In short: this set stands for the hardest problems in NP, but we don t actually have a proof that they are harder than P problems!) These definitions might seem a little tricky. Why can t we have something simpler?... We will hopefully see why they are like this in what follows Complexity / NP-Completeness , March 20th 2015

4 Polynomial-Time is Tractable Some notation: Π A problem π A problem instance π The number of symbols needed to describe π. Also written as n, the input size to an algorithm for solving π. Running times of algorithms are expressed as functions of the size of the input. E.g. O(n 2 ). Very few polynomial algorithms run in a complexity worse than say O(n 5 ) time. (There are no O(n 1000 ) algorithms). So, polynomial tractable. Complexity / NP-Completeness , March 20th 2015

5 The Need for Problem Complexity Classes Efficient algorithms have been designed for many problems, e.g.: integer arithmetic operations, sorting, matrix multiplication, shortest path in a graph, minimum spanning tree, and so on. These problems are polynomial-time solvable: there are known algorithms to solve each of these problems in polynomial time in the worst case. But for many other well-defined problems, often very simple to state, no efficient (polynomial) algorithm has been found. Confronted with a new problem, how do we know whether or not it will be futile to look for an efficient algorithm for it? Complexity / NP-Completeness , March 20th 2015

6 The Need for Problem Complexity Classes The bandersnatch problem (*Garey and Johnson, 1979) A fictitious scenario: Your boss has asked you, her chief algorithm designer, to design a procedure to check whether new components can be made for a bandersnatch machine that meet all the design requirements. You design an algorithm, but it keeps running too slowly. The result is that the whole production is held up. What can you do, as your boss wants a faster algorithm, but you can t think of one? Computers and Intractability, A Guide to the Theory of NP-Completeness, Freeman and Co., New York, Complexity / NP-Completeness , March 20th 2015

7 The Need for Problem Complexity Classes One strategy: If you can t do something, prove it can t be done. Good idea, but for many problems no such proof can be found either. This is the case for many optimization problems and whether they are polynomial-time solvable....now your bandersnatch boss asks to see you about the SLOWNESS of your algorithm. What can you say? Complexity / NP-Completeness , March 20th 2015

8 The Need for Problem Complexity Classes You should say: I can t prove it can t be done, but I can prove that the problem is as hard as all these problems: traveling salesman problem, propositional satisfiability, solving quadratic Diophantine equations, shortest common supersequence, bounded post correspondance problem,... And NO-ONE ELSE has found a polynomial time solution to those either! I.e. you might be able to prove an equivalence of your problem with some others that seem difficult. This is a key motivation for the class of NP-Complete problems (which are problems that all have an equivalent maximum level of difficulty). Complexity / NP-Completeness , March 20th 2015

9 The Need for Problem Complexity Classes c Bell Telephone Laboratories, (Reproduced without permission) Complexity / NP-Completeness , March 20th 2015

10 Optimization Problems, Evaluation Problems and Decision Problems Optimization Problems are Three Problems in One. 1. An optimization version instance has the form: Given the parameters of the instance, find the optimal feasible solution. (The solution must define the values of the variables to achieve the optimum.) 2. An evaluation version of the problem instance requires only that the optimal solution cost is returned. 3. A decision version of the problem instance gives the parameters of the instance and an additional integer L and asks for a yes/no answer to the question, Is there a feasible solution of cost c L? + The theory of computation studies problems like the decision version above, not optimization problems, so the classes P, NP (and NP-Complete) are defined on decision problems only. In the course textbook this is called a recognition version. + We assume a minimization problem, without loss of generality. Complexity / NP-Completeness , March 20th 2015

11 The Class P Informally the class P is the class of decision problems solvable by some algorithm within a number of steps bounded by some fixed polynomial in the length of the input. Stephen Cook The length of the input refers to the number of symbols written in some alphabet (or language) needed to describe a problem instance. Recall that O(n log n), which is a log-linear complexity, and not polynomial, is still bounded by a polynomial, e.g. O(n 2 ). Similarly other complexities involving roots will generally be bounded by polynomials if exponents and factorials of n are absent. Complexity / NP-Completeness , March 20th 2015

12 The Class NP Definition: The class NP is the set of decision problems such that a yes certificate can be checked in polynomial time. Thus, the class NP encapsulates all problems where it is easy/quick to verify a proposed yes solution (a certificate) to every problem instance that admits a yes solution. Proof: pi is a YES instance since: Algorithm Certificate is verified Every combinatorial optimization problem in its decision version is in NP (see p.351, course textbook). Complexity / NP-Completeness , March 20th 2015

13 Important Note: It is not a requirement of members of NP for there to be a method of efficiently generating the certificate. Second Important Note: If the problem instance is a no instance then any proposed solution (or certificate) must be verified as being a no in polynomial time. But there is no need to be able to arrive at the NO decision for the problem (one might have to check every certificate) quickly to be in NP. E.g. the problem may be Is there a route around the 50 state capitals of the United States in at most 5000 miles?. The answer is no, but it can be difficult to prove because we may have to check a lot of certificates. ALL that is required for the problem to be in NP is that we can check each no certificate in polynomial time. E.g. we can verify that a particular tour does NOT visit all the state capitals and have a length 5000 miles. Complexity / NP-Completeness , March 20th 2015

14 Example of a Member of the Class NP TSP (decision version) instance: Is there a Hamiltoninan Cycle in this graph of length less than L=41? A 7 B 9 6 F 6 G 7 5 C E D certificate: <ABCDEGF> = 40 Certificate: a sequence of cities, such as ABCDEGF Checking algorithm: check each edge in the certificate (e.g. A-B, B-C,..., F-A) exists in the given graph, and that every vertex in the sequence appears exactly twice. Complexity / NP-Completeness , March 20th 2015

15 Proof (TSP (decision) problem is in NP.) Clearly, the checking can be done in polynomial time since the number of symbols in the certificate is less than the number in the problem instance (input), and each check that must be done is a simple look-up or a simple increment to a counter. More examples: Papadimitriou and Steiglitz, p Complexity / NP-Completeness , March 20th 2015

16 All P problems are NP Theorem: Every (decision) problem in P is a member of NP. Argument: Assume a problem is in P but not in NP. The consequence of being in P: the algorithm for generating the decision (yes/no) is polynomial. But the trace of this algorithm can serve legitimately as the certificate of the decision (a polynomial certificate) contradicting the assumption the problem is not in NP. See page 351 of Papadimitriou and Steiglitz for fuller treatment. Complexity / NP-Completeness , March 20th 2015

17 Is P a proper subset of NP? This is the notorious P? NP (or P vs NP) problem. Computer scientists have not been able to prove that problems such as Traveling Salesman, Graph Coloring (scheduling and timetabling), Diophantine Equations, cannot be solved in polynomial time. Either: P = NP OR NP P P = NP P NP Complexity / NP-Completeness , March 20th 2015

18 P vs NP P vs NP One of the Clay Millenium Prizes vs NP/ is offering a Million US Dollars for a legitimate proof either way. Complexity / NP-Completeness , March 20th 2015

19 P vs NP (Wider Implications) Apart from being a rigourously defined mathematical problem of great interest to theoretical computer science, the P vs NP problem is also taken to have wider (even philosophical) implications about the limits of AI, and questions around human creativity. If P=NP, then painting a (new) masterpiece should be as easy as recognizing that it is a masterpiece. Check out the painting. vs. I verify it is good. Doing a painting versus verifying a painting is good. Complexity / NP-Completeness , March 20th 2015

20 P vs NP Some time in but Deolalikar is not a prize millionaire yet. Complexity / NP-Completeness , March 20th 2015

21 P vs NP In fact, when Deolalikar announced his proof, Scott Aaronson, a respected theoretical computer scientist wrote on his blog I hereby announce the following offer: If Vinay Deolalikar is awarded the 1,000,000-dollar Clay Millennium Prize for his proof of P NP, then I, Scott Aaronson, will personally supplement his prize by the amount of 200,000 dollars. I m dead serious and I can afford it about as well as you d think I can. even before the proof had been seen or checked. Complexity / NP-Completeness , March 20th 2015

22 Polynomial Transformations There is a polynomial transformation from a problem Π 1 to a problem Π 2 if there is a function F such that for every instance π 1 Π 1 there is an instance π 2 Π 2 = F (π 1 ), such that π 1 is a yes instance of Π 1 if and only if π 2 is a yes instance of Π 2, and F is a polynomially bounded function. We write Π 1 Π 2. Note 1: Π 2 is a least as hard a problem as Π 1 since the above says that all instances of Π 1 can be mapped to instances of Π 2, but not (necessarily) the other way around. Note 2: polynomial transformation is transitive, i.e. if Π 1 Π 2 and Π 2 Π 3 then Π 1 Π 3. We can use this idea to define a class of most difficult problems in NP. Complexity / NP-Completeness , March 20th 2015

23 The Definition of NP-Complete A Problem Π is NP-complete if and only if 1. Π is in NP, and 2. Π Π for every other Π in NP. It would be most inconvenient if, to show a problem is hard, we had to show that every problem in NP transforms to it. Fortunately, we can instead prove that just one NP-complete problem transforms to it. Why is this sufficient? Complexity / NP-Completeness , March 20th 2015

24 Updated View of Complexity Classes NP Complete NP P It has been shown that if P NP then there are problems in NP that are not in P or NP-Complete, including integer factoring. Complexity / NP-Completeness , March 20th 2015

25 The Usefulness of the NP-Complete Class 1. No NP-Complete problem can be solved by a known polynomial-time algorithm 2. If a polynomial-time algorithm is found for one NP-Complete problem, then all NP-Complete problems will be solvable in polynomial time. So, NP-Complete serves as a kind of badge of difficulty. One can now say, my problem Π is hard unless P=NP. (And most people who know, think P NP). It also allows every theoretical computer scientist to be working on the same problem, i.e., searching for a proof that P NP, or searching for a polynomial time algorithm for any NP-Complete problem (and hence proving P=NP). Complexity / NP-Completeness , March 20th 2015

26 The First NP-Complete Problem(s) Timeline: 1971 Stephen Cook proves that SATISFIABILITY is NP-Complete by showing that every other problem in NP can be transformed to it Cook also proves that another problem, SUBGRAPH ISOMORPHISM is NP-Complete Richard Karp gives NP-Completeness proofs for 21 further problems in NP Levin gives NP-Completeness proofs for a number of natural problems of interest Garey and Johnson publish the landmark book Computers and Intractability..., which contains NP-Completeness results (references to proofs) for more than 300 problems across Graph theory, Network design, Sets and partitions, Sequencing and scheduling, Logic, Storage and retrieval, Games, and Number theory. Complexity / NP-Completeness , March 20th 2015

27 Generalizations and Special Cases One easy way to prove a problem is NP-Complete is to show it is a generalization of an existing NP-Complete problem. E.g. The 0/1 Knapsack problem is a generalization of the problem, PARTITION. PARTITION asks whether or not it is possible to divide a set of items (each with a weight) into two sets with equal total weight. Can you show that 0/1 Knapsack is a generalization of this? Notice: This does not mean PARTITION is easier than 0/1 Knapsack; there must also be a way to transform 0/1 Knapsack to PARTITION too, so that each instance of 0/1 knapsack is a yes instance if and only if the corresponding PARTITION instance is a yes. Special cases (or restrictions) of NP-Complete problems need not be NP-complete. For you to do: Give a restriction of 0/1 Knapsack that makes it easy (polynomial-time solvable). Complexity / NP-Completeness , March 20th 2015

28 The Definition of NP-Hard Recall the definition of NP-Complete: A Problem Π is NP-complete if and only if 1. Π is in NP, and 2. Π Π for every other Π in NP. If Π satisfies condition (2) and whether or not it satisfies condition (1), then it is NP-hard. So, NP-hard problems are not necessarily in NP. The hard optimization problems (in the original optimization version) are NP-hard, and not in NP. NP-hard is the class of problems (not necessarily decision problems) that are at least as hard as the NP-Complete problems. Complexity / NP-Completeness , March 20th 2015

29 Warning: Some people mistakenly think NP-Complete are really hard problems, and NP-hard are slightly less hard. This is incorrect. NP-hard problems are a larger class than NP-Complete and at least as difficult. Complexity / NP-Completeness , March 20th 2015

30 The Complement of NP: co-np A problem is said to be in co-np if its complement is in NP. Hamiltonian Cycle Complement Problem: Is there NOT a Hamiltonian cycle in the given graph G? The above problem is in co-np (since if you remove the NOT you have a problem in NP). The above problem is not in NP. Why? (Hint: certificates?) Every P decision problem is in both NP and co-np. Can you see why? Complexity / NP-Completeness , March 20th 2015

31 Summary of Key Information decision problems P NP P NP P NP probably problems with yes/no answer set of poly.-time solvable decision problems set of problems whose yes instances can be verified (not solved) in poly. time, given a suitable certificate (since solution constitutes certificate) though proof is seeming difficult! NP-Complete Problem Π 1. Π is in NP, and 2. Π polynomially transforms to Π for all Π in NP NP-Hard Problem Π At least as hard as any problem in NP. Not necessarily in NP; so does not need to be a decision Π co-np P NP co-np problem complement of problem Π. Yes instances of Π are no instances of Π problems where no instances can be verified in poly. time (since run of polynomial algorithm is certificate for both a problem and its complement) Complexity / NP-Completeness , March 20th 2015

32 Part II Pseudopolynomial Algorithms and Approximation Algorithms Complexity / NP-Completeness , March 20th 2015

33 0/1 Knapsack Problem is Weakly NP-Hard You are currently studying algorithms for the 0/1 Knapsack problem. Later, you will study a Dynamic Programming algorithm for the problem. It works by constructing a table of size N.C, where N is the number of items in the knapsack and C is the knapsack capacity. Each operation for constructing the table is simple [i.e., O(1)], so the complexity of the algorithm as a whole is O(N.C). But 0/1 Knapsack is NP-Hard (or NP-Complete in decision form), so how can this be? Complexity / NP-Completeness , March 20th 2015

34 The Dynamic Programming Approach to Knapsack Here is a problem with Knapsack capacity=6 being solved. There are 3 items of sizes 3,1 and 5. (The profits equal the sizes.) Capacity item The optimal solution is in the last (bottom right) cell of the table. This looks like polynomial time. So did we prove P=NP? Complexity / NP-Completeness , March 20th 2015

35 0/1 Knapsack is Still Exponential Time :-( Why is the DP algorithm for Knapsack not polynomial? Because O(N.C) is not polynomial in the input size of the instance! Consider a problem with just 5 items but knapsack capacity 1,000,005 with items of sizes 743,004; 225,025; 78,000; 634,451; 362,017. What is the size of the instance? Well, count the digits in the input above. There are just 36. But N.C is 5, 000, 000. What has happened here? The input size is N. log C since it takes roughly log C digits to express each number (item weight). So O(N.C) is exponential in the input size. However, practically, Knapsack problems often grow in the number of items, not (so much) by the capacity. So in practice DP approach to Knapsack is fast. Complexity / NP-Completeness , March 20th 2015

36 0/1 Knapsack is Pseudopolynomial Time Solvable It seems useful to be able to describe the relative easiness of Knapsack in practice (despite it being NP-hard). Definition: A problem can be solved in pseudopolynomial time if there is an algorithm that can solve every instance π in a polynomial function of n and number(π). Above, number(π) can be taken as the largest number described in the input. Complexity / NP-Completeness , March 20th 2015

37 Strongly NP-Hard Means Not Pseudo-Poly. Solvable Many NP-hard problems cannot be solved in pseudopolynomial time. E.g. dynamic programming when applied to the travelling salesman still gives a running time exponential in V, so it does not matter if the weights in the graph are small or large. No other pseudopolynomial time algorithm is possible either unless P=NP. Such problems are called Strongly NP-Hard. Definition: A problem is strongly NP-hard if it remains NP-hard for instances π n Π that are restricted to the set of instances where number(π n ) is polynomial in π n. An NP-hard problem that is not strongly NP-hard is called weakly NP-hard. Complexity / NP-Completeness , March 20th 2015

38 Approximation Algorithms One approach to tackling NP-hard optimization problems, is to use polynomial-time algorithms that give guaranteed levels of approximation of the optimal solution. Definition: An algorithm is a p-approximation algorithm for a problem Π if it gives a solution not worse than p times the optimal cost on any instance π Π. If a p-approximation algorithm is in addition a polynomial time algorithm for Π, then it is known as a polynomial time approximation algorithm for Π. Complexity / NP-Completeness , March 20th 2015

39 Approximation Ratios The performance ratio of an algorithm is for a minimization problem max π Π max π Π C alg (π) C opt (π) C opt (π) C alg (π) for a maximization problem where C alg (π) is the value of the algorithm solution on input π and C opt (π) is the value of the optimal solution on input π. Complexity / NP-Completeness , March 20th 2015

40 A 2-Approximation Scheme for Metric TSP One form of metric TSP is: Given a weighted, complete graph G = (V, E) with edge weights satisfying the triangle inequality, give a minimal weight Hamiltonian cycle in G. A 2-Approximation Scheme for it is the following: [HOMEWORK: Prove this] Step 1. Construct the minimum spanning tree (e.g. using Kruskal or Prim s algorithm.) This takes O( V 2 ) time. Step 2. Create an Eulerian graph from the tree by duplicating all edges. This step is O( V ). Step 3. Construct an Euler cycle in the Eulerian graph by using a cycle finding algorithm. The complexity is O( V + E ). Step 4. Finally, construct a Hamiltonian Cycle in G by shortcutting the Euler cycle. That is, skip out vertices already visited and proceed to the next unvisited vertex on the Eulerian cycle. This takes O( V 2 ) steps. Complexity / NP-Completeness , March 20th 2015

41 Illustration: A 2-Approximation Scheme for Metric TSP D E C B A F G D E C B A F G 2 D E C B A F G 2 TSP Instance STEP 1 STEPS 2 & 3 D E C B A F G STEP 4 Note: the scheme is illustrated here on a sparse graph. However, in general it only works on complete graphs. Why? Complexity / NP-Completeness , March 20th 2015

42 The APX Class Any problem that is approximable within a fixed constant factor of the best cost is in the class APX (approximable). In other words, if there is a polynomial time p-approximation algorithm for a problem, then the problem is in APX. Metric TSP is in APX. Complexity / NP-Completeness , March 20th 2015

43 Summary The P, NP and NP-Complete complexity classes are important for classifying problem difficulty. NP-hard class characterizes optimization problems (optimization versions) as being very difficult. Strongly NP-hard problems remain hard even when all the numbers (e.g. weights) in a problem instance are small (i.e. bounded by a polynomial of the instance size). Weakly NP-hard problems are only hard when some of the numbers in an instance are large. They can be solved in pseudo-polynomial time. Constant-factor approximation algorithms for NP-hard problems that run in polynomial time are known as polynomial approximation algorithms. The class of problems for which a PTAA exists is called APX. Complexity / NP-Completeness , March 20th 2015

44 Further Reading Chapters 15, 16, 17 of Papadimitriou and Steiglitz Garey and Johnson (1979). Look at the NP-Complete problems listed in the back, and how they are related Zoo Some more links are on the course webpage. Complexity / NP-Completeness , March 20th 2015