NP-Complete and Non-Computable Problems. COMP385 Dr. Ken Williams

Transcription

1 NP-Complete and Non-Computable Problems COMP385 Dr. Ken Williams

2 Start by doing what s necessary; then do what s possible; and suddenly you are doing the impossible. Francis of Assisi

3 Our Goal Define classes of problems that are very hard or impossible to solve Know some examples of NP-complete problems so you might know one when you see it Understand that some simple problems are not computable Understand that software tools cannot always check properties of programs

4 Algorithm Speed O(c) O(log N) O( N) O(N) O(N log N) O(N 2 ) O(N c ) O(2 N ) Longer yet

5 Exponential Growth f(n) f(nlogn) f(n2) f(n3) f(2n)

6 Further Exponential Growth squared cubed 2 to the N

7 Hard Some problems are hard to solve There exists interesting, reasonable problems for which you cannot compute a solution

8 Graph Examples Consider an arbitrary weighted graph. We want to find a path from A to B without any cycles Easy, no worse than O(N 2 ) Is there a path from A to B with weight < X? Hard, O(2 N ) Is there a path from A to B with weight > X?

9 Intractable Algorithms that are O(2 N ) or harder are considered intractable It usually takes a very long time to solve Increasing N by one doubles the time There are many problems that fall in this category

10 Traveling Salesman Problem Given a list of cities and the distances between each pair of cities, what is the shortest possible route that visits each city exactly once and returns to the origin city? The problem was mathematically formulated in the 1800s by the Irish mathematician William Rowan Hamilton

11 Boolean Satisfiability Problem Given a boolean expression of N boolean variables, what values of the N variables will make the expression true? (X + Y) & (X + Z) & (Y + Z) Easy solution is to try all possible combinations of true and false for all N variables This is the only known solution There are 2 N possible combination of N boolean values The algorithm is O(2 N )

12 More Hard Problems Knapsack problem - Given a set of items, each with a weight and a value, determine the number of each item to include in a collection so that the total weight is less than or equal to a given limit and the total value is as large as possible Subset Sum problem given a set of integers { 7, 3, 2, 5, 8} is there a subset that will sum to zero?

13 NP Hard Problems Non-deterministic Polynomial-time hard Imagine you have an oracle that can guess solutions correctly If you can solve the problem in polynomial time [O(N c ) or less] using the oracle, then the problem is NP-hard

14 Oracles In algorithm theory we assume we have an oracle that can guess solutions and will always guess correctly. If you want to find a path through all nodes of a graph, you could ask the oracle what is the next node to traverse. It would take n steps to find the shortest path through an n node graph.

15 Verifying the Solution Once you have a solution from an oracle, you can verify that it is correct If you can verify that the solution is correct in polynomial time, the problem is NP-hard Non-deterministic Polynomial time

16 NP-complete NP-complete is a class of problems that are all NP hard A problem X is NP-complete if you can show that a known NP-complete problem can be converted to X in polynomial time If you can solve X in polynomial time, then you can solve the NP-complete problem in polynomial time

17 Problem Reduction If you can convert (or reduce) problem A to problem B, then you know if you can solve problem B, you can always solve problem A The subset sum problem can be reduced to the knapsack problem Have the subset sum to the size of the knapsack Let the numbers represent knapsack values Assume all weights are the same

18 NP-complete Examples Boolean satisfiability problem N-puzzle Knapsack problem Hamiltonian path problem Travelling salesman problem Graph coloring problem Longest common subsequence

19 NP = P? While it appears that there is no polynomial time solution to NP-Complete problems, this has never been proven If there exists a polynomial time solution to NP-Complete problems, then NP problems are the same as polynomial (P) time problems The question of whether NP = P is a great unsolved problem in Computer Science

20 Donald Knuth s Opinion At the present time very few people believe that P = NP. In other words, almost everybody who has studied the subject thinks that satisfiability cannot be decided in polynomial time. The author of this book [Knuth], however, suspects that N O(1) step algorithms do exist, yet that they re unknowable. Almost all polynomial time algorithms are so complicated that they lie beyond human comprehension, and could never ben programmed for an actual computer in the real world. Existence is different from embodiment. The Art of Computer Programming, vol.4, fas. 6, Satisfiability, Addison-Wesley, 2015

21 NP Venn Diagrams

22 Harder Yet Some problems are harder than O(2 N ) Even if you have an oracle to guess solutions, you cannot guess the solution in polynomial time

23 Chomsky Hierarchy

24 Algorithms and Language Level What is the average execution time, Big O, for recognizing a regular language? A. O(1) B. O(n) C. O(n 2 ) D. O(n 3 ) E. O(2 n )

25 What is a Non-Computable Problem? Non-Computable Problems are well defined problems that are impossible to solve for all cases, by any algorithm These problems are provably non-computable. They cannot be solved with a faster computer or a better algorithm The best known undecidable problem is the Halting Problem

26 Halting Problem Can you write a program that inputs another program and some data to determine if the other program will eventually terminate (halt) when using the data? Proof by contradiction: Assume we have such a program (called Q) figure from Algorithmics, The Spirit of Computing, 2 nd ed. by David Harel

27 Halting Problem Proof Create program S calling program Q as a method. If Q answers Yes, go into an infinite loop figure from Algorithmics, The Spirit of Computing, 2 nd ed. by David Harel

28 Call program S using S as the input If Q says S halts, then it doesn t If Q says S does not halt, then it does This is a contradiction Therefore Q cannot exist Halting Problem Proof figure from Algorithmics, The Spirit of Computing, 2 nd ed. by David Harel

29 Sometimes but not Always You can frequently solve a problem known to be noncomputable The simple Hello World program will halt while (true) {} never halts While you may be able to solve specific instances of the problem, you cannot write a program to always solve the problem

30 Will this program always halt? This program runs very quickly for some input values, but takes a long time for other input values Collatz conjecture is that it will always halt, but this has not been proven 10 takes 6 steps 9 takes 19 steps 97 takes 118 steps read cat; while (cat > 1) { if (cat%2 == 0) cat = cat / 2; else cat + cat * 3 + 1; }

31 Beyond Just Halting If you cannot prove the a program will halt, you cannot prove that a program will execute a given statement Imagine replacing the given statement with Halt While static program analysis maybe able to find many potential problems, it cannot always find all problems

32 Non-Computable Problems There are a number of problems that have been proved noncomputable including: Post Correspondence Problem Wang Tiles Domino Snakes

33 Post correspondence problem Given a set of domino-like tiles with numbers on the top and bottom, can a series of tiles be placed side by side so that the string of numbers on the top matches the string of numbers on the bottom Assume you have an infinite number of each type of domino

34 Solution

35 A slightly more difficult problem

36 Solution to Example Simple solution to this instance of the problem requiring seven dominos

37 More Difficult Problem If you change the bottom number on the last tile so that it only contains one zero, the problem becomes much more difficult This simple change has a solution, but it requires 75 dominos

38 Algorithm for Post Correspondence Here is a possible algorithm for solving the Post Correspondence problem for (d = 0; d < # dominos; d++) { listofdominos first = domino[d]; boolean foundanswer = postc( first ); if (foundanswer) return true; } return false;

39 Algorithm for Post Correspondence boolean postc( listofdominos sofar) { for (i = 0, i < #dominos, i++) { if (domino[i] fits at end of sofar) { put domino[i] at end of sofar; if (top and bottom match) return true; boolean works = postc( sofar); if (works) return true; remove domino[i] from end of sofar; } } return false; }

40 How Long Will It Take? This is a depth first search How deep do you have to search? The problem becomes very difficult because you do not know how far to search before you are sure it cannot be done

41 Wang tiles When using a set of square tiles with colored edges, can the tiles be arranged without rotation or reflection so that they tile a plane with adjacent tiles having edges of the same color? Assume you have an infinite supply of each tile

42 Tiling Solution

43 Periodic and Aperiodic The previous solution has a 2 by 4 block that can be repeated forever Some sets of tiles can tile an infinite plane without repeating

44 Aperiodic Tiling

45 Domino Snakes Using tiles similar to Wang Tiles, can a given set of tiles form a path from two given points so that all adjacent tile edges have matching colors?

46 Snake Solution

47 Mu Puzzle Start with the string MI and convert to MU Rule 1 If the string ends in I, you can add U to the end MI can change to MIU Rule 2 A string Mx can be changed to Mxx MIU can change to MIUIU

48 More Mu Rule 3 If the string ends in III, you can replace the III with a U MUIII can change to MUU Rule 4 If UU occurs in a string, you can drop the UU MUUII can change to MII

49 To Be Discussed Wednesday There you have it. Now you may begin trying to make MU. Don t worry if you don t get it. Just try it out a bit the main thing is for you to get the flavor of this MU-puzzle. Have fun. Douglas Hofstadter Gödel, Escher, Bach: an Eternal Golden Braid