11. Tractability and Decidability

Transcription

1 11. Tractability and Decidability Frank Stephan April 1, 2014

2 Introduction Computational Complexity In 1844 Gabriel Lamé studied and analysed the running time of Euclid s algorithm. In 1982, Stephen Cook received the Turing Award for his contribution to complexity of computation. In 1985 Richard Karp received the Turing Award for his contributions to the theory of algorithms.

3 Introduction Complexity Theory The foundations [for complexity theory] were laid in the 1930s by a group of logicians, including Alan Turing, who were concerned with the existence or nonexistence of automatic procedures for deciding whether mathematical statements were true or false. Karp s Turing Award Lecture, Combinatorics, Complexity, and Randomness (

4 R Decidability A problem is decidable if there exists an algorithm that can solve all instances of the problem. We call R the class of all decidable problems. Undecidability A problem is undecidable if there does not exist an algorithm that can solve all instances of the problem. There are many more Problems than Algorithms Most problems are undecidable.

5 R Problems R

6 R Cardinality The set has 0 elements. The set {Cantor} and the set {Turing} have 1 element. The set {Cantor, Hilbert, Gödel} and the set {Turing, VonNeumann, Karp} have 3 elements. Natural Numbers N = {0, 1, 2, 3, 4, } is the set of possible cardinalities of finite sets.

7 R Infinity There are infinitely many natural numbers. The number of natural numbers is a not a natural number. There are ℵ 0 (pronounce aleph-naught, aleph-null, or aleph-zero) (countably many, countably infinite) natural numbers (they are countable). ℵ 0

8 R Countability The set of odd natural numbers is {1, 3, 5, 7, }. There are as many odd numbers as there are natural numbers. There are ℵ 0 (countably many) natural numbers. Countability m odd = 2 n + 1 The set of rational numbers is {0, 1 3, 1 2, 1, 5 4, }. There are as many rational numbers as there are natural numbers. There are ℵ 0 (countably many) natural numbers.

9 R

10 R Sets of Sets The set {1, 2, 3} contains 2 3 = 8 subsets of numbers:, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3} Sets of Natural Numbers How many sets of natural numbers are there? 2 ℵ 0 2 ℵ 0 = ℵ 1? (Continuum Hypothesis)

11 R Sets of Natural Numbers Let us represent a set of natural number as a (infinite) vector. =< 0, 0, 0, 0, 0, 0, 0,... > {0, 1} =< 1, 1, 0, 0, 0, 0, 0,... > {2, 4, 5} =< 0, 0, 1, 0, 1, 1, 0,... >

12 R set /number {0} {1, 2} {4} {0, 1}

13 R set /number {0} {1, 2} {4} {0, 1}

14 R Diagonal Argument Let us select a set A which differs from the diagonal everywhere: < 0, 0, 1, 0,, > A =< 1, 1, 0, 1,, > The new set A is distinct from all rows. Proof If there was a row identical to A, say the n th row, then the n th entry of A would be identical to the n th entry in the n th row, although A differs at the n th entry from the n th entry of the diagonal. Thus A does not occur in the list.

15 R Algorithms are Countable An algorithm can be represented by pseudo-code, by a program in JavaScript, by the program input to a universal Turing machine, namely, by a finite string of 1s and 0s. An algorithm is a natural number. Problems are Uncountable A decision problem is a function from a set of inputs to the set {yes, no}. There are ℵ 0 possible inputs and 2 ℵ 0 many possible answer-vectors to these inputs. A decision problem is the subset of those possible inputs which have yes as an answer. There are 2 ℵ 0 problems. There are 2 ℵ 0 problems, 2 ℵ 0 subsets of N and 2 ℵ 0 real numbers.

16 R Example: Primality The primality problem seeks to decide whether a natural number is a prime number. An algorithm is a natural number: it can be represented by the finite sequence of bits of its JavaScript code. < 1, 1, 0, 1, 1, 0, 0, 1, 1, 0,, 1, 0 > A problem is a real number: for each natural number, the following infinite vector indicates whether it is a prime or not. < 0, 0, 1, 1, 0, 1, 0, 1, 0, 0, 0, >

17 R Do we Know Undecidable Problems? The halting problem, Post correspondence problem, Wang tiles, Diophantine equations, etc.

18 R The Halting Problem is Undecidable There is no algorithm that can decide whether an algorithm halts on every input. Proof (by Turing) Assume that there exists a function Halt(F, i) that decides whether the function F halts on input i. It returns 1 if F halts on input i, and 0 otherwise. Let us consider the function X below. If we call X(X), the call will loop forever if and only if Halt says that X with input X halts. This is a contradiction. 1 f u n c t i o n X( i ) { 2 i f ( H a l t ( i, i ) == 0) then r e t u r n 0 ; 3 e l s e l o o p f o r e v e r ; }

19 R Post Correspondence Problem Given a finite list of pairs of words, is there a sequence of any length of these pairs possibly repeated that yields the same sentence on the first and second line? ( ) ( ) ( ) ( ) ( ) ( ) s Amer a ic il l Po Pos wa an s t ( ) ( ) ( ) ( ) ( ) Em Emi t was n an A l merican logici og

20 R Wang Tiles A Wang tile is square split into four equilateral triangles painted in colors. As proposed by Wang Hao in 1961, the Wang tiles problem is the question whether a finite set of tiles can tile the plane such that adjacent triangles from different tiles have the same color. Wang thought the problem was decidable. In 1966, Robert Berger proved the problem is undecidable.

21 R Diophantine equations A Diophantine equation is a multivariable polynomial equation whose solutions should be integers. Deciding whether any given Diophantine equation has a solution is David Hilbert s tenth problem. In 1970, Yuri Matiyasevich proved that it is undecidable. Fermat s Last Theorem Andrew Wiles proved in 1995, as Pierre de Fermat suspected in 1637, that there is no positive integer solution to x n + y n = z n except for n = 1 and for n = 2 (Pythagorean triples).

22 R Is everything solvable? Time (1984): Put the right kind of software into a computer and it will do whatever you want it to. There may be limits on what you can do with the machines themselves but there are no limits on what you can do with software. David Harel (1993 and 2012): Bullshit!

23 P Karps Turing Award Citation For his continuing contributions to the theory of algorithms including the development of efficient algorithms for network flow and other combinatorial optimization problems, the identification of polynomial-time computability with the intuitive notion of algorithmic efficiency

24 P P P is the set of problems that can be solved in polynomial time (O(p(n)) where p(n) is a polynomial). Examples Searching a list, finding peaks, computing the maximum, sorting, Euclid algorithm, primality testing (2004 by Manindra Agrawal Neeraj Kayal and Nitin Saxena), circuit value problem, context-free grammar membership (membership in a language recognised by some push down automaton), some famous data-compression algorithms (Lempel-Ziv-Welch, 1978 and 1984).

25 P Problems R P

26 P Tractability The Cobham thesis proposes to call the problems in P tractable and the others untractable. Alan Cobham discussed the concept of the complexity class P in a 1965 paper.

27 EXP Combinatorial Explosion The input to the program was a set of Boolean formulas specifying how the outputs of the circuit were to depend on the inputs: the program was supposed to generate a circuit to do the job using a minimum number of logic gates. [...] The number of circuits that the program had to comb through grew at a furious rate as the number of input variables increased Combinatorial Explosion and Disillusionment Today, our optimism in even trying an enumerative approach may seem utterly naive [...] with an initial surge of excitement as toy problems were successfully solved, followed by disillusionment as the full seriousness of the combinatorial explosion phenomenon became apparent.

28 EXP EXP EXP is the set of problems that can be solved in exponential time (O(2 p(n) ) where p(n) is a polynomial). Examples Deciding if a position in chess, checkers, or Go, is a winning position (with an infinite board).

29 EXP n n Chess The n n Chess problem seeks to decide whether a given configuration on a n n chess board wins.

30 EXP Problems R EXP P

31 NP NP NP is the set of problems whose solutions can be verified in polynomial time. Examples Prime factorization, minimum circuit size, satisfiability of Boolean expressions, Sudoku, Tetris, Minesweeper, Nonograms, Eternity II, Traveling Salesman Problem, etc. Let us play Tetris (Can you survive versus can you die)

32 NP NP NP is the set of problems that can be solved in polynomial time (O(p(n)) where p(n) is a polynomial) by a non-deterministic Turing machine. NP stands for non-deterministic polynomial time. ( ;1;R) start q 0 q 1 ( ;0;R) ( ; ; ) ( ; ; ) ( ; ; ) ( ; ; ) q 2

33 NP SAT Satisfiability seeks to decide whether a given Boolean formula can be true. p q r (p q) (r q) true true true false true true false true true false true true true false false false false true true false false true false true false false true true false false false true SAT is not known to be in P.

34 NP Problems R EXP NP P

35 NP NP-hard A problem is NP-hard if it is at least as hard as any problem in NP. NP-complete A problem is NP-complete if it is in NP and NP-hard. SAT NPC Stephen Cook proved that SAT is NP-complete in 1971 (Cook-Levin theorem, also known as Cook s theorem). Cook Turing Award Citation For his advancement of our understanding of the complexity of computation in a significant and profound way.

36 NP Reduction A reduction is an algorithm for transforming one problem into another problem. A many-one reduction is an algorithm for transforming one decision problem into another decision problem.

37 NP NP-complete Problems Richard Karp, in 1972, proved the NP-completeness of 21 problems by showing a polynomial time many-one reduction of SAT to each of these problems. Karps Turing Award Citation most notably, [for his] contributions to the theory of NP-completeness. Karp introduced the now standard methodology for proving problems to be NP-complete which has led to the identification of many theoretical and practical problems as being computationally difficult.

38 NP DNF A Boolean formula is in disjunctive normal form if and only if it is a disjunction of conjunctions of propositions and their negation. ( p q) (r q) (q r) CNF A Boolean formula is in conjunctive normal form if and only if it is a conjunction of disjunctions (called clauses) of propositions and their negation. ( p q r) ( q r)

39 NP 3-SAT 3-SAT is the special case in which a Boolean formula in CNF has at most three variables per clause. ( p q r) ( q r s) ( q r s) 3SAT NPC 3-SAT is NP-complete.

40 NP Proof Sketch We rewrite any k-sat formula into a 3-SAT one. (x 1 x 2 ) becomes (x 1 x 2 x new ) (x 1 x 2 x new ) (x 1 ) becomes (x 1 x new1 x new2 ) (x 1 x new1 x new2 ) (x 1 x new1 x new2 ) (x 1 x new1 x new2 ) (x 1 x k ) becomes (x 1 x 2 x new1 ) (x 3 x new1 x new2 ) (x 4 x new2 x new3 )... (x k 1 x k x newk 3 ) This transformation is polynomial. (x 1 x 2 x 3 x 4 ) (x 1 x 2 x new1 ) (x 3 x 4 x new1 )

41 NP Now we Use 3-SAT Now we can prove that an NP problem is NP-complete by reducing 3-SAT to it.

42 NP 2SAT P 2-SAT is in P. ( p q) ( q r) ( p r)

43 NP The Exponential Time Hypothesis The (unproven) exponential time hypothesis, by Russell Impagliazzo and Ramamohan Paturi (1999), proposes that NP-complete problems cannot be solved in subexponential time (faster than polynomial but slower than exponential) in the worst case. NPI Prime factorization and minimum circuit size are candidate to be NP-intermediate, that is NP, not in P and not NP-complete.

44 NP Problems R EXP NP P NPI? NPC

45 Conclusion Complexity Classes There are 495 classes and counting in the complexity zoo. What do we Know? There are problems which can be solved in exponential time but cannot be solved in polynomial time. If P NP then NPI, NPC and P are three disjoint nonempty classes. All regular languages are solvable in polynomial time, but not vice versa; the set of all natural numbers with as many digits 1 as digits 2 as digits 3 can be decided in polynomial time but is not regular.

46 Conclusion P=NP? The question whether P=NP is one of the six remaining Millennium Prize Problems. Millennium Prize The Millennium Prize problems are seven problems proposed by the Clay Mathematics Institute in There is a one million US dollars prize for being the first one solving any one of these problems.

47 Conclusion Stories... As of today, six of the Millennium Prize problems remain unsolved. The Poincaré conjecture was solved by Grigori Perelman. Grigori Perelman thought that his proof owed much to Richard Hamilton s work and declined the price. Richard Hamilton did not get Millennium Prize but he got half of the one million US dollar Shaw Prize. The Shaw Prize is a prize by the Shaw Prize Foundation. The prize is named after Sir Run Run Shaw who, with his brother, created the Shaw Organisation in Singapore.

48 Conclusion And more Stories Logicomix: An Epic Search for Truth is a comics by Apostolos Doxiad and Christos Papadimitriou. It tells the story of Cantor, Hilbert, Gödel, Turing and infinity, logic, the Entscheindung problem and many other things.

49 Attribution Attribution The images and media files used in this presentation are either in the public domain or are licensed under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation, the Creative Commons Attribution-Share Alike 3.0 Unported license or the Creative Commons Attribution-Share Alike 2.5 Generic, 2.0 Generic and 1.0 Generic license.