Decidability: Church-Turing Thesis While there are a countably infinite number of languages that are described by TMs over some alphabet Σ, there are an uncountably infinite number that are not Are there machines that are more powerful than TMs that can describe some of these other languages? Note: Since there can only be a countably infinite number of machines that are described by some finite representation, there will always be some languages that cannot be described Theory of computation based on the concept of algorithms Important concepts relating to theory of computation introduced in the early 20th century 1. Is it possible to axiomatize all mathematical structures of interest so that every true statement becomes a theorem? The set of axioms may be infinite, but they must be decidable - there must be an algorithm that can determine whether a string is an axiom 2. Given a set of axioms, does there exist a machine that can decide whether an axiom is a theorem? These questions were proved to be unanswerable 1. Godel s Incompleteness Theorem Statement: Given a potentially infinite set of true statements in number theory (Peano arithmetic), some cannot be proven 2. Entscheidungsproblem (decision problem) of Hilbert and Achermann Statement: (a) Does there exist an algorithm for deciding, for an arbitrary sentence w in first order logic, whether w is valid? (b) Given a set of axioms A and a sentence w, does there exist an algorithm for deciding whether w is entailed by A? (c) Given a set of axioms A and a sentence w, does there exist an algorithm for deciding whether w can be proved from A? 1
Decidability: Church-Turing Thesis (2) Concept of algorithm is important for proof of the Entscheidungsproblem Turing and Church independently formalized what is meant by algorithm 1. Turing s approach was the Turing machine 2. Church s approach was the lambda calculus Church-Turing Thesis Statement: Lambda calculus and Turing machines are equivalent in power Generalization: Any formalisms capable of describing computational algorithms are equivalent in power This has not been proven, hence it is a thesis, not a theorem Implications are that 1. There is no computational model that is more powerful 2. There are problems (languages) that have no computational solutions Thus, there are limitations to what computers can do 2
Decidability: The Halting Problem The theory of computation is concerned with the classes D, SD, SD/D = SD D, and SD The halting problem is concerned with the language H = {< M, w >: TM M halts on input string w} Our concern here - and throughout the rest of this section - is whether a language is D, SD/D, or SD? While correct syntax of strings input to decision procedures (i.e., < M, w >) is important, that is not our concern here Our concern is whether machine M halts when it runs on input w We also do not consider general strings from Σ Thus, H = {< M, w >: TM M does not halt on w} The halting problem is: Is H decidable, or semidecidable? Theorem 19.1 Statement: Language H = {< M, w >: TM M halts on input string w} SD Proof: By construction Boolean M SH (<M, w>) { M(w); return TRUE; } 3
Decidability: The Halting Problem (2) Theorem 19.2 Statement: Language H = {< M, w >: TM M halts on input string w} / D Proof: If H were decidable, there would be a TM M H that decides H It could be implemented as Boolean M H (string <M>, string <w>) { if (<M> represents a TM that halts on w) return TRUE; else return FALSE; } Now consider TM C: Boolean C (<M>) { if (M H (<M>, <M>)) loop; else halt; } The crux of the proof is the answer to the question What is the result of C(< C >)? 1. If C were decidable, C would never halt 2. If C were not decidable, C would halt The premise on which C is based - the existence of M H - must be incorrect Therefore, M H does not exist 4
Another way of looking at things: Decidability: The Halting Problem (3) Consider the table below, which indicates encodings which a TM accepts < M 1 > < M 2 > < M 3 > < M 4 >... M 1 accept accept... M 2 accept... M 3 accept accept accept... M 4 accept......... Now consider this table, which indicates the result of running M H on each (row, col) pair < M 1 > < M 2 > < M 3 > < M 4 >... M 1 accept accept... M 2 reject reject accept... M 3 accept accept accept... M 4 accept reject......... Since C is a TM, it must appear in some row, and its encoding in some column The values in C s row are opposite the values of the diagonal 1. If [1, 1] is accept 2. Then [c, 1] is reject < M 1 > < M 2 > < M 3 > < M 4 >... M 1 accept accept... M 2 reject reject accept... M 3 accept accept accept... M 4 accept reject......... C reject accept reject accept......... What value should be stored in [c, c]? If [c, c] is reject, it should be accept, and vice-versa! 5
Decidability: The Halting Problem (4) Theorem 19.3 Statement: If H were in D, then every language in SD would be in D too Discussion: Let L SD Then there exists a TM M L that semidecides L Suppose H were in D Then it would be decidable by some machine O To decide whether some w L, we could use O to determine whether M L halts on w If the answer were yes, then we could run M L on w to see if it accepts Proof: By construction of M to decide above Construction Boolean M (string w) { result = O(<M L, w>); if (result == accept) { result2 = M L (w); if (result2 == accept) accept(); else reject(); } else reject(); } O always halts, so M always halts Therefore, if H were in D, all languages in SD would be too 6
Decidability: The Relationship Between D and SD The following theorems explore the relations among the languages D, SD, and SD Theorem 20.1 Statement: The Context Free Grammars are a proper subset of D Proof: By Theorem 14.1, there is a decision procedure to determine whether a string w L, where L is a CFL But A n B n C n is not a CFL, and it is decidable Therefore CF Ls D Theorem 20.2 Statement: Every language L D is also in SD Proof: If L D, L is decided by some TM M This means that M accepts all strings in L and only strings in L This meets the definition of semidecidability, so L SD 7
Theorem 20.3 Decidability: The Relationship Between D and SD (2) Statement: There are languages that are not in SD Lemma: There are a countably infinite number of semidecidable languages over an alphabet Σ Proof (of lemma): Every language L SD is semidecided by some TM with alphabet Σ The TMs with alphabet Σ can be lexicographically ordered By Theorem A.1, there are a countably infinite number of semideciding TMs Therefore, there are at most a countably infinite number of languages in SD, since there cannot be more semidecidable languages than TMs There is not a 1:1 correspondence between TMs and semidecidable languages, since there is more than 1 TM to semidecide a language The number of semidecidable languages is at least countably infinite because D SD, and D is countably infinite Therefore, there are at least and at most a countably infinite number of semidecidable languages Proof (of theorem): By Theorem 2.2, there are an uncountably infinite number of languages over an alphabet. So there must be more languages over an alphabet than there are in SD So there is at least 1 language in SD Theorem 20.4 Statement: D is closed under complement Proof: By construction Let L D There must be a deterministic TM that decides L Construct M as follows 1. Initially, let M = M 2. Make the non-accepting states accepting 3. Make the accepting states non-accepting M now rejects what M accepts, and vice-versa Since M always halts, M always halts: M decides L 8
Theorem 20.5 Decidability: The Relationship Between D and SD (3) Statement: SD is not closed under complement Proof: By contradiction Suppose SD were closed under complementation Then L SD would mean that there exists a TM M that semidecides L Construct M as follows 1. M simulates M and M in parallel on input w 2. Since w L or w L, w is accepted by either M or M This means that M will halt on w So if all L SD were closed under complement, all would also be decidable But this is not the case, so SD is not closed under complement Theorem 20.6 Statement: Language L D iff L SD and L SD Proof(if part): If L D, then L SD By Theorem 20.4, L D Therefore, L SD Proof(only-if part): By construction Since L SD and L SD, both are semidecided by some TM Call these TMs M and M Construct M as follows 1. M simulates M and M in parallel on input w Since w is accepted by one of M and M, M must halt Since M decides L, L D Theorem 20.7 Statement: H / SD Proof: H SD by Theorem 19.1 If H SD, then H D by Theorem 20.6 But H / D by Theorem 19.2 Therefore, H / SD 9
Decidability: And Enumeration TM M enumerates a language L if 1. It starts with a blank tape 2. Writes some string w L onto the tape 3. Signals that it has written a string 4. Repeats If L is finite, M will halt M has a special state p to indicate when a string has been generated M enumerates L iff, for some fixed state p, L = {w : (s, ) M (p, w)} Language L is said to be Turing-enumerable iff there exists a TM that enumerates it Theorem 20.8 Statement: A language L is semidecidable iff it is Turing-enumerable Proof (only if): By construction If L is Turing-enumerable, then there is a TM M that enumerates it Construct M (w) as follows 1. M copies input w onto a second tape 2. M executes M 3. On each new string w generated by M, M compares w with w 4. If w == w, M accepts Since M semidecides L, M SD 10
Proof (if): Discussion Decidability: And Enumeration (2) If L SD, then there is a TM M that semidecides it Generate all strings in Σ lexicographically Run M on each string w If M accepts, w L Problem is that M may not halt on a given input Solution is to use technique called dovetailing M controls M so that it generates strings in parallel M will be allowed to execute one step on each string at any time Strings will be generated in a diagonal manner w 1 [0] w 1 [1] w 2 [0] w 1 [2] w 2 [1] w 3 [0] w 1 [3] w 2 [2] w 3 [1] w 4 [0] w 1 [4] w 3 [2] w 4 [1] w 5 [0] If M accepts a string, M will output that string If M halts on a string, computation will stop on that string (w 2 above) Proof(if): By construction of M as described above 11