Theory of Computation CS3102 Spring 2014 A tale of computers, math, problem solving, life, love and tragic death Nathan Brunelle Department of Computer Science University of Virginia www.cs.virginia.edu/~njb2b/theory
Rice s Theorem Henry Gordon Rice, 1951 Any nontrivial property about the language of a Turing machine is undecidable. A property is trivial if it either applies to ALL languages of Turing Machines, or NONE of them. M w Machine R: 1) Build (but don t run!) machine M x 2) Return B( M ) Machine M : 1) Simulate M(w) 2) Check if x is a TM with property P Let have property P, otherwise replace P with P
Which of these are Undecidable? Does TM M accept any strings? Does TM M accept Hello? Undecidable Undecidable Does TM M take more than 1000 steps to process input w? Does TM M accept a finite number of strings? Does TM M accept a Decidable Language? Does TM M accept a Recognizable Language? Undecidable Undecidable Decidable Decidable Warning: Rice s theorem applies to properties (i.e., sets of languages), not (directly to) TM s or other object types!
Undecidability + Rice + Church-Turing Language and Problems: any problem can be restated as a language recognition problem Undecidability: undecidable languages that cannot be decided by any Turing Machine Rice s Theorem: all nontrivial properties about the language of a TM are undecidable Church-Turing Thesis: any mechanical computation can be done by some TM Conclusion: any nontrivial property about general mechanical computations cannot be decided!
Some useful undecidable Problems Malware detection Virus Scanning Bug detection Program equivalence Type checking Solutions to polynomials N-body problems Fluid dynamics M Compressibility w Enscheidungsproblem Machine R: 1) Build (but don t run!) machine M x Machine M : 1) Simulate M(w) safely 2) Infect machine! 2) Return Malware( M )
Recognizing vs. Deciding Turing-decidable: A language L is decidable if there exists a TM M such that for all strings w: If w L, M enters q accept. If w L, M enters q reject. Computable Turing-recognizable: A language L is Turing-recognizable if there exists a TM M such that for all strings w: If w L eventually M enters q accept If w L either M enters q reject or M never terminates Non-Computable
Revised Hierarchy Recognizable Languages Decidable Languages finite Languages Regular Languages Deterministic CFLs
Example of a Recognizable Language The Halting Problem, or A TM Algorithm: Universal Turing Machine. M w Universal Turing Machine Output Tape for running TM-M starting on tape w
Closure Properties of Recognizable Languages Theorem: The recognizable languages are closed under. 1. Copy input 2. Run machine 1 on copy of input (never cross $) 1. If reject then move into reject state 3. Erase everything to right of $ M2 M1 4. Run machine 2 a b b a $ a b b a input copy
Closure Properties of Recognizable Languages Theorem: The recognizable languages are closed under union. 1. Copy input 2. Run machine 1 on copy of input (never cross $) 1. If accept then move into accept state 3. Erase everything to right of $ Problem: What if the machine runs forever? 4. Run machine 2
Closure Properties of Recognizable Languages Theorem: The recognizable languages are closed under union. 1. Copy input to second tape 2. Run machine 1 on tape 1, machine 2 on tape 2 in parallel, time-multiplexed 1. If either accept then move into accept state Time M2 M2 M1 M1 M1 M2
Closure Properties of Recognizable Languages Theorem: The recognizable languages are closed under concat. Using non-determinism: Tape 1: input choose Nondeterministically! a b b a Tape 2: tape for Machine 1 Tape 3: tape for Machine 2 a b b a Copy some characters from the first tape to tape 2 Nondeterminstically decide when to copy the rest onto tape 3
Closure Properties of Recognizable Languages Theorem: The recognizable languages are closed under Kleene star. Similar to concatenation, but now we also try all possible numbers of concatenations. Inserted Nondeterministically! a $ b b $ a Each substring then used as input to the machine Non-deterministically insert $ s into the input. The machine must accept each of these substrings.
Closure Properties of Recognizable Languages Theorem: The recognizable languages are not closed under complement. Hint: reduction from halting problem. This one takes some new tools
Co-Recognizable Languages Turing-recognizable: A language L is Turing-recognizable if there exists a TM M such that for all strings w: If w L eventually M enters q accept If w L either M enters q reject or M never terminates Co-Turing-recognizable: A language L is Co-Turing-recognizable if there exists a TM M such that for all strings w: If w L eventually M enters q accept If w L either M enters q reject or M never terminates A language L is Co-Recognizable if L is Recognizable.
Closure Properties of Recognizable Languages Theorem: If a language is both Recognizable and Co-Recognizable then it is decidable. Proof: Let M1 be a recognizer for L, M2 be a Co-Recognizer for L. We build MD, a Decider for L: Simulate M1 and M2 in parallel (time multiplexed)on input w. If M1 accepts then accept, else if M2 accepts, then reject. Every string will be accepted by exactly one machine in finite time. Time M2 M2 M1 M1 M1 M2
Closure Properties of Recognizable Languages Theorem: If a language is both Recognizable and Co-Recognizable then it is decidable. Corollary 1: A TM is unrecognizable since A TM is recognizable and undecidable. Corollary 2: The recognizable languages are not closed under complement.
Oracles Originated in Turing s Ph.D. thesis Named after the Oracle of Apollo at Delphi, ancient Greece Black-box subroutine / language Can compute arbitrary functions Instant computations for free Can greatly increase computation power of basic TMs E.g., oracle for halting problem
The Oracle of Omaha
The Oracle of the Matrix
A special case of hyper-computation Allows what if analysis: assumes certain undecidable languages can be recognized An oracle can profoundly impact the decidability & tractability of a language Any language / problem can be relativized WRT an arbitrary oracle Undecidability / intractability exists even for oracle machines! Turing Machines with Oracles Theorem [Turing]: Some problems are still not computable, even by Turing machines with an oracle for the halting problem!
Reaching a Contradiction Assume there exists some TM H that decides A TM *. * * Define D (<M>) = Construct a TM that: Outputs the opposite of the result of simulating * * H on input (M, <M>) * * * If M accepts its own description <M>, D(<M>) rejects. * * If M rejects its own description <M>, D(<M>) accepts. * M* is Relativized to O H* is Relativized to O D* is Relativized to O What happens if we run D on its own description, <D>? * * substituting D for M * * If D accepts its own description <D>, D(<D>) * * rejects. * * * * If D rejects its own description <D>, D(<D>) accepts.
Turing Degrees Turing (1937); studied by Post (1944) and Kleene (1954) Quantifies the non-computability (i.e., algorithmic unsolvability) of (decision) problems and languages Some problems are more unsolvable than others! Students of Alonzo Church: Alan Turing 1912-1954 H * Defines computation relative to an oracle. Georg Cantor 1845-1918 Emil Post 1897-1954 Relativized computation - an infinite hierarchy! H H A relativity theory of computation! Turing degree 2 Turing degree 1 Turing degree 0 Ø Stephen Kleene 1909-1994
Turing Degrees Turing degree of a set X is the set of all Turing-equivalent (i.e., mutually-reducible) sets: an equivalence class [X] Turing degrees form a partial order / join-semilattice [0]: the unique Turing degree containing all computable sets For set X, the Turing jump operator X is the set of indices of oracle TMs which halt when using X as an oracle [0 ]: Turing degree of the halting problem H; [0 ]: Turing degree of the halting problem H* for TMs with oracle H. Students of Alonzo Church: Alan Turing 1912-1954 Emil Post 1897-1954 Turing jump Turing jump Stephen Kleene 1909-1994
Turing Degrees Each Turing degree is countably infinite (has exactly 0 sets) There are uncountably many (2 0 ) Turing degrees A Turing degree X is strictly smaller than its Turing jump X For a Turing degree X, the set of degrees smaller than X is countable; set of degrees larger than X is uncountable (2 0 ) For every Turing degree X there is an incomparable degree (i.e., neither X Y nor Y X holds). There are 2 0 pairwise incomparable Turing degrees For every degree X, there is a degree D strictly between X and X so that X < D < X (there are actually 0 of them) Students of Alonzo Church: Alan Turing 1912-1954 Emil Post 1897-1954 The structure of the Turing degrees semilattice is extremely complex! Turing jump Turing jump Stephen Kleene 1909-1994
Not finitely describable Not Recognizable Recognizable EXPSPACE-complete =RE EXPTIME-complete Go PSPACE-complete QBF NP-complete SAT The Extended Chomsky Hierarchy 2 S*? H H* H Turing degrees Decidable Presburger arithmetic EXPSPACE EXPTIME PSPACE Context sensitive LBA NP P a n b n c n Context-free ww R Det. CF a n b n Regular a* Finite {a,b}