CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY

CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY RYAN DOUGHERTY If we want to talk about a program running on a real computer, consider the following: when a program reads an instruction, the contents of the memory may change based on what the current memory contents were, and what the input is. In fact, we can conceive of a program as a state machine that has some sort of memory component. Turing Machines are a particular type of state-based model of computation. Specifically, a Turing machine (TM) has a finite control via states and an infinite tape. The first usage in the literature about TMs, oddly enough, comes from Turing s famous paper. 1 In this class, our basic model will have a tape that is infinite only in one direction. A Turing machine starts up with the input string written at the left end of the tape. The rest of the tape is filled with a special blank character (i.e., ). Initially, the head is located at the first tape position and the start state of the machine. Thus, the initial configuration of a Turing machine for the input cse355 is as follows. Tape: c s e 3 5 5... Read/Write tape head Each step of the Turing machine first reads the symbol on the cell of the tape under the head. Depending on the symbol and the current state of the controller, it then writes a symbol at the current tape position, moves either left or right, and changes state. For example, the following transition is taken if the controller is in state q and the symbol under the read head is c. It replaces the c with the character f and then moves right, switching the controller to state r. We will design Turing machines to be deterministic. That is, once we know the state and which symbol is under the read/write head, there is exactly one choice for what the machine can (and must) do. This leads to the interesting observation that since there is always a transition to do, then the machine will run forever; in the definition, we will handle when the machine should stop. For now, we proceed with an example. Date: Last Updated July 11, 2018. 1 Turing, Alan Mathison. On computable numbers, with an application to the Entscheidungsproblem. Proceedings of the London mathematical society 2.1 (1937): 230-265. No, he did not call them Turing machines. 1

2 RYAN DOUGHERTY Example 1. Consider the language Let s describe a TM for L. L = {0 n $0 n : n 0}, Definition 1. A Turing Machine (TM) is a 7-tuple (Q, Σ, Γ, δ, q 0, q accept, q reject ) where: Q is a finite set of states; Σ Γ is a finite input alphabet; Γ is a finite tape alphabet which contains a symbol / Σ; δ : (Q \ {q accept, q reject }) Γ Q Γ {L, R} is the transition function; q 0 Q is the start state; q accept Q is the accept state; q reject Q with q accept q reject is the reject state. For a TM, the input provided to the TM is on an infinite one-way tape, leftadjusted. We now define what it means for a TM to accept (or reject) a given input. Example 2. Can you determine the language of this TM?

CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY 3 x R 0 L R, 1 R x R,1 R 0 L x L 0, R 1 x, R L q 1 q 2 q 3 q 5 R x R 1 R R 0 R q reject q accept q 4 0 x, R x R 1 L R Exercise: show that the above TM always halts on any input. Another nice exercise: given an input of length n, what is the run-time of the above TM in terms of n? Definition 2. A configuration C of a TM currently in state q and tape contents s 1 s k and observing tape cell i is the string s 1 s i 1 qs i s k. Definition 3. A computation of a TM M on input w is a sequence of configurations C 0, C 1, C s where: C 0 is the starting configuration (i.e., its state is q 0 ) C i is obtained from C i 1 by a single transition of M for all i (often said that C i 1 yields C i ). Suppose that the original state is q and goes to state q. (1) C i 1 = s 1 s i 1 qs i s i+1 s k and the next transition moves right and writes symbol b, then C i = s 1 s i 1 bq s i+1 s k. (2) C i 1 = s 1 s k 1 qs k and the next transition moves right and writes symbol b, then C i = s 1 s k 1 bq. (3) C i 1 = s 1 s i 2 s i 1 qs i s k and the next transition moves left and writes symbol b, then C i = s 1 s i 2 q s i 1 b s k. A computation is accepting if the last configuration contains q accept, and is rejecting if it contains q reject. Definition 4. The language recognized by M, L(M), is the set of all strings w such that M has an accepting computation on w. A language is Turing-recognizable (or recognizable) if there is some TM that recognizes it. Unfortunately, it may be possible for a TM to run forever. We will make a distinction now about TMs that can run forever and the ones that do not: Definition 5. A TM is a decider if it halts on all inputs (i.e., doesn t run forever, and says accept or reject in a finite amount of time). A language L is Turingdecidable (or decidable) if some decider has language L. Some languages that are decidable are: Σ

4 RYAN DOUGHERTY Any finite language The language {0 n $0 n : n 0} from earlier. The language of the machine in Example 2. Theorem 1. Any decidable language L is also recognizable. Proof. Follows from the definition. It will turn out that the converse is not true (i.e., there are recognizable languages that are not decidable). Like writing assembly language explicitly, it is a good idea to do it once or a few times to get the general understanding of how it works; but after that, it is much more convenient and useful to stay in a higher-level programming language. The same is true here, we will write high-level descriptions of TMs moving forward (unless specific details are needed): Example 3. Let L 1 = {0 2n : n 0} ( powers of 2 ). Here is a high-level description of a TM to decide L 1 : M = On input w: (1) Sweep left and right across the input, crossing off every other 0 (by writing a character x). (2) If in step 1 we only find a single 0, accept. (3) If in step 1 we find more than one 0 and the number of 0s was odd, reject. (4) Return the tape head to the left end of the tape, and go back to Step 1. Piazza Question 1. Let L 2 = {a i b j c k : i j = k}. Make a TM that decides L 2. Example 4. Let L 3 = {#w 1 #w 2 # #w k # : w i {0, 1}, i j w i w j }. Here is a high-level description of a TM to decide L 3 : M = On input w: (1) If the leftmost tape symbol is, accept. If it is not #, reject. If it is #, mark it. (2) Scan right to the next # and place a marker on it (with a special symbol indicating so); if not found, accept. (3) Zig-zag across the adjacent strings to the marked # symbol to check for equality (by marking symbols in each). If the strings are the same, reject. If not, then restore the strings without marked symbols. (4) Move to the next pair of strings by moving the second # marker. If none found, move the first marker forward to the next #, and the second # marker the one after that. If none is found, accept. (5) Go back to Step 3. Piazza Question 2. Are L 1, L 2, L 3 decidable? Now we make characterizations as to what languages are decidable or recognizable. When a class of languages C has the property that for any L 1, L 2 C and a specific operation has that L 1 L 2 C, then we say that C is closed under. Theorem 2. Let L 1, L 2 be recognizable (resp. decidable) languages. Then L 1 L 2, L 1 L 2, L 1 L 2, L 1 are recognizable (resp. decidable) languages. If L 1 is decidable, then L 1 is also decidable.

CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY 5 Showing the above theorem will have that the recognizable (and decidable) languages are closed under intersection, union, concatenation, and star. Furthermore, decidable languages are closed under complement. Proof: Piazza Question 3. Why does the above idea not work to show recognizable languages are closed under complement? There is a characterization between recognizable and decidable languages: Theorem 3. L is decidable if and only if L is recognizable and L is recognizable. Proof:

6 RYAN DOUGHERTY 1. TM Variants To show evidence for why TMs are a good model for our everyday use of computers, we want to show that for any reasonable modification to the definition of a TM, that it is equivalent to the standard definition. By this, we want to show that a normal TM can decide/recognize a language if and only if the variant can decide/recognize it. When this happens, we say that the variant is equivalent to the standard model. Example 5. A stay-put TM (STM) is one that can move left or right, but also stay put and not move the tape head. Formally, its transition function is δ : Q Γ Q Γ {L, R, S}, and the next configuration has the tape head at the same location. Theorem 4. STMs are equivalent to TMs. Proof. A STM can simulate a TM (it just has no stay-put transitions). For the other direction, it suffices to show we can simulate a stay-put transition with only left/right transitions. We can do this by moving right, then moving left, without changing the symbol on the right. Example 6. A multitape TM is one that has a fixed number k of tapes that all operate independently. A transition moves all of the tapes, but we also allow for the tape heads to stay put instead of moving left or right. Theorem 5. Multitape TMs are equivalent to TMs. Proof: it s clear that a multitape TM can simulate a normal TM, since it only uses one tape (the multitape TM can do whatever it wants on the other tapes). So we show how to simulate a multitape TM using a normal TM: Example 7. Nondeterminism is where the TM has multiple choices it can make for transitions, it picks the correct one, and continues from there. Formally, nondeterminism means that if there is a choice that will eventually lead to acceptance, the TM will pick that choice (and if not, it makes an arbitrary choice).

CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY 7 Formally, the transition function is: δ : Q Γ P(Q Γ {L, R}). Theorem 6. Nondeterministic TMs (NTMs) are equivalent to TMs. Proof: it s clear that a NTM can simulate an arbitrary TM (it isn t required to use multiple choices for transitions). Now we show that a TM can simulate an arbitrary NTM: Definition 6. A TM M is an enumerator for a language L if M, starting with empty tape, lists out strings in L so that for any string w L, M will eventually print out w. A language L is enumerable if there is an enumerator for L. Piazza Question 4. Give an enumerator for Σ (as a high-level algorithm).

8 RYAN DOUGHERTY You may wonder if there is a relation between enumerable languages and decidable/recognizable ones. In fact, there is: Theorem 7. A language L is enumerable if and only if L is recognizable. Proof. We prove this in both directions: 2. Church-Turing Thesis One of the very interesting aspects of Turing Machines is that they do compute and solve the same problems as what we can with everyday computers. This was established through the works of Alonzo Church and Alan Turing in the 1930s, which essentially stated the following: The languages decided by Turing Machines are exactly those that correspond to an intuitive computer algorithm. Piazza Question 5. This is not a theorem that can be proven why? Let s establish why this statement is true, however:

CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY 9 So in fact, if we have a language L we are interested in, and show that it is undecidable for Turing Machines, the Church-Turing thesis says that that is impossible to write an algorithm for on a computer. 3. Undecidability of Problems We have seen several examples of languages that are decidable (and hence recognizable). Here, we show that certain problems are undecidable. The main result we will show is the following: Theorem 8. Let A T M = { M, w : M is a TM and M accepts w}. Then A T M is undecidable. We proceed in steps. Theorem 9. A T M is recognizable. Remember that a recognizer only has to halt on the strings M, w that are in A T M, not guarantee to halt on these inputs that are not (a decider has to halt on both these cases). Proof:

10 RYAN DOUGHERTY Theorem 10. A T M is a recognizable, but not decidable language. Proof: So how do we go about proving that A T M is undecidable? We will proceed with proof by contradiction; i.e., we start with a supposed TM that decides A T M, and then we will at some point derive a contradiction. First, a high-level overview. Consider the language { M : M is a TM with at least 7 states}. Note that this is decidable; why? Because we can look through the input TM given and count the states, and accept iff this number is at least 7. So TMs can in fact answer questions about other TMs. Now onto the proof.

CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY 11 But we also prove a striking result between decidable and recognizable languages: Theorem 11. L is decidable if and only if L and L are recognizable. Proof: Piazza Question 6. What can we conclude about A T M? Using the undecidable result of A T M, we can in fact show that some other languages are also undecidable through reductions. Definition 7. A reduction from A to B, written A B, is a computable (i.e., finite time to compute) function f : Σ Σ such that x A if and only if f(x) B. Theorem 12. If A B and B is decidable (resp. recognizable), then A is decidable (resp. recognizable). If A B and A is undecidable (resp. unrecognizable), then B is undecidable (resp. unrecognizable). Proof. The second is the contrapositive of the first, and hence is equivalent.

12 RYAN DOUGHERTY We now apply reductions to show that other languages are undecidable: Theorem 13. E T M = { M : M is a TM and L(M) = } is undecidable. Proof. Suppose E decides E T M. We now make a decider for A T M as follows: It would be useful in programming (such as a grader for a course of that nature) to determine if a given program is equivalent to the sample solution you have; i.e., it has the same behavior on all inputs. Unfortunately, deciding whether this is true is impossible: Theorem 14. EQ T M = { M 1, M 2 : M 1, M 2 are TMs and L(M 1 ) = L(M 2 )} is undecidable. Proof. We construct a reduction E T M EQ T M.

CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY 13 Piazza Question 7. What about ALL T M = { M : L(M) = Σ }? 4. Rice s Theorem The above undecidability proofs make it seem as though there is a general theme underlying them: we construct a new TM that has different behaviors based on what the input is. For example, for A T M E T M the machine we constructed had nonempty language if and only if M accepts w. In fact, there is something more general that can be said, which is Rice s theorem. Definition 8. A property of TM languages P is a language of TM descriptions such that if M 1, M 2 are arbitrary TMs with L(M 1 ) = L(M 2 ), then either: (1) M 1, M 2 P, or (2) M 1, M 2 / P. In other words, the language of TM descriptions is of the form { M : M is a TM and L(M) has some property}. If we have two machines with the same language, then it cannot be the case that one of them has the property (is in the language), and the other does not have the property (is not in the language). Definition 9. A property of TM languages P is nontrivial if there exist TMs M 1, M 2 such that M 1 P, M 2 / P. Now we can prove Rice s theorem. Theorem 15 (Rice s Theorem). Every nontrivial property of TM languages is undecidable. As a corollary, we immediately can prove, using the theorem, that E T M is undecidable. Proof. Let P be such a nontrivial property, and we will show A T M P.

14 RYAN DOUGHERTY 5. LBAs We consider the restriction of TMs where we can only use the tape where the input is initially given. Formally, if the input is w 1 w n, the tape contents are w 1 w n, where, are special characters to help determine when the tape head is at the beginning or the end of the tape. Also, the TM is not allowed to write, anywhere other than the beginning and end. This type of machine is called a linear bounded automaton (LBA), and other than the tape, behaves identically to a standard TM. The use of such an object would be that we restrict the amount of space used, such as if we are dealing with a large data set and cannot afford more space than what we are given. One may ask the question of whether LBAs decide/recognize the same languages as TMs. In fact, they are different! 2 Hopefully we get to time complexity where we can prove using a powerful theorem there. And the following theorem certifies that this must be true for recognizable languages, but not necessarily decidable ones. Theorem 16. A LBA = { M, w : M is an LBA and M accepts w} is decidable. Proof. Suppose M has n states, q tape symbols, and the input w has length g. But they behave similarly to TMs with regards to emptiness: Theorem 17. E LBA is undecidable. Proof. Let an accepting computation history (ACH) for a given TM M (yes, I mean TM) and input w to be a string #C 0 #C 1 # #C m # where C 0 is a start configuration, C m is an accepting computation, and C i is obtained from C i 1 via a single transition. Consider the language, for a given M and w, L M,w = {w : w is an accepting computation history of M on w}. 2 If you re interested, the languages of LBAs are called context-sensitive languages, named for a completely different reason than LBAs.

CSE355 SUMMER 2018 LECTURES TURING MACHINES AND (UN)DECIDABILITY 15 This seems to suggest that even restricting to just the input for space is not very useful, in the sense that we cannot even determine whether or not an LBA actually accepts anything! We would like to have a model that is maybe not as computationally powerful as an LBA, but we can get potentially many more problems that we could solve. Piazza Question 8. What about EQ LBA?