CSCE 551: Theory of Computation Chin-Tser Huang huangct@cse.sc.edu University of South Carolina
Church-Turing Thesis The definition of the algorithm came in the 1936 papers of Alonzo Church h and Alan Turing. Church s h definition iti used λ-calculus and Turing s definition used machines. Their definitions turned out to be equivalent. Since then the connection between the informal notion of an algorithm and the precise definition is called the Church- Turing thesis. Hilbert's Tenth Problem basically asks to construct a decider for the language: g D = {p p is a polynomial with an integer root} That is, for any given polynomial p, we need to decide whether p D (in a finite number of steps). 03/03/2010 2
Example: Finding an Integer Root A Turing machine for finding a integer root of a given polynomial (of a single variable x for simplicity) can be informally described as: Evaluate a given polynomial p(x) at x set successively to the values 0, 1, -1, 2, -2, 3, -3, If at any such value of x the value p(x) is zero, then accept. Q: Is this Turing machine recognizer or decider? For a univariate polynomial, it is possible to compute an upper bound for an absolute value of its integer root. If no root found below this bound, the polynomial has no integer roots, and its TM can reject (making it a decider solving the problem). However, as shown by Matijasevic, for multivariate polynomials no such bound can be computed, making it impossible to have a decider for this problem in general case. 03/03/2010 3
Description of TM Algorithms From now on, we focus on algorithms, and use TM as a precise model for the definition iti of an algorithm. Algorithms can be described in three ways: Formal : Explicit description of a Turing machine (its states, transition function etc.); Implementation : Description of TM using English prose, i.e., how TM moves head, read/writes symbols etc. (without explicitly specifying its states and transition function); High-level : Description of an algorithm using English prose, ignoring implementation details. We start with the first two levels and later will switch to the third one, when you feel confident on how to go from one level l of algorithm description to another. 03/03/2010 4
Example of High-Level Description Let A be the language consisting of all strings representing connected undirected d graphs. We write A = { G G is a connected undirected graph} Ahigh-level description of a TM M that decides A is as follows: M = On input G, the encoding of a graph G: 1. Select the first node of G and mark it. 2. Repeat the following stage until no new nodes are marked: For each node in G, G mark it if it is attached by an edge to a node that is already marked. 3. Scan all nodes of G to determine whether they all are marked. If they are, accept; otherwise, reject. 03/03/2010 5
Chapter 4: Decidability
Turing Machine: Languages Recall that a collection of strings that a TM M accepts is called the language of M or language recognized by M, M denoted L(M). Definition A language g is Turing-recognizable g (or recursively enumerable) ) if it is recognized by some Turing machine. A TM M on an input that is not in L(M) can either reject or loop. We distinguish TMs that never loop and call them deciders. A decider M decides the language that it recognizes. Definition A language is decidable (or recursive) if it is decided by some Turing machine. It is easy to see that the complement of a decidable language g is also decidable. 03/03/2010 7
Decidable Languages We will study some examples of languages concerning automata and grammars and show that they are decidable by algorithms Some problems of this kind are related to important t applications, such as compilers Some other problems concerning automata and grammars are not decidable; learning decidable languages first will help us understand undecidable languages 03/03/2010 8
Decidable Languages concerning RL Acceptance problems: A DFA = { B, w B is a DFA that accepts string w} A NFA = { B, w B is a NFA that accepts string w} A REX = { R, w R is a regexp that t generates string w} Emptiness testing: E DFA = { A A is a DFA and L(A) = } Equality testing: EQ DFA = { A, B A, B are DFAs and L(A) = L(B)} 03/03/2010 9
Deciding Languages concerning RL A DFA = { B, w B is a DFA that accepts string w} We need to present a TM that decides A DFA. Idea is to execute given B on the given input w within our TM. Q: How to implement such a TM? A few implementation details: Check that input represent a valid DFA B (a list of five components Q, Σ, δ, q 0, and F) and a valid string w over the alphabet of B (i.e., Σ). Make that TM store and maintain the configuration of B somewhere on its tape. Transitions from state to state in B can be done in a finite number of steps in TM. Check that execution of B on w will terminate at some point. 03/03/2010 10
Decidable Languages concerning RL A NFA = { B, w B is an NFA that accepts string w} A REX = { R, w R is a regexp that generates string w} Idea is to construct a TM that first converts a given NFA into an equivalent DFA (or first converts a given regexp into an equivalent NFA), and then executes this DFA as before. 03/03/2010 11
Decidable Languages concerning RL E DFA = { A A is a DFA and L(A) = } Idea is to construct a TM that view given DFA A as a directed graph with its vertices representing states and its edges representing transitions, and explores all paths from the start state in this graph. First, mark the start state of A. Then repeat the following until no new states t get marked: mark any state that has a transition coming into it from any state that is already marked. If no accept state t of A is marked, it accepts; otherwise it rejects. 03/03/2010 12
Decidable Languages concerning RL EQ DFA = { A, B A, B are DFAs and L(A) = L(B)} First notice that L(A) = L(B) is equivalent to emptiness of both set differences L(A) \ L(B) = L(A) L(B) and L(B) \ L(A) = L(B) L(A), and thus their union: (L(A) L(B)) (L(B) L(A)) called the symmetric difference of L(A) and L(B). The symmetric difference is a result of regular operations over L(A) and L(B), thus it represents a regular language recognized of some DFA C. C Therefore, we can design a TM that first constructs a DFA C from the given DFAs A and B, and then tests emptiness for L(C). 03/03/2010 13
Decidable Languages concerning CFL A CFG = { G, w G is a CFG that generates string w} Emptiness testing: E CFG = { G G is a CFG and L(G) = } But equality testing for CFG is NOT decidable: EQ CFG = { G, H G, H are CFGs and L(G) = L(H)} 03/03/2010 14
Decidable Languages concerning CFL A CFG = { G, w G is a CFG that generates string w} Q: Can we try to go through all derivation of G to determine whether any is a derivation of w? To show that A CFG is decidable, we need to ensure that the TM only tries finitely many derivations. It has been shown that if G is in Chomsky normal form, any derivation of w has 2n -1 steps, where n is the length of w. 03/03/2010 15
Decidable Languages concerning CFL E CFG = { G G is a CFG and L(G) = } Similar to the proof that A CFG is decidable, we cannot try all possible strings. Instead, we test whether the start variable can generate a string of terminals. First, mark all terminal symbols in G. Then repeat the following until no new variables get marked: mark any variable A where G has a rule A U 1 U 2 U k and each symbol U 1,, U k has already been marked. If no accept state of A is marked, it accepts; otherwise it rejects. 03/03/2010 16
CFL is Decidable Theorem Every context-free language is decidable. Proof Let A be a CFL. Let G be a CFG for A. We can make use of the TM we just constructed for G to decide whether any input w can be generated by G. 03/03/2010 17
Relationship among Languages RL CFL DECIDABLE RECOGNIZABLE 03/03/2010 18
Decidable vs. Undecidable Languages EQ DFA = { A, B A, B are DFAs and L(A) = L(B)} EQ CFG = { G, H G, H are CFGs and L(G) = L(H)} We proved that EQ DFA is decidable. In contrast to EQ DFA, it is not clear how to prove that EQ CFG is decidable. In fact, it is NOT decidable. But it is still unclear how to prove that. Our goal is to prove undecidability of another somewhat simpler language A TM = { M, w M is a TM and M accepts w} 03/03/2010 19
Decidable vs. Undecidable Languages A CFG = { G, w G is a CFG that generates string w} A TM = { M, w M is a TM and M accepts w} We proved that A CFG is decidable and that was used to prove that every context-free language is decidable (by simulating a TM for A CFG ). Q: What would happen if we prove that A TM is decidable? If A TM were decidable, the language of any TM would be decidable. In particular, recognizable languages would coincide with decidable languages (which is very unrealistic!). So, it is natural to expect that A TM is NOT decidable. 03/03/2010 20
Halting Problem It is easy to establish that A TM is recognizable: for a given pair M, w we need simply to simulate M on the input w, w and accept or reject depending on whether M accepts or rejects. Note that M may loop in which case our simulation will loop as well, making it just a recognizer not a decider. If we were able to determine whether M would ever stop on a given input w, a TM for A TM could reject given M, w as soon as it determined that M would loop on w. In this case it would be a decider for A TM (which accepts if M accepts; and rejects if M rejects or loops). Therefore, the crucial point is determination of whether M would ever stop on a given input w, giving the name halting problem to A TM.In these new terms, our goal is to prove that the halting problem is undecidable. 03/03/2010 21
Sets and Their Cardinalities It is easy to tell whether two given finite sets (such as {5, 7, 23} and {a, b, c}) have the same size -- just count up their elements and compare the counts. But is there a way to compare sizes of two infinite sets? Obviously, counting elements won't help as it would never end. But can we determine that two sets have equal sizes without counting their elements? 03/03/2010 22
Sets and Their Cardinalities Cantor proposed a technique called diagonalization to solve this problem. The idea is to set up a mapping between the elements of two sets such that each element of one set is mapped to an element of the other, and vice versa. This idea works equally well for finite and infinite sets. For example, we can map elements of the two sets above as follows: 5 a, 7 b, and 23 c; implying that these sets are the same size. 03/03/2010 23
One-to-one, one, Onto, Correspondence Let A, B be two sets and f : A B be a function from A to B. Definition f is called one-to-one (injective) if it maps different elements of A to different elements of B. Definition f is called onto (surjective) if for every elements b B, there exists a A such that f (a) = b. b Definition f is called correspondence (bijective)( ) if it is one-to-one and onto. Two set are of the same size if there exists a correspondence between them. 03/03/2010 24
An Example Which of the following two sets, E = {2, 4, 6, } of even positive integers and N = {1, 2, 3, } of natural numbers, have a larger size? It turns out they are of the same size! Why? Define f : N E such that f (n) = 2n for every n N. It is one-to-one since for n m we have 2n 2m. It is onto since for k E, k /2 is a natural number and f (k/2) = k. k Hence, f is a correspondence between E and N, implying that they are of the same size. Definition A set is called countable if either it is finite or has the same size as N. 03/03/2010 25
Announcement Homework 2 is due TODAY in class; ;please submit it before you leave Reading assignment: Ch. 4.2 Happy spring break! 03/03/2010 26