Computability and Complexity

Computability and Complexity Decidability, Undecidability and Reducibility; Codes, Algorithms and Languages CAS 705 Ryszard Janicki Department of Computing and Software McMaster University Hamilton, Ontario, Canada janicki@mcmaster.ca Ryszard Janicki Computability and Complexity 1 / 34

Some Denitions The problem is decidable if there is an algorithm that takes an instance of the problem and determines in a nite number of steps that instance is YES or NO. The problem is undecidable is such an algorithm does not exist. Formally, a problem is decidable if the language associated with problem is recursive. Why, and what does it mean? Ryszard Janicki Computability and Complexity 2 / 34

Decidable Problems Let L be a regular or context-free language. 1 The problem if x L is decidable. 2 The problem if L = is decidable. 3 The problem if L = is decidable. 4 The problem if 0 < L < is decidable. Ryszard Janicki Computability and Complexity 3 / 34

Reducibility How to show that a problem is undecidable? How to show that an algorithm does not exists? Maybe it just has not been invented yet? Suppose that somehow it was proven that a problem A is not decidable. I have a problem B to consider and I suspect B is not decidable. I try to prove the following result: Any algorithm that solves B can be modied in such a way that it solves A. But A is undecidable, so no algorithm to solve A do exist! Hence B is also undecidable. A is reduced to B! B is decidable = A is decidable. A is undecidable = B is undecidable. Ryszard Janicki Computability and Complexity 4 / 34

Denition (Post Correspondence Problem) Let Σ be an alphabet and A, B are two lists of sequences over Σ, i.e. A = w 1,..., w k, B = x 1,..., x k, x i, w i Σ. Does there exist any sequence of integers i 1, i 2,..., i m, m 1, such that w i1 w i2... w im = x i1 x i2... x im? The above part is blue is called `Post Property'. Theorem The Post Correspondence Problem is undecidable if Σ 2. Example w 1 = a, w 2 = abaaa, w 3 = ab x 1 = aaa, x 2 = ab, x 3 = b m = 4, i 1 = 2, i 2 = 1, i 3 = 1, i 4 = 3 w 2 w 1 w 1 w 3 = abaaaaaab = x 2 x 1 x 1 x 3 so these two lists have the Post property. Ryszard Janicki Computability and Complexity 5 / 34

The trick is to nd some problem that is undecidable, at least one, then we can use REDUCTION. Theorem Let L 1, L 2 be context-free grammars over Σ such that Σ 2. Then the? problem L 1 L 2 = is undecidable. Proof. Assume it is decidable. Consider A = x 1,..., x n, B = y 1,..., y n, x i, w i Σ, c, d, e / Σ. Dene f : {1,..., } de as: f (i) = de i = d e... e. }{{} i times Dene G x = ({σ x }, Σ, σ x, P x ), where Σ = Σ {d, c, e}, and σ x f (1)σ x x 1 f (2)σ x x 2... f (n)σ x x n c and dene G y = ({σ y }, Σ, σ y, P y ), σ y f (1)σ y y 1 f (2)σ y y 2... f (n)σ y y n c. Note that L(G x ) L(G y ) = lists A and B have Post property. Hence any algorithm for nding L 1 L 2 = will solve the Post Correspondence Problem, which is undecidable! Ryszard Janicki Computability and Complexity 6 / 34

How to nd this rst undecidable problem? How should we think? Some ideas: Every nite entity can be described as a string. Every book is a string. Jorge Luis Borges, Babel Library, 1941 Each book consists of 409 pages; each page 40 lines; each line about 80 letters of black colour... There is no two identical books in the library Library is total... It describes everything and in all languages. Everything: detailed history of the past... Every book in Borges library is a string. L = all books in Borges library, L is nite! Ryszard Janicki Computability and Complexity 7 / 34

All Turing machines may be coded as strings. All congurations of a given Turing machines can be coded as strings. We just need to code the current state, current state position, perhaps current tape number, etc. Each primitive recursive or recursive function is fully described by its derivation which can be coded as a string. For example: add(x, 0) = x add(x, y + 1) = s(sdd(x, y)) can be coded as: #add(x, 0) = x#add(x, y + 1) = s(sdd(x, y)), which is a string over Σ, where Σ = {,, #, a, d, x, 0, (, ), y, +, 1, s}. Algorithm does not exist if its code does not exists! Ryszard Janicki Computability and Complexity 8 / 34

Languages Dened by Problems For any concept C (i.e. graph, primitive recursive function, Turing machine, etc.), C denotes a code of C. Details of coding are not important, but we assume its existence, and one coding technique for a given concept. C is a string of symbols, i.e. C Σ for some nite Σ. A = { G G is a connected undirected graph } B = { PR PR is a primitive recursive function} Both A and B are languages, i.e. A Σ A, B Σ B, where Σ A, Σ B are alphabets of codings. Ryszard Janicki Computability and Complexity 9 / 34

Denition (Another Decidability) Problem P is decidable i the language A P = { P P} is Turing-decidable. Example Assumption Turing machines understand codings A DFA = { B, w B is deterministic automaton that accepts w } B, w is a code of deterministic nite state automaton that accepts input strings w. Proposition A DFA is Turing-decidable. Proof. Turing machine simulate B on w, if B ends in its nal state, Turing machine accepts, otherwise Turing machine rejects. Ryszard Janicki Computability and Complexity 10 / 34

A REX = { R, w R is a regular expression that generates w } Proposition A REX is Turing-decidable. Proof. Using standard vocabulary: the problem: `if a given regular expression R generates a string w is decidable. Turing machine takes R and w as input. It converts R into non-deterministic automaton and next into deterministic automaton. Then it simulates the obtained deterministic automaton on w. If simulations ends in nal state, it accepts, otherwise it rejects. Decidability of problems Turing Decidability of languages of their codings. Ryszard Janicki Computability and Complexity 11 / 34

A CFG = { G, w G is a context-free grammar that generates w } Proposition A CFG is Turing-decidable. Proof. Turing machine takes G and w as input. It converts G into Chomsky normal form (with ε here). Next Turing machine list all derivations with 2n 1 steps, where n = w, unless n = 0, where all derivations with one step are given. Now Turing machine checks all these derivations, if one of them deriver w, then accept, otherwise reject. Ryszard Janicki Computability and Complexity 12 / 34

? - maybe A L1 L 1 = { L 1, L 2 L 1, L 2 are context-free and L 1 L 2 = }? Ryszard Janicki Computability and Complexity 13 / 34

A TM = { M, w M is a Turing Machine and M accepts w } Theorem A TM is undecidable. Theorem A TM is recognizable. Proof. Let U be a Turing machine (called universal Turing machine), that takes M and w as input. Next U simulates M on w, and, if M accepts, the U accepts, if M rejects, then U rejects. But U loops when M loops, so U cannot decide A TM The construction of universal Turing machine is intuitively obvious: rst copy, then execute. Ryszard Janicki Computability and Complexity 14 / 34

Problem: We know that A B. How can we show that A B? Method 1 (explicit) We just construct x such that x B but x / A. Method 2 (implicit) Suppose that A and B are nite and suppose that we can calculate A and B. Then, B > A = B A. What if A and B are innite? Ryszard Janicki Computability and Complexity 15 / 34

Countable and Uncountable sets Let N = {1, 2, 3,...} be the set of `natural numbers'. Denition A set X is countable i it is nite or there is a one-to-one function f : N X. Proposition (Basic Properties) Assume A, B are countable. Then A B is countable. A B is countable. A B is countable. If A B and B is countable, then A is countable. Denition A set X is uncountable i it is not countable. Ryszard Janicki Computability and Complexity 16 / 34

Uncountable sets are so big that they cannot uniquely be labeled by natural numbers. The set of real numbers is uncountable. If X is countable but innite 2 X = {y Y X } is uncountable. Now, if A B, A and B are not nite; if A is countable and B is not countable, then A B. Ryszard Janicki Computability and Complexity 17 / 34

Theorem Some languages are not Turing-recognizable. Proof. Let Σ be any (nite) alphabet. Then the set Σ is countable. For example the lexicographical enumeration (just rst order arbitrarily Σ from 1 to Σ ) is a possible mapping from N. We will show that the set of all Turing machines working on languages over Σ is countable. We assume that each M, a code for the machine M, is built from Σ = Σ Σ s, where Σ s is a set of special symbols that make coding easier. Hence for each M, M (Σ ). This means { M M is Turing machine } (Σ ). Since (Σ ) is countable, { M M is Turing machine } is countable too. However the set 2 Σ is not countable, hence there is L 2 Σ, i.e. L Σ which is not recognizable by any Turing machine. Ryszard Janicki Computability and Complexity 18 / 34

Simply, there is much more languages than Turing machines (as Turing machines correspond to strings). In fact, there is much more languages that are not Turing-recognizable, than that are Turing-recognizable. We just cannot explicitly dene them. One example will be given later. Ryszard Janicki Computability and Complexity 19 / 34

Theorem The language: A TM = { M, w M is Turing Machine and M accepts w } is not Turing-decidable (i.e. the problem is undecidable). Proof. Suppose A TM is decidable. Let H be a Turing machine that decides A TM, i.e. for any input M, w, H accepts M, w if M accepts w, and H rejects M, w if M either rejects w or loops. In other words: { accept M accepts w H( M, w ) = reject M does not accept w Note that the question: `Does M accept M?' is a valid question. We do not have any assumption about w, it is just a string. M is also just a string (a code of M). So we can call H( M, M ) for any Turing machine M. Ryszard Janicki Computability and Complexity 20 / 34

Let D be the following Turing machine: 1 D reads M 2 D calls H( M, M ) 3 If H return accept, D produces reject, if H return reject, D produces accept. In other words: { accept M does not accept M D( M ) = reject M accepts M Now consider D( D ). { We have accept D does not accept D D( D ) = reject D accepts D A contradiction, so such D does not exist. But if we have H, D does exist, so H does not exist either. Ryszard Janicki Computability and Complexity 21 / 34

This result is often formulated as: Theorem (self acceptance problem) The language: A STM = { M M is Turing machine and M accepts M } is undecidable. The proof is almost identical Then we can use reduction to show that A TM is also undecidable. Ryszard Janicki Computability and Complexity 22 / 34

Denition A language L Σ is co-turing-recognizable if Σ \ L is Turing recognizable, i.e. the there exists Turing machine M, such that L(M) = Σ \ L. Theorem L is decidable if it is Turing-recognizable and co-turing-recognizable. Proof. Let M 1, M 2 be Turing machines such that L(M 1 ) = L and L(M 2 ) = Σ \ L. Dene M as follows: 1 read w 2 run M 1 and M 2 on w in parallel (one step of M 1 followed by one step of M 2 ). 3 if M 1 accepts then accept, if M 2 accepts, reject. Ryszard Janicki Computability and Complexity 23 / 34

Corollary Σ \ A TM is not Turing recognizable. Proof. A TM is Turing recognizable. If Σ \ A TM were is Turing recognizable, then A TM would be decidable, but it is not. Ryszard Janicki Computability and Complexity 24 / 34

Reducibility Again If problem A reduces to B, we can use a solution to B to solve A. Reducibility says nothing about solving A or B alone, only about the solvability of A in the presence of a solution to B. When A is reducible to B: 1 solving A cannot be harder than solving B 2 if B is decidable, A is also decidable. 3 if A is not decidable, B is undecidable. Most of undecidability results have been obtain using (3) above. One can use reducibility either directly, or via languages and codings. Ryszard Janicki Computability and Complexity 25 / 34

Example While using coding and languages is essential to prove fundamental results as A TM is undecidable or A STM is undecidable, when using reducibility, it is often easier, and more intuitive, to switch back to the language of algorithms and problems. A TM = { M, w M is Turing Machine and M accepts w } is undecidable can alternatively be formulated as follows: Theorem (Acceptance Problem) Let M be a Turing machine and w Σ. The problem w? L(M) is undecidable. Note that we do not have such an alternative formulation for A STM = { M M is Turing machine and M accepts M } as here the use of M is essential not only for proofs but for the problem formulation as well. Ryszard Janicki Computability and Complexity 26 / 34

Halting Problem Theorem Let: HALT TM = { M, w M is a Turing machine and M halts on w } The language HALT TM is undecidable. Proof. We will show that A TM is reducible to HALT TM. Suppose that { a Turing machine R decides HALT TM, so accept M halts on w R( M, w ) = reject M loops on w Let S be a Turing machine that works as follows: 1 takes M, w as input 2 run R on M, w 3 if R rejects, reject 4 if R accepts, simulate M on w until it halts 5 if M accepts, accept; if M reject, reject. Ryszard Janicki Computability and Complexity 27 / 34

Proof. In other words: reject R( M, w ) = reject S( M, w ) = reject R( M, w ) = accept M(w) = reject accept R( M, w ) = accept M(w) = accept Note that S decides A TM, but A TM is undecidable, so such R does not exist. Hence HALT TM is undecidable. Using dierent language and approach might result in easier proofs. Theorem (Dierent formulation of Halting problem) Let T be a Turing machine and let w Σ. The problem:'does M halts on w?' is undecidable. Proof. Suppose that it is decidable. So we can decide if either w L(M) or w / L(M). Hence we can clearly decide if w L(M), when it w is really in L(M). But this problem is undecidable. Ryszard Janicki Computability and Complexity 28 / 34

Theorem The language E TM = { M M is Turing machine and L(M) = } is undecidable. Theorem (Alternative Formulation) The problem `if for a given Turing machine M, L(M) = ' is undecidable. Ryszard Janicki Computability and Complexity 29 / 34

Theorem (Alternative Formulation) The problem `if for a given Turing machine M, L(M) = ' is undecidable. Proof. Suppose we have an algorithm that can decide if L(M) =. We have such algorithms for regular and context-free languages. We may assume that L(M) Σ. Let w Σ. We construct a Turing machine M w as follows reject x w M w (x) = accept x = w M(w) = accept reject x = w M(w) = reject If M loops for w, so does M w. Note that: L(M w ) = {w} w L(M) L(M w ) = w / L(M). Suppose that we have w and M. We apply our algorithm to M w, and if L(M w ) = then w / L(M) and if L(M w ) then w L(M), so we can decide halting problem, which is not decidable. Ryszard Janicki Computability and Complexity 30 / 34

Theorem The language REGULAR TM = { M M is Turing machine and L(M) is regular} is undecidable. Theorem (Alternative Formulation) The problem `if a given language accepted by a Turing machine is regular' is undecidable. Ryszard Janicki Computability and Complexity 31 / 34

Theorem (Alternative Formulation) The problem `if a given language accepted by a Turing machine is regular' is undecidable. Proof. Suppose that this problem is decidable. We will show that we can the decide if M accepts w. Let M be a Turing machine and let w be a string. We dene M w such that M w (x) = accept x {a n b n n 0} accept x / {a n b n n 0} M accepts w reject x / {a n b n n 0} M rejects w If M loops{ for w, so does M w. Hence: {a L(M w ) = n b n n 0} M does not accept w Σ M accepts w Now, when we apply our algorithm to M w and if the answer is `yes', we know that M accepts w, which is undecidable. Ryszard Janicki Computability and Complexity 32 / 34

Rice's Theorem We can replace `regular' by `context-free', `decidable', `nite', etc. All proofs have the similar structure. Theorem (Rice's Theorem) Determining any nontrivial property of the languages recognized by Turing machines is undecidable. Ryszard Janicki Computability and Complexity 33 / 34

Theorem The language EQ TM = { M 1, M 2 M 1, M 2 are Turing machines and L(M 1 ) = L(M 2 )} is undecidable. Theorem (Alternative Formulation) The problem `if two Turing Machines recognize the same language' is undecidable. Proof. Suppose this problem is decidable. Let M be a Turing machine that accepts nothing, i.e. L(M ) =. Then, by using the decider for M 1 = M and M 2 = M, we can decide the problem `L(M)? = ', which is undecidable. Ryszard Janicki Computability and Complexity 34 / 34