Theory of Computation - PDF Free Download

Thomas Zeugmann Hokkaido University Laboratory for Algorithmics http://www-alg.ist.hokudai.ac.jp/ thomas/toc/ Lecture 14: Applications of PCP

Goal of this Lecture Our goal is to present some typical undecidability results for problems arising naturally in formal language theory. Here our main focus is on context-free languages but we shall also look at regular languages and the family L 0.

Undecidability Results for Context-free Languages I As we have seen in Lecture 6, there are context-free languages L 1 and L 2 such that L 1 L 2 CF. So, it would be nice to have an algorithm which, on input any two context-free grammars G 1, G 2, returns 1, if L(G 1 ) L(G 2 ) CF, and 0 otherwise. Unfortunately, such an algorithm does not exist. Also, some closely related problems cannot be solved algorithmically as our next theorem states.

Undecidability Results for Context-free Languages II Theorem 1 The following problems are undecidable for any context-free grammars G 1, G 2 : (1) L(G 1 ) L(G 2 ) =, (2) L(G 1 ) L(G 2 ) is infinite, (3) L(G 1 ) L(G 2 ) CF, (4) L(G 1 ) L(G 2 ) REG.

Proof I Proof. The general proof idea is as follows. We construct a context-free language L S (here S stands for standard) and for any PCP [{a, b}, n, P, Q] a language L(P, Q) such that L S L(P, Q) if and only if [{a, b}, n, P, Q] is solvable. Let Σ = {a, b, c} and define L S = {pcqcq T cp T p, q {a, b} +, c Σ}. (1) Furthermore, for any p 1,..., p n, where p i {a, b} +, we set L(p 1,..., p n ) = {ba i k b ba i 1 cp i1 p i2 p ik k 1, j[1 j k 1 i j n]}. Here the idea is to encode in a i j the index i j.

Proof II Now, let [{a, b}, n, P, Q] be any PCP, then we define the language L(P, Q) as follows: L(P, Q) = L(p 1,..., p n ){c}l T (q 1,..., q n ). (2) Claim 1. L S, L(p 1,..., p n ) and L(P, Q) are context-free. First, we define a grammar G S = [{a, b, c}, {σ, h}, σ, P], where the production set P is defined as follows: σ aσa σ bσb σ chc h aha h bhb h c

Proof III Clearly, G S is context-free. Furthermore, we have: σ wσw T wchcw T wcvhv T cw T wcvcv T cw T, where w, v {a, b} +. Hence L(G S ) L S. The inclusion L S L(G S ) is obvious. Consequently, L S CF. Next, let P = [p 1,..., p n ]. We define a grammar G P = [{a, b, c}, {σ}, σ, P], where P = {σ ba i σp i i = 1,..., n} {σ c}. Clearly, G P is context-free, L(G P ) = L(p 1,..., p n ) and thus L(p 1,..., p n ) CF.

Proof IV By Theorem 6.4 we know that CF is closed under transposition. Therefore, we can conclude that L T (p 1,..., p n ) CF, too. Moreover, CF is closed under product. Consequently, L(P, Q) is context-free for any PCP. This proves Claim 1. Claim 2. For every PCP we have: L S L(P, Q) if and only if [{a, b}, n, P, Q] is solvable. Necessity. Let L S L(P, Q) and consider any string r L S L(P, Q), i.e., r = ba i k b ba i 1 } {{ } c p i1 p ik c (q j1 q jm ) T c (ba jm b ba j1 ) T } {{ }} {{ }} {{ } w 1 w 3 w 4 w 2 Since r L S, we directly see that w 1 = w T 2 and w 3 = w T 4. Consequently, we get k = m and i l = j l for l = 1,..., k. Thus, the equality w 3 = w T 4 provides a solution of [{a, b}, n, P, Q].

Proof V Sufficiency. Let [{a, b}, n, P, Q] be solvable. Then there exist a finite sequence i 1, i 2..., i k of natural numbers such that i j n for all 1 j k, and p i1 p i2 p ik = q i1 q i2 q ik. So, one directly gets a string r L S L(P, Q). This proves Claim 2. Claim 1 and 2 together directly imply Assertion (1) via the undecidability of PCP.

Proof VI For showing Assertion (2), we can use the same ideas plus the following Claim. Claim 3. L S L(P, Q) is infinite if and only if L S L(P, Q). The necessity is trivial. For showing the sufficiency, let i 1, i 2..., i k be a finite sequence of natural numbers such that i j n for all 1 j k, and p i1 p i2 p ik = q i1 q i2 q ik. Therefore, we also have (p i1 p i2 p ik ) m = (q i1 q i2 q ik ) m, that is, (i 1, i 2..., i k ) m is a solution of [{a, b}, n, P, Q] for every m 1. But this means, if then also w 1 cw 3 cw 4 cw 2 L S L(P, Q), w m 1 cwm 3 cwm 4 cwm 2 L S L(P, Q) for every m N +. This proves Claim 3, and thus Assertion (2) is shown.

Proof VII It remains to prove Assertions (3) and (4). This done via the following claim. Claim 4. L S L(P, Q) does not contain any infinite context-free language. Assuming Claim 4, Assertions (3) and (4) can be obtained, since the following assertions are equivalent. (α) L S L(P, Q) =, (β) L S L(P, Q) REG, (γ) L S L(P, Q) CF. Obviously, (α) implies (β) and (β) implies (γ). Thus, we have only to prove that (γ) implies (α). This is equivalent to showing that the negation of (α) implies the negation of (γ).

Proof VIII So, let us assume the negation of (α), i.e., L S L(P, Q). By Claim 3, we then know that L S L(P, Q) is infinite. Now Claim 4 tells us that L S L(P, Q) CF. Thus, we have shown the negation of (γ). Under the assumption that Claim 4 is true, we have thus established the equivalence of (α), (β) and (γ). Consequently, by Claim 2 we then know that L S L(P, Q) CF if and only if [{a, b}, n, P, Q] is not solvable and also that L S L(P, Q) REG if and only if [{a, b}, n, P, Q] is not solvable. This proves Assertions (3) and (4).

Proof IX So, it remains to show Claim 4. Let [{a, b}, n, P, Q] be arbitrarily fixed. Suppose there is a language L L S L(P, Q) such that L is infinite and context-free. Now we apply the qrsuv-theorem. Thus, there exists a k N such that for all w L with w k there are strings q, r, s, u, v such that w = qrsuv and ru λ and qr i su i v L for all i N. Since by supposition L is infinite there must exist a string w L with w k. Furthermore, L L(P, Q) and therefore w must have the form w = w 1 cw 2 cw 3 cw 4, where w i {a, b} +, i = 1, 2, 3, 4. (3) That is, w contains exactly three times the letter c. Now, let w = qrsuv. We distinguish the following cases.

Proof X Case 1. c is a substring of ru. Then qr i su i v L for every i N and hence qr i su i v contains at least four c s for every i 4. So, Case 1 cannot happen.

Proof XI Case 2. c is not a substring of ru. Then, neither r nor u could be substrings of ba i 1b ba i k. If r or u start and end with b, then qr 2 su 2 v L(P, Q). If both r and u do not start and end with b, then r {a} + or u {a} + is impossible, since qr n+1 su n+1 v then violates the condition that we have at most n consecutive a s. Otherwise, we get a contradiction to the definition of L(P, Q), since then we have more blocks of a s than there are p i s or q j s. Finally, the only remaining possibility is that r is a substring of w 2 or r is a substring of w 3 (cf. (3)). In either case, then qr i su i v L(P, Q) for i 2, since the length of w 1 and w 2 as well as the length of w 3 and w 4 are related. This is one of the reasons we have included the a i j into the definition of L(P, Q). This proves Claim 4, and thus the theorem is shown.

Undecidability Results for Context-free Languages III Next, we turn our attention to problems involving the complement of context-free languages. Recall that CF is not closed under complement. However, due to lack of time, we have to omit a certain part of the following proof. Theorem 2 The following problems are undecidable for any context-free grammar G: (1) L(G) =, (2) L(G) is infinite, (3) L(G) CF, (4) L(G) REG, (5) L(G) REG.

Proof I We use the notions from the demonstration of the previous Theorem. Consider L = df L S L(P, Q). Note that L is always context-free. We do not prove this assertion here. The interested reader is referred to Ginsburg (1966). For showing (1), suppose the converse. Then, given the fact that L is context-free, there is a context-free grammar G such that L = L(G). So, we could run the algorithm on input G. On the other hand, L = L S L(P, Q). Thus, we could decide whether or not L S L(P, Q) =. By Claim 2 in the proof of the previous Theorem, this implies that we can decide whether or not a PCP is solvable; a contradiction to the undecidability of PCP.

Proof II Assertion (2) is shown analogously via (2) of the previous Theorem. Assertion (3) and (4) also follow directly from the previous Theorem by using its Assertions (3) and (4), respectively. Finally, Assertion (5) is a direct consequence of Assertion (4) and the fact that L REG if and only if L REG (cf. Problem 4.2).

Undecidability Results for Context-free Languages IV Furthermore, the theorems shown above directly allow for the following corollary. Corollary 3 The following problems are undecidable for any context-free grammars G 1, G 2 : (1) L(G 1 ) = L(G 2 ), (2) L(G 1 ) L(G 2 ).

Proof of the Corollary I Proof. Suppose (1) is decidable. Let G 1 be any context-free grammar such that L(G 1 ) = L = L S L(P, Q) (see the proof of the latter theorem). Furthermore, let G 2 be any context-free grammar such that L(G 2 ) = {a, b, c} = Σ. Then L = Σ L S L(P, Q) = Σ L S L(P, Q) = [{a, b}, n, P, Q] is not solvable, a contradiction to the undecidability of PCP. This proves Assertion (1).

Proof of the Corollary II If (2) would be decidable, then (1) would be decidable, too, since L(G 1 ) = L(G 2 ) L(G 1 ) L(G 2 ) and L(G 2 ) L(G 1 ), (4) a contradiction. Therefore, we obtain that (2) is not decidable and Assertion (2) is shown.

Regular Languages I This is a good place to summarize our knowledge about regular languages and to compare the results obtained to the undecidability results for context-free languages shown above. As we have shown, L REG if and only if there exists a deterministic finite automaton A such that L = L(A). Therefore, we can directly get the following corollary. Corollary 4 The regular languages are closed under complement.

Regular Languages II Furthermore, the regular languages are closed under union, product and Kleene closure. So, recalling a bit set theory, we know that L 1 L 2 = L 1 L 2. Hence we directly get the following corollary. Corollary 5 The regular languages are closed under intersection. Moreover, as shown in Lecture 4, there is an algorithm which on input any regular grammar G decides whether or not L(G) is infinite. Using the algorithm given in the proof of this Theorem, we can also conclude that there is an algorithm which on input any regular grammar G decides whether or not L(G) is finite. Looking at the proof of Theorem 4.5, we also get the following corollary.

Regular Languages III Corollary 6 There is an algorithm which on input any regular grammar G decides whether or not L(G) =. Proof. Let G be a regular grammar. The algorithm first constructs a deterministic finite automaton A = [Σ, Q, δ, q 0, F] such that L(G) = L(A). Let card(q) = n. Then, the algorithm checks whether or not there is a string s such that n + 1 s 2n + 2 with s L(A). If there is no such string, then by the proof of Theorem 4.5 we already know that L(G) is finite. Thus, it suffices to check whether or not there is a string s such that s n and s L(A). If there is such string, output L(G). Otherwise, output L(G) =. Of course, this is not the most efficient algorithm.

Regular Languages IV Finally, you have shown that REG is closed under set difference (cf. Problem 4.2 of our Exercise sets). Thus, we also have the following corollary. Corollary 7 There is an algorithm which on input any regular grammars G 1 and G 2 decides whether or not L(G 1 ) L(G 2 ). Proof. Since REG is closed under set difference, we know that L(G 1 ) \ L(G 2 ) REG. Furthermore, a closer inspection of the proof given shows that we can construct a grammar G such that L(G) = L(G 1 ) \ L(G 2 ). Again recalling a bit set theory, we have L(G 1 ) L(G 2 ) if and only if L(G) =. By the corollary above, it is decidable whether or not L(G) =.

Regular Languages V Using the same idea as above (see (4)), we also see that there is an algorithm which on input any regular grammars G 1 and G 2 decides whether or not L(G 1 ) = L(G 2 ).

Regular Languages V Using the same idea as above (see (4)), we also see that there is an algorithm which on input any regular grammars G 1 and G 2 decides whether or not L(G 1 ) = L(G 2 ). Thus, we can conclude that all the above problems considered for context-free languages when stated mutatis mutandis for regular languages are decidable.

Definitions We need some more notations which we introduce below. Let A N be any set. We have already defined the characteristic function χ A, i.e., χ A (x) = 1 if x A and χ A (x) = 0 if x A. Next, we introduce the partial characteristic function π A defined as follows. π A (x) = { 1, if x A ; not defined, otherwise. Definition 1 Let A N; then A is said to (1) be recursive if χ A R, (2) be recursively enumerable if π A P.

Examples (1) The set of all prime numbers is recursive. (2) The set of all odd numbers is recursive. (3) The set of all even numbers is recursive. (4) is recursive. (5) Every finite set is recursive. (6) Every recursive set is recursively enumerable. (7) Let A be any recursive set. Then its complement A is recursive, too.

Characterization Theorem 8 Let A N; then we have: A is recursive if and only if A is recursively enumerable and A is recursively enumerable. Proof. Necessity. If A is recursive then A is recursively enumerable. As mentioned above, the complement A of every recursive set A is recursive, too. Thus, A is also recursively enumerable. Sufficiency. If both A and A are recursively enumerable, then π A P and π A P. Thus, we can express χ A as follows. χ A (x) = Consequently, χ A R. { 1, if πa (x) = 1 ; 0, if π A (x) = 1.

Consequences I The latter theorem can be used to show that there are recursively enumerable sets which are not recursive. Corollary 9 The halting set K is recursively enumerable but not recursive. Proof. Recall that K = {i i N, ψ i (i) }, where ψ P 2 is universal for P. Since ψ P 2, there must exist a TM M such that f 2 M = ψ. Hence, we have the following equivalence: π K (i) = 1 ψ i (i) M(i, i) stops. Thus, in order to compute π K (i) it suffices to start the Turing machine M on input (i, i). If the computation terminates, the algorithm for computing π K (i) outputs 1. Otherwise, the algorithm does not terminate. Consequently, π K P.

Consequences II Suppose K to be recursive. Then, χ K R, a contradiction to the undecidability of the halting problem. Corollary 10 The complement K of the halting set K is not recursively enumerable. So, it remains to relate the language family L 0 to Turing machines. This is done by the following theorems.

Turing Machines and L 0 Theorem 11 Let G be any type-0 grammar. Then there exists a Turing machine M such that L(G) = L(M).

Turing Machines and L 0 Theorem 11 Let G be any type-0 grammar. Then there exists a Turing machine M such that L(G) = L(M). The opposite is also true. Theorem 12 For every Turing machine M there exists a type-0 grammar G such that L(M) = L(G). Due to the lack of time, we do not prove these theorems here.

Turing Machines and L 0 So, we directly get the following corollary. Corollary 13 L 0 is equal to the family of all recursively enumerable sets. Now, we are ready to show the theorem already announced at the end of Lecture 10, i.e., that membership is not decidable for languages from L 0. Theorem 14 There does not exist any algorithm that on input any type-0 grammar G = [T, N, σ, P] and any string s T decides whether or not s L(G).

Proof We know that K is recursively enumerable but not recursive. Since π K P, there exists a Turing machine computing π K. Then we can conclude that there is a grammar G such that L(G) = K. Thus, if we could decide s L(G) for every s N, then K would be recursive, a contradiction.

Further Properties I Since K is not recursively enumerable, we also have the following corollary. Corollary 15 L 0 is not closed under complement and set difference. Interestingly, the ideas used in the proof of Theorem 2.1 directly yield the following theorem. Theorem 16 The language family L 0 is closed under union, product and Kleene closure.

Further Properties II Theorem 17 There does not exist any algorithm which, on input any type-0 grammar G decides whether or not L(G) =. Proof. Suppose the converse. Let G = [T, N, σ, P] be any grammar and let w T be arbitrarily fixed. We construct a grammar G = [T, N { σ, #}, σ, P], where P is as follows: P = P { σ #σ#, #w# λ}. Note that #w# λ is the only production which can remove the # symbols. Thus, we get L( G) = { {λ}, if w L(G) ;, otherwise.

Further Properties III Consequently, w L(G) if and only if L( G) =. Thus, we can decide L( G) = if and only if we can decide w L(G). But the latter problem is undecidable. Since the construction of G can be done algorithmically, we obtain a contradiction.

Further Properties IV Corollary 18 The following problems are undecidable for any type-0 grammars G 1, G 2 : (1) L(G 1 ) = L(G 2 ), (2) L(G 1 ) L(G 2 ). Proof. Let G be any type-0 grammar such that L(G ) =. Then we have for any type-0 grammar G: L(G) = L(G) L(G ) L(G) = L(G ), and thus both the inclusion and equivalence problem, respectively, are reduced to the decidability of the emptiness problem.

Remark As a general rule of thumb you should memorize that every non-trivial problem is undecidable for type-0 grammars. Here by non-trivial we mean that there are infinitely many grammars satisfying the problem and infinitely many grammars not satisfying the problem.

Problem REG CF CS Type 0 1) s L(G) + + + 2) L(G 1 ) L(G 2 ) + 3) L(G 1 ) = L(G 2 ) + 4) L(G) = + + 5) L(G) finite + + 6) L(G) infinite + + 7) L(G) = + 8) L(G) infinite + 9) L(G) has the + + same type as L(G) 10) L(G) REG + 11) L(G) CF + 12) L(G 1 ) L(G 2 ) = + 13) L(G 1 ) L(G 2 ) finite + 14) L(G 1 ) L(G 2 ) infinite + 15) L(G) = T + 16) L(G) REG +

Thank you!