1 CSCI- 2200 FOUNDATIONS OF COMPUTER SCIENCE Spring 2015 My 7, 2015
2 Announcements Homework 9 is due now. Some finl exm review problems will be posted on the web site tody. These re prcqce problems not complete review of everything tht might be on the finl.
3 Review: Lnguges A string is sequence of lewers defined over n lphbet, Exmple lphbet: = {, b} is the set of ll possible strings mde from the lphbet Exmple: = {,,b,,b,b,bb,,...} A lnguge is set of strings. It is subset of How mny lnguges re there over the lphbet = {, b}?
We mesure the computqonl power of computqonl model by its bility to recognize lnguges. 4 Turing-Recognizble Lnguges n b n c n ww Context-Free Lnguges n b n Regulr Lnguges R ww * *b*
5 Review: Universl Turing Mchine A universl Turing mchine is progrmmble Turing mchine. It cn simulte ny Turing mchine. Tkes descripqon of Turing s input. *A Turing mchine cn be encoded s bit string.
6 Review: The Lnguge of Turing Mchines A Turing Mchine cn be described by string of 0s nd 1s. Therefore, the set of Turing Mchines form lnguge. Ech string of the lnguge is the binry encoding of Turing Mchine. L = { 010100101, 00100100101111, 111010011110010101, (Turing Mchine 1) (Turing Mchine 2) }
7 LimitQons of Turing Mchines? The lnguge consisqng of ll Turing Mchine encodings is n infinite set. If there re infinitely mny Turing Mchines, does tht men tht every lnguge is recognized by some Turing Mchine? Not necessrily. Infinite sets come in different sizes. Countble vs. uncountble sets
8 Are there ny lnguges out here? Turing-Recognizble Lnguges n b n c n ww Context-Free Lnguges n b n Regulr Lnguges R ww * *b*
9 Review: Countble Sets An infinite set S is countble iff there exists bijecqve funcqon f: S Z + All infinite countble sets re the sme size. They ll hve the sme size s Z + All finite sets re countble.
10 Countble Sets An infinite set S is countble iff there exists bijecqve funcqon f: S Z + f mps ech element of S to exctly one element of Z +. Every element of Z + is mpped to by some element of S, under f. All infinite countble sets re the sme size. They ll hve the sme size s Z + All finite sets re countble.
11 Enumertors We cn lso show set is countble by describing n enumer;on procedure for the correspondence to nturl numbers. An enumerqon procedure is n lgorithm. A Turing Mchine cn do it. An enumertor for lnguge S is Turing Mchine tht genertes (prints on tpe) ll the strings in S one by one. Ech string is generted in finite Qme.
12
13 Enumertion procedure: Proper Order (Cnonicl Order) 1. Produce ll strings of length 1 2. Produce ll strings of length 2 3. Produce ll strings of length 3 4. Produce ll strings of length 4...
14 CounQng Turing Mchines Recll, Turing Mchine cn be encoded s bit string. A string of 0s nd 1s. The lnguge of Turing mchines is the set contining ll of these bit strings. Theorem: The set of ll Turing Mchines is countble. We will prove this theorem by finding n enumerqon procedure for the set.
15 Enumertor: Repet 1. Generte the next binry string of 0 s nd 1 s in cnonicl order 2. Check if the string describes Turing Mchine if YES: print string on output tpe if NO: ignore string
Binry strings 0 1 00 01 16 Turing Mchines! 1 0 1 0 11 0 11 0 0 s 1 1 0 1 0 11 0 11 0 1 1 0 1 0 11 0 11 0 1! 1 0 11 0 1010010101101 s 2 1 0 11 0 1010010101101! QED
17 Note tht: X = { L, L, L, } 1 2 3 is multi-set (elements my repet) since lnguge my be ccepted by more thn one Turing mchine However, if we remove the repeted elements, the resulting set is gin countble since every element still corresponds to positive integer
18 Are there ny lnguges out here? Turing-Recognizble Lnguges n b n c n ww Context-Free Lnguges n b n Regulr Lnguges R ww * *b*
19 LimitQons of Turing Mchines? The lnguge consisqng of ll Turing Mchine encodings is n infinite set. If there re infinitely mny Turing Mchines, does tht men tht every lnguge is recognized by some Turing Mchine? The set of Turing Mchines is countble. Is the set ll of lnguges countble?
20 Definition: A set is uncountble if it is not countble Theorem: If S is n infinite countble set, then P(S) (the power set of S) is n uncountble set.
Proof: Since S is countble, we cn write 21 S = { s1, s2, s3, } Elements of the power set { 3 { 10 s 1, s } s5, s7, s9, s } P(S) hve the form:
We encode ech element of the power set with binry string of 0 s nd 1 s 22 Powerset element (in rbitrry order) Binry encoding s1 s2 s 3 s 4! { s 1 } 1 0 0 0! { s 2, s 3} 0 1 1 0! { s1, s3, s 4 } 1 0 1 1!
Observtion: Every infinite binry string corresponds to n element of the powerset: 23 Exmple: 10 0111 0! Corresponds to: {s 1, s 4, s 5, s 6, }
24 Let s ssume (for contrdiction) tht the power set of S is countble Then: we cn enumerte the elements of the powerset P ( S) = { t, t, t,...} 1 2 3
Powerset element suppose tht this is the respective Binry encoding 25 t 1 1 0 0 0 0! t 2 1 1 0 0 0! t 3 1 1 0 1 0! t 4 1 1 0 0 1!!!
Tke the binry string whose bits re the complement of the digonl 26 t 1 1 0 0 0 0! t 2 t 3 t 4 1 1 0 0 0 1 1 0 1 0 1 1 0 0 1!!! Binry string: t = 0011 (birry complement of digonl)
27 The binry string t =0011 corresponds to n element of the power set of S : t = { s, s,...} P( S) 3 4
Thus, t must be equl to some ti 28 t =t i However, the i-th bit in the encoding of the complement of the i-th bit of, thus: t is ti t t i Contrdiction!!!
29 Since we hve contrdiction: The power set of S is uncountble. QED Cntor s Digonl Argument
30 Consider Alphbet : A = {, b} The set of ll Strings: S = {, b} * = { λ,, b,, b, b, bb,, b, } infinite nd countble Any lnguge is subset of : S L = {, b, b}
31 Consider Alphbet : A = {, b} The set of ll Strings: S = A * = {, b} * = { λ,, b,, b, b, bb,, b, } infinite nd countble The power set of S contins ll lnguges: P( S) = {,{ λ},{ },{, b},{, b},...,{, b, b},...} uncountble
Lnguges: uncountble 32 L1 L2 L3!! Lk M1 M2 M3? Turing mchines: countble There re more lnguges thn Turing Mchines
33 Conclusion: There re some lnguges not ccepted by Turing Mchines (These lnguges cnnot be described by lgorithms)
Are there ny lnguges out here? YES Turing-Recognizble Lnguges n b n c n ww 34 Context-Free Lnguges n b n R ww Regulr Lnguges * *b*
Fll 2006 Costs Busch - RPI 35 TURING- RECOGNIZABLE AND TURING DECIDABLE LANGUAGES
36 Lnguges Accepted by Turing Mchine A lnguge is clled Turing- recognizble if some Turing Mchine recognizes it (k ccepts it). If w 2 L the TM hlts in n ccept stte If w/2 L the TM hlts in reject stte OR loops forever A lnguge is clled Turing- decidble if some Turing Mchine decides it. If w 2 L the TM hlts in n ccept stte If w/2 L the TM hlts in reject stte The TM lwys hlts. The set of Turing- decidble lnguges is subset of the set of Turing- recognizble lnguges
37 Turing-Recognizble Lnguges Turing-Decidble Lnguges Context-Free Lnguges Regulr Lnguges
38 Lnguges Accepted by Turing Mchine We showed tht there re lnguges tht re not recognized by ny Turing Mchine. We used nonconstrucqve existence proof. We will show: There is specific lnguge tht is not recognized by ny Turing mchine There is specific lnguge tht is recognized by Turing mchine but is not decided by ny Turning mchine.
39 Let the lphbet be {} We cn generte lnguges over this lphbet, e.g. L 1 = {,,,,.} L 2 = {,,, } Consider TMs tht recognize these lnguges, i.e., L(M 1 ) = L 1 L(M 2 ) = L 2 The set of these TMs is countble {M 1, M 2, M 3, }
40 Exmple lnguge ccepted by M i L( Mi ) = {,, } Alterntive representtion of lnguge = bit string 1 2 3 4 5 6 7! L( Mi ) 0 1 1 1 0 0 0!
41 1 2 3 4! L( M1) L( M2 ) L( M3 ) 0 1 0 1! 1 0 0 1! 0 1 1 1! L( M4 ) 0 0 0 1!
1 2 3 4 42! L( M1) L( M2 ) L( M3 ) 0 1 0 1! 1 0 0 1! 0 1 1 1! L( M4 ) 0 0 0 1! 3 4 L = {,, } L = { i : i 2 L(M i ), for i 1}
1 2 3 4 43! L( M1) L( M2 ) L( M3 ) 0 1 0 1! 1 0 0 1! 0 1 1 1! L( M4 ) 0 0 0 1! L = { 1, 2, } Is there TM tht recognizes this lnguge?
44 A Non- Turing- Recognizble Lnguge Theorem: The lnguge is not Turing Recognizble L i i = { : L( Mi )}
45 Proof: This is proof by contrdiction. Assume tht is Turing-recognizble. L Then, there must exist some TM tht recognizes L. Cll this TM M. L(M ) = L
1 2 3 4 46! L( M1) L( M2 ) L( M3 ) 0 1 0 1! 1 0 0 1! 0 1 1 1! L( M4 ) 0 0 0 1! Question: M = M 1?
1 2 3 4 47! L( M1) L( M2 ) L( M3 ) 0 1 0 1! 1 0 0 1! 0 1 1 1! L( M4 ) 0 0 0 1! Answer: M M 1 1 L(M ) 1 L(M 1 )
1 2 3 4 48! L( M1) L( M2 ) L( M3 ) 0 1 0 1! 1 0 0 1! 0 1 1 1! L( M4 ) 0 0 0 1! Question:? M = M 2
1 2 3 4 49! L( M1) L( M2 ) L( M3 ) 0 1 0 1! 1 0 0 1! 0 1 1 1! L( M4 ) 0 0 0 1! Answer: M = 2 L( M 2) M 2 2 L( M )
1 2 3 4 50! L( M1) L( M2 ) L( M3 ) 0 1 0 1! 1 0 0 1! 0 1 1 1! L( M4 ) 0 0 0 1! Question: M =M 3?
1 2 3 4 51! L( M1) L( M2 ) L( M3 ) 0 1 0 1! 1 0 0 1! 0 1 1 1! L( M4 ) 0 0 0 1! Answer: M 3 L( M ) 3 M L( M ) 3 3
52 Similrly: M M i for ny i Becuse either: i L(M ) i L(M i ) or i L(M ) i L(M i )
53 Therefore, the mchine M cnnot exist Therefore, the lnguge L is not Turing-recognizble QED
54 A Non- Turing- Recognizble Lnguge Theorem: The lnguge is not Turing Recognizble L i i = { : L( Mi )} We proved this with Cntor s digonl rgument.
The Chomsky Hierrchy L i i 55 = { : L( Mi )} Turing-Recognizble Lnguges Turing-Decidble Lnguges Context-Free Lnguges Regulr Lnguges
56 Lnguges Accepted by Turing Mchine We showed tht there is specific lnguge tht is not recognized by ny Turing mchine. We will show: There is specific lnguge tht is recognized by Turing mchine but is not decided by ny Turning mchine.
A lnguge tht is Turing- Recognizble but not Turing- Decidble Recll, for the lphbet {}, we cn generte lnguges over this lphbet, e.g. L 1 = {,,,,.} L 2 = {,,, } We enumerted the TMs tht recognize these lnguges: L(M 1 ) = L 1 57 L(M 2 ) = L 2 We defined the lnguge L = { i : i 2 L(M i ), for i 1} Clim: (1) L is Turing- recognizble nd (2) L is not Turing- decidble.
58 Proof tht L is Turing- Recognizble We will describe TM tht recognizes For ny input string w = i, compute i. Find Turing Mchine M i Using enumerqon procedure for Turing Mchines. Simulte M i on w If M i ccepts, then output ccept QED L = { i : i 2 L(M i ), for i 1}
59 Proof tht L is not Turing- Decidble This is proof by contrdicqon. L = { i : i 2 L(M i ), for i 1} Assume is Turing- decidble. Then, there exists mchine M tht decides L: If w 2 L, M hlts in n ccept stte. If w/2 L, M hlts in reject stte. i i This implies tht L = { : L( Mi )} is lso Turing- decidble. Why? But, L is not Turing- recognizble; therefore, it is not not Turing- decidble. Our ssumpqon hs led to contrdicqon. Therefore L is not Turing- decidble. QED
60 Hilbert s 10 th Problem In 1900, mthemqcin Dvid Hilbert posed 23 significnt mth problems s chllenge to the coming century. Hilbert s 10 th problem: Devise n lgorithm tht tests whether polynomil hs n integrl root. A polynomil is sum of terms, where ech term is product of vribles nd constnt ( coefficient). E.g. 6x 3 yz 2 + 3xy 2 x 3 10 A root of polynomil is n ssignment of vlues to its vribles tht mkes the polynomil equl 0. A root is n integrl root if the vribles vlues re ll integers. In 1970, Yuri MQjsevic proved tht no such lgorithm exists. Using noqons lid out in the Church- Turing thesis.
UNDECIDABILITY 61
62 Decision Problems Decision problems re problems with YES or NO nswer Does mchine M hve three sttes? Is w binry number? Does DFA M ccept ny input? A problem is decidble if some Turing mchine solves it. Outputs YES (ccept) when the nswer is yes nd outputs NO (reject) when the nswer is no. This is sme s sying the lnguge for the problem is decidble. A problem is undecidble if no Turing mchine decides it.
63 The Turing mchine tht decides (solves) problem nswers YES or NO for ech instnce of the problem. The Turing Mchine MUST HALT! Input problem instnce Turing Mchine YES NO We cll this TM Decider
64 Undecidble Problems Some decision problems re undecidble. There is no Turing mchine tht solves the problem for ll inputs. Exmple: The membership problem Input: Turing mchine M nd string w QuesQon: Does M ccept w? Theorem: The membership problem is undecidble There is no TM tht cn decide whether M ccepts w for ll M nd w.
65 Proof This is proof by contrdicqon. Assume the membership problem is decidble. This mens tht there is TM (cll it H) tht solves the membership problem every input (M,w). M YES M ccepts w H w NO M rejects w
66 Proof (cont.) Let L be ny Turing- recognizble lnguge, nd let M be the TM tht recognizes L. If the membership problem is decidble, then we cn construct TM tht decides L. This TM lwys hlts. M H YES ccept w M ccepts w? w NO reject w
67 Proof (cont.) We showed tht, if the membership problem is decidble, then: For ll L, if L is Turing- recognizble, then L is Turing- decidble. But not ll Turing- recognizble lnguges re Turing- decidble. This is contrdicqon. Therefore, the membership problem is undecidble. QED
68 The HlQng Problem Input: Turing Mchine M nd string w QuesQon: Does M hlt on input w? Theorem: The hlqng problem is undecidble. There is no TM tht cn determine whether M hlts on input w for ll M nd w. Proved in Turing s 1936 pper.
69 Proof This is proof by contrdicqon. Assume the hlqng problem is decidble. Then, there exists Turing mchine H tht solves the problem. M YES M hlts on w H w NO M doesn t hlt on w
70 Proof (cont.) Let L be Turing- recognizble lnguge Let M be the Turing mchine tht ccepts L. If the hlqng problem is decidble, then we cn construct Turing mchine tht decides L.
71 Turing Mchine tht decides L M H NO reject w w M hlts on w? YES Run M with input w ccept w Hlts on ccept stte reject w Hlts on reject stte
72 Proof (cont.) If the hlqng problem is decidble, this implies: For ll L, If L is Turing- recognizble, then L is lso Turing- decidble. But there re Turing- recognizble lnguges tht re not Turing- decidble. ContrdicQon! Therefore, the hlqng problem is undecidble. QED