Decidbility We investigte the power of lgorithms to solve problems. We demonstrte tht certin problems cn be solved lgorithmiclly nd others cnnot. Our objective is to explore the limits of lgorithmic solvbility. Models of Computtion 1
Encoding Turing Mchines s strings We cn encode Turing Mchines s strings over ny non-empty lphbet. E.g. if the string begins with the prefix 0 n 10 m 10 k 10 s 10 t 10 r 10 u 10 v this might indicte tht the mchine hs n sttes (0, 1,, n 1); it hs m tpe symbols (0, 1,,m 1), of which the first k re input symbols; the strt, ccept nd reject sttes re s, t nd r respectively, nd the endmrker nd blnk symbols re u nd v respectively. The reminder of the string cn consist of sequence of substrings specifying the trnsitions in δ. E.g. 0 p 10 10 q 10 b 10might indicte tht δ contins the trnsition ((p, ), (q, b, L)). The exct detils re unimportnt. Models of Computtion 2
There re lnguges tht re not RE There re only countbly mny TMs. Observe tht Σ is countble, for ny lphbet Σ. Since M Σ for ech TM M,wecnenumertethesetofTMsbysimplyomittingthosestringsthtrenot encodings of TMs. The set of lnguges over Σ, P(Σ ),isuncountble. Suppose there were n enumertion of P(Σ ),nmely,l 1,L 2,L 3,.Let x 1,x 2,x 3, be n enumertion of Σ.DefinenewlngugeL by: for i 1 x i L x i L i Now L P(Σ ),butl is not equl to ny of the L i s. Thus P(Σ ) is not in 1-1 correspondence with the set of Turing mchines (there re more lnguges thn TMs). It follows tht there is some lnguge tht is not ccepted by ny TM. Models of Computtion 3
DFA Acceptnce Problem DFA Acceptnce Problem: GivenDFAB nd n input w, doesb ccept w? The corresponding lnguge is A DFA = { B,w : B is DFA tht ccepts input string w }. Lemm. A DFA is decidble lnguge. Proof. WeconstructTMM tht decides A DFA :Oninput B,w where B is DFA nd w n input 1. Simulte B on input w 2. If the simultion ends in n ccept stte, ccept. Ifitendsinnon-ccepting stte, reject. Convince yourself tht it follows tht A NFA nd A regexp (cceptnce problems for NFA nd regulr expressions respectively) re lso decidble. Models of Computtion 4
Emptiness Problem for DFA: Given DFA A, is L(A) empty? The corresponding lnguge is E DFA = { A : A is DFA such tht L(A) = } Lemm. E DFA is decidble lnguge. Proof. WedesignTMT tht uses mrking lgorithm. On input A where A is DFA: 1. Mrk the strt stte of A. 2. Repet until no new sttes get mrked: Mrk ny stte tht hs trnsition coming into it from ny mrked stte. 3. If no ccept stte is mrked, ccept; otherwisereject. Models of Computtion 5
Equivlence Problem for DFA: Given two DFAs, re they equivlent? The corresponding lnguge: EQ DFA = { A, B : A nd B re DFA s.t. L(A) =L(B) }. Lemm. EQ DFA is decidble lnguge. Proof. Keyide:ForsetsP nd Q, P Q iff P Q =. WeuseT,the lgorithm for the Emptiness Problem E DFA. On input A, B, wherea nd B re DFAs: 1. Construct C =(A B) (B A). (DFAsreclosedunderunion, intersection nd complementtion.) 2. Run T on input C. 3. If T ccepts, ccept; if T rejects, reject. Models of Computtion 6
Acceptnce problem for CFG Consider the lnguge: A CFG = { G, w : G is CFG tht genertes w } Lemm. A CFG is decidble lnguge. We use fct: If G is in Chomsky norml form, ny derivtion of w hs 2n 1 steps, where n is the length of w. Proof. TheTMS for A CFG is: On input G, w, whereg is CFG nd w is string: 1. Convert G to n equivlent one in Chomsky norml form. 2. List ll derivtions with 2n 1 steps, where n is the length of w. 3. If ny of these derivtions generte w, ccept;otherwise,reject. This is of course extremely inefficient, nd will not be used in prctice. Models of Computtion 7
Emptiness Problem for CFG: Given CFG G, isl(g) empty? Lemm. E CFG is decidble lnguge. Proof. TheTMR for E CFG is: On input G, whereg =(V,Σ,R,S) is CFG: 1. Mrk ll terminl symbols in G. 2. Repet until no new vribles get mrked: Mrk ny vrible A where G contins rule A U 1 U k nd ech symbol U i Σ V hs lredy been mrked. 3. If the strt symbol is not mrked, ccept; otherwise, reject. Models of Computtion 8
Equiv. Problem for CFG: Given two CFGs, re they equivlent? Obvious ttempt: Use the strtegy for deciding EQ DFA. Unfortuntely CFGs re neither closed under intersection nor complementtion! In fct EQ CFG is undecidble. Models of Computtion 9
Universl Turing Mchines A universl TM is TM U tht cn simulte the ctions of ny Turing mchine. I.e. for ny TM M nd ny input x of M, U ccepts (respectively rejects or loops on) M,x, ifndonlyif,m ccepts (respectively rejects or loops on) x. This is equivlent to n interpreter of C written in C. Note this is different (quntifier order) from results such s every NDTM cn be simulted by deterministic TM. Models of Computtion 10
Universl Turing Mchines Simultion. The tpe of U is prtitioned into 3 trcks: The top trck holds the trnsition function δ M of the input TM M. The middle trck will be used to hold the simulted contents of M s tpe. The bottom trck will hold the current stte of M,ndthecurrentpositionof M s tpe hed. U simultes M on input x one step t time, shuttling bck nd forth between the three trcks. In ech step, it updtes M s stte nd simulted tpe contents nd hed position s dictted by δ M.IfeverM hlts nd ccepts or hlts nd rejects, then U does the sme. Note: The input lphbet of M nd U my be different. Models of Computtion 11
A TM :AcceptnceProblemforTMs Acceptnce Problem for TM A TM = { M,w : M is TM tht ccepts input w } Theorem. A TM is RE. Proof. WedefineTMU tht ccepts A TM :Oninput M,w where M is TM nd w is string 1. Simulte M on inputw. 2. If M ever enters its ccept stte, ccept; if M ever enters its reject stte, reject. Note tht U loops on M,w if M loops on w. ThisiswhyU is not decider. If the lgorithm hd some wy to determine tht M ws not hlting on w, itcould reject. Models of Computtion 12
A TM :AcceptnceProblemforTMs Theorem. A TM is undecidble. Proof. Suppose,forcontrdiction,TMH is decider for A TM.I.e. H( M,w ) = ccept reject if M ccepts w if M does not ccept w We construct new TM D with H s subroutine. D: Oninput M, wherem is TM 1. Run H on input M, M 2. Output the opposite of H. I.e.ifH ccepts, reject, ndifh rejects, ccept. Models of Computtion 13
A TM is undecidble (proof cont d) Thus D( M ) = reject ccept if M ccepts M if M does not ccept M Now we run D on input D. Wehve reject D( D ) = ccept if D ccepts D if D does not ccept D I.e. D ccepts D iff D rejects D, whichiscontrdiction.thusneitherd nor H cn exist. To summrize, A TM is (RE but) undecidble. Models of Computtion 14
AlngugethtisnotRE We sy tht L is co-re if its complement L is RE. Theorem. AlngugeisdecidbleiffitisbothREndco-RE. Proof. Suppose L nd L re cceptble by M nd M respectively. Define TM N s follows: On input x 1. Write x on the input tpes of M nd M. 2. Run M nd M in prllel (e.g. by dove-tiling the two computtions). 3. If M ccepts, ccept; ifm ccepts, reject. N is decider for L becuse for ny input x, eitherx L or x L: ifthe former then N must ccept becuse M must hlt nd ccept t some point, if the ltter, then N must reject becuse M must hlt nd ccept t some point. Corollry. A TM is not RE. Models of Computtion 15
The Hlting Problem: Given M nd x, doesm hlt on x? HALT TM = { M,w : M is TM nd M hlts on input w } Theorem. TheHltingProblem,HALT TM,isundecidble. Proof. Suppose,forcontrdiction,H is decider for HALT TM.WeuseH to construct TM S to decide A TM s follows: On input M,w where M is TM nd w is n input: 1. Run TM H on input M,w 2. If H rejects, reject. 3. If H ccepts, run M on w until it hlts. 4. If M hs ccepted, ccept; ifm hs rejected, reject. Since A TM is undecidble, H cnnot be decider for HALT TM. Models of Computtion 16
The Emptiness Problem for TM is undecidble E TM = { M : M is TM nd L(M) = } Theorem. TheEmptinessProblemforTMisundecidble. Proof. Suppose,forcontrdiction,E TM is decidble by TM E. Weshlluse E to construct TM S tht decides A TM. Definition of S: On input M,x where M is TM nd x n input 1. Construct TM M x defined s: On input y () If y x then reject. (b) Run M (input is x) ndccept if M does. 2. Run E on input M x.ife ccepts, reject, ife rejects, ccept. Models of Computtion 17
The Emptiness Problem for TM is undecidble (proof cont d) Now we hve M ccepts x L(M x )={ x } i.e. S ccepts M,x M does not ccept x L(M x )= i.e. S rejects M,x Thus S decides A TM,whichiscontrdiction. Models of Computtion 18
The Equivlence Problem for TM is undecidble EQ TM = { M 1,M 2 : M 1 nd M 2 re TMs nd L(M 1 )=L(M 2 ) } Theorem. TheEquivlenceProblemforTMisundecidble. Proof. Suppose,forcontrdiction,EQ TM is decidble by Q. WedefineTM tht decides E TM by: On input M 1. Run Q on input M,M 1 where M 1 is TM tht ccepts nothing i.e. L(M 1 )=. 2. If Q ccepts, ccept; ifq rejects, reject. This gives contrdiction. Models of Computtion 19
Proving Undecidbilty We hve seen three min methods of proving undecidbility results: digonliztion, Turing reduction nd mny-one reduction. Digonliztion. E.g. A TM :ShowtheproblemcnnotbedecidblebycretingTMwhich cnnot exist without contrdiction. Advntge: Donotneedtoknowboutundecidbleproblems Disdvntge: Onlyworksesilyforsmllnumberofproblems:A TM, HALT TM,... Models of Computtion 20
Proving Undecidbility: Reductions Reductions trnsform one problem to nother in computble wy. If there is reduction from known undecidble problem to some problem P,thenP is lso undecidble. Reductions lso work lso for proving tht problem is or is not RE. Turing reduction. E.g. E TM :Cretenlgorithmforknownundecidbleproblemusingthe problem to be reduced to s subroutine. Advntge: Esywytoshowundecidbilitywithknownundecidbleproblems Disdvntge: Algorithmicpproch doesnotprovideinsightintony reltionship between the problems Models of Computtion 21
Proving Undecidbility: Reductions Mny-one reduction. E.g. EQ TM :LetA be decision problem over Σ A nd let B decisionproblem over Σ B.Amny-onereductioniscomputblefunctionf :Σ A Σ B such tht for ll w Σ A w A f(w) B Advntge: Trueproblemtrnsformtion givesreltionshipsusefulinother res (e.g. complexity) Disdvntge: StrongerthnTuringreduction,sohrdertofindreductions between problems Another method of proving tht problem is not RE uses the fct tht problem is decidble if nd only if the problem nd its complement re both RE. Models of Computtion 22
TOTAL = { M : M hlts on ll inputs} Theorem. TOTALisneitherREnorco-RE. Proof. HALT TM is not decidble, but it is RE. Hence HALT TM is not RE. Therefore it suffices to reduce HALT TM to (i) TOTAL nd to (ii) TOTAL. (i) Given M,x where M is TM nd x is n input, we set N M,x to be the following TM: On input y, 1. Run M on input x for up to y steps. 2. If M hs hlted, then loop; if M hs not yet hlted, ccept. We hve M,x HALT TM N M,x hlts on ll input y N M,x TOTAL. Models of Computtion 23
TOTAL is neither RE nor co-re. (proof cont d) (ii) Given M,x where M is TM nd x is n input, we construct N M,x to be the following TM: On input y, 1. Erse y. 2. Write x on input tpe, nd run M on x. 3. If M ccepts, ccept; ifm rejects, reject. Note tht either N M,x loops on ll inputs, or it hlts on ll inputs. Now M,x HALT TM N M,x loops on ll inputs N M,x / TOTAL. Models of Computtion 24
Rice s Theorem Rice s Theorem. Everynon-trivilpropertyoftheREsetsisundecidble. Reformultion Let P be ny decision problem bout TMs (expressed s lnguge) tht stisfies the following properties: (1) For ny TMs M 1 nd M 2 where L(M 1 )=L(M 2 ),wehve M 1 P iff M 2 P.I.e.themembershipofTMM in P depends only on the lnguge of M. (2) There re TMs M 1 nd M 2 where M 1 P nd M 2 P.I.e.P is non-trivil. Then P is undecidble. Models of Computtion 25
Proof of Rice s Theorem Tke property P tht stisfies (1) nd (2). Wlog, ssume tht for ny TM E, if L(E) = then E P.TkeTMK such tht K P.Given M,x where M is TM nd x is n input, we construct N M,x s follows: On input y 1. Sve y on seprte trck somewhere. 2. Write x on the input tpe, nd run M on input x. 3. If M hlts on x, runk on y. 4. If K ccepts, ccept. Either M hlts on x or not. If M does not hlt on x, thesimultionin3.willnothlt,ndtheinputy of N M,x will not be ccepted, nd so, L(N M,x )= i.e. N M,x P. On the other hnd, if M hlts on x, y is ccepted by N M,x iff y is ccepted by K i.e. L(N M,x )=L(K) i.e. N M,x P. Models of Computtion 26
Proof of Rice s Theorem (cont d) Thus M hlts on x L(N M,x )=L(K) N M,x P M does not ccept x L(N M,x )= N M,x / P Thus we hve reduced HALT TM to P.SinceHALT TM is undecidble, so is P. [Assume for contrdiction P ws decidble, sy by decider A, then we cn construct decider S for HALT TM s follows: on input M,x construct N M,x nd run A on N M,x.IfAccepts ccept, ifa rejects reject.] Models of Computtion 27
Appliction of Rice s Theorem For n rbitrry TM M with Σ={, b}, thefollowingproblemsreundecidble: whether M ccepts bb (i.e. bb L(M)) whether L(M) contins two different strings of the sme length; whether L(M) contins ny string of length five; whether L(M) is regulr lnguge; whether L(M) is context-free lnguge; Note The following properties of TMs re not properties of RE sets nd Rice s Theorem is not pplicble: M hs t lest 341 sttes; M hlts on ll inputs; M rejects bb there exists smller mchine equivlent to M. Models of Computtion 28
PCP: Post Correspondence Problem Apuzzle:Given collection of dominos such s e.g. P = {[ b c ], [ b ], [ c ] [ ]} bc, c find list of dominos from P (repetitions permitted) so tht the string we get by reding off the symbols on the top is the sme s the sting of symbols on the bottom. This list is clled mtch. E.g. the following is mtch tking dominos 2, 1, 3, 2 nd 4 in this order: [ ] [ ] [ ] [ ] [ ] b c bc,,,, b c b c (Reding off the top string we get bcbc, thesmesredingoffthebottom one.) Models of Computtion 29
AformlsttementofthePCP Instnce: AcollectionP of dominos: P = {[ t1 b 1 ], [ t2 b 2 ],, [ tk where k 1, ndllt i nd b j re strings over Σ. Question: Is there mtch for P?I.e.non-emptysequencei 1,,i l where ech 1 i j k nd b k ]} t i1 t i2 t il = b i1 b i2 b il Theorem. PCPisundecidble. Models of Computtion 30
Exmples 1. Let Σ={ 0, 1 }, nd P = {[ ] [ ] [ ]} 11 111 1100,,. 111 11 001 Here is mtch for P : 1, 1, 3, 2, i.e. [ ] [ ] [ 11 11 1100,, 111 111 001 ], [ 111 11 2. For some collection of dominos, finding mtch my not be possible. E.g. P = {[ bc b ], [ c ] [ ]} cc, bc (becuse every top string is longer thn the bottom string) ] Models of Computtion 31
Undecidbility of PCP: outline proof We modify PCP to MPCP = {P is n instnce of PCP with mtch tht strts with the first domino} 1. Suppose, for contrdiction, there is TM R tht decides the PCP. We use R to construct TM S tht decide A TM.Let M = (Q, Σ, Γ,,,δ,q 0,q cc,q rej ) Tke n input w for M.WeconstructninstnceP of the MPCP (over the lphbet Q Γ {}) thtsimultesm on w i.e. P MPCP M,w A TM. Models of Computtion 32
Ide. Writingconfigurtion(u i,q i,v i ) s u i q i v i,mtchofp gives string; the first prt q 0 wu 1 q 1 v 1 u 2 q 2 v 2 u i q i v i u n q cc v n corresponds exctly to n ccepting configurtion; the rest of the string q cc consists of progressively shorter segments of the form tht finlly ends in. 2. We convert P to P,ninstnceofPCP which still simultes M on w. Models of Computtion 33
Definition of P [ ] [ ] Prt 1. Put into P t1 s the first domino q 0 w 1 w n b 1 [ ] q Prt 2. For ech, b Γ, q, r Q, ifδ(q, ) =(r, b, R), put br [ ] cq Prt 3. For ech, b, c Γ, q, r Q, ifδ(q, ) =(r, b, L), put rcb [ ] Prt 4. For every Γ, put into P. [ ] [ ] Prt 5. Put nd into P. [ ] [ ] qcc qcc Prt 6. For every Γ, put nd into P. Prt 7. Add [ ] qcc to P. q cc q cc into P into P Models of Computtion 34
An exmple Consider the following TM M on input w = b M =({ q 0,q 3,q 4,q cc,q rej }, {, b, c }, Σ {, },,,δ,q 0,q cc,q rej ) }{{}}{{}}{{} Q Σ Γ where δ is prtly given by δ(q 0, ) = (q 0,,R) δ(q 0,) = (q 3,c,R) δ(q 3,b) = (q 4,,R) δ(q 4, ) = (q cc,,l) An ccepting computtion: (ϵ, q 0, b) (,q 0,b) ( c, q 3,b) ( c, q 4, ) ( c, q cc,b) Models of Computtion 35
An exmple (cont d) q 0 b prt 1 (ϵ, q 0, b) q 0 q 0 b q 0 δ(q 0, ) =(q 0,,R) prt 2 q 0 b (,q 0,b) q 0 b q 0 b prts 4 4 5 Models of Computtion 36
An exmple (cont d) q 0 b q 0 b ( c, q 3,b) q 0 b q 0 b c q 3 b 4 2 4 5 q 0 b q 0 b c q 3 b ( c, q 4, ) q 0 b q 0 b c q 3 b c q 4 4 4 2 5 c q 3 b c q 4 ( c, q cc,b) c q 4 c q cc b 4 4 3 5 Models of Computtion 37
An exmple (cont d) Tidying up. c q 4 c q cc b c q cc b q cc b 4 6 4 4 5 cq cc b q cc b q cc b q cc b q cc q cc b q cc b q cc b q cc 6 4 4 5 6 4 5 6 5 7 Models of Computtion 38
Converting MPCP instnce to PCP instnce Finlly we show how to convert P to n instnce of PCP which still simultes M on w. Theideistobuildtherequirementofstrtingwiththefirstdominodirectly into the problem. Nottion. Tkestringu = u 1 u n.define u = u 1 u 2 u n u = u 1 u 2 u n u = u 1 u 2 u n Models of Computtion 39
Converting MPCP instnce to PCP instnce (cont d) Suppose P is the MPCP instnce (over lphbet Θ) {[ t1 b 1 ], [ t2 b 2 ], [ t3 b 3 ],, [ tk b k ]} Now define P to be the PCP instnce (over lphbet Θ {, }) {[ ] [ ] [ ] [ ] [ ]} t1 t2 t3 tk,,,,, b 1 b 2 b 3 b k Note tht the only domino tht cn strt mtch is the first one, becuse it is the only one where both the top nd bottom strt with the sme symbol, nmely,. The originl symbols in Θ now occur in even positions of the mtch. The domino [ ] is to llow the top to dd the extr t the end of the mtch. Models of Computtion 40
Lnguge regulr context-free semi-decidble/re Grmmr right-liner context-free unrestricted rules A wb, A w A α α β Mchine DFA or NFA NPDA Turing mchine memory finite finite + one stck unrestricted Other Descr n Regulr expression Myhill-Nerode-Thm λ-clculus µ-recursive funct s RM, Hugs, Oberon,... Exmple { w : w contins 10 } { 0 n 1 m : n, m 0 } { ww R } { 0 n 1 n : n 0 } { 0 n 1 m 0 n m : n 0 } { Mx : M hlts on x } HP Counterexmple { 0 n 1 n : n 0 } { 0 n 1 n 0 n : n 0 } { ww} { M : M hlts on every x } TOTAL Appliction Tokenizer Prser Generl computing Model checker Models of Computtion 41