Decidability. Models of Computation 1

Transcription

1 Decidbility We investigte the power of lgorithms to solve problems. We demonstrte tht certin problems cn be solved lgorithmiclly nd others cnnot. Our objective is to explore the limits of lgorithmic solvbility. Models of Computtion 1

2 Encoding Turing Mchines s strings We cn encode Turing Mchines s strings over ny non-empty lphbet. E.g. if the string begins with the prefix 0 n 10 m 10 k 10 s 10 t 10 r 10 u 10 v this might indicte tht the mchine hs n sttes (0, 1,, n 1); it hs m tpe symbols (0, 1,,m 1), of which the first k re input symbols; the strt, ccept nd reject sttes re s, t nd r respectively, nd the endmrker nd blnk symbols re u nd v respectively. The reminder of the string cn consist of sequence of substrings specifying the trnsitions in δ. E.g. 0 p q 10 b 10might indicte tht δ contins the trnsition ((p, ), (q, b, L)). The exct detils re unimportnt. Models of Computtion 2

3 There re lnguges tht re not RE There re only countbly mny TMs. Observe tht Σ is countble, for ny lphbet Σ. Since M Σ for ech TM M,wecnenumertethesetofTMsbysimplyomittingthosestringsthtrenot encodings of TMs. The set of lnguges over Σ, P(Σ ),isuncountble. Suppose there were n enumertion of P(Σ ),nmely,l 1,L 2,L 3,.Let x 1,x 2,x 3, be n enumertion of Σ.DefinenewlngugeL by: for i 1 x i L x i L i Now L P(Σ ),butl is not equl to ny of the L i s. Thus P(Σ ) is not in 1-1 correspondence with the set of Turing mchines (there re more lnguges thn TMs). It follows tht there is some lnguge tht is not ccepted by ny TM. Models of Computtion 3

4 DFA Acceptnce Problem DFA Acceptnce Problem: GivenDFAB nd n input w, doesb ccept w? The corresponding lnguge is A DFA = { B,w : B is DFA tht ccepts input string w }. Lemm. A DFA is decidble lnguge. Proof. WeconstructTMM tht decides A DFA :Oninput B,w where B is DFA nd w n input 1. Simulte B on input w 2. If the simultion ends in n ccept stte, ccept. Ifitendsinnon-ccepting stte, reject. Convince yourself tht it follows tht A NFA nd A regexp (cceptnce problems for NFA nd regulr expressions respectively) re lso decidble. Models of Computtion 4

5 Emptiness Problem for DFA: Given DFA A, is L(A) empty? The corresponding lnguge is E DFA = { A : A is DFA such tht L(A) = } Lemm. E DFA is decidble lnguge. Proof. WedesignTMT tht uses mrking lgorithm. On input A where A is DFA: 1. Mrk the strt stte of A. 2. Repet until no new sttes get mrked: Mrk ny stte tht hs trnsition coming into it from ny mrked stte. 3. If no ccept stte is mrked, ccept; otherwisereject. Models of Computtion 5

6 Equivlence Problem for DFA: Given two DFAs, re they equivlent? The corresponding lnguge: EQ DFA = { A, B : A nd B re DFA s.t. L(A) =L(B) }. Lemm. EQ DFA is decidble lnguge. Proof. Keyide:ForsetsP nd Q, P Q iff P Q =. WeuseT,the lgorithm for the Emptiness Problem E DFA. On input A, B, wherea nd B re DFAs: 1. Construct C =(A B) (B A). (DFAsreclosedunderunion, intersection nd complementtion.) 2. Run T on input C. 3. If T ccepts, ccept; if T rejects, reject. Models of Computtion 6

7 Acceptnce problem for CFG Consider the lnguge: A CFG = { G, w : G is CFG tht genertes w } Lemm. A CFG is decidble lnguge. We use fct: If G is in Chomsky norml form, ny derivtion of w hs 2n 1 steps, where n is the length of w. Proof. TheTMS for A CFG is: On input G, w, whereg is CFG nd w is string: 1. Convert G to n equivlent one in Chomsky norml form. 2. List ll derivtions with 2n 1 steps, where n is the length of w. 3. If ny of these derivtions generte w, ccept;otherwise,reject. This is of course extremely inefficient, nd will not be used in prctice. Models of Computtion 7

8 Emptiness Problem for CFG: Given CFG G, isl(g) empty? Lemm. E CFG is decidble lnguge. Proof. TheTMR for E CFG is: On input G, whereg =(V,Σ,R,S) is CFG: 1. Mrk ll terminl symbols in G. 2. Repet until no new vribles get mrked: Mrk ny vrible A where G contins rule A U 1 U k nd ech symbol U i Σ V hs lredy been mrked. 3. If the strt symbol is not mrked, ccept; otherwise, reject. Models of Computtion 8

9 Equiv. Problem for CFG: Given two CFGs, re they equivlent? Obvious ttempt: Use the strtegy for deciding EQ DFA. Unfortuntely CFGs re neither closed under intersection nor complementtion! In fct EQ CFG is undecidble. Models of Computtion 9

10 Universl Turing Mchines A universl TM is TM U tht cn simulte the ctions of ny Turing mchine. I.e. for ny TM M nd ny input x of M, U ccepts (respectively rejects or loops on) M,x, ifndonlyif,m ccepts (respectively rejects or loops on) x. This is equivlent to n interpreter of C written in C. Note this is different (quntifier order) from results such s every NDTM cn be simulted by deterministic TM. Models of Computtion 10

11 Universl Turing Mchines Simultion. The tpe of U is prtitioned into 3 trcks: The top trck holds the trnsition function δ M of the input TM M. The middle trck will be used to hold the simulted contents of M s tpe. The bottom trck will hold the current stte of M,ndthecurrentpositionof M s tpe hed. U simultes M on input x one step t time, shuttling bck nd forth between the three trcks. In ech step, it updtes M s stte nd simulted tpe contents nd hed position s dictted by δ M.IfeverM hlts nd ccepts or hlts nd rejects, then U does the sme. Note: The input lphbet of M nd U my be different. Models of Computtion 11

12 A TM :AcceptnceProblemforTMs Acceptnce Problem for TM A TM = { M,w : M is TM tht ccepts input w } Theorem. A TM is RE. Proof. WedefineTMU tht ccepts A TM :Oninput M,w where M is TM nd w is string 1. Simulte M on inputw. 2. If M ever enters its ccept stte, ccept; if M ever enters its reject stte, reject. Note tht U loops on M,w if M loops on w. ThisiswhyU is not decider. If the lgorithm hd some wy to determine tht M ws not hlting on w, itcould reject. Models of Computtion 12

13 A TM :AcceptnceProblemforTMs Theorem. A TM is undecidble. Proof. Suppose,forcontrdiction,TMH is decider for A TM.I.e. H( M,w ) = ccept reject if M ccepts w if M does not ccept w We construct new TM D with H s subroutine. D: Oninput M, wherem is TM 1. Run H on input M, M 2. Output the opposite of H. I.e.ifH ccepts, reject, ndifh rejects, ccept. Models of Computtion 13

14 A TM is undecidble (proof cont d) Thus D( M ) = reject ccept if M ccepts M if M does not ccept M Now we run D on input D. Wehve reject D( D ) = ccept if D ccepts D if D does not ccept D I.e. D ccepts D iff D rejects D, whichiscontrdiction.thusneitherd nor H cn exist. To summrize, A TM is (RE but) undecidble. Models of Computtion 14

15 AlngugethtisnotRE We sy tht L is co-re if its complement L is RE. Theorem. AlngugeisdecidbleiffitisbothREndco-RE. Proof. Suppose L nd L re cceptble by M nd M respectively. Define TM N s follows: On input x 1. Write x on the input tpes of M nd M. 2. Run M nd M in prllel (e.g. by dove-tiling the two computtions). 3. If M ccepts, ccept; ifm ccepts, reject. N is decider for L becuse for ny input x, eitherx L or x L: ifthe former then N must ccept becuse M must hlt nd ccept t some point, if the ltter, then N must reject becuse M must hlt nd ccept t some point. Corollry. A TM is not RE. Models of Computtion 15

16 The Hlting Problem: Given M nd x, doesm hlt on x? HALT TM = { M,w : M is TM nd M hlts on input w } Theorem. TheHltingProblem,HALT TM,isundecidble. Proof. Suppose,forcontrdiction,H is decider for HALT TM.WeuseH to construct TM S to decide A TM s follows: On input M,w where M is TM nd w is n input: 1. Run TM H on input M,w 2. If H rejects, reject. 3. If H ccepts, run M on w until it hlts. 4. If M hs ccepted, ccept; ifm hs rejected, reject. Since A TM is undecidble, H cnnot be decider for HALT TM. Models of Computtion 16

17 The Emptiness Problem for TM is undecidble E TM = { M : M is TM nd L(M) = } Theorem. TheEmptinessProblemforTMisundecidble. Proof. Suppose,forcontrdiction,E TM is decidble by TM E. Weshlluse E to construct TM S tht decides A TM. Definition of S: On input M,x where M is TM nd x n input 1. Construct TM M x defined s: On input y () If y x then reject. (b) Run M (input is x) ndccept if M does. 2. Run E on input M x.ife ccepts, reject, ife rejects, ccept. Models of Computtion 17

18 The Emptiness Problem for TM is undecidble (proof cont d) Now we hve M ccepts x L(M x )={ x } i.e. S ccepts M,x M does not ccept x L(M x )= i.e. S rejects M,x Thus S decides A TM,whichiscontrdiction. Models of Computtion 18

19 The Equivlence Problem for TM is undecidble EQ TM = { M 1,M 2 : M 1 nd M 2 re TMs nd L(M 1 )=L(M 2 ) } Theorem. TheEquivlenceProblemforTMisundecidble. Proof. Suppose,forcontrdiction,EQ TM is decidble by Q. WedefineTM tht decides E TM by: On input M 1. Run Q on input M,M 1 where M 1 is TM tht ccepts nothing i.e. L(M 1 )=. 2. If Q ccepts, ccept; ifq rejects, reject. This gives contrdiction. Models of Computtion 19

20 Proving Undecidbilty We hve seen three min methods of proving undecidbility results: digonliztion, Turing reduction nd mny-one reduction. Digonliztion. E.g. A TM :ShowtheproblemcnnotbedecidblebycretingTMwhich cnnot exist without contrdiction. Advntge: Donotneedtoknowboutundecidbleproblems Disdvntge: Onlyworksesilyforsmllnumberofproblems:A TM, HALT TM,... Models of Computtion 20

21 Proving Undecidbility: Reductions Reductions trnsform one problem to nother in computble wy. If there is reduction from known undecidble problem to some problem P,thenP is lso undecidble. Reductions lso work lso for proving tht problem is or is not RE. Turing reduction. E.g. E TM :Cretenlgorithmforknownundecidbleproblemusingthe problem to be reduced to s subroutine. Advntge: Esywytoshowundecidbilitywithknownundecidbleproblems Disdvntge: Algorithmicpproch doesnotprovideinsightintony reltionship between the problems Models of Computtion 21

22 Proving Undecidbility: Reductions Mny-one reduction. E.g. EQ TM :LetA be decision problem over Σ A nd let B decisionproblem over Σ B.Amny-onereductioniscomputblefunctionf :Σ A Σ B such tht for ll w Σ A w A f(w) B Advntge: Trueproblemtrnsformtion givesreltionshipsusefulinother res (e.g. complexity) Disdvntge: StrongerthnTuringreduction,sohrdertofindreductions between problems Another method of proving tht problem is not RE uses the fct tht problem is decidble if nd only if the problem nd its complement re both RE. Models of Computtion 22

23 TOTAL = { M : M hlts on ll inputs} Theorem. TOTALisneitherREnorco-RE. Proof. HALT TM is not decidble, but it is RE. Hence HALT TM is not RE. Therefore it suffices to reduce HALT TM to (i) TOTAL nd to (ii) TOTAL. (i) Given M,x where M is TM nd x is n input, we set N M,x to be the following TM: On input y, 1. Run M on input x for up to y steps. 2. If M hs hlted, then loop; if M hs not yet hlted, ccept. We hve M,x HALT TM N M,x hlts on ll input y N M,x TOTAL. Models of Computtion 23

24 TOTAL is neither RE nor co-re. (proof cont d) (ii) Given M,x where M is TM nd x is n input, we construct N M,x to be the following TM: On input y, 1. Erse y. 2. Write x on input tpe, nd run M on x. 3. If M ccepts, ccept; ifm rejects, reject. Note tht either N M,x loops on ll inputs, or it hlts on ll inputs. Now M,x HALT TM N M,x loops on ll inputs N M,x / TOTAL. Models of Computtion 24

25 Rice s Theorem Rice s Theorem. Everynon-trivilpropertyoftheREsetsisundecidble. Reformultion Let P be ny decision problem bout TMs (expressed s lnguge) tht stisfies the following properties: (1) For ny TMs M 1 nd M 2 where L(M 1 )=L(M 2 ),wehve M 1 P iff M 2 P.I.e.themembershipofTMM in P depends only on the lnguge of M. (2) There re TMs M 1 nd M 2 where M 1 P nd M 2 P.I.e.P is non-trivil. Then P is undecidble. Models of Computtion 25

26 Proof of Rice s Theorem Tke property P tht stisfies (1) nd (2). Wlog, ssume tht for ny TM E, if L(E) = then E P.TkeTMK such tht K P.Given M,x where M is TM nd x is n input, we construct N M,x s follows: On input y 1. Sve y on seprte trck somewhere. 2. Write x on the input tpe, nd run M on input x. 3. If M hlts on x, runk on y. 4. If K ccepts, ccept. Either M hlts on x or not. If M does not hlt on x, thesimultionin3.willnothlt,ndtheinputy of N M,x will not be ccepted, nd so, L(N M,x )= i.e. N M,x P. On the other hnd, if M hlts on x, y is ccepted by N M,x iff y is ccepted by K i.e. L(N M,x )=L(K) i.e. N M,x P. Models of Computtion 26

27 Proof of Rice s Theorem (cont d) Thus M hlts on x L(N M,x )=L(K) N M,x P M does not ccept x L(N M,x )= N M,x / P Thus we hve reduced HALT TM to P.SinceHALT TM is undecidble, so is P. [Assume for contrdiction P ws decidble, sy by decider A, then we cn construct decider S for HALT TM s follows: on input M,x construct N M,x nd run A on N M,x.IfAccepts ccept, ifa rejects reject.] Models of Computtion 27

28 Appliction of Rice s Theorem For n rbitrry TM M with Σ={, b}, thefollowingproblemsreundecidble: whether M ccepts bb (i.e. bb L(M)) whether L(M) contins two different strings of the sme length; whether L(M) contins ny string of length five; whether L(M) is regulr lnguge; whether L(M) is context-free lnguge; Note The following properties of TMs re not properties of RE sets nd Rice s Theorem is not pplicble: M hs t lest 341 sttes; M hlts on ll inputs; M rejects bb there exists smller mchine equivlent to M. Models of Computtion 28

29 PCP: Post Correspondence Problem Apuzzle:Given collection of dominos such s e.g. P = {[ b c ], [ b ], [ c ] [ ]} bc, c find list of dominos from P (repetitions permitted) so tht the string we get by reding off the symbols on the top is the sme s the sting of symbols on the bottom. This list is clled mtch. E.g. the following is mtch tking dominos 2, 1, 3, 2 nd 4 in this order: [ ] [ ] [ ] [ ] [ ] b c bc,,,, b c b c (Reding off the top string we get bcbc, thesmesredingoffthebottom one.) Models of Computtion 29

30 AformlsttementofthePCP Instnce: AcollectionP of dominos: P = {[ t1 b 1 ], [ t2 b 2 ],, [ tk where k 1, ndllt i nd b j re strings over Σ. Question: Is there mtch for P?I.e.non-emptysequencei 1,,i l where ech 1 i j k nd b k ]} t i1 t i2 t il = b i1 b i2 b il Theorem. PCPisundecidble. Models of Computtion 30

31 Exmples 1. Let Σ={ 0, 1 }, nd P = {[ ] [ ] [ ]} ,, Here is mtch for P : 1, 1, 3, 2, i.e. [ ] [ ] [ ,, ], [ For some collection of dominos, finding mtch my not be possible. E.g. P = {[ bc b ], [ c ] [ ]} cc, bc (becuse every top string is longer thn the bottom string) ] Models of Computtion 31

32 Undecidbility of PCP: outline proof We modify PCP to MPCP = {P is n instnce of PCP with mtch tht strts with the first domino} 1. Suppose, for contrdiction, there is TM R tht decides the PCP. We use R to construct TM S tht decide A TM.Let M = (Q, Σ, Γ,,,δ,q 0,q cc,q rej ) Tke n input w for M.WeconstructninstnceP of the MPCP (over the lphbet Q Γ {}) thtsimultesm on w i.e. P MPCP M,w A TM. Models of Computtion 32

33 Ide. Writingconfigurtion(u i,q i,v i ) s u i q i v i,mtchofp gives string; the first prt q 0 wu 1 q 1 v 1 u 2 q 2 v 2 u i q i v i u n q cc v n corresponds exctly to n ccepting configurtion; the rest of the string q cc consists of progressively shorter segments of the form tht finlly ends in. 2. We convert P to P,ninstnceofPCP which still simultes M on w. Models of Computtion 33

34 Definition of P [ ] [ ] Prt 1. Put into P t1 s the first domino q 0 w 1 w n b 1 [ ] q Prt 2. For ech, b Γ, q, r Q, ifδ(q, ) =(r, b, R), put br [ ] cq Prt 3. For ech, b, c Γ, q, r Q, ifδ(q, ) =(r, b, L), put rcb [ ] Prt 4. For every Γ, put into P. [ ] [ ] Prt 5. Put nd into P. [ ] [ ] qcc qcc Prt 6. For every Γ, put nd into P. Prt 7. Add [ ] qcc to P. q cc q cc into P into P Models of Computtion 34

35 An exmple Consider the following TM M on input w = b M =({ q 0,q 3,q 4,q cc,q rej }, {, b, c }, Σ {, },,,δ,q 0,q cc,q rej ) }{{}}{{}}{{} Q Σ Γ where δ is prtly given by δ(q 0, ) = (q 0,,R) δ(q 0,) = (q 3,c,R) δ(q 3,b) = (q 4,,R) δ(q 4, ) = (q cc,,l) An ccepting computtion: (ϵ, q 0, b) (,q 0,b) ( c, q 3,b) ( c, q 4, ) ( c, q cc,b) Models of Computtion 35

36 An exmple (cont d) q 0 b prt 1 (ϵ, q 0, b) q 0 q 0 b q 0 δ(q 0, ) =(q 0,,R) prt 2 q 0 b (,q 0,b) q 0 b q 0 b prts Models of Computtion 36

37 An exmple (cont d) q 0 b q 0 b ( c, q 3,b) q 0 b q 0 b c q 3 b q 0 b q 0 b c q 3 b ( c, q 4, ) q 0 b q 0 b c q 3 b c q c q 3 b c q 4 ( c, q cc,b) c q 4 c q cc b Models of Computtion 37

38 An exmple (cont d) Tidying up. c q 4 c q cc b c q cc b q cc b cq cc b q cc b q cc b q cc b q cc q cc b q cc b q cc b q cc Models of Computtion 38

39 Converting MPCP instnce to PCP instnce Finlly we show how to convert P to n instnce of PCP which still simultes M on w. Theideistobuildtherequirementofstrtingwiththefirstdominodirectly into the problem. Nottion. Tkestringu = u 1 u n.define u = u 1 u 2 u n u = u 1 u 2 u n u = u 1 u 2 u n Models of Computtion 39

40 Converting MPCP instnce to PCP instnce (cont d) Suppose P is the MPCP instnce (over lphbet Θ) {[ t1 b 1 ], [ t2 b 2 ], [ t3 b 3 ],, [ tk b k ]} Now define P to be the PCP instnce (over lphbet Θ {, }) {[ ] [ ] [ ] [ ] [ ]} t1 t2 t3 tk,,,,, b 1 b 2 b 3 b k Note tht the only domino tht cn strt mtch is the first one, becuse it is the only one where both the top nd bottom strt with the sme symbol, nmely,. The originl symbols in Θ now occur in even positions of the mtch. The domino [ ] is to llow the top to dd the extr t the end of the mtch. Models of Computtion 40

41 Lnguge regulr context-free semi-decidble/re Grmmr right-liner context-free unrestricted rules A wb, A w A α α β Mchine DFA or NFA NPDA Turing mchine memory finite finite + one stck unrestricted Other Descr n Regulr expression Myhill-Nerode-Thm λ-clculus µ-recursive funct s RM, Hugs, Oberon,... Exmple { w : w contins 10 } { 0 n 1 m : n, m 0 } { ww R } { 0 n 1 n : n 0 } { 0 n 1 m 0 n m : n 0 } { Mx : M hlts on x } HP Counterexmple { 0 n 1 n : n 0 } { 0 n 1 n 0 n : n 0 } { ww} { M : M hlts on every x } TOTAL Appliction Tokenizer Prser Generl computing Model checker Models of Computtion 41