Theory of Computation COMP363/COMP6363 Prerequisites: COMP4 and COMP 6 (Foundations of Computing) Textbook: Introduction to Automata Theory, Languages and Computation John E. Hopcroft, Rajeev Motwani, and Jeffrey D. Ullman [HMU]. Course assumes one knows: - Sets, functions, relations - Mathematical induction - Any other background material related to Chapter of HMU.
The First Half of the Course... Covered by Badri Vellambi badri.vellambi@anu.edu.au Models of Computation and Languages: - Automata and Regular Languages [.5 weeks] - Pushdown Automata and Context-free Languages [.5 weeks] - Turing Machines and Recursively Enumerable Languages [.5 weeks] Computational Problems: - Decidability, Undecidability, and Intractable Problems [.5 weeks] 2
This Lecture Covers Finite Automata (Chapter 2 of HMU) Deterministic Finite Automaton Nondeterministic Finite Automaton NFA with -transitions An Equivalence among the above three. Reading (from HMU): All of Chapter 2. 3
Preliminary Concepts Alphabet : A finite set of symbols E.g., = {; } (binary alphabet) = {a; b; : : : ; z} (Roman alphabet) String (or word) is a finite sequence of symbols -Usually represented without commas, e.g., instead of (; ; ; ) Concatenation of strings x and y is the string xy = x followed by y is the identity element for concatenation, i.e., x = x = x. Concatenation of sets of strings: AB = {ab : a 2 A; b 2 B} Concatenation of the same set: A 2 = AA; A 3 =(AA)A, etc Kleene or closure operator: = {} [ [ 2 [ 3 denotes the set of all strings. A (formal) language is a subset of. 4
Deterministic Finite Automaton Informally: s s 2 s 3 ::: s [Input tape] [Movable Reading Head] Finite Control q q q 5 q 4 q 2 q 3 The device consisting of: (a) input tape; (b) reading head; and (c) finite control (Finite-state machine) The input is read from left to right Each read operation changes the internal state of the FSM Input is accepted/rejected based on the final state after reading all symbols 5
Deterministic Finite Automaton (DFA) A DFA A =(Q; ; ;q ;F) Q: A finite set (of internal states) : The alphabet corresponding to the input : Q! Q (Transition Function) [If present state is q 2 Q, and a 2 is read, the DFA moves to (q;a).] q : The (unique) starting state of the DFA (prior to any reading). (q 2 Q) F Q is the set of final (or accepting) states Transition Table: Transition Diagram: ; q q 2 q q q 2 q q q 2 q q q 2 q F = {q } (q ; ) = q 2 (q ; ) = q Remark: Each state has exactly one outgoing edge labelled by a symbol 6
Language Accepted by a DFA The language L(A) accepted by a DFA A =(Q; ; ;q ;F) is: The set of all input strings that move the state of the DFA from q to a state in F This is formalized via the extended transition function ˆ : Q! Q: Basis: ) 2) Induction: 3) ˆ (q;) =q ˆ (q;s) = (q;s) if ˆ (q;w) =p, then ˆ (q;ws) = (p; s). [No state change] s 2 s 2 ;w 2 L(A) :=all strings that take q to some final state = {w 2 : ˆ (q ;w) 2 F }: In other words, (a) 2 L(A), q 2 F (b) For k>, w = s s 2 s k 2 L(A), q s! P s 2! P 2! s k! P k 2 F 7
An Example A: ; q q 2 q Is accepted by A? - Need to determine ˆ (q ; ) q! q 2! q 2 =2 F Thus, is not accepted by A Is accepted by A? q! q 2! q 2! q 2 F Thus, is accepted by A. The only way one can reach q from q is if the string contains. L(A) is the set of strings containing. Remark : In general, each string corresponds to a unique path of states. Remark 2: The converse isn t true. For example, and have the same sequence of states. 8
Limitations of DFAs Can all languages be accepted by DFAs? DFAs have a finite number of states (and hence finite memory). Given a DFA, there is always a long pattern it cannot remember or track e.g., L = { n n : n 2 N} cannot be accepted by any DFA. Can generalize DFAs in one of many ways: - Allow transitions to multiple states at each symbol reading. - Allow transitions without reading any symbol - Allow the device to have an additional tape to store symbols - Allow the device to edit the input tape - Allow bidirectional head movement 9
Non-deterministic Finite Automaton (NFA) - Allow transitions to multiple states at each symbol reading. - Multiple transitions allows the device to: (a) clone itself, traverse through and consider all possible parallel outcomes. (b) hypothesize/guess multiple eventualities concerning its input. - Seems bizarre, but aids the implication of describing the automaton. Formal Definition: A =(Q; ; ;q ;F) : Q! 2 Q [Transition Function] Remark 3: Remark 4: (q;s) can be a set with two or more states, or even be empty! If ( ; ) is a singleton for all argument pairs, then NFA is a DFA. [So every DFA is an NFA, by definition!]
Language Accepted by an NFA This is formalized via the extended transition function ˆ : Q! 2 Q : Basis: ) 2) Induction: 3) ˆ (q;) ={q} ˆ (q;s) = (q;s) ˆ (q;ws) = [ k i= (p i ;s); ˆ (q;w) ={p ;:::;p k } ; ˆ (q;w) =; [No state change] s 2 s 2 ;w 2 ˆ (q;ws) w ˆ (q;w) p s. (p ;s) q p k p 2... s.. (p k ;s) L(A) :={w 2 : ˆ (q ;w) \ F 6= ;}. In other words, (a) (b) 2 L(A), q 2 F For k>, w = s s 2 s k 2 L(A), q s! P s 2! P 2! s k! P k 2 F
An Example L(A) ={w : penultimate symbol in w is a }. q q {q ;q } ; q q 2 q 2 q 2 ; ; ; q q q 2 ˆ (q ; ) = {q } ˆ (q ; ) = {q ;q } q! q! q q! q! q q! q! q ˆ (q ; ) = {q ;q 2 } q! q! q q! q! q2 ˆ (q ; ) = {q } q! q! q! q An input can move the state from q to q 2 only if it ends in or. Each time the NFA reads a (in state q ) it considers two parallel possibilities: (a) the is the penultimate symbol. [These paths die if the is not actually the penultimate symbol] (b) the is not the penultimate symbol. 2
Is Non-determinism Better? Non-determinism was introduced to increase the computational power. So is there a language L that is accepted by an NDA, but not by any DFA? Theorem : Every Language L that is accepted by an NFA is also accepted by some DFA. 3
Proof of Theorem ) Let N =(Q N ; ; N ;q ;F N ) generate the given language L Idea: Devise a DFA D such that at any time instant the state of the DFA is the set of all states that NFA N can be in. 2) Define DFA D =(Q D ; ; D ;q D; ;F D ) from N using the following subset construction: Q D =2 Q N q D; = {q } F D = {S Q N : S \ F N 6= ;} Example: N : ; D : {q } {q ;q 2 } ; q q q 2 ; {q 2 } {q ;q } {q ;q ;q 2 } {q } {q ;q 2 } 3) Hence, 2 L(N), q 2 F, {q } 2 F D, 2 L(D) 4
Proof of Theorem 4) To define D (P; s) for each P Q and s 2 : -Assume NFA N is simultaneously in all states of P -Let P be the states to which N can transition from states in P upon reading s -Set D (P; s) :=P = S p2p N(p; s). s N: D : P P, s P! P 5) Induction step: Basis: Let s 2 ˆ N (q ;s) def = N (q ;s)= [ p2{q } N (p; s) def = D ({q };s) def = ˆ D ({q };s) Induction: Let ˆ N (q ;w)=ˆ D ({q };w) for w 2 \{} ˆ N (q ;ws) def [ = N (q ;s) ind [ = N (q ;s) def = D (ˆ D ({q };w);s) def = ˆ D ({q };ws) p2ˆ N (q ;w) p2ˆ D ({q };w) 6) Thus, ˆ N (q ; ) =ˆ D ({q }; ), and hence the languages have to be identical. [QED] 5
Comments about Subset construction Generally, the DFA constructed using subset construction has 2 n states (n = number of states in the NFA) Not all states are reachable! (see example below) The state corresponding to the empty set is never a final state. ; q q ; q 2 D : ; ; ; ; {q 2 } {q } ; {q ;q 2 } {q ;q } {q ;q ;q 2 } {q } {q ;q 2 } 6
-Transitions State transitions occur without reading any symbols. An -Nondeterministic Finite Automaton is a 5-tuple (Q; ; ;q ;F), where: Q; ;q ; and F are as in an NFA : Q ( [ {})! 2 Q a b Example: q q 2 q 3 q q q 2 {q ;q 4 } {q 2 } {q 3 } ; ; ; ; ; ; q b q 4 a q 5 q 6 q 3 q 4 q 5 q 6 ; ; {q 6 } ; ; ; {q 5 } ; ; {q 3 } ; ; Without reading any input symbols, the state of the -NFA can transition - From q to q, q 4, q 2, or q 3. - From q to q 2, or q 3. - From q 2 to q 3. 7 - From q 5 to q 6.
Language accepted by an -NFA -closure of a state ECLOSE(q) = all states that are reachable from q by -transitions alone. q q 2 q 3 q b q 4 q 5 q 6 a ECLOSE(q )={q ;q ;q 4 ;q 2 ;q 3 } ECLOSE(q )={q ;q 2 ;q 3 } ECLOSE(q 2 )={q 2 ;q 3 } ECLOSE(q 3 )={q 3 } ECLOSE(q 4 )={q 4 } ECLOSE(q 5 )={q 5 ;q 6 } ECLOSE(q 6 )={q 6 } 8
Language accepted by an -NFA Given -NFA N =(Q; ; ;q ;F) extended transition function ˆ : Q! 2 Q by Basis: ) ˆ (q;) =ECLOSE(q) q q ::: q = 2 = 3 = 2) ˆ (q;s) = [ @ [ ECLOSE(p ) A p2eclose(q) p 2 (p;s) [s = {z } finitely many s {z } ] finitely many q s q ::: q p p ::: p Induction: 3) ˆ (q;ws) = [ @ [ ECLOSE(p ) A p2ˆ (q;w) p 2 (p;s) q w s ˆ (q;ws) w 2 L(N) if and only if ˆ (q ; w) \ F 6= ; ˆ (q;w)
Language accepted by an -NFA w 2 L(N) if and only if ˆ (q ; w) \ F 6= ; In other words, (a) 2 L(N), ECLOSE(q ) \ F 6= ; q p ::: p r 2 F (b) For k>, q ::: s p p ::: s 2 p 2 w = s s 2 s k 2 L(A),. p k ::: s k p k p k ::: q F 2 F 2
Are -NFAs Better? Theorem 2: Every Language L that is accepted by an -NFA is also accepted by some DFA. Proof: Given L that is accepted by some -NFA, we must find an NFA that accepts L. [NFA to DFA conversion can be done as in Theorem ]. Let -NFA N =(Q N ; ; N ;q ;F N ) accept L. Let us devise NFA N =(Q N ; ; N ;q ;F N ) as follows: Q N = Q N : q = q FN = {q 2 Q N : ECLOSE(q) \ F N 6= ;} N : Q N! 2 Q N defined by: N (q;s) = [ p2eclose(q) (p; s) s N : q ::: p p N : q can transition to p after a few -transitions, and a single read of s 2. m s N : q p N : q can transition to p after reading s. 2
Are -NFAs Better? (Proof continued) [Handwavy, but can be formalized!] N : s :::s k is accepted by -NFA N s :::s k is accepted by NFA N q ::: p p k p k m ::: :::. ::: s k s s 2 q F 2 F p p 2 p k, N : q s m p p 2. p k s 2 s k ECLOSE(p k ) \ F N 6= ; 22 [QED]
To Summarize... Nondeterminism and -transitions do not offer computational benefits Languages accepted by DFAs Languages accepted by NFAs = = Languages accepted by -NFAs 23