Thomas Zeugmann Hokkaido University Laboratory for Algorithmics http://www-alg.ist.hokudai.ac.jp/ thomas/toc/ Lecture 3: Finite State Automata
Motivation In the previous lecture we learned how to formalize the generation of languages, i.e., we looked at formal languages from the perspective of a speaker.
Motivation In the previous lecture we learned how to formalize the generation of languages, i.e., we looked at formal languages from the perspective of a speaker. Now, we turn our attention to accepting languages, i.e., we are going to formalize the perspective of a listener.
Motivation In the previous lecture we learned how to formalize the generation of languages, i.e., we looked at formal languages from the perspective of a speaker. Now, we turn our attention to accepting languages, i.e., we are going to formalize the perspective of a listener. In this lecture we deal with regular languages, and the machine model accepting them.
Overall Goal Let Σ be again an alphabet, and let L Σ be any regular language. Now, for every string s Σ we want to have a possibility to decide whether or not s L.
Overall Goal Let Σ be again an alphabet, and let L Σ be any regular language. Now, for every string s Σ we want to have a possibility to decide whether or not s L. Looking at the definition of a regular grammar, the following method may be easily discovered. We start generating strings until one of the following two conditions happens: (1) The string s is generated. Clearly, then s L. (2) The length of our string s is exceeded. Taking into account that all further generable strings must be longer, we may conclude that s L.
Overall Goal Let Σ be again an alphabet, and let L Σ be any regular language. Now, for every string s Σ we want to have a possibility to decide whether or not s L. Looking at the definition of a regular grammar, the following method may be easily discovered. We start generating strings until one of the following two conditions happens: (1) The string s is generated. Clearly, then s L. (2) The length of our string s is exceeded. Taking into account that all further generable strings must be longer, we may conclude that s L. But this method lacks efficiency. It may take time that is exponential in the length of the input string s to terminate.
Nondeterministic Finite Automata I The approach described at the previous slide hardly reflects what humans are doing when accepting sentences of natural languages. Therefore, we favor another approach.
Nondeterministic Finite Automata I The approach described at the previous slide hardly reflects what humans are doing when accepting sentences of natural languages. Therefore, we favor another approach. Idea: Looking at a regular grammar, we have productions σ a σ aσ
Nondeterministic Finite Automata I The approach described at the previous slide hardly reflects what humans are doing when accepting sentences of natural languages. Therefore, we favor another approach. Idea: Looking at a regular grammar, we have productions σ a σ aσ Now, we simply change the direction of the, i.e., σ a σ aσ
Nondeterministic Finite Automata II This leads to the following definition: Definition 1 (NDFA) A 5-tuple A = [Σ, Q, δ, q 0, F] is said to be a nondeterministic finite automaton if (1) Σ is an alphabet (the so-called input alphabet); (2) Q is a finite nonempty set (the set of states); (3) δ: Q Σ (Q), the transition relation; (4) q 0 Q, the initial state, and (5) F Q, the set of final states.
Nondeterministic Finite Automata II This leads to the following definition: Definition 1 (NDFA) A 5-tuple A = [Σ, Q, δ, q 0, F] is said to be a nondeterministic finite automaton if (1) Σ is an alphabet (the so-called input alphabet); (2) Q is a finite nonempty set (the set of states); (3) δ: Q Σ (Q), the transition relation; (4) q 0 Q, the initial state, and (5) F Q, the set of final states. In the definition above, (Q) denotes the power set of Q, i.e., the set of all subsets of Q.
Deterministic Finite Automata There is also a deterministic counterpart of a nondeterministic finite automaton which we define next.
Deterministic Finite Automata There is also a deterministic counterpart of a nondeterministic finite automaton which we define next. Definition 2 (DFA) A 5-tuple A = [Σ, Q, δ, q 0, F] is said to be a deterministic finite automaton if (1) Σ is an alphabet (the so-called input alphabet); (2) Q is a finite nonempty set (the set of states); (3) δ: Q Σ Q, the transition function, which must be defined for every input; (4) q 0 Q, the initial state, and (5) F Q, the set of final states.
Computing with Finite Automata I When we do not want to specify whether an automaton is deterministic or nondeterministic, we simply refer to it as a finite automaton.
Computing with Finite Automata I When we do not want to specify whether an automaton is deterministic or nondeterministic, we simply refer to it as a finite automaton. So far, we have explained what an automaton is but not what it does. In order to explain how to compute with an automaton, we need some more definitions. For the deterministic case, we can easily define the language accepted by a finite automaton.
Computing with Finite Automata I When we do not want to specify whether an automaton is deterministic or nondeterministic, we simply refer to it as a finite automaton. So far, we have explained what an automaton is but not what it does. In order to explain how to compute with an automaton, we need some more definitions. For the deterministic case, we can easily define the language accepted by a finite automaton. First, we extend the definition of δ to strings. That is, formally we inductively define a function by setting δ : Q Σ Q, δ (q, λ) = q for all q Q, δ (q, sa) = δ(δ (q, s), a) for all s Σ, a Σ, and q Q.
Computing with Finite Automata II Furthermore, we also need the following lemma: Lemma 1 Let A = [Σ, Q, δ, q 0, F] be a deterministic finite automaton. Then for all strings v, w Σ and all q Q we have δ (q, vw) = δ (δ (q, v), w).
Computing with Finite Automata II Furthermore, we also need the following lemma: Lemma 1 Let A = [Σ, Q, δ, q 0, F] be a deterministic finite automaton. Then for all strings v, w Σ and all q Q we have δ (q, vw) = δ (δ (q, v), w). Now, we can define the language accepted by a deterministic finite automaton. Definition 3 Let A = [Σ, Q, δ, q 0, F] be a deterministic finite automaton. The language L(A) accepted by A is L(A) = {s s Σ, δ (q 0, s) F}.
Computing with Finite Automata III If we have s L(A) for a string s Σ then we say that there is an accepting computation for s. We adopt this notion also to nondeterministic automata. Note that λ L(A) iff q 0 F.
Computing with Finite Automata III If we have s L(A) for a string s Σ then we say that there is an accepting computation for s. We adopt this notion also to nondeterministic automata. Note that λ L(A) iff q 0 F. In order to keep notation simple, in the following we shall identify δ with δ. It should be clear from context what is meant.
Computing with Finite Automata III If we have s L(A) for a string s Σ then we say that there is an accepting computation for s. We adopt this notion also to nondeterministic automata. Note that λ L(A) iff q 0 F. In order to keep notation simple, in the following we shall identify δ with δ. It should be clear from context what is meant. Well, this seems very abstract, and so some explanation is in order.
Computing with Finite Automata IV Conceptually, a finite automaton possesses an input tape that is divided into cells. Each cell can store a symbol from Σ or it may be empty. Additionally, a finite automaton has a head to read what is stored in the cells. Initially, a string s = s 1 s 2 s k is written on the tape and the head is positioned on the leftmost symbol of the input, i.e., on s 1.
Computing with Finite Automata IV Conceptually, a finite automaton possesses an input tape that is divided into cells. Each cell can store a symbol from Σ or it may be empty. Additionally, a finite automaton has a head to read what is stored in the cells. Initially, a string s = s 1 s 2 s k is written on the tape and the head is positioned on the leftmost symbol of the input, i.e., on s 1. s 1 s 2 s k s k+1 head moves in this direction, one finite state control cell at a time
Computing with Finite Automata V s 1 s 2 s k s k+1 head moves in this direction, one finite state control cell at a time The automaton is put into its initial state q 0. Now, it reads s 1. Then, it changes its state to one of the possible states in δ(q 0, s 1 ), say q, and the head moves right to the next cell. In the deterministic case, the state δ(q 0, s 1 ) is uniquely defined. Next, s 2 is read, and the automaton changes its state to one of the possible states in δ(q, s 2 ). This process is iterated until the automaton reaches the first cell which is empty. After having read the whole string, the automaton is in some state, say r. If r F, then the computation has been an accepting one, otherwise, the string s is rejected.
Computing with Finite Automata VI Now, we see what the problem is in defining the language accepted by a nondeterministic finite automaton. On input a string s, there are many possible computations. Some of these computations may finish in an accepting state and some may not. We therefore define the language accepted by a nondeterministic finite automaton as follows:
Computing with Finite Automata VI Now, we see what the problem is in defining the language accepted by a nondeterministic finite automaton. On input a string s, there are many possible computations. Some of these computations may finish in an accepting state and some may not. We therefore define the language accepted by a nondeterministic finite automaton as follows: Definition 4 Let A = [Σ, Q, δ, q 0, F] be a nondeterministic finite automaton. The language L(A) accepted by A is the set of all strings s Σ such that there exists an accepting computation for s.
State Diagrams I Finite automata may be conveniently represented by their state diagram. The state diagram is a directed graph whose nodes are labeled by the states of the automaton. The edges are labeled by symbols from Σ. Let p and q be nodes. Then, there is a directed edge from p to q if and only if there exists an a Σ such that q = δ(p, a) (deterministic case) or q δ(p, a) (nondeterministic case). Final states have an extra circle. The state diagram of a finite automaton accepting the language L = {a i b j i 0, j 0} is shown below.
State Diagrams I Finite automata may be conveniently represented by their state diagram. The state diagram is a directed graph whose nodes are labeled by the states of the automaton. The edges are labeled by symbols from Σ. Let p and q be nodes. Then, there is a directed edge from p to q if and only if there exists an a Σ such that q = δ(p, a) (deterministic case) or q δ(p, a) (nondeterministic case). Final states have an extra circle. The state diagram of a finite automaton accepting the language L = {a i b j i 0, j 0} is shown below. a b a, b b 1 2 a 3
Illustrating the Example First, let us consider the computation performed by the state diagram on input abb. Having read a a b a, b b 1 2 a 3
Illustrating the Example First, let us consider the computation performed by the state diagram on input abb. Having read ab a b a, b b 1 2 a 3
Illustrating the Example First, let us consider the computation performed by the state diagram on input abb. Having read abb a b a, b b 1 2 a 3 Thus, the string abb is accepted.
Illustrating the Example Second, let us consider the computation performed by the state diagram on input ba. Having read b a b a, b b 1 2 a 3
Illustrating the Example Second, let us consider the computation performed by the state diagram on input ba. Having read ba a b a, b b 1 2 a 3 Thus, the string ba is rejected.
Automata and Regular Languages Now that we know what finite automata are, we can answer the question you probably have already in mind, i.e., Question What have finite automata to do with regular languages?
Automata and Regular Languages Now that we know what finite automata are, we can answer the question you probably have already in mind, i.e., Question What have finite automata to do with regular languages? For answering this question, we show the following theorem:
Main Theorem Theorem 2 Let L Σ be any language. Then, the following three assertions are equivalent: (1) There exists a deterministic finite automaton A such that L = L(A). (2) There exists a nondeterministic finite automaton A such that L = L(A). (3) L is regular. We show the equivalence by proving (1) implies (2), (2) implies (3), and (3) implies (1).
Proof of the Main Theorem Claim 1. (1) implies (2). This is obvious by definition, since a deterministic finite automaton is a special case of a nondeterministic one.
Proof of the Main Theorem Claim 2. (2) implies (3). Let A = [Σ, Q, δ, q 0, F] be a nondeterministic finite automaton such that L = L(A). We have to construct a grammar G generating L. Let G = [Σ, Q {σ}, σ, P], where P is the following set of productions: P = {σ q 0 } {p aq a Σ, p, q Q, q δ(p, a)} {p λ p F}. Obviously, G is regular. We have to show L(G) = L(A). First we prove L(A) L(G).
Proof of the Main Theorem Let s = a 1 a k L. Then, there exists an accepting computation of A for s. Let q 0, p 1,..., p k be the sequence of states through which A goes while performing this accepting computation. Therefore, p 1 δ(q 0, a 1 ), p 2 δ(p 1, a 2 ),..., p k δ(p k 1, a k ), and p k F. Thus, σ q 0 a 1 p 1 a 1 a k 1 p k 1 a 1 a k p k a 1 a k. Hence, s L(G).
Proof of the Main Theorem Let s = a 1 a k L. Then, there exists an accepting computation of A for s. Let q 0, p 1,..., p k be the sequence of states through which A goes while performing this accepting computation. Therefore, p 1 δ(q 0, a 1 ), p 2 δ(p 1, a 2 ),..., p k δ(p k 1, a k ), and p k F. Thus, σ q 0 a 1 p 1 a 1 a k 1 p k 1 a 1 a k p k a 1 a k. Hence, s L(G). The direction L(G) L(A) can be proved analogously, and is therefore left as an exercise.
Proof of the Main Theorem Claim 3. (3) implies (1). In principle, we want to use an idea similar to the one used to prove Claim 2. But productions may have strings on their right hand side, while the transition function has to be defined over states and letters. Therefore, we first have to show a normal form lemma for regular grammars.
Proof of the Main Theorem Claim 3. (3) implies (1). In principle, we want to use an idea similar to the one used to prove Claim 2. But productions may have strings on their right hand side, while the transition function has to be defined over states and letters. Therefore, we first have to show a normal form lemma for regular grammars. Normal Form Lemma For every regular grammar G = [T, N, σ, P] there exists a grammar G such that L(G) = L(G ) and all productions of G have the form h ah or h λ, where h, h N and a T.
Proof of the Normal Form Lemma First, each production of the form h a 1 a k h, k > 0, is equivalently replaced by the following productions: h a 1 h a2 a k h, h a 2 a k h a 2h a3 a k h,..., h ak h a kh.
Proof of the Normal Form Lemma First, each production of the form h a 1 a k h, k > 0, is equivalently replaced by the following productions: h a 1 h a2 a k h, h a 2 a k h a 2h a3 a k h,..., h ak h a kh. Next, each production of the form h a 1 a k, k > 0, is equivalently replaced by the following productions: h a 1 h a2 a k, h a2 a k a 2 h a3 a k,..., h ak a k h λ and h λ λ.
Proof of the Normal Form Lemma First, each production of the form h a 1 a k h, k > 0, is equivalently replaced by the following productions: h a 1 h a2 a k h, h a 2 a k h a 2h a3 a k h,..., h ak h a kh. Next, each production of the form h a 1 a k, k > 0, is equivalently replaced by the following productions: h a 1 h a2 a k, h a2 a k a 2 h a3 a k,..., h ak a k h λ and h λ λ. Finally, we have to deal with productions of the form h h where h, h N.
Proof of the Normal Form Lemma continued Let Ĝ = [T, ˆN, σ, ˆP] be the grammar constructed so far. Furthermore, let U(h) = df {h h ˆN and h h }.
Proof of the Normal Form Lemma continued Let Ĝ = [T, ˆN, σ, ˆP] be the grammar constructed so far. Furthermore, let U(h) = df {h h ˆN and h h }. Clearly, U(h) is computable. Now, we delete all productions of the form h h and add the following productions to ˆP: If h λ ˆP and h U(h), then we add h λ. Moreover, we add all h xh for all productions in ˆP such that there is a h U(h) with h xh ˆP.
Proof of the Normal Form Lemma continued Let Ĝ = [T, ˆN, σ, ˆP] be the grammar constructed so far. Furthermore, let U(h) = df {h h ˆN and h h }. Clearly, U(h) is computable. Now, we delete all productions of the form h h and add the following productions to ˆP: If h λ ˆP and h U(h), then we add h λ. Moreover, we add all h xh for all productions in ˆP such that there is a h U(h) with h xh ˆP. Let G be the resulting grammar. Clearly, now all productions have the desired form. One easily verifies that L(G) = L(G ). This proves the lemma.
Illustration of Normal Form Lemma Let G = [{a, b}, {σ, h}, σ, P] be the grammar given, where P = {σ abσ, σ h, h aah, h λ}. Applying the transformation, first we obtain: ˆP = {σ ah bσ, h bσ bσ, σ h, h ah ah, h ah ah, h λ}. Now, U(σ) = {h}, and U(h) =. Thus, we delete the production σ h and replace it by σ ah ah and σ λ. Summarizing this construction, we now have the following set P of productions: P = {σ ah bσ, h bσ bσ, σ ah ah, σ λ, h ah ah, h ah ah, h λ} as well as the following set N of nonterminals: N = {σ, h ah, h bσ, h}.
Proof of (3) implies (1) continued Now, assume that we are given a grammar G = [T, N, σ, P] that is already in the normal form described in the lemma above. We define a deterministic finite automaton A = [T, Q, δ, q 0, F] as follows:
Proof of (3) implies (1) continued Now, assume that we are given a grammar G = [T, N, σ, P] that is already in the normal form described in the lemma above. We define a deterministic finite automaton A = [T, Q, δ, q 0, F] as follows: Let Q = (N) and q 0 = {σ}. The transition function δ is defined as δ(p, a) = {h h[h p and h ah P]}.
Proof of (3) implies (1) continued Now, assume that we are given a grammar G = [T, N, σ, P] that is already in the normal form described in the lemma above. We define a deterministic finite automaton A = [T, Q, δ, q 0, F] as follows: Let Q = (N) and q 0 = {σ}. The transition function δ is defined as δ(p, a) = {h h[h p and h ah P]}. Finally, we set F = {p h[h p and h λ P]}. We leave it as an exercise to prove L(A) = L(G).
Deciding Membership Theorem 2 directly allows for the following corollary: Corollary 3 There is an algorithm that on input any regular grammar G = [T, N, σ, P] and any string s T decides whether or not s L(G).
Deciding Membership Theorem 2 directly allows for the following corollary: Corollary 3 There is an algorithm that on input any regular grammar G = [T, N, σ, P] and any string s T decides whether or not s L(G). Proof. As the proof of Theorem 2 shows, given any regular grammar G we can construct a deterministic finite automaton A such that L(G) = L(A). Since A is deterministic, on input any string s T it either accepts s or it rejects it.
Showing Non-Regularity of Languages I Finally, we show how to use finite automata to prove that particular languages are not regular. Consider the following grammar: G = [{a, b}, {σ}, σ, {σ aσb, σ λ}]. Then, L(G) = {a i b i i 0}.
Showing Non-Regularity of Languages I Finally, we show how to use finite automata to prove that particular languages are not regular. Consider the following grammar: G = [{a, b}, {σ}, σ, {σ aσb, σ λ}]. Then, L(G) = {a i b i i 0}. Clearly, G is not regular, but this does not prove that there is no regular grammar at all that can generate this language.
Showing Non-Regularity of Languages I Finally, we show how to use finite automata to prove that particular languages are not regular. Consider the following grammar: G = [{a, b}, {σ}, σ, {σ aσb, σ λ}]. Then, L(G) = {a i b i i 0}. Clearly, G is not regular, but this does not prove that there is no regular grammar at all that can generate this language. Theorem 4 The language L = {a i b i i 0} is not regular.
Showing Non-Regularity of Languages II Proof. Suppose the converse. Then, by our Main Theorem there must be a deterministic finite automaton A = [Σ, Q, δ, q 0, F] such that L(A) = L. Clearly, {a, b} Σ, and thus δ(q 0, a i ) must be defined for all i. However, there are only finitely many states, but infinitely many i, and hence there must exist i, j such that i j but δ(q 0, a i ) = δ(q 0, a j ).
Showing Non-Regularity of Languages II Proof. Suppose the converse. Then, by our Main Theorem there must be a deterministic finite automaton A = [Σ, Q, δ, q 0, F] such that L(A) = L. Clearly, {a, b} Σ, and thus δ(q 0, a i ) must be defined for all i. However, there are only finitely many states, but infinitely many i, and hence there must exist i, j such that i j but δ(q 0, a i ) = δ(q 0, a j ). Since the automaton is deterministic, δ(q 0, a i ) = δ(q 0, a j ) implies δ(q 0, a i b i ) = δ(q 0, a j b i ). Let q = δ(q 0, a i b i ); then, if q F we directly obtain a j b i L, a contradiction, since i j. But if q F, then a i b i is rejected. This is again a contradiction, since a i b i L. Thus, L is not regular.
Thank you!