Deterministic Finite Automata (DFAs)

CS/ECE 374: Algorithms & Models of Computation, Fall 28 Deterministic Finite Automata (DFAs) Lecture 3 September 4, 28 Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Part I DFA Introduction Chandra Chekuri (UIUC) CS/ECE 374 2 Fall 28 2 / 33

DFAs also called Finite State Machines (FSMs) The simplest model for computers? State machines that are very common in practice. Vending machines Elevators Digital watches Simple network protocols Programs with fixed memory Chandra Chekuri (UIUC) CS/ECE 374 3 Fall 28 3 / 33

A simple program Program to check if a given input string w has odd length int n = While input is not finished read next character c n n + endwhile If (n is odd) output YES Else output NO Chandra Chekuri (UIUC) CS/ECE 374 4 Fall 28 4 / 33

A simple program Program to check if a given input string w has odd length int n = While input is not finished read next character c n n + endwhile If (n is odd) output YES Else output NO bit x = While input is not finished read next character c x flip(x) endwhile If (x = ) output YES Else output NO Chandra Chekuri (UIUC) CS/ECE 374 4 Fall 28 4 / 33

Another view Machine has input written on a read-only tape Start in specified start state Start at left, scan symbol, change state and move right Circled states are accepting Machine accepts input string if it is in an accepting state after scanning the last symbol. Chandra Chekuri (UIUC) CS/ECE 374 5 Fall 28 5 / 33

Graphical Representation/State Machine q q 3 q q 2, Directed graph with nodes representing states and edge/arcs representing transitions Figure : labeled DFA M by forsymbols problem 5in Σ For each state (vertex) q and symbol a Σ there is exactly one outgoing edge labeled by a hange the initial state to B? Initial/start state has a pointer (or labeled as s, q or start ) Some states with double circles labeled as accepting/final states hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 6 Fall 28 6 / 33

Graphical Representation q q 3 q q 2, Convention: Machine reads symbols from left to right Figure : DFA M for problem 5 Where does lead?? hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 7 Fall 28 7 / 33

Graphical Representation q q 3 q q 2, Convention: Machine reads symbols from left to right Figure : DFA M for problem 5 Where does lead?? Which strings end up in accepting state? hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 7 Fall 28 7 / 33

Graphical Representation q q 3 q q 2, Convention: Machine reads symbols from left to right Figure : DFA M for problem 5 Where does lead?? Which strings end up in accepting state? Every string w has a unique walk that it follows from a given hange the initial state to B? state q by reading one letter of w from left to right. hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 7 Fall 28 7 / 33

Graphical Representation q q 3 q q 2, Definition Figure : DFA M for problem 5 A DFA M accepts a string w iff the unique walk starting at the start state and spelling out w ends in an accepting state. hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 8 Fall 28 8 / 33

Graphical Representation q q 3 q q 2, Definition Figure : DFA M for problem 5 A DFA M accepts a string w iff the unique walk starting at the start state and spelling out w ends in an accepting state. hange Definition the initial state to B? The language accepted (or recognized) by a DFA M is denote by L(M) and defined as: L(M) = {w M accepts w}. hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 8 Fall 28 8 / 33

Warning M accepts language L does not mean simply that that M accepts each string in L. It means that M accepts each string in L and no others. Equivalently M accepts each string in L and does not accept/rejects strings in Σ \ L. Chandra Chekuri (UIUC) CS/ECE 374 9 Fall 28 9 / 33

Formal Tuple Notation Definition A deterministic finite automata (DFA) M = (Q, Σ, δ, s, A) is a five tuple where Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Formal Tuple Notation Definition A deterministic finite automata (DFA) M = (Q, Σ, δ, s, A) is a five tuple where Q is a finite set whose elements are called states, Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Formal Tuple Notation Definition A deterministic finite automata (DFA) M = (Q, Σ, δ, s, A) is a five tuple where Q is a finite set whose elements are called states, Σ is a finite set called the input alphabet, δ : Q Σ Q is the transition function, s Q is the start state, Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Example q q 3 q q 2, Q = Figure : DFA M for problem 5 hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Example q q 3 q q 2, Q = {q, q, q, q 3 } Figure : DFA M for problem 5 hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Example q q 3 q q 2, Q = {q, q, q, q 3 } Figure : DFA M for problem 5 Σ = hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Example q q 3 q q 2, Q = {q, q, q, q 3 } Figure : DFA M for problem 5 Σ = {, } hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Example q q 3 q q 2, Q = {q, q, q, q 3 } Figure : DFA M for problem 5 Σ = {, } δ hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Example q q 3 q q 2, Q = {q, q, q, q 3 } Figure : DFA M for problem 5 Σ = {, } δ s = hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Example q q 3 q q 2, Q = {q, q, q, q 3 } Figure : DFA M for problem 5 Σ = {, } δ s = q hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Example q q 3 q q 2, Q = {q, q, q, q 3 } Figure : DFA M for problem 5 Σ = {, } δ s = q hange the initial state to B? A = hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Example q q 3 q q 2, Q = {q, q, q, q 3 } Figure : DFA M for problem 5 Σ = {, } δ s = q hange the initial state to B? A = {q } hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 Fall 28 / 33

Extending the transition function to strings Given DFA M = (Q, Σ, δ, s, A), δ(q, a) is the state that M goes to from q on reading letter a Useful to have notation to specify the unique state that M will reach from q on reading string w Transition function δ : Q Σ Q defined inductively as follows: δ (q, w) = q if w = ɛ δ (q, w) = δ (δ(q, a), x) if w = ax. Chandra Chekuri (UIUC) CS/ECE 374 2 Fall 28 2 / 33

Formal definition of language accepted by M Definition The language L(M) accepted by a DFA M = (Q, Σ, δ, s, A) is {w Σ δ (s, w) A}. Chandra Chekuri (UIUC) CS/ECE 374 3 Fall 28 3 / 33

Example q q 3 q q 2, What is: δ (q, ɛ) Figure : DFA M for problem 5 δ (q, ) δ (q, ) hange the initial state to B? δ (q 4, ) hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 4 Fall 28 4 / 33

Example continued q q 3 q q 2, What is L(M) if start state is changed to q? Figure : DFA M for problem 5 What is L(M) if final/accepte states are set to {q 2, q 3 } instead of {q }? hange the initial state to B? hange the set of final states to be {B} (with initial state A)? Chandra Chekuri (UIUC) CS/ECE 374 5 Fall 28 5 / 33

Advantages of formal specification Necessary for proofs Necessary to specify abstractly for class of languages Exercise: Prove by induction that for any two strings u, v, and any state q, δ (q, uv) = δ (δ (q, u), v). Chandra Chekuri (UIUC) CS/ECE 374 6 Fall 28 6 / 33

Part II Constructing DFAs Chandra Chekuri (UIUC) CS/ECE 374 7 Fall 28 7 / 33

DFAs: State = Memory How do we design a DFA M for a given language L? That is L(M) = L. DFA is a like a program that has fixed amount of memory independent of input size. The memory of a DFA is encoded in its states The state/memory must capture enough information from the input seen so far that it is sufficient for the suffix that is yet to be seen (note that DFA cannot go back) Chandra Chekuri (UIUC) CS/ECE 374 8 Fall 28 8 / 33

DFA Construction: Example Assume Σ = {, } L =, L = Σ, L = {ɛ}, L = {}. Chandra Chekuri (UIUC) CS/ECE 374 9 Fall 28 9 / 33

DFA Construction: Example Assume Σ = {, } L =, L = Σ, L = {ɛ}, L = {}. L = {w {, } w is divisible by 5} Chandra Chekuri (UIUC) CS/ECE 374 9 Fall 28 9 / 33

DFA Construction: Example Assume Σ = {, } L =, L = Σ, L = {ɛ}, L = {}. L = {w {, } w is divisible by 5} L = {w {, } w ends with } Chandra Chekuri (UIUC) CS/ECE 374 9 Fall 28 9 / 33

DFA Construction: Example Assume Σ = {, } L =, L = Σ, L = {ɛ}, L = {}. L = {w {, } w is divisible by 5} L = {w {, } w ends with } L = {w {, } w contains as substring} Chandra Chekuri (UIUC) CS/ECE 374 9 Fall 28 9 / 33

DFA Construction: Example Assume Σ = {, } L =, L = Σ, L = {ɛ}, L = {}. L = {w {, } w is divisible by 5} L = {w {, } w ends with } L = {w {, } w contains as substring} L = {w {, } w contains or as substring} Chandra Chekuri (UIUC) CS/ECE 374 9 Fall 28 9 / 33

DFA Construction: Example L = {Binary numbers congruent to mod 5} Example: = 7 = 2 mod 5, = = mod 5 Chandra Chekuri (UIUC) CS/ECE 374 2 Fall 28 2 / 33

DFA Construction: Example L = {Binary numbers congruent to mod 5} Example: = 7 = 2 mod 5, = = mod 5 Key observation: w mod 5 = a implies w mod 5 = 2a mod 5 and w mod 5 = (2a + ) mod 5 Chandra Chekuri (UIUC) CS/ECE 374 2 Fall 28 2 / 33

Part III Product Construction and Closure Properties Chandra Chekuri (UIUC) CS/ECE 374 2 Fall 28 2 / 33

Part IV Complement Chandra Chekuri (UIUC) CS/ECE 374 22 Fall 28 22 / 33

gnized by M i L = L(M). Complement Question: If M is a DFA, is there a DFA M such that L(M ) = Σ \ L(M)? That is, are languages recognized by DFAs closed under complement? q q 3 q q 2, Figure : DFA M for problem 5 Chandra Chekuri (UIUC) CS/ECE 374 23 Fall 28 23 / 33

Complement Theorem Languages accepted by DFAs are closed under complement. Chandra Chekuri (UIUC) CS/ECE 374 24 Fall 28 24 / 33

Complement Theorem Languages accepted by DFAs are closed under complement. Proof. Let M = (Q, Σ, δ, s, A) such that L = L(M). Let M = (Q, Σ, δ, s, Q \ A). Claim: L(M ) = L. Why? Chandra Chekuri (UIUC) CS/ECE 374 24 Fall 28 24 / 33

Complement Theorem Languages accepted by DFAs are closed under complement. Proof. Let M = (Q, Σ, δ, s, A) such that L = L(M). Let M = (Q, Σ, δ, s, Q \ A). Claim: L(M ) = L. Why? δ M = δ M. Thus, for every string w, δ M (s, w) = δ M (s, w). δ M (s, w) A δ M (s, w) Q \ A. δ M (s, w) A δ M (s, w) Q \ A. Chandra Chekuri (UIUC) CS/ECE 374 24 Fall 28 24 / 33

Part V Product Construction Chandra Chekuri (UIUC) CS/ECE 374 25 Fall 28 25 / 33

Union and Intersection Question: Are languages accepted by DFAs closed under union? That is, given DFAs M and M 2 is there a DFA that accepts L(M ) L(M 2 )? How about intersection L(M ) L(M 2 )? Idea from programming: on input string w Simulate M on w Simulate M 2 on w If both accept than w L(M ) L(M 2 ). If at least one accepts then w L(M ) L(M 2 ). Chandra Chekuri (UIUC) CS/ECE 374 26 Fall 28 26 / 33

Union and Intersection Question: Are languages accepted by DFAs closed under union? That is, given DFAs M and M 2 is there a DFA that accepts L(M ) L(M 2 )? How about intersection L(M ) L(M 2 )? Idea from programming: on input string w Simulate M on w Simulate M 2 on w If both accept than w L(M ) L(M 2 ). If at least one accepts then w L(M ) L(M 2 ). Catch: We want a single DFA M that can only read w once. Chandra Chekuri (UIUC) CS/ECE 374 26 Fall 28 26 / 33

Union and Intersection Question: Are languages accepted by DFAs closed under union? That is, given DFAs M and M 2 is there a DFA that accepts L(M ) L(M 2 )? How about intersection L(M ) L(M 2 )? Idea from programming: on input string w Simulate M on w Simulate M 2 on w If both accept than w L(M ) L(M 2 ). If at least one accepts then w L(M ) L(M 2 ). Catch: We want a single DFA M that can only read w once. Solution: Simulate M and M 2 in parallel by keeping track of states of both machines Chandra Chekuri (UIUC) CS/ECE 374 26 Fall 28 26 / 33

Example M accepts # = odd M 2 accepts # = odd Chandra Chekuri (UIUC) CS/ECE 374 27 Fall 28 27 / 33

Example M accepts # = odd M 2 accepts # = odd Cross-product machine Chandra Chekuri (UIUC) CS/ECE 374 28 Fall 28 28 / 33

Product construction for intersection M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Chandra Chekuri (UIUC) CS/ECE 374 29 Fall 28 29 / 33

Product construction for intersection M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Q = Chandra Chekuri (UIUC) CS/ECE 374 29 Fall 28 29 / 33

Product construction for intersection M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Q = Q Q 2 = {(q, q 2 ) q Q, q 2 Q 2 } Chandra Chekuri (UIUC) CS/ECE 374 29 Fall 28 29 / 33

Product construction for intersection M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Q = Q Q 2 = {(q, q 2 ) q Q, q 2 Q 2 } s = Chandra Chekuri (UIUC) CS/ECE 374 29 Fall 28 29 / 33

Product construction for intersection M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Q = Q Q 2 = {(q, q 2 ) q Q, q 2 Q 2 } s = (s, s 2 ) Chandra Chekuri (UIUC) CS/ECE 374 29 Fall 28 29 / 33

Product construction for intersection M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Q = Q Q 2 = {(q, q 2 ) q Q, q 2 Q 2 } s = (s, s 2 ) δ : Q Σ Q where δ((q, q 2 ), a) = Chandra Chekuri (UIUC) CS/ECE 374 29 Fall 28 29 / 33

Product construction for intersection M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Q = Q Q 2 = {(q, q 2 ) q Q, q 2 Q 2 } s = (s, s 2 ) δ : Q Σ Q where δ((q, q 2 ), a) = (δ (q, a), δ 2 (q 2, a)) Chandra Chekuri (UIUC) CS/ECE 374 29 Fall 28 29 / 33

Product construction for intersection M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Q = Q Q 2 = {(q, q 2 ) q Q, q 2 Q 2 } s = (s, s 2 ) δ : Q Σ Q where δ((q, q 2 ), a) = (δ (q, a), δ 2 (q 2, a)) A = Chandra Chekuri (UIUC) CS/ECE 374 29 Fall 28 29 / 33

Product construction for intersection M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Q = Q Q 2 = {(q, q 2 ) q Q, q 2 Q 2 } s = (s, s 2 ) δ : Q Σ Q where δ((q, q 2 ), a) = (δ (q, a), δ 2 (q 2, a)) A = A A 2 = {(q, q 2 ) q A, q 2 A 2 } Chandra Chekuri (UIUC) CS/ECE 374 29 Fall 28 29 / 33

Product construction for intersection M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Q = Q Q 2 = {(q, q 2 ) q Q, q 2 Q 2 } s = (s, s 2 ) δ : Q Σ Q where δ((q, q 2 ), a) = (δ (q, a), δ 2 (q 2, a)) A = A A 2 = {(q, q 2 ) q A, q 2 A 2 } Theorem L(M) = L(M ) L(M 2 ). Chandra Chekuri (UIUC) CS/ECE 374 29 Fall 28 29 / 33

Correctness of construction Lemma For each string w, δ (s, w) = (δ (s, w), δ 2 (s 2, w)). Chandra Chekuri (UIUC) CS/ECE 374 3 Fall 28 3 / 33

Correctness of construction Lemma For each string w, δ (s, w) = (δ (s, w), δ 2 (s 2, w)). Exercise: Assuming lemma prove the theorem in previous slide. Chandra Chekuri (UIUC) CS/ECE 374 3 Fall 28 3 / 33

Correctness of construction Lemma For each string w, δ (s, w) = (δ (s, w), δ 2 (s 2, w)). Exercise: Assuming lemma prove the theorem in previous slide. Proof of lemma by induction on w Chandra Chekuri (UIUC) CS/ECE 374 3 Fall 28 3 / 33

Product construction for union M = (Q, Σ, δ, s, A ) and M 2 = (Q, Σ, δ 2, s 2, A 2 ) Create M = (Q, Σ, δ, s, A) where Q = Q Q 2 = {(q, q 2 ) q Q, q 2 Q 2 } s = (s, s 2 ) δ : Q Σ Q where δ((q, q 2 ), a) = (δ (q, a), δ 2 (q 2, a)) A = Chandra Chekuri (UIUC) CS/ECE 374 3 Fall 28 3 / 33

Set Difference Theorem M, M 2 DFAs. There is a DFA M such that L(M) = L(M ) \ L(M 2 ). Exercise: Prove the above using two methods. Using a direct product construction Using closure under complement and intersection and union Chandra Chekuri (UIUC) CS/ECE 374 32 Fall 28 32 / 33

Things to know: 2-way DFA Question: Why are DFAs required to only move right? Can we allow DFA to scan back and forth? Caveat: Tape is read-only so only memory is in machine s state. Chandra Chekuri (UIUC) CS/ECE 374 33 Fall 28 33 / 33

Things to know: 2-way DFA Question: Why are DFAs required to only move right? Can we allow DFA to scan back and forth? Caveat: Tape is read-only so only memory is in machine s state. Can define a formal notion of a 2-way DFA Can show that any language recognized by a 2-way DFA can be recognized by a regular (-way) DFA Proof is tricky simulation via NFAs Chandra Chekuri (UIUC) CS/ECE 374 33 Fall 28 33 / 33