Lecture 5: Minimizing DFAs

6.45 Lecture 5: Minimizing DFAs

6.45 Announcements: - Pset 2 is up (as of last night) - Dylan says: It s fire. - How was Pset? 2

DFAs NFAs DEFINITION Regular Languages Regular Expressions 3

4

Some Languages Are Not Regular: Limitations on DFAs/NFAs a.k.a. Lower Bounds on DFAs/NFAs 5

Minimizing DFAs 6

Does this DFA have a minimal number of states? NO 7

8

Is this minimal? How can we tell in general? 9

DFA Minimization Theorem: For every regular language A, there is a unique (up to re-labeling of the states) minimal-state DFA M* such that A = L(M*). Furthermore, there is an efficient algorithm which, given any DFA M, will output this unique M*. If such algorithms existed for more general models of computation, that would be an engineering breakthrough!!

In general, there isn t a uniquely minimal NFA

Distinguishing states with strings For a DFA M = (Q, Σ,, q, F), and q Q, let M q be the DFA equal to (Q, Σ,, q, F) Def. w Σ* distinguishes states p and q if: M p accepts w M q rejects w p M w accept OR p M w reject q w reject q w accept 2

Distinguishing states with strings For a DFA M = (Q, Σ,, q, F), and q Q, let M q be the DFA equal to (Q, Σ,, q, F) Def. w Σ* distinguishes states p and q if: M p and M q have different outputs on input w p M w accept OR p M w reject q w reject q w accept 3

Distinguishing two states Def. w Σ* distinguishes states q and q 2 M p and M q have different outputs on w iff I m in q or q 2, but which? How can I tell? 4

Distinguishing two states Def. w Σ* distinguishes states p and q iff M p and M q have different outputs on w Here read this Ok, I m accepting! Must have been p W Ok, I m rejecting! Must have been q 5

Fix M = (Q, Σ,, q, F) and let p, q Q Let M p = (Q, Σ,, p, F) and M q = (Q, Σ,, q, F) Definition(s): State p is distinguishable from state q iff there is a w Σ* that distinguishes p and q iff there is a w Σ* so that M p accepts w M q rejects w State p is indistinguishable from state q iff p is not distinguishable from q iff for all w Σ*, M p accepts w M q accepts w Big Idea: Pairs of indistinguishable states are redundant! 6

Which pairs of states are distinguishable? q, q q 2 Are q and q distinguishable? q 3 The empty string ε distinguishes all final states from all non-final states 7

q, q q 2 Are q and q 3 distinguishable? q 3 The string distinguishes q and q 3 8

q, q q 2 Are q and q 2 distinguishable? q 3 The string distinguishes q and q 2 In this DFA, all pairs of states are distinguishable! 9

Fix M = (Q, Σ,, q, F) and let p, q, r Q Define a binary relation on the states of M: p q iff p is indistinguishable from q p q iff p is distinguishable from q Proposition: is an equivalence relation p p (reflexive) p q q p (symmetric) p q and q r p r (transitive) Proof? Just look at the definition! p q means for all w, M p accepts w M q accepts w 2

Fix M = (Q, Σ,, q, F) and let p, q, r Q Therefore, the relation partitions Q into disjoint equivalence classes Proposition: is an equivalence relation Q [q] := { p p q } q q 2

22

Algorithm: MINIMIZE-DFA Input: DFA M Output: DFA M MIN such that:. L(M) = L(M MIN ) not reachable from start 2. M MIN has no inaccessible states 3. M MIN is irreducible for all states p q of M MIN, p and q are distinguishable Theorem: Every M MIN satisfying,2,3 is the unique minimal DFA equivalent to M 23

States of M MIN Intuition: = Equivalence classes of states of M We ll uncover these equivalent states with a dynamic programming algorithm 24

The Table-Filling Algorithm Input: DFA M = (Q, Σ,, q, F) Output: () D M = { (p, q) p, q Q and p q } (2) EQUIV M = { [q] q Q } Idea: We know how to find those pairs of states that the string ε distinguishes Use this and iteration to find those pairs distinguishable with longer strings The pairs of states left over will be indistinguishable 25

The Table-Filling Algorithm Input: DFA M = (Q, Σ,, q, F) Output: () D M = { (p, q) p, q Q and p q } (2) EQUIV M = { [q] q Q } q q q i q n q q q i q n 26

The Table-Filling Algorithm Input: DFA M = (Q, Σ,, q, F) Output: () D M = { (p, q) p, q Q and p q } (2) EQUIV M = { [q] q Q } q q Base Case: For all (p, q) such that p accepts and q rejects mark p q q i q n q q q i q n 27

q q q i D D q n D q q q i q n The Table-Filling Algorithm Input: DFA M = (Q, Σ,, q, F) Output: () D M = { (p, q) p, q Q and p q } (2) EQUIV M = { [q] q Q } Base Case: For all (p, q) such that p accepts and q rejects mark p q Iterate: If there are states p, q and symbol σ Σ satisfying: (p, ) = p mark p q (q, ) = q Repeat until no more D s can be added 28

q Are q and q 2 distinguishable? q q 2 q 3 D D D D D D Are q and q distinguishable? Are q and q 2 distinguishable? q q q 2 q 3, q q q 2 q 3 29

Claim: If (p, q) is marked D by the Table-Filling algorithm, then p q Proof: Induction on the number of iterations n in the algorithm when (p, q) is marked D n = : If (p, q) is marked D in the base case, then exactly one of them is final, so ε distinguishes p and q I.H. For all (p, q ) marked D in the first n iterations, p q Suppose (p, q) is marked D in (n + )th iteration. To be marked, there must be states p, q such that:. p = (p, ) and q = (q, ), for some Σ 2. (p, q ) is marked D p q (by induction) So there s a w s.t. M p and M q have different output on w Then, the string w distinguishes p and q! 3

Claim: If (p, q) is not marked D by the Table-Filling algorithm, then p q Proof (by contradiction): Suppose the pair (p, q) is not marked D by the algorithm, yet p q (call this a bad pair ) Then there is a string w such that w > and: M p and M q have different outputs on w (Why is w >?) Of all such bad pairs, let (p, q) be a pair with a minimum-length distinguishing string w 3

Claim: If (p, q) is not marked D by the Table-Filling algorithm, then p q Proof (by contradiction): Suppose the pair (p, q) is not marked D by the algorithm, yet p q (call this a bad pair ) Of all such bad pairs, let (p, q) be a pair with a minimum-length distinguishing string w M p and M q have different outputs on w We have w = w, for some string w and some Σ Let p = (p, ) and q = (q, ). Then (p, q ) is also a bad pair! (p,q ) distinguished by w But then (p, q ) has a SHORTER distinguishing string, w Contradiction! 32

Algorithm MINIMIZE Input: DFA M Output: Equivalent minimal-state DFA M MIN. Remove all inaccessible states from M 2. Run Table-Filling algorithm on M to get: EQUIV M := { [q] q is an accessible state of M } 3. Define: M MIN = (Q MIN, Σ, MIN, q MIN, F MIN ) Q MIN = EQUIV M, q MIN = [q ], F MIN = { [q] q F } MIN ( [q], ) = [ ( q, ) ] Claim: L(M MIN ) = L(M) (well-defined??) 33

The MINIMIZE Algorithm in Pictures. Remove all inaccessible states M q q 34

The MINIMIZE Algorithm in Pictures 2. Run Table-Filling to get equiv classes M [q] := { p p q } q q 35

The MINIMIZE Algorithm in Pictures 3. Define M MIN with states = equiv classes M q q States of M MIN = Equivalence classes of states of M 36

q q q q D q 3 q 2 q 2 D q 3 D D q q q 2 q 3 37

q q q 4, q, q 5 q D q 2 q 3 q 3 D D q 4 D D q 5 D D D D q q q 3 q 4 q 5 38

q q q 4, q q 5 q D q 3 q 3 D D q 4 D D q 5 D D D D q q q 3 q 4 q 5 39

Thm: M MIN is the unique minimal DFA equivalent to M Claim: Let M be any DFA where L(M )=L(M MIN ) and M has no inaccessible states and M is irreducible. Then there is an isomorphism between M and M MIN Suppose we have proved the Claim is true. Assuming the Claim we can prove the Thm: Proof of Thm: Let M be any minimal DFA for M. Since M is minimal, M has no inaccessible states and is irreducible (otherwise, we could make M smaller!) By the Claim, there is an isomorphism between M and the DFA M MIN that is output by MINIMIZE(M). That is, M MIN is isomorphic to every minimal M. 4

Thm: M MIN is the unique minimal DFA equivalent to M Claim: Let M be any DFA where L(M )=L(M MIN ) and M has no inaccessible states and M is irreducible. Then there is an isomorphism between M and M MIN Proof: We recursively construct a map from the states of M MIN to the states of M Base Case: q MIN q Recursive Step: If p p q q Then q q 4

Base Case: q MIN q Recursive Step: If p p Then q q q q 42

Base Case: q MIN q Recursive Step: If p p q Then q q Claim: Map is an isomorphism. Need to prove: The map is defined everywhere The map is well defined The map is a bijection (one-to-one and onto) The map preserves all transitions: If p p then MIN (p, ) (p, ) (this follows from the definition of the map!) q 43

Base Case: q MIN q Recursive Step: If p p The map is defined everywhere That is, for all states q of M MIN there is a state q of M such that q q If q M MIN, there is a string w such that M MIN is in state q after reading in w q Then q q q Let q be the state of M after reading in w. Claim: q q (proof by induction on w ) 44

Base Case: q MIN q Recursive Step: If p p The map is onto: q q such that q q q Then q q Want to show: For all states q of M there is a state q of M MIN such that q q For every q in M there is a string w such that M reaches state q after reading in w Let q be the state of M MIN after reading in w. Claim: q q (proof by induction on w ) q 45

Base Case: q MIN q Recursive Step: If p p The map is well defined: q! q such that q q Suppose there are states q and q such that q q and q q We show that q and q are indistinguishable, so it must be that q = q (why?) q Then q q q 46

Suppose there are states q and q such that q q and q q Assume for contradiction q and q are distinguishable M MIN M q MIN u q w Accept q u q w Accept q MIN v Contradiction! w q Reject q v q w Reject 47

Map is -to-: p q, p q and q q q q Proof by contradiction. Suppose there are states p ¹ q such that p q and q q If p q, then p and q are distinguishable M MIN M q MIN u p w Accept q u q w Accept q MIN v q w Reject q v Contradiction! w q Reject 48

How can we prove that two regular expressions are equivalent? 49