Non-deterministic Finite Automata (NFAs)

Algorithms & Models of Computation CS/ECE 374, Fall 27 Non-deterministic Finite Automata (NFAs) Part I NFA Introduction Lecture 4 Thursday, September 7, 27 Sariel Har-Peled (UIUC) CS374 Fall 27 / 39 Sariel Har-Peled (UIUC) CS374 2 Fall 27 2 / 39 Non-deterministic Finite State Automata (NFAs) NFA behavior,,,, q q q q q q Differences from DFA From state q on same letter a Σ multiple possible states No transitions from q on some letters -transitions! Questions: Is this a real machine? What does it do? Machine on input string w from state q can lead to set of states (could be empty) From q on From q on From q on From q on From q on Sariel Har-Peled (UIUC) CS374 3 Fall 27 3 / 39 Sariel Har-Peled (UIUC) CS374 4 Fall 27 4 / 39

NFA acceptance: informal NFA acceptance: example,,,, q q q q q q Informal definition: An NFA N accepts a string w iff some accepting state is reached by N from the start state on input w. The language accepted (or recognized) by a NFA N is denote by L(N) and defined as: L(N) = {w N accepts w}. Is accepted? Is accepted? Is accepted? Are all strings in accepted? What is the language accepted by N? Comment: Unlike DFAs, it is easier in NFAs to show that a string is accepted than to show that a string is not accepted. Sariel Har-Peled (UIUC) CS374 5 Fall 27 5 / 39 Sariel Har-Peled (UIUC) CS374 6 Fall 27 6 / 39 Simulating NFA the first (N) a b a b A B C D E Run it on input ababa. Idea: Keep track of the states where the NFA might be at any given time. t = : a b a b A B C D E Remaining input: ababa. t = : a b a b A B C D E Sariel Har-Peled (UIUC) CS374 7 Fall 27 7 / 39 Formal Tuple Notation A non-deterministic finite automata (NFA) N = (Q, Σ, δ, s, A) is a five tuple where Q is a finite set whose elements are called states, Σ is a finite set called the input alphabet, δ : Q Σ {} P(Q) is the transition function (here P(Q) is the power set of Q), s Q is the start state, A Q is the set of accepting/final states. δ(q, a) for a Σ {} is a subset of Q a set of states. Sariel Har-Peled (UIUC) CS374 8 Fall 27 8 / 39

Reminder: Power set For a set Q its power set is: P(Q) = 2 Q = {X X Q} is the set of all subsets of Q. Q = {, 2, 3, 4} P(Q) = {, 2, 3, 4}, {2, 3, 4}, {, 3, 4}, {, 2, 4}, {, 2, 3}, {, 2}, {, 3}, {, 4}, {2, 3}, {2, 4}, {3, 4}, {}, {2}, {3}, {4}, {}, q Q = {q, q, q, } Σ = {, } δ s = q A = { } q q, Sariel Har-Peled (UIUC) CS374 9 Fall 27 9 / 39 Sariel Har-Peled (UIUC) CS374 Fall 27 / 39 Transition function in detail..., q δ(q, ) = {q } δ(q, ) = {q, q } δ(q, ) = {q } δ(q, ) = {q } δ(q, ) = {} δ(q, ) = { } q q, δ(q, ) = {q, q } δ(q, ) = {q } δ(q, ) = {} δ(, ) = { } δ(, ) = { } δ(, ) = { } Extending the transition function to strings NFA N = (Q, Σ, δ, s, A) 2 δ(q, a): set of states that N can go to from q on reading a Σ {}. 3 Want transition function δ : Q Σ P(Q) 4 δ (q, w): set of states reachable on input w starting in state q. Sariel Har-Peled (UIUC) CS374 Fall 27 / 39 Sariel Har-Peled (UIUC) CS374 2 Fall 27 2 / 39

. "-Transitions the NFA somehow chose a path to an accept state still exist. One slight disadvantage of this metaphor is that if an NFA reads a string that is not in its language, it destroys all universes. t is fairly common for NFAs to include so-called "-transitions, which allow the machine to hange state Extending without reading the transition input symbol. function An NFA with to "-transitions strings accepts a string w a a 2 a 3 Extending the transition function to strings a` Proofs/oracles. Finally, we can treat NFAs not as a mechanism for computing something, but as and only if there is a sequence of transitions s! q! q 2!! q` where the final a mechanism for verifying proofs. If we want to prove that a string w contains one of the suffixes tate q` is accepting, each a i is either " or a symbol in, and a a 2 a` = w. or, it suffices to demonstrate a single walk in our example NFA that starts at s and ends For example, For NFA consider N = the (Q, following Σ, δ, s, NFA A) and with q"-transitions. Q the ɛreach(q) (For this example, is the setwe indicate at c, and whose For NFA edgesn are= labeled (Q, Σ, with δ, the s, A) symbols and qin w. QEquivalently, the ɛreach(q) whenever is the the setnfa faces a he "-transitions of all states using large that red q can arrows; reach we using won t only normally -transitions. do that.) This NFA deliberately has nontrivial choice, of all states the prover that can q can simply reach tellusing the NFA onlywhich -transitions. state to move to next. ore "-transitions than necessary. This intuition can be formalized as follows. Consider a deterministic finite state machine whose input alphabet is the product of an input alphabet and an oracle alphabet a b c Equivalently, we can imagine that this Inductive definition of δ DFA reads : Q Σ simultaneously from two strings of the same P(Q): length: the input string w and the oracle string!. In either formulation, the transition function, s g, has the form : ifq w ( =, ) δ!(q, Q. w) As usual, = ɛreach(q) this DFA accepts the pair (w,!) 2 ( ) if and only if (s, (w, if!)) w 2= A. afinally, where Ma nondeterministically Σ accepts the string w 2 if there is d e f an oracle string δ! (q, 2 a) with =! p ɛreach(q) = w such ( that r δ(p,a) (w, ɛreach(r))!) 2 L(M). An NFA with "-transitions if w = ax,. "-Transitions δ (q, w) = p ɛreach(q) ( r δ(p,a) δ (r, x)) The NFA starts as usual in state s. If the input string is, the the machine might It is fairly common for NFAs to include so-called "-transitions, which allow the machine to on-deterministically choose the following transitions and then accept. change state without reading an input symbol. An NFA with "-transitions accepts a string w a a 2 a 3 a` " " " " " " if and only if there is a sequence of transitions s! q s Sariel! s Har-Peled! d (UIUC)! a! b! c CS374! d! 3 e! f! e! f Fall! 27 c! 3 g! q 2!! q` where the final / 39 Sariel Har-Peled (UIUC) CS374 4 Fall 27 4 / 39 state q` is accepting, each a i is either " or a symbol in, and a a 2 a` = w. More formally, the transition function in an NFA with "-transitions has a slightly larger For example, consider the following NFA with "-transitions. (For this example, we indicate omain : Q ( [ {"})! 2 Q. The "-reach of a state q 2 Q consists of all states r that satisfy the "-transitions using large red arrows; we won t normally do that.) This NFA deliberately has ne of the following conditions: more "-transitions than necessary. Formal definition of language accepted by N either r = q, or r 2 (q, ") for some state q in the "-reach of q. A string w is accepted by NFA N if δ N (s, w) A. n other words, r is in the "-reach of q if there is a (possibly empty) sequence of "-transitions ading from q to r. For example, in the example NFA above, the "-reach of state f is {a, c, d, f, g}. The language L(N) accepted by a NFA N = (Q, Σ, δ, s, A) is {w Σ δ (s, w) A }. Important: Formal definition of the language of NFA above uses δ and not δ. As such, one does not need to include -transitions closure when specifying δ, since δ takes care of that. What is: b, s g, a d e An NFA with "-transitions The NFA starts δ (s, as ɛ) usual in state s. If the input string is, the the machine might non-deterministically δ (s, choose ) the following transitions and then accept. s δ (c, " ) "! s! d! a! b! c "! d! e! f "! e! f "! c "! g δ (b, ) More formally, the transition function in an NFA with "-transitions has a slightly larger domain : Q ( [ {"})! 2 Q. The "-reach of a state q 2 Q consists of all states r that satisfy one of the following conditions: c f Sariel Har-Peled (UIUC) CS374 5 Fall 27 5 / 39 either r = q, Sariel Har-Peled (UIUC) CS374 6 Fall 27 6 / 39 or r 2 (q, ") for some state q in the "-reach of q.

Another definition of computation w q N p: State p of NFA N is reachable from q on w there exists a sequence of states r, r,..., r k and a sequence x, x 2,..., x k where x i Σ {}, for each i, such that: r = q, for each i, r i + δ(r i, x i + ), r k = p, and w = x x 2 x 3 x k. Why non-determinism? Non-determinism adds power to the model; richer programming language and hence (much) easier to design programs Fundamental in theory to prove many theorems Very important in practice directly and indirectly Many deep connections to various fields in Computer Science and Mathematics Many interpretations of non-determinism. Hard to understand at the outset. Get used to it and then you will appreciate it slowly. δ N(q, w) = { p Q q } w N p. Sariel Har-Peled (UIUC) CS374 7 Fall 27 7 / 39 Sariel Har-Peled (UIUC) CS374 8 Fall 27 8 / 39 DFAs and NFAs Part II Constructing NFAs Every DFA is a NFA so NFAs are at least as powerful as DFAs. NFAs prove ability to guess and verify which simplifies design and reduces number of states Easy proofs of some closure properties Sariel Har-Peled (UIUC) CS374 9 Fall 27 9 / 39 Sariel Har-Peled (UIUC) CS374 2 Fall 27 2 / 39

Strings that represent decimal numbers.,,,2,...,9 2 3. 5 {strings that contain CS374 as a substring} {strings that contain CS374 or CS473 as a substring} {strings that contain CS374 and CS473 as substrings},2,...,9 4.,,2,...,9,,2,...,9 Sariel Har-Peled (UIUC) CS374 2 Fall 27 2 / 39 Sariel Har-Peled (UIUC) CS374 22 Fall 27 22 / 39 L k = {bitstrings that have a k positions from the end} A simple transformation For every NFA N there is another NFA N such that L(N) = L(N ) and such that N has the following two properties: N has single final state f that has no outgoing transitions The start state s of N is different from f Sariel Har-Peled (UIUC) CS374 23 Fall 27 23 / 39 Sariel Har-Peled (UIUC) CS374 24 Fall 27 24 / 39

Closure properties of NFAs Part III Closure Properties of NFAs Are the class of languages accepted by NFAs closed under the following operations? union intersection concatenation Kleene star complement Sariel Har-Peled (UIUC) CS374 25 Fall 27 25 / 39 Sariel Har-Peled (UIUC) CS374 26 Fall 27 26 / 39 Closure under union For any two NFAs N and N 2 there is a NFA N such that L(N) = L(N ) L(N 2 ). Closure under concatenation For any two NFAs N and N 2 there is a NFA N such that L(N) = L(N ) L(N 2 ). q N f q N f q 2 f 2 N 2 q 2 N f 2 2 Sariel Har-Peled (UIUC) CS374 27 Fall 27 27 / 39 Sariel Har-Peled (UIUC) CS374 28 Fall 27 28 / 39

Closure under Kleene star For any NFA N there is a NFA N such that L(N) = (L(N )). Closure under Kleene star For any NFA N there is a NFA N such that L(N) = (L(N )). q N f q N f Does not work! Why? Sariel Har-Peled (UIUC) CS374 29 Fall 27 29 / 39 Sariel Har-Peled (UIUC) CS374 3 Fall 27 3 / 39 Closure under Kleene star For any NFA N there is a NFA N such that L(N) = (L(N )). Part IV q q N f NFAs capture Regular Languages Sariel Har-Peled (UIUC) CS374 3 Fall 27 3 / 39 Sariel Har-Peled (UIUC) CS374 32 Fall 27 32 / 39

Regular Languages Recap Regular Languages Regular Expressions regular denotes {ɛ} regular ɛ denotes {ɛ} {a} regular for a Σ a denote {a} R R 2 regular if both are r + r 2 denotes R R 2 R R 2 regular if both are r r 2 denotes R R 2 R is regular if R is r denote R NFAs and Regular Language For every regular language L there is an NFA N such that L = L(N). Proof strategy: For every regular expression r show that there is a NFA N such that L(r) = L(N) Induction on length of r Regular expressions denote regular languages they explicitly show the operations that were used to form the language Sariel Har-Peled (UIUC) CS374 33 Fall 27 33 / 39 Sariel Har-Peled (UIUC) CS374 34 Fall 27 34 / 39 NFAs and Regular Language For every regular expression r show that there is a NFA N such that L(r) = L(N) Induction on length of r Base cases:, {}, {a} for a Σ. NFAs and Regular Language For every regular expression r show that there is a NFA N such that L(r) = L(N) Induction on length of r Inductive cases: r, r 2 regular expressions and r = r + r 2. By induction there are NFAs N, N 2 s.t L(N ) = L(r ) and L(N 2 ) = L(r 2 ). We have already seen that there is NFA N s.t L(N) = L(N ) L(N 2 ), hence L(N) = L(r) r = r r 2. Use closure of NFA languages under concatenation r = (r ). Use closure of NFA languages under Kleene star Sariel Har-Peled (UIUC) CS374 35 Fall 27 35 / 39 Sariel Har-Peled (UIUC) CS374 36 Fall 27 36 / 39

(+)(+) * (+) (+) * (+) * (+) * * * Sariel Har-Peled (UIUC) CS374 37 Fall 27 37 / 39 Sariel Har-Peled (UIUC) CS374 38 Fall 27 38 / 39 * 3 2 4 Final NFA simplified slightly to reduce states Sariel Har-Peled (UIUC) CS374 39 Fall 27 39 / 39