Automata Theory, Computability and Complexity Mridul Aanjaneya Stanford University June 26, 22 Mridul Aanjaneya Automata Theory / 64
Course Staff Instructor: Mridul Aanjaneya Office Hours: 2:PM - 4:PM, Gates 26 (Mon). Course Assistant: Sean Kandel Office Hours: 6:PM - 8:PM, Gates 372 (Tue/Thu). Textbook: Michael T. Sipser, Introduction to the Theory of Computation (second edition). Website: http://www.stanford.edu/class/cs54/ Discussion: Lore (http://www.lore.com/) Access code: R4ML4Z Mridul Aanjaneya Automata Theory 2/ 64
Course Reuirements Homework Midterm Final (optional) Note: Final grade will be computed from Homework (6%) and best of Midterm/Final (4%). Projected grade will be out one week before the Final, taking Homework (6%) and Midterm (4%). Can skip Final if satisfied with grade. Can collaborate on homeworks, but separate write-up per person and must mention collaborators. 4 homeworks, 2 free late days, use them however you want. No late days for final homework. Mridul Aanjaneya Automata Theory 3/ 64
Why Study Automata? Regular expressions: are used in many systems, e.g., UNIX. DTD s describe XML tags with RE like format. Finite automata model protocols, electronic circuits. Theory is used in model-checking. Context free languages: are used as syntax descriptors for PL s, parsers (YACC). have an important role in describing natural languages. find use in procedural modeling. Mridul Aanjaneya Automata Theory 4/ 64
Course Outline Deterministic/Nondeterministic finite automata. Regular expressions. Decision/closure properties of regular languages. Context-free languages. Parse trees. Normal forms. Pushdown automata. Euivalence of CFG s and PDA s. Pumping lemma for CFL s. Properties of CFL s. Enumerations, Turing machines. Undecidable problems. NP-completeness. Satisfiability. Cook s theorem. Mridul Aanjaneya Automata Theory 5/ 64
Automata, Computability and Complexity Main Question: What are the fundamental capabilities and limitations of computers? Dates back to the 93s. Different interpretations in all three areas. Mridul Aanjaneya Automata Theory 6/ 64
Complexity Question Why are some problems hard and others easy? Subset sum: Given a set of integers, does any non-empty subset of them add up to zero? So far only a classification according to hardness. Can help alter the root of difficulty, settle for an approximate solution, some problems are hard only in the worst case. Mridul Aanjaneya Automata Theory 7/ 64
Computability Question Which problems are solvable? Halting problem: Given an arbitrary computer program, decide if it finishes or continues running forever. Computability and complexity are closely related. Mridul Aanjaneya Automata Theory 8/ 64
Automata Deals with different models of computation. Mridul Aanjaneya Automata Theory 9/ 64
Preliminaries: Mathematical Induction Mridul Aanjaneya Automata Theory / 64
Preliminaries: Mathematical Induction Example Prove that 2 + 2 2 + 3 2 +... + n 2 is n(n+)(2n+) 6. Basis (n=): (+)(2 +) 6 = 2 3 6 = = 2. Induction: Let P(n) := 2 +... + n 2 is n(n+)(2n+) 6. P(n + ) = { 2 +... + n 2 } + (n + ) 2 = { n(n+)(2n+) 6 } + (n + ) 2 = (n + ) n(2n+)+6n+6 6 = (n + ) 2n 2 +7n+6 6 { } = (n + ) 2n 2 +4n+3n+6 6 = (n+)(n+2)(2n+3) 6 Since P() is true and P(n) P(n + ), we conclude that P(n) is true for all n. Mridul Aanjaneya Automata Theory / 64
Preliminaries: Graphs Undirected graph Mridul Aanjaneya Automata Theory 2/ 64
Preliminaries: Graphs Degree of a vertex Mridul Aanjaneya Automata Theory 3/ 64
Preliminaries: Graphs Subgraph Mridul Aanjaneya Automata Theory 4/ 64
Preliminaries: Graphs Path Mridul Aanjaneya Automata Theory 5/ 64
Preliminaries: Graphs Cycle Mridul Aanjaneya Automata Theory 6/ 64
Preliminaries: Graphs Directed graph Mridul Aanjaneya Automata Theory 7/ 64
Preliminaries: Pigeonhole Principle Pigeonhole Principle If there are n + pigeons and n pigeonholes, then some pigeonhole must contain at least two pigeons. Mridul Aanjaneya Automata Theory 8/ 64
Preliminaries: Pigeonhole Principle Generalized Pigeonhole Principle If there are kn + pigeons and n pigeonholes, then some pigeonhole must contain at least k + pigeons. Mridul Aanjaneya Automata Theory 9/ 64
Preliminaries: Pigeonhole Principle Example Given twelve integers, show that two of them can be chosen whose difference is divisible by. Example 2 Twenty-five crates of apples are delivered to a store. The apples are of three different sorts, and all the apples in each crate are of the same sort. Show that among these crates there are at least nine containing the same sort of apple. Example 3 Show that in any group of five people, there are two who have an identical number of friends within the group. Mridul Aanjaneya Automata Theory 2/ 64
Alphabet An alphabet is any finite set of symbols. Some examples include: ASCII, {,} (binary), {a,b,c}. The set of strings over an alphabet Σ is the set of lists, each element of which is a member of Σ. Note: Strings are shown with no commas, e.g., abc,. Σ denotes this set of strings. ε stands for the empty string (string of length ). Example: For the alphabet Σ = {, }, the set of strings in Σ is: {ε,,,,,,,,,,,,,,,...} For the alphabet Σ = {a, b, c}, the set of strings in Σ is: {ε,a,b,c,aa,ab,ac,ba,bb,bc,ca,cb,cc,...} Mridul Aanjaneya Automata Theory 2/ 64
Alphabet An alphabet is any finite set of symbols. Some examples include: ASCII, {,} (binary), {a,b,c}. The set of strings over an alphabet Σ is the set of lists, each element of which is a member of Σ. Note: Strings are shown with no commas, e.g., abc,. Σ denotes this set of strings. ε stands for the empty string (string of length ). Example: For the alphabet Σ = {, }, the set of strings in Σ is: {ε,,,,,,,,,,,,,,,...} Subtlety: as a string, and as an alphabet look the same. Context determines the type. Mridul Aanjaneya Automata Theory 22/ 64
Languages A language is a subset of Σ for some alphabet Σ. Example : The set of strings over {,} with no two consecutive s. {ε,,,,,,,,,,,,,,,,,,,...} Example 2: The set of strings over {a,b,c} with uneual number of a s and b s. {a,b,aa,ac,bb,bc,ca,cb,aaa,bbb,aab,baa,aba,aac,caa, aca,bbc,cbb,bcb,bba,abb,bab,cca,ccb,cac,cbc,bcc,acc,...} Mridul Aanjaneya Automata Theory 23/ 64
Finite Automata Informally, finite automata are finite collections of states with transition rules for going from one state to another. There is a start state and (one or more) accept states. Representation: Simplest representation is often a graph. Nodes denote states, and arcs indicate state transitions. Labels on arcs denote the cause of transition. 2 The above automaton only accepts binary strings ending in. Mridul Aanjaneya Automata Theory 24/ 64
An example 2 Mridul Aanjaneya Automata Theory 25/ 64
An example 2 Mridul Aanjaneya Automata Theory 26/ 64
An example 2 Mridul Aanjaneya Automata Theory 27/ 64
An example 2 Mridul Aanjaneya Automata Theory 28/ 64
An example 2 Mridul Aanjaneya Automata Theory 29/ 64
An example 2 Mridul Aanjaneya Automata Theory 3/ 64
An example 2 Mridul Aanjaneya Automata Theory 3/ 64
An example 2 Mridul Aanjaneya Automata Theory 32/ 64
An example 2 Mridul Aanjaneya Automata Theory 33/ 64
An example 2 Mridul Aanjaneya Automata Theory 34/ 64
An example 2 Mridul Aanjaneya Automata Theory 35/ 64
An example 2 Mridul Aanjaneya Automata Theory 36/ 64
An example 2 Mridul Aanjaneya Automata Theory 37/ 64
An example 2 Mridul Aanjaneya Automata Theory 38/ 64
An example 2 Mridul Aanjaneya Automata Theory 39/ 64
An example Accept! 2 Mridul Aanjaneya Automata Theory 4/ 64
An example Exercise: Verify that is rejected. 2 Mridul Aanjaneya Automata Theory 4/ 64
Deterministic Finite Automata Definition A DFA is a 5-tuple (Q, Σ, δ,, F ) consisting of: A finite set of states Q, A set of input alphabets Σ, A transition function δ : Q Σ Q, A start state, and A set of accept states F Q. The transition function δ: Takes two arguments, a state and an alphabet a. δ(,a) = the state the DFA goes to when it is in state and the alphabet a is received. Mridul Aanjaneya Automata Theory 42/ 64
Graph representation of DFA s Nodes correspond to states. Arcs represent transition function. Arc from state p to state labeled by all those input symbols that have transitions from p to. Incoming arrow from outside denotes start state. Accept states indicated by double circles. 2, 3 Accepts all binary strings without two consecutive s. Mridul Aanjaneya Automata Theory 43/ 64
Example 2, 3 Mridul Aanjaneya Automata Theory 44/ 64
Example 2, 3 Mridul Aanjaneya Automata Theory 45/ 64
Example 2, 3 Mridul Aanjaneya Automata Theory 46/ 64
Example 2, 3 Mridul Aanjaneya Automata Theory 47/ 64
Example 2, 3 Mridul Aanjaneya Automata Theory 48/ 64
Transition table for DFA s 2 2 3 3 3 3 A row for each state, a column for each alphabet. Accept states are starred. Arrow for the start state. 2, 3 Mridul Aanjaneya Automata Theory 49/ 64
Extended transition function Effect of a string on a DFA can be described by extending δ to a state and a string. Induction on length of the input string. Basis: δ(,ε)= Induction: δ(,wa)=δ(δ(,w),a) w is a string, a is an input alphabet. Convention: w,x,y,... are strings, a,b,c,... are alphabets. For a DFA, extended δ is computed for state and inputs a a 2...a n by following a path starting at and selecting arcs with labels a,a 2,...,a n in turn. Mridul Aanjaneya Automata Theory 5/ 64
Example using transition table 2 2 3 3 3 3 δ( 2, ) = δ(δ( 2, ), ) = δ(δ(δ( 2, ), ), ) = δ(δ(, ), ) = δ( 2, ) = 3 2, 3 Mridul Aanjaneya Automata Theory 5/ 64
Language of a DFA Automata of all kinds define languages. If A is an automaton, L(A) is its language. For a DFA A, L(A) is the set of strings labeling paths from the start state to an accept state. Formally, L(A) = set of strings w such that δ(,w) is in F. Example: is in L(A), where A is: 2 Mridul Aanjaneya Automata Theory 52/ 64
Proofs of set euivalence Question How to prove that two different sets are in fact the same? Here, T is the set of strings of s and s with no consecutive s and L = {w M accepts w}, where M is: 2, 3 Mridul Aanjaneya Automata Theory 53/ 64
Proofs of set euivalence Question How to prove that two different sets are in fact the same? In general, to show T = L, we need to prove two parts: If w is in L, then w is in T, i.e., L T. If w is in T, then w is in L, i.e., T L. 2, 3 Mridul Aanjaneya Automata Theory 54/ 64
Part : L T To prove: If w L, then w has no consecutive s. Proof is induction on length of w. Important trick: Expand the inductive hypothesis to be more detailed than you need. Mridul Aanjaneya Automata Theory 55/ 64
Part : L T (Inductive Hypothesis) If δ(,w)=, then w has no consecutive s and does not end in. 2 If δ(,w)= 2, then w has no consecutive s and ends in a single. Basis: w =, i.e., w = ε. holds as ε has no s at all. 2 holds vacuously, since δ(,ε) is not 2. Note: p is always true, if p is false. 2, 3 Mridul Aanjaneya Automata Theory 56/ 64
Part : L T If δ(,w)=, then w has no consecutive s and does not end in. 2 If δ(,w)= 2, then w has no consecutive s and ends in a single. Inductive step: Assume IH is true for strings shorter than w, where w. Because w is not empty, we can write w = xa, where a is the last alphabet of w and x is the string that precedes. IH is true for x. 2, 3 Mridul Aanjaneya Automata Theory 57/ 64
Part : L T If δ(,w)=, then w has no consecutive s and does not end in. 2 If δ(,w)= 2, then w has no consecutive s and ends in a single. Inductive step: Assume IH is true for strings shorter than w, where w. for w is δ(,w) =. Since δ(,w) =, δ(,x) must be or 2 and a must be. 2, 3 Mridul Aanjaneya Automata Theory 58/ 64
Part : L T If δ(,w)=, then w has no consecutive s and does not end in. 2 If δ(,w)= 2, then w has no consecutive s and ends in a single. Inductive step: Assume IH is true for strings shorter than w, where w. 2 for w is δ(,w) = 2. Since δ(,w) = 2, δ(,x) must be and a must be. 2, 3 Mridul Aanjaneya Automata Theory 59/ 64
Part 2: T L To prove: If w has no consecutive s, then w L. We prove the contrapositive. Note: p is euivalent to p. To prove (contrapositive): If w / L, then w has no s. Mridul Aanjaneya Automata Theory 6/ 64
Part 2: T L Simple induction on length of w. The only way w is not accepted is if it gets to 3. The only way to get to 3 is if w = xy. If δ(,x) = 2, then surely x = z (from Part : L T). w = zy! 2, 3 Mridul Aanjaneya Automata Theory 6/ 64
Regular languages Definition A DFA M = (Q, Σ, δ,, F ) accepts w if there exists a seuence of states r, r,..., r n in Q with three conditions: r = δ(r i, w i+ ) = r i+ for i =,..., n, and r n F Condition says that M starts in the start state. Condition 2 says that M follows δ between two states. Condition 3 says that last state is an accept state. We say that M recognizes L if L = {w M accepts w}. Mridul Aanjaneya Automata Theory 62/ 64
Regular languages A language L is regular if it is the language of some DFA. Note: the DFA must accept only the strings in L. Some languages are not regular (more later). Intuitively, DFA s are memoryless. Mridul Aanjaneya Automata Theory 63/ 64
A regular language Example Let L = {w w {,} and w, viewed as a binary integer, is divisible by 5.} 3 4 2 Mridul Aanjaneya Automata Theory 64/ 64