Topics COSC Administrivia. Topics Today. Administrivia (II) Acknowledgements. Slides presented May 9th and 16th,

Transcription

1 Topics COSC 2001 Introduction to the Theory of Computation Dr. David Forster Basic concepts of theoretical CS, with practical applications: Regular Languages Context Free Languages Recursively Enumerable Languages Finite Automata & Regular Expressions Context Free Grammars & Pushdown Automata Turing Machines, Computability, Halting Problem NP completeness (time permitting!) Text editor Parser, Compiler General computer model COSC Topics Today Administrivia Motivation & Overview Review: Set theory, relations, proofs Alphabets, Strings, and Languages Administrivia Lecture hours: R 19:00 to 22:00, CSB 121 Office hours: TR 16:00 to 17:00, CSB david@cs.yorku.ca subject: 2001: web page: Text: Kinber & Smith, Theory of Computing: A Gentle Introduction HMU = Hopcroft, Motwani & Ullman, Introduction to Automata Theory, Languages, and Computation COSC COSC Administrivia (II) Evaluation: 5 assignments 10% midterm 30% final 60% Acknowledgements These slides are courtesy Dr. Nancy Ide, Vassar, with some modifications and additions by David Forster. COSC COSC Slides presented May 9th and 16th,

2 Motivation Overview (I) Why do we care about automata? modelling circuits, protocols pattern matching, search lexical analysers (RE) (RL) Automata states (including start & final) input arcs Push compilers (CFL) Start off on Push COSC COSC Overview (II) "Structural Representation" grammars Overview (III) Complexity decidable: what's possible? tractable: what's practical? (Sentence Subject Verb Object) regular expressions [a-za-z_][a-za-z_0-9]* COSC COSC Concept Review sets, membership (x R), subset (R S), union (R S), intersection (R S), cartesian product (R S), complement (R) pigeonhole principle relations: reflexive, symmetric, transitive equivalence relations, equivalence classes Formal Proof need it for this course need it to reason about design? programs? systems? 1. seat of your pants 2. verify everything 3. somewhere in between functions, bijections, inverses COSC COSC Slides presented May 9th and 16th,

3 Deductive Proofs sequence of statements: hypothesis deductions conclusion deduction based on: accepted logical principle given fact previous step combination Example of a Formal Proof Theorem: If x is the sum of the squares of four positive integers, then 2 x x 2 Step Statement x = a 2 + b 2 + c 2 + d 2 a 1; b 1; c 1; d 1 a 2 1; b 2 1; c 2 1; d 2 1 x 4 2 x x 2 Justification Given Given (2) & properties of arithmetic (1), (3), & properties of arithmetic (4) & Theorem 1.3* From HMU: theorem 1.4 & fig. 1.3 *: Previously proved in HMU: If x 4, then2 x x 2 COSC COSC Starting a Proof don't know how to start? reduce terms in hypothesis to their definitions e.g., some definitions: def n 1: a set S is finite if S = n, for some n def n 2: if sets S and T are subsets of a set U, then T is the complement of S w.r.t. U if S T = U and S T = φ now prove: Let S be a finite subset of some infinite set U. Let T be the complement of S with respect to U. Then T is infinite. COSC Example: Restating the givens Theorem: Let S be a finite subset of some infinite set U. Let T be the complement of S with respect to U. Then T is infinite. Original Statement S is finite U is infinite T is the complement of S From HMU: fig. 1.4: Restating the givens of Theorem 1.5 New Statement There is an integer n such that S = n For no integer p is U = p S T = U and S T = φ COSC A Proof 1) S and T are disjoint, so S + T = U 2) S is finite, so S = n for some integer n 3) U is infinite, so there is no p s.t. U = p 4) assume T is finite, so T = m 5) now, U = S + T = n + m 6) but this contradicts (3), so assumption (4) must be false, i.e., T must be infinite. Q.E.D. An "If-then" by any other name H C H implies C H only if C C if H Whenever H holds, C follows C hidden hypothesis? ex., "sin 2 θ + cos 2 θ = 1" is really: "if θ is an angle, sin 2 θ + cos 2 θ = 1" COSC COSC Slides presented May 9th and 16th,

4 A if and only if B: If-and-only-If A B A iffb A is equivalent to B A exactly when B Iff (II) two parts, need to prove both: 1. if A then B 2. if B then A if and only-if are converses of each other "if A then B" says nothing about whether B is true when A is false (and vice versa) COSC COSC Quantifiers for all: there exists: order is important: d represents a driver, c a car d c: driver(d) (car(c) & drives(d,c)) "all drivers drive some car" (not necessarily the same one) c d: driver(d) (car(c)& drives(d,c)) "all drivers drive the same car" Additional forms of proof proofs about sets contrapositive proof by contradiction counterexamples COSC COSC Set Equivalences often need to prove two sets are same, e.g., sets of strings, languages, etc. we write: E = F we mean: "x is in E iff x is in F" e.g.: R S = S R so prove: x is in R S iff x is in S R The Contrapositive "if H then C" "if not C then not H" H C if con. F F T T F T T T T F F F T T T T don't confuse with "converse" COSC COSC Slides presented May 9th and 16th,

5 Proof by Contradiction already demonstrated (slide "A Proof") assume opposite of something to prove show that the assumption must be false therefore, thing to prove must be true Counterexamples theorems are statements about infinite number of cases otherwise: merely an "observation" one false case disproves theorem "All computer scientists wear plaid." COSC COSC Inductive Proofs Prove a statement S(X) about a family of objects X (e.g., integers, trees) in two parts: 1. Basis: Prove for one or several small values of X directly. 2. Inductive step: Assume S(Y) for Y smaller than X; prove S(X) using that assumption. COSC Structural Induction (I) structures with recursive definitions: basis case(s): elementary structures inductive step(s): define complex structure in terms of previously defined structures e.g., complex noun phrases: Noun : "rocket", "manifold", "engine" Complex noun phrase : Noun or noun followed by a complex noun phrase thus: "rocket", "rocket engine", "rocket engine manifold", COSC Structural Induction (II) Use following form to prove theorems about recursively defined structures: Basis: Prove S(X) for the basis structure(s) X. Inductive Step: Take structure X formed from Y 1,,Y k, assume S(Y 1 ),,S(Y k ), use to prove S(X) Example (I) A binary tree with n leaves has 2n - 1 nodes Formally, S(T): if T is a binary tree with n leaves, then T has 2n-1 nodes Induction is on the size = number of nodes of T COSC COSC Slides presented May 9th and 16th,

6 Example (II) Basis: If T has 1 leaf, it is a one-node tree. 2(1) 1 = 1. Induction: Assume S(U) for trees with fewer nodes than T. In particular, assume for the subtrees of T. (so now prove S(T) with this assumption) Example (III) 1. T must be a root plus two subtrees U and V 2. If U and V have u and v leaves, respectively, and T has t leaves, then u + v = t 3. By the inductive hypothesis, U and V have 2u-1 and 2v-1 nodes, respectively 4. Then T has 1+(2u-1)+(2v-1) nodes = 2(u+v) 1 = 2t-1, proving the inductive step. COSC COSC Example: Balanced Parentheses Here are two ways to define "balanced parentheses": 1. Grammatically: a) The empty string ε is balanced b) If w is balanced, then (w) is balanced c) If w and x are balanced, then so is wx 2. By Scanning: w is balanced if and only if: a) w has an equal number of left and right parentheses b) Every prefix of w has at least as many left as right parentheses Call these GB and SB properties, respectively. Theorem: A string of parentheses w is GB if and only if it is SB. If An induction on w (length of w). Assume w is SB; prove it is GB. Basis: If w = ε (length = 0), then w is GB by rule (a). Notice that we do not even have to address the question of whether ε is SB (it is, however). Induction: Suppose the statement "SB implies GB" is true for strings shorter than w. COSC COSC Case 1: w is not ε, but has no non-empty prefix that has an equal number of ( and ). Then w must begin with ( and end with ); i.e., w = (x). x must be SB (why?) By the IH, x is GB GB/SB (II) By rule 1(b), (x) is GB; but (x) = w, so w is GB Case 2: w = xy, where x is the shortest non-empty prefix of w with an equal number of ( and ), and y is not ε. x and y are both SB (why?) By the IH, x and y are GB w is GB by rule 1(c) GB/SB (III) COSC COSC Slides presented May 9th and 16th,

7 Only-If GB/SB (IV) An induction on w. Assume w is GB; prove it is SB. Basis:w = ε. Clearly w obeys the conditions for being SB. Induction: Assume GB implies SB" for strings shorter than w, and assume w is not ε. Case 1: w is GB because of rule 1(b); i.e., w = (x) and x is GB. by the IH, x is SB. GB/SB (V) Since x has equal numbers of ('s and )'s, so does (x). Since x has no prefix with more ('s than )'s, so does (x). COSC COSC Case 2: GB/SB (VI) w is not ε and is GB because of rule (c); i.e., w = xy, and x and y are GB. By the IH, x and y are SB. (Aside) Trickier than it looks: we have to argue that neither x nor y could be ε, because if one were, the other would be w, and this rule application could not be the one that first shows w to be GB. xy has equal numbers of ('s and )'s because x and y both do. If w had a prefix with more )'s than ('s, that prefix would either be a prefix of x (contradicting the fact that x has no such prefix) or it would be x followed by a prefix of y (contradicting the fact that y also has no such prefix). Above is an example of proof by contradiction. We assumed our conclusion about w was false and showed it would imply something that we know is false. COSC Languages Alphabet (Σ): finite set of symbols, e.g., - Σ = {0, 1} (binary alphabet) - ASCII Word or String = finite sequence of symbols chosen from some alphabet, e.g., or abracadabra Language = set of strings chosen from some alphabet COSC Language Examples The set of all binary strings consisting of at least one 0, followed by an equal number of 1's, i.e., {01, 0011, , } C (the set of compilable C programs) English COSC Strings Empty string:ε Length: abcde= 5 uv = u + v Reverse: w R If w= abc, w R = cba Substring: any string of consecutive characters in some w Prefix and suffix: if w = vu, v and u are a prefix and a suffix of w, respectively Convention: Use lower case letters from the beginning of the alphabet for symbols, lower case letters from end of alphabet for strings COSC Slides presented May 9th and 16th,

8 Strings (cont'd) Concatenation : Given strings x and y, concatenation is xy e.g., x = abc, y = def, xy = abcdef Powers Σ k = set of all strings from alphabet Σ with length k Σ 0 = {ε} Σ * = set of all strings from alphabet Σ Σ + = Σ * -{ε} COSC More About Languages Can concatenate languages L 1 L 2 = {xy x L 1, y L 2 } L n = L concatenated with itself n times L 0 = {ε}; L 1 = L Star-closure L * = L 0 L 1 L 2 (note: L + = L * -L 0 ) Languages can be finite or infinite L = {a, aba, bba} L = {a n n > 0} Subtle point: a language may be infinite, but there is some finite set of symbols of which all its strings are composed COSC Finite Automata describe regular languages: simple, highly useful L s graphwith finite number of nodes (states) arcs labelled with one or more symbols from Σ one state is start / initial state some states are final / favoured / accepting states The language of the FA is the set of strings that label paths that go from the start state to some accepting state Example Below FA scans HTML documents, looking for a list of what could be title-author pairs Accepts whenever it finds the end of a list item Could be basis for application COSC COSC Scan HTML Documents HTML Document Example start <ol>, <ul> <li> <li> Any non-tag space b y space <ul> <li> start <ol>, <ul> <li> IALC by HM&U </li> <li> <li> LotR by T </li> </ul> </ol>, </ul> </li> Any non-tag space b y space Any non-tag Any non-tag </ol>, </ul> </li> COSC COSC Slides presented May 9th and 16th,

9 Recognize Simple Identifiers Newspaper Vending Machine quarter start letter letter or digit start dime dime dime quarter dime, quarter other character (delimiter) nickel nickel nickel nickel nickel, dime, quarter quarter COSC COSC Formal Definition of Finite Automaton A FA is represented as the five-tuple: A = (Q, Σ, δ, q 0, F) Q Finite set of states Σ Alphabet of input symbols δ A transition function q 0 start/initial state F Zero or more final/favoured/accepting states FA Transition Function We write: δ(q,a) = p where: q, p are states a is an input symbol Intuitively: if the FA is in state q, and input a is received, then the FA goes to state p (note: q = p is OK) COSC COSC Example: Clamping Logic We may think of an accepting state as representing a "1" output and non-accepting states as representing "0" output. A "clamping" circuit waits for a 1 input, and forever after makes a 1 output. However, to avoid clamping on spurious noise, we'll design an FA that waits for two 1's in a row, and "clam" only then. In general, we may think of a state as representing a summary of the history of what has been seen on the input so far. COSC The states we need are: State q 0, the start state, says that the most recent input (if there was one) was not a 1, and we have never seen two 1's in a row. State q 1 says we have never seen 11, but the previous input was 1. State q 2 is the only accepting state, it says that we have at some time seen 11. Thus, A = ({q 0, q 1, q 2 }, {0, 1}, δ, q 0, {q 2 }), where δ is given by: 0 1 >q 0 q 0 q 1 q 1 q 0 q 2 *q 2 q 2 q 2 By marking the start state with > and accepting states with *, the transition table that defines δ also specifies the entire FA COSC Slides presented May 9th and 16th,

10 Conventions It hel if we can avoid mentioning the type of every name by following some rules: Input symbols are a, b, etc., or digits. Strings of input symbols are u, v,..., z. States are q, p, etc. Transition Diagram An FA can be represented by a graph: nodes = states arc from q to p is labeled by the set of input symbols a such that δ(q, a) = p. No arc if no such a. Start state indicated by word "start" and an arrow. Accepting states get double circles. COSC COSC Example Extension of δ to Paths For the Clamping FA: start ,1 Intuitively, a FA accepts a string w = a 1 a 2 a n if there is a path in the transition diagram that: 1. Begins at the start state, 2. Ends at an accepting state, and 3. Has sequence of labels a 1,a 2,,a n COSC COSC Extension of δ to Paths (II) Formally, extend δ to Basis: i.e., stay put on no input Induction: i.e., see where the FA goes on w, then look for the transition on the last symbol from that state δ (q,w) δ (q,ε) = q δ (q,wa) = δ (δ (q,w),a) Aside Important fact with a straightforward, inductive proof: δ really represents paths. That is, if w = a 1 a 2 a n, and δ ( p i-1,a i ) = p i for all i = 1,..., n, then δ ( p 0,w) = p n COSC COSC Slides presented May 9th and 16th,

11 Acceptance of Strings FA A = (Q, Σ, δ, q 0, F) accepts string w if δ (q 0, w) is in F. Language of a FA FA A accepts the language L(A) = {w δ (q 0, w) is in F}. COSC Type Errors A major source of confusion when dealing with automata (or mathematics in general) is making "type errors." Example: Don't confuse A, a FA, i.e., a program, with L(A), which is of type "set of strings." Example: the start state q 0 is of type "state," but the accepting states F is of type "set of states." Trickier example: Is a a symbol or a string of length 1? Answer: it depends on the context, e.g., is it used in δ(q, a), where it is a symbol, or δ(q, a), where it is a string? COSC Non-deterministic Finite Automata Allow (deterministic) FA to have a choice of 0 or more next states for each state-input pair. Important tool for designing string processors, e.g., grep, lexical analyzers. But "imaginary," in the sense that it has to be implemented deterministically. Example Design an NFA to accept strings over alphabet {1, 2, 3} such that the last symbol appears previously, without any intervening higher symbol, e.g., Trick: use start state to mean "I guess I haven't seen the symbol that matches the ending symbol yet." Three other states represent a guess that the matching symbol has been seen, and remembers what that symbol is. COSC COSC Formal NFA (I) 1,2,3 p q r 2 2 t N = (Q, Σ, δ, q 0, F) where all is as DFA, but δ(q, a) is a set of states, rather than a single state. 3 3 s 1,2 COSC COSC Slides presented May 9th and 16th,

12 Formal NFA (II) Extension to δ Basis: δ (q, ε) = {q}. Induction: Let: δ (q, w) = {p 1, p 2,, p k } δ (p i, a) = S i for i = 1, 2,, k Then: δ (q, wa) = S 1 S 2 S k. COSC Language of a NFA An NFA accepts w if any path from the start state to an accepting state is labeled w. Formally: L(N) = {w δ (q 0, w) F φ}. COSC Equivalence of DFA and NFA For every NFA there is an equivalent (accepts the same language) DFA. But the DFA can have exponentially many states. From NFA N = (Q N, Σ, δ N, q 0, F N ) create equivalent DFA D = (Q D, Σ, δ D, {q 0 }, F D ): 1. Q D = 2 QN, i.e., the set of all subsets of Q N 2. F D is the set of sets S in Q D such that S F N φ 3. δ D ({q 1,q 2, q k }, a) = δ N (p 1,a) δ N (p 2,a) δ N (p k,a) Key theorem (induction on w ): δ D ({q 0 }, w) = δ N (q 0,w) Consequence: L(D) = L(N) DFA from NFA COSC COSC Subset Construction Starting with q 0 in N, for each a Σ, compute the set S of states accessible from q 0 on input a: this set becomes a new state in D, so δ D ({q 0 }, a) = S Repeat for each new state so discovered, so that for each such set S, and each p i S δ D (S, a) =δ N (p 1, a) δ N (p 2, a) δ N (p k, a) COSC Example: DFA from Clamping NFA 1,2,3 p q r s 1,2 COSC t Subset construction: δ({p},1)={p,q} δ({p},2)={p,r} δ({p},3)={p,s} δ({p,q},1)={p,q,t} δ({p,q},2)={p,r} δ({p,q},3)={p,s} δ({p,r},1)={p,q,r} ¼ Slides presented May 9th and 16th,

13 Example: δ for Clamping DFA For the NFA example above, there are 32 possible subsets computing transitions "on demand" gives a DFA with 15 states and this δ D : >p pq *pqt pr *prt *t prs *prst pqs *pqst pqr *pqrt pqrs *pqrst pq pqt pqt pqr pqr pqs pqs pqrs pqrs pqst pqst pqrt pqrt pqrst pqrst pr pr pr prt prt prs prs prst prst prs prs prt prt prst prst t t t t t t t t COSC These slides were presented in the first two lectures of COSC B, Summer COSC Slides presented May 9th and 16th,