The View Over The Horizon

Transcription

1 The View Over The Horizon enumerable decidable context free regular

2 Context-Free Grammars An example of a context free grammar, G 1 : A 0A1 A B B # Terminology: Each line is a substitution rule or production. Each rule has the form: symbol string. The left-hand symbol is a variable (usually upper-case). A string consists of variables and terminals. One variable is the start variable.

3 Rules for Generating Strings Write down the start variable (lhs of top rule). Pick a variable written down in current string and a derivation that starts with that variable. Replace that variable with right-hand side of that derivation. Repeat until no variables remain. Return final string (concatenation of terminals). Process is inherently non deterministic.

4 Example Grammar G 1 : A 0A1 A B B # Derivation with G 1 : A 0A1 00A11 000A B #111

5 AParseTree A A A B # Question: What strings can be generated in this way from the grammar G 1? Answer: Exactly those of the form 0 n #1 n (n 0).

6 Context-Free Languages The language generated in this way is called the language of the grammar. For example, L(G 1 ) is {0 n #1 n n 0}. Any language generated by a context-free grammar is called a context-free language.

7 AUsefulAbbreviation Rules with same variable on left hand side A 0A1 A B are written as: A 0A1 B

8 English-like Sentences AgrammarG 2 to describe a few English sentences: < SENTENCE > < NOUN-PHRASE >< VERB > < NOUN-PHRASE > < ARTICLE >< NOUN > < NOUN > boy girl flower < ARTICLE > a the < VERB > touches likes sees

9 Deriving English-like Sentences A specific derivation in G 2 : < SENTENCE > < NOUN-PHRASE >< VERB > < ARTICLE >< NOUN >< VERB > a < NOUN >< VERB > aboy < VERB > aboysees More strings generated by G 2 : aflowersees the girl touches

10 Derivation and Parse Tree < SENTENCE > < NOUN-PHRASE >< VERB > < ARTICLE >< NOUN >< VERB > a < NOUN >< VERB > aboy < VERB > aboysees SENTENCE NOUN-PHRASE VERB ARTICLE NOUN a boy sees

11 Formal Definitions A context-free grammar is a 4-tuple (V,Σ,R,S) where V is a finite set of variables, Σ is a finite set of terminals, R is a finite set of rules: each rule is a variable and a finite string of variables and terminals. S is the start symbol.

12 Formal Definitions If u and v are strings of variables and terminals, and A w is a rule of the grammar, then we say uav yields uwv, written uav uwv. We write u v if u = v or u u 1... u k v. for some sequence u 1,u 2,...,u k. Definition: { Thelanguage } of the grammar is w Σ S w.

13 Example Consider G 4 =(V,{a, b},r,s). R (Rules): S asb SS ε. Some words in the language: aabb, aababb. Q.: Butwhatis this language? Hint: Think of parentheses.

14 Arithmetic Example Consider (V,Σ,R,E) where V = {E,T,F} Σ = {a, +,, (, )} Rules: E E + T T T T F F F (E) a Strings generated by the grammer: a + a a and (a + a) a. What is the language of this grammer? Hint: arithmetic expressions. E = expression, T = term, F = factor.

15 Parse Tree for a + a a E E + T T T T F F F (E) a E E T T T F F F a + a X a

16 Parse Tree for (a + a) a E T E E + T T T T F F F (E) a T F E E T T F F F ( a + a ) X a

17 Designing Context-Free Grammars No recipe in general, but few rules-of-thumb If CFG is the union of several CFGs, rename variables (not terminals) so they are disjoint, and add new rule S S 1 S 2... S i. To construct CFG for a regular language, follow a DFA for the language. For initial state q 0,makeR 0 the start variable. For state transition δ(q i,a)=q j add rule R i ar j to grammar. For each final state q f, add rule R f ε to grammar. For languages with linked substrings (like {0 n #1 n n 0} ), a rule of form R urv may be helpful, forcing desired relation between substrings.

18 Closure Properties Regular languages are closed under union concatenation star Context-Free Languages are closed under union : S S 1 S 2 concatenation S S 1 S 2 star S ε SS

19 More Closure Properties Regular languages are also closed under complement (reverse accept/non-accept states of DFA) ) intersection (L 1 L 2 = L 1 L 2. What about complement and intersection of context-free languages? Not clear...

20 Chomsky Normal Form A simplified, canonical form of context free grammers. Every rule has the form A BC A a S ε where S is the start symbol, A, B and C are any variable, except B and C not the start symbol, and A can be the start symbol.

21 CFN: Theorem Theorem: Any context-free language is generated by a context-free grammar in Chomsky normal form. Basic idea: Add new start symbol S 0. Eliminate all ε rules of the form A ε. Eliminate all unit rules of the form A B. Patch up rules so that grammar generates the same language. Convert remaining long rules to proper form.

22 Proof Add new start symbol S 0 and rule S 0 S. Guarantees that new start symbol does not appear on right-hand-side of a rule.

23 Proof Eliminating ε rules. Repeat: remove some A ε. for each R uav, add rule R uv. and so on: for R uavaw add R uvaw, R uavw, and R uvw. for R A add R ε, except if R ε has already been removed. until all ε-rules not involving the original start variable have been removed.

24 Proof Eliminate unit rules. Repeat: remove some A B. for each B u, add rule A u, unless this is previously removed unit rule. (u is a string of variables and terminals.) until all unit rules have been removed.

25 Proof Finally, convert long rules. To replace each A u 1 u 2...u k (for k 3), introduce new non-terminals N 1,N 2,...,N k 1 and rules A u 1 N 1 N 1 u 2 N 2. N k 3 u k 2 N k 2 N k 2 u k 1 u k

26 Conversion Example Initial Grammar: S ASA ab A B S B b ε (1) Add new start state: S 0 S S ASA ab A B S B b ε

27 Conversion Example (2) S 0 S S ASA ab A B S B b ε (2) Remove ε-rule B ε: S 0 S S ASA ab a A B S ε B b ε

28 Conversion Example (3) S 0 S S ASA ab a A B S ε B b (3) Remove ε-rule A ε: S 0 S S ASA ab a AS SA S A B S ε B b

29 Conversion Example (4) S 0 S S ASA ab a AS SA S A B S B b (4) Remove unit rule S S S 0 S S ASA ab a AS SA S A B S B b

30 Conversion Example (5) S 0 S S ASA ab a AS SA A B S B b (5) Remove unit rule S 0 S: S 0 S ASA ab a AS SA S ASA ab a AS SA A B S B b

31 Conversion Example (6) S 0 ASA ab a AS SA S ASA ab a AS SA A B S B b (6) Remove unit rule A B: S 0 ASA ab a AS SA S ASA ab a AS SA A B S b B b

32 Conversion Example (7) S 0 ASA ab a AS SA S ASA ab a AS SA A S b B b Remove unit rule A S: S 0 ASA ab a AS SA S ASA ab a AS SA A S b ASA ab a AS SA B b

33 Conversion Example (8) S 0 ASA ab a AS SA S ASA ab a AS SA A b ASA ab a AS SA B b (8) Final simplification treat long rules: S 0 AA 1 UB a SA AS S AA 1 UB a SA AS A b AA 1 UB a SA AS A 1 SA U a B b

34 Example 2 Consider G 5 =(V,{a, b},r,s). R (Rules): S asb bsa ε. Some words in the language: abba, aabaabaa. Q.: Butwhatis this language? L(G 5 )={ww R w {a, b} }

35 Example 3 Consider G 6 =(V,{a, b},r,s). R (Rules): S ab ba ε. A a as baa. B b bs abb. Some words in the language: aababb, baabba. Q.: But what is this language? L(G 6 )={w {a, b} # a (w) =# b (w)}

36 Example L(G 6 )={w {a, b} # a (w) =# b (w)} What do we need to show?! (1) If w generated by G 6 then # a (w) =# b (w). (2) If # a (w) =# b (w) then w generated by G 6. Claim: If S α then # a (α)+# A (α) =# b (α)+# B (α). Claim: For any w let k =# a (w) # b (w). Then: (i) if k =0then S ws, (ii) if k>0then S wa k, (iii) if k<0then S wb k Are we done?!

37 Non-Context-Free Languages The pumping lemma for finite automata and Myhill-Nerode theorem are our tools for showing that languages are not regular. We will now show a similar pumping lemma for context-free languages. It is slightly more complicated...

38 Pumping Lemma for CFL Also known as the uvxyz Theorem. Theorem: If A is a CFL, there is an l (critical length), such that if s A and s l, thens = uvxyz where for every i 0, uv i xy i z A vy > 0, (non-triviality) vxy l.

39 Basic Intuition Let A be a CFL and G its CFG. Let s be a very long string in A. Then s must have a tall parse tree. And some root-to-leaf path must repeat a symbol. ahmmm,... why is that so? T R R u v x y z

40 Proof let G be a CFG for CFL A. let b be the max number of symbols in right-hand-side of any rule. no node in parse tree has >bchildren. at depth d, canhaveatmostb d leaves. let V be the number of variables in G. set l = b V +2.

41 Proof (2) let s be a string where s l let T be parse tree for s with fewest nodes T has height V +2 some path in T has length V +2 that path repeats a variable R

42 Proof (3) Split s = uvxyz T R R u v x y z each occurrence of R produces a string upper produces string vxy lower produces string x

43 Proof Replacing smaller by larger yields uv i xy i z,fori>0. T R R u v Rx y z

44 Proof (4) Replacing larger by smaller yields uxz. T R x u z Together, they establish: for all i 0, uv i xy i z A

45 Proof (5) Next condition is: vy > 0 (out of uvxyz) If v and y are both ε, then T R x u z is a parse tree for s with fewer nodes than T, contradiction.

46 Proof (6) Final condition is vxy l: T R R V +2 u v x y z b V +2 the upper occurrence of R generates vxy. can choose symbols such that both occurrences of R lie in bottom V +1variables on path. subtree where R generates vxy is V +2high. strings by subtree at most b V +2 = l long.

47 Non CFL Example Theorem: B = {a n b n c n } is not a CFL. Proof: By contradiction. Let l be the critical length. Consider s = a l b l c l. If s = uvxyz, neither v nor y can contain both a s and b s, or both b s and c s, because otherwise uv 2 xy 2 z would have out-of-order symbols. But if v and y contain only one letter, then uv 2 xy 2 z has an imbalance!

48 Non CFL Example (2) The language C = { a i b j c k 0 i j k } is not context-free. Let l be the critical length, and s = a l b l c l. Let s = uvxyz neither v nor y contains two distinct symbols, because otherwise uv 2 xy 2 z would have out-of-order symbols. vxy cannot be all the same letter (why?) vxy l, so either v contains only a s and y contains only b s, but then uv 2 xy 2 z has too few c s. v contains only b s and y contains only c s. but then uv 0 xy 0 z has too many a s.

49 Non CFL Example (3) The language D = { ww w {0, 1} } is not context-free. Let s =0 l 1 l 0 l 1 l As before, suppose s = uvxyz. Recall that vxy l. if vxy is in the first half of s, uv 2 xy 2 z moves a 1 into the first position in second half. if vxy is in the second half, uv 2 xy 2 z moves a 0 into the last position in first half. if vxy straddles the midpoint, then pumping down to uxz yields 0 l 1 i 0 j 1 l where i and j cannot both be l. Note that D = { ww R w {0, 1} } is CFL.

50 CFL Closure Properties: Example Is L = {(0 1 2) : #of0 s = # of 1 s = # of 2 s } context free?

51 CFL Closure Properties L {(0 1 2) : # 0 s = # 1 s = # 2 s } Is L context free? L ={0 n 1 n 2 n : n>0} which is not context free. Context free languages intersected with a regular languages are context free is regular. So L is not a context free language! This could also be established using pumping lemma, but proof above is much more elegant.

52 More CFL Closure Properties Assignments Homomorphism Inverse Homomorphism

53 Algorithmic Questions for NFAs Q.: GivenanNFA,N, andastrings, iss L(N)? Answer: ConstructtheDFAequivalenttoN and run it on w. Q.: IsL(N) =? Answer: This is a reachability question in graphs: Is there a path in the states graph of N from the start state to some accepting state. There are simple, efficient algorithms for this task.

54 More Algorithmic Questions for NFAs Q.: IsL(N) =Σ? Answer: CheckifL(N) =. Q.: GivenN 1 and N 2,isL(N 1 ) L(N 2 )? Answer: CheckifL(N 2 ) L(N 1 )=. Q.: GivenN 1 and N 2,isL(N 1 )=L(N 2 )? Answer: CheckifL(N 1 ) L(N 2 ) and L(N 2 ) L(N 1 ). In the future, we will see that for stronger models of computations, many of these problems cannot be solved by any algorithm.

55 Algorithmic Questions Regarding CFGs Given a CFG, G, and a string w, doesg generate w? Initial Idea: Design an algorithm that tries all derivations. Problem: If G does not generate w, we ll never stop.

56 Algorithmic Questions for CFGs (2) Lemma: If G is in Chomsky normal form, w = n, andw is generated by G, thenw has a derivation of length 2n 1 or less. Algorithm s idea: First, convert G to Chomsky normal form. Now need only consider a finite number of derivations those of length 2n 1 or less.

57 Algorithmic Questions for CFGs: membership Build a function derive(a, x) that returns TRUE if A x PROCEDURE derive(a, x). If x =1then if A x R return TRUE, otherwise return FALSE. For each A BC and each partition x = x 1 x 2, Call derive(b,x 1 ) and derive(c, x 2 ). If both return true, exit and return TRUE. Return FALSE. Test membership by calling derive(s, w). Why did we need CNF?!

58 Emptiness of CFGs Given a CFG, G, is L(G) =? In other words, is there any string w, suchthatg generate w? Theorem: There is an algorithm that solves this problem (and always halts). Possible approaches for a proof: Bad Idea: We know how to test whether w L(G) for any string w, so just try it for each w. (criticize this...) Better Idea: Can the start variable generate a string of terminals? Even Better Idea: Can a particular variable generate a string of terminals?

59 Removing redundant variables and terminals of a CFG 1. Mark all terminal symbols in G. 2. Repeat until no new variable become marked: Mark any A where A U 1 U 2...U k and all U i have already been marked. 4. Remove all unmarked variables, and any rule they appear in. 5. If S is removed, then L(G) =. 6. Remove any variable A not reachable from S. 7. Remove any terminal which does not appear in some rule.

60 CFG Emptiness (2) Algorithm: On input G (a CFG), 1. Remove redundant variables and terminals, get G. 2. L(G )= iff S is removed

61 Finiteness of CFGs Given a CFG, G, is L(G) finite? 1. Remove redundant variables and terminals. 2. Create a graph where nodes are variables and directed edges are derivations. 3. L(G) is infinite iff the graph has a cycle.

62 CFGs Fullness Given a CFG, G, is L(G) =Σ? We just saw an algorithm to determine, given a CFG, G, if L(G) = L(G) =Σ iff L(G) =. Why not modify the algorithm so it determines emptiness of the complement? Unfortunately, CFGs are not closed under complement. Fact: There is no algorithm to solve this problem. We are not prepared to prove this remarkable fact (yet).

63 When Are Two CFGs equivalent? Given two CFGs, G, H, is L(G) =L(H)? Hey, we did this already for equivalence of DFAs! We constructed C from A and B: ( ) L(C) = L(A) L(B) and tested whether L(C) is empty. ( ) L(A) L(B).

64 When Are Two CFGs equivalent? This approach was fine for DFAs, but not for CFLs! The class of context-free languages is not closed under complementation or intersection. Fact: There is no algorithm to solve this problem. We are not prepared to prove this remarkable fact (yet).

65 Computations A computation of a Turing machine is a sequence of configurations beginning with an initial configuration and ending with a halting configuration. Computations are (primitive) recursive.

66 Non-computations The non-computations of a Turing machine M with input x can be presented as a context-free language L(G), which include sequences without an initial state sequences without a final state configurations with more than one state adjacent configurations that are wrong etc.

67 Fullness Problem: Full(G) = [L(G)=Σ*] Reduction: Halt Full halt(m,x) = Full(G(M,x)) So fullness of CFG is undecidable

68 Context-Sensitive Grammars A context-sensitive grammar has production rules of the form uaw uvw where u,v,w are strings of terminal symbols.

69 Arbitrary Grammars An arbitrary grammar has production rules of the form u v where u and v are strings of terminal symbols and variables (non-terminals), and u is non-empty. (Alternatively, one can require that u have exactly one variable.)

70 Arbitrary Grammars Turing machines can be simulated by an arbitrary grammar. For normal computations contextsensitive grammars suffice.

71 Language Classes In general, if a class C of recursive languages is recursively enumerable (e.g. regular, context-free), then there is a recursive language not in C. Proof by diagonalization.

72 Arbitrary Grammar for TMs Terminals: Tape symbols plus $, Variables: States Q i, plus A j for each tape symbol a j, plus S,B,C Three stages: 1. S => => w$q 0 w$ (initial state q 0 ) 2. Q 0 w => => uq f v (final state q f ) 3. $uq f v$ => => ε

73 Stage 1 S CB B $ B a j A j B for each a j A j a k a k A j A j a j Ca k a k C C $Q 0

74 Stage 2 Q i a j a k Q l for each right move Q i $ a k Q l $ a j =blank a m Q i a j Q l a m a k for each left move a m Q i $ Q l a m a k $ a j =blank

75 Stage 3 Q f a j Q f a m Q f Q f $Q f $ ε

76 Language Classes In general, if a class C of recursive languages is recursively enumerable (e.g. regular, contextfree), then there is a recursive language not in C. Proof by diagonalization. Moral: One cannot define a set of grammars to exactly capture the recursive languages.

77 A Short Summary Regular Languages Finite Automata. Context Free Languages Push Down Automata. Closure properties of regular languages and of CFLs. Most algorithmic problems for finite automata are solvable. Some algorithmic problems for finite automata are not solvable. Pumping lemmata for both classes of languages. There are additional languages out there.