CS375: Logic and Theory of Computing

CS375: Logic and Theory of Computing Fuhua (Frank) Cheng Department of Computer Science University of Kentucky 1

Table of Contents: Week 1: Preliminaries (set algebra, relations, functions) (read Chapters 1-4) Weeks 3-6: Regular Languages, Finite Automata (Chapter 11) Weeks 7-9: Context-Free Languages, Pushdown Automata (Chapters 12) Weeks 10-12: Turing Machines (Chapter 13) 2

Table of Contents (conti): Weeks 13-14: Propositional Logic (Chapter 6), Predicate Logic (Chapter 7), Computational Logic (Chapter 9), Algebraic Structures (Chapter 10) 3

7. Context-Free Languages & Pushdown Automata- Context-Free Languages Goal: To study a non-regular language called context-free language, and the machine called pushdown automaton that recognizes this language. Why do we want to study context-free language? 4

Can a machine recognize a language that is not regular? YES. Can have more than one production A context-free grammar has productions of the form N w where N is a non-terminal and w is any string containing terminals and/or non-terminals. 5

Why is the grammar called context-free grammar? Consider the rule S asb What this says is wherever you find S, you can replace it with asb. 6

Why is the grammar called context-free grammar? Now consider the rule CSB CaSbB This says You can replace S with asb only if it is preceded by C and followed by B. Here it imposes a condition on when S can be replaced with asb. You can apply this rule only if S appears in this particular context. Here, context is used as is generally 7 used in normal English.

Why is the grammar called context-free grammar? S asb CSB CaSbB ================================================== In summary, in the first case, you didn't need any context to apply the rule. You can apply it irrespective of the context in which S appears. So, grammars which contain only rules of first kind are called context-free grammars. 8

A context-free language is the set of strings derived from a context-free grammar. Example. { a n b n n N} is a C-F language derived from the C-F grammar Think of the strings in the following form: (ab)bb, (aabb)bb, (aaabbb)bb, S asb Λ. Example. Any regular grammar is context-free. So regular languages are C-F languages. Quiz. Find a grammar for { Answer. S asb bb. a n b n 2 n N}. 9

Quiz. Find a grammar for { is the reverse of w. Answer. S asa bsb Λ. R w w w {a, b}* }, where R w Example. S asa absba ab ba abba Question. What if w { a, b, c, d}? Would S asa bsb csc dsd Λ work? 10

Techniques for Constucting Grammars: Let L and M be two C-F grammars with disjoint sets of non-terminals and with start symbols A and B, respectively. Then L U M has grammar S A B LM has grammar S AB L* has grammar S AS Λ How about the alphabets? Why do we need a Ʌ here? 11

Let L and M be two C-F grammars with disjoint sets of non-terminals and with start symbols A and B, respectively. Then L U M has grammar S A B LM has grammar S AB L* has grammar S AS Λ (1) (2) (3) L*M has grammar S AS B Why? For L*, from (3), we have S AA AAS AA AAɅ S AA AAS But then from (2), we have AA AAB 12 (AA AAɅ)B = AA AAB

Let L and M be two C-F grammars with disjoint sets of non-terminals and with start symbols A and B, respectively. Then L U M has grammar S A B LM has grammar S AB L* has grammar S AS Λ L*M has grammar S AS B (1) (2) (3) (4) Example: find a grammar for a*b + c S A c (1) a* b A aa b (4) 13

Let L and M be two C-F grammars with disjoint sets of non-terminals and with start symbols A and B, respectively. Then L U M has grammar S A B LM has grammar S AB L* has grammar S AS Λ L*M has grammar S AS B Example: find a grammar for a*b*c + ac S A ac (1) (1) (2) (3) (4) a* b* c A aa B (4) b* c B bb c (4) 14

S asbs aasbbs a n b n S a n b n bsa a n b n bbsaa a n b n b m a m Example. Let L be the language of strings over {a, b} with the same number of a s and b s. Does L have the following grammar? S asbs bsas Λ Let the language of this grammar be called L Note that S asb bsa abs bas asbs bsas Ʌ We have a n b n, b n a n, (ab) n, (ba) n L 15 a n b n b m a m, a m b n a n b m, b m (a n b n )a m L

Example. Let L be the language of strings over {a, b} with the same number of a s and b s. Does L have the following grammar? S asbs bsas Λ a 2 b 5 a 3 ϵ L Why? because a 2 b 5 a 3 = (a 2 b 2 ) b 3 a 3 a 3 b 2 a 4 b 5 L Why? Because a 3 b 2 a 4 b 5 = a(a 2 b 2 ) a 4 b 4 b L L 16

7. Context-Free Languages & Pushdown Automata- Context-Free Languages Example. Let L be the language of strings over {a, b} with the same number of a s and b s. Does L have the following grammar? S asbs bsas Λ some notations first: S Λ : derives in one step :derives in one or moresteps :derives in zero or moresteps S + asbasbs 17 S S

7. Context-Free Languages & Pushdown Automata- Context-Free Languages Example. Let L be the language of strings over {a, b} with the same number of a s and b s. Does L have the following grammar? S asbs bsas Λ. Toshow that L L we assume that w L and show w is derived by the grammar (by induction) If w = Λ, then S Λ. Let w Λ and assume that if s L and s < w, then S + s. Show that S + w. 18

7. Context-Free Languages & Pushdown Automata- Context-Free Languages Example. Let L be the language of strings over {a, b} with the same number of a s and b s. Does L have the following grammar? S asbs bsas Λ. Consider four cases: 1. w = asb for some string s. In this case, s L and s < w. So by induction we have S + s. Therefore, we have S asbs asb + asb = w. 19

7. Context-Free Languages & Pushdown Automata- Context-Free Languages Example. Let L be the language of strings over {a, b} with the same number of a s and b s. Does L have the following grammar? S asbs bsas Λ. 3. w = axa for some string x. x has two more b s than a s, so x L. What do we do now? Notice, for example, if w = 4, then w = abba = aʌb bʌa. If w = 6, then w has one of the forms aabbba, ababba, abbaba, abbbaa. = aa bb bʌa, ab ab bʌa, aʌb ba ba, aʌb bb aa. derivable by the grammar 21 derivable by the grammar

7. Context-Free Languages & Pushdown Automata- Context-Free Languages Example. Let L be the language of strings over {a, b} with the same number of a s and b s. Does L have the following grammar? S asbs bsas Λ. 3. (conti) We claim that x can be written in the form x = ubbv where u, v L. (Can you prove it?) So by induction we have derivations S + u and S + v. Therefore, we have S asbs + aubs aubbsas + aubbvas aubbva = axa = w. 4. w = bxb for some string x. Similar to case 3. QED. 3/20/2017 22

First, we show that x contains at least two consecutive b s x = bb If each substring of b contains only one b, then w is of the following form: w = a a b a a b a a b a. a b a a b a a b a a If the number of substrings in b is m, then the number of substrings in a is m+1 The number of a s and the number of b s in w satisfy the following inequality: number of b s = m < m+1 <= number of a s So, x must contain at least two consecutive b s 23

Next, we show that x = ubbv where u, v ϵ L We will prove this by induction on the length of x. If x = 2, x = bb = ɅbbɅ, the proposition is true. We assume the proposition is true when x < 2k and show that the proposition is true when x = 2k. Let x = 2k and find the two right most consecutive b s in x. If x is of the form x = ubb, since u must be in L, so we have x = ubb = ubbʌ, u, Ʌ ϵ L Otherwise, then x must be of the following form: x = b b a y 24 for some substring y

Next, we show that x = ubbv (conti.) where u, v ϵ L Otherwise, then x must be of the following form: x = b b a y for some substring y Now consider the entry next to the first b on the left hand side. We have two cases: Or x = z a b b a y (1) x = z b b b a y (2) In the first case, azya ϵ L and zy < 2k, hence, by induction zy can be expressed as 25 zy = u bb s, u, s ϵ L

Next, we show that x = ubbv (conti.) where u, v ϵ L In the first case, azya ϵ L and zy < 2k, hence, by induction zy can be expressed as zy = u bb s, u, s ϵ L Note that bb can not be part of y. Why? If bb is part of z, we have zy = u bb t y, u, ty ϵ L or x = u bb t a b b a y u, tabbay ϵ L So if we set v = tabbay then x = ubbv, u,v ϵ L 26

Next, we show that x = ubbv (conti.) where u, v ϵ L In the first case, azya ϵ L and zy < 2k, hence, by induction zy can be expressed as zy = u bb s, u, s ϵ L If bb is not completely in z, then we have z = ub, y = bs, But then u, s ϵ L x = u b a b b a b s uba, abs ϵ L So x can still be expresses as x = U bb V, U, V ϵ L 27 with U = uba and V = abs

Next, we show that x = ubbv (conti.) where u, v ϵ L In the second case, azbbya ϵ L and zbby < 2k, hence, by induction, zbby can be expressed as zbby = s bb t, s, t ϵ L Since bb cannot be part of y, so the only possibility is y = bt, t = y, t = by or t = r bb y for some r If y = bt then s = zb and x = zbbbay = zb bb abt = u bb v, u,v ϵ L If t = y then z = s and x = z bb ba y = s bb ba y = s bb bat = u bb v with u = s, v = bat and u, v ϵ L 28

Next, we show that x = ubbv (conti.) where u, v ϵ L In the second case, azbbya ϵ L and zbby < 2k, hence, by induction, zbby can be expressed as zbby = s bb t, s, t ϵ L If t = by then z = sb and x = z bb ba y = sb bb ba y = s bb bbay = u bb v with u = s, v = bbay If t = r bb y for some r then z = sbbr and and u, v ϵ L x = z bb ba y = sbbr bb ba y = s bb rbbbay = u bb v with u = s, v = rbbbay and u, v ϵ L 29 Hence, the proposition is proved.

Examples. For a string x and letter a let na(x) be the number of a s in x. Let L = { x {a, b}* na(x) = nb(x) }. A grammar for L with start symbol E can be written as: E aebe beae Λ. Use this information to find grammars for following languages. 1. { x {a, b}* na(x) = 1 + nb(x) }. Solution: S EaE. Would S ae or S Ea work? Can this grammar generate a 4 ab 4, a 2 b 3 ab 5 a 6, b 2 aa 5 b 3? 30

1. { x {a, b}* na(x) = 1 + nb(x) }. Solution: S EaE. Woul S S Ea ae or work? Can this grammar generate a 4 ab 4, a 2 b 3 ab 5 a 6, b 2 aa 5 b 3? a 4 ab 4 = a (a 4 b 4 ) a 2 b 3 ab 5 a 6 = a 2 b 3 a b 5 a 5 a b 2 aa 5 b 3 = b 2 a 2 a(a 3 b 3 ) 31

Examples. For a string x and letter a let na(x) be the number of a s in x. Let L = {x {a, b}* na(x) = nb(x)}. A grammar for L with start symbol E can be written as: E aebe beae Λ. Use this information to find grammars for following languages. 2. {x {a, b}* na(x) = 2 + nb(x) }. Solution: S EaEaE. Why not S EaaE? Can this grammar generate a 4 ab 2 ab 2, a 2 b 3 ab 5 aa 6? a 4 ab 2 ab 2 = aa(a 2 ab 2 ab 2 ) 32 a 2 b 3 ab 5 aa 6 = (a 2 b 3 ab 5 aa 4 )aa

Examples. For a string x and letter a let na(x) be the number of a s in x. Let L = {x {a, b}* na(x) = nb(x)}. A grammar for L with start symbol E can be written as: E aebe beae Λ. Use this information to find grammars for following languages. 3. { x {a, b}* na(x) > nb(x) }. Solution: S EaET T aet Λ. To ensure the number of a s can be as many as possible What is T for? S EaET EaEaET EaEaEaET. 33

Examples. For a string x and letter a let na(x) be the number of a s in x. Let L = {x {a, b}* na(x) = nb(x)}. A grammar for L with start symbol E can be written as: E aebe beae Λ. Use this information to find grammars for following languages. 3. { x {a, b}* na(x) > nb(x) }. Solution: S EaET T aet Λ. Questions: would S EaES Ʌ or S EaES a work? None would work!!! Why? 34

Examples. For a string x and letter a let na(x) be the number of a s in x. Let L = {x {a, b}* na(x) = nb(x)}. A grammar for L with start symbol E can be written as: E aebe beae Λ. Use this information to find grammars for following languages. 4. {x {a, b}* na(x) < nb(x)}. Solution: S EbET T bet Λ. compare #4 with #3 35

7. Context-Free Languages Infinite & memory Pushdown but Automata- Pushdown Automata restricted access A pushdown automaton (PDA): a finite automaton with a stack stack operations:, push, and nop. Do nothing a PDA is an automaton that can memorize things What can we do with a PDA? can be used to model a parser (Most programming languages have deterministic PDA s ) 36

A PDA starts with one designated symbol on the stack. A state transition is determined by: current state input symbol top of the stack Representation: If the machine is in state i and the input letter is b and C is on top of the stack, then the stack and enter state j 37

Deterministic PDA: at most one legal transition for the same combination of input symbol, state and top stack symbol Nondeterminism can occur in two ways: Acceptance: A string w is accepted by a PDA if there is a path from the start state to a final state such that the input symbols 38 on the path edges concatenate to w. Otherwise, w is rejected.

Deterministic PDA: at most one legal transition for the same b, D combination of input symbol, state and top stack symbol Nondeterminism: the 2 nd case: b=ʌb, hence, if δ(i, Ʌ, C) is not empty, then processing δ(i, b, C) could give us two different legal transitions. Why? In addition to getting δ(i, b, C), we can get δ(i, Ʌ, C) first and then followed by δ(j, b, D) where j is the next state of δ(i, Ʌ, C) and D is a stack symbol right below C (after ping C). 39

Intuitively, Deterministic PDA: never have a choice Nondeterminisitic PDA: may have several alternatives how to continue A PDA is deterministic if δ(i, a, C) + δ(i, Ʌ, C) 1 for any state i, any alphabet symbol a, any stack symbol C. 40

Example. A PDA to accept the language { graph and as a set of 5-tuples. a n b n n > 0} as a Deterministic or nondeterministic? Consider: aaabbb Accepted a X 41

Quiz. How would you modify the machine to accept { n N}? a n b n Deterministic or nondeterministic? Answer: Add the instruction ( 0,, X, nop, 2), X nop 42

Example. A PDA to accept the language { graph and as a set of 5-tuples. a n b n n > 0} as a Instead of a, X push( a) a, a push( a) Would this PDA also work? Start X 0 1 b, a, X nop b, a 43

Example. A PDA to accept the language { graph and as a set of 5-tuples. a n b n n > 0} as a Take a closer look Would this PDA also work? a, X push( a) Start X a, a push( a) 0 1 b, a, X nop Deterministic or nondeterministic? b, a 44

Example. A PDA to accept the language { graph and as a set of 5-tuples. a n b n n > 0} as a a, X push( a) a, a push( a) Would this PDA also work? NO Start X 0 1 b, a, X nop b, a 45

Example. A PDA to accept the language { graph and as a set of 5-tuples. a n b n n > 0} as a a, X push( a) a, a push( a) Start X 0 b, a, X nop 1 b, a { a n b n n This PDA accepts and { a n n 0} 0} 46

Example. Find a PDA to accept the language { a 2n b n n N}. A solution: 2 a s for one b Deterministic, or nondeterministic? 47

Example. Find a PDA to accept the language { a 2n b n n N}. A solution: But, first, a definition Plays the role of a stack Instantaneous Description (ID) : ( current state, unconsumed input, stack contents ) e.g., (0, aaaabb, X) : start in state 0 with X on the stack and the input string being aaaabb 48

Example. Find a PDA to accept the language { a 2n b n n N}. A solution: 49 accepted

#4, HW6 Not accepted 0, aabaabbb, X (0, abaabbb, ax) 0, baabbb, aax (1, aabbb, ax) Execution stopped 50

Example. Find a PDA to accept the language { a 2n b n n N}. A solution: How do I know the PDA should be built this way? 51

Example. Find a PDA to accept the language { a 2n b n n N}. A solution: First, to accept, we have two choices: Start X Start X, X nop, X nop 52

Example. Find a PDA to accept the language { a 2n b n n N}. A solution: Then, build mechanism to push m a onto the stack a, X push( a) a, a push( a) a, X push ( a) a, a push ( a) Start X Start X, X nop, X nop 53

Example. Find a PDA to accept the language { a 2n b n n N}. A solution: Then, develop mechanism to stack 2 times for a given b a from the a, X push( a) a, a push( a) b = bʌ Start X, X nop b, a, a 55

Example. Find a PDA to accept the language { a 2n b n n N}. A solution: Then, build mechanism to 2 times for a given, to accept b a from the stack a 2 b a, X push( a) a, a push( a) Start X, X nop b, a, a, X nop 56

Example. Find a PDA to accept the language { a 2n b n n N}. A solution: Then, build mechanism to 2 times for each, to accept b a a 2n b n from the stack a, X push( a) a, a push( a) b, a Start X, X nop b, a, a, X nop 57

Example. Find a PDA to accept the language { a 2n b n n N}. A solution: Then, build mechanism to 2 times for each, to accept b a a 2n b n from the stack a, X push( a) a, a push( a) b, a Remove this edge if n>0 Start X, X nop b, a, a, X nop 58

Example. Find a PDA to accept the language { a n b 2n n N}. Idea: build mechanism to every two s b a from the stack once for b, a nop b, a b, a nop 59

Example. Find a PDA to accept the language { a n b 2n n N}. Idea: build mechanism to for every two s b a from the stack once for OR b, a b, a nop b, a 60

Empty-Stack Acceptance. Instead of final-state acceptance, a PDA can also be defined to accept a string by the condition that the stack is empty. The two types of acceptance are equivalent in that they both define PDAs that accept the same class of languages. empty-stack PDA final-state PDA 61

if xxxx is accepted by the final-state Transform final-state PDA then acceptance ɅxxxxɅ is accepted to empty-stack by the acceptance: empty-stack PDA Introduce a new start state s, a new stack symbol Y, a new empty state e, and construct edges and actions as shown in the diagram below. Note that? stands for any stack symbol. 62

Example: given the following final-state PDA, convert it to a empty-stack PDA, X nop 63

Example: given the following final-state PDA, convert it to a empty-stack PDA, Y Start Y s, Y push( X ), X e, X nop 64

Example: given the following final-state PDA, convert it to a empty-stack PDA OR Start Y s, Y push( X ), X, Y e, X 65

Transform empty-stack acceptance to final-state acceptance: Introduce a new start state s, a new stack symbol Y, a new final state ƒ, and construct edges and actions as shown in the diagram below. 66

Theorem: The context-free (C-F) languages are exactly the languages accepted by PDAs. Proof: We ll see two algorithms, one to transform a C-F grammar into an empty-stack PDA, & one to transform an empty-stack PDA into a C-F grammar. QED. C-F grammar mpty-stack PDA final-state PDA 67

Transform a C-F grammar to empty-stack PDA: - The idea is to use the stack of the PDA to simulate the (leftmost) derivation of a string in the grammar. - The PDA has ONE STATE, but we allow multiple stack operations. - The stack symbols are the terminals and nonterminals of the grammar. - The designated starting stack symbol is the grammar start symbol. 3/20/2017 68

Transform a C-F grammar to empty-stack PDA: Basic idea: Use the stack of the PDA to simulate the (leftmost) derivation of a string in the grammar Push S (start symbol of CFG) on the stack From this point on, there are two moves the PDA can make: 1. 1. If a non-terminal A is on the top of the stack, it and push the right-hand side of a production A B1B2 Bn from G 2. If a terminal a is on the top of the stack, it and match it with whatever symbol is being read from the input string 3/20/2017 69

Leftmost and Rightmost Derivations Consider the grammar G = ({S, A, B, C}, {a, b, c}, S, P) where P = { S ABC, A aa Ʌ, B bb Ʌ, C cc Ʌ } Leftmost derivation (always expand the leftmost variable first): S ABC aabc abc abbc abbbc abbc abbcc abbc Rightmost derivation (always expand the rightmost variable first): S ABC ABcC ABc AbBc AbbBc Abbc aabbc abbc Note: 1. Different derivations result in different sentential forms, but 2. For a C-F grammar, it doesn't make difference in what order we expand the variables. 70

How to simulate a leftmost derivation of abbc for the grammar S ABC A aa B bb C cc S ABC aabc abc abbc abbbc abbc abbcc abbc S ABC A aa A Ʌ B bb B bb B Ʌ C cc C Ʌ 71

S ABC abbbc aabc abc abbc abbc abbcc abbc S ABC A aa A Ʌ B bb B bb B Ʌ C cc C Ʌ push S + push A B C + push a A B C A B C B C + push b B C 72 Output: a b

S ABC abbbc aabc abc abbc abbc abbcc abbc S ABC A aa A Ʌ B bb B bb B Ʌ C cc C Ʌ B C + push b B C B C C + push c C C 73 Output: a b b c

How to simulate a leftmost derivation of aaab for the grammar S asb aas S asb a( aas) b a( aa) b S asb ; S aas ; S Ʌ push S + push a S b S b + push a a S b a S b S b Output: a a a b 74 b

Transform a C-F grammar to empty-stack PDA: Algorithm: 1. For each terminal a, deterministic or non-deterministic? create instruction (0, a, a,, 0) a, a S 0 Why? 75

Algorithm: 1. For each terminal a, create instruction (0, a, a,, 0) S a, a 0 Why? A terminal a exists in a production of the form B a or B aw. Once B is replaced with a or aw, the stack looks like: 76 a or For a PDA, we need an instruction that can be used to remove a from the stack (to empty the stack): a, a a w

Transform a C-F grammar to empty-stack PDA: Algorithm: 2. For each production A B1B2 Bn create instruction (0, Ʌ, A,, push(bn),, push(b1), 0) Ʌ, A, push B n,, push (B 1 ) S 0 Why? 77

Algorithm: 2. For each production A B1B2 Bn create instruction (0, Ʌ, A,, push(bn),, push(b1), 0) Why do we know A would be in the stack? Because before this production, we would have a production like: S XAY Therefore, by executing (0, Ʌ, S, <, push(y), push(a), push(x)>, 0) S 78 X A Y

Transform a C-F grammar to empty-stack PDA: Algorithm: 3. For each production A Ʌ create instruction (0, Ʌ, A,, 0) Ʌ, A S 0 Why? 79

Transform a C-F grammar to empty-stack PDA: Algorithm: 3. For each production A Ʌ create instruction (0, Ʌ, A,, 0) Ʌ, A Why? The production A Ʌ is for us to terminate a derivation process. S 0 For a PDA, we need an instruction that can be used to empty a stack: Ʌ, A 80

Example. For the grammar: S Ʌ asb aas, do a leftmost derivation of aaab and show the IDs for executing it. First, get PDA instructions for terminals and productions: Terminals Corresponding PDA instructions a ( 0, a, a,, 0 ) b ( 0, b, b,, 0 ) Start with this one Productions Corresponding PDA instructions S asb (0,, S,, push( b), push( S), push( a), 0) S aas (0,, S,, push( S), push( a), push( a), 0) S (0,, S,, 81 0)

Example. For the grammar: S Ʌ asb aas, do a leftmost derivation of aaab and show the IDs for executing it. S a, a 0 Ʌ, S b, b Ʌ, S, push b, push S, push(a) Ʌ, S, push S, push a, push(a) 82

Example. For the grammar: S Ʌ asb aas, do a leftmost derivation of aaab and show the IDs for executing it. Leftmost derivation of aaab S asb a( aas) b a( aa) b 83

Example. For the grammar: S Ʌ asb aas, do a leftmost derivation of aaab and show the IDs for executing it. ID for aaab: PDA instruction: (0, aaab, S) (0, aaab, asb) (0, aab, Sb) (0, aab, aasb) (0, ab, asb) (0, b, Sb) (0, b, b) (0,, ) S aa a S b S ( 0,, S,, push( b), push( S), push( a) ( 0, a, a,, 0) ( 0,, S,, push( S), push( a), push( a) ( 0, a, a,, 0) ( 0, a, a,, 0) ( 0,, S,, 0) ( 0, b, b,, 0) 84 Accepted, 0), 0)

End of Context-Free Languages and Pushdown Automata I 85