1 CSCI- 2200 FOUNDATIONS OF COMPUTER SCIENCE Spring 2015 May 11, 2015
2 Announcements The bonus quiz points will be posted on LMS today. You can earn up to 3 bonus points on the final exam. I will have office hours on Thursday 12pm 2pm (usual Ume). The TAs will hold usual office hours this week. Please check LMS to make sure your grades are correct. Email me if you see any issues. Homeworks 5-7 will be in front of my office door M- W 12pm 4pm this week.
FINAL EXAM INFO 3
4 Final Exam LogisUcs Final Exam is Wednesday, May 20, 2015 from 11:30am 2:30pm in West Hall Auditorium. You can bring two 8.5 x11 double- sided crib sheets Calculators are not allowed (you won t need one) Logical equivalences, set idenuues, rules of inference, and useful summauon formulae will be provided. See FOCS final exam tables link on web site Usual disclaimer: anyone caught cheaung on the final will receive an immediate F in the course.
5 Final Exam Info The final will last 3 hours. There will be about 10 problems. The quesuon formats will be like the homework problems and the good problems to review listed on the slides. There are review problems posted on the course web site. I will not post soluuons. The TAs will go over some of the problems in recitauon. You can also ask about review problems in office hours.
6 Final Exam Topics Disclaimer: This is a high level overview of what will be on the final. Unless I specifically say something will not be on the final, it may be on the final. The exam will be cumulauve. There will be more emphasis (at least 50%) on the material from aeer Exam 2.
7 Final Exam Topics (Pre- Exam 2) Pre- Exam 2 material Informal proofs Direct proof, proof by contraposiuon, proof by contraposiuon, existence proofs, etc. Proofs by inducuon Concepts from logic, number theory sets, funcuons, sequences, and series may appear in quesuons (and answers), but the focus will be proofs. Combinatorics PermutaUons and CombinaUons, the pigeonhole principle, etc.
8 Final Exam Topics (Post- Exam 2) Discrete Probability Laplace s Theory of Probability Probability DistribuUons CondiUonal Probability Bayes Theorem Independence
9 Final Exam Topics (Post- Exam 2) Models of ComputaUon Regular Languages Finite Automata Context- Free Languages and Grammars Turing machines Complete descripuons, high level descripuons, TM variants LimitaUons of computauonal models Decision problems P and NP
10 Review: Languages A string is a sequence of lehers defined over an alphabet, Example alphabet: = {a, b} is the set of all possible strings made from the alphabet Example: = {,a,b,aa,ab,ba,bb,aaa,...} A language is a set of strings. It is a subset of
The Chomsky Hierarchy 11 Turing-Recognizable Languages Turing-Decidable Languages Context-Free Languages Regular Languages
12 Regular Languages The class of languages accepted by the weakest computauonal model that we have studied finite automata. Small computauonal power, e.g. vending machine, elevator, etc. Theorem: A language is regular if and only if it is accepted by a determinisuc finite automaton. Theorem: A language is regular if and only if it is accepted by a nondeterminisuc finite automaton. Theorem: A language is regular if and only if it there is a regular expression that describes it.
13 Regular Languages Examples of languages that are not regular: {ww R w Σ*} {0 k 1 k k 1} {w w {0,1}* and w contains and equal number of 0 s and 1 s}
The Chomsky Hierarchy 14 Turing-Recognizable Languages Turing-Decidable Languages Context-Free Languages Regular Languages
15 Context- Free Languages A context free language is a language that can be recognized by another type of computauonal model a pushdown automata stack Input string Output automaton accept or reject A context free language can be described by a context- free grammar (CFG). Important for compilers and NLP.
16 Context- Free Grammars You will not have to write a context- free grammar on the final exam. You may have to derive strings using the rules of a CFG. You may also be asked to idenufy the language defined by a CFG.
The Chomsky Hierarchy 17 Turing-Recognizable Languages Turing-Decidable Languages Context-Free Languages Regular Languages
A Turing Machine Tape... 18... Control Unit Read-Write head - The Read- Write head may move both lee and right over the input tape. - A TM may both read from and write to its input tape. - A TM halts immediately when it reaches an accept or reject state.
19 Turing Machines A complete descripuon of a Turing Machine is the state machine diagram. A high- level descripuon of a Turing Machine is a paragraph. If a TM accepts a language L, then The TM halts in the accept state for all strings in L For strings not in L, the TM may halt in the reject state or loop forever If a TM decides a language L, then The TM halts in the accept state for all strings in L The TM halts in the reject state for all strings not in L
20 Turing Machines The standard Turing Machine is determinisuc, with a single tape that is infinite in one direcuon. Variants: doubly infinite tape, muluple tape, nondeterminisuc Each variauon has the same power as the standard model Running Ume may be different. A Turing machine can be encoded as a bit string. Given a TM as input, a universal Turing machine can simulate that TM.
21 Turing Machines and Languages There are infinitely many Turing machines and infinitely many languages. Theorem: there are some languages that are not recognized by any Turing machine. Proof outline: The set of Turing machines is countable (the smaller infinity). The set of languages is uncountable (the bigger infinity). Therefore, there are more languages than Turing machines. Theorem: If S is an infinite countable set, then P (S) is an uncountable set.
L = {a i : a i 2 L(M i )} Turing-Recognizable Languages i i 22 L = { a : a L( Mi )} Turing-Decidable Languages Context-Free Languages Regular Languages
23 Decision Problems Decision problems are problems with a yes or no answer A problem is decidable if some Turing machine solves it. Outputs YES (accept) when the answer is yes and outputs NO (reject) when the answer is no. The machine must halt. This is same as saying the language for the problem is Turing- decidable.
24 The HalUng Problem Input: Turing Machine M and string w QuesUon: Does M halt on input w? Theorem: The halung problem is undecidable. There is no TM that can determine whether M halts on input w for all M and w. Key Idea of Proof: If the halung problem is decidable, this implies: For all languages L, If L is Turing- recognizable, then L is also Turing- decidable This is not true, so the halung problem is undecidable.
REDUCIBILITY 25
26 Reducibility A reduc@on is a way of converung (reducing) one problem to another problem, so that the soluuon to the second problem can be used to solve the first problem. Finding the area of a rectangle reduces to measuring its width and height. Solving a set of linear equauons reduces to inverung a matrix. If problem A can be reduced to problem B then, if we can solve B, we can solve A. If B is decidable, then A is decidable. If A is undecidable, then B is undecidable. (contraposiuve)
27 The Blank- Tape HalUng Problem The blank- tape halung problem: Input: Turing machine M QuesUon: Does M halt when started with a blank tape? Theorem: The blank- tape halung problem is undecidable. Proof: We will reduce the halung problem to the blank tape halung problem. This means that, if there is a TM that solves the blank tape halung problem, there is a TM that solves the halung problem. But, this is impossible since the halung problem is undecidable.
Suppose we have a decider for the blank-tape halting problem: 28 M ' blank-tape halting problem decider YES NO M ' halts on blank tape M ' doesn t halt on blank tape
We want to build a decider for the halting problem: 29 M halting problem YES M halts on w decider w NO M doesn t halt on w
We want to reduce the halting problem to the blank-tape halting problem: 30 M M ' Blank-tape halting problem decider YES NO YES NO w Halting problem decider
We need to convert one problem instance to the other problem instance 31 Halting problem decider M w Convert Inputs? M ' Blank-tape halting problem decider YES NO YES NO
Construct a new Turing machine M ' 32 When started on blank tape, writes w Then continues execution like M M ' step 1 if blank tape then write w step2 execute M with input w
33 M halts on input string w if and only if M ' halts when started with blank tape
34 M w Generate M ' M ' blank-tape halting problem decider YES NO YES NO Halting problem decider
35 We reduced the halting problem to the blank-tape halting problem Since the halting problem is undecidable, the blank-tape halting problem is undecidable QED
36 Summary of Undecidable Problems The membership problem Does machine M accept a string w? The halung problem Does machine M halt on input w? The blank tape halung problem Does machine M halt when starung with a blank tape?
RICE S THEOREM 37
38 Non- Trivial ProperUes of Languages A property Q defines a subset of all Turing- recognizable languages S. A property is trivial if Q = S or Q = Otherwise it is non- trivial. A non- trivial property of a Turing- recognizable language is a property such that Some Turing- recognizable language has this property Some Turing- recognizable language does not have this property.
39 Rice s Theorem Theorem: Any non- trivial property of a Turing- recognizable language is undecidable. Examples of non- trivial properues The language is empty The language is finite The language contains two different strings of the same length
P AND NP 40
41 Complexity Theory For pracucal purposes, it is not enough for a problem to be decidable. What if it takes an inordinate amount of Ume and memory to solve? We also care about the Ume, memory, and other resources needed to solve the problem. This is the focus of computauonal complexity theory. We will consider the @me complexity of problems.
42 Time Complexity Consider the language A = {0 k 1 k k 0}. A is decidable. What is the @me complexity needed to decide this language? How many steps does a Turing machine need to take to decide an input string of length n? What is its running @me? A step is a move of the read- write head over a single slot on the tape. We measure the worst- case @me complexity. The longest running Ume needed for any input of length n. We use big- O notauon.
43 Time Complexity (cont.) Consider the language A = {0 k 1 k k 0}. A is decidable. What is the Ume complexity of the TM that decides this language? We described two Turing machines that decide A. Single- tape machine Zig- zag back and forth crossing off a 0 and then a corresponding O(n 2 ) 1. MulU- tape machine (2 tapes) Leave all 0s on tape 1 Copy all 1s to tape 2 Compare tape 1 and tape 2 Running Time: Running Time: O(n)
44 Time Complexity (cont.) The Ume complexity depends on the computauonal model. We will use the mulu- tape Turing machine model. We define a class of languages DTIME(n). This is the set of languages that can be decided by a determinisuc mulutape Turing Machine with Ume complexity O(n). Languages in this class include: A = {0 k 1 k k 0} B = {ww w {a,b}*}
45 More Time Complexity Classes We can define other classes in a similar way to DTIME(n) E.g, DTIME(n 2 ), DTIME(n 3 ), etc. DTIME(n 2 ) is the set of languages that can be decided by a mulu- tape Turing machine in O(n 2 ). Theorem: DTIME(n k ) DTIME(n k+1 ) Polynomial Ume algorithms are those in DTIME(n k ) for some integer k. These are tractable algorithms. For small k, they can be computed quickly.
46 The class P P is the class of languages that are decidable on a determinis@c Turing machine in polynomial Ume. P = [ k DTIME(n k ) All determinisuc computauonal models are polynomially equivalent. The class P does not depend on the choice of determinisuc computauonal model.
47 ExponenUal Time Algorithms ExponenUal Ume algorithms are the class DTIME(2 n ) This class contains intractable problems. Some problem instances may take centuries to solve. Example: The SaUsfiability Problem Input: a boolean expression (a logical proposiuon) in conjuncuve normal form (CNF) QuesUon: Is this expression sausfiable?
48 The SaUsfiability Problem Is this sausfiable? ( x 1 _ x 2 ) ^ (x 1 _ x 3 ) Yes. x 1 = F, x 2 = T,x 3 = T Is this sausfiable? (x 1 _ x 2 ) ^ x 1 ^ x 2 No. We can define the language L = { w w is a proposiuon in CNF and w is sausfiable} L DTIME(2 n ) Need brute force search of all possibiliues
49 Non- Determinism So far, we have analyzed the running Ume on determinisuc Turing machines. We can also consider running Ume on a non- determinisuc Turning machine. Example: Non- determinisuc algorithm for the sausfiability problem. Guess an assignment for the variables. Check if it makes the expression true. In a determinisuc algorithm, we try each assignment one at a Ume. In a nondeterminisuc algorithm, we try all assignments at the same Ume, in parallel.
50 The SaUsfiability Problem Revisited L = { w w is a proposiuon in CNF and w is sausfiable} Algorithm: Guess an assignment of variables O(n) Check if this is a sausfying assignment O(n) Total running Ume on a non- determinisuc TM is O(n). L NTIME(n). NTIME(n) is the set of languages that can be decided by a non- determinis@c Turing machine in O(n).
51 The class NP We can define other classes (similarly to NTIME(n)) NTIME(n 2 ), NTIME(n 3 ), etc. The class NTIME(n 2 ) is the set of languages that can be decided by a non- determinisuc Turing machine in O(n 2 ). The class NP is the set of languages that can be decided by a non- determinisuc Turing machine in NTIME(n k ), for all k. NP = [ k NTIME(n k ) The sausfiability problem is an NP problem.
52 P vs NP ObservaUon: P NP Every language that can be decided by a determinisuc TM in polynomial Ume can be decided by a nondeterminisuc TM in polynomial Ume. Every determinisuc TM is a nondeterminisuc TM.
53 P vs NP Is P = NP? We don t know. Does the sausfiability problem have a polynomial Ume determinisuc algorithm? We don t know the answer.
54 NP Completeness Informally, NP complete problems are the hardest problems in the class NP. If we prove that an NP complete problem has a determinisuc polynomial Ume algorithm, then we will know that P = NP. If we prove that an NP complete problem does not have a determinisuc polynomial Ume algorithm, then we will know that P NP.
55 Cook s Theorem Theorem: The sausfiability problem is NP- complete.
56 The Traveling Salesman Problem Suppose a salesman must visit all 6 ciues. The distances between the ciues are given. What is the shortest tour she can take that visits every city exactly once and then returns to the starung city?
57 The Traveling Salesman Problem Theorem: The traveling salesman problem is NP complete.