# Finish K-Complexity, Start Time Complexity

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 6.045 Finish K-Complexity, Start Time Complexity 1

2 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string <M,w> such that M(w) halts with only x on its tape. Definition: The Kolmogorov complexity of x, denoted as K(x), is d(x). 2

3 There Exist Incompressible Strings Theorem: For all n, there is an x ϵ {0,1} n such that K(x) n There are incompressible strings of every length Proof: (Number of binary strings of length n) = 2 n but (Number of descriptions of length < n) (Number of binary strings of length < n) = n-1 = 2 n 1 Therefore, there is at least one n-bit string x that does not have a description of length < n. 3

4 Random Strings Are Incompressible! Theorem: For all n and c 1, Pr x ϵ {0,1} n[ K(x) n-c ] 1 1/2 c Most strings are highly incompressible Proof: (Number of binary strings of length n) = 2 n but (Number of descriptions of length < n-c) (Number of binary strings of length < n-c) = 2 n-c 1 Hence the probability that a random x satisfies K(x) < n-c is at most (2 n-c 1)/2 n < 1/2 c. 4

5 Kolmogorov Complexity: Try it! Give short algorithms for generating the strings:

6 Kolmogorov Complexity: Try it! Give short algorithms for generating the strings:

7 Kolmogorov Complexity: Try it! Give short algorithms for generating the strings:

8 Kolmogorov Complexity: Try it! Give short algorithms for generating the strings: This seems hard to determine in general. Why? 8

9 Computing Compressibility? Can an algorithm perform optimal compression? Can algorithms tell us if a given string is compressible? COMPRESS = { (x,c) K(x) c} Theorem: COMPRESS is undecidable! Idea: If decidable, we could design an algorithm that prints the shortest incompressible string of length n But such a string could then be succinctly described, by providing the algorithm code and n in binary! Berry Paradox: The smallest integer that cannot be defined in less than thirteen words. 9

10 Computing Compressibility? COMPRESS = {(x,c) K(x) c} Theorem: COMPRESS is undecidable! Proof: Suppose it is decidable. Consider the TM: M = On input x ϵ {0,1}*, let N = 2 x. For all y ϵ {0,1}* in lexicographical order, If (y, N) COMPRESS then print y and halt. M(x) prints the lex. first string y with K(y ) > 2 x. <M,x> is a description of y, <M,x> (2 M +1) + x So 2 x < K(y ) c + x. CONTRADICTION for large x! 10

11 Proving Theorems With K-Complexity Theorem: L = {x x x 0, 1 } is not regular. Proof: Suppose L is recognized by a DFA D. Let n 0 and choose an x 0, 1 n such that K x n Let q x be the state of D reached after reading in x Define a TM M(D, q, n): Find a path P in DFA D of length n that starts from state q and ends in a final state. Print the n-bit string along path P, and halt. Claim: The pair <M,(D, q x, n)> is a description of x! So n K x <M,(D, q x, n)> c + log n CONTRADICTION for large n! 11

12 Computational Complexity Theory 12

13 Computational Complexity Theory What can and can t be computed with limited resources on computation, such as time, space, and so on Captures many of the significant issues in practical problem solving The field is rich with important open questions that no one has any idea how to begin answering! We ll start with: Time complexity 13

14 Very Quick Review of Big-O Let f, g : N N. We say that f(n) O(g(n)) if there are c, n 0 N so that for every integer n n 0 f(n) c g(n) We say g(n) is an upper bound on f(n) if f(n) O(g(n)) 5n 3 + 2n n + 6 O(n 3 ) If c = 6 and n 0 = 10, then 5n 3 + 2n n + 6 cn 3 14

15 2n n O(n 4.1 ) 3n log 2 n + 5n log 2 log 2 n O(n log 2 n) n log 10 n 78 O(n log 10 n) log 10 n = log 2 n / log 2 10 O(n log 2 n) O(n log 10 n) O(n log n) Big-O isolates the dominant term of a function 15

16 A Simpler Big-O Definition Let f, g : N N We say f(n) O(g(n)) if there is a c N so that for all n N, f(n) c g(n) + c 16

17 Measuring Time Complexity of a TM We measure time complexity by counting the steps taken for a Turing machine to halt on an input Example: Let A = { 0 k 1 k k 0 } Here s a TM for A. On input x of length n: 1. Scan across the tape and reject if x is not of the form 0 i 1 j 2. Repeat the following if both 0s and 1s remain on the tape: Scan across the tape, crossing off a single 0 and a single 1 3. If 0s remain after all 1s have been crossed off, or vice-versa, reject. Otherwise accept. O(n) O(n 2 ) O(n) 17

18 Let M be a TM that halts on all inputs. (We will only consider decidable languages now!) Definition: The running time or time complexity of M is the function T : N N such that T(n) = maximum number of steps taken by M over all inputs of length n 18

19 Time-Bounded Complexity Classes Definition: TIME(t(n)) = { L there is a Turing machine M with time complexity O(t(n)) so that L = L(M) } = { L L is a language decided by a Turing machine with c t(n) + c running time } We showed: A = { 0 k 1 k k 0 } TIME(n 2 ) Is there a faster Turing machine? 19

20 A = { 0 k 1 k k 0 } TIME(n log n) M(w) := If w is not of the form 0*1*, reject. Repeat until all bits of w are crossed out: If (parity of 0 s) (parity of 1 s), reject. Cross out every other 0. Cross out every other 1. Once all bits are crossed out, accept x0x0x0x0x0x0xx1x1x1x1x1x1x xxx0xxx0xxx0xxxx1xxx1xxx1x xxxxxxx0xxxxxxxxxxxx1xxxxx xxxxxxxxxxxxxxxxxxxxxxxxxx 20

21 A = { 0 k 1 k k 0 } TIME(n log n) M(w) := If w is not of the form 0*1*, reject. Repeat until all bits of w are crossed out: If (parity of 0 s) (parity of 1 s), reject. Cross out every other 0. Cross out every other 1. Once all bits are crossed out, accept. For a fixed w = 0 k 1 k : Let zero i be number of 0s left in w, after iteration i Let ones i be number of 1s left in w, after iteration i Start with zero 0 = k, ones 0 = k Key Observation: zero i+1 = floor(zero i /2), ones i+1 = floor(ones i /2) Number of iterations O log n 21

22 It can be proved that there is no (one-tape) Turing Machine that can decide A in less than O(n log n) time! Let f(n) = O n log n α n Puzzle: where α n is unbounded. Prove: TIME(f(n)) contains only regular languages(!) For example, TIME(n log log n) contains only regular languages! 22

23 In particular, define the class REGULAR = {L L is a regular language}. Proposition: REGULAR TIME(n) Assuming the Puzzle is true, then for every f(n) = O n log n α n unbounded, where α n is TIME(f(n)) = TIME(n) A collapse of complexity classes! 23

24 Two Tapes Can Be More Efficient Theorem: A = { 0 k 1 k k 0 } can be decided in O(n) time with a two-tape TM. Proof Idea: Sweep over all 0s, copy them over on the second tape. Sweep over all 1s. For each 1, cross off a 0 from the second tape. 24

25 Different models of computation can yield different running times for the same language! Let s revisit some of the key concepts from computability theory 25

26 Theorem: Let t : N N satisfy t(n) n, for all n. Then every t(n) time multi-tape TM has an equivalent O(t(n) 2 ) time one-tape TM Our simulation of multitape TMs by one-tape TMs achieves this! Corollary: Suppose language A can be decided by a multi-tape TM in p(n) time, for some polynomial p. Then A can also be decided by a one-tape TM in q(n) time, for some polynomial q. 26

27 Theorem: For every t(n) time multi-tape TM, there is an equivalent O(t(n) 2 ) time one-tape TM FINITE STATE CONTROL FINITE STATE CONTROL # # # 27

28 Theorem: For every t(n) time multi-tape TM, there is an equivalent O(t(n) 2 ) time one-tape TM FINITE STATE CONTROL FINITE STATE CONTROL # # # 28

29 Theorem: For every t(n) time multi-tape TM, there is an equivalent O(t(n) 2 ) time one-tape TM FINITE STATE CONTROL FINITE STATE CONTROL # # # 29

30 Theorem: For every t(n) time multi-tape TM, there is an equivalent O(t(n) 2 ) time one-tape TM t(n) time FINITE STATE CONTROL O(t(n) 2 ) FINITE STATE CONTROL # # # 30

31 An Efficient Universal TM Theorem: There is a (one-tape) Turing machine U which takes as input: - the code of an arbitrary TM M - an input string w - and a string of t 1s, t > w such that U(M, w, 1 t ) halts in O( M 2 t 2 ) steps and U accepts (M, w, 1 t ) M accepts w in t steps The Universal TM with a Clock Idea: Make a multi-tape TM U that does the above, and runs in O( M t) steps 31

32 The Time Hierarchy Theorem Intuition: If you get more time to compute, then you can solve strictly more problems. Theorem: For all reasonable f, g : N! N where for all n, g(n) > n 2 f(n) 2, TIME(f(n)) TIME(g(n)) Proof Idea: Diagonalization with a clock Make a TM N that on input M, simulates the TM M on input M for f( M ) steps, then flips the answer. We will show L(N) cannot have time complexity f(n) 32

33 The Time Hierarchy Theorem Theorem: For reasonable f, g where g(n) > n 2 f(n) 2, TIME(f(n)) TIME(g(n)) Proof Sketch: Define a TM N as follows: N(M) = Compute t = f( M ) Run U(M, M, 1 t ) and output the opposite answer. Claim: L(N) does not have time complexity f(n). Proof: Assume N runs in f(n) time, and L(N ) = L(N). By assumption, N (N ) runs in f( N ) time and outputs the opposite answer of U(N, N, 1 f( N ) ) So N (N ) accepts U(N, N, 1 f( N ) ) rejects N (N ) rejects in f( N ) steps [U is universal] This is a contradiction! 33

34 The Time Hierarchy Theorem Theorem: For reasonable f, g where g(n) > n 2 f(n) 2, TIME(f(n)) TIME(g(n)) Proof Sketch: Define a TM N as follows: N(M) = Compute t = f( M ) Run U(M, M, 1 t ) and output the opposite answer. So, L(N) does not have time complexity f(n). For what functions g(n) will N run in O(g(n)) time? 1. Compute t = f( M ) in O(g( M )) time [ reasonable ] 2. Simulate U(M, M, 1 t ) in O(g( M )) time Recall: U(M, w, 1 t ) halts in O( M 2 t 2 ) steps Set g(n) so that g( M ) > M 2 f( M ) 2 for all n. QED Remark: Time hierarchy also holds for multitape TMs! 34

35 A Better Time Hierarchy Theorem Theorem: For reasonable f, g where g(n) > f(n) log 2 f(n), TIME(f(n)) TIME(g(n)) Corollary: TIME(n) TIME(n 2 ) TIME(n 3 ) There is an infinite hierarchy of increasingly more time-consuming problems Question: Are there important everyday problems that are high up in this time hierarchy? A natural problem that needs exactly n 10 time? THIS IS AN OPEN QUESTION! 37

36 k N P = TIME(n k ) Polynomial Time The analogue of decidability 38

37 The EXTENDED Church-Turing Thesis Everyone s Intuitive Notion of Efficient Algorithms = Polynomial-Time Turing Machines A controversial thesis! Potential counterexamples include n 100 time algorithms, randomized algorithms, quantum algorithms 39

38 Nondeterminism and NP The analogue of recognizability 40

39 Nondeterministic Turing Machines are just like standard TMs, except: 1. The machine may proceed according to several possible transitions (like an NFA) 2. The machine accepts an input string if there exists an accepting computation history for the machine on the string 41

40 read write move 0 0, R 0 0, R, R q accept, R 0 0, R q reject 42

41 Definition: A nondeterministic TM is a 7-tuple T = (Q, Σ, Γ,, q 0, q accept, q reject ), where: Q is a finite set of states Σ is the input alphabet, where Σ Γ is the tape alphabet, where Γ and Σ Γ : Q Γ 2 (Q Γ {L,R}) q 0 Q is the start state q accept Q is the accept state q reject Q is the reject state, and q reject q accept 43

42 Defining Acceptance for NTMs Let N be a nondeterministic Turing machine An accepting computation history for N on w is a sequence of configurations C 0,C 1,,C t where 1. C 0 is the start configuration q 0 w, 2. C t is an accepting configuration, 3. Each configuration C i yields C i+1 Def. N(w) accepts in t time Such a history exists N has time complexity T(n) if for all n, for all inputs of length n and for all histories, N halts in T(n) time 44

43 Definition: NTIME(t(n)) = { L L is decided by a O(t(n)) time nondeterministic Turing machine } Note: TIME(t(n)) NTIME(t(n)) Is TIME(t(n)) = NTIME(t(n)) for all t(n)? THIS IS AN OPEN QUESTION! What can be done in short NTIME that cannot be done in short TIME? 45

44 Boolean Formulas logical parentheses operations A satisfying assignment is a setting of the variables that makes the formula true = ( x y) z x = 1, y = 1, z = 1 is a satisfying assignment for Boolean variables (0 or 1) (x y) (z x)

45 A Boolean formula is satisfiable if there exists a true/false setting to the variables that makes the formula true YES NO a b c d (x y) x SAT = { is a satisfiable Boolean formula } 47

46 A 3cnf-formula has the form: (x 1 x 2 x 3 ) (x 4 x 2 x 5 ) (x 3 x 2 x 1 ) literals clauses Ex: (x 1 x 2 x 1 ) (x 3 x 1 ) (x 3 x 2 x 1 ) (x 1 x 2 x 3 ) ( x 4 x 2 x 1 ) (x 3 x 1 x 1 ) (x 1 x 2 x 3 ) (x 3 x 2 x 1 ) 3SAT = { is a satisfiable 3cnf-formula } 48

47 3SAT = { is a satisfiable 3cnf-formula } Theorem: 3SAT NTIME(n 2 ) Proof Idea: On input : 1. Check if the formula is in 3cnf 2. For each variable v in, nondeterministically substitute either 0 or 1 in place of v ( x y x ) ( 0 y 0 ) ( 1 y 1 ) ( ) ( ) 3. Evaluate the formula and accept iff is true 49

48 NP = NTIME(n k ) k N Nondeterministic Polynomial Time 50

49 Theorem: L NP There is a constant k and polynomial-time TM V such that L = { x y ϵ Σ* [ y x k and V(x,y) accepts ] } Proof: (1) If L = { x y y x k and V(x,y) accepts } then L NP Given the poly-time TM V, our NP machine for L is: N(x): Nondeterministically guess y. Run V(x,y) (2) If L NP then L = { x y y x k and V(x,y) accepts } Let N be a nondet. poly-time TM that decides L. Define a TM V(x,y) which accepts y encodes an accepting computation history of N on x 51

50 Moral: A language L is in NP if and only if there are polynomial-length proofs for membership in L 3SAT = { y such that is in 3cnf and y is a satisfying assignment to } SAT = { y such that is a Boolean formula and y is a satisfying assignment to } 52

51 NP = Problems with the property that, once you have the answer, it is easy to verify the answer SAT is in NP because a satisfying assignment is a polynomial-length proof that a formula is satisfiable When ϵ SAT, I can prove that fact to you with a short proof you can quickly verify 53

52 The Hamiltonian Path Problem b f e g a d i c h A Hamiltonian path traverses through each node exactly once 55

53 Assume a reasonable encoding of graphs (example: the adjacency matrix is reasonable) HAMPATH = { (G,s,t) G is a directed graph with a Hamiltonian path from s to t } Theorem: HAMPATH NP A Hamiltonian path P in G from s to t is a proof that (G,s,t) is in HAMPATH Given P (as a permutation on the nodes) can easily check that it is a path through all nodes exactly once 56

54 The k-clique Problem a d f c b e g k-clique = complete subgraph on k nodes 57

55 CLIQUE = { (G,k) G is an undirected graph with a k-clique } Theorem: CLIQUE NP A k-clique in G is a proof that (G, k) is in CLIQUE Given a subset S of k nodes from G, we can efficiently check that all possible edges are present between the nodes in S 58

56 A language is in NP if and only if there are polynomial-length proofs for membership in the language P = the problems that can be efficiently solved NP = the problems where proposed solutions can be efficiently verified Is P = NP? can problem solving be automated? 59

57 \$\$\$ P = NP? \$\$\$ 60

58 If P = NP Mathematicians/creators may be out of a job Cryptography as we know it may be impossible In principle, every aspect of daily life could be efficiently and globally optimized life as we know it would be different Conjecture: P NP 61

59 Next Episode: NP-Complete Problems 62

### Lecture 16: Time Complexity and P vs NP

6.045 Lecture 16: Time Complexity and P vs NP 1 Time-Bounded Complexity Classes Definition: TIME(t(n)) = { L there is a Turing machine M with time complexity O(t(n)) so that L = L(M) } = { L L is a language

### TIME COMPLEXITY AND POLYNOMIAL TIME; NON DETERMINISTIC TURING MACHINES AND NP. THURSDAY Mar 20

TIME COMPLEXITY AND POLYNOMIAL TIME; NON DETERMINISTIC TURING MACHINES AND NP THURSDAY Mar 20 COMPLEXITY THEORY Studies what can and can t be computed under limited resources such as time, space, etc Today:

### CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression

CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression Rosencrantz & Guildenstern Are Dead (Tom Stoppard) Rigged Lottery? And the winning numbers are: 1, 2, 3, 4, 5, 6 But is

### Lecture 13: Foundations of Math and Kolmogorov Complexity

6.045 Lecture 13: Foundations of Math and Kolmogorov Complexity 1 Self-Reference and the Recursion Theorem 2 Lemma: There is a computable function q : Σ* Σ* such that for every string w, q(w) is the description

### CS5371 Theory of Computation. Lecture 19: Complexity IV (More on NP, NP-Complete)

CS5371 Theory of Computation Lecture 19: Complexity IV (More on NP, NP-Complete) Objectives More discussion on the class NP Cook-Levin Theorem The Class NP revisited Recall that NP is the class of language

### FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY KOLMOGOROV-CHAITIN (descriptive) COMPLEXITY TUESDAY, MAR 18 CAN WE QUANTIFY HOW MUCH INFORMATION IS IN A STRING? A = 01010101010101010101010101010101

### Advanced Topics in Theoretical Computer Science

Advanced Topics in Theoretical Computer Science Part 5: Complexity (Part II) 30.01.2014 Viorica Sofronie-Stokkermans Universität Koblenz-Landau e-mail: sofronie@uni-koblenz.de 1 Contents Recall: Turing

### Lecture 17: Cook-Levin Theorem, NP-Complete Problems

6.045 Lecture 17: Cook-Levin Theorem, NP-Complete Problems 1 Is SAT solvable in O(n) time on a multitape TM? Logic circuits of 6n gates for SAT? If yes, then not only is P=NP, but there would be a dream

### Decidability and Undecidability

Decidability and Undecidability Major Ideas from Last Time Every TM can be converted into a string representation of itself. The encoding of M is denoted M. The universal Turing machine U TM accepts an

### Computability Theory. CS215, Lecture 6,

Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David

### Theory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death

Theory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death Robbie Hott www.cs.virginia.edu/~jh2jf Department of Computer Science University of Virginia

### 1 Computational Problems

Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

### Definition: conp = { L L NP } What does a conp computation look like?

Space Complexity 28 Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode: Guess string y of x k length and the machine accepts

### The Polynomial Hierarchy

The Polynomial Hierarchy Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Motivation..synthesizing circuits is exceedingly difficulty. It is even

### Lecture Notes: The Halting Problem; Reductions

Lecture Notes: The Halting Problem; Reductions COMS W3261 Columbia University 20 Mar 2012 1 Review Key point. Turing machines can be encoded as strings, and other Turing machines can read those strings

### CS154, Lecture 10: Rice s Theorem, Oracle Machines

CS154, Lecture 10: Rice s Theorem, Oracle Machines Moral: Analyzing Programs is Really, Really Hard But can we more easily tell when some program analysis problem is undecidable? Problem 1 Undecidable

### 6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch

6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch Today: More Complexity Theory Polynomial-time reducibility, NP-completeness, and the Satisfiability (SAT) problem Topics: Introduction

### satisfiability (sat) Satisfiability unsatisfiability (unsat or sat complement) and validity Any Expression φ Can Be Converted into CNFs and DNFs

Any Expression φ Can Be Converted into CNFs and DNFs φ = x j : This is trivially true. φ = φ 1 and a CNF is sought: Turn φ 1 into a DNF and apply de Morgan s laws to make a CNF for φ. φ = φ 1 and a DNF

### Theory of Computation

Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation 1 / 33 Lecture 20: Overview Incompressible strings Minimal Length Descriptions Descriptive Complexity Dr. Sarmad Abbasi

### Turing Machines Part III

Turing Machines Part III Announcements Problem Set 6 due now. Problem Set 7 out, due Monday, March 4. Play around with Turing machines, their powers, and their limits. Some problems require Wednesday's

### Ma/CS 117c Handout # 5 P vs. NP

Ma/CS 117c Handout # 5 P vs. NP We consider the possible relationships among the classes P, NP, and co-np. First we consider properties of the class of NP-complete problems, as opposed to those which are

### Computability Theory

Computability Theory Cristian S. Calude May 2012 Computability Theory 1 / 1 Bibliography M. Sipser. Introduction to the Theory of Computation, PWS 1997. (textbook) Computability Theory 2 / 1 Supplementary

### DRAFT. Diagonalization. Chapter 4

Chapter 4 Diagonalization..the relativized P =?NP question has a positive answer for some oracles and a negative answer for other oracles. We feel that this is further evidence of the difficulty of the

### CSCE 551: Chin-Tser Huang. University of South Carolina

CSCE 551: Theory of Computation Chin-Tser Huang huangct@cse.sc.edu University of South Carolina Church-Turing Thesis The definition of the algorithm came in the 1936 papers of Alonzo Church h and Alan

### Time Complexity. Definition. Let t : n n be a function. NTIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM}

Time Complexity Definition Let t : n n be a function. TIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM} NTIME(t(n)) = {L L is a language decidable by a O(t(n)) non-deterministic

### De Morgan s a Laws. De Morgan s laws say that. (φ 1 φ 2 ) = φ 1 φ 2, (φ 1 φ 2 ) = φ 1 φ 2.

De Morgan s a Laws De Morgan s laws say that (φ 1 φ 2 ) = φ 1 φ 2, (φ 1 φ 2 ) = φ 1 φ 2. Here is a proof for the first law: φ 1 φ 2 (φ 1 φ 2 ) φ 1 φ 2 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0 0 a Augustus DeMorgan

### Lecture 3: Reductions and Completeness

CS 710: Complexity Theory 9/13/2011 Lecture 3: Reductions and Completeness Instructor: Dieter van Melkebeek Scribe: Brian Nixon Last lecture we introduced the notion of a universal Turing machine for deterministic

### Computer Sciences Department

Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y

### 6.045 Final Exam Solutions

6.045J/18.400J: Automata, Computability and Complexity Prof. Nancy Lynch, Nati Srebro 6.045 Final Exam Solutions May 18, 2004 Susan Hohenberger Name: Please write your name on each page. This exam is open

### P, NP, NP-Complete, and NPhard

P, NP, NP-Complete, and NPhard Problems Zhenjiang Li 21/09/2011 Outline Algorithm time complicity P and NP problems NP-Complete and NP-Hard problems Algorithm time complicity Outline What is this course

### ENEE 459E/CMSC 498R In-class exercise February 10, 2015

ENEE 459E/CMSC 498R In-class exercise February 10, 2015 In this in-class exercise, we will explore what it means for a problem to be intractable (i.e. it cannot be solved by an efficient algorithm). There

### Lecture 18: More NP-Complete Problems

6.045 Lecture 18: More NP-Complete Problems 1 The Clique Problem a d f c b e g Given a graph G and positive k, does G contain a complete subgraph on k nodes? CLIQUE = { (G,k) G is an undirected graph with

### Chapter 1 - Time and Space Complexity. deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE

Chapter 1 - Time and Space Complexity deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE 1 / 41 Deterministic Turing machines Definition 1.1 A (deterministic

### FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY Chomsky Normal Form and TURING MACHINES TUESDAY Feb 4 CHOMSKY NORMAL FORM A context-free grammar is in Chomsky normal form if every rule is of the form:

CMPSCI611: NP Completeness Lecture 17 Essential facts about NP-completeness: Any NP-complete problem can be solved by a simple, but exponentially slow algorithm. We don t have polynomial-time solutions

### Intro to Theory of Computation

Intro to Theory of Computation LECTURE 24 Last time Relationship between models: deterministic/nondeterministic Class P Today Class NP Sofya Raskhodnikova Homework 9 due Homework 0 out 4/5/206 L24. I-clicker

### Computational Complexity: A Modern Approach. Draft of a book: Dated January 2007 Comments welcome!

i Computational Complexity: A Modern Approach Draft of a book: Dated January 2007 Comments welcome! Sanjeev Arora and Boaz Barak Princeton University complexitybook@gmail.com Not to be reproduced or distributed

### Introduction to Turing Machines. Reading: Chapters 8 & 9

Introduction to Turing Machines Reading: Chapters 8 & 9 1 Turing Machines (TM) Generalize the class of CFLs: Recursively Enumerable Languages Recursive Languages Context-Free Languages Regular Languages

### Lecture 22: PSPACE

6.045 Lecture 22: PSPACE 1 VOTE VOTE VOTE For your favorite course on automata and complexity Please complete the online subject evaluation for 6.045 2 Final Exam Information Who: You On What: Everything

### Theory of computation: initial remarks (Chapter 11)

Theory of computation: initial remarks (Chapter 11) For many purposes, computation is elegantly modeled with simple mathematical objects: Turing machines, finite automata, pushdown automata, and such.

### Computability and Complexity Theory

Discrete Math for Bioinformatics WS 09/10:, by A Bockmayr/K Reinert, January 27, 2010, 18:39 9001 Computability and Complexity Theory Computability and complexity Computability theory What problems can

### Decision Problems with TM s. Lecture 31: Halting Problem. Universe of discourse. Semi-decidable. Look at following sets: CSCI 81 Spring, 2012

Decision Problems with TM s Look at following sets: Lecture 31: Halting Problem CSCI 81 Spring, 2012 Kim Bruce A TM = { M,w M is a TM and w L(M)} H TM = { M,w M is a TM which halts on input w} TOTAL TM

### Topics in Complexity Theory

Topics in Complexity Theory Announcements Final exam this Friday from 12:15PM-3:15PM Please let us know immediately after lecture if you want to take the final at an alternate time and haven't yet told

### 1 Computational problems

80240233: Computational Complexity Lecture 1 ITCS, Tsinghua Univesity, Fall 2007 9 October 2007 Instructor: Andrej Bogdanov Notes by: Andrej Bogdanov The aim of computational complexity theory is to study

### Chomsky Normal Form and TURING MACHINES. TUESDAY Feb 4

Chomsky Normal Form and TURING MACHINES TUESDAY Feb 4 CHOMSKY NORMAL FORM A context-free grammar is in Chomsky normal form if every rule is of the form: A BC A a S ε B and C aren t start variables a is

### 6.840 Language Membership

6.840 Language Membership Michael Bernstein 1 Undecidable INP Practice final Use for A T M. Build a machine that asks EQ REX and then runs M on w. Query INP. If it s in P, accept. Note that the language

### 6.841/18.405J: Advanced Complexity Wednesday, February 12, Lecture Lecture 3

6.841/18.405J: Advanced Complexity Wednesday, February 12, 2003 Lecture Lecture 3 Instructor: Madhu Sudan Scribe: Bobby Kleinberg 1 The language MinDNF At the end of the last lecture, we introduced the

### Further discussion of Turing machines

Further discussion of Turing machines In this lecture we will discuss various aspects of decidable and Turing-recognizable languages that were not mentioned in previous lectures. In particular, we will

### 1 Deterministic Turing Machines

Time and Space Classes Exposition by William Gasarch 1 Deterministic Turing Machines Turing machines are a model of computation. It is believed that anything that can be computed can be computed by a Turing

### ,

Kolmogorov Complexity Carleton College, CS 254, Fall 2013, Prof. Joshua R. Davis based on Sipser, Introduction to the Theory of Computation 1. Introduction Kolmogorov complexity is a theory of lossless

### Time and space classes

Time and space classes Little Oh (o,

### Definition: Alternating time and space Game Semantics: State of machine determines who

CMPSCI 601: Recall From Last Time Lecture Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins

### CSE 105 Theory of Computation

CSE 105 Theory of Computation http://www.jflap.org/jflaptmp/ Professor Jeanne Ferrante 1 Undecidability Today s Agenda Review and More Problems A Non-TR Language Reminders and announcements: HW 7 (Last!!)

### Lecture 2: Tape reduction and Time Hierarchy

Computational Complexity Theory, Fall 2010 August 27 Lecture 2: Tape reduction and Time Hierarchy Lecturer: Peter Bro Miltersen Scribe: Andreas Hummelshøj Jakobsen Tape reduction Theorem 1 (k-tapes 2-tape

### Notes on Complexity Theory Last updated: October, Lecture 6

Notes on Complexity Theory Last updated: October, 2015 Lecture 6 Notes by Jonathan Katz, lightly edited by Dov Gordon 1 PSPACE and PSPACE-Completeness As in our previous study of N P, it is useful to identify

### The Complexity of Optimization Problems

The Complexity of Optimization Problems Summary Lecture 1 - Complexity of algorithms and problems - Complexity classes: P and NP - Reducibility - Karp reducibility - Turing reducibility Uniform and logarithmic

### Verifying whether One-Tape Turing Machines Run in Linear Time

Electronic Colloquium on Computational Complexity, Report No. 36 (2015) Verifying whether One-Tape Turing Machines Run in Linear Time David Gajser IMFM, Jadranska 19, 1000 Ljubljana, Slovenija david.gajser@fmf.uni-lj.si

### CpSc 421 Homework 9 Solution

CpSc 421 Homework 9 Solution Attempt any three of the six problems below. The homework is graded on a scale of 100 points, even though you can attempt fewer or more points than that. Your recorded grade

### A.Antonopoulos 18/1/2010

Class DP Basic orems 18/1/2010 Class DP Basic orems 1 Class DP 2 Basic orems Class DP Basic orems TSP Versions 1 TSP (D) 2 EXACT TSP 3 TSP COST 4 TSP (1) P (2) P (3) P (4) DP Class Class DP Basic orems

### Lecture 8. MINDNF = {(φ, k) φ is a CNF expression and DNF expression ψ s.t. ψ k and ψ is equivalent to φ}

6.841 Advanced Complexity Theory February 28, 2005 Lecture 8 Lecturer: Madhu Sudan Scribe: Arnab Bhattacharyya 1 A New Theme In the past few lectures, we have concentrated on non-uniform types of computation

### Definition: Alternating time and space Game Semantics: State of machine determines who

CMPSCI 601: Recall From Last Time Lecture 3 Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins

### MTAT Complexity Theory October 13th-14th, Lecture 6

MTAT.07.004 Complexity Theory October 13th-14th, 2011 Lecturer: Peeter Laud Lecture 6 Scribe(s): Riivo Talviste 1 Logarithmic memory Turing machines working in logarithmic space become interesting when

### Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar

IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar Turing Machine A Turing machine is an abstract representation of a computing device. It consists of a read/write

### Lecture 17. In this lecture, we will continue our discussion on randomization.

ITCS:CCT09 : Computational Complexity Theory May 11, 2009 Lecturer: Jayalal Sarma M.N. Lecture 17 Scribe: Hao Song In this lecture, we will continue our discussion on randomization. 1 BPP and the Polynomial

### Turing Machines Part II

Turing Machines Part II Problem Set Set Five Five due due in in the the box box up up front front using using a late late day. day. Hello Hello Condensed Slide Slide Readers! Readers! This This lecture

### An introduction to Parameterized Complexity Theory

An introduction to Parameterized Complexity Theory May 15, 2012 A motivation to parametrize Definitions Standard Complexity Theory does not distinguish between the distinct parts of the input. It categorizes

### Chapter 2 Algorithms and Computation

Chapter 2 Algorithms and Computation In this chapter, we first discuss the principles of algorithm and computation in general framework, common both in classical and quantum computers, then we go to the

### Turing machines and linear bounded automata

and linear bounded automata Informatics 2A: Lecture 29 John Longley School of Informatics University of Edinburgh jrl@inf.ed.ac.uk 25 November, 2011 1 / 13 1 The Chomsky hierarchy: summary 2 3 4 2 / 13

### Automata Theory CS Complexity Theory I: Polynomial Time

Automata Theory CS411-2015-17 Complexity Theory I: Polynomial Time David Galles Department of Computer Science University of San Francisco 17-0: Tractable vs. Intractable If a problem is recursive, then

### Homework 8. a b b a b a b. two-way, read/write

Homework 8 309 Homework 8 1. Describe a TM that accepts the set {a n n is a power of 2}. Your description should be at the level of the descriptions in Lecture 29 of the TM that accepts {ww w Σ } and the

### 6.045J/18.400J: Automata, Computability and Complexity Final Exam. There are two sheets of scratch paper at the end of this exam.

6.045J/18.400J: Automata, Computability and Complexity May 20, 2005 6.045 Final Exam Prof. Nancy Lynch Name: Please write your name on each page. This exam is open book, open notes. There are two sheets

### Theory of Computation Lecture Notes. Problems and Algorithms. Class Information

Theory of Computation Lecture Notes Prof. Yuh-Dauh Lyuu Dept. Computer Science & Information Engineering and Department of Finance National Taiwan University Problems and Algorithms c 2004 Prof. Yuh-Dauh

### The Quantum and Classical Complexity of Translationally Invariant Tiling and Hamiltonian Problems

The Quantum and Classical Complexity of Translationally Invariant Tiling and Hamiltonian Problems Daniel Gottesman Perimeter Institute for Theoretical Physics Waterloo, Ontario, Canada dgottesman@perimeterinstitute.ca

### 2 P vs. NP and Diagonalization

2 P vs NP and Diagonalization CS 6810 Theory of Computing, Fall 2012 Instructor: David Steurer (sc2392) Date: 08/28/2012 In this lecture, we cover the following topics: 1 3SAT is NP hard; 2 Time hierarchies;

### 1 Acceptance, Rejection, and I/O for Turing Machines

1 Acceptance, Rejection, and I/O for Turing Machines Definition 1.1 (Initial Configuration) If M = (K,Σ,δ,s,H) is a Turing machine and w (Σ {, }) then the initial configuration of M on input w is (s, w).

### NP Complete Problems. COMP 215 Lecture 20

NP Complete Problems COMP 215 Lecture 20 Complexity Theory Complexity theory is a research area unto itself. The central project is classifying problems as either tractable or intractable. Tractable Worst

### Complexity and NP-completeness

Lecture 17 Complexity and NP-completeness Supplemental reading in CLRS: Chapter 34 As an engineer or computer scientist, it is important not only to be able to solve problems, but also to know which problems

### Lecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009

CS 373: Theory of Computation Sariel Har-Peled and Madhusudan Parthasarathy Lecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009 This lecture covers Rice s theorem, as well as

### Undecidable Problems and Reducibility

University of Georgia Fall 2014 Reducibility We show a problem decidable/undecidable by reducing it to another problem. One type of reduction: mapping reduction. Definition Let A, B be languages over Σ.

### Polynomial-Time Reductions

Reductions 1 Polynomial-Time Reductions Classify Problems According to Computational Requirements Q. Which problems will we be able to solve in practice? A working definition. [von Neumann 1953, Godel

### DRAFT. Algebraic computation models. Chapter 14

Chapter 14 Algebraic computation models Somewhat rough We think of numerical algorithms root-finding, gaussian elimination etc. as operating over R or C, even though the underlying representation of the

### NP-Complete problems

NP-Complete problems NP-complete problems (NPC): A subset of NP. If any NP-complete problem can be solved in polynomial time, then every problem in NP has a polynomial time solution. NP-complete languages

### Computational Models Lecture 9, Spring 2009

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. p. 1 Computational Models Lecture 9, Spring 2009 Reducibility among languages Mapping reductions More undecidable

### 7.1 The Origin of Computer Science

CS125 Lecture 7 Fall 2016 7.1 The Origin of Computer Science Alan Mathison Turing (1912 1954) turing.jpg 170!201 pixels On Computable Numbers, with an Application to the Entscheidungsproblem 1936 1936:

### FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY REVIEW for MIDTERM 1 THURSDAY Feb 6 Midterm 1 will cover everything we have seen so far The PROBLEMS will be from Sipser, Chapters 1, 2, 3 It will be

### Regular Languages and Finite Automata

Regular Languages and Finite Automata Theorem: Every regular language is accepted by some finite automaton. Proof: We proceed by induction on the (length of/structure of) the description of the regular

### Turing Machines Part II

Turing Machines Part II Hello Hello Condensed Slide Slide Readers! Readers! This This lecture lecture is is almost almost entirely entirely animations that that show show how how each each Turing Turing

### Complexity Theory. Jörg Kreiker. Summer term Chair for Theoretical Computer Science Prof. Esparza TU München

Complexity Theory Jörg Kreiker Chair for Theoretical Computer Science Prof. Esparza TU München Summer term 2010 Lecture 4 NP-completeness Recap: relations between classes EXP PSPACE = NPSPACE conp NP conp

### CSCI 1590 Intro to Computational Complexity

CSCI 1590 Intro to Computational Complexity Space Complexity John E. Savage Brown University February 11, 2008 John E. Savage (Brown University) CSCI 1590 Intro to Computational Complexity February 11,

### CISC 876: Kolmogorov Complexity

March 27, 2007 Outline 1 Introduction 2 Definition Incompressibility and Randomness 3 Prefix Complexity Resource-Bounded K-Complexity 4 Incompressibility Method Gödel s Incompleteness Theorem 5 Outline

### Boolean circuits. Lecture Definitions

Lecture 20 Boolean circuits In this lecture we will discuss the Boolean circuit model of computation and its connection to the Turing machine model. Although the Boolean circuit model is fundamentally

### Mm7 Intro to distributed computing (jmp) Mm8 Backtracking, 2-player games, genetic algorithms (hps) Mm9 Complex Problems in Network Planning (JMP)

Algorithms and Architectures II H-P Schwefel, Jens M. Pedersen Mm6 Advanced Graph Algorithms (hps) Mm7 Intro to distributed computing (jmp) Mm8 Backtracking, 2-player games, genetic algorithms (hps) Mm9

### On the Expressive Power of Monadic Least Fixed Point Logic

On the Expressive Power of Monadic Least Fixed Point Logic Nicole Schweikardt Institut für Informatik, Humboldt-Universität Berlin, Unter den Linden 6, D-10099 Berlin, Germany, Email: schweika@informatik.hu-berlin.de,

### Turing Machines (TM) The Turing machine is the ultimate model of computation.

TURING MACHINES Turing Machines (TM) The Turing machine is the ultimate model of computation. Alan Turing (92 954), British mathematician/engineer and one of the most influential scientists of the last

### On Rice s theorem. Hans Hüttel. October 2001

On Rice s theorem Hans Hüttel October 2001 We have seen that there exist languages that are Turing-acceptable but not Turing-decidable. An important example of such a language was the language of the Halting

### 198:538 Lecture Given on February 23rd, 1998

198:538 Lecture Given on February 23rd, 1998 Lecturer: Professor Eric Allender Scribe: Sunny Daniels November 25, 1998 1 A Result From theorem 5 of the lecture on 16th February 1998, together with the

### Introduction to Computability

Chapter 2 Introduction to Computability This subject is primarily concerned with the limitations of computing. As one of the highlights of this study, we will learn several specific problems that computers