# Computational Complexity: A Modern Approach. Draft of a book: Dated January 2007 Comments welcome!

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 i Computational Complexity: A Modern Approach Draft of a book: Dated January 2007 Comments welcome! Sanjeev Arora and Boaz Barak Princeton University Not to be reproduced or distributed without the authors permission This is an Internet draft. Some chapters are more finished than others. References and attributions are very preliminary and we apologize in advance for any omissions (but hope you will nevertheless point them out to us). Please send us bugs, typos, missing references or general comments to Thank You!!

2 ii

3 Chapter 3 Diagonalization..the relativized P =?NP question has a positive answer for some oracles and a negative answer for other oracles. We feel that this is further evidence of the difficulty of the P =?NP question. Baker, Gill, Solovay. [?] One basic goal in complexity theory is to separate interesting complexity classes. To separate two complexity classes we need to exhibit a machine in one class that gives a different answer on some input from every machine in the other class. This chapter describes diagonalization, essentially the only general technique known for constructing such a machine. We have already seen diagonalization in Section 1.4, where it was used to show the existence of uncomputable functions. In this chapter, we first use diagonalization to prove hierarchy theorems, according to which giving Turing machines more computational resources (such as time, space, and non-determinism) allows them to solve a strictly larger number of problems. We will also use it to show that if P NP then there exist problems that are neither in P nor NP-complete. Though diagonalization led to some of these early successes of complexity theory, researchers realized in the 1970s that diagonalization alone may not resolve P versus NP and other interesting questions; see Section 3.5. Interestingly, the limits of diagonalization are proved using diagonalization. This last result caused diagonalization to go out of favor for many years. But some recent results (see Section 16.3 for an example) use diagonalization as a key component. Thus future complexity theorists should master this simple idea before going on to anything fancier! Machines as strings and the universal TM. The one common tool used in all diagonalization proofs is the representation of TMs by strings, such that given a string x a universal TM can simulate the machine M x represented by x with a small (i.e. at most logarithmic) overhead, see Theorems 1.13,?? and??. Recall that we assume that every string x represents some machine and every machine is represented by infinitely many strings. For i N, we will also use the notation M i for the machine represented by the string that is the binary expansion of the number i (ignoring the leading 1). p3.1 (63) Complexity Theory: A Modern Approach Sanjeev Arora and Boaz Barak. References and attributions are still incomplete.

4 p3.2 (64) 3.1. TIME HIERARCHY THEOREM 3.1 Time Hierarchy Theorem The Time Hierarchy Theorem shows that allowing Turing Machines more computation time strictly increases the class of languages that they can decide. Recall that a function f : N N is a timeconstructible function if there is a Turing machine that, given the input 1 n, writes down 1 f(n) on its tape in O(f(n)) time. Usual functions like n log n or n 2 satisfy this property, and we will restrict attention to running times that are time-constructible. Theorem 3.1 If f, g are time-constructible functions satisfying f(n) log f(n) = o(g(n)), then DTIME(f(n)) DTIME(g(n)) (1) Proof: To showcase the essential idea of the proof of Theorem 3.1, we prove the simpler statement DTIME(n) DTIME(n 1.5 ). Consider the following Turing Machine D: On input x, run for x 1.4 steps the Universal TM U of Theorem 1.13 to simulate the execution of M x on x. If M x outputs an answer in this time, namely, M x (x) {0, 1} then output the opposite answer (i.e., output 1 M x (x)). Else output 0. Here M x is the machine represented by the string x. By definition, D halts within n 1.4 steps and hence the language L decided by D is in DTIME(n 1.5 ). We claim that L DTIME(n). For contradiction s sake assume that some TM M decides L but runs in time cn on inputs of size n. Then every x {0, 1}, M(x) = D(x). The time to simulate M by the universal Turing machine U on every input x is at most c c x log x for some constant c (depending on the alphabet size and number of tapes and states of M, but independent of x ). There exists a number n 0 such that for every n n 0, n 1.4 > c cn log n. Let x be a string representing the machine M of length at least n 0 (there exists such a string since M is represented by infinitely many strings). Then, D(x) will obtain the output M(x) within x 1.4 steps, but by definition of D, we have D(x) = 1 M(x) M(x). Thus we have derived a contradiction. 3.2 Space Hierarchy Theorem The space hierarchy theorem is completely analogous to the time hierarchy theorem. One restricts attention to space-constructible functions, which are functions f : N N for which there is a machine that, given any n-bit input, constructs f(n) in space O(f(n)). The proof of the next theorem is completely analogous to that of Theorem 3.1. (The theorem does not have the log f(n) factor because the universal machine for space-bounded computation incurs only a constant factor overhead in space; see Theorem??.) Theorem 3.2 If f, g are space-constructible functions satisfying f(n) = o(g(n)), then SPACE(f(n)) SPACE(g(n)) (2)

5 3.3. NONDETERMINISTIC TIME HIERARCHY THEOREM p3.3 (65) 3.3 Nondeterministic Time Hierarchy Theorem The following is the hierarchy theorem for non-deterministic Turing machines. Theorem 3.3 If f, g are time constructible functions satisfying f(n + 1) = o(g(n)), then NTIME(f(n)) NTIME(g(n)) (3) Proof: Again, we just showcase the main idea by proving NTIME(n) NTIME(n 1.5 ). The technique from the previous section does not directly apply, since it has to determine the answer of a TM in order to flip it. To determine the answer of a nondeterminisitic that runs in O(n) time, we may need to examine as many as 2 Ω(n) possible strings of non-deterministic choices. So it is unclear that how the diagonalizer machine can determine in O(n 1.5 ) (or even O(n 100 )) time how to flip this answer. Instead we introduce a technique called lazy diagonalization, which is only guaranteed to flip the answer on some input in a fairly large range. For every i N we denote by M i the non-deterministic TM represented by i s binary expansion according to the universal NDTM N U (see Theorem??). We define the function f : N N as follows: f(1) = 2 and f(i + 1) = 2 f(i)1.2. Note that given n, we can can easily find in O(n 1.5 ) time the number i such that n is sandwiched between f(i) and f(i + 1). Our diagonalizing machine D will try to flip the answer of M i on some input in the set {1 n : f(i) < n f(i + 1)}. It is defined as follows: On input x, if x 1, reject. If x = 1 n, then compute i such that f(i) < n f(i + 1) and 1. If f(i) < n < f(i + 1) then simulate M i on input 1 n+1 using nondeterminism in n 1.1 time and output its answer. (If the simulation takes more than that then halt and accept.) 2. If n = f(i + 1), accept 1 n iff M i rejects 1 f(i)+1 in (f(i) + 1) 1.5 time. Note that Part 2 requires going through all possible exp((f(i) + 1) 1.1 ) branches of M i on input 1 f(i)+1, but that is fine since the input size f(i + 1) is 2 f(i)1.2. We conclude that NDTM D runs in O(n 1.5 ) time. Let L be the language decided by D. We claim that L NTIME(n). Indeed, suppose for the sake of contradiction that L is decided by an NDTM M running in cn steps (for some constant c). Since each NDTM is represented by infinitely many strings, we can find i large enough such that M = M i and on inputs of length n f(i), M i can be simulated in less than n 1.1 steps. Thus the two steps in the description of D ensure respectively that If f(i) < n < f(i + 1), then D(1 n ) = M i (1 n+1 ) (4) D(1 f(i+1) ) M i (1 f(i)+1 ) (5) see Figure 3.1. By our assumption M i and D agree on all inputs 1 n for n (f(i), f(i + 1)]. Together with (4), this implies that D(1 f(i+1) ) = M i (1 f(i)+1 ), contradicting(5).

6 p3.4 (66) 3.4. LADNER S THEOREM: EXISTENCE OF NP-INTERMEDIATE PROBLEMS. D(1 f(i)+1 ) D(1 f(i)+2 )... D(1 f(i+1) ) = = = = = = = = = M i (1 f(i)+1 ) M i (1 f(i)+2 )... M i (1 f(i+1) ) = Figure 3.1: The values of D and M i on inputs 1 n for n (f(i), f(i + 1)]. Full lines denote equality by the design of D, dashed lines denote equality by the assumption that D(x) = M i(x) for every x, and the dashed arrow denotes inequality by the design of D. Note that together all these relations lead to contradiction. 3.4 Ladner s Theorem: Existence of NP-intermediate problems. One of the striking aspects of NP-completeness is the surprisingly large number of NP-problems including some that were studied for many decades that turned out to be NP-complete. This phenomenon suggests a bold conjecture: every problem in NP is either in P or NP complete. We show that if P NP then this is false there is a language L NP \ P that is not NP-complete. (If P = NP then the conjecture is trivially true but uninteresting.) The rest of this section proves this. Theorem 3.4 (Ladner s Theorem [?]) Suppose that P NP. Then there exists a language L NP \ P that is not NP-complete. Proof: If P NP then we know at least one language in NP \ P: namely, the NP-complete language SAT. Consider the language SAT H of length n satisfiable formulae that { are padded with n H(n) } 1 s for some polynomial-time computable function H : N N (i.e., SAT H = ψ01 nh(n) : ψ SAT and n = ψ ). Consider two possibilities: (a) H(n) is at most some constant c for every n. In this case SAT H is simply SAT with a polynomial amount of padding. Thus, SAT H is also NP-complete and is not in P if P NP. (b) H(n) tends to infinity with n, and thus the padding is of superpolynomial size. In this case, we claim that SAT H cannot be NP-complete. Indeed, if there is a O(n i )-time reduction f from SAT to SAT H then such a reduction reduces the satisfiability of SAT instances of length n to instances of SAT H of length O(n i ), which must have the form ψ01 ψ H( ψ ), where ψ + ψ H( ψ ) = O(n i ), and hence ψ = o(n). In other words, we have a polynomial-time reduction from SAT instances of length n to SAT instances of length o(n), which implies SAT can be solved in polynomial time. (The algorithm consists of applying the reduction again and again, reducing the size of the instances each time until the instance is of size O(1) and can be solved in O(1) time by brute force) This is a contradiction to the assumption P NP.

7 3.4. LADNER S THEOREM: EXISTENCE OF NP-INTERMEDIATE PROBLEMS. p3.5 (67) The proof of the Theorem uses a language SAT H for a function H that in some senses combines the two cases above. This function tends to infinity with n, so that SAT H is not NP-complete as in Case (b), but grows slowly enough to assure SAT H P as in Case (a). Function H is defined as follows: H(n) is the smallest number i < log log n such that for every x {0, 1} with x log n, M i halts on x within i x i steps and M i outputs 1 iff x SAT H where M i is the machine represented by the binary expansion of i according to the representation scheme of the universal Turing machine U of Theorem If there is no such i then we let H(n) = log log n. Notice, this is implicitly a recursive definition since the definition of H depends on SAT H, but a moment s thought shows that H is well-defined since H(n) determines membership in SAT H of strings whose length is greater than n, and the definition of H(n) only relies upon checking the status of strings of length at most log n. There is a trivial algorithm to compute H(n) in O(n 3 ) time. After all, we only need to (1) compute H(k) for every k log n, (2) simulate at most log log n machines for every input of length at most log n for log log n(log n) log log n = o(n) steps, and (3) compute SAT on all the inputs of length at most log n. Now we have the following two claims. claim 1: SAT H is not in P. Suppose, for the sake of contradiction, that there is a machine M solving SAT H in at most cn c steps. Since M is represented by infinitely many strings, there is a number i > c such that M = M i. By the definition of H(n) this implies that for n > 2 2i, H(n) i. But this means that for all sufficiently large input lengths, SAT H is simply the language SAT padded with a polynomial (i.e., n i ) number of 1 s, and so cannot be in P unless P = NP. claim 2: SAT H is not NP-complete. As in Case (b), it suffices to show that H(n) tends to infinity with n. We prove the equivalent statement that for every integer i, there are only finitely many n s such that H(n) = i: since SAT H P, for each i we know that there is an input x such that given i x i time, M i gives the incorrect answer to whether or not x SAT H. Then the definition of H ensures that for every n > 2 x, H(x) i. Remark 3.5 We do not know of a natural decision problem that, assuming NP P, is proven to be in NP \ P but not NP-complete, and there are remarkably few candidates for such languages. However, there are a few fascinating examples for languages not known to be either in P nor NP-complete. Two such examples are the Factoring and Graph isomorphism languages (see Example 2.2). No polynomial-time algorithm is currently known for these languages, and there is some evidence that they are not NP complete (see Chapter 8).

11 3.5. ORACLE MACHINES AND THE LIMITS OF DIAGONALIZATION? p3.9 (71) Chapter notes and history Georg Cantor invented diagonalization in the 19th century to show that the set of real numbers is uncountable. Kurt Gödel used a similar technique in his proof of the Incompleteness Theorem. Computer science undergraduates often encounter diagonalization when they are taught the undecidabilty of the Halting Problem. The time hierarchy theorem is from Hartmanis and Stearns pioneering paper [?]. The space hierarchy theorem is from Stearns, Hartmanis, and Lewis [?]. The nondeterministic time hierarchy theorem is from Cook [?], though the simple proof given here is essentially from [?]. A similar proof works for other complexity classes such as the (levels of the) polynomial hierarchy discussed in the next chapter. Ladner s theorem is from [?] but the proof here is due to an unpublished manuscript by Impagliazzo. The notion of relativizations of the P versus NP question is from Baker, Gill, and Solovay [?], though the authors of that paper note that other researchers independently discovered some of their ideas. The notion of relativization is related to similar ideas in logic (such as independence results) and recursive function theory. The notion of oracle Turing machines can be used to study interrelationships of complexity classes. In fact, Cook [?] defined NP-completeness using oracle machines. A subfield of complexity theory called structural complexity has carried out a detailed study of oracle machines and classes defined using them; see []. Whether or not the Cook-Levin theorem is a nonrelativizing fact depends upon how you formalize the question. There is a way to allow the 3SAT instance to query the oracle, and then the Cook-Levin theorem does relativize. However, it seems safe to say that any result that uses the locality of computation is looking at the internal workings of the machine and hence is potentially nonrelativizing. The term superiority introduced in the exercises does not appear in the literature but the concept does. In particular,??? have shown the limitations of relativizing techniques in resolving certain similar open questions. Exercises 1 Show that the following language is undecidable: { M : M is a machine that runs in 100n time }. 2 Show that SPACE(n) NP. (Note that we do not know if either class is contained in the other.) 3 Show that there is a language B EXP such that NP B P B. 4 Say that a class C 1 is superior to a class C 2 if there is a machine M 1 in class C 1 such that for every machine M 2 in class C 2 and every large enough n, there is an input of size between n and n 2 on which M 1 and M 2 answer differently. (a) Is DTIME(n 1.1 ) superior to DTIME(n)?

12 p3.10 (72) 3.5. ORACLE MACHINES AND THE LIMITS OF DIAGONALIZATION? (b) Is NTIME(n 1.1 ) superior to NTIME(n)? 5 Show that there exists a function that is not time-constructible. 6 Show that there is an oracle A and a language L NP A such that L is not polynomial-time reducible to 3SAT even when the machine computing the reduction is allowed access to A. 7 Suppose we pick a random language B, by deciding for each string independently and with probability 1/2 whether or not it is in B. Show that with high probability P B NP B. (To give an answer that is formally correct you may need to know elementary measure theory.)

### DRAFT. Diagonalization. Chapter 4

Chapter 4 Diagonalization..the relativized P =?NP question has a positive answer for some oracles and a negative answer for other oracles. We feel that this is further evidence of the difficulty of the

### The Polynomial Hierarchy

The Polynomial Hierarchy Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Motivation..synthesizing circuits is exceedingly difficulty. It is even

### 1 Computational Problems

Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

### MTAT Complexity Theory October 13th-14th, Lecture 6

MTAT.07.004 Complexity Theory October 13th-14th, 2011 Lecturer: Peeter Laud Lecture 6 Scribe(s): Riivo Talviste 1 Logarithmic memory Turing machines working in logarithmic space become interesting when

### 2 P vs. NP and Diagonalization

2 P vs NP and Diagonalization CS 6810 Theory of Computing, Fall 2012 Instructor: David Steurer (sc2392) Date: 08/28/2012 In this lecture, we cover the following topics: 1 3SAT is NP hard; 2 Time hierarchies;

### Lecture 3: Reductions and Completeness

CS 710: Complexity Theory 9/13/2011 Lecture 3: Reductions and Completeness Instructor: Dieter van Melkebeek Scribe: Brian Nixon Last lecture we introduced the notion of a universal Turing machine for deterministic

### satisfiability (sat) Satisfiability unsatisfiability (unsat or sat complement) and validity Any Expression φ Can Be Converted into CNFs and DNFs

Any Expression φ Can Be Converted into CNFs and DNFs φ = x j : This is trivially true. φ = φ 1 and a CNF is sought: Turn φ 1 into a DNF and apply de Morgan s laws to make a CNF for φ. φ = φ 1 and a DNF

### De Morgan s a Laws. De Morgan s laws say that. (φ 1 φ 2 ) = φ 1 φ 2, (φ 1 φ 2 ) = φ 1 φ 2.

De Morgan s a Laws De Morgan s laws say that (φ 1 φ 2 ) = φ 1 φ 2, (φ 1 φ 2 ) = φ 1 φ 2. Here is a proof for the first law: φ 1 φ 2 (φ 1 φ 2 ) φ 1 φ 2 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0 0 a Augustus DeMorgan

### Finish K-Complexity, Start Time Complexity

6.045 Finish K-Complexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts

### Ma/CS 117c Handout # 5 P vs. NP

Ma/CS 117c Handout # 5 P vs. NP We consider the possible relationships among the classes P, NP, and co-np. First we consider properties of the class of NP-complete problems, as opposed to those which are

### Lecture 16: Time Complexity and P vs NP

6.045 Lecture 16: Time Complexity and P vs NP 1 Time-Bounded Complexity Classes Definition: TIME(t(n)) = { L there is a Turing machine M with time complexity O(t(n)) so that L = L(M) } = { L L is a language

### Computer Sciences Department

Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y

### Lecture 20: conp and Friends, Oracles in Complexity Theory

6.045 Lecture 20: conp and Friends, Oracles in Complexity Theory 1 Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode:

### 6.045 Final Exam Solutions

6.045J/18.400J: Automata, Computability and Complexity Prof. Nancy Lynch, Nati Srebro 6.045 Final Exam Solutions May 18, 2004 Susan Hohenberger Name: Please write your name on each page. This exam is open

### Space Complexity. The space complexity of a program is how much memory it uses.

Space Complexity The space complexity of a program is how much memory it uses. Measuring Space When we compute the space used by a TM, we do not count the input (think of input as readonly). We say that

### 9 Nondeterministic Turing Machines

Caveat lector: This is the zeroth (draft) edition of this lecture note. In particular, some topics still need to be written. Please send bug reports and suggestions to jeffe@illinois.edu. If first you

### Notes on Complexity Theory Last updated: October, Lecture 6

Notes on Complexity Theory Last updated: October, 2015 Lecture 6 Notes by Jonathan Katz, lightly edited by Dov Gordon 1 PSPACE and PSPACE-Completeness As in our previous study of N P, it is useful to identify

### DRAFT. Algebraic computation models. Chapter 14

Chapter 14 Algebraic computation models Somewhat rough We think of numerical algorithms root-finding, gaussian elimination etc. as operating over R or C, even though the underlying representation of the

### 15-855: Intensive Intro to Complexity Theory Spring Lecture 7: The Permanent, Toda s Theorem, XXX

15-855: Intensive Intro to Complexity Theory Spring 2009 Lecture 7: The Permanent, Toda s Theorem, XXX 1 #P and Permanent Recall the class of counting problems, #P, introduced last lecture. It was introduced

### Definition: conp = { L L NP } What does a conp computation look like?

Space Complexity 28 Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode: Guess string y of x k length and the machine accepts

### Randomized Complexity Classes; RP

Randomized Complexity Classes; RP Let N be a polynomial-time precise NTM that runs in time p(n) and has 2 nondeterministic choices at each step. N is a polynomial Monte Carlo Turing machine for a language

### The space complexity of a standard Turing machine. The space complexity of a nondeterministic Turing machine

298 8. Space Complexity The space complexity of a standard Turing machine M = (Q,,,, q 0, accept, reject) on input w is space M (w) = max{ uav : q 0 w M u q av, q Q, u, a, v * } The space complexity of

### CISC 876: Kolmogorov Complexity

March 27, 2007 Outline 1 Introduction 2 Definition Incompressibility and Randomness 3 Prefix Complexity Resource-Bounded K-Complexity 4 Incompressibility Method Gödel s Incompleteness Theorem 5 Outline

### CS154, Lecture 10: Rice s Theorem, Oracle Machines

CS154, Lecture 10: Rice s Theorem, Oracle Machines Moral: Analyzing Programs is Really, Really Hard But can we more easily tell when some program analysis problem is undecidable? Problem 1 Undecidable

### Introduction to Turing Machines. Reading: Chapters 8 & 9

Introduction to Turing Machines Reading: Chapters 8 & 9 1 Turing Machines (TM) Generalize the class of CFLs: Recursively Enumerable Languages Recursive Languages Context-Free Languages Regular Languages

### 1 Computational problems

80240233: Computational Complexity Lecture 1 ITCS, Tsinghua Univesity, Fall 2007 9 October 2007 Instructor: Andrej Bogdanov Notes by: Andrej Bogdanov The aim of computational complexity theory is to study

### Decision Problems with TM s. Lecture 31: Halting Problem. Universe of discourse. Semi-decidable. Look at following sets: CSCI 81 Spring, 2012

Decision Problems with TM s Look at following sets: Lecture 31: Halting Problem CSCI 81 Spring, 2012 Kim Bruce A TM = { M,w M is a TM and w L(M)} H TM = { M,w M is a TM which halts on input w} TOTAL TM

### Lecture 12: Interactive Proofs

princeton university cos 522: computational complexity Lecture 12: Interactive Proofs Lecturer: Sanjeev Arora Scribe:Carl Kingsford Recall the certificate definition of NP. We can think of this characterization

### Computability Theory. CS215, Lecture 6,

Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David

CMPSCI611: NP Completeness Lecture 17 Essential facts about NP-completeness: Any NP-complete problem can be solved by a simple, but exponentially slow algorithm. We don t have polynomial-time solutions

### Bounded rationality of restricted Turing machines

Bounded rationality of restricted Turing machines Lijie Chen, Pingzhong Tang IIIS, Tsinghua University, Beijing, China September 10, 2015 Abstract Bounded rationality aims to understand the effects of

### Lecture 22: PSPACE

6.045 Lecture 22: PSPACE 1 VOTE VOTE VOTE For your favorite course on automata and complexity Please complete the online subject evaluation for 6.045 2 Final Exam Information Who: You On What: Everything

### Turing Machines Part III

Turing Machines Part III Announcements Problem Set 6 due now. Problem Set 7 out, due Monday, March 4. Play around with Turing machines, their powers, and their limits. Some problems require Wednesday's

### Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar

IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar Turing Machine A Turing machine is an abstract representation of a computing device. It consists of a read/write

### Notes on Complexity Theory Last updated: November, Lecture 10

Notes on Complexity Theory Last updated: November, 2015 Lecture 10 Notes by Jonathan Katz, lightly edited by Dov Gordon. 1 Randomized Time Complexity 1.1 How Large is BPP? We know that P ZPP = RP corp

### Verifying whether One-Tape Turing Machines Run in Linear Time

Electronic Colloquium on Computational Complexity, Report No. 36 (2015) Verifying whether One-Tape Turing Machines Run in Linear Time David Gajser IMFM, Jadranska 19, 1000 Ljubljana, Slovenija david.gajser@fmf.uni-lj.si

### Theory of Computation

Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation 1 / 33 Lecture 20: Overview Incompressible strings Minimal Length Descriptions Descriptive Complexity Dr. Sarmad Abbasi

### Final Exam Comments. UVa - cs302: Theory of Computation Spring < Total

UVa - cs302: Theory of Computation Spring 2008 Final Exam Comments < 50 50 59 60 69 70 79 80 89 90 94 95-102 Total 2 6 8 22 16 16 12 Problem 1: Short Answers. (20) For each question, provide a correct,

### Theory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death

Theory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death Robbie Hott www.cs.virginia.edu/~jh2jf Department of Computer Science University of Virginia

### U.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan 9/6/2004. Notes for Lecture 3

U.C. Berkeley CS278: Computational Complexity Handout N3 Professor Luca Trevisan 9/6/2004 Notes for Lecture 3 Revised 10/6/04 1 Space-Bounded Complexity Classes A machine solves a problem using space s(

### 6.045J/18.400J: Automata, Computability and Complexity Final Exam. There are two sheets of scratch paper at the end of this exam.

6.045J/18.400J: Automata, Computability and Complexity May 20, 2005 6.045 Final Exam Prof. Nancy Lynch Name: Please write your name on each page. This exam is open book, open notes. There are two sheets

### Time Complexity. Definition. Let t : n n be a function. NTIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM}

Time Complexity Definition Let t : n n be a function. TIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM} NTIME(t(n)) = {L L is a language decidable by a O(t(n)) non-deterministic

### The Complexity of Optimization Problems

The Complexity of Optimization Problems Summary Lecture 1 - Complexity of algorithms and problems - Complexity classes: P and NP - Reducibility - Karp reducibility - Turing reducibility Uniform and logarithmic

### Space is a computation resource. Unlike time it can be reused. Computational Complexity, by Fu Yuxi Space Complexity 1 / 44

Space Complexity Space is a computation resource. Unlike time it can be reused. Computational Complexity, by Fu Yuxi Space Complexity 1 / 44 Synopsis 1. Space Bounded Computation 2. Logspace Reduction

### Randomness and non-uniformity

Randomness and non-uniformity JASS 2006 Course 1: Proofs and Computers Felix Weninger TU München April 2006 Outline Randomized computation 1 Randomized computation 2 Computation with advice Non-uniform

### Notes on Space-Bounded Complexity

U.C. Berkeley CS172: Automata, Computability and Complexity Handout 7 Professor Luca Trevisan April 14, 2015 Notes on Space-Bounded Complexity These are notes for CS278, Computational Complexity, scribed

### 2 Natural Proofs: a barrier for proving circuit lower bounds

Topics in Theoretical Computer Science April 4, 2016 Lecturer: Ola Svensson Lecture 6 (Notes) Scribes: Ola Svensson Disclaimer: These notes were written for the lecturer only and may contain inconsistent

### Lecture 2. 1 More N P-Compete Languages. Notes on Complexity Theory: Fall 2005 Last updated: September, Jonathan Katz

Notes on Complexity Theory: Fall 2005 Last updated: September, 2005 Jonathan Katz Lecture 2 1 More N P-Compete Languages It will be nice to find more natural N P-complete languages. To that end, we ine

### CSCI 1590 Intro to Computational Complexity

CSCI 1590 Intro to Computational Complexity Space Complexity John E. Savage Brown University February 11, 2008 John E. Savage (Brown University) CSCI 1590 Intro to Computational Complexity February 11,

### Advanced Topics in Theoretical Computer Science

Advanced Topics in Theoretical Computer Science Part 5: Complexity (Part II) 30.01.2014 Viorica Sofronie-Stokkermans Universität Koblenz-Landau e-mail: sofronie@uni-koblenz.de 1 Contents Recall: Turing

### 6.841/18.405J: Advanced Complexity Wednesday, February 12, Lecture Lecture 3

6.841/18.405J: Advanced Complexity Wednesday, February 12, 2003 Lecture Lecture 3 Instructor: Madhu Sudan Scribe: Bobby Kleinberg 1 The language MinDNF At the end of the last lecture, we introduced the

### Notes for Lecture 3... x 4

Stanford University CS254: Computational Complexity Notes 3 Luca Trevisan January 18, 2012 Notes for Lecture 3 In this lecture we introduce the computational model of boolean circuits and prove that polynomial

### Lecture 17: Cook-Levin Theorem, NP-Complete Problems

6.045 Lecture 17: Cook-Levin Theorem, NP-Complete Problems 1 Is SAT solvable in O(n) time on a multitape TM? Logic circuits of 6n gates for SAT? If yes, then not only is P=NP, but there would be a dream

### More Complexity. Klaus Sutner Carnegie Mellon University. Fall 2017

More Complexity Klaus Sutner Carnegie Mellon University Fall 2017 1 Below NP Parallelism Above NP Where Are We? 3 We have a somewhat elaborate zoo of low complexity classes. For the flagship P and NP,

### Complexity and NP-completeness

Lecture 17 Complexity and NP-completeness Supplemental reading in CLRS: Chapter 34 As an engineer or computer scientist, it is important not only to be able to solve problems, but also to know which problems

### Technische Universität München Department of Computer Science. Joint Advanced Students School 2009 Propositional Proof Complexity.

Technische Universität München Department of Computer Science Joint Advanced Students School 2009 Propositional Proof Complexity May 2009 Frege Systems Michael Herrmann Contents Contents ii 1 Introduction

### 3 Probabilistic Turing Machines

CS 252 - Probabilistic Turing Machines (Algorithms) Additional Reading 3 and Homework problems 3 Probabilistic Turing Machines When we talked about computability, we found that nondeterminism was sometimes

### Lecture Notes: The Halting Problem; Reductions

Lecture Notes: The Halting Problem; Reductions COMS W3261 Columbia University 20 Mar 2012 1 Review Key point. Turing machines can be encoded as strings, and other Turing machines can read those strings

### Primitive recursive functions: decidability problems

Primitive recursive functions: decidability problems Armando B. Matos October 24, 2014 Abstract Although every primitive recursive (PR) function is total, many problems related to PR functions are undecidable.

### 6.840 Language Membership

6.840 Language Membership Michael Bernstein 1 Undecidable INP Practice final Use for A T M. Build a machine that asks EQ REX and then runs M on w. Query INP. If it s in P, accept. Note that the language

### CSCE 551: Chin-Tser Huang. University of South Carolina

CSCE 551: Theory of Computation Chin-Tser Huang huangct@cse.sc.edu University of South Carolina Church-Turing Thesis The definition of the algorithm came in the 1936 papers of Alonzo Church h and Alan

### Chapter 2 Algorithms and Computation

Chapter 2 Algorithms and Computation In this chapter, we first discuss the principles of algorithm and computation in general framework, common both in classical and quantum computers, then we go to the

### Chapter 1 - Time and Space Complexity. deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE

Chapter 1 - Time and Space Complexity deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE 1 / 41 Deterministic Turing machines Definition 1.1 A (deterministic

### Computational Models Lecture 9, Spring 2009

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. p. 1 Computational Models Lecture 9, Spring 2009 Reducibility among languages Mapping reductions More undecidable

### TIME COMPLEXITY AND POLYNOMIAL TIME; NON DETERMINISTIC TURING MACHINES AND NP. THURSDAY Mar 20

TIME COMPLEXITY AND POLYNOMIAL TIME; NON DETERMINISTIC TURING MACHINES AND NP THURSDAY Mar 20 COMPLEXITY THEORY Studies what can and can t be computed under limited resources such as time, space, etc Today:

### Computability Theory

Computability Theory Cristian S. Calude May 2012 Computability Theory 1 / 1 Bibliography M. Sipser. Introduction to the Theory of Computation, PWS 1997. (textbook) Computability Theory 2 / 1 Supplementary

### Decidability and Undecidability

Decidability and Undecidability Major Ideas from Last Time Every TM can be converted into a string representation of itself. The encoding of M is denoted M. The universal Turing machine U TM accepts an

### 16.1 Countability. CS125 Lecture 16 Fall 2014

CS125 Lecture 16 Fall 2014 16.1 Countability Proving the non-existence of algorithms for computational problems can be very difficult. Indeed, we do not know how to prove P NP. So a natural question is

### P, NP, NP-Complete, and NPhard

P, NP, NP-Complete, and NPhard Problems Zhenjiang Li 21/09/2011 Outline Algorithm time complicity P and NP problems NP-Complete and NP-Hard problems Algorithm time complicity Outline What is this course

### Computational Complexity CSCI-GA Subhash Khot Transcribed by Patrick Lin

Computational Complexity CSCI-GA 3350 Subhash Khot Transcribed by Patrick Lin Abstract. These notes are from a course in Computational Complexity, as offered in Spring 2014 at the Courant Institute of

### NP Complete Problems. COMP 215 Lecture 20

NP Complete Problems COMP 215 Lecture 20 Complexity Theory Complexity theory is a research area unto itself. The central project is classifying problems as either tractable or intractable. Tractable Worst

### Theory of Computation. Ch.8 Space Complexity. wherein all branches of its computation halt on all

Definition 8.1 Let M be a deterministic Turing machine, DTM, that halts on all inputs. The space complexity of M is the function f : N N, where f(n) is the maximum number of tape cells that M scans on

### Polynomial Space. The classes PS and NPS Relationship to Other Classes Equivalence PS = NPS A PS-Complete Problem

Polynomial Space The classes PS and NPS Relationship to Other Classes Equivalence PS = NPS A PS-Complete Problem 1 Polynomial-Space-Bounded TM s A TM M is said to be polyspacebounded if there is a polynomial

### 1 Deterministic Turing Machines

Time and Space Classes Exposition by William Gasarch 1 Deterministic Turing Machines Turing machines are a model of computation. It is believed that anything that can be computed can be computed by a Turing

### 1 Acceptance, Rejection, and I/O for Turing Machines

1 Acceptance, Rejection, and I/O for Turing Machines Definition 1.1 (Initial Configuration) If M = (K,Σ,δ,s,H) is a Turing machine and w (Σ {, }) then the initial configuration of M on input w is (s, w).

### Lecture 13: Foundations of Math and Kolmogorov Complexity

6.045 Lecture 13: Foundations of Math and Kolmogorov Complexity 1 Self-Reference and the Recursion Theorem 2 Lemma: There is a computable function q : Σ* Σ* such that for every string w, q(w) is the description

### CS5371 Theory of Computation. Lecture 19: Complexity IV (More on NP, NP-Complete)

CS5371 Theory of Computation Lecture 19: Complexity IV (More on NP, NP-Complete) Objectives More discussion on the class NP Cook-Levin Theorem The Class NP revisited Recall that NP is the class of language

### CSC 5170: Theory of Computational Complexity Lecture 5 The Chinese University of Hong Kong 8 February 2010

CSC 5170: Theory of Computational Complexity Lecture 5 The Chinese University of Hong Kong 8 February 2010 So far our notion of realistic computation has been completely deterministic: The Turing Machine

### CS 361 Meeting 26 11/10/17

CS 361 Meeting 26 11/10/17 1. Homework 8 due Announcements A Recognizable, but Undecidable Language 1. Last class, I presented a brief, somewhat inscrutable proof that the language A BT M = { M w M is

### Further discussion of Turing machines

Further discussion of Turing machines In this lecture we will discuss various aspects of decidable and Turing-recognizable languages that were not mentioned in previous lectures. In particular, we will

### ON COMPUTAMBLE NUMBERS, WITH AN APPLICATION TO THE ENTSCHENIDUGSPROBLEM. Turing 1936

ON COMPUTAMBLE NUMBERS, WITH AN APPLICATION TO THE ENTSCHENIDUGSPROBLEM Turing 1936 Where are We? Ignoramus et ignorabimus Wir mussen wissen Wir werden wissen We do not know We shall not know We must know

Introduction to Advanced Results Master Informatique Université Paris 5 René Descartes 2016 Master Info. Complexity Advanced Results 1/26 Outline Boolean Hierarchy Probabilistic Complexity Parameterized

### Resource-Bounded Computation

Resource-Bounded Computation Previously: can something be done? Now: how efficiently can it be done? Goal: conserve computational resources: Time, space, other resources? Def: L is decidable within time

### CpSc 421 Homework 9 Solution

CpSc 421 Homework 9 Solution Attempt any three of the six problems below. The homework is graded on a scale of 100 points, even though you can attempt fewer or more points than that. Your recorded grade

### Theory of Computation Lecture Notes. Problems and Algorithms. Class Information

Theory of Computation Lecture Notes Prof. Yuh-Dauh Lyuu Dept. Computer Science & Information Engineering and Department of Finance National Taiwan University Problems and Algorithms c 2004 Prof. Yuh-Dauh

### Notes for Lecture 2. Statement of the PCP Theorem and Constraint Satisfaction

U.C. Berkeley Handout N2 CS294: PCP and Hardness of Approximation January 23, 2006 Professor Luca Trevisan Scribe: Luca Trevisan Notes for Lecture 2 These notes are based on my survey paper [5]. L.T. Statement

### Nondeterministic Circuit Lower Bounds from Mildly Derandomizing Arthur-Merlin Games

Nondeterministic Circuit Lower Bounds from Mildly Derandomizing Arthur-Merlin Games Barış Aydınlıo glu Department of Computer Sciences University of Wisconsin Madison, WI 53706, USA baris@cs.wisc.edu Dieter

### Problems, and How Computer Scientists Solve Them Manas Thakur

Problems, and How Computer Scientists Solve Them PACE Lab, IIT Madras Content Credits Introduction to Automata Theory, Languages, and Computation, 3rd edition. Hopcroft et al. Introduction to the Theory

### Lecture 24: Randomized Complexity, Course Summary

6.045 Lecture 24: Randomized Complexity, Course Summary 1 1/4 1/16 1/4 1/4 1/32 1/16 1/32 Probabilistic TMs 1/16 A probabilistic TM M is a nondeterministic TM where: Each nondeterministic step is called

### 6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch

6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch Today: More Complexity Theory Polynomial-time reducibility, NP-completeness, and the Satisfiability (SAT) problem Topics: Introduction

### A Turing Machine Distance Hierarchy

A Turing Machine Distance Hierarchy Stanislav Žák and Jiří Šíma Institute of Computer Science, Academy of Sciences of the Czech Republic, P. O. Box 5, 18207 Prague 8, Czech Republic, {stan sima}@cs.cas.cz

### Complexity Theory. Jörg Kreiker. Summer term Chair for Theoretical Computer Science Prof. Esparza TU München

Complexity Theory Jörg Kreiker Chair for Theoretical Computer Science Prof. Esparza TU München Summer term 2010 Lecture 4 NP-completeness Recap: relations between classes EXP PSPACE = NPSPACE conp NP conp

### CSE 105 Theory of Computation

CSE 105 Theory of Computation http://www.jflap.org/jflaptmp/ Professor Jeanne Ferrante 1 Undecidability Today s Agenda Review and More Problems A Non-TR Language Reminders and announcements: HW 7 (Last!!)

### Time and space classes

Time and space classes Little Oh (o,

### Alternation Trading Proofs and Their Limitations

and Their Limitations Mathematical Foundations of Computer Science (MFCS) IST Austria, Klosterneuburg August 27, 2013 NP and Satisfiability Lower bounds Fundamental problems for computer science include

### Lecture 20: PSPACE. November 15, 2016 CS 1010 Theory of Computation

Lecture 20: PSPACE November 15, 2016 CS 1010 Theory of Computation Recall that PSPACE = k=1 SPACE(nk ). We will see that a relationship between time and space complexity is given by: P NP PSPACE = NPSPACE

### The Complexity of Maximum. Matroid-Greedoid Intersection and. Weighted Greedoid Maximization

Department of Computer Science Series of Publications C Report C-2004-2 The Complexity of Maximum Matroid-Greedoid Intersection and Weighted Greedoid Maximization Taneli Mielikäinen Esko Ukkonen University