Computational complexity
|
|
|
- Charlene Hicks
- 5 years ago
- Views:
Transcription
1 COMS11700 Computational complexity Department of Computer Science, University of Bristol Bristol, UK 2 May 2014 COMS11700: Computational complexity Slide 1/23
2 Introduction If we can prove that a language is decidable, that does not mean we can solve the corresponding decision problem in practice. Computational complexity theory studies the question of which problems we can solve given restricted resources: in particular, restricted time or space. We are generally interested in how the resources we need to solve a family of problems grow with problem size. This allows us to compare the complexity of different problems and formalise the intuitive notion that some problems are harder than others. COMS11700: Computational complexity Slide 2/23
3 Time complexity Time complexity Let M be a deterministic Turing machine that halts on all inputs. The running time or time complexity of M is the function f : N N where f (n) is the maximum number of steps that M uses on any input of length n. COMS11700: Computational complexity Slide 3/23
4 Time complexity Time complexity Let M be a deterministic Turing machine that halts on all inputs. The running time or time complexity of M is the function f : N N where f (n) is the maximum number of steps that M uses on any input of length n. Some notes on this definition: A step is a transition, which includes reading a symbol, changing state and writing a new symbol to the tape. This is a worst-case notion of complexity, i.e. we define the running time of M on inputs of a certain length to be the number of steps that M takes on the worst possible input of that length. We usually use n to represent the length of the input. COMS11700: Computational complexity Slide 3/23
5 When comparing running times we often only care about the scaling behaviour with the input size n. COMS11700: Computational complexity Slide 4/23
6 When comparing running times we often only care about the scaling behaviour with the input size n. For example, given two Turing machines M 1 and M 2 with running times 3n 2 + 2n and 137n respectively, for large n the 3n 2 term in the running time of the first machine will dominate n 2 + 2n n f (n) n COMS11700: Computational complexity Slide 4/23
7 Big-O notation Big-O notation allows us to simplify expressions for running times etc. while still retaining the important features. Definition Let f, g : N R + be functions. We write f (n) = O(g(n)) if there exist positive integers c and n 0 such that, for all integers n n 0, f (n) c g(n). COMS11700: Computational complexity Slide 5/23
8 Big-O notation Big-O notation allows us to simplify expressions for running times etc. while still retaining the important features. Definition Let f, g : N R + be functions. We write f (n) = O(g(n)) if there exist positive integers c and n 0 such that, for all integers n n 0, f (n) c g(n). For example: f (n) = 3n 2 + n: f (n) = O(n 2 ) f (n) = 0.01n n 3 : f (n) = O(n 3 ), but f (n) is not O(n 2 ). COMS11700: Computational complexity Slide 5/23
9 Time complexity classes Definition Let t : N R + be a function. Then TIME(t(n)) is the set of all languages which are decidable by a Turing machine running in time O(t(n)). COMS11700: Computational complexity Slide 6/23
10 Time complexity classes Definition Let t : N R + be a function. Then TIME(t(n)) is the set of all languages which are decidable by a Turing machine running in time O(t(n)). For example, recall the language L EQ = {w#w w {0, 1} } of two equal bit-strings, separated by a # symbol. Claim: L EQ TIME(n 2 ). COMS11700: Computational complexity Slide 6/23
11 Time complexity classes Recall the following algorithm for deciding this language: Algorithm for deciding L EQ (sketch) 1. Scan forwards and backwards, testing each corresponding pair of bits either side of the # for equality in turn. 2. Overwrite each bit with an x symbol after checking it so we don t check the same bits twice. 3. Accept if all pairs of bits match each other; otherwise reject. COMS11700: Computational complexity Slide 7/23
12 Time complexity classes Recall the following algorithm for deciding this language: Algorithm for deciding L EQ (sketch) 1. Scan forwards and backwards, testing each corresponding pair of bits either side of the # for equality in turn. 2. Overwrite each bit with an x symbol after checking it so we don t check the same bits twice. 3. Accept if all pairs of bits match each other; otherwise reject. Bounding the time complexity (sketch): In the worst case, the input string is of the form w#w for some string w {0, 1} m. COMS11700: Computational complexity Slide 7/23
13 Time complexity classes Recall the following algorithm for deciding this language: Algorithm for deciding L EQ (sketch) 1. Scan forwards and backwards, testing each corresponding pair of bits either side of the # for equality in turn. 2. Overwrite each bit with an x symbol after checking it so we don t check the same bits twice. 3. Accept if all pairs of bits match each other; otherwise reject. Bounding the time complexity (sketch): In the worst case, the input string is of the form w#w for some string w {0, 1} m. For each symbol in the part of the string to the left of #, m + 1 moves to the right are made, the corresponding symbol to the right of # is checked and m + 1 moves to the left are made. COMS11700: Computational complexity Slide 7/23
14 Time complexity classes Recall the following algorithm for deciding this language: Algorithm for deciding L EQ (sketch) 1. Scan forwards and backwards, testing each corresponding pair of bits either side of the # for equality in turn. 2. Overwrite each bit with an x symbol after checking it so we don t check the same bits twice. 3. Accept if all pairs of bits match each other; otherwise reject. Bounding the time complexity (sketch): In the worst case, the input string is of the form w#w for some string w {0, 1} m. For each symbol in the part of the string to the left of #, m + 1 moves to the right are made, the corresponding symbol to the right of # is checked and m + 1 moves to the left are made. So the algorithm runs in time O(m 2 ), which is also O(n 2 ). COMS11700: Computational complexity Slide 7/23
15 Polynomial and exponential time We would like to distinguish between algorithms which are efficient (run in a reasonable length of time) and algorithms which are inefficient. One way to do this is via the concepts of polynomial-time and exponential-time algorithms. COMS11700: Computational complexity Slide 8/23
16 Polynomial and exponential time We would like to distinguish between algorithms which are efficient (run in a reasonable length of time) and algorithms which are inefficient. One way to do this is via the concepts of polynomial-time and exponential-time algorithms. Definitions P = k 0 TIME(n k ) EXP = k 0 TIME(2 nk ) So P is the class of languages that are decided by Turing machines with runtime polynomial in the input size. EXP is the class of languages decided by Turing machines with runtime exponential in the input size. COMS11700: Computational complexity Slide 8/23
17 Polynomial and exponential time n 0.8 f (n) n n COMS11700: Computational complexity Slide 9/23
18 Some examples of languages in P Consider the language PATH defined by PATH = { G, s, t G is a directed graph that has a path from s to t} B E A D G C F COMS11700: Computational complexity Slide 10/23
19 Some examples of languages in P Consider the language PATH defined by PATH = { G, s, t G is a directed graph that has a path from s to t} B E A D G C F PATH is a formalisation of the problem of determining whether there is a path between two specified points in a directed graph. COMS11700: Computational complexity Slide 10/23
20 Some examples of languages in P Consider the language PATH defined by PATH = { G, s, t G is a directed graph that has a path from s to t} B E A D G C F PATH is a formalisation of the problem of determining whether there is a path between two specified points in a directed graph. A naïve algorithm would be to try every possible path (sequence of vertices) from s to t in turn to see if that path exists. But if G has m vertices, in the worst case this could involve checking m m paths... COMS11700: Computational complexity Slide 10/23
21 Some examples of languages in P A polynomial-time algorithm deciding PATH On input G, s, t : 1. Place a mark on vertex s. 2. Repeat until no further vertices are found: Check all the edges in G. If an edge is found from a marked vertex to an unmarked vertex, mark the target of the edge. 3. If t is marked, accept; otherwise reject. COMS11700: Computational complexity Slide 11/23
22 Some examples of languages in P A polynomial-time algorithm deciding PATH On input G, s, t : 1. Place a mark on vertex s. 2. Repeat until no further vertices are found: Check all the edges in G. If an edge is found from a marked vertex to an unmarked vertex, mark the target of the edge. 3. If t is marked, accept; otherwise reject. A rough upper bound on the running time of this algorithm: Assume the input graph G is on m vertices and is provided as an adjacency matrix, so the input size n = O(m 2 ). COMS11700: Computational complexity Slide 11/23
23 Some examples of languages in P A polynomial-time algorithm deciding PATH On input G, s, t : 1. Place a mark on vertex s. 2. Repeat until no further vertices are found: Check all the edges in G. If an edge is found from a marked vertex to an unmarked vertex, mark the target of the edge. 3. If t is marked, accept; otherwise reject. A rough upper bound on the running time of this algorithm: Assume the input graph G is on m vertices and is provided as an adjacency matrix, so the input size n = O(m 2 ). Step 2 is repeated at most m times and checks at most m 2 edges. COMS11700: Computational complexity Slide 11/23
24 Some examples of languages in P A polynomial-time algorithm deciding PATH On input G, s, t : 1. Place a mark on vertex s. 2. Repeat until no further vertices are found: Check all the edges in G. If an edge is found from a marked vertex to an unmarked vertex, mark the target of the edge. 3. If t is marked, accept; otherwise reject. A rough upper bound on the running time of this algorithm: Assume the input graph G is on m vertices and is provided as an adjacency matrix, so the input size n = O(m 2 ). Step 2 is repeated at most m times and checks at most m 2 edges. Checking each edge in step 2 can be done in time O(m k ) for some fixed k (depending on the computational model). COMS11700: Computational complexity Slide 11/23
25 Some examples of languages in P A polynomial-time algorithm deciding PATH On input G, s, t : 1. Place a mark on vertex s. 2. Repeat until no further vertices are found: Check all the edges in G. If an edge is found from a marked vertex to an unmarked vertex, mark the target of the edge. 3. If t is marked, accept; otherwise reject. A rough upper bound on the running time of this algorithm: Assume the input graph G is on m vertices and is provided as an adjacency matrix, so the input size n = O(m 2 ). Step 2 is repeated at most m times and checks at most m 2 edges. Checking each edge in step 2 can be done in time O(m k ) for some fixed k (depending on the computational model). So the running time of the algorithm is O(m k+3 ), which is poly(n). COMS11700: Computational complexity Slide 11/23
26 Some examples of languages in P Many other important problems are known to be in P. For example: The language PRIMES = {x {0, 1} x is a prime number written in binary} is in P but this was only proven in 2002 (by two undergraduate students and a professor). COMS11700: Computational complexity Slide 12/23
27 Some examples of languages in P Many other important problems are known to be in P. For example: The language PRIMES = {x {0, 1} x is a prime number written in binary} is in P but this was only proven in 2002 (by two undergraduate students and a professor). Every context-free language is in P. The algorithm we presented showing decidability of CFLs does not imply this (why not?), but the CYK Algorithm you saw in Algorithms and Programming does. COMS11700: Computational complexity Slide 12/23
28 Some examples of languages in P Many other important problems are known to be in P. For example: The language PRIMES = {x {0, 1} x is a prime number written in binary} is in P but this was only proven in 2002 (by two undergraduate students and a professor). Every context-free language is in P. The algorithm we presented showing decidability of CFLs does not imply this (why not?), but the CYK Algorithm you saw in Algorithms and Programming does. Other examples include evaluation of circuits, finding shortest paths, pattern matching, linear programming,... COMS11700: Computational complexity Slide 12/23
29 Problems whose solutions are easy to check There are problems which may not be easy to solve, but for which it is easy to check a claimed solution. Pics: Wikipedia/Sudoku COMS11700: Computational complexity Slide 13/23
30 Problems whose solutions are easy to check There are problems which may not be easy to solve, but for which it is easy to check a claimed solution. Pics: Wikipedia/Sudoku COMS11700: Computational complexity Slide 13/23
31 Problems whose solutions are easy to check We can formalise this using the notion of a verifier. Definition A verifier for a language L is a Turing machine V such that L = {x there exists a string c such that V accepts x, c }. We think of c as a proof or witness that x L, which V can check. COMS11700: Computational complexity Slide 14/23
32 Problems whose solutions are easy to check We can formalise this using the notion of a verifier. Definition A verifier for a language L is a Turing machine V such that L = {x there exists a string c such that V accepts x, c }. We think of c as a proof or witness that x L, which V can check. V should accept a correct proof, but not be fooled by any claimed incorrect proof. COMS11700: Computational complexity Slide 14/23
33 Problems whose solutions are easy to check We can formalise this using the notion of a verifier. Definition A verifier for a language L is a Turing machine V such that L = {x there exists a string c such that V accepts x, c }. We think of c as a proof or witness that x L, which V can check. V should accept a correct proof, but not be fooled by any claimed incorrect proof. A polynomial-time verifier is a verifier which runs in time polynomial in the length of the input x, i.e. O( x k ) for some fixed k. Definition NP is the class of languages which have a polynomial-time verifier. COMS11700: Computational complexity Slide 14/23
34 Problems whose solutions are easy to check An example of a language in NP: FACTORING = { x, y x is an integer with a prime factor lower than y}. For example, 15, 4 is in the language, but 15, 2 is not. COMS11700: Computational complexity Slide 15/23
35 Problems whose solutions are easy to check An example of a language in NP: FACTORING = { x, y x is an integer with a prime factor lower than y}. For example, 15, 4 is in the language, but 15, 2 is not. For any x, the list of prime factors of x can be used as a proof to determine whether x, y is in the language. Given some claimed prime factors of x, we can multiply them together to determine whether we get x. If so, we check whether the smallest of them is lower than y. COMS11700: Computational complexity Slide 15/23
36 Problems whose solutions are easy to check Another example is graph colouring problems. We say a graph can be properly k-coloured if each vertex can be assigned one of k colours, such that all pairs of adjacent vertices have different colours. COMS11700: Computational complexity Slide 16/23
37 Problems whose solutions are easy to check Another example is graph colouring problems. We say a graph can be properly k-coloured if each vertex can be assigned one of k colours, such that all pairs of adjacent vertices have different colours. COMS11700: Computational complexity Slide 16/23
38 Problems whose solutions are easy to check Another example is graph colouring problems. We say a graph can be properly k-coloured if each vertex can be assigned one of k colours, such that all pairs of adjacent vertices have different colours. COMS11700: Computational complexity Slide 16/23
39 Problems whose solutions are easy to check Another example is graph colouring problems. We say a graph can be properly k-coloured if each vertex can be assigned one of k colours, such that all pairs of adjacent vertices have different colours. We can formalise this decision problem as the language 3-COLOURING = { G G is a graph which can be properly 3-coloured}. Given a graph, we can be convinced that it can be properly 3-coloured by being given a 3-colouring and checking that it is proper; so 3-COLOURING is in NP. COMS11700: Computational complexity Slide 16/23
40 P vs. NP Every language in P is also in NP. For any language in P, the verifier can ignore any claimed proof and just decide the language directly. COMS11700: Computational complexity Slide 17/23
41 P vs. NP Every language in P is also in NP. For any language in P, the verifier can ignore any claimed proof and just decide the language directly. Also, every language in NP is also in EXP. If there exists an m-bit witness that the input is in a language, by looping over all possible witnesses a Turing machine can find that witness in time O(2 m ). COMS11700: Computational complexity Slide 17/23
42 P vs. NP Every language in P is also in NP. For any language in P, the verifier can ignore any claimed proof and just decide the language directly. Also, every language in NP is also in EXP. If there exists an m-bit witness that the input is in a language, by looping over all possible witnesses a Turing machine can find that witness in time O(2 m ). So P NP EXP. COMS11700: Computational complexity Slide 17/23
43 P vs. NP Every language in P is also in NP. For any language in P, the verifier can ignore any claimed proof and just decide the language directly. Also, every language in NP is also in EXP. If there exists an m-bit witness that the input is in a language, by looping over all possible witnesses a Turing machine can find that witness in time O(2 m ). So P NP EXP. It is not known whether P = NP and this question is considered the most important open problem in computer science! Resolving it would win you everlasting fame (as well as $1M). COMS11700: Computational complexity Slide 17/23
44 Nondeterministic polynomial-time Recall that a nondeterministic Turing machine (NDTM) can explore multiple computational paths simultaneously. It accepts if and only if at least one of the computational paths accepts. q q 3 q q COMS11700: Computational complexity Slide 18/23
45 Nondeterministic polynomial-time Definitions Let t : N R + be a function. Then NTIME(t(n)) is the set of all languages which are decidable by a nondeterministic Turing machine running in time O(t(n)). We say that an NDTM runs in time O(t(n)) if all of its computational paths halt in time O(t(n)). COMS11700: Computational complexity Slide 19/23
46 Nondeterministic polynomial-time Definitions Let t : N R + be a function. Then NTIME(t(n)) is the set of all languages which are decidable by a nondeterministic Turing machine running in time O(t(n)). We say that an NDTM runs in time O(t(n)) if all of its computational paths halt in time O(t(n)). We will now see that there is a close connection between nondeterministic Turing machines and verification of proofs. This will explain the name NP, which stands for Nondeterministic Polynomial-time (and not non-polynomial time ). COMS11700: Computational complexity Slide 19/23
47 Nondeterministic polynomial-time Theorem NP = k 0 NTIME(n k ). COMS11700: Computational complexity Slide 20/23
48 Nondeterministic polynomial-time Theorem NP = k 0 NTIME(n k ). Proof (sketch) We first show that L NP implies that L is decided by an NDTM running in polynomial time (i.e. NP k 0 NTIME(nk )). COMS11700: Computational complexity Slide 20/23
49 Nondeterministic polynomial-time Theorem NP = k 0 NTIME(n k ). Proof (sketch) We first show that L NP implies that L is decided by an NDTM running in polynomial time (i.e. NP k 0 NTIME(nk )). From the definition of NP, there exists a verifier V for L running in time O(n k ) for some fixed k. COMS11700: Computational complexity Slide 20/23
50 Nondeterministic polynomial-time Theorem NP = k 0 NTIME(n k ). Proof (sketch) We first show that L NP implies that L is decided by an NDTM running in polynomial time (i.e. NP k 0 NTIME(nk )). From the definition of NP, there exists a verifier V for L running in time O(n k ) for some fixed k. We define an NDTM N which behaves as follows: 1. Nondeterministically guess a string c of length at most O(n k ). 2. Run V on input x, c. If V accepts, accept; otherwise, reject. COMS11700: Computational complexity Slide 20/23
51 Theorem NP = k 0 NTIME(n k ). Proof (sketch) For the other direction, we show that if L is decided by an NDTM running in polynomial time, L NP (i.e. k 0 NTIME(nk ) NP). COMS11700: Computational complexity Slide 21/23
52 Theorem NP = k 0 NTIME(n k ). Proof (sketch) For the other direction, we show that if L is decided by an NDTM running in polynomial time, L NP (i.e. k 0 NTIME(nk ) NP). From the definition of NTIME, there exists an NDTM N which decides L and runs in time O(n k ) for some fixed k. COMS11700: Computational complexity Slide 21/23
53 Theorem NP = k 0 NTIME(n k ). Proof (sketch) For the other direction, we show that if L is decided by an NDTM running in polynomial time, L NP (i.e. k 0 NTIME(nk ) NP). From the definition of NTIME, there exists an NDTM N which decides L and runs in time O(n k ) for some fixed k. We define a verifier V which takes as input a string x and a witness c. c is the description of a sequence of computational path choices made by N. COMS11700: Computational complexity Slide 21/23
54 Theorem NP = k 0 NTIME(n k ). Proof (sketch) For the other direction, we show that if L is decided by an NDTM running in polynomial time, L NP (i.e. k 0 NTIME(nk ) NP). From the definition of NTIME, there exists an NDTM N which decides L and runs in time O(n k ) for some fixed k. We define a verifier V which takes as input a string x and a witness c. c is the description of a sequence of computational path choices made by N. V simulates the computation of N on input x according to c, checking whether each transition made is valid. COMS11700: Computational complexity Slide 21/23
55 Theorem NP = k 0 NTIME(n k ). Proof (sketch) For the other direction, we show that if L is decided by an NDTM running in polynomial time, L NP (i.e. k 0 NTIME(nk ) NP). From the definition of NTIME, there exists an NDTM N which decides L and runs in time O(n k ) for some fixed k. We define a verifier V which takes as input a string x and a witness c. c is the description of a sequence of computational path choices made by N. V simulates the computation of N on input x according to c, checking whether each transition made is valid. If N accepts in the end, V accepts; otherwise, V rejects. COMS11700: Computational complexity Slide 21/23
56 NP-completeness Some languages in the class NP are known to actually be NP-complete. A language L is said to be NP-complete if: L NP; Every language L NP reduces to L in polynomial time. So NP-complete problems are the hardest problems in NP: if we can solve them efficiently, we can solve every other problem in NP efficiently. COMS11700: Computational complexity Slide 22/23
57 NP-completeness Some languages in the class NP are known to actually be NP-complete. A language L is said to be NP-complete if: L NP; Every language L NP reduces to L in polynomial time. So NP-complete problems are the hardest problems in NP: if we can solve them efficiently, we can solve every other problem in NP efficiently. Examples of NP-complete problems include 3-colouring, clique finding in graphs, optimal packing problems... in fact, many of the most practically important problems in computer science. COMS11700: Computational complexity Slide 22/23
58 Summary and further reading Computational complexity theory allows us to study the resources required to solve problems. Problems can be classified according to the time required to solve them in the deterministic Turing machine model. P is the class of problems which can be solved in time polynomial in the input size, while NP is the class of problems whose solutions can be verified in time polynomial in the input size. NP has an alternative definition in terms of nondeterministic Turing machines. Further reading: Sipser COMS11700: Computational complexity Slide 23/23
Chapter 7: Time Complexity
Chapter 7: Time Complexity 1 Time complexity Let M be a deterministic Turing machine that halts on all inputs. The running time or time complexity of M is the function f: N N, where f(n) is the maximum
CSCI3390-Lecture 14: The class NP
CSCI3390-Lecture 14: The class NP 1 Problems and Witnesses All of the decision problems described below have the form: Is there a solution to X? where X is the given problem instance. If the instance is
Time Complexity. CS60001: Foundations of Computing Science
Time Complexity CS60001: Foundations of Computing Science Professor, Dept. of Computer Sc. & Engg., Measuring Complexity Definition Let M be a deterministic Turing machine that halts on all inputs. The
CSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION Fall 2016 http://cseweb.ucsd.edu/classes/fa16/cse105-abc/ Logistics HW7 due tonight Thursday's class: REVIEW Final exam on Thursday Dec 8, 8am-11am, LEDDN AUD Note card allowed
Time Complexity (1) CSCI Spring Original Slides were written by Dr. Frederick W Maier. CSCI 2670 Time Complexity (1)
Time Complexity (1) CSCI 2670 Original Slides were written by Dr. Frederick W Maier Spring 2014 Time Complexity So far we ve dealt with determining whether or not a problem is decidable. But even if it
NP, polynomial-time mapping reductions, and NP-completeness
NP, polynomial-time mapping reductions, and NP-completeness In the previous lecture we discussed deterministic time complexity, along with the time-hierarchy theorem, and introduced two complexity classes:
Undecidability COMS Ashley Montanaro 4 April Department of Computer Science, University of Bristol Bristol, UK
COMS11700 Undecidability Department of Computer Science, University of Bristol Bristol, UK 4 April 2014 COMS11700: Undecidability Slide 1/29 Decidability We are particularly interested in Turing machines
CISC 4090 Theory of Computation
CISC 4090 Theory of Computation Complexity Professor Daniel Leeds dleeds@fordham.edu JMH 332 Computability Are we guaranteed to get an answer? Complexity How long do we have to wait for an answer? (Ch7)
Lecture 16: Time Complexity and P vs NP
6.045 Lecture 16: Time Complexity and P vs NP 1 Time-Bounded Complexity Classes Definition: TIME(t(n)) = { L there is a Turing machine M with time complexity O(t(n)) so that L = L(M) } = { L L is a language
Finish K-Complexity, Start Time Complexity
6.045 Finish K-Complexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts
CSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION "Winter" 2018 http://cseweb.ucsd.edu/classes/wi18/cse105-ab/ Today's learning goals Sipser Ch 7 Distinguish between computability and complexity Articulate motivation questions
Theory of Computation
Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation 1 / 38 Lecture 21: Overview Big-Oh notation. Little-o notation. Time Complexity Classes Non-deterministic TMs The Class
Advanced topic: Space complexity
Advanced topic: Space complexity CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Chinese University of Hong Kong Fall 2016 1/28 Review: time complexity We have looked at how long it takes to
Complexity (Pre Lecture)
Complexity (Pre Lecture) Dr. Neil T. Dantam CSCI-561, Colorado School of Mines Fall 2018 Dantam (Mines CSCI-561) Complexity (Pre Lecture) Fall 2018 1 / 70 Why? What can we always compute efficiently? What
Lecture 3: Nondeterminism, NP, and NP-completeness
CSE 531: Computational Complexity I Winter 2016 Lecture 3: Nondeterminism, NP, and NP-completeness January 13, 2016 Lecturer: Paul Beame Scribe: Paul Beame 1 Nondeterminism and NP Recall the definition
Friday Four Square! Today at 4:15PM, Outside Gates
P and NP Friday Four Square! Today at 4:15PM, Outside Gates Recap from Last Time Regular Languages DCFLs CFLs Efficiently Decidable Languages R Undecidable Languages Time Complexity A step of a Turing
Theory of Computation Time Complexity
Theory of Computation Time Complexity Bow-Yaw Wang Academia Sinica Spring 2012 Bow-Yaw Wang (Academia Sinica) Time Complexity Spring 2012 1 / 59 Time for Deciding a Language Let us consider A = {0 n 1
Announcements. Problem Set 7 graded; will be returned at end of lecture. Unclaimed problem sets and midterms moved!
N P NP Announcements Problem Set 7 graded; will be returned at end of lecture. Unclaimed problem sets and midterms moved! Now in cabinets in the Gates open area near the drop-off box. The Complexity Class
Complexity: moving from qualitative to quantitative considerations
Complexity: moving from qualitative to quantitative considerations Textbook chapter 7 Complexity Theory: study of what is computationally feasible (or tractable) with limited resources: running time (main
Computability Theory
CS:4330 Theory of Computation Spring 2018 Computability Theory The class NP Haniel Barbosa Readings for this lecture Chapter 7 of [Sipser 1996], 3rd edition. Section 7.3. Question Why are we unsuccessful
CSE200: Computability and complexity Space Complexity
CSE200: Computability and complexity Space Complexity Shachar Lovett January 29, 2018 1 Space complexity We would like to discuss languages that may be determined in sub-linear space. Lets first recall
Time Complexity. Definition. Let t : n n be a function. NTIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM}
Time Complexity Definition Let t : n n be a function. TIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM} NTIME(t(n)) = {L L is a language decidable by a O(t(n)) non-deterministic
Computational Models Lecture 11, Spring 2009
Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. p. 1 Computational Models Lecture 11, Spring 2009 Deterministic Time Classes NonDeterministic Time Classes
Could we potentially place A in a smaller complexity class if we consider other computational models?
Introduction to Complexity Theory Big O Notation Review Linear function: r(n) =O(n). Polynomial function: r(n) =2 O(1) Exponential function: r(n) =2 no(1) Logarithmic function: r(n) = O(log n) Poly-log
Turing machines COMS Ashley Montanaro 21 March Department of Computer Science, University of Bristol Bristol, UK
COMS11700 Turing machines Department of Computer Science, University of Bristol Bristol, UK 21 March 2014 COMS11700: Turing machines Slide 1/15 Introduction We have seen two models of computation: finite
CS154, Lecture 13: P vs NP
CS154, Lecture 13: P vs NP The EXTENDED Church-Turing Thesis Everyone s Intuitive Notion of Efficient Algorithms Polynomial-Time Turing Machines More generally: TM can simulate every reasonable model of
CSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION Spring 2016 http://cseweb.ucsd.edu/classes/sp16/cse105-ab/ Today's learning goals Sipser Ch 7.2, 7.3 Distinguish between polynomial and exponential DTIME Define nondeterministic
Complexity Theory Part I
Complexity Theory Part I Problem Problem Set Set 77 due due right right now now using using a late late period period The Limits of Computability EQ TM EQ TM co-re R RE L D ADD L D HALT A TM HALT A TM
FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY. FLAC (15-453) Spring l. Blum TIME COMPLEXITY AND POLYNOMIAL TIME;
15-453 TIME COMPLEXITY AND POLYNOMIAL TIME; FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY NON DETERMINISTIC TURING MACHINES AND NP THURSDAY Mar 20 COMPLEXITY THEORY Studies what can and can t be computed
Turing Machines and Time Complexity
Turing Machines and Time Complexity Turing Machines Turing Machines (Infinitely long) Tape of 1 s and 0 s Turing Machines (Infinitely long) Tape of 1 s and 0 s Able to read and write the tape, and move
CS151 Complexity Theory. Lecture 1 April 3, 2017
CS151 Complexity Theory Lecture 1 April 3, 2017 Complexity Theory Classify problems according to the computational resources required running time storage space parallelism randomness rounds of interaction,
CS154, Lecture 13: P vs NP
CS154, Lecture 13: P vs NP The EXTENDED Church-Turing Thesis Everyone s Intuitive Notion of Efficient Algorithms Polynomial-Time Turing Machines More generally: TM can simulate every reasonable model of
Computer Sciences Department
Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y
Harvard CS 121 and CSCI E-121 Lecture 20: Polynomial Time
Harvard CS 121 and CSCI E-121 Lecture 20: Polynomial Time Harry Lewis November 12, 20123 Review of Asymptotic Notation For f, g : N R + f = O(g): c > 0 s.t. f(n) c g(n) for all sufficiently large n. f
Lecture 2 (Notes) 1. The book Computational Complexity: A Modern Approach by Sanjeev Arora and Boaz Barak;
Topics in Theoretical Computer Science February 29, 2016 Lecturer: Ola Svensson Lecture 2 (Notes) Scribes: Ola Svensson Disclaimer: These notes were written for the lecturer only and may contain inconsistent
CSE 105 Theory of Computation
CSE 105 Theory of Computation http://www.jflap.org/jflaptmp/ Professor Jeanne Ferrante 1 Today s Agenda P and NP (7.2, 7.3) Next class: Review Reminders and announcements: CAPE & TA evals are open: Please
1 Computational Problems
Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation
Computability and Complexity
Computability and Complexity Lecture 10 More examples of problems in P Closure properties of the class P The class NP given by Jiri Srba Lecture 10 Computability and Complexity 1/12 Example: Relatively
P and NP. Or, how to make $1,000,000.
P and NP Or, how to make $1,000,000. http://www.claymath.org/millennium-problems/p-vs-np-problem Review: Polynomial time difference between single-tape and multi-tape TMs Exponential time difference between
TIME COMPLEXITY AND POLYNOMIAL TIME; NON DETERMINISTIC TURING MACHINES AND NP. THURSDAY Mar 20
TIME COMPLEXITY AND POLYNOMIAL TIME; NON DETERMINISTIC TURING MACHINES AND NP THURSDAY Mar 20 COMPLEXITY THEORY Studies what can and can t be computed under limited resources such as time, space, etc Today:
Notes for Lecture Notes 2
Stanford University CS254: Computational Complexity Notes 2 Luca Trevisan January 11, 2012 Notes for Lecture Notes 2 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation
Computational Complexity
CS311 Computational Structures Computational Complexity Lecture 16 Andrew P. Black Andrew Tolmach 1 So, itʼs computable! But at what cost? Some things that are computable in principle are in practice intractable
Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar
IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar Turing Machine A Turing machine is an abstract representation of a computing device. It consists of a read/write
Algorithms and Theory of Computation. Lecture 19: Class P and NP, Reduction
Algorithms and Theory of Computation Lecture 19: Class P and NP, Reduction Xiaohui Bei MAS 714 October 29, 2018 Nanyang Technological University MAS 714 October 29, 2018 1 / 26 Decision Problems Revisited
Theory of Computation
Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation 1 / 33 Lecture 20: Overview Incompressible strings Minimal Length Descriptions Descriptive Complexity Dr. Sarmad Abbasi
COMPUTATIONAL COMPLEXITY
ATHEATICS: CONCEPTS, AND FOUNDATIONS Vol. III - Computational Complexity - Osamu Watanabe COPUTATIONAL COPLEXITY Osamu Watanabe Tokyo Institute of Technology, Tokyo, Japan Keywords: {deterministic, randomized,
In complexity theory, algorithms and problems are classified by the growth order of computation time as a function of instance size.
10 2.2. CLASSES OF COMPUTATIONAL COMPLEXITY An optimization problem is defined as a class of similar problems with different input parameters. Each individual case with fixed parameter values is called
Notes on Complexity Theory Last updated: December, Lecture 2
Notes on Complexity Theory Last updated: December, 2011 Jonathan Katz Lecture 2 1 Review The running time of a Turing machine M on input x is the number of steps M takes before it halts. Machine M is said
NP-Completeness. Until now we have been designing algorithms for specific problems
NP-Completeness 1 Introduction Until now we have been designing algorithms for specific problems We have seen running times O(log n), O(n), O(n log n), O(n 2 ), O(n 3 )... We have also discussed lower
Complexity Theory Part II
Complexity Theory Part II Time Complexity The time complexity of a TM M is a function denoting the worst-case number of steps M takes on any input of length n. By convention, n denotes the length of the
Lecture 19: Finish NP-Completeness, conp and Friends
6.045 Lecture 19: Finish NP-Completeness, conp and Friends 1 Polynomial Time Reducibility f : Σ* Σ* is a polynomial time computable function if there is a poly-time Turing machine M that on every input
P is the class of problems for which there are algorithms that solve the problem in time O(n k ) for some constant k.
Complexity Theory Problems are divided into complexity classes. Informally: So far in this course, almost all algorithms had polynomial running time, i.e., on inputs of size n, worst-case running time
P, NP, NP-Complete, and NPhard
P, NP, NP-Complete, and NPhard Problems Zhenjiang Li 21/09/2011 Outline Algorithm time complicity P and NP problems NP-Complete and NP-Hard problems Algorithm time complicity Outline What is this course
Theory of Computation
Theory of Computation Lecture #6 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 39 Lecture 6: Overview Prove the equivalence of enumerators and TMs. Dovetailing
Lecture 20: conp and Friends, Oracles in Complexity Theory
6.045 Lecture 20: conp and Friends, Oracles in Complexity Theory 1 Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode:
conp, Oracles, Space Complexity
conp, Oracles, Space Complexity 1 What s next? A few possibilities CS161 Design and Analysis of Algorithms CS254 Complexity Theory (next year) CS354 Topics in Circuit Complexity For your favorite course
Lecture 18: P & NP. Revised, May 1, CLRS, pp
Lecture 18: P & NP Revised, May 1, 2003 CLRS, pp.966-982 The course so far: techniques for designing efficient algorithms, e.g., divide-and-conquer, dynamic-programming, greedy-algorithms. What happens
CSCE 551 Final Exam, Spring 2004 Answer Key
CSCE 551 Final Exam, Spring 2004 Answer Key 1. (10 points) Using any method you like (including intuition), give the unique minimal DFA equivalent to the following NFA: 0 1 2 0 5 1 3 4 If your answer is
an efficient procedure for the decision problem. We illustrate this phenomenon for the Satisfiability problem.
1 More on NP In this set of lecture notes, we examine the class NP in more detail. We give a characterization of NP which justifies the guess and verify paradigm, and study the complexity of solving search
Lecture 6: Oracle TMs, Diagonalization Limits, Space Complexity
CSE 531: Computational Complexity I Winter 2016 Lecture 6: Oracle TMs, Diagonalization Limits, Space Complexity January 22, 2016 Lecturer: Paul Beame Scribe: Paul Beame Diagonalization enabled us to separate
UNIT-VIII COMPUTABILITY THEORY
CONTEXT SENSITIVE LANGUAGE UNIT-VIII COMPUTABILITY THEORY A Context Sensitive Grammar is a 4-tuple, G = (N, Σ P, S) where: N Set of non terminal symbols Σ Set of terminal symbols S Start symbol of the
Introduction to Complexity Theory
Introduction to Complexity Theory Read K & S Chapter 6. Most computational problems you will face your life are solvable (decidable). We have yet to address whether a problem is easy or hard. Complexity
Announcements. Friday Four Square! Problem Set 8 due right now. Problem Set 9 out, due next Friday at 2:15PM. Did you lose a phone in my office?
N P NP Completeness Announcements Friday Four Square! Today at 4:15PM, outside Gates. Problem Set 8 due right now. Problem Set 9 out, due next Friday at 2:15PM. Explore P, NP, and their connection. Did
Chapter 2. Reductions and NP. 2.1 Reductions Continued The Satisfiability Problem (SAT) SAT 3SAT. CS 573: Algorithms, Fall 2013 August 29, 2013
Chapter 2 Reductions and NP CS 573: Algorithms, Fall 2013 August 29, 2013 2.1 Reductions Continued 2.1.1 The Satisfiability Problem SAT 2.1.1.1 Propositional Formulas Definition 2.1.1. Consider a set of
FORMAL LANGUAGES, AUTOMATA AND COMPUTATION
FORMAL LANGUAGES, AUTOMATA AND COMPUTATION DECIDABILITY ( LECTURE 15) SLIDES FOR 15-453 SPRING 2011 1 / 34 TURING MACHINES-SYNOPSIS The most general model of computation Computations of a TM are described
Lecture 3: Reductions and Completeness
CS 710: Complexity Theory 9/13/2011 Lecture 3: Reductions and Completeness Instructor: Dieter van Melkebeek Scribe: Brian Nixon Last lecture we introduced the notion of a universal Turing machine for deterministic
CS5371 Theory of Computation. Lecture 18: Complexity III (Two Classes: P and NP)
CS5371 Theory of Computation Lecture 18: Complexity III (Two Classes: P and NP) Objectives Define what is the class P Examples of languages in P Define what is the class NP Examples of languages in NP
The purpose here is to classify computational problems according to their complexity. For that purpose we need first to agree on a computational
1 The purpose here is to classify computational problems according to their complexity. For that purpose we need first to agree on a computational model. We'll remind you what a Turing machine is --- you
Umans Complexity Theory Lectures
Complexity Theory Umans Complexity Theory Lectures Lecture 1a: Problems and Languages Classify problems according to the computational resources required running time storage space parallelism randomness
U.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan August 30, Notes for Lecture 1
U.C. Berkeley CS278: Computational Complexity Handout N1 Professor Luca Trevisan August 30, 2004 Notes for Lecture 1 This course assumes CS170, or equivalent, as a prerequisite. We will assume that the
Intro to Theory of Computation
Intro to Theory of Computation LECTURE 24 Last time Relationship between models: deterministic/nondeterministic Class P Today Class NP Sofya Raskhodnikova Homework 9 due Homework 0 out 4/5/206 L24. I-clicker
Advanced Topics in Theoretical Computer Science
Advanced Topics in Theoretical Computer Science Part 5: Complexity (Part II) 30.01.2014 Viorica Sofronie-Stokkermans Universität Koblenz-Landau e-mail: sofronie@uni-koblenz.de 1 Contents Recall: Turing
FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY THURSDAY APRIL 3 REVIEW for Midterm TUESDAY April 8 Definition: A Turing Machine is a 7-tuple T = (Q, Σ, Γ, δ, q, q accept, q reject ), where: Q is a
6-1 Computational Complexity
6-1 Computational Complexity 6. Computational Complexity Computational models Turing Machines Time complexity Non-determinism, witnesses, and short proofs. Complexity classes: P, NP, conp Polynomial-time
Computational Complexity. IE 496 Lecture 6. Dr. Ted Ralphs
Computational Complexity IE 496 Lecture 6 Dr. Ted Ralphs IE496 Lecture 6 1 Reading for This Lecture N&W Sections I.5.1 and I.5.2 Wolsey Chapter 6 Kozen Lectures 21-25 IE496 Lecture 6 2 Introduction to
INTRODUCTION TO THE THEORY OF NP-COMPLETENESS
INTRODUCTION TO THE THEORY OF NP-COMPLETENESS CSI4105 (3 TO 5 LECTURES) Lucia Moura University of Ottawa Winter 2006 These notes/slides are intended as an introduction to the theory of NP-completeness,
Definition: conp = { L L NP } What does a conp computation look like?
Space Complexity 28 Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode: Guess string y of x k length and the machine accepts
Week 2: Defining Computation
Computational Complexity Theory Summer HSSP 2018 Week 2: Defining Computation Dylan Hendrickson MIT Educational Studies Program 2.1 Turing Machines Turing machines provide a simple, clearly defined way
NP-Completeness. Algorithmique Fall semester 2011/12
NP-Completeness Algorithmique Fall semester 2011/12 1 What is an Algorithm? We haven t really answered this question in this course. Informally, an algorithm is a step-by-step procedure for computing a
Computability and Complexity Theory: An Introduction
Computability and Complexity Theory: An Introduction meena@imsc.res.in http://www.imsc.res.in/ meena IMI-IISc, 20 July 2006 p. 1 Understanding Computation Kinds of questions we seek answers to: Is a given
The Polynomial Hierarchy
The Polynomial Hierarchy Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Motivation..synthesizing circuits is exceedingly difficulty. It is even
Time to learn about NP-completeness!
Time to learn about NP-completeness! Harvey Mudd College March 19, 2007 Languages A language is a set of strings Examples The language of strings of all zeros with odd length The language of strings with
CS154, Lecture 17: conp, Oracles again, Space Complexity
CS154, Lecture 17: conp, Oracles again, Space Complexity Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode: Guess string
Complexity Theory Part I
Complexity Theory Part I Outline for Today Recap from Last Time Reviewing Verifiers Nondeterministic Turing Machines What does nondeterminism mean in the context of TMs? And just how powerful are NTMs?
Lecture 21: Space Complexity (The Final Exam Frontier?)
6.045 Lecture 21: Space Complexity (The Final Exam Frontier?) 1 conp NP MIN-FORMULA conp P NP FIRST-SAT TAUT P FACTORING SAT NP NP NP 2 VOTE VOTE VOTE For your favorite course on automata and complexity
NP Complete Problems. COMP 215 Lecture 20
NP Complete Problems COMP 215 Lecture 20 Complexity Theory Complexity theory is a research area unto itself. The central project is classifying problems as either tractable or intractable. Tractable Worst
Chapter 2: The Basics. slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens
Chapter 2: The Basics slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens Problem instances vs. decision problems vs. search problems Decision
Non-Deterministic Time
Non-Deterministic Time Master Informatique 2016 1 Non-Deterministic Time Complexity Classes Reminder on DTM vs NDTM [Turing 1936] (q 0, x 0 ) (q 1, x 1 ) Deterministic (q n, x n ) Non-Deterministic (q
1. Introduction Recap
1. Introduction Recap 1. Tractable and intractable problems polynomial-boundness: O(n k ) 2. NP-complete problems informal definition 3. Examples of P vs. NP difference may appear only slightly 4. Optimization
NP and NP Completeness
CS 374: Algorithms & Models of Computation, Spring 2017 NP and NP Completeness Lecture 23 April 20, 2017 Chandra Chekuri (UIUC) CS374 1 Spring 2017 1 / 44 Part I NP Chandra Chekuri (UIUC) CS374 2 Spring
22c:135 Theory of Computation. Analyzing an Algorithm. Simplifying Conventions. Example computation. How much time does M1 take to decide A?
Example computation Consider the decidable language A = {0 n 1 n n 0} and the following TM M1 deciding A: M1 = "On input string w: 1. Scan across the tape and reject if a 0 appears after a 1 2. Repeat
Lecture 18: PCP Theorem and Hardness of Approximation I
Lecture 18: and Hardness of Approximation I Arijit Bishnu 26.04.2010 Outline 1 Introduction to Approximation Algorithm 2 Outline 1 Introduction to Approximation Algorithm 2 Approximation Algorithm Approximation
1 Non-deterministic Turing Machine
1 Non-deterministic Turing Machine A nondeterministic Turing machine is a generalization of the standard TM for which every configuration may yield none, or one or more than one next configurations. In
15.1 Proof of the Cook-Levin Theorem: SAT is NP-complete
CS125 Lecture 15 Fall 2016 15.1 Proof of the Cook-Levin Theorem: SAT is NP-complete Already know SAT NP, so only need to show SAT is NP-hard. Let L be any language in NP. Let M be a NTM that decides L
Time-bounded computations
Lecture 18 Time-bounded computations We now begin the final part of the course, which is on complexity theory. We ll have time to only scratch the surface complexity theory is a rich subject, and many
SAT, NP, NP-Completeness
CS 473: Algorithms, Spring 2018 SAT, NP, NP-Completeness Lecture 22 April 13, 2018 Most slides are courtesy Prof. Chekuri Ruta (UIUC) CS473 1 Spring 2018 1 / 57 Part I Reductions Continued Ruta (UIUC)
CP405 Theory of Computation
CP405 Theory of Computation BB(3) q 0 q 1 q 2 0 q 1 1R q 2 0R q 2 1L 1 H1R q 1 1R q 0 1L Growing Fast BB(3) = 6 BB(4) = 13 BB(5) = 4098 BB(6) = 3.515 x 10 18267 (known) (known) (possible) (possible) Language:
Quantum Computing Lecture 8. Quantum Automata and Complexity
Quantum Computing Lecture 8 Quantum Automata and Complexity Maris Ozols Computational models and complexity Shor s algorithm solves, in polynomial time, a problem for which no classical polynomial time
1 Computational problems
80240233: Computational Complexity Lecture 1 ITCS, Tsinghua Univesity, Fall 2007 9 October 2007 Instructor: Andrej Bogdanov Notes by: Andrej Bogdanov The aim of computational complexity theory is to study