# Complexity Theory of Polynomial-Time Problems

Size: px
Start display at page:

## Transcription

1 Complexity Theory of Polynomial-Time Problems Lecture 1: Introduction, Easy Examples Karl Bringmann and Sebastian Krinninger

2 Audience no formal requirements, but: NP-hardness, satisfiability problem, how to multiply two matrices, dynamic programming, all-pairs-shortest-path problem, Dijkstra s algorithm if you (vaguely) remember at least half of these things, then you should be able to follow the course

3 NP-hardness suppose we want to solve some problem fastest algorithm that we can come up with: O(2 \$ ) should we search further for an efficient algorithm? prove NP-hardness! an efficient algorithm would show P=NP we may assume that no efficient algorithm exists relax the problem: approximation algorithms, fixed parameter tractability, average case, heuristics

4 What about polynomial time? suppose we want to solve another problem we come up with an O(n ' ) algorithm polynomial time = efficient big data: input is too large for O(n ' ) algorithm should we search for faster algorithms? P vs NP is too coarse we need fine-grained complexity to study limits of big data

5 Conditional Lower Bounds which barriers prevent subquadratic algorithms? consider a well-studied problem P: best-known running time O(n * ) conjecture that it has no O(n *+, ) algorithm for any ε > 0 relate another problem Q to P via a reduction conditional lower bound

6 Hard problems SAT: OV: APSP: 3SUM: given a formula in conj. normal form on n variables is it satisfiable? conjecture: no O(2 :+, \$ ) algorithm (SETH) given n vectors in {0,1} 8 (for small d) are any two orthogonal? conjecture: no O(n '+, ) algorithm given a weighted graph with n vertices compute the distance between any pair of vertices conjecture: no O(n 3+, ) algorithm given n integers do any three sum to 0? conjecture: no O(n '+, ) algorithm

7 Relations = Reductions transfer hardness of one problem to another one by reductions problem P instance I size n reduction r(n) time problem Q instance J size s(n) I is a yes -instance () J is a yes -instance t(n) algorithm for Q implies a r n + t(s n ) algorithm for P if P has no r n + t(s n ) algorithm then Q has no t(n) algorithm

8 Relations = Reductions transfer hardness of one problem to another one by reductions problem P instance I size n reduction total time r(n) problem Q instance I 1 size s : (n) instance I k size s E (n) t(n) algorithm for Q implies a r n + E DF: t(s D n ) algorithm for P

9 Showcase Results longest common subseq. edit distance, longest palindromic subsequence, Fréchet distance... O(n ' ) SETH-hard n '+, [B.,Künnemann 15, Abboud,Backurs,V-Williams 15] given two strings x, y of length n, compute the longest string z that is a subsequence of both x and y a b b c a d a c d a a b d

10 Showcase Results longest common subseq. edit distance, longest palindromic subsequence, Fréchet distance... O(n ' ) SETH-hard n '+, [B.,Künnemann 15, Abboud,Backurs,V-Williams 15] we can stop searching for faster algorithms! in this sense, conditional lower bounds replace NP-hardness O(n '+, ) algorithms are unlikely to exist improvements are at least as hard as a breakthrough for SAT

11 Showcase Results longest common subseq. edit distance, longest palindromic subsequence, Fréchet distance... O(n ' ) SETH-hard n '+, [B.,Künnemann 15, Abboud,Backurs,V-Williams 15] bitonic TSP O(n ' ) O n log N n longest increasing subsequence, matrix chain multiplication... [de Berg,Buchin,Jansen,Woeginger 16] given n points in the plane, compute a minimum tour connecting all points among all tours consisting of two x-monotone parts

12 Showcase Results longest common subseq. edit distance, longest palindromic subsequence, Fréchet distance... O(n ' ) SETH-hard n '+, [B.,Künnemann 15, Abboud,Backurs,V-Williams 15] bitonic TSP O(n ' ) O n log N n longest increasing subsequence, matrix chain multiplication... [de Berg,Buchin,Jansen,Woeginger 16] maximum submatrix O(n 3 ) APSP hard n 3+, minimum weight triangle, graph centrality measures... [Backurs,Dikkala,Tzamos 16] given matrix A over Z, choose a submatrix (consisting of consecutive rows and columns of A) maximizing the sum of all entries

13 Showcase Results longest common subseq. edit distance, longest palindromic subsequence, Fréchet distance... O(n ' ) SETH-hard n '+, [B.,Künnemann 15, Abboud,Backurs,V-Williams 15] bitonic TSP O(n ' ) O n log N n longest increasing subsequence, matrix chain multiplication... [de Berg,Buchin,Jansen,Woeginger 16] maximum submatrix O(n 3 ) APSP hard n 3+, minimum weight triangle, graph centrality measures... [Backurs,Dikkala,Tzamos 16] colinearity motion planning, polygon containment... O(n ' ) 3SUM hard n '+, [Gajentaan,Overmars 95] given n points in the plane, are any three of them on a line?

14 Showcase Results longest common subseq. edit distance, longest palindromic subsequence, Fréchet distance... O(n ' ) SETH-hard n '+, [B.,Künnemann 15, Abboud,Backurs,V-Williams 15] bitonic TSP O(n ' ) O n log N n longest increasing subsequence, matrix chain multiplication... [de Berg,Buchin,Jansen,Woeginger 16] maximum submatrix O(n 3 ) APSP hard n 3+, minimum weight triangle, graph centrality measures... [Backurs,Dikkala,Tzamos 16] colinearity motion planning, polygon containment... O(n ' ) 3SUM hard n '+, [Gajentaan,Overmars 95] Open: optimal binary search tree O(n ' ) knapsack O(nW) many more...

15 Complexity Inside P SAT 2 n 3SUM n 2 OV n 2 Colinearity n 2 3SUM-hard APSP n 3 APSP equivalent Radius n 3 LCS n 2 SETH-hard Negative Triangle n 3 EDIT n 2 Fréchet n 2 classification of polynomial time problems diameter n 2 problem-centric view on complexity theory

16 Machine Model complexity theory is (to some extent) independent of the machine model but only up to polynomial factors we have to fix a machine model! Turing Machine: any (single-tape) Turing machine takes time Ω(n ' ) for recognizing palindromes abbcaacbba this does not apply to real computers!

17 Machine Model Random Access Machine (RAM): CPU memory of O(1) cells can perform all reasonable operations on two cells in O(1) time (arithmetic + logical operations ) can read/write any cell in O(1) time storage: cell / word consisting of Θ(log n) bits can recognize palindromes in time O(n) the details do not matter!

18 More Discussion What about unconditional lower bounds? no tools available beyond Ω(n log n) What if the hypotheses are wrong? NP-hardness was in the same situation 40 years ago relations between problems will stay suggests ways to attack a problem + which problems to attack

19 Conditional Lower Bounds allow to classify polynomial time problems are an analogue of NP-hardness yield good reasons to stop searching for faster algorithms should belong to the basic toolbox of theoretical computer scientists allow to search for new algorithms with better focus improve SAT before longest common subsequence non-matching lower bounds suggest better algorithms motivate new algorithms relax the problem and study approximation algorithms, parameterized running time,

20 Content of the Course we study core problems SAT, OV, APSP, 3SUM, and others conditional lower bounds: from each of these hypotheses algorithms: learn fastest known algorithms for core problems fine-grained complexity is a young field of research we will see many open problems & possibilities for BA/MA-theses

21 II. An Example for OV-hardness

22 Orthogonal Vectors Hypothesis Input: Task: Sets A, B 0,1 8 of size n Decide whether there are a A, b B such that a b 8 [ a D \ b D = 0 DF: for all 1 i d: a D = 0 or b D = 0 A = { 1,1,1, 1,1,0, 1,0,1, 0,0,1 } B = { 0,1,0, 0,1,1, 1,0,1, 1,1,1 } trivial O(n ' d) algorithm best known algorithm: O(n '+:/e(fgh *) ) where d = c log n [Lecture 03] OV-Hypothesis: no O(n '+, poly(d)) algorithm for any ε > 0 OV has no O(n '+, ) algorithm, even if d = polylog n

23 Graph Diameter Problem Input: Task: An unweighted graph G = (V, E) Compute the largest distance between any pair of vertices = max s,t u d v(u, v) diameter 2 Easy algorithm: Single-source-shortest-paths: Dijkstra s algorithm: O m + n log n All-pairs-shortest-paths: Dijkstra from every node: O n m + n log n O(n m log n) from this information we can compute the diameter in time O(n ' )

24 OV-Hardness Result OV sets A, B 0,1 8 of size n reduction time O(dn) diameter graph G O(n) vertices O(dn) edges O(n '+, poly(d)) algorithm O((nm) :+, ) algorithm Thm: Diameter has no O((nm) :+, ) algorithm unless the OV-Hypothesis fails. [Roditty,V-Williams 13]

25 Proof OV sets A, B 0,1 8 of size n reduction time O(dn) diameter graph G O(n) vertices O(dn) edges Proof: can assume: every vector has at least one 1 put edge if a D = d i put edge if b D = 1 d a, b = 2 a, b not orthogonal A a b B diameter = 3 there exists an orthogonal pair

26 Proof OV sets A, B 0,1 8 of size n reduction time O(dn) diameter graph G O(n) vertices O(dn) edges Remark: Even deciding whether the diameter is 2 or 3 has no O((nm) :+, ) algorithm unless OVH fails. There is no ( 3 ' ε)-approximation for the diameter in time O((nm) :+, ) unless OVH fails.

27 III. Another Example for OV-hardness

28 NFA Acceptance Problem nondeterministic finite automaton G accepts input string s if there is a walk in G from starting state to some accepting state, labelled with s 0,1 0,1 start 1 1 string: dynamic programming algorithm in time O( s G ): T i set of states reachable via walks labelled with s[1.. i] T 0 {starting state} T i v u T i 1 and transition u v labelled s i }

29 OV-Hardness Result OV sets A, B 0,1 8 of size n reduction time O(dn) NFA acceptance grammar G, string s G = O(dn) s = O(dn) O(n '+, poly(d)) algorithm O(( s G ) :+, ) algorithm Thm: NFA acceptance has no O(( s G ) :+, ) algorithm unless OVH fails. [Impagliazzo]

30 Proof OV sets A, B 0,1 8 of size n reduction time O(dn) NFA acceptance grammar G, string s G = O(dn) s = O(dn) Proof: fix some a A: in string s: 0011 fix some b B: in NFA G: = a : a ' a 8 if b D = 1 if b D = 0

31 Proof OV sets A, B 0,1 8 of size n reduction time O(dn) NFA acceptance grammar G, string s G = O(dn) s = O(dn) Proof: fix some a A: in string s: 0011 fix some b B: in NFA G: = a : a ' a 8 if b D = 1 if b D = 0

32 Proof fix some a A: in string s: 0011 fix some b B: in NFA G: string s = \$1100\$0110\$ \$0011\$ (for all a A) equivalent to OV instance size s = G = O(dn) NFA G: start 0,1, \$ \$ (for all b B) 0 1 \$ 0,1, \$ \$ \$ 1 1

33 VI. Four Russians

34 Method of Four Russians Arlazarov, Dinic, Kronrod, and Faradzev 1970 Algorithm for Boolean matrix multiplication Not all of them were Russian Better name: Four Soviets??

35 Boolean Matrix Multiplication Input: Boolean (0/1) matrices A and B Output: A B where + is OR and is AND A B C n = n

36 Naïve Algorithm A i, 1 B 1,j A i, k B k, j B Running time: time O n per inner product #inner products: n ' O n 3 total time A C C i, j = A i, k B k, j : E \$ Is there a k such that A i, k = 1 and B k, j = 1?

37 Main Idea Divide A into blocks of size t t We use t = 0.5 log n t Number of blocks: n ' = n' t log ' n t 1. Preprocess blocks and construct lookup table 2. Speed up naïve algorithm using lookup table

38 Preprocessing a Block X 1. For every t-dimensional vector v: precompute product X v v # inner products: t X O t ' time 2. Store results in lookup table: retrieve X v in time O(t) # t-dimensional vectors: 2 = 2. fgh\$ = n. Total time: \$ ' 2 t ' = n ' 2 = n '. = O \$ fgh \$ #blocks #vectors inner product

39 Multiplying with Lookup Table For each of output: 1 O(t) \$ times 2 O(t) \$ times O n n t n t t = O n3 t = O n3 log n

40 Summary Key property: small number of possible cell entries Main idea: speedup from lookup tables after preprocessing Discussion: In preprocessing: need to prepare for all t-dimensional input vectors Counter-intuitive at first Only some of the t-dimensional vectors might really appear in B # different t-dimensional vectors in general: 2 = n. # t-dimensional vectors per block in matrix multiplication: n Reusability outweighs preprocessing cost Four Russians in some sense a charging trick: Charge running time to the t-dim. vectors instead of the columns

41 Beyond Four Russians Very general technique: Matrix problems, dynamic programming problems (e.g. LCS) Shaving off logarithmic factors popular for certain problems Race to fastest algorithm Example: All-pairs shortest paths (APSP) Floyd-Warshall: O(n 3 ) State of the art: O \$ ' ( ª«) The Polynomial Method 2 fgh\$ grows faster than log * n for any constant c Ideas from circuit complexity We will teach it in May

42 Transitive Closure Problem Directed graph G, n nodes TC matrix TC i, j = 1 iff i can reach j Thm: BMM in time O(T n ) TC in time O(T n ) Naïve Algorithm: O n 3 by breadth-first search from each node

43 Reduction: From BMM (T (n)) to TC (T(n)) A B n 2-layered graph: edge from i to j iff A i, j = 1 i j TC = A B n T ± n = T 3n

44 Reduction: From TC (T (n)) to BMM (T(n)) A: I: adjacency matrix of G A i, j = 1 if and only if G has edge (i, j) identity matrix Fact: (A I) E i, j = 1 if and only if path from i to j of length at most k Fact: TC = (A I) \$ Repeated squaring: M ' = M M M N = M ' M ' M ³ = M N M N Lem: TC can be computed using log n Boolean matrix multiplications

45 Drawback of Reduction Lem: TC can be computed with log n Boolean matrix multiplications T ± n = T n log n = n3 log n log n = n3 Log-factor improvement of Four Russians is gone! Goal: Better reduction with T ± n = O(T n )

46 Reducing Problem to DAG I 1 III IV II 4 1. Compute strongly connected components (SCCs) i and j in same component iff i can reach j and j can reach i Suffices to solve problem on graph of SCCs Graph of SCCs is directed acyclic graph (DAG) 2. Compute topological order on DAG: edge i, j in DAG i j O(n ' )

47 Recursive Transitive Closure Input: DAG G with n nodes in topological order X: First n 2 edges in topological order Y: Last n RECURSE 2 edges in topological order M: Adjacency matrix of edges between X and Y X ± X Y Y ± TC G = TC X M TC(Y) Master Theorem* T ± n = 2T ± \$ ' + O T n = O(T n ) * 2T ± ct (n) for some 0 < c < 1

48 Summary BMM and TC have same asymptotic time complexity Same status: All-pairs shortest paths (APSP) and Min-plus matrix multiplication

### Complexity Theory of Polynomial-Time Problems

Complexity Theory of Polynomial-Time Problems Lecture 13: Recap, Further Directions, Open Problems Karl Bringmann I. Recap II. Further Directions III. Open Problems I. Recap Hard problems SAT: OV: APSP:

### Complexity Theory of Polynomial-Time Problems

Complexity Theory of Polynomial-Time Problems Lecture 11: Nondeterministic SETH Karl Bringmann Complexity Inside P SAT 2 n 3SUM n 2 OV n 2 Colinearity n 2 3SUM-hard APSP n 3 APSP equivalent Radius n 3

### Complexity Theory of Polynomial-Time Problems

Complexity Theory of Polynomial-Time Problems Lecture 2: SETH and OV Karl Bringmann Tutorial Slot Tuesday, 16:15-18:00 works for everybody? alternative time slot: Mo Tue Wed Thu Fri 10-12 12-14 new 14-16

### Complexity Theory of Polynomial-Time Problems

Complexity Theory of Polynomial-Time Problems Lecture 3: The polynomial method Part I: Orthogonal Vectors Sebastian Krinninger Organization of lecture No lecture on 26.05. (State holiday) 2 nd exercise

### Hardness for easy problems. An introduction

Hardness for easy problems An introduction The real world and hard problems I ve got data. I want to solve this algorithmic problem but I m stuck! Ok, thanks, I feel better that none of my attempts worked.

### Complexity Theory of Polynomial-Time Problems

Complexity Theory of Polynomial-Time Problems Lecture 5: Subcubic Equivalences Karl Bringmann Reminder: Relations = Reductions transfer hardness of one problem to another one by reductions problem P instance

### A FINE-GRAINED APPROACH TO

A FINE-GRAINED APPROACH TO ALGORITHMS AND COMPLEXITY Virginia Vassilevska Williams MIT ICDT 2018 THE CENTRAL QUESTION OF ALGORITHMS RESEARCH ``How fast can we solve fundamental problems, in the worst case?

### Limitations of Algorithm Power

Limitations of Algorithm Power Objectives We now move into the third and final major theme for this course. 1. Tools for analyzing algorithms. 2. Design strategies for designing algorithms. 3. Identifying

### Complexity: Some examples

Algorithms and Architectures III: Distributed Systems H-P Schwefel, Jens M. Pedersen Mm6 Distributed storage and access (jmp) Mm7 Introduction to security aspects (hps) Mm8 Parallel complexity (hps) Mm9

### Computational Complexity

Computational Complexity Algorithm performance and difficulty of problems So far we have seen problems admitting fast algorithms flow problems, shortest path, spanning tree... and other problems for which

### Complexity Theory of Polynomial-Time Problems

Complexity Theory of Polynomial-Time Problems Lecture 8: (Boolean) Matrix Multiplication Karl Bringmann Recall: Boolean Matrix Multiplication given n n matrices A, B with entries in {0,1} compute matrix

### Outline. Complexity Theory. Example. Sketch of a log-space TM for palindromes. Log-space computations. Example VU , SS 2018

Complexity Theory Complexity Theory Outline Complexity Theory VU 181.142, SS 2018 3. Logarithmic Space Reinhard Pichler Institute of Logic and Computation DBAI Group TU Wien 3. Logarithmic Space 3.1 Computational

### 6.S078 A FINE-GRAINED APPROACH TO ALGORITHMS AND COMPLEXITY LECTURE 1

6.S078 A FINE-GRAINED APPROACH TO ALGORITHMS AND COMPLEXITY LECTURE 1 Not a requirement, but fun: OPEN PROBLEM Sessions!!! (more about this in a week) 6.S078 REQUIREMENTS 1. Class participation: worth

### On some fine-grained questions in algorithms and complexity

On some fine-grained questions in algorithms and complexity Virginia Vassilevska Williams Massachusetts Institute of Technology, EECS and CSAIL Abstract In recent years, a new fine-grained theory of computational

### A FINE-GRAINED APPROACH TO ALGORITHMS AND COMPLEXITY. Virginia Vassilevska Williams Stanford University

A FINE-GRAINED APPROACH TO ALGORITHMS AND COMPLEXITY Virginia Vassilevska Williams Stanford University THE CENTRAL QUESTION OF ALGORITHMS RESEARCH ``How fast can we solve fundamental problems, in the worst

Advanced topic: Space complexity CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Chinese University of Hong Kong Fall 2016 1/28 Review: time complexity We have looked at how long it takes to

### CSE 200 Lecture Notes Turing machine vs. RAM machine vs. circuits

CSE 200 Lecture Notes Turing machine vs. RAM machine vs. circuits Chris Calabro January 13, 2016 1 RAM model There are many possible, roughly equivalent RAM models. Below we will define one in the fashion

### Informatique Fondamentale IMA S8

Informatique Fondamentale IMA S8 Cours 4 : graphs, problems and algorithms on graphs, (notions of) NP completeness Laure Gonnord http://laure.gonnord.org/pro/teaching/ Laure.Gonnord@polytech-lille.fr Université

### CS 350 Algorithms and Complexity

CS 350 Algorithms and Complexity Winter 2019 Lecture 15: Limitations of Algorithmic Power Introduction to complexity theory Andrew P. Black Department of Computer Science Portland State University Lower

### Unit 1A: Computational Complexity

Unit 1A: Computational Complexity Course contents: Computational complexity NP-completeness Algorithmic Paradigms Readings Chapters 3, 4, and 5 Unit 1A 1 O: Upper Bounding Function Def: f(n)= O(g(n)) if

### CS 350 Algorithms and Complexity

1 CS 350 Algorithms and Complexity Fall 2015 Lecture 15: Limitations of Algorithmic Power Introduction to complexity theory Andrew P. Black Department of Computer Science Portland State University Lower

### Mm7 Intro to distributed computing (jmp) Mm8 Backtracking, 2-player games, genetic algorithms (hps) Mm9 Complex Problems in Network Planning (JMP)

Algorithms and Architectures II H-P Schwefel, Jens M. Pedersen Mm6 Advanced Graph Algorithms (hps) Mm7 Intro to distributed computing (jmp) Mm8 Backtracking, 2-player games, genetic algorithms (hps) Mm9

### Connections between exponential time and polynomial time problem Lecture Notes for CS294

Connections between exponential time and polynomial time problem Lecture Notes for CS294 Lecturer: Russell Impagliazzo Scribe: Stefan Schneider November 10, 2015 We connect problems that have exponential

### CSC 1700 Analysis of Algorithms: Warshall s and Floyd s algorithms

CSC 1700 Analysis of Algorithms: Warshall s and Floyd s algorithms Professor Henry Carter Fall 2016 Recap Space-time tradeoffs allow for faster algorithms at the cost of space complexity overhead Dynamic

### Geometric Problems in Moderate Dimensions

Geometric Problems in Moderate Dimensions Timothy Chan UIUC Basic Problems in Comp. Geometry Orthogonal range search preprocess n points in R d s.t. we can detect, or count, or report points inside a query

### CSC 5170: Theory of Computational Complexity Lecture 4 The Chinese University of Hong Kong 1 February 2010

CSC 5170: Theory of Computational Complexity Lecture 4 The Chinese University of Hong Kong 1 February 2010 Computational complexity studies the amount of resources necessary to perform given computations.

### NP-completeness. Chapter 34. Sergey Bereg

NP-completeness Chapter 34 Sergey Bereg Oct 2017 Examples Some problems admit polynomial time algorithms, i.e. O(n k ) running time where n is the input size. We will study a class of NP-complete problems

### CSCI 1590 Intro to Computational Complexity

CSCI 59 Intro to Computational Complexity Overview of the Course John E. Savage Brown University January 2, 29 John E. Savage (Brown University) CSCI 59 Intro to Computational Complexity January 2, 29

### 1 Circuit Complexity. CS 6743 Lecture 15 1 Fall Definitions

CS 6743 Lecture 15 1 Fall 2007 1 Circuit Complexity 1.1 Definitions A Boolean circuit C on n inputs x 1,..., x n is a directed acyclic graph (DAG) with n nodes of in-degree 0 (the inputs x 1,..., x n ),

### MTAT Complexity Theory October 13th-14th, Lecture 6

MTAT.07.004 Complexity Theory October 13th-14th, 2011 Lecturer: Peeter Laud Lecture 6 Scribe(s): Riivo Talviste 1 Logarithmic memory Turing machines working in logarithmic space become interesting when

### 1. Introduction Recap

1. Introduction Recap 1. Tractable and intractable problems polynomial-boundness: O(n k ) 2. NP-complete problems informal definition 3. Examples of P vs. NP difference may appear only slightly 4. Optimization

### arxiv: v2 [cs.ds] 3 Oct 2017

Orthogonal Vectors Indexing Isaac Goldstein 1, Moshe Lewenstein 1, and Ely Porat 1 1 Bar-Ilan University, Ramat Gan, Israel {goldshi,moshe,porately}@cs.biu.ac.il arxiv:1710.00586v2 [cs.ds] 3 Oct 2017 Abstract

### Computational Complexity and Intractability: An Introduction to the Theory of NP. Chapter 9

1 Computational Complexity and Intractability: An Introduction to the Theory of NP Chapter 9 2 Objectives Classify problems as tractable or intractable Define decision problems Define the class P Define

### CSE 431/531: Analysis of Algorithms. Dynamic Programming. Lecturer: Shi Li. Department of Computer Science and Engineering University at Buffalo

CSE 431/531: Analysis of Algorithms Dynamic Programming Lecturer: Shi Li Department of Computer Science and Engineering University at Buffalo Paradigms for Designing Algorithms Greedy algorithm Make a

### CHAPTER 3 FUNDAMENTALS OF COMPUTATIONAL COMPLEXITY. E. Amaldi Foundations of Operations Research Politecnico di Milano 1

CHAPTER 3 FUNDAMENTALS OF COMPUTATIONAL COMPLEXITY E. Amaldi Foundations of Operations Research Politecnico di Milano 1 Goal: Evaluate the computational requirements (this course s focus: time) to solve

### Introduction to Complexity Theory

Introduction to Complexity Theory Read K & S Chapter 6. Most computational problems you will face your life are solvable (decidable). We have yet to address whether a problem is easy or hard. Complexity

### The space complexity of a standard Turing machine. The space complexity of a nondeterministic Turing machine

298 8. Space Complexity The space complexity of a standard Turing machine M = (Q,,,, q 0, accept, reject) on input w is space M (w) = max{ uav : q 0 w M u q av, q Q, u, a, v * } The space complexity of

### Lecture 6: Oracle TMs, Diagonalization Limits, Space Complexity

CSE 531: Computational Complexity I Winter 2016 Lecture 6: Oracle TMs, Diagonalization Limits, Space Complexity January 22, 2016 Lecturer: Paul Beame Scribe: Paul Beame Diagonalization enabled us to separate

### A Complexity Theory for Parallel Computation

A Complexity Theory for Parallel Computation The questions: What is the computing power of reasonable parallel computation models? Is it possible to make technology independent statements? Is it possible

### Lecture 4: NP and computational intractability

Chapter 4 Lecture 4: NP and computational intractability Listen to: Find the longest path, Daniel Barret What do we do today: polynomial time reduction NP, co-np and NP complete problems some examples

### 6.841/18.405J: Advanced Complexity Wednesday, February 12, Lecture Lecture 3

6.841/18.405J: Advanced Complexity Wednesday, February 12, 2003 Lecture Lecture 3 Instructor: Madhu Sudan Scribe: Bobby Kleinberg 1 The language MinDNF At the end of the last lecture, we introduced the

### Lecture 4. 1 Circuit Complexity. Notes on Complexity Theory: Fall 2005 Last updated: September, Jonathan Katz

Notes on Complexity Theory: Fall 2005 Last updated: September, 2005 Jonathan Katz Lecture 4 1 Circuit Complexity Circuits are directed, acyclic graphs where nodes are called gates and edges are called

### On the Space Complexity of Parameterized Problems

On the Space Complexity of Parameterized Problems Michael Elberfeld Christoph Stockhusen Till Tantau Institute for Theoretical Computer Science Universität zu Lübeck D-23538 Lübeck, Germany {elberfeld,stockhus,tantau}@tcs.uni-luebeck.de

### Complexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler

Complexity Theory Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität Wien 15 May, 2018 Reinhard

### Outline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181.

Complexity Theory Complexity Theory Outline Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität

### Umans Complexity Theory Lectures

Complexity Theory Umans Complexity Theory Lectures Lecture 1a: Problems and Languages Classify problems according to the computational resources required running time storage space parallelism randomness

### Speeding up the Four Russians Algorithm by About One More Logarithmic Factor

Speeding up the Four Russians Algorithm by About One More Logarithmic Factor Timothy M. Chan September 19, 2014 Abstract We present a new combinatorial algorithm for Boolean matrix multiplication that

### Limits to Approximability: When Algorithms Won't Help You. Note: Contents of today s lecture won t be on the exam

Limits to Approximability: When Algorithms Won't Help You Note: Contents of today s lecture won t be on the exam Outline Limits to Approximability: basic results Detour: Provers, verifiers, and NP Graph

### NP-complete problems. CSE 101: Design and Analysis of Algorithms Lecture 20

NP-complete problems CSE 101: Design and Analysis of Algorithms Lecture 20 CSE 101: Design and analysis of algorithms NP-complete problems Reading: Chapter 8 Homework 7 is due today, 11:59 PM Tomorrow

### arxiv: v1 [cs.ds] 27 Mar 2017

Tree Edit Distance Cannot be Computed in Strongly Subcubic Time (unless APSP can) Karl Bringmann Paweł Gawrychowski Shay Mozes Oren Weimann arxiv:1703.08940v1 [cs.ds] 27 Mar 2017 Abstract The edit distance

### NP-Completeness. Algorithmique Fall semester 2011/12

NP-Completeness Algorithmique Fall semester 2011/12 1 What is an Algorithm? We haven t really answered this question in this course. Informally, an algorithm is a step-by-step procedure for computing a

### Tree Edit Distance Cannot be Computed in Strongly Subcubic Time (unless APSP can)

Tree Edit Distance Cannot be Computed in Strongly Subcubic Time (unless APSP can) Karl Bringmann Paweł Gawrychowski Shay Mozes Oren Weimann Abstract The edit distance between two rooted ordered trees with

### Final exam study sheet for CS3719 Turing machines and decidability.

Final exam study sheet for CS3719 Turing machines and decidability. A Turing machine is a finite automaton with an infinite memory (tape). Formally, a Turing machine is a 6-tuple M = (Q, Σ, Γ, δ, q 0,

### The Complexity Classes P and NP. Andreas Klappenecker [partially based on slides by Professor Welch]

The Complexity Classes P and NP Andreas Klappenecker [partially based on slides by Professor Welch] P Polynomial Time Algorithms Most of the algorithms we have seen so far run in time that is upper bounded

### Easy Problems vs. Hard Problems. CSE 421 Introduction to Algorithms Winter Is P a good definition of efficient? The class P

Easy Problems vs. Hard Problems CSE 421 Introduction to Algorithms Winter 2000 NP-Completeness (Chapter 11) Easy - problems whose worst case running time is bounded by some polynomial in the size of the

### Undecidable Problems. Z. Sawa (TU Ostrava) Introd. to Theoretical Computer Science May 12, / 65

Undecidable Problems Z. Sawa (TU Ostrava) Introd. to Theoretical Computer Science May 12, 2018 1/ 65 Algorithmically Solvable Problems Let us assume we have a problem P. If there is an algorithm solving

### Limits of Feasibility. Example. Complexity Relationships among Models. 1. Complexity Relationships among Models

Limits of Feasibility Wolfgang Schreiner Wolfgang.Schreiner@risc.jku.at Research Institute for Symbolic Computation (RISC) Johannes Kepler University, Linz, Austria http://www.risc.jku.at 1. Complexity

### Chapter 2: The Basics. slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens

Chapter 2: The Basics slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens Problem instances vs. decision problems vs. search problems Decision

### CS151 Complexity Theory. Lecture 1 April 3, 2017

CS151 Complexity Theory Lecture 1 April 3, 2017 Complexity Theory Classify problems according to the computational resources required running time storage space parallelism randomness rounds of interaction,

### CS 241 Analysis of Algorithms

CS 241 Analysis of Algorithms Professor Eric Aaron Lecture T Th 9:00am Lecture Meeting Location: OLB 205 Business Grading updates: HW5 back today HW7 due Dec. 10 Reading: Ch. 22.1-22.3, Ch. 25.1-2, Ch.

### Algorithms and Theory of Computation. Lecture 22: NP-Completeness (2)

Algorithms and Theory of Computation Lecture 22: NP-Completeness (2) Xiaohui Bei MAS 714 November 8, 2018 Nanyang Technological University MAS 714 November 8, 2018 1 / 20 Set Cover Set Cover Input: a set

### Lecture 22: Counting

CS 710: Complexity Theory 4/8/2010 Lecture 22: Counting Instructor: Dieter van Melkebeek Scribe: Phil Rydzewski & Chi Man Liu Last time we introduced extractors and discussed two methods to construct them.

### Models of Computation

Models of Computation Analysis of Algorithms Week 1, Lecture 2 Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Models of Computation (RAM) a) Random Access Machines

### Lecture 19: Finish NP-Completeness, conp and Friends

6.045 Lecture 19: Finish NP-Completeness, conp and Friends 1 Polynomial Time Reducibility f : Σ* Σ* is a polynomial time computable function if there is a poly-time Turing machine M that on every input

### A New Combinatorial Approach For Sparse Graph Problems

A New Combinatorial Approach For Sparse Graph Problems Guy E. Blelloch, Virginia Vassilevska, and Ryan Williams Computer Science Department, Carnegie Mellon University, Pittsburgh, PA {guyb,virgi,ryanw}@cs.cmu.edu

### The Polynomial Hierarchy

The Polynomial Hierarchy Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Motivation..synthesizing circuits is exceedingly difficulty. It is even

### Combinatorial Optimization

Combinatorial Optimization Problem set 8: solutions 1. Fix constants a R and b > 1. For n N, let f(n) = n a and g(n) = b n. Prove that f(n) = o ( g(n) ). Solution. First we observe that g(n) 0 for all

### Definition: Alternating time and space Game Semantics: State of machine determines who

CMPSCI 601: Recall From Last Time Lecture Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins

### Definition: Alternating time and space Game Semantics: State of machine determines who

CMPSCI 601: Recall From Last Time Lecture 3 Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins

### NP-Completeness. Until now we have been designing algorithms for specific problems

NP-Completeness 1 Introduction Until now we have been designing algorithms for specific problems We have seen running times O(log n), O(n), O(n log n), O(n 2 ), O(n 3 )... We have also discussed lower

### ECS 120 Lesson 23 The Class P

ECS 120 Lesson 23 The Class P Oliver Kreylos Wednesday, May 23th, 2001 We saw last time how to analyze the time complexity of Turing Machines, and how to classify languages into complexity classes. We

### INAPPROX APPROX PTAS. FPTAS Knapsack P

CMPSCI 61: Recall From Last Time Lecture 22 Clique TSP INAPPROX exists P approx alg for no ε < 1 VertexCover MAX SAT APPROX TSP some but not all ε< 1 PTAS all ε < 1 ETSP FPTAS Knapsack P poly in n, 1/ε

### Parallelism and Machine Models

Parallelism and Machine Models Andrew D Smith University of New Brunswick, Fredericton Faculty of Computer Science Overview Part 1: The Parallel Computation Thesis Part 2: Parallelism of Arithmetic RAMs

### Intractable Problems [HMU06,Chp.10a]

Intractable Problems [HMU06,Chp.10a] Time-Bounded Turing Machines Classes P and NP Polynomial-Time Reductions A 10 Minute Motivation https://www.youtube.com/watch?v=yx40hbahx3s 1 Time-Bounded TM s A Turing

### Logarithmic space. Evgenij Thorstensen V18. Evgenij Thorstensen Logarithmic space V18 1 / 18

Logarithmic space Evgenij Thorstensen V18 Evgenij Thorstensen Logarithmic space V18 1 / 18 Journey below Unlike for time, it makes sense to talk about sublinear space. This models computations on input

### NP-Completeness. Andreas Klappenecker. [based on slides by Prof. Welch]

NP-Completeness Andreas Klappenecker [based on slides by Prof. Welch] 1 Prelude: Informal Discussion (Incidentally, we will never get very formal in this course) 2 Polynomial Time Algorithms Most of the

### Nondeterministic Polynomial Time

Nondeterministic Polynomial Time 11/1/2016 Discrete Structures (CS 173) Fall 2016 Gul Agha Slides based on Derek Hoiem, University of Illinois 1 2016 CS Alumni Awards Sohaib Abbasi (BS 78, MS 80), Chairman

### A An Overview of Complexity Theory for the Algorithm Designer

A An Overview of Complexity Theory for the Algorithm Designer A.1 Certificates and the class NP A decision problem is one whose answer is either yes or no. Two examples are: SAT: Given a Boolean formula

### Space Complexity. The space complexity of a program is how much memory it uses.

Space Complexity The space complexity of a program is how much memory it uses. Measuring Space When we compute the space used by a TM, we do not count the input (think of input as readonly). We say that

### Acknowledgments 2. Part 0: Overview 17

Contents Acknowledgments 2 Preface for instructors 11 Which theory course are we talking about?.... 12 The features that might make this book appealing. 13 What s in and what s out............... 14 Possible

### P is the class of problems for which there are algorithms that solve the problem in time O(n k ) for some constant k.

Complexity Theory Problems are divided into complexity classes. Informally: So far in this course, almost all algorithms had polynomial running time, i.e., on inputs of size n, worst-case running time

### Yale university technical report #1402.

The Mailman algorithm: a note on matrix vector multiplication Yale university technical report #1402. Edo Liberty Computer Science Yale University New Haven, CT Steven W. Zucker Computer Science and Appled

### Notes for Lecture 3... x 4

Stanford University CS254: Computational Complexity Notes 3 Luca Trevisan January 18, 2012 Notes for Lecture 3 In this lecture we introduce the computational model of boolean circuits and prove that polynomial

### COMP/MATH 300 Topics for Spring 2017 June 5, Review and Regular Languages

COMP/MATH 300 Topics for Spring 2017 June 5, 2017 Review and Regular Languages Exam I I. Introductory and review information from Chapter 0 II. Problems and Languages A. Computable problems can be expressed

### CSE 105 THEORY OF COMPUTATION

CSE 105 THEORY OF COMPUTATION "Winter" 2018 http://cseweb.ucsd.edu/classes/wi18/cse105-ab/ Today's learning goals Sipser Ch 7 Distinguish between computability and complexity Articulate motivation questions

### 15.083J/6.859J Integer Optimization. Lecture 2: Efficient Algorithms and Computational Complexity

15.083J/6.859J Integer Optimization Lecture 2: Efficient Algorithms and Computational Complexity 1 Outline Efficient algorithms Slide 1 Complexity The classes P and N P The classes N P-complete and N P-hard

### Reductions. Reduction. Linear Time Reduction: Examples. Linear Time Reductions

Reduction Reductions Problem X reduces to problem Y if given a subroutine for Y, can solve X. Cost of solving X = cost of solving Y + cost of reduction. May call subroutine for Y more than once. Ex: X

### Lecture 14 - P v.s. NP 1

CME 305: Discrete Mathematics and Algorithms Instructor: Professor Aaron Sidford (sidford@stanford.edu) February 27, 2018 Lecture 14 - P v.s. NP 1 In this lecture we start Unit 3 on NP-hardness and approximation

### Show that the following problems are NP-complete

Show that the following problems are NP-complete April 7, 2018 Below is a list of 30 exercises in which you are asked to prove that some problem is NP-complete. The goal is to better understand the theory

### 1 Review of Vertex Cover

CS266: Parameterized Algorithms and Complexity Stanford University Lecture 3 Tuesday, April 9 Scribe: Huacheng Yu Spring 2013 1 Review of Vertex Cover In the last lecture, we discussed FPT algorithms for

### Simulating Branching Programs with Edit Distance and Friends

Simulating Branching Programs with Edit Distance and Friends Or: A Polylog Shaved Is a Lower Bound Made Amir Abboud Stanford University, USA abboud@cs.stanford.edu Thomas Dueholm Hansen Aarhus University,

### Lecture 8: Alternatation. 1 Alternate Characterizations of the Polynomial Hierarchy

CS 710: Complexity Theory 10/4/2011 Lecture 8: Alternatation Instructor: Dieter van Melkebeek Scribe: Sachin Ravi In this lecture, we continue with our discussion of the polynomial hierarchy complexity

### CP405 Theory of Computation

CP405 Theory of Computation BB(3) q 0 q 1 q 2 0 q 1 1R q 2 0R q 2 1L 1 H1R q 1 1R q 0 1L Growing Fast BB(3) = 6 BB(4) = 13 BB(5) = 4098 BB(6) = 3.515 x 10 18267 (known) (known) (possible) (possible) Language:

### Lecture Notes Each circuit agrees with M on inputs of length equal to its index, i.e. n, x {0, 1} n, C n (x) = M(x).

CS 221: Computational Complexity Prof. Salil Vadhan Lecture Notes 4 February 3, 2010 Scribe: Jonathan Pines 1 Agenda P-/NP- Completeness NP-intermediate problems NP vs. co-np L, NL 2 Recap Last time, we

### Discrete Optimization 2010 Lecture 10 P, N P, and N PCompleteness

Discrete Optimization 2010 Lecture 10 P, N P, and N PCompleteness Marc Uetz University of Twente m.uetz@utwente.nl Lecture 9: sheet 1 / 31 Marc Uetz Discrete Optimization Outline 1 N P and co-n P 2 N P-completeness

### Computer Sciences Department

Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y

### Lecture Notes 4. Issued 8 March 2018

CM30073 Advanced Algorithms and Complexity 1. Structure of the class NP Lecture Notes 4 Issued 8 March 2018 Recall that it is not known whether or not P = NP, the widely accepted hypothesis being that

### Pr[X = s Y = t] = Pr[X = s] Pr[Y = t]

Homework 4 By: John Steinberger Problem 1. Recall that a real n n matrix A is positive semidefinite if A is symmetric and x T Ax 0 for all x R n. Assume A is a real n n matrix. Show TFAE 1 : (a) A is positive