Preliminaries and Complexity Theory

Size: px
Start display at page:

Download "Preliminaries and Complexity Theory"

Transcription

1 Preliminaries and Complexity Theory Oleksandr Romanko CAS Advanced Topics in Combinatorial Optimization McMaster University, January 16, 2006

2 Introduction Book structure: 2

3 Part I Linear Algebra : Introduction: Book Structure theory of linear spaces and of systems of linear equations complexity of solving linear systems sizes of solutions and the running time of the Gaussian elimination method which is a polynomial-time one (its running time is bounded by a polynomial in the size of the input data) Part II Lattices and Linear Diophantine Equations : solving systems of linear diophantine equations (solve systems of linear equations in integer variables) Hermite normal form problem of diophantine approximation (the continued fraction method approximates a real number by a rational number with low denominator) 3

4 Introduction: Book Structure Part III Polyhedra, Linear Inequalities, and Linear Programming : solving systems of linear inequalities solving systems of linear equations in nonnegative variables solving linear programming problems Polynomial-time Part IV Integer Linear Programming : solving systems of linear diophantine inequalities, i.e. solving linear inequalities in integers solving systems of linear equations in nonnegative integer variables solving integer linear programming problems NP-complete 4

5 If a is a real number, then General Preliminaries denote the lower integer part and the upper integer part of α. Notation: - sets of integers, rationals, and real numbers - sets of nonnegative integers, rationals, and real numbers Notation: Greatest common divisor: If are rational numbers (not all equal to 0) then the largest rational number dividing each of exists, and is called the greatest common divisor or g.c.d. The numbers are relatively prime if 5

6 General Preliminaries Big O notation is used to describe the asymptotic behavior of functions - an asymptotic upper bound for the magnitude of a function in terms of another, usually simpler, function Optimization problem: 6

7 Preliminaries: Linear Algebra, Linear equations and inequalities: Matrix Theory, Geometry A system of linear inequalities can have several alternative forms: 7

8 Euclidian norm: Preliminaries: Linear Algebra, Matrix Theory, Geometry Euclidian ball: Other norms: 8

9 Matrices: Preliminaries: Linear Algebra, Row (column) rank Row (column) submatrix Upper and low triangular matrices Diagonal matrices Matrix Theory, Geometry Incidence vector (characteristic vector) of a set T S (universe S) is a {0,1} vector χ T whose entries are labeled with the members of S: The support of a vector is the set of coordinates at which the vector is non-zero 9

10 Preliminaries: Linear Algebra, A set C of vectors is convex if it satisfies: Matrix Theory, Geometry Linear hull, affine hull and convex hull of a set X of vectors. Convex hull is the smallest convex set containing X. A (convex) cone is a nonempty set of vectors C satisfying The cone generated by a set X of vectors is the smallest convex cone containing X 10

11 Preliminaries: Pivoting 11

12 Preliminaries: Some Inequalities Cauchy-Schwarz inequality: Hadamard inequality (B is the m x n matrix consisting of column vectors b 1,,b m ): 12

13 Preliminaries: Positive Definite Matrices A real-valued matrix D is called positive definite if D is symmetric and its eigenvalues all are positive. The following are equivalent for a realvalued symmetric matrix D: Let D be a positive definite matrix, and let D = C T C, for nonsingular C. Two vectors c and d are orthogonal (relative to the inner product defined by D) if c T Dd = 0. The norm. defined by D is given by 13

14 Preliminaries: Balls and Ellipsoids 14

15 Graph Theory: Undirected Graphs Undirected graph is a pair G = (V, E), where V is a finite set, and E is a family of unordered pairs of elements of V. The elements of V are called the vertices or points of G, and the elements of E are called the edges or lines of G. A pair occurring more than once in E is a multiple edge. Graphs without multiple edges are called simple. Loops are edges of the form {v,v} We say that an edge {v,w} connects the vertices v and w The vertices v and w are adjacent if there is an edge connecting v and w The edge {v,w} is incident with the vertex v and w, and conversely The vertices v and w are called the ends of the edge {v,w} The number of edges incident with a vertex v is called the valency or degree of, usually denoted by d G (v). The maximum and minimum degree of the vertices of G are denoted by (G) and δ(g) 15

16 Graph Theory: Undirected Graphs Example: G=(V,E), V(G)={u,v,w,x} and E(G)={{u,v}.{v,w},{v,x},{w,x}} e u v w f v and w are the ends (endpoints) of the edge e={v,w} e is incident with both v and w w is incident with the edges e and f={w,x} The vertex x is adjacent to the vertices v and w The edge {v,w} is adjacent to the edges {u,v}, {v,w} and {w,x} The degree of v is d G (v)=3 The maximum and minimum degree of the vertices of G are (G)=3 and δ(g)=1 x 16

17 Graph Theory: Undirected Graphs The complementary graph of G is the simple graph with the same vertex set as G, and with edges all pairs {v, w} of vertices which are not in E. A simple graph is complete if E is the set of all pairs of vertices (K n ). A graph G = (V,E ) is a subgraph of G = (V,E) if V V and E E. A planar graph is one that can be drawn on a plane in such a way that there are no "edge crossings," i.e. edges intersect only at their common vertices. Example: K 4 is complete planar graph, K 5 and K 3,3 are complete nonplanar graphs K 4 K 5 K 3,3 17

18 Graph Theory: Undirected Graphs A path in the graph G=(V,E) from v 0 to v t is an alternating sequence of vertices and edges of the form If v 0 = v t, we have closed path. A graph is connected if each two vertices of the graph are connected by a path 18

19 Graph Theory: Undirected Graphs Algorithm for finding the distance between vertices r and s. It consists of recursively determining the sets V 0, V 1, V 2,..., where V i, is the set of vertices v of distance i to r : 19

20 Graph Theory: Undirected Graphs A closed path (v 0 = v t ) of length at least one and without repeated edges or vertices (except for the end points) is called a circuit. A Hamilton circuit is a circuit containing each vertex of the graph exactly once (except for the end points). Famous traveling salesman problem (TSP): given a graph G = (V,E) and a 'length' function f, find a Hamilton circuit of minimum length. (The length of a circuit is the sum of the lengths of its edges.) 20

21 Graph Theory: Undirected Graphs A forest is a graph having no circuits, and a tree is a connected forest Equivalently, a tree is a graph in which any two vertices are connected by exactly one path. A forest is a graph in which any two vertices are connected by at most one path. Forests Trees 21

22 Graph Theory: Undirected Graphs A clique (coclique, respectively) in the graph G = (V,E) is a set of pairwise adjacent (nonadjacent, respectively) vertices of G. The (maximum) clique problem is the problem: given a graph G = (V,E), find a clique of maximum size A clique of a graph is its maximal complete subgraph!!! A matching is a set of pairwise disjoint edges (a set of edges of G such that no two of them share a vertex in common). A matching covering all vertices is called a perfect matching. 22

23 Graph Theory: Undirected Graphs A graph G = (V,E) is called bipartite if V can be partitioned into two classes V 1 and V 2 such that each edge of G contains a vertex in V 1 and a vertex in V 2. The sets V 1 and V 2 are called colour classes. A complete bipartite graph is a simple bipartite graph, with colour classes V 1 and V 2, so that each vertex in V 1, is adjacent to each vertex in V 2. A complete bipartite graph with colour classes of sizes m and n is denoted by K m,n. K 3,3 23

24 Graph Theory: Undirected Graphs The adjacency matrix of an undirected graph G = (V,E) is the matrix with both rows and columns indexed by V, where the entry in position (v, w) is the number of edges connecting v and w. So G is simple if and only if its adjacency matrix is a {0,1}-matrix. The incidence matrix of G is the {0, 1}-matrix with rows and columns indexed by V and E, respectively, where the entry in position (v, e) is 1 if and only if vertex v is incident with edge e. 24

25 Graph Theory: Directed Graphs A directed graph or a digraph is a pair D = (V, A), where V is a finite set, and A is a finite family of ordered pairs of elements of V. The elements of V are called the vertices or points, and the elements of E are called the arcs of D. The vertices v and w are called the tail and the head of the arc (v, w), respectively. Arc (v, w) enters w and leaves v If W is a set of vertices such that v W and w W, then (v,w) is said to enter W and to leave V\ W. If W V, then δ A (W) or δ (W) denotes the set of arcs in A entering W, and δ + A (W) or δ+ (W) denotes the set of arcs in A leaving W 25

26 Graph Theory: Directed Graphs Given a digraph D = (V, A) and vertices r,s V, a flow from r to s, or an r-s-flow, is a function x:a R satisfying: The second set of constraints here means that at any vertex v r, s, the total 'amount' of flow entering v is equal to the total 'amount' of flow leaving v the flow conservation law. The value of the flow is the net amount of flow leaving r is equal to the net amount of flow entering s. Flow x is said to be subject to a given 'capacity' function c:a R + if x(a) c(a) for each arc a. A circulation is a flow for which the flow conservation law holds at each vertex (i.e. it is an r-s-flow of value 0, or an r-r-flow, for arbitrary r and s). 26

27 Graph Theory: Directed Graphs An undirected path (undirected circuit, respectively) for a directed graph is a path (circuit, respectively) in the underlying undirected graph. An undirected path or circuit in a directed graph has forward arcs and backward arcs. A digraph D = (V, A) is called strongly connected if for each two vertices r and s of D there is a directed r-s-path. D is (weakly) connected if its underlying undirected graph is connected. The incidence matrix of a digraph D = (V, A) is the matrix with rows and columns indexed by V and A, respectively, where the entry in position (v,a) is -1, +1, or 0, if vertex v is the head of a, the tail of a, or neither, respectively. 27

28 Problems, Algorithms and Complexity: Letters, Words and Sizes Need symbols and strings of symbols for formalizing problem complexity. Let Σ be a finite set (often Σ = {0,1}). Σ is called the alphabet and its elements are called symbols or letters. An ordered finite sequence of symbols from Σ is called a string {of symbols) or a word. Σ* stands for the collection of all strings of symbols from Σ. The size of a string is the number of its components. The string of size 0 is the empty string, denoted by. Strings can have the form of (finite sequences of) rational numbers, vectors, etc. Standard way of encoding strings of symbols is binary notation (fixed alphabet like {0,1}). Fix the sizes of a rational number (where p and q are relatively prime integers), of a rational vector and of a rational matrix are 28

29 Problems, Algorithms and Complexity: Letters, Words and Sizes Strings can have the form of (finite sequences of) rational numbers, vectors, etc. Standard way of encoding strings of symbols is binary notation (fixed alphabet like {0,1}). Fix the sizes of a rational number (where p and q are relatively prime integers), of a rational vector and of a rational matrix are 29

30 Problems, Algorithms and Complexity: Problems A problem can have the form of a question or a task. Mathematically, a (search) problem is a subset Π of Σ* Σ*, where Σ is some alphabet. The corresponding problem then is: Here the string z is called an instance or the input of the problem, and y is a solution or the output. Problem Π is called a decision problem or a yes/no problem if, for each (z,y) in Π, y is the empty string. In that case, the problem is often identified with the set if of strings z in Σ* for which (z, ) belongs to Π. The problem can have form like 30

31 Problems, Algorithms and Complexity: Examples of search problems: Problems Corresponding informal forms of these problems: 31

32 Problems, Algorithms and Complexity: Algorithms and Running Time An algorithm can be formalized in terms of Turing Machine (TM). For a given input z Σ*, an algorithm for problem Π Σ* Σ* determines an output y such that (z,y) is in Π, or stops without delivering an output if there exists no such y. Mathematically, an algorithm is a finite string A of 0's and 1's. A solves problem Π or A is an algorithm for Π, if for any instance z of Π, when giving the string (A,z) to a 'universal Turing machine', the machine stops after a finite number of steps, while delivering a string y with (z,y) Π, or delivering no string in the case where such a string y does not exist. Running time - the number of 'elementary bit operations' in the execution of an algorithm. Mathematically, the running time of an algorithm A for a certain problem instance z is the number of moves the 'head' of a universal Turing machine makes before stopping when given the input (A,z). Define the running time (function) of an algorithm A as the function 32

33 Problems, Algorithms and Complexity: Polynomial Algorithms Complicated definition A function f(n) is said to be polynomially bounded if f(n) = O(n k ). Easier definition 33

34 Problems, Algorithms and Complexity: Polynomial Algorithms A function f(n) is said to be polynomially bounded if f(n) = O(n k ). An algorithm is polynomial-time or polynomial if its running time function is polynomially bounded. A problem is said to be solvable in polynomial time or polynomially solvable if the problem can be solved by a polynomial-time algorithm. We are interested mostly in the asymptotic behavior of the running time of an algorithm. Therefore, we say that the running time is O(g(σ)), for some function g(σ), meaning that there is a constant C such that the running time is upper bounded by Cg(σ). The elementary arithmetic operations are: adding, subtracting, multiplying, dividing, and comparing numbers. In rational arithmetic, they can be executed by polynomial-time algorithms. Therefore, for deriving the polynomiality of an algorithm which performs a sequence of elementary arithmetic operations, it suffices to show that the total number of these operations is polynomially bounded by the size of the input, and that the sizes of the intermediate numbers to which these 34 elementary arithmetic operations are applied are polynomially

35 Problems, Algorithms and Complexity: the Classes P, NP and co-np The class of decision problems solvable in polynomial time is denoted by P. Another, possibly larger, 'complexity class' is the class NP. Informally, the class NP can be described as the class of decision problems satisfying Formally, a decision problem belongs to NP if there exists a polynomially solvable decision problem and a polynomial such that for each z in Σ*: 35

36 Problems, Algorithms and Complexity: the Classes P, NP and co-np Formally, a decision problem belongs to NP if there exists a polynomially solvable decision problem and a polynomial such that for each z in Σ*: Interpretation: y here fulfils the role of a 'polynomial-length proof of the fact that z is in. This 'proof can be checked in polynomial time, as is polynomially solvable. It is not required that y in is to be found in polynomial time. The string y can be seen as a 'certificate' for z, which can be given to the 'boss' in order to convince him that z is in. To test whether z belongs to, we could 'guess' a string y such that (z,y) is in. Since there are at most such guesses, it follows that if is in NP, there is a polynomial and an algorithm which solves the problem in time, where is the input size. 36

37 Problems, Algorithms and Complexity: the Classes P, NP and co-np The complement of a decision problem is the decision problem. The class of decision problems whose complement is in NP is denoted by co-np. So, co-np consists of those decision problems for which the fact that a certain string z is not in has a proof of length polynomially bounded by size(z). Since the complement of every polynomially solvable decision problem is polynomially solvable again, we know that P co-np, and hence P NP co-np. The class NP co-np consists of those decision problems for which both a positive answer and a negative answer have a proof of polynomial length. That is, it consists of all problems for which there exist polynomially solvable decision problem and a polynomial, such that for each string z Σ*: 37

38 Problems, Algorithms and Complexity: the Classes P, NP and co-np It is as yet a big open problem in complexity theory whether the class P is equal to the class NP. It is generally believed that P NP. The traveling salesman problem is one of the many problems in NP for which no polynomial-time algorithm has been found as yet. Most of these problems turned out to belong to the hardest in NP, the socalled NP-complete problems. Similarly, it is unknown whether NP = co-np or P = NP co-np. Thus we have the following Venn diagram: 38

39 Problems, Algorithms and Complexity: NP-Complete Problems Certain problems in the class NP are the hardest among all problems in NP. Among these hardest problems are the integer linear programming problem, the traveling salesman problem, and the maximum clique problem. A decision problem is called Karp-reducible to decision problem if there is a polynomial-time algorithm A such that for any input string z in Σ*, A delivers a string x as output, such If is polynomially solvable, and is Karp-reducible to, then also is polynomially solvable. Similarly, if belongs to NP, and is Karp-reducible to, then also belongs to NP. The same applies to co-np. 39

40 Problems, Algorithms and Complexity: NP-Complete Problems 40

Combinatorial Optimization

Combinatorial Optimization Combinatorial Optimization Problem set 8: solutions 1. Fix constants a R and b > 1. For n N, let f(n) = n a and g(n) = b n. Prove that f(n) = o ( g(n) ). Solution. First we observe that g(n) 0 for all

More information

In complexity theory, algorithms and problems are classified by the growth order of computation time as a function of instance size.

In complexity theory, algorithms and problems are classified by the growth order of computation time as a function of instance size. 10 2.2. CLASSES OF COMPUTATIONAL COMPLEXITY An optimization problem is defined as a class of similar problems with different input parameters. Each individual case with fixed parameter values is called

More information

Acyclic Digraphs arising from Complete Intersections

Acyclic Digraphs arising from Complete Intersections Acyclic Digraphs arising from Complete Intersections Walter D. Morris, Jr. George Mason University wmorris@gmu.edu July 8, 2016 Abstract We call a directed acyclic graph a CI-digraph if a certain affine

More information

CS 320, Fall Dr. Geri Georg, Instructor 320 NP 1

CS 320, Fall Dr. Geri Georg, Instructor 320 NP 1 NP CS 320, Fall 2017 Dr. Geri Georg, Instructor georg@colostate.edu 320 NP 1 NP Complete A class of problems where: No polynomial time algorithm has been discovered No proof that one doesn t exist 320

More information

NP-completeness. Chapter 34. Sergey Bereg

NP-completeness. Chapter 34. Sergey Bereg NP-completeness Chapter 34 Sergey Bereg Oct 2017 Examples Some problems admit polynomial time algorithms, i.e. O(n k ) running time where n is the input size. We will study a class of NP-complete problems

More information

Combinatorial optimization problems

Combinatorial optimization problems Combinatorial optimization problems Heuristic Algorithms Giovanni Righini University of Milan Department of Computer Science (Crema) Optimization In general an optimization problem can be formulated as:

More information

Computability and Complexity Theory: An Introduction

Computability and Complexity Theory: An Introduction Computability and Complexity Theory: An Introduction meena@imsc.res.in http://www.imsc.res.in/ meena IMI-IISc, 20 July 2006 p. 1 Understanding Computation Kinds of questions we seek answers to: Is a given

More information

ACYCLIC DIGRAPHS GIVING RISE TO COMPLETE INTERSECTIONS

ACYCLIC DIGRAPHS GIVING RISE TO COMPLETE INTERSECTIONS ACYCLIC DIGRAPHS GIVING RISE TO COMPLETE INTERSECTIONS WALTER D. MORRIS, JR. ABSTRACT. We call a directed acyclic graph a CIdigraph if a certain affine semigroup ring defined by it is a complete intersection.

More information

CS3719 Theory of Computation and Algorithms

CS3719 Theory of Computation and Algorithms CS3719 Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational tool - analysis

More information

CS6901: review of Theory of Computation and Algorithms

CS6901: review of Theory of Computation and Algorithms CS6901: review of Theory of Computation and Algorithms Any mechanically (automatically) discretely computation of problem solving contains at least three components: - problem description - computational

More information

CHAPTER 3 FUNDAMENTALS OF COMPUTATIONAL COMPLEXITY. E. Amaldi Foundations of Operations Research Politecnico di Milano 1

CHAPTER 3 FUNDAMENTALS OF COMPUTATIONAL COMPLEXITY. E. Amaldi Foundations of Operations Research Politecnico di Milano 1 CHAPTER 3 FUNDAMENTALS OF COMPUTATIONAL COMPLEXITY E. Amaldi Foundations of Operations Research Politecnico di Milano 1 Goal: Evaluate the computational requirements (this course s focus: time) to solve

More information

NP-Complete Problems and Approximation Algorithms

NP-Complete Problems and Approximation Algorithms NP-Complete Problems and Approximation Algorithms Efficiency of Algorithms Algorithms that have time efficiency of O(n k ), that is polynomial of the input size, are considered to be tractable or easy

More information

Technische Universität München, Zentrum Mathematik Lehrstuhl für Angewandte Geometrie und Diskrete Mathematik. Combinatorial Optimization (MA 4502)

Technische Universität München, Zentrum Mathematik Lehrstuhl für Angewandte Geometrie und Diskrete Mathematik. Combinatorial Optimization (MA 4502) Technische Universität München, Zentrum Mathematik Lehrstuhl für Angewandte Geometrie und Diskrete Mathematik Combinatorial Optimization (MA 4502) Dr. Michael Ritter Problem Sheet 1 Homework Problems Exercise

More information

Topics in Graph Theory

Topics in Graph Theory Topics in Graph Theory September 4, 2018 1 Preliminaries A graph is a system G = (V, E) consisting of a set V of vertices and a set E (disjoint from V ) of edges, together with an incidence function End

More information

NP-Completeness. Algorithmique Fall semester 2011/12

NP-Completeness. Algorithmique Fall semester 2011/12 NP-Completeness Algorithmique Fall semester 2011/12 1 What is an Algorithm? We haven t really answered this question in this course. Informally, an algorithm is a step-by-step procedure for computing a

More information

Lecture 4: NP and computational intractability

Lecture 4: NP and computational intractability Chapter 4 Lecture 4: NP and computational intractability Listen to: Find the longest path, Daniel Barret What do we do today: polynomial time reduction NP, co-np and NP complete problems some examples

More information

Introduction to Complexity Theory

Introduction to Complexity Theory Introduction to Complexity Theory Read K & S Chapter 6. Most computational problems you will face your life are solvable (decidable). We have yet to address whether a problem is easy or hard. Complexity

More information

Hermite normal form: Computation and applications

Hermite normal form: Computation and applications Integer Points in Polyhedra Gennady Shmonin Hermite normal form: Computation and applications February 24, 2009 1 Uniqueness of Hermite normal form In the last lecture, we showed that if B is a rational

More information

ON COST MATRICES WITH TWO AND THREE DISTINCT VALUES OF HAMILTONIAN PATHS AND CYCLES

ON COST MATRICES WITH TWO AND THREE DISTINCT VALUES OF HAMILTONIAN PATHS AND CYCLES ON COST MATRICES WITH TWO AND THREE DISTINCT VALUES OF HAMILTONIAN PATHS AND CYCLES SANTOSH N. KABADI AND ABRAHAM P. PUNNEN Abstract. Polynomially testable characterization of cost matrices associated

More information

Show that the following problems are NP-complete

Show that the following problems are NP-complete Show that the following problems are NP-complete April 7, 2018 Below is a list of 30 exercises in which you are asked to prove that some problem is NP-complete. The goal is to better understand the theory

More information

Chapter 34: NP-Completeness

Chapter 34: NP-Completeness Graph Algorithms - Spring 2011 Set 17. Lecturer: Huilan Chang Reference: Cormen, Leiserson, Rivest, and Stein, Introduction to Algorithms, 2nd Edition, The MIT Press. Chapter 34: NP-Completeness 2. Polynomial-time

More information

ACO Comprehensive Exam March 17 and 18, Computability, Complexity and Algorithms

ACO Comprehensive Exam March 17 and 18, Computability, Complexity and Algorithms 1. Computability, Complexity and Algorithms (a) Let G(V, E) be an undirected unweighted graph. Let C V be a vertex cover of G. Argue that V \ C is an independent set of G. (b) Minimum cardinality vertex

More information

Computational Complexity. IE 496 Lecture 6. Dr. Ted Ralphs

Computational Complexity. IE 496 Lecture 6. Dr. Ted Ralphs Computational Complexity IE 496 Lecture 6 Dr. Ted Ralphs IE496 Lecture 6 1 Reading for This Lecture N&W Sections I.5.1 and I.5.2 Wolsey Chapter 6 Kozen Lectures 21-25 IE496 Lecture 6 2 Introduction to

More information

1.1 P, NP, and NP-complete

1.1 P, NP, and NP-complete CSC5160: Combinatorial Optimization and Approximation Algorithms Topic: Introduction to NP-complete Problems Date: 11/01/2008 Lecturer: Lap Chi Lau Scribe: Jerry Jilin Le This lecture gives a general introduction

More information

2 GRAPH AND NETWORK OPTIMIZATION. E. Amaldi Introduction to Operations Research Politecnico Milano 1

2 GRAPH AND NETWORK OPTIMIZATION. E. Amaldi Introduction to Operations Research Politecnico Milano 1 2 GRAPH AND NETWORK OPTIMIZATION E. Amaldi Introduction to Operations Research Politecnico Milano 1 A variety of decision-making problems can be formulated in terms of graphs and networks Examples: - transportation

More information

Algorithms and Theory of Computation. Lecture 22: NP-Completeness (2)

Algorithms and Theory of Computation. Lecture 22: NP-Completeness (2) Algorithms and Theory of Computation Lecture 22: NP-Completeness (2) Xiaohui Bei MAS 714 November 8, 2018 Nanyang Technological University MAS 714 November 8, 2018 1 / 20 Set Cover Set Cover Input: a set

More information

CS 583: Algorithms. NP Completeness Ch 34. Intractability

CS 583: Algorithms. NP Completeness Ch 34. Intractability CS 583: Algorithms NP Completeness Ch 34 Intractability Some problems are intractable: as they grow large, we are unable to solve them in reasonable time What constitutes reasonable time? Standard working

More information

Limitations of Algorithm Power

Limitations of Algorithm Power Limitations of Algorithm Power Objectives We now move into the third and final major theme for this course. 1. Tools for analyzing algorithms. 2. Design strategies for designing algorithms. 3. Identifying

More information

NP-Completeness. NP-Completeness 1

NP-Completeness. NP-Completeness 1 NP-Completeness Reference: Computers and Intractability: A Guide to the Theory of NP-Completeness by Garey and Johnson, W.H. Freeman and Company, 1979. NP-Completeness 1 General Problems, Input Size and

More information

Computational complexity theory

Computational complexity theory Computational complexity theory Introduction to computational complexity theory Complexity (computability) theory deals with two aspects: Algorithm s complexity. Problem s complexity. References S. Cook,

More information

NP-Completeness. Andreas Klappenecker. [based on slides by Prof. Welch]

NP-Completeness. Andreas Klappenecker. [based on slides by Prof. Welch] NP-Completeness Andreas Klappenecker [based on slides by Prof. Welch] 1 Prelude: Informal Discussion (Incidentally, we will never get very formal in this course) 2 Polynomial Time Algorithms Most of the

More information

CMPT307: Complexity Classes: P and N P Week 13-1

CMPT307: Complexity Classes: P and N P Week 13-1 CMPT307: Complexity Classes: P and N P Week 13-1 Xian Qiu Simon Fraser University xianq@sfu.ca Strings and Languages an alphabet Σ is a finite set of symbols {0, 1}, {T, F}, {a, b,..., z}, N a string x

More information

More on NP and Reductions

More on NP and Reductions Indian Institute of Information Technology Design and Manufacturing, Kancheepuram Chennai 600 127, India An Autonomous Institute under MHRD, Govt of India http://www.iiitdm.ac.in COM 501 Advanced Data

More information

Graph Theory and Optimization Computational Complexity (in brief)

Graph Theory and Optimization Computational Complexity (in brief) Graph Theory and Optimization Computational Complexity (in brief) Nicolas Nisse Inria, France Univ. Nice Sophia Antipolis, CNRS, I3S, UMR 7271, Sophia Antipolis, France September 2015 N. Nisse Graph Theory

More information

Computer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms

Computer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms Computer Science 385 Analysis of Algorithms Siena College Spring 2011 Topic Notes: Limitations of Algorithms We conclude with a discussion of the limitations of the power of algorithms. That is, what kinds

More information

Polynomial-time Reductions

Polynomial-time Reductions Polynomial-time Reductions Disclaimer: Many denitions in these slides should be taken as the intuitive meaning, as the precise meaning of some of the terms are hard to pin down without introducing the

More information

Polynomial-time reductions. We have seen several reductions:

Polynomial-time reductions. We have seen several reductions: Polynomial-time reductions We have seen several reductions: Polynomial-time reductions Informal explanation of reductions: We have two problems, X and Y. Suppose we have a black-box solving problem X in

More information

P P P NP-Hard: L is NP-hard if for all L NP, L L. Thus, if we could solve L in polynomial. Cook's Theorem and Reductions

P P P NP-Hard: L is NP-hard if for all L NP, L L. Thus, if we could solve L in polynomial. Cook's Theorem and Reductions Summary of the previous lecture Recall that we mentioned the following topics: P: is the set of decision problems (or languages) that are solvable in polynomial time. NP: is the set of decision problems

More information

Chapter 3. Complexity of algorithms

Chapter 3. Complexity of algorithms Chapter 3 Complexity of algorithms In this chapter, we see how problems may be classified according to their level of difficulty. Most problems that we consider in these notes are of general character,

More information

Week 8. 1 LP is easy: the Ellipsoid Method

Week 8. 1 LP is easy: the Ellipsoid Method Week 8 1 LP is easy: the Ellipsoid Method In 1979 Khachyan proved that LP is solvable in polynomial time by a method of shrinking ellipsoids. The running time is polynomial in the number of variables n,

More information

Preliminaries. Graphs. E : set of edges (arcs) (Undirected) Graph : (i, j) = (j, i) (edges) V = {1, 2, 3, 4, 5}, E = {(1, 3), (3, 2), (2, 4)}

Preliminaries. Graphs. E : set of edges (arcs) (Undirected) Graph : (i, j) = (j, i) (edges) V = {1, 2, 3, 4, 5}, E = {(1, 3), (3, 2), (2, 4)} Preliminaries Graphs G = (V, E), V : set of vertices E : set of edges (arcs) (Undirected) Graph : (i, j) = (j, i) (edges) 1 2 3 5 4 V = {1, 2, 3, 4, 5}, E = {(1, 3), (3, 2), (2, 4)} 1 Directed Graph (Digraph)

More information

Discrete Optimization in a Nutshell

Discrete Optimization in a Nutshell Discrete Optimization in a Nutshell Integer Optimization Christoph Helmberg : [,] Contents Integer Optimization 2: 2 [2,2] Contents Integer Optimization. Bipartite Matching.2 Integral Polyhedra ( and directed

More information

SAT, NP, NP-Completeness

SAT, NP, NP-Completeness CS 473: Algorithms, Spring 2018 SAT, NP, NP-Completeness Lecture 22 April 13, 2018 Most slides are courtesy Prof. Chekuri Ruta (UIUC) CS473 1 Spring 2018 1 / 57 Part I Reductions Continued Ruta (UIUC)

More information

Complexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler

Complexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler Complexity Theory Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität Wien 15 May, 2018 Reinhard

More information

Outline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181.

Outline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181. Complexity Theory Complexity Theory Outline Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität

More information

REVIEW QUESTIONS. Chapter 1: Foundations: Sets, Logic, and Algorithms

REVIEW QUESTIONS. Chapter 1: Foundations: Sets, Logic, and Algorithms REVIEW QUESTIONS Chapter 1: Foundations: Sets, Logic, and Algorithms 1. Why can t a Venn diagram be used to prove a statement about sets? 2. Suppose S is a set with n elements. Explain why the power set

More information

Computational Models Lecture 11, Spring 2009

Computational Models Lecture 11, Spring 2009 Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. p. 1 Computational Models Lecture 11, Spring 2009 Deterministic Time Classes NonDeterministic Time Classes

More information

Turing Machines and Time Complexity

Turing Machines and Time Complexity Turing Machines and Time Complexity Turing Machines Turing Machines (Infinitely long) Tape of 1 s and 0 s Turing Machines (Infinitely long) Tape of 1 s and 0 s Able to read and write the tape, and move

More information

Lecture #14: NP-Completeness (Chapter 34 Old Edition Chapter 36) Discussion here is from the old edition.

Lecture #14: NP-Completeness (Chapter 34 Old Edition Chapter 36) Discussion here is from the old edition. Lecture #14: 0.0.1 NP-Completeness (Chapter 34 Old Edition Chapter 36) Discussion here is from the old edition. 0.0.2 Preliminaries: Definition 1 n abstract problem Q is a binary relations on a set I of

More information

1 Reductions and Expressiveness

1 Reductions and Expressiveness 15-451/651: Design & Analysis of Algorithms November 3, 2015 Lecture #17 last changed: October 30, 2015 In the past few lectures we have looked at increasingly more expressive problems solvable using efficient

More information

Lecture 5: Computational Complexity

Lecture 5: Computational Complexity Lecture 5: Computational Complexity (3 units) Outline Computational complexity Decision problem, Classes N P and P. Polynomial reduction and Class N PC P = N P or P = N P? 1 / 22 The Goal of Computational

More information

Nonnegative Matrices I

Nonnegative Matrices I Nonnegative Matrices I Daisuke Oyama Topics in Economic Theory September 26, 2017 References J. L. Stuart, Digraphs and Matrices, in Handbook of Linear Algebra, Chapter 29, 2006. R. A. Brualdi and H. J.

More information

Computational complexity theory

Computational complexity theory Computational complexity theory Introduction to computational complexity theory Complexity (computability) theory deals with two aspects: Algorithm s complexity. Problem s complexity. References S. Cook,

More information

Chapter 9: Relations Relations

Chapter 9: Relations Relations Chapter 9: Relations 9.1 - Relations Definition 1 (Relation). Let A and B be sets. A binary relation from A to B is a subset R A B, i.e., R is a set of ordered pairs where the first element from each pair

More information

Deciding Emptiness of the Gomory-Chvátal Closure is NP-Complete, Even for a Rational Polyhedron Containing No Integer Point

Deciding Emptiness of the Gomory-Chvátal Closure is NP-Complete, Even for a Rational Polyhedron Containing No Integer Point Deciding Emptiness of the Gomory-Chvátal Closure is NP-Complete, Even for a Rational Polyhedron Containing No Integer Point Gérard Cornuéjols 1 and Yanjun Li 2 1 Tepper School of Business, Carnegie Mellon

More information

Polynomiality of Linear Programming

Polynomiality of Linear Programming Chapter 10 Polynomiality of Linear Programming In the previous section, we presented the Simplex Method. This method turns out to be very efficient for solving linear programmes in practice. While it is

More information

Algorithms Design & Analysis. Approximation Algorithm

Algorithms Design & Analysis. Approximation Algorithm Algorithms Design & Analysis Approximation Algorithm Recap External memory model Merge sort Distribution sort 2 Today s Topics Hard problem Approximation algorithms Metric traveling salesman problem A

More information

Design and Analysis of Algorithms

Design and Analysis of Algorithms Design and Analysis of Algorithms CSE 5311 Lecture 25 NP Completeness Junzhou Huang, Ph.D. Department of Computer Science and Engineering CSE5311 Design and Analysis of Algorithms 1 NP-Completeness Some

More information

Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar

Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar Turing Machine A Turing machine is an abstract representation of a computing device. It consists of a read/write

More information

NP-Complete Problems. More reductions

NP-Complete Problems. More reductions NP-Complete Problems More reductions Definitions P: problems that can be solved in polynomial time (typically in n, size of input) on a deterministic Turing machine Any normal computer simulates a DTM

More information

ECE 695 Numerical Simulations Lecture 2: Computability and NPhardness. Prof. Peter Bermel January 11, 2017

ECE 695 Numerical Simulations Lecture 2: Computability and NPhardness. Prof. Peter Bermel January 11, 2017 ECE 695 Numerical Simulations Lecture 2: Computability and NPhardness Prof. Peter Bermel January 11, 2017 Outline Overview Definitions Computing Machines Church-Turing Thesis Polynomial Time (Class P)

More information

1 Sequences and Summation

1 Sequences and Summation 1 Sequences and Summation A sequence is a function whose domain is either all the integers between two given integers or all the integers greater than or equal to a given integer. For example, a m, a m+1,...,

More information

The Mixed Chinese Postman Problem Parameterized by Pathwidth and Treedepth

The Mixed Chinese Postman Problem Parameterized by Pathwidth and Treedepth The Mixed Chinese Postman Problem Parameterized by Pathwidth and Treedepth Gregory Gutin, Mark Jones, and Magnus Wahlström Royal Holloway, University of London Egham, Surrey TW20 0EX, UK Abstract In the

More information

ACO Comprehensive Exam October 14 and 15, 2013

ACO Comprehensive Exam October 14 and 15, 2013 1. Computability, Complexity and Algorithms (a) Let G be the complete graph on n vertices, and let c : V (G) V (G) [0, ) be a symmetric cost function. Consider the following closest point heuristic for

More information

Paths and cycles in extended and decomposable digraphs

Paths and cycles in extended and decomposable digraphs Paths and cycles in extended and decomposable digraphs Jørgen Bang-Jensen Gregory Gutin Department of Mathematics and Computer Science Odense University, Denmark Abstract We consider digraphs called extended

More information

NP-problems continued

NP-problems continued NP-problems continued Page 1 Since SAT and INDEPENDENT SET can be reduced to each other we might think that there would be some similarities between the two problems. In fact, there is one such similarity.

More information

Notes for Lecture Notes 2

Notes for Lecture Notes 2 Stanford University CS254: Computational Complexity Notes 2 Luca Trevisan January 11, 2012 Notes for Lecture Notes 2 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

More information

Linear-Time Algorithms for Finding Tucker Submatrices and Lekkerkerker-Boland Subgraphs

Linear-Time Algorithms for Finding Tucker Submatrices and Lekkerkerker-Boland Subgraphs Linear-Time Algorithms for Finding Tucker Submatrices and Lekkerkerker-Boland Subgraphs Nathan Lindzey, Ross M. McConnell Colorado State University, Fort Collins CO 80521, USA Abstract. Tucker characterized

More information

NP-COMPLETE PROBLEMS. 1. Characterizing NP. Proof

NP-COMPLETE PROBLEMS. 1. Characterizing NP. Proof T-79.5103 / Autumn 2006 NP-complete problems 1 NP-COMPLETE PROBLEMS Characterizing NP Variants of satisfiability Graph-theoretic problems Coloring problems Sets and numbers Pseudopolynomial algorithms

More information

Index coding with side information

Index coding with side information Index coding with side information Ehsan Ebrahimi Targhi University of Tartu Abstract. The Index Coding problem has attracted a considerable amount of attention in the recent years. The problem is motivated

More information

Chapter 2 : Time complexity

Chapter 2 : Time complexity Dr. Abhijit Das, Chapter 2 : Time complexity In this chapter we study some basic results on the time complexities of computational problems. concentrate our attention mostly on polynomial time complexities,

More information

1: Introduction to Lattices

1: Introduction to Lattices CSE 206A: Lattice Algorithms and Applications Winter 2012 Instructor: Daniele Micciancio 1: Introduction to Lattices UCSD CSE Lattices are regular arrangements of points in Euclidean space. The simplest

More information

4/12/2011. Chapter 8. NP and Computational Intractability. Directed Hamiltonian Cycle. Traveling Salesman Problem. Directed Hamiltonian Cycle

4/12/2011. Chapter 8. NP and Computational Intractability. Directed Hamiltonian Cycle. Traveling Salesman Problem. Directed Hamiltonian Cycle Directed Hamiltonian Cycle Chapter 8 NP and Computational Intractability Claim. G has a Hamiltonian cycle iff G' does. Pf. Suppose G has a directed Hamiltonian cycle Γ. Then G' has an undirected Hamiltonian

More information

CS Algorithms and Complexity

CS Algorithms and Complexity CS 50 - Algorithms and Complexity Linear Programming, the Simplex Method, and Hard Problems Sean Anderson 2/15/18 Portland State University Table of contents 1. The Simplex Method 2. The Graph Problem

More information

Eigenvectors Via Graph Theory

Eigenvectors Via Graph Theory Eigenvectors Via Graph Theory Jennifer Harris Advisor: Dr. David Garth October 3, 2009 Introduction There is no problem in all mathematics that cannot be solved by direct counting. -Ernst Mach The goal

More information

1 Non-deterministic Turing Machine

1 Non-deterministic Turing Machine 1 Non-deterministic Turing Machine A nondeterministic Turing machine is a generalization of the standard TM for which every configuration may yield none, or one or more than one next configurations. In

More information

6-1 Computational Complexity

6-1 Computational Complexity 6-1 Computational Complexity 6. Computational Complexity Computational models Turing Machines Time complexity Non-determinism, witnesses, and short proofs. Complexity classes: P, NP, conp Polynomial-time

More information

Formal definition of P

Formal definition of P Since SAT and INDEPENDENT SET can be reduced to each other we might think that there would be some similarities between the two problems. In fact, there is one such similarity. In SAT we want to know if

More information

conp = { L L NP } (1) This problem is essentially the same as SAT because a formula is not satisfiable if and only if its negation is a tautology.

conp = { L L NP } (1) This problem is essentially the same as SAT because a formula is not satisfiable if and only if its negation is a tautology. 1 conp and good characterizations In these lecture notes we discuss a complexity class called conp and its relationship to P and NP. This discussion will lead to an interesting notion of good characterizations

More information

The cocycle lattice of binary matroids

The cocycle lattice of binary matroids Published in: Europ. J. Comb. 14 (1993), 241 250. The cocycle lattice of binary matroids László Lovász Eötvös University, Budapest, Hungary, H-1088 Princeton University, Princeton, NJ 08544 Ákos Seress*

More information

A An Overview of Complexity Theory for the Algorithm Designer

A An Overview of Complexity Theory for the Algorithm Designer A An Overview of Complexity Theory for the Algorithm Designer A.1 Certificates and the class NP A decision problem is one whose answer is either yes or no. Two examples are: SAT: Given a Boolean formula

More information

Computational complexity

Computational complexity COMS11700 Computational complexity Department of Computer Science, University of Bristol Bristol, UK 2 May 2014 COMS11700: Computational complexity Slide 1/23 Introduction If we can prove that a language

More information

Lecture 5 January 16, 2013

Lecture 5 January 16, 2013 UBC CPSC 536N: Sparse Approximations Winter 2013 Prof. Nick Harvey Lecture 5 January 16, 2013 Scribe: Samira Samadi 1 Combinatorial IPs 1.1 Mathematical programs { min c Linear Program (LP): T x s.t. a

More information

Classical Complexity and Fixed-Parameter Tractability of Simultaneous Consecutive Ones Submatrix & Editing Problems

Classical Complexity and Fixed-Parameter Tractability of Simultaneous Consecutive Ones Submatrix & Editing Problems Classical Complexity and Fixed-Parameter Tractability of Simultaneous Consecutive Ones Submatrix & Editing Problems Rani M. R, Mohith Jagalmohanan, R. Subashini Binary matrices having simultaneous consecutive

More information

1 Computational Problems

1 Computational Problems Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

More information

Reductions. Reduction. Linear Time Reduction: Examples. Linear Time Reductions

Reductions. Reduction. Linear Time Reduction: Examples. Linear Time Reductions Reduction Reductions Problem X reduces to problem Y if given a subroutine for Y, can solve X. Cost of solving X = cost of solving Y + cost of reduction. May call subroutine for Y more than once. Ex: X

More information

1.3 Vertex Degrees. Vertex Degree for Undirected Graphs: Let G be an undirected. Vertex Degree for Digraphs: Let D be a digraph and y V (D).

1.3 Vertex Degrees. Vertex Degree for Undirected Graphs: Let G be an undirected. Vertex Degree for Digraphs: Let D be a digraph and y V (D). 1.3. VERTEX DEGREES 11 1.3 Vertex Degrees Vertex Degree for Undirected Graphs: Let G be an undirected graph and x V (G). The degree d G (x) of x in G: the number of edges incident with x, each loop counting

More information

NP-Completeness I. Lecture Overview Introduction: Reduction and Expressiveness

NP-Completeness I. Lecture Overview Introduction: Reduction and Expressiveness Lecture 19 NP-Completeness I 19.1 Overview In the past few lectures we have looked at increasingly more expressive problems that we were able to solve using efficient algorithms. In this lecture we introduce

More information

ECS122A Handout on NP-Completeness March 12, 2018

ECS122A Handout on NP-Completeness March 12, 2018 ECS122A Handout on NP-Completeness March 12, 2018 Contents: I. Introduction II. P and NP III. NP-complete IV. How to prove a problem is NP-complete V. How to solve a NP-complete problem: approximate algorithms

More information

1. Introduction Recap

1. Introduction Recap 1. Introduction Recap 1. Tractable and intractable problems polynomial-boundness: O(n k ) 2. NP-complete problems informal definition 3. Examples of P vs. NP difference may appear only slightly 4. Optimization

More information

Strongly chordal and chordal bipartite graphs are sandwich monotone

Strongly chordal and chordal bipartite graphs are sandwich monotone Strongly chordal and chordal bipartite graphs are sandwich monotone Pinar Heggernes Federico Mancini Charis Papadopoulos R. Sritharan Abstract A graph class is sandwich monotone if, for every pair of its

More information

Induction and recursion. Chapter 5

Induction and recursion. Chapter 5 Induction and recursion Chapter 5 Chapter Summary Mathematical Induction Strong Induction Well-Ordering Recursive Definitions Structural Induction Recursive Algorithms Mathematical Induction Section 5.1

More information

Finding k disjoint paths in a directed planar graph

Finding k disjoint paths in a directed planar graph Finding k disjoint paths in a directed planar graph Alexander Schrijver CWI Kruislaan 413 1098 SJ Amsterdam The Netherlands and Department of Mathematics University of Amsterdam Plantage Muidergracht 24

More information

On the adjacency matrix of a block graph

On the adjacency matrix of a block graph On the adjacency matrix of a block graph R. B. Bapat Stat-Math Unit Indian Statistical Institute, Delhi 7-SJSS Marg, New Delhi 110 016, India. email: rbb@isid.ac.in Souvik Roy Economics and Planning Unit

More information

CS675: Convex and Combinatorial Optimization Fall 2014 Combinatorial Problems as Linear Programs. Instructor: Shaddin Dughmi

CS675: Convex and Combinatorial Optimization Fall 2014 Combinatorial Problems as Linear Programs. Instructor: Shaddin Dughmi CS675: Convex and Combinatorial Optimization Fall 2014 Combinatorial Problems as Linear Programs Instructor: Shaddin Dughmi Outline 1 Introduction 2 Shortest Path 3 Algorithms for Single-Source Shortest

More information

Rao s degree sequence conjecture

Rao s degree sequence conjecture Rao s degree sequence conjecture Maria Chudnovsky 1 Columbia University, New York, NY 10027 Paul Seymour 2 Princeton University, Princeton, NJ 08544 July 31, 2009; revised December 10, 2013 1 Supported

More information

Branch-and-Bound for the Travelling Salesman Problem

Branch-and-Bound for the Travelling Salesman Problem Branch-and-Bound for the Travelling Salesman Problem Leo Liberti LIX, École Polytechnique, F-91128 Palaiseau, France Email:liberti@lix.polytechnique.fr March 15, 2011 Contents 1 The setting 1 1.1 Graphs...............................................

More information

P and NP. Or, how to make $1,000,000.

P and NP. Or, how to make $1,000,000. P and NP Or, how to make $1,000,000. http://www.claymath.org/millennium-problems/p-vs-np-problem Review: Polynomial time difference between single-tape and multi-tape TMs Exponential time difference between

More information

CS675: Convex and Combinatorial Optimization Fall 2016 Combinatorial Problems as Linear and Convex Programs. Instructor: Shaddin Dughmi

CS675: Convex and Combinatorial Optimization Fall 2016 Combinatorial Problems as Linear and Convex Programs. Instructor: Shaddin Dughmi CS675: Convex and Combinatorial Optimization Fall 2016 Combinatorial Problems as Linear and Convex Programs Instructor: Shaddin Dughmi Outline 1 Introduction 2 Shortest Path 3 Algorithms for Single-Source

More information