NP-complete problems. CSE 101: Design and Analysis of Algorithms Lecture 20

NP-complete problems CSE 101: Design and Analysis of Algorithms Lecture 20

CSE 101: Design and analysis of algorithms NP-complete problems Reading: Chapter 8 Homework 7 is due today, 11:59 PM Tomorrow s discussion section is the final exam review No instructor, TA, or tutor office hours during finals week. Ask questions on Piazza. Final exam is Friday, Dec 14, 7:00 PM-9:59 PM Practice problems released tomorrow CSE 101, Fall 2018 2

Optimization problems Find the best solution from among a large space of possibilities The format for an optimization problem is Instance: what does the input look like? Solution format: what does an output look like? Constraints: what properties must a solution have? Objective function: what makes a solution better or worse? Based on slides courtesy of Miles Jones CSE 101, Fall 2018 3

Examples of optimization problems Shortest path: Dijkstra s algorithm (fast) Minimum spanning tree: Kruskal s algorithm (fast) Optimal event scheduling: greedy algorithm (fast) Maximum independent set: backtracking (slow) Longest increasing subsequence: dynamic programming (relatively fast) CSE 101, Fall 2018 4

Search problems Find a solution from among a large space of possibilities that meets the constraints, or say none exists The format for a search problem is Instance: what does the input look like? (same as optimization) Solution format: what does an output look like? (same as optimization) Constraints: what properties must a solution have? (same as optimization) Objective: find any solution meeting constraints, or certify none exists Decision version: is there a solution meeting the constraints? Yes/No CSE 101, Fall 2018 5

Examples of decision problems Is there a path? Depth-first search, breadth-first search (fast) Is there a topological sort (i.e., is a graph a directed acyclic graph (DAG))? Greedy-ish algorithm Is there a Hamiltonian path (a path in an undirected or directed graph that visits each vertex exactly once)? Backtracking (slow) Is there a negative cycle? Bellman-Ford algorithm, dynamic programming (somewhat fast) Does a bipartite graph have a perfect matching? (somewhat fast) CSE 101, Fall 2018 6

Examples of other search/optimization problems Finding the minimum area arrangement for components in a chip Solving a Sudoku puzzle Packing your car trunk Decrypting a message Proving a theorem CSE 101, Fall 2018 7

8 queens puzzle Put 8 queens on a chessboard such that no two are attacking Solution format: what does an output look like? Constraints: what properties must a solution have? CSE 101, Fall 2018 8

8 queens puzzle Put 8 queens on a chessboard such that no two are attacking Solution format: arrangement of 8 queens on chessboard Constraints: no two queens are attacking CSE 101, Fall 2018 9

Maximal independent set Given a graph with nodes representing people, and there is an edge between A in B if A and B are enemies, find the largest set of people such that no two are enemies. In other words, given an undirected graph, find the largest set of vertices such that no two are connected with an edge. Instance: what does the input look like? Solution format: what does an output look like? Constraints: what properties must a solution have? Objective function: what makes a solution better or worse? CSE 101, Fall 2018 10

Maximal independent set Given a graph with nodes representing people, and there is an edge between A in B if A and B are enemies, find the largest set of people such that no two are enemies. In other words, given an undirected graph, find the largest set of vertices such that no two are connected with an edge. Instance: an undirected graph A Solution format: subset S of vertices V B C D Constraints: u, v S (u, v) E G H E Objective function: maximize S F CSE 101, Fall 2018 11 I Example K J L

n queens puzzle How can we view n queens as a special case of independent set? Break into groups of 4 or 5, and discuss approaches Do not worry about getting the answer right, just brainstorm ideas CSE 101, Fall 2018 12

n queens puzzle How can we view n queens as a special case of independent set? Graph G is? V = deg u = 4 n 1 E = 4 n 1 n 2 If G has an independent set of size n, then there is a solution to the n queens problem CSE 101, Fall 2018 14

n queens puzzle How can we view n queens as a special case of independent set? Graph G is n n board V = deg u = 4 n 1 E = 4 n 1 n 2 If G has an independent set of size n, then there is a solution to the n queens problem CSE 101, Fall 2018 15

n queens puzzle How can we view n queens as a special case of independent set? Graph G is n n board V =? deg u = 4 n 1 E = 4 n 1 n 2 If G has an independent set of size n, then there is a solution to the n queens problem CSE 101, Fall 2018 16

n queens puzzle How can we view n queens as a special case of independent set? Graph G is n n board V = n 2 deg u = 4 n 1 E = 4 n 1 n 2 If G has an independent set of size n, then there is a solution to the n queens problem CSE 101, Fall 2018 17

n queens puzzle How can we view n queens as a special case of independent set? Graph G is n n board V = n 2 deg u ~? E = 4 n 1 n 2 If G has an independent set of size n, then there is a solution to the n queens problem CSE 101, Fall 2018 18

n queens puzzle How can we view n queens as a special case of independent set? Graph G is n n board V = n 2 deg u ~4 n 1 E = 4 n 1 n 2 If G has an independent set of size n, then there is a solution to the n queens problem CSE 101, Fall 2018 19

n queens puzzle How can we view n queens as a special case of independent set? Graph G is n n board V = n 2 deg u ~4 n 1 E ~? If G has an independent set of size n, then there is a solution to the n queens problem CSE 101, Fall 2018 20

n queens puzzle How can we view n queens as a special case of independent set? Graph G is n n board V = n 2 deg u ~4 n 1 E ~4 n 1 n 2 If G has an independent set of size n, then there is a solution to the n queens problem CSE 101, Fall 2018 21

How hard are these problems? We have a number of approaches that give fast algorithms for a wide variety of search and optimization problems But, other search and optimization problems seem to defeat our standard algorithmic methods (e.g., factoring, independent set, Hamiltonian path) Are all reasonable search and optimization problems easy? If not, what makes some hard? Can we identify the hard ones? CSE 101, Fall 2018 23

Nondeterministic polynomial (NP) NP is a class of decision versions of reasonable search problems Instance: what does the input look like? x, n bits Solution format: what does an output look like? y, poly(n) bits to describe Constraints: what properties must a solution have? R(x,y) can be checked in polynomial time Objective: find any solution meeting constraints, or certify none exists Given x, does there exist a y so that R(x,y)? CSE 101, Fall 2018 24

Polynomial (P) P is a class of decision versions of reasonable search problems that have a polynomial time solution Examples of problems in P Shortest path in graphs Median Almost all algorithms covered in this course CSE 101, Fall 2018 26

P vs NP Any polynomial time decidable yes/no problem is in NP So P is a subset of NP Statement of P=NP: Every decision version of a reasonable search problem is solvable in polynomial time Implications: using binary search, can reduce search/optimization to decision. So, P=NP implies: Every reasonable search and optimization problem can be solved in polynomial time Totally open after many years, one of the Clay Mathematics Institute s list of most important open problems in mathematics CSE 101, Fall 2018 28

Optimization vs decision Example: Traveling salesman (optimization) problem Given a (complete) graph with vertices v 0, v 1,, v n and positive edge weights, what is the cheapest route one can take starting from v 0, going through every node (exactly once), and ending back at v 0? Instance: what does the input look like? Solution format: what does an output look like? Constraints: what properties must a solution have? Objective function: what makes a solution better or worse? CSE 101, Fall 2018 29

Optimization vs decision Example: Traveling salesman (optimization) problem Given a (complete) graph with vertices v 0, v 1,, v n and positive edge weights, what is the cheapest route one can take starting from v 0, going through every node (exactly once), and ending back at v 0? Instance: complete graph with positive edge weights Solution format: sequence of vertices Constraints: all vertices appear exactly once Objective function: minimize total sum of edges CSE 101, Fall 2018 30

Optimization vs decision Example: Traveling salesman (decision) problem Given a (complete) graph with vertices v 0, v 1,, v n and positive edge weights and a budget B, is there a route one can take starting from v 0, going through every node (exactly once), and ending back at v 0 such that the total cost of all edges is less than B? Instance: what does the input look like? Solution format: what does an output look like? Constraints: what properties must a solution have? Objective (decision version): is there a solution meeting the constraints? Yes/No CSE 101, Fall 2018 31

Optimization vs decision Example: Traveling salesman (decision) problem Given a (complete) graph with vertices v 0, v 1,, v n and positive edge weights and a budget B, is there a route one can take starting from v 0, going through every node (exactly once), and ending back at v 0 such that the total cost of all edges is less than B? Instance: complete graph with positive edge weights (same as optimization problem) Solution format: sequence of vertices (same as optimization problem) Constraints: all vertices appear exactly once (same as optimization problem) Objective (decision version): is there a sequence where the total sum of edges is B? CSE 101, Fall 2018 32

Optimization vs decision If one could solve the traveling salesman decision problem in O n k time for some integer k, then could one use that to solve the optimization problem in polynomial time? A binary search on the range of possible distances (total sums of edges) can convert the decision version of the problem to the optimization version, by calling the decision version repeatedly (a polynomial number of times) CSE 101, Fall 2018 33

Brief history The famous logician Kurt Godel asked the famous computer scientist, mathematician, and economist John von Neumann the P vs NP question in a private letter, written shortly before von Neumann s death (1957) CSE 101, Fall 2018 35

Brief history In the Soviet Union, S.V. Yablonksi invents the term perebor or brute force search to describe the combinatorial explosion limiting algorithms, especially for circuit design problems (1959) CSE 101, Fall 2018 36

Brief history Matching and P vs NP. In 1965, Jack Edmonds gives the first polynomial time algorithm for perfect matching on general graphs. To explain the significance to referees, he introduces a section defining P, NP, and posing the P vs NP question. CSE 101, Fall 2018 37

Brief history NP-completeness. In 1971, Steve Cook defines NP-completeness and proves that several problems from logic and combinatorics are NPcomplete, meaning that P=NP if and only if any of these problems are polynomial time solvable. CSE 101, Fall 2018 38

Brief history Plethora of NP-complete problems. Following Cook s work, Richard Karp showed that a large number of the most important optimization problems from all sub-areas (scheduling, graph theory, number theory, logic, puzzles and games, packing, coding, etc.) are NPcomplete. CSE 101, Fall 2018 39

Brief history Garey and Johnson s classic textbook (1979) includes an appendix listing hundreds of NPcomplete problems CSE 101, Fall 2018 40

Brief history NP-complete problems everywhere. Since then, thousands of NP-complete problems have been identified in pretty much any area with computational problems: physics, biology, chemistry, economics, sociology, linguistics, games, engineering, etc. CSE 101, Fall 2018 41

Some of our favorite NP-complete problems Sudoku CSE 101, Fall 2018 42

Some of our favorite NP-complete problems Tetris CSE 101, Fall 2018 43

Some of our favorite NP-complete problems Candy Crush CSE 101, Fall 2018 44

Some of our favorite NP-complete problems Minesweeper CSE 101, Fall 2018 45

Reductions We can view decision problems as the set of all instances for which the answer is yes A reduction from decision problem A to decision problem B is a polynomial time computable function F so that x A F(x) B Then, if we have an algorithm Alg B solving B, we can get an Alg A solving A by using Alg A x Alg B F x If A reduces to B, we write A mp B, where mp is polynomial time map reduction CSE 101, Fall 2018 46

Reductions We have used reductions many times to design algorithms For example Reducing max bandwidth path to reachability Reducing longest increasing subsequence to longest path in directed acyclic graphs Reducing arbitrage to the single-source shortest paths problem, checking for negative cycles (Bellman-Ford algorithm) CSE 101, Fall 2018 47

Reductions and relative hardness If A mp B and B P, then A P So, if A reduces to B and B is easy, then A is easy Contrapositive: if A reduces to B and A is hard, then B is hard Caution: If A reduces to B and B is hard, then nothing happens If A reduces to B and A is easy, then nothing happens CSE 101, Fall 2018 48

Graph coloring Originally came up in making maps: vertices = countries, colors must be distinct to show borders. Famous 4-color theorem said all planar graphs (including possible maps) can be colored with 4 colors. CSE 101, Fall 2018 49

Graph coloring Example: can you color the vertices of this graph with 3 colors so that no two adjacent vertices are the same color? B E D C Possible Colors: R, G, B I G K H A F J L CSE 101, Fall 2018 50

3-coloring Can you color the vertices of a given graph with 3 colors so that no two adjacent vertices are the same color? Instance: what does the input look like? Solution format: what does an output look like? Constraints: what properties must a solution have? Objective (decision version): is there a solution meeting the constraints? Yes/No CSE 101, Fall 2018 51

3-coloring Can you color the vertices of a given graph with 3 colors so that no two adjacent vertices are the same color? Instance: undirected graph G Solution format: give each vertex v a color C v R, G, B Constraints: if u, v is an edge, C(v) C(u) Objective (decision version): is there any 3-coloring of G? CSE 101, Fall 2018 52

3-coloring 3-coloring reduces to maximum independent set Given any undirected connected graph G, if one can solve maximum independent set in polynomial time, then one can use that to find a 3- coloring of G How? Given a graph G = (V, E), we make a new graph H = (V, E ), where H = F(G) For each u V, create three vertices in V, namely u G, u R, u B and connect these three in a triangle For each edge {u, v} E, add edges u G, v G, u R, v R, u B, v B in E CSE 101, Fall 2018 53

3-coloring B E D C I G H J A F K L CSE 101, Fall 2018 54

3-coloring Equivalence Claim: H has an independent set of size n if and only if G is 3-colorable If H has an independent set of size n, it must select exactly one of {u R, u G, u B } for each u in G. Color u in G by the color of the subscript in the independent set of H. This must be a valid coloring because two adjacent vertices in G cannot be both Red since u R, v R is an edge in H Conversely, if there is a coloring of G, use that coloring to pick the corresponding vertex in H CSE 101, Fall 2018 55

Implication 3-coloring If we can solve independent set quickly, we can solve 3-coloring quickly Can we solve independent set quickly? CSE 101, Fall 2018 56

Implication 3-coloring If we can solve independent set quickly, we can solve 3-coloring quickly Can we solve independent set quickly? No (not yet) CSE 101, Fall 2018 57

Search problems (satisfiability) A very basic search problem is the problem of satisfiability (SAT) (SAT) Given a Boolean formula in conjunctive normal form, find an assignment of the variables that makes the formula true or state that none exists Example: x y z x y y z CSE 101, Fall 2018 58

Search problems (3-satisfiability) (3-SAT) Given a Boolean formula in conjunctive normal form such that each clause has at most 3 literals, find an assignment of the variables that makes the formula true or state that none exists If we relax the 2-SAT problem to at most 3 literals per clause, then there is no known polynomialtime algorithm to solve it CSE 101, Fall 2018 60

P vs NP A decision problem is an algorithm that takes as input an instance I and a solution S and determines if S is a valid solution of I in polynomial time of I The set of all decision problems is called NP The set of all decision problems that have a polynomial time algorithm that can find a solution is called P CSE 101, Fall 2018 61

Other Reductions Sometimes it is useful to reduce problem A to problem B (denoted A B) Given that you can get a solution to problem B, you can solve problem A in polynomial time Example: (2SAT) (3SAT) Clearly you can just consider (2SAT) as a subset of (3SAT) Example: (3SAT) (SAT) By simply considering that there could be more than 3 literals per clause Example: (SAT) (3SAT) This may seem surprising CSE 101, Fall 2018 62

NP-complete A decision problem A in NP is said to be NP-complete if all other decision problems reduce to A Existence of Hamiltonian path TSP optimization TSP decision Factoring large numbers CSE 101, Fall 2018 63

NP-complete A problem A is NP-complete if A NP and for every B NP, B reduces to A It follows that, if A is NP-complete, then A P P = NP So, each NP-complete problem determines the fate of all search and optimization problems. In particular, they are all equivalent to each other. CSE 101, Fall 2018 64

(SAT) is NP-complete NP-complete There is a way to reduce any decision problem in NP to (SAT) CSE 101, Fall 2018 65

(SAT) (3SAT) Any clause of the form a 1 a 2 a n can be written in the form a 1 a 2 y 1 y 1 a 3 y 2 y 2 a 4 y 3 (y n 2 a n 1 a n ) So, if we can solve (3SAT) then we can solve (SAT) because (SAT) can be reduced to (3SAT) CSE 101, Fall 2018 66

Loopholes If A is an NP-complete problem, then there are no known polynomial time solutions that can completely solve it on every input. CSE 101, Fall 2018 67

Loopholes Change problem Restrict instances. Are there some restrictions on the instances you have to solve that you did not include in the problem definition? Approximation algorithms: is coming close to optimal good enough? Multiparameter analysis: can you be exponential in a typically small value? Hope that an exponential time algorithm runs faster on your instance: SAT solvers, integer linear programming Average-case analysis Search heuristics CSE 101, Fall 2018 68

NP-hard A problem H is called NP-hard if all problems in NP can be reduced to H The difference between NP-hard and NPcomplete NP-complete NP NP-hard NP CSE 101, Fall 2018 69

P = NP? Does there exist a polynomial time algorithm for all problems in NP? This is not known CSE 101, Fall 2018 70

Summary of CSE 101 Graph search Using data structures (mix between graph search and greedy algorithms) Greedy algorithms Divide and conquer Dynamic programming Reducing to a previously solved problem CSE 101, Fall 2018 71