Algorithms and Theory of Computation. Lecture 19: Class P and NP, Reduction

Algorithms and Theory of Computation Lecture 19: Class P and NP, Reduction Xiaohui Bei MAS 714 October 29, 2018 Nanyang Technological University MAS 714 October 29, 2018 1 / 26

Decision Problems Revisited Many optimization problems can be reposed as decision problems. Nanyang Technological University MAS 714 October 29, 2018 2 / 26

Decision Problems Revisited Many optimization problems can be reposed as decision problems. Travelling Salesman Problem TSP: Given a complete undirected graph with edge weights, find a tour (cycle visiting each vertex once and exactly once) of minimum total weight. Nanyang Technological University MAS 714 October 29, 2018 2 / 26

Decision Problems Revisited Many optimization problems can be reposed as decision problems. Travelling Salesman Problem TSP: Given a complete undirected graph with edge weights, find a tour (cycle visiting each vertex once and exactly once) of minimum total weight. TSP dec : { G, k G is a complete undirected graph with edge weights and there exists a tour of total weight k}. Nanyang Technological University MAS 714 October 29, 2018 2 / 26

TSP in NP Given G, k, what is a nondeterministic polynomial time algorithm to decide whether or not G has a tour of cost at most k? Nanyang Technological University MAS 714 October 29, 2018 3 / 26

TSP in NP Given G, k, what is a nondeterministic polynomial time algorithm to decide whether or not G has a tour of cost at most k? Guess a tour t, and accept if cost(t) k runs in polynomial time (linear in fact) if there is a tour of cost at most k, then there is an accepting computation if there is no tour of cost at most k, then all computations reject. Nanyang Technological University MAS 714 October 29, 2018 3 / 26

Certifier/Proof Interpretation of NP L NP implies that there is a nondeterministic poly-time algorithm/tm M that accepts L. Claim L NP if and only if there is a poly-time TM M that accepts L with the following properties: On input w, M first non-deterministically guesses a string y where y p( w ) for some fixed polynomial p(). M then runs a deterministic poly-time TM M on the string w#y and accepts w if M accepts w#y and rejects w if M rejects w#y. Nanyang Technological University MAS 714 October 29, 2018 4 / 26

Certifier/Proof Interpretation of NP L NP implies that there is a nondeterministic poly-time algorithm/tm M that accepts L. Claim L NP if and only if there is a poly-time TM M that accepts L with the following properties: On input w, M first non-deterministically guesses a string y where y p( w ) for some fixed polynomial p(). M then runs a deterministic poly-time TM M on the string w#y and accepts w if M accepts w#y and rejects w if M rejects w#y. Non-determinism is used to guess a proof/solution y for w. Verifier/Certifier M is a deterministic algorithm that checks if y is a valid solution for w. Nanyang Technological University MAS 714 October 29, 2018 4 / 26

Certifier/Proof Interpretation of NP L NP implies that there is a nondeterministic poly-time algorithm/tm M that accepts L. Alternate definition of NP L NP if and only if there is a deterministic TM/algorithm M and two polynomials p() and q() such that M runs in time p( x ) where x is its input (hence efficient) if w L then there is a string y with y q( w ) such that M accepts w#y if w / L then for every y with y q( w ), M on input w#y rejects Nanyang Technological University MAS 714 October 29, 2018 5 / 26

Certifier/Proof Interpretation of NP L NP implies that there is a nondeterministic poly-time algorithm/tm M that accepts L. Alternate definition of NP L NP if and only if there is a deterministic TM/algorithm M and two polynomials p() and q() such that M runs in time p( x ) where x is its input (hence efficient) if w L then there is a string y with y q( w ) such that M accepts w#y if w / L then for every y with y q( w ), M on input w#y rejects NP is natural because there are plenty of problems where verification of solutions is easy. Hundreds of well-studied problems are in NP. Nanyang Technological University MAS 714 October 29, 2018 5 / 26

Maximum Independent Set Given undirected graph G = (V, E), a subset of vertices S V is an independent set if there are no edges between vertices in S. That is, if u, v S then (u, v) / E. A B C F E D Some independent sets in graph above: {D}, {A, C}, {B, E, F} Nanyang Technological University MAS 714 October 29, 2018 6 / 26

Maximum Independent Set Given undirected graph G = (V, E), a subset of vertices S V is an independent set if there are no edges between vertices in S. That is, if u, v S then (u, v) / E. A B C F E D Some independent sets in graph above: {D}, {A, C}, {B, E, F} Maximum Independent Set Problem(MIS) Input: Graph G = (V, E) Goal: Find maximum sized independent set in G. Nanyang Technological University MAS 714 October 29, 2018 6 / 26

Maximum Independent Set Decision Version of MIS Input: Graph G = (V, E) and integer k written as G, k Question: Is there an independent set in G of size at least k? Nanyang Technological University MAS 714 October 29, 2018 7 / 26

Maximum Independent Set Decision Version of MIS Input: Graph G = (V, E) and integer k written as G, k Question: Is there an independent set in G of size at least k? The answer to G, k is YES if G has an independent set of size at least k. Otherwise the answer is NO. Sometimes we say G, k is a YES instance or a NO instance. The language associated with this decision problem is L MIS = { G, k G has an independent set of size k} Nanyang Technological University MAS 714 October 29, 2018 7 / 26

L MIS is in NP A non-deterministic polynomial-time algorithm for L MIS. Nanyang Technological University MAS 714 October 29, 2018 8 / 26

L MIS is in NP A non-deterministic polynomial-time algorithm for L MIS. Input: a string G, k encoding graph G = (V, E) and integer k 1 Non-deterministically guess a subset S V of vertices. 2 Verify (deterministically) that S forms an independent set in G by checking that there is no edge in E between any pair of vertices in S S k 3 If S passes the above two tests output YES else NO Nanyang Technological University MAS 714 October 29, 2018 8 / 26

L MIS is in NP A non-deterministic polynomial-time algorithm for L MIS. Input: a string G, k encoding graph G = (V, E) and integer k 1 Non-deterministically guess a subset S V of vertices. 2 Verify (deterministically) that S forms an independent set in G by checking that there is no edge in E between any pair of vertices in S S k 3 If S passes the above two tests output YES else NO Key Points: 1 string encoding S, S has length polynomial in length of input G, k 2 verification of guess is easily seen to be polynomial in length of S and G, k Nanyang Technological University MAS 714 October 29, 2018 8 / 26

Sudoku Given a n n sudoku puzzle, does it have a solution? Nanyang Technological University MAS 714 October 29, 2018 9 / 26

Recap P: set of decision problems that have polynomial time algorithms. NP: set of decision problems that have polynomial time nondeterministic algorithms. Big Question Does every problem in NP have an efficient algorithm? Nanyang Technological University MAS 714 October 29, 2018 10 / 26

Recap P: set of decision problems that have polynomial time algorithms. NP: set of decision problems that have polynomial time nondeterministic algorithms. Big Question Does every problem in NP have an efficient algorithm? Same as asking whether P = NP. We dont know the answer and many people believe that P NP. Nanyang Technological University MAS 714 October 29, 2018 10 / 26

Reductions A reduction from Problem X to Problem Y means (informally) that if we have an algorithm for Problem Y, we can use it to find an algorithm for Problem X. Nanyang Technological University MAS 714 October 29, 2018 11 / 26

Reductions A reduction from Problem X to Problem Y means (informally) that if we have an algorithm for Problem Y, we can use it to find an algorithm for Problem X. We use reductions to find algorithms to solve problems. show that we can t find algorithms for some problems. (We say that these problems are hard.) Nanyang Technological University MAS 714 October 29, 2018 11 / 26

Reductions for Decision Problems/Languages Reductions for Languages For languages L X, L Y, a reduction from L X to L Y is an algorithm with input w Σ output w Σ such that w L Y w L X (Actually, this is only one type of reduction, but this is the one well use most often.) There are other kinds of reductions. Nanyang Technological University MAS 714 October 29, 2018 12 / 26

Reductions for Decision Problems/Languages Reductions for Decision Problems For decision problems X, Y, a reduction from X to Y is an algorithm with such that input: I X, an instance of X output: I Y. an instance of Y I Y is YES instance of Y I X is YES instance of X Nanyang Technological University MAS 714 October 29, 2018 13 / 26

Using Reductions to Solve Problems R: Reduction X Y A Y : algorithm for Y Nanyang Technological University MAS 714 October 29, 2018 14 / 26

Using Reductions to Solve Problems R: Reduction X Y A Y : algorithm for Y New algorithm for X: Algorithm: A X (I X ): I Y = R(I X ); return A Y (I Y ) Nanyang Technological University MAS 714 October 29, 2018 14 / 26

Using Reductions to Solve Problems R: Reduction X Y A Y : algorithm for Y New algorithm for X: Algorithm: A X (I X ): I Y = R(I X ); return A Y (I Y ) If R and A Y polynomial time = A X polynomial time Nanyang Technological University MAS 714 October 29, 2018 14 / 26

Comparing Problems If Problem X reduces to Problem Y (we write X Y), then Nanyang Technological University MAS 714 October 29, 2018 15 / 26

Comparing Problems If Problem X reduces to Problem Y (we write X Y), then Problem X is no harder to solve than Problem Y. Nanyang Technological University MAS 714 October 29, 2018 15 / 26

Comparing Problems If Problem X reduces to Problem Y (we write X Y), then Problem X is no harder to solve than Problem Y. X Y: X is no harder than Y, or Y is at least as hard as X. Nanyang Technological University MAS 714 October 29, 2018 15 / 26

Example of Reductions Given a graph G, a set of vertices S is: independent set: no two vertices of S connected by an edge. clique: every pair of vertices in S is connected by an edge of G. Nanyang Technological University MAS 714 October 29, 2018 16 / 26

Independent Set and Clique Problems Independent Set Instance: A graph G and an integer k. Question: Does G have an independent set of size k? Clique Instance: A graph G and an integer k. Question: Does G have an clique of size k? Nanyang Technological University MAS 714 October 29, 2018 17 / 26

Recall For decision problems X, Y, a reduction from X to Y is an algorithm that takes I X, an instance of X as input and returns I Y, an instance of Y as output such that the solution (Y/N) to I Y is the same as the solution to I X. Nanyang Technological University MAS 714 October 29, 2018 18 / 26

Reducing Independent Set to Clique An instance of Independent Set is a graph G and an integer k. Reduction given G, k outputs G, k, where G is the complement of G, G has an edge (u, v) if and only if (u, v) is not an edge of G. Nanyang Technological University MAS 714 October 29, 2018 19 / 26

Correctness of Reduction Lemma G has an independent set of size at least k if and only if G has a clique of size at least k. Nanyang Technological University MAS 714 October 29, 2018 20 / 26

Correctness of Reduction Lemma G has an independent set of size at least k if and only if G has a clique of size at least k. Proof. Need to prove two facts: G has an independent set of size at least k implies that G has a clique of size at least k. G has a clique of size at least k implies that G has an independent set of size at least k. Nanyang Technological University MAS 714 October 29, 2018 20 / 26

Correctness of Reduction Lemma G has an independent set of size at least k if and only if G has a clique of size at least k. Proof. Need to prove two facts: G has an independent set of size at least k implies that G has a clique of size at least k. G has a clique of size at least k implies that G has an independent set of size at least k. Easy to see both from the fact that S V is an independent set in G if and only if S is a clique in G. Nanyang Technological University MAS 714 October 29, 2018 20 / 26

Independent Set and Clique Independent Set Clique what does it mean? Nanyang Technological University MAS 714 October 29, 2018 21 / 26

Independent Set and Clique Independent Set Clique what does it mean? If we have an algorithm for Clique, then we have an algorithm for Independent Set. Clique is at least as hard as Independent Set. Also... Independent Set is at least as hard as Clique. Nanyang Technological University MAS 714 October 29, 2018 21 / 26

Independent Set and Clique Assume you can solve the Clique problem in time T(n), then you can solve the Independent Set problem in time (A) O(T(n)) (B) O(n log n + T(n)) (C) O(n 2 T(n 2 )) (D) O(n 4 T(n 4 )) (E) O(n 2 + T(n 2 )) Nanyang Technological University MAS 714 October 29, 2018 22 / 26

Independent Set and Clique Assume you can solve the Clique problem in time T(n), then you can solve the Independent Set problem in time (A) O(T(n)) (B) O(n log n + T(n)) (C) O(n 2 T(n 2 )) (D) O(n 4 T(n 4 )) (E) O(n 2 + T(n 2 )) (F) We don t care all these are polynomial if T(n) is a polynomial, which is good enough for our purposes. Nanyang Technological University MAS 714 October 29, 2018 22 / 26

Polynomial Time Reductions We say that an algorithm is efficient if it runs in polynomial-time. To find efficient algorithms for problems, we are only interested in polynomial-time reductions. Reductions that take longer are not useful. If we have a polynomial-time reduction from problem X to problem Y (we write X P Y), and a poly-time algorithm A Y for Y, we have a polynomial-time/efficient algorithm for X. Nanyang Technological University MAS 714 October 29, 2018 23 / 26

Polynomial Time Reductions For decision problems X, Y, a polynomial reduction from X to Y is an algorithm A that has the following properties: A takes I X, an instance of X as input returns I Y, an instance of Y as output runs in polynomial time such that the solution (Y/N) to I Y is the same as the solution to I X. Proposition If X P Y then a polynomial time algorithm for Y implies a polynomial time algorithm for X. Such a reduction is called a Karp reduction. Most reductions we will need are Karp reductions. Nanyang Technological University MAS 714 October 29, 2018 24 / 26

Transitivity of Reductions Proposition X P Y and Y P Z implies that X P Z. Nanyang Technological University MAS 714 October 29, 2018 25 / 26

Transitivity of Reductions Proposition X P Y and Y P Z implies that X P Z. Note: X P Y does not imply that Y P X and hence it is very important to know the FROM and TO in a reduction. To prove X P Y you need to show a reduction FROM X TO Y That is, show that an algorithm for Y implies an algorithm for X. Nanyang Technological University MAS 714 October 29, 2018 25 / 26

Polynomial Time Reductions and Hardness For decision problems X and Y, if X P Y, and Y has an efficient algorithm, X has an efficient algorithm. If you believe that Independent Set does not have an efficient algorithm, why should you believe the same of Clique? Because we showed Independent Set P Clique. If Clique had an efficient algorithm, so would Independent Set. If X P Y and X does not have an efficient algorithm, Y cannot have an efficient algorithm! Nanyang Technological University MAS 714 October 29, 2018 26 / 26