1. Tractable and Intractable Computational Problems So far in the course we have seen many problems that have polynomial-time solutions; that is, on

Transcription

1 . Tractable and Intractable Comptational Problems So far in the corse we have seen many problems that have polynomial-time soltions; that is, on a problem instance of size n, the rnning time T (n) = O(n k ) for some k, with k typically small. For example, we discssed sorting algorithms with T (n) = O(n lg n), and we showed how to find shortest paths in time T ( E, V ) = O ( E lg( V )) sing Dijkstra s algorithm. On the other hand, for the TSP we gave a dynamic programming soltion that solved the problem in T (n) = O(n n ), where n is the nmber of vertices. We did not give any polynomial-time bonded soltion for this problem. Problems that can be solved by a polynomial-time algorithm are considered tractable, and those that can not be solved in time bonded by a polynomial in the size of the inpt are considered intractable. Therefore, sorting and shortest-path are tractable. From or discssion so far we do not know if TSP is tractable (in fact it is nknown at present). The prpose of these notes is to explore this dichotomy between tractable and intractable problems. Or final reslt will not qite be a neat division into tractable and intractable problems. Rather, we will end p with two important classes: the problems we know are tractable (this class will be called P ), and the problems which are sspected to be intractable (NP C). A very famos nsolved problem is the qestion of whether or not NP C P ; that is, whether or not the seemingly intractable problems are in fact intractable. Why aim for sch a categorization? Before yo set ot to solve a problem, it is sefl to determine first if it is in NP C. If it is, yo know that most experts believe it nlikely that yo will ever find an efficient (polynomial-time) soltion. Unless yo are very ambitios and have a lot of time to devote to the problem, yo may decide at this point to abandon hope for a polynomial-time soltion and go with a non-polynomial time algorithm and confine yorself to relatively small problem instances. Alternatively, yo can se an approximation algorithm that might give a good answer (bt not garanteed to be the best) in polynomial time. Finally, yo might reformlate the original problem to a version that is in P. (Often the distinction between a problem in P and NP C can be sbtle.) We will develop a theory for decision problems; that is, problems that have a yes or no answer. For example, we considered the shortest path problem: Given a weighted digraph G and vertices s and t, what is the weight of the lightest path from vertex s to vertex t? The decision version of this problem is: Given a weighted digraph G, vertices s and t, and integer, is there a path of weight at most between s and t? The decision version is certainly no harder than the original optimization problem. If we can solve the optimization problem, then we only need to compare the optimal soltion with to solve the decision problem. The class of decision problems for which polynomial time algorithms exist is denoted P. Sppose that we are given an instance of the decision version of the shortest path problem with a yes answer, and someone offers a certificate that demonstrates that the problem is a yes instance. What form might this certificate take? Sppose it is a path from s to t that has length. The path has at most V edges, and so we can certainly check the certificate in time that is bonded by a polynomial in the size of the problem instance. This problem therefore has the sccinct certificate property: given a yes instance, there is at least one sccinct certificate (and possibly many) that verifies that the problem is a yes instance

2 in time that is bonded by a polynomial in the size of the problem instance. A no instance of the problem has no sch certificate. The decision version of the traveling salesman problem, which we will call TSP is: given n cities and distance matrix d and integer, is there a tor of the n cities with total length? This problem has the sccinct certificate property. If we are given a tor (permtation of the n cities), we can check in O(n) time to see if it has length. The class of yes-no problems that have the sccinct certificate property is called NP. (NP stands for nondeterministic polynomial.) Clearly P NP, since if the problem can be solved in polynomial time, then the soltion constrcted by the algorithm can certainly be checked in polynomial time. The decision version of the knapsack problem we discssed in class, which we call NAPSAC, is: Given a weight bdget W, n items of vale v i and weight w i, i =,,..., n, and vale, is there a choice of the items that weigh W and have total vale? We saw (in homework problem 6.-) that the optimization version can be solved in time O(nW ), so of corse the decision version can also be solved in time O(nW ). Does this mean that NAPSAC is in P? To answer this qestion, we mst be carefl abot the size of the inpt. It is fair to consider the data abot the n items to have size O(n), bt what abot W? In binary representation it has lg(w ) bits. So the size of the inpt is really abot n lg W. Therefore, the rnning time is exponential in the size of the problem instance. The decision version of knapsack does have the sccinct certificate property, however, and so it is in NP. Is it in P? We don t know; we have only arged that the particlar algorithm we came p with does not show it to be in P since it is not a polynomial time algorithm. The field of logic provides many examples of decision problems. The CIRCUIT- VALUE problem is: Given a Boolean circit and its inpts, is the otpt T? This problem is in P (see below). The SAT problem is: Given a Boolean circit, is there a way to assign vales to its inpts so that the otpt is T? While sperficially similar, SAT seems to be a mch more difficlt problem. The obvios soltion is to try all inpts, of which there are exponentially many. The linear programming problem (LP) is: Given an n m matrix A and an m vector b, are there nonnegative real nmbers x, x,..., x n satisfying Ax b? The integer linear programming problem (ILP), is: Given an n m matrix A and an m vector b, are there nonnegative integers x, x,..., x n satisfying Ax b? It trns ot that LP is in P, while ILP appears to be exponential. We next define a class of decision problems which are the hardest problems in the class NP. We first need to introdce redctions. Let A and B be two decision problems. A redction from A to B is a polynomialtime algorithm R which transforms inpts of A to eqivalent inpts to B. If x is an instance of problem A, then R(x) is an instance of problem B, and x is a yes instance of A if and only if R(x) is a yes instance of B. If we have a polynomial-time algorithm for problem B, and a redction from A to B, then we have a polynomial-time algorithm for problem A. To see this, sppose that or polynomial-time algorithm for problem B has rnning time T (n) = O(n k ) for some k, and sppose that or redction R has rnning time O(n j ). To solve an instance

3 x of A of length n, first convert x to R(x), an instance of B, in time O(n j ); notice that the length of R(x) is at most O(n j ). Then solve R(x), which takes time O((n j ) k ) = O(n jk ). Cook s Theorem: A problem is in NP if and only if it can be redced to SAT. Part of the theorem is easy to prove. If a problem can be redced to SAT, then it is in NP. A certificate of any yes instance wold be the redction of the instance with a certificate for the reslting inpt of SAT. Here is a sketch of the plasibility of the other half of the theorem: If a problem A is in NP then it can be redced to SAT. Since A is in NP, there exists a polynomial-time algorithm to check problem instances and certificates for validity. By combining polynomially many Boolean circits, we can implement the algorithm that does the validity checking (the inpt gates correspond to the bits of the problem instance and the certificate). Let x be an instance of problem A. Plg in the correct T/F vales for the inpt gates of the circit corresponding to the problem instance, and leave nknown the gates corresponding to the certificate. We now have an instance of SAT which captres the qestion of whether a valid certificate for x exists; that is, whether x is a yes instance of A. Therefore, we have redced A to SAT. If a problem is in NP and all other problems in NP redce to it, then we say that the problem is NP-Complete. Ths an NP-complete problem is (p to a polynomial) as hard as any other problem in NP. Cook s theorem gives s or first example of an NP-Complete problem.

4 . Examples of Redctions In these notes we will illstrate polynomial redctions. This allows one to show that a problem is NP-complete if we redce a known NP-complete problem to it. Let {,,, n } be a collection of Boolean variables. A trth assignment assigns a vale T of F to each variable. We se a bar to denote complement: ū is T if and only if is F. If is a variable, then and ū are literals. A clase is a set of literals, representing the disjnction (OR) of those literals, for example ( ū 5 ). This clase will be satisfied nless = F, = T, and 5 = F. A collection C of clases is satisfiable if and only if there is a trth assignment that simltaneosly satisfies all the clases in C. We will write sch a collection of clases by connecting them with (read AND ); sch a Boolean expression is in conjnctive normal form (CNF). The satisfiability problem (abbreviated SAT) is: Given a collection of clases C over variables U, is there a satisfying trth assignment for C? Cook s theorem states that SAT is NP-complete. We will now redce SAT to some other interesting problems, ths showing that they are NP-complete. A set of clases is in conjnctive normal form (-CNF) if each clase has exactly literals. The -SAT problem is the special case of SAT in which each clase has exactly three literals. SAT redces to -SAT Consider any formla F in CNF, consisting of clases C, C,..., C m over variables {,,..., n }. We shall constrct a new formla F with exactly three literals per clase that is satisfiable if and only if F is. We do this by examining each clase C i in trn, replacing it by an eqivalent set of clases, introdcing new variables if necessary. If C i already has exactly three literals, we do nothing. Otherwise, there are two cases to consider: C i has fewer than three literals and C i has more than three literals. First sppose that C i has one or two literals. If C i =, then introdce new variables z, z, α, β and replace C i by the clases ( z z ) and ( z α β) ( z ᾱ β) ( z α β) ( z ᾱ β) ( z α β) ( z ᾱ β) ( z α β) ( z ᾱ β). These last clases force the variables z and z to be false in any trth assignment satisfying F, so that the clase is eqivalent to its replacement. If C i = ( ), introdce only z, α, β and disregard the clases containing z. If C i has k > literals, say ( k ), then introdce new variables z, z,..., z k, and replace C i by ( z ) ( z z ) ( z z ) ( z k k k ). This last set of clases is satisfiable if and only if C i is: If C i has a trth assignment, let l be the smallest integer sch that l is tre. Then the new clase is satisfied with the same trth assignment and with z i = T for i l and z i = F for i > l. Conversely, sppose that the new clase is satisfied by some trth assignment. If C i is not satisfied by the trth assignment, then i = F for i k. Bt this wold imply that the new clase is not satisfied. It is clear that the redction can be accomplished in polynomial time. Let G = (V, E) be a graph. A sbset I V is called an independent set if no two vertices in I are directly connected by an edge in E. For example, in Figre, {,, 6} and {, 5} are two examples of independent sets.

5 5 The Independent Set problem is: Given a graph G and an integer, is there an independent set I V containing at least vertices? Independent Set is NP-complete. Proof. It is clearly in NP. We will redce -SAT to Independent Set; i.e., given a Boolean formla F in -CNF, prodce a graph G = (V, E) and integer sch that G has an independent set of size at least if and only if F is satisfiable. Here is the constrction: Let be the nmber of clases. For each clase, create a node for each literal in the clase, connected in all possible ways (a triangle for each clase). Also draw an edge between literals of different clases if contradictory; e.g., if i appears in one clase and ū i appears in a different clase, connect the corresponding nodes with an edge. For example, consider the formla ( ) (ū ) (ū ū 5 ). The graph is illstrated in Figre. 5 Figre. Graph sed in Independent Set redction. If there is an independent set of size at least, then it mst have exactly one node from each grop. Think of the chosen node in a grop as a literal that satisfies that clase. Since contradictory literals are connected by an edge, no nodes in I are contradictory, so the literals together comprise a satisfying trth assignment (if not sed, a variable can be set to either T or F ). Conversely, sppose that we have a satisfying trth assignment for F. Pick a T literal from each clase. The corresponding set I of nodes is an independent set (they are from the same trth assignment, so not contradictory). A cliqe in a graph G = (V, E) is a flly connected set of nodes. For example, in the following graph, {,, } and {,, 5, 6} are cliqes, as is {, } G G Figre. A graph and its complement.

6 6 The complement of a graph G, denoted Ḡ, is the graph with the same nodes as G, bt only the missing edges; see Figre for an example. Cliqe is NP-complete. This follows from the fact that a cliqe of size for G corresponds to an independent set of size for G: If each pair of nodes in a set is connected by an edge (that is, the nodes form a cliqe), then in the complementary graph none of the nodes will be connected by an edge (so they form an independent set). Finally, we consider the mltiprocessor schedling problem. There are m identical processors working in parallel. There are T discrete time slots, dring which a set J of jobs mst be schedled. In addition, there are precedence constraints of the type: job j i mst be processed before job j k. These constraints are represented by an acyclic directed graph (J, A), where (j i, j k ) A means j i mst be schedled before j k. A feasible schedle is a mapping S from the jobs to time slots, sch that at most m jobs are schedled dring any one slot, and all jobs complete within the allotted time T. Mltiprocessor schedling is NP-complete. Proof. The problem is clearly in NP, since it is easy to check if a given schedle is feasible. We will now redce CLIQUE to mltiprocessor schedling. Consider a graph G = (V, E) and an integer. We will constrct a set J of jobs, an acyclic digraph (J, A) representing precedence constraints, and integers m and T sch that there exists a feasible schedle for J if and only if G has a cliqe of size. Let the set of jobs be given by J = V E {b, b,, b nb } E {c, c,, c nc } E {d, d,, d nd } where the nmbers n b, n c, n d are chosen sch that ( ) ( ) (.) k + n b = + V + n c = E + n d and min{n b, n c, n d } =. Clearly sch a choice is possible. Let m be the common vale in (.) and set T =. (Notation: ( n m) (read n choose m ) is the nmber of sbsets of size m in a set with n elements.) To constrct the precedence constraint digraph, first add to A all arcs of the form (b i, c j ) and (c j, d k ). Then add all arcs (v, e) for all v V and e E sch that e is incident on v. We will now arge that the instance of mltiprocessor schedling jst constrcted has a feasible schedle if and only if G has a cliqe of size. Sppose first that there is a feasible schedle. Since T =, it mst be that S(b i ) =, S(c i ) =, and S(d i ) = for all i, since the b jobs mst precede the c jobs which mst precede the d jobs. Also, since J = m = T m by constrction, all m machines mst be bsy dring all three time slots. Now, in the last period, besides the d i jobs, only m n d = E ( ) jobs corresponding to edges can be exected, since if a vertex job were exected, then its incident edges cold not be processed ntil period, violating the feasibility of the schedle. We conclde that the remaining ( ) edge jobs mst be exected in the second period, and the

7 7 vertex jobs are exected in the first and second periods (with m n b = of them processed in the first period). Frthermore, the vertices corresponding to jobs that are exected in the first period mst inclde the endpoints of all ( ) edges correspond ( to jobs in the second period. Bt vertices can inclde endpoints of ) edges only if they form a cliqe of size. So the existence of a feasible schedle implies the existence of a cliqe of size. Conversely, sppose that G has a cliqe of size. Define a feasible schedle S by: S(b i ) =, S(c i ) =, S(d i ) = for all i, S(v) = if v is a member of the cliqe and S(v) = otherwise; S([v, ]) = if v and are in the cliqe and S([v, ]) = otherwise. (Here [v, ] is the job corresponding to the edge between vertices v and, if there is one.)