Reducibility and Completeness

Transcription

1 Reducibility and Copleteness Chapter 28 of the forthcoing CRC Handbook on Algoriths and Theory of Coputation Eric Allender 1 Rutgers University Michael C. Loui 2 University of Illinois at Urbana-Chapaign 1 Introduction Kenneth W. Regan 3 State University of New York at Buffalo There is little doubt that the notion of reducibility is the ost useful tool that coplexity theory has delivered to the rest of the coputer science counity. For ost coputational probles that arise in real-world applications, such as the Traveling Salesperson Proble, we still know little about their deterinistic tie or space coplexity. We cannot now tell whether classes such as P and NP are distinct. And yet, even without such hard knowledge, it has been useful in practice to take soe new proble A whose coplexity needs to be analyzed, and announce that A has roughly the sae coplexity as Traveling Salesperson, by exhibiting efficient ways of reducing each proble to the other. Thus we can say a lot about probles being equivalent in coplexity to each other, even if we cannot pinpoint what that coplexity is. One reason this has succeeded is that, when one partitions the any thousands of real-world coputational probles into equivalence classes according to the reducibility relation, there is a surprisingly sall nuber of classes of this partition. Thus, the coplexity of alost any proble arising in practice can be classified by showing that it is equivalent to one of a short list of representative probles. It was not originally expected that this would be the case. 1 Supported by the National Science Foundation under Grant CCR Portions of this work were perfored while a visiting scholar at the Institute of Matheatical Sciences, Madras, India. 2 Supported by the National Science Foundation under Grant CCR Supported by the National Science Foundation under Grant CCR

2 Even ore aazingly, these representative probles correspond in a very natural way to abstract odels of coputation that is, they correspond to coplexity classes. These classes were defined in the last chapter using a sall set of abstract achine concepts: Turing achines, nondeterinis, alternation, tie, space. With this and a few siple functions that define tie and space bounds, we are able to characterize the coplexity of the overwheling ajority of natural coputational probles ost of which bear no topical reseblance to any question about Turing achines. This tool has been uch ore successful than we had any right to expect it would be. All this leads us to believe that it is no ere accident that probles easily lend theselves to being placed in one class or another. That is, we are disposed to think that these classes really are distinct, that the classification is real, and that the atheatics developed to deal with the really does describe soe iportant aspect of nature. Nondeterinistic Turing achines, with their ability to agically soar through iense search spaces, see to be uch ore powerful than our undane deterinistic achines, and this reinforces our belief. However, until P vs. NP and siilar long-standing questions of coplexity theory are copletely resolved, our best ethod of understanding the coplexity of real-world probles is to use the classification provided by reducibility, and to trust in a few plausible conjectures. In this chapter, we discuss reducibility. We will learn about different types of reducibility, and the related notion of copleteness. It is especially useful to understand NP-copleteness. We define NP-copleteness precisely, and give exaples of NP-coplete probles. We show how to prove that a proble is NP-coplete, and give soe help for coping with NP-copleteness. After that, we describe probles that are coplete for other coplexity classes, under the ost efficient reducibility relations. Finally, we cover two other iportant topics in coplexity theory that are otivated by reducibility: relativized coputation and the study of sparse languages. 2 Reducibility Relations In atheatics, as in everyday life, a typical way to solve a new proble is to reduce it to a previously solved proble. Frequently, an instance of the new proble is expressed copletely in ters of an instance of the prior proble, and the solution is then interpreted in the ters of the new proble. This kind of reduction is called any-one reducibility, and is defined below. A different way to solve the new proble is to use a subroutine that solves the prior proble. 2

3 For exaple, we can solve an optiization proble whose solution is feasible and axiizes the value of an objective function g by repeatedly calling a subroutine that solves the corresponding decision proble of whether there exists a feasible solution x whose value g(x) satisfies g(x) k. This kind of reduction is called Turing reducibility, and is also defined below. Let A 1 and A 2 be languages. A 1 is any-one reducible to A 2, written A 1 A 2, if there exists a total recursive function f such that for all x, x A 1 if and only if f(x) A 2. The function f is called the transforation function. A 1 is Turing reducible to A 2, written A 1 T A 2, if A 1 can be decided by a deterinistic oracle Turing achine M using A 2 as its oracle, i.e., A 1 = L(M A 2 ). (Recursive functions are defined in Chapter 26, and oracle achines are defined in Chapter 24. The oracle for A 2 odels a hypothetical efficient subroutine for A 2.) If f or M above consues too uch tie or space, the reductions they copute are not helpful. To study coplexity classes defined by bounds on tie and space resources, it is natural to consider resource-bounded reducibilities. Let A 1 and A 2 be languages. A 1 is Karp reducible to A 2, written A 1 p A 2, if A 1 is any-one reducible to A 2 via a transforation function that is coputable deterinistically in polynoial tie. A 1 is Cook reducible to A 2, written A 1 p T A 2, if A 1 is Turing reducible to A 2 via a deterinistic oracle Turing achine of polynoial tie coplexity. The ter polynoial-tie reducibility usually refers to Karp reducibility. If A 1 p A 2 and A 2 p A 1, then A 1 and A 2 are equivalent under Karp reducibility. Equivalence under Cook reducibility is defined siilarly. Karp and Cook reductions are useful for finding relationships between languages of high coplexity, but they are not useful at all for distinguishing between probles in P, because all probles in P are equivalent under Karp (and hence Cook) reductions. (Here and later we ignore the special cases A 1 = and A 1 = Σ, and consider the to reduce to any language.) To investigate the any interesting coplexity classes inside P, we will want to define ore restrictive reducibilities, and this we do beginning in Section 5. For the tie being, however, we focus on Cook and Karp reducibility. The key property of Cook and Karp reductions is that they preserve polynoial-tie feasibility. Suppose A 1 p A 2 via a transforation f. If M 2 decides A 2, and M f coputes f, then to decide 3

4 whether an input word x is in A 1, we ay use M f to copute f(x), and then run M 2 on input f(x). If the tie coplexities of M 2 and M f are bounded by polynoials t 2 and t f, respectively, then on inputs x of length n = x, the tie taken by this ethod of deciding A 1 is at ost t f (n)+t 2 (t f (n)), which is also a polynoial in n. In suary, if A 2 is feasible, and there is an efficient reduction fro A 1 to A 2, then A 1 is feasible. Although this is a siple observation, this fact is iportant enough to state as a theore. First, though, we need the concept of closure. A class of languages C is closed under a reducibility r if for all languages A 1 and A 2, whenever A 1 r A 2 and A 2 C, necessarily A 1 C. Theore 2.1 P is closed under both Cook and Karp reducibility. Note that this is an instance of an idea that otivated our identification of P with the class of feasible probles in Section 2.2 of Chapter 27, naely that the coposition of two feasible functions should be feasible. Siilar considerations give us the following theore. Theore 2.2 Karp reducibility and Cook reducibility are transitive; i.e.: 1. If A 1 p A 2 and A 2 p A 3, then A 1 p A If A 1 p T A 2 and A 2 p T A 3, then A 1 p T A 3. We shall see the iportance of closure under a reducibility in conjunction with the concept of copleteness, which we define in the next section. 3 Coplete Languages and Cook s Theore Let C be a class of languages that represent coputational probles. A language A 0 is C-hard under a reducibility r if for all A in C, A r A 0. A language A 0 is C-coplete under r if A 0 is C-hard, and A 0 C. Inforally, if A 0 is C-hard, then A 0 represents a proble that is at least as difficult to solve as any proble in C. If A 0 is C-coplete, then in a sense, A 0 is one of the ost difficult probles in C. There is another way to view copleteness. Copleteness provides us with tight lower bounds on the coplexity of probles. If a language A is coplete for coplexity class C, then we have a lower bound on its coplexity. Naely, A is as hard as the ost difficult proble in C, assuing 4

5 that the coplexity of the reduction itself is sall enough not to atter. The lower bound is tight because A is in C; that is, the upper bound atches the lower bound. In the case C = NP, the reducibility r is usually taken to be Karp reducibility unless otherwise stated. Thus we say: A language A 0 is NP-hard if A 0 is NP-hard under Karp reducibility. A 0 is NP-coplete if A 0 is NP-coplete under Karp reducibility. However, any sources take the ter NP-hard to refer to Cook reducibility. Many iportant languages are now known to be NP-coplete. Before we get to the, let us discuss soe iplications of the stateent A 0 is NP-coplete, and also soe things this stateent doesn t ean. The first iplication is that if there exists a deterinistic Turing achine that decides A 0 in polynoial tie that is, if A 0 P then because P is closed under Karp reducibility (Theore 2.1 in Section 2), it would follow that NP P, hence P = NP. In essence, the question of whether P is the sae as NP coes down to the question of whether any particular NP-coplete language is in P. Put another way, all of the NP-coplete languages stand or fall together: if one is in P, then all are in P; if one is not, then all are not. Another iplication, which follows by a siilar closure arguent applied to co-np, is that if A 0 co-np then NP = co-np. It is also believed unlikely that NP = co-np, as was noted in connection with whether all tautologies have short proofs in Section 2.2 of Chapter 27. A coon isconception is that the above property of NP-coplete languages is actually their definition, naely: if A NP, and A P iplies P = NP, then A is NP-coplete. This definition is wrong. A theore due to Ladner [Ladner, 1975b] shows that P NP if and only if there exists a language A in NP P such that A is not NP-coplete. Thus, if P NP, then A is a counterexaple to the definition. Another coon isconception arises fro a isunderstanding of the stateent If A 0 is NPcoplete, then A 0 is one of the ost difficult probles in NP. This stateent is true on one level: if there is any proble at all in NP that is not in P, then the NP-coplete language A 0 is one such proble. However, note that there are NP-coplete probles in NTIME[n] and these probles 5

6 are, in soe sense, uch sipler than any probles in NTIME[n ]. We discuss the difficulty of NP-coplete probles in ore detail below after seeing several exaples. We now prove Cook s Theore, which established the first iportant NP-coplete proble. Recall the definition of SAT, the language of satisfiable Boolean forulas, fro Section 2.2 of Chapter 27. In this and later Karp-reduction proofs, we highlight the construction of the transforation f, check that the coplexity of f is polynoial, and verify the correctness of the reduction. Theore 3.1 (Cook s Theore) SAT is NP-coplete. Proof. Let A NP. Without loss of generality we ay assue that A { 0, 1 }. There is a polynoial q and a polynoial-tie coputable relation R such that for all x, x A ( y : y = q( x )) R(x, y). By the construction in Theore 3.1 of Chapter 27, there is a polynoial p such that for all n, we can build in tie p(n) a Boolean circuit C n, using only binary NAND gates, that decides R on inputs of length n + q(n). C n has n input nodes labeled x 1,..., x n and q = q(n) ore input nodes labeled y 1,..., y q. C n has at ost p(n) wires, which we label by w 1,..., w, where p(n) and w is a special wire leading out of the output gate. Construction. We first write a Boolean forula φ n in the x, y, and w variables to express that every gate in C n functions correctly and C n outputs 1. For every NAND gate in C n with incoing wires u, v, and for each outgoing wire w of the gate, we add to φ n the following conjunction of three clauses (u w) (v w) (ū v w). These clauses are satisfied by those assignents to u, v, w such that w = (u v). Intuitively, they assert that the given NAND gate functions correctly. Thus φ n has three clauses for every wire w except those wires leading fro the inputs, each of which carries the label of the corresponding input variable. Finally for the output wire, φ n has the singleton clause (w ). So φ n has at ost 3p(n) + 1 clauses in all. Now given x, we for the desired forula f(x) = φ x by building φ n, where n = x, and siply appending n singleton clauses that force the corresponding assignent to the x 1,..., x n variables. (For exaple, if x = 1001, append x 1 x 2 x 3 x 4.) 6

7 Coplexity. C n is built up in roughly O(p(n)) tie. Building φ n fro C n, and appending the singleton clauses for x, takes a siilar polynoial aount of tie. Correctness. Forally, we need to show that for all x, x A f(x) SAT. By construction, for all x, x A if and only if there exists an assignent to the y variables and to the w variables that satisfies φ x. Hence the reduction is correct. A glance at the proof shows that φ x is always a Boolean forula in conjunctive noral for (CNF) with clauses of one, two, or three literals each. By introducing soe new duy variables, we can arrange that each clause has exactly three literals. Thus we have actually shown that the following restricted for of the satisfiability proble is NP-coplete: 3-Satisfiability (3SAT) Instance: A Boolean expression φ in conjunctive noral for with three literals per clause. Question: Is φ satisfiable? One concrete iplication of Cook s Theore is that if deciding SAT is easy (i.e., in polynoial tie), then factoring integers is likewise easy, because the decision version of factoring belongs to NP. (See Chapter 39.) This is a surprising connection between two ostensibly unrelated probles. The ain ipact, however, is that once one language has been proved coplete for a class such as NP, others can be proved coplete by constructing transforations. If A 0 is NP-coplete, then to prove that another language A 1 is NP-coplete, it suffices to prove that A 1 NP, and to construct a polynoial-tie transforation that establishes A 0 p A 1. Since A 0 is NP-coplete, for every language A in NP, A p A 0, hence by transitivity (Theore 2.2 in Section 2), A p A 1. Hundreds of coputational probles in any fields of science and engineering have been proved to be NP-coplete, alost always by reduction fro a proble that was previously known to be NP-coplete. We give soe practically-otivated exaples of these reductions, and also soe advice on how to cope with NP-copleteness. 4 NP-Coplete Probles and Copleteness Proofs This and the next two sections are directed toward practitioners who have a coputational proble, don t know how to solve it, and want to know how hard it is specifically, is it NP-coplete, or 7

8 NP-hard? The following step-by-step procedure will help in answering these questions, and ay help in identifying cases of the proble that are tractable even if the proble is NP-hard for general cases. In brief, the steps are: 1. State the proble in general atheatical ters, and foralize the stateent. 2. Ascertain whether the proble belongs to NP. 3. If so, try to find it in a copendiu of known NP-coplete probles. 4. If you cannot find it, try to construct a reduction fro a related proble that is known to be NP-coplete or NP-hard. 5. Try to identify special cases of your proble that are (i) hard, (ii) easy, and/or (iii) the ones you need. Your work in steps ay help you here. 6. Even if your cases are NP-hard, they ay still be aenable to direct attack by sophisticated ethods on high-powered hardware. These steps are interspersed with a traditional theore proof presentation and several long exaples, but the sae sequence is aintained. We ephasize that trying to do the foralization and proofs asked for in these steps ay give you useful positive inforation about your proble. Step 1. Give a foral stateent of the proble. State it without using ters that are specific to your own particular discipline. Use coon ters fro atheatics and data objects in coputer science, e.g. graphs, trees, lists, vectors, atrices, alphabets, strings, logical forulas, atheatical equations. For exaple, a proble in evolutionary biology that a phylogenist would state in ters of species and characters and cladogras can be stated in ters of trees and strings, using an alphabet that represents the taxonoic characters. Standard notions of size, depth, and distance in trees can express the objectives of the proble. If your proble involves coputing a function that produces a lot of output, look for associated yes/no decision probles, because decision probles have been easier to characterize and classify. For instance, if you need to copute atrices of a certain kind, see whether the essence of your proble can be captured by yes/no questions about the atrices, perhaps about individual entries 8

9 of the. Many optiization probles looking for a solution of a certain iniu cost or axiu value can be turned into decision probles by including a target cost/value k as an input paraeter, and phrasing the question of whether a solution exists of cost less than (or value greater than) the target k. Several probles given in the exaples below have this for. It ay also help to siplify, even over-siplify, your proble by reoving or ignoring soe particular eleents of it. Doing so ay ake it easier to ascertain what general category of decision proble yours is in or closest to. In the process, you ay learn useful inforation about the proble that tells you what the effects of those specific eleents are we say soe ore about this in Section 4.2 below. Step 2. When you have an adequate foralization, ask first: does your decision proble belong to NP? This is true if and only if candidate solutions that would bring about a yes answer can be tested in polynoial tie see the extended discussion in Chapter 27, Section 2.2. If it does belong to NP, that s good news for now! Even if not, you ay proceed to deterine whether it is NP-hard. The proble ay be coplete for a class such as PSPACE that contains NP. Exaples of such probles are given later in this chapter. Step 3. See whether your proble is already listed in a copendiu of (NP-)coplete probles. The book [Garey and Johnson, 1988] lists several hundred NP-coplete probles arranged by category. The following is intended as a sall representative saple. The first five (together with 3SAT) receive extended treatent in [Garey and Johnson, 1988], while the last five receive coparable treatent here. (The language corresponding to each proble is the set of instances whose answers are yes. ) Vertex Cover Instance: A graph G and an integer k. Question: Does G have a set W of k vertices such that every edge in G is incident on a vertex in W? Clique Instance: A graph G and an integer k. 9

10 Question: Does G have a set K of k vertices such that every two vertices in K are adjacent in G? Hailtonian Circuit Instance: A graph G. Question: Does G have a circuit that includes every vertex exactly once? 3-Diensional Matching Instance: Sets W, X, Y with W = X = Y = q and a subset S W X Y. Question: Is there a subset S S of size q such that no two triples in S agree in any coordinate? Partition Instance: A set S of positive integers. Question: Is there a subset S S such that the su of the eleents of S equals the su of the eleents of S S? Independent Set Instance: A graph G and an integer k. Question: Does G have a set U of k vertices such that no two vertices in U are adjacent in G? Graph Colorability Instance: A graph G and an integer k. Question: Is there an assignent of colors to the vertices of G so that no two adjacent vertices receive the sae color, and at ost k colors are used overall? Traveling Salesperson (TSP) Instance: A set of cities C 1,..., C, with a distance d(i, j) between every pair of cities C i and C j, and an integer D. Question: Is there a tour of the cities whose total length is at ost D, i.e., a perutation c 1,..., c of {1,..., }, such that d(c 1, c 2 ) + + d(c 1, c ) + d(c, c 1 ) D? 10

11 Knapsack Instance: A set U = {u 1,..., u } of objects, each with an integer size size(u i ) and an integer profit profit(u i ), a target size s 0, and a target profit p 0. Question: Is there a subset U U whose total cost and total profit satisfy u i U size(u i ) s 0 and u i U profit(u i ) p 0? The languages of all of these probles are easily seen to belong to NP. For exaple, to show that TSP is in NP, one can build a nondeterinistic Turing achine that siply guesses a tour and checks that the tour s total length is at ost D. Soe coents on the last two probles are relevant to steps 1. and 2. above. Traveling Salesperson provides a single abstract for for any concrete probles about sequencing a series of test exaples so as to iniize the variation between successive ites. The Knapsack proble odels the filling of a knapsack with ites of various sizes, with the goal of axiizing the total value (profit) of the ites. Many scheduling probles for ultiprocessor coputers can be expressed in the for of Knapsack instances, where the size of an ite represents the length of tie a job takes to run, and the size of the knapsack represents an available block of achine tie. If yours is on the list of NP-coplete probles, you ay skip Step 4, and the copendiu ay give you further inforation for Steps 5 and 6. You ay still wish to pursue Step 4 if you need ore study of particular transforations to and fro your proble. If your proble is not on the list, it ay still be close enough to one or ore probles on the list to help with the next step. Step 4. Construct a reduction fro an already-known NP-coplete proble. Broadly speaking, Karp reductions coe in three kinds. A restriction fro your proble to a special case that is already known to be NP-coplete. A inor adjustent of an already-known proble. A cobinatorial transforation. 11

12 The first two kinds of reduction are usually quite easy to do, and we give several exaples before proceeding to the third kind. Exaple. Partition p Knapsack, by restriction: Given a Partition instance with integers s i, the corresponding instance of Knapsack takes size(u i ) = profit(u i ) = s i (for all i), and sets the targets s 0 and p 0 both equal to ( i s i)/2. The condition in the definition of the Knapsack proble of not exceeding s 0 nor being less than p 0 requires that the su of the selected ites eet the target ( i s i)/2 exactly, which is possible if and only if the original instance of Partition is solvable. In this way, the Partition proble can be regarded as a restriction or special case of the Knapsack proble. Note that the reduction itself goes fro the ore-special proble to the oregeneral proble, even though one thinks of the ore-general proble as the one being restricted. The iplication is that if the restricted proble is NP-hard, then the ore-general proble is NP-hard as well, not vice-versa. Exaple. Hailtonian Circuit p TSP by restriction: Let a graph G be given as an instance of the Hailtonian Circuit proble, and let G have vertices v 1,..., v. These vertices becoe the cities of the TSP instance that we build. Now define a distance function d as follows: { 1 if (vi, v d(i, j) = j ) is an edge in G + 1 otherwise. Set D =. Clearly, d and D can be coputed in polynoial tie fro G. If G has a Hailtonian circuit, then the length of the tour that corresponds to this circuit is exactly. Conversely, if there is a tour whose length is at ost, then each step of the tour ust have distance 1, not +1. Then each step corresponds to an edge of G, so the corresponding sequence of vertices fors a Hailtonian circuit in G. Thus the function f defined by f(g) = ({ d(i, j) : 1 i, j }, D) is a polynoial-tie transforation fro Hailtonian Circuit to TSP. 4 Minor Adjustents. Here we consider cases where two probles look different but are really closely connected. Consider Clique, Independent Set, and Vertex Cover. A graph G has a clique of size k if and only if its copleentary graph G has an independent set of size k. It follows 4 Technically we need f to be a function fro Σ to Σ. However, given a string x we can decide in polynoial tie whether x encodes a graph G that can be given as an instance of Hailtonian Circuit. If x does not encode a well-fored instance, then define f(x) to be a fixed instance I 0 of TSP for which the answer is no. Because this sort of thing can generally always be done, we are free to regard the doain of a reduction function f to be the set of well-fored instances of the proble we are reducing fro. Henceforth we try to ignore such encoding details. 12

13 that the function f defined by f(g, k) = (G, k) is a Karp reduction fro Independent Set to Clique. To forge a link to the Vertex Cover proble, note that all vertices not in a given vertex cover for an independent set, and vice versa. Thus a graph G on n vertices has a vertex cover of size at ost k if and only if G has an independent set of size at least n k. Hence the function g(g, k) = (G, n k) is a Karp reduction fro Independent Set to Vertex Cover. (Note that the sae f and g also provide reductions fro Clique to Independent Set and fro Vertex Cover to Independent Set, respectively. This does not happen for all reductions, and gives a sense in which these three probles are unusually close to each other.) 4.1 NP-Copleteness by Cobinatorial Transforation The following exaples show how the cobinatorial echanis of one proble (here, 3SAT) can be transfored by a reduction into the seeingly uch different echanis of another proble. Theore 4.1 Independent Set is NP-coplete. Hence also Clique and Vertex Cover are NP-coplete. Proof. We have rearked already that the languages of these three probles belong to NP, and shown already that Independent Set p Clique and Independent Set p Vertex Cover. It suffices to show that 3SAT p Independent Set. Construction. Let the Boolean forula φ be a given instance of 3SAT with variables x 1,..., x n and clauses C 1,..., C. The graph G φ we build consists of a ladder on 2n vertices labeled x 1, x 1,..., x n, x n, with edges (x i, x i ) for 1 i n foring the rungs, and clause coponents. Here the coponent for each clause C j has one vertex for each literal x i or x i in the clause, and all pairs of vertices within each clause coponent are joined by an edge. Finally, each clausecoponent node with a label x i is connected by a crossing edge to the node with the opposite label x i in the ith rung, and siilarly each occurrence of x i in a clause is joined to the rung node x i. This finishes the construction of G φ. See Figure 1. Also set k = n +. Then the reduction function f is defined for all arguents φ by f(φ) = (G φ, k). Coplexity. It is not hard to see that f is coputable in polynoial tie given (a straightforward encoding of) φ. 13

14 x 1 x 2 x 3 x 1 I I x x 1 1 x 3 1 I 1 1 x 2 1 x 2 x 1 1 I x 1 3 x I Figure 1: Construction in the proof of NP-copleteness of Independent Set for the forula (x 1 x 2 x 3 ) (x 1 x 2 x 3 ). The independent set of size 5 corresponding to the satisfying assignent x 1 = false, x 2 = true, and x 3 = true is shown by nodes arked I. 14

15 Correctness. To coplete the proof, we need to argue that φ is satisfiable if and only if G φ has an independent set of size n +. To see this, first note that any independent set I of that size ust contain exactly one of the two nodes fro each rung, and exactly one node fro each clause coponent because the edges in the rungs and the clause coponent prevent any ore nodes fro being added. And if I selects a node labeled x i in a clause coponent, then I ust also select x i in the ith rung. If I selects x j in a clause coponent, then I ust also select x j in the rung. In this anner I induces a truth assignent in which x i = true and x j = false, and so on for all variables. This assignent satisfies φ, because the node selected fro each clause coponent tells how the corresponding clause is satisfied by the assignent. Going the other way, if φ has a satisfying assignent, then that assignent yields an independent set I of size n + in like anner. Since the φ in this proof is a 3SAT instance, every clause coponent is a triangle. The idea, however, also works for CNF forulas with any nuber of variables in a clause, such as the φ x in the proof of Cook s Theore. Now we odify the above idea to give another exaple of an NP-copleteness proof by cobinatorial transforation. Theore 4.2 Graph Colorability is NP-coplete Proof. Construction. Given the 3SAT instance φ, we build G φ siilarly to the last proof, but with several changes. See Figure 2. On the left, we add a special node labeled B and connect it to all 2n rung nodes. On the right we add a special node G with an edge to B. In any possible 3-coloring of G φ, without loss of generality B will be colored blue and the adjacent G will be colored green. The third color red stands for literals ade true, whereas green stands for falsity. Now for each occurrence of a positive literal x i in a clause, the corresponding clause coponent has two nodes labeled x i and x i with an edge between the; and siilarly an occurrence of a negated literal x j gives nodes x j and x j with an edge between the. The pried ( inner ) nodes in each coponent are connected by edges into a triangle, but the unpried ( outer ) nodes are not. Each outer node of each clause coponent is instead connected by an edge to G. Finally, each 15

16 x 1 1 R 1 R x x x 3 x 2 x 2 R 1 x 2 x x 3 x 3 x R 1 1 R B x 1 G Figure 2: Construction in the proof of NP-copleteness of Graph Colorability for the forula (x 1 x 2 x 3 ) (x 1 x 2 x 3 ). The nodes shown colored R correspond to the satisfying assignent x 1 = false, x 2 = true, and x 3 = true, and these together with G and B essentially force a 3-coloring of the graph, which the reader ay coplete. Note the reseblance to Figure 1. 16

17 outer node x i is connected by a crossing edge to the rung node x i, and each outer node x j to rung node x j, exactly as in the Independent Set reduction. This finishes the construction of G φ. Coplexity. The function f that given any φ outputs G φ, also fixing k = 3, is clearly coputable in polynoial tie. Correctness. The key idea is that every three-coloring of B, G, and the rung nodes, which corresponds to a truth assignent to the variables of φ, can be extended to a 3-coloring of a clause coponent if and only if at least one of the three crossing edges fro the coponent goes to a green rung node. If all three of these edges go to red nodes, then the links to G force each outer node in the coponent to be colored blue but then it is ipossible to three-color the inner triangle since blue cannot be used. Conversely, any crossing edge to a green node allows the outer node x i or x j to be colored red, so that one red and two blues can be used for the outer nodes, and this allows the inner triangle to be colored as well. Hence G φ is 3-colorable if and only if φ is satisfiable. Note that we have also shown that the restricted for of Graph Colorability with k fixed to be 3 (i.e., given a graph G, is G 3-colorable?) is NP-coplete. Had we stated the proble this way originally, we would now conclude instead that the ore-general graph-colorability proble is NP-coplete, siilarly to the Knapsack and TSP exaples above. Many other reductions fro 3SAT use the sae basic pattern of a truth-assignent selection coponent for the variables, coponents for the clauses (whose behavior depends on whether a variable in the clause is satisfied), and links between these coponents that ake the reduction work correctly. For another exaple of this pattern, a standard proof that Hailtonian Circuit is NP-coplete uses subgraphs V i for each pair x i, x i and C j for each clause. There are two possible ways a circuit can enter V i, and these correspond to the choices of x i = true or x i = false in an assignent. The whole graph is built so that if the circuit enters V i on the x i = true side, then the circuit has the opportunity to visit all nodes in the C j coponents for all clauses in which x i occurs positively, and siilarly for occurrences of x i if the circuit enters on the negative side. Hence the circuit can run through every C j if and only if φ is satisfiable. Full details ay be found in [Papadiitriou, 1994]. For our last fully-worked-out exaple, we show a soewhat different pattern in which the individual variables as well as the clauses correspond to top-level coponents 17

18 of the following proble. Disjoint Connecting Paths Instance: A graph G with two disjoint sets of distinguished vertices s 1,..., s k and t 1,..., t k, where k 1. Question: Does G contain paths P 1,..., P k, with each P i going fro s i to t i, such that no two paths share a vertex? Theore 4.3 Disjoint Connecting Paths is NP-coplete. Proof. First, it is easy to see that Disjoint Connecting Paths belongs to NP: one can design a polynoial-tie nondeterinistic Turing achine that siply guesses k paths and then deterinistically checks that no two of these paths share a vertex. Now let φ be a given instance of 3SAT with n variables and clauses. Take k = n +. Construction and coplexity. The graph G φ we build has distinguished path-origin vertices s 1,..., s n for the variables and S 1,..., S for the clauses of φ. G φ also has corresponding sets of path-destination nodes t 1,..., t n and T 1,..., T. The other vertices in G φ are nodes u ij for each occurrence of a positive literal x i in a clause C j, and nodes v ij for each occurrences of a negated literal x i in C j. For each i, 1 i n, G φ is given the edges for a directed path fro s i through all u ij nodes to t i, and another fro s i through all v ij nodes to t i. (If there are no occurrences of the positive literal x i in any clause then the forer path is just an edge fro s i right to t i, and likewise for the latter path if the negated literal x i does not appear in any clause.) Finally, for each j, 1 j, G φ has an edge fro S j to every node u ij or v ij for the jth clause, and edges fro those nodes to T j. Clearly these instructions can be carried out to build G φ in polynoial tie given φ. (See Figure 3.) Correctness. The first point is that for each i, no path fro s i to t i can go through both a u-node and a v-node. Setting x i true corresponds to avoiding u-nodes, and setting x i false entails avoiding v-nodes. Thus the choices of such paths for all i represent a truth assignent. The key point is that for each j, one of the three nodes between S j and T j will be free for the taking if and only if the corresponding positive or negative literal was ade true in the assignent, thus satisfying the clause. Hence G φ has the n + required paths if and only if φ is satisfiable. 18

19 s 1 s 2 s S 1 1 1u 11 1u 21 1v 31 1 T 1 S 2 1 1u 12 1v 22 1u 32 1 T t 1 t 2 t 3 Figure 3: Construction in the proof of NP-copleteness of Disjoint Connecting Paths for the forula (x 1 x 2 x 3 ) (x 1 x 2 x 3 ). 19

20 4.2 Significance of NP-Copleteness Suppose that you have proved that your proble is NP-coplete. What does this ean, and how should you approach the proble now? Exactly what it eans is that your proble does not have a polynoial-tie algorith, unless every proble in NP has a polynoial-tie algorith; i.e., unless NP P. We have discussed above the reasons for believing that NP P. In practical ters, you can draw one definite conclusion: Don t bother looking for a agic bullet to solve the proble. A siple forula or an easily-tested deciding condition will not be available; otherwise it probably would have been spotted already during the thousands of person-years that have been spent trying to solve siilar probles. For exaple, the NP-copleteness of Graph 3-Colorability effectively ended hopes that an efficient atheatical forula for deciding the proble would pop out of research on chroatic polynoials associated to graphs. Notice that NP-hardness does not say that one needs to be extra clever to find a feasible solving algorith it says that one probably does not exist at all. The proof itself eans that the cobinatorial echanis of the proble is rich enough to siulate Boolean logic. The proof, however, ay also unlock the door to finding saving graces in Steps 5 and 6. Step 5. Analyze the instances of your proble that are in the range of the reduction. You ay tentatively think of these as hard cases of the proble. If these differ arkedly fro the kinds of instances that you expect to see, then this difference ay help you refine the stateent and conditions of your proble in ways that ay actually define a proble in P after all. To be sure, avoiding the range of one reduction still leaves wide-open the possibility that another reduction will ap into your instances of interest. However, it often happens that special cases of NP-coplete probles belong to P and often the boundary between these and the NP-coplete cases is sudden and sharp. For one exaple, consider SAT. The restricted case of three variables per clause is NP-coplete, but the case of two variables per clause belongs to P. For another exaple, note that the proof of NP-copleteness for Disjoint Connecting Paths given above uses instances in which k = n + ; i.e., in which k depends on the nuber of variables. The case k = 2, where you are given G and s 1, s 2, t 1, t 2 and need to decide whether there are vertex- 20

21 disjoint paths fro s 1 to t 1 and fro s 2 to t 2, belongs to P. (The polynoial-tie algorith for this case is nontrivial and was not discovered until 1978, as noted in [Garey and Johnson, 1988].) However, one ust also be careful in one s expectations. Suppose we alter the stateent of Disjoint Connecting Paths by requiring also that no two vertices in two different paths ay have an edge between the. Then the case k = 2 of the new proble is NP-coplete. (Showing this is a nice exercise; the idea is to ake one path clib the variable ladder and send the other path through all the clause coponents.) 4.3 Strong NP-copleteness for nuerical probles An iportant difference between hard and easy cases applies to certain NP-coplete probles that involve nubers. For exaple, above we stated that the Partition proble is NP-coplete; thus, it is unlikely to be solvable by an efficient algorith. Clearly, however, we can solve the Partition proble by a siple dynaic prograing algorith, as follows. For an instance of Partition, let S be a set of positive integers {s 1,..., s }, and let s be the total, s = i=1 s i. Initialize a linear array B of Boolean values so that B[0] = true, and each other entry of B is false. For i = 1 to, and for t = s down to 0, if B[t] = true, then set B[t + s i ] to true. After the i th iteration, B[t] is true if and only if a subset of {s 1,..., s i } sus to t. The answer to this instance of Partition is yes if B[s /2] is ever set to true. The running tie of this algorith depends critically on the representation of S. If each integer in S is represented in binary, then the running tie is exponential in the total length of the representation. If each integer is represented in unary that is, each s i is represented by s i consecutive occurrences of the sae sybol then total length of the representation would be greater than s, and the running tie would be only a polynoial in the length. Put another way, if the agnitudes of the nubers involved are bounded by a polynoial in, then the above algorith runs in tie bounded by a polynoial in. Since the length of the encoding of such a low-agnitude instance is O( log ), the running tie is polynoial in the length of the input. The botto line is that these cases of the Partition proble are feasible to solve copletely. A proble is NP-coplete in the strong sense if there is a fixed polynoial p such that for each instance x of the proble, the value of the largest nuber encoded in x is at ost p( x ). That is, the integer values are polynoial in the length of the standard representation of 21

22 the proble. By definition, the 3SAT, Vertex Cover, Clique, Hailtonian Circuit, and 3 Diensional Matching probles defined in Section 4 are NP-coplete in the strong sense, but Partition and Knapsack are not. The Partition and Knapsack probles can be solved in polynoial tie if the integers in their stateents are bounded by a polynoial in n for instance, if nubers are written in unary rather than binary notation. The concept of strong NP-copleteness reinds us that the representation of inforation can have a ajor ipact on the coputational coplexity of a proble. 4.4 Coping with NP-hardness Step 6. Even if you cannot escape NP-hardness, the cases you need to solve ay still respond to sophisticated algorithic ethods, possibly needing high-powered hardware. There are two broad failies of direct attack that have been ade on hard probles. Exact solvers typically take exponential tie in the worst case, but provide feasible runs in certain concrete cases. Whenever they halt, they output a correct answer and soe exact solvers also output a proof that their answer is correct. Heuristic algoriths typically run in polynoial tie in all cases, and often ai to be correct only ost of the tie, or to find approxiate solutions (see Sections 6.1 and 6.2 in the next chapter). They are ore coon. Popular heuristic ethods include genetic algoriths, siulated annealing, neural networks, relaxation to linear prograing, and stochastic (Markov) process siulation. Experiental systes dedicated to certain NP-coplete probles have recently yielded soe interesting results an extensive survey on solvers for Traveling Salesperson is given by [Johnson and McGeogh, 1997]. There are two ways to attept to use this research. One is to find a proble close to yours for which people have produced solvers, and try to carry over their ethods and heuristics to the specific features of your proble. The other (uch ore speculative) is to construct a Karp reduction fro your proble to their proble, ask to run their progra or achine itself on the transfored instance, and then try to ap the answer obtained back to a solution of your proble. The hitches are (1) that the currently-known Karp reductions f tend to lose uch of the potentially helpful structure of the source instance x when they for f(x), and (2) that approxiate solutions for f(x) ay ap back to terribly sub-optial or even infeasible answers to x. (See, however, 22

23 the notion of L-reductions in Chapter 29, Section 6.2.) All of this indicates that there is uch scope for further research on iportant practical features of relationships between NP-coplete probles. See also Chapter 34 in this Handbook. 4.5 Beyond NP-Hardness If your proble belongs to NP and you cannot prove that it is NP-hard, it ay be an NPinterediate proble; i.e., neither in P nor NP-coplete. The theore of Ladner entioned in Section 3 shows that NP-interediate probles exist, assuing NP P. However, very few natural probles are currently counted as good candidates for such interediate status: factoring, discrete logarith, graph-isoorphis, and several probles relating to lattice bases for a very representative list. For the first two, see Chapter 38. The vast ajority of natural probles in NP have resolved theselves as being either in P or NP-coplete. Unless you uncover a specific connection to one of those four interediate probles, it is ore likely offhand that your proble siply needs ore work. The observed tendency of natural probles in NP to cluster as either being in P or NPcoplete, with little in between, reinforces the arguents ade early in this chapter that P is really different fro NP. Finally, if your proble sees not to be in NP, or alternatively if soe ore-stringent notion of feasibility than polynoial tie is at issue, then you ay desire to know whether your proble is coplete for soe other coplexity class. We now turn to this question. 5 Coplete Probles for NL, P, and PSPACE We first investigate the log-space analogue of the P vs. NP question, naely whether NL = L. We show that there are natural coputational probles that are NL-coplete. The question is, under which reducibility? Polynoial-tie reducibility is too blunt an instruent here, because NL is contained in P, and so all languages in NL are technically coplete for NL under both p and p T reductions. We need a reducibility that is fine enough to preserve the distinction between deterinistic and nondeterinistic log-space that we are attepting to establish and study. The siplest way is to replace the polynoial-tie bound in p reductions by a log-space bound. A language A 1 is log-space reducible to a language A 2, written A 1 log A 2, if A 1 is any- 23

24 one reducible to A 2 via a transforation function that is coputable by a deterinistic Turing achine in O(log n) space. There is a log-space analogue of p T reductions have the properties we desire: reducibility, but we do not use it here. Now we show that log Theore 5.1 (a) (Closure) If A 1 log A 2 and A 2 L, then A 1 L. (b) (Transitivity) If A 1 log A 2 and A 2 log A 3, then A 1 log A 3. (c) (Refineent of p reductions) If A 1 log A 2, then A 1 p A 2. The proof of (a) and (b) is soewhat tricky and rests on the fact that if two functions f and g fro strings to strings are coputable in log space, then so is the function h defined by h(x) = g(f(x)). The hitch is that a log space Turing achine M f coputing f can output the characters of f(x) serially but does not have space to store the. This becoes a proble whenever the achine M g coputing g, whose input is the output fro M f, requests the ith character of f(x), where i ay be less than the index j of the previous request. The solution is that since only space and not tie is constrained, we ay restart the coputation of M f (x) fro scratch upon the request, and let M g count the characters that M f outputs serially until it sees the i th one. Such a counter, and siilar ones tracking the oveents of M f s actual input head and M g s virtual input head, can be aintained in O(log n) space. Thus we need no physical output tape for M f or input tape for M g, and we obtain a tande achine that coputes g(f(x)) in log space. Part (c) is iediate by the function-class counterpart of the inclusion L P. The definition of NL-coplete is an instance of the general definition of copleteness at the beginning of Section 3: A language A 1 is NL-coplete ( under log NL and for every language A 2 NL, A 2 log again with log have A 1 log reductions is assued) if A 1 A 1. One defines P-coplete in a siilar anner reductions assued. Together with the observation that whenever A 1, A 2 L we A 2 (ignoring technicalities for A 1 or A 2 equal to or Σ ), we obtain a siilar state of affairs to what is known about NP-copleteness and the P vs. NP question: Theore 5.2 Let A be NL-coplete. Then the following stateents are equivalent: NL = L. 24

25 A L. Soe NL-coplete language belongs to L. All NL-coplete languages belong to L. All languages in L are NL-coplete. Substitute P for NL and the sae equivalence holds. Note that here we are applying copleteness to a class, naely P itself, whose definition does not involve nondeterinis. Cook s Theore provides two significant inferences: evidence of intractability, and a connection between coputation and Boolean logic. NL-copleteness is not to any coparable degree a notion of intractability, but does provide a fundaental link between coputations and graphs, via the following iportant proble. Graph Accessibility Proble (GAP) Instance: A directed graph G, and two nodes s and t of G. Question: Does G have a directed path fro node s to node t? Other naes are the s-t connectivity proble and the reachability proble. The link involves the concept of an instantaneous description (ID) of a Turing achine M. Let us suppose for siplicity that M has just two tapes: one read-only input tape that holds the input x, and one work tape with alphabet { 0, 1, B }, where B is the blank character. Let us also suppose that M never writes a B on its work tape. Then any step of a coputation of M on the fixed input x is describable by giving: the current state q of M, the contents y of the work tape, the position i of the input tape head, and the position j of the work tape head. Then the 4-tuple (q, y, i, j) is called an ID of M on input x. The restriction on writing B allows us to identify y with a string in { 0, 1 }. Without loss of generality, we always have 1 i n + 1, 25

26 where n = x, and if s(n) is a space bound on M, also 1 j s(n). An ID is also called a configuration. Now define G x to be the graph whose nodes are all possible IDs of M on input x, and whose directed edges coprise all pairs (I, J) such that M, if set up in configuration I, has a transition that takes it to configuration J in one step. If M is deterinistic, then every node in G x has at ost one outgoing arc. Nondeterinistic TMs, however, give rise to directed graphs G x of out-degree ore than one. Note that G x does depend on x, since the step(s) taken fro an ID (q, y, i, j) ay depend on bit x i of x. Theore 5.3 GAP is NL-coplete. Proof. GAP belongs to NL because guessing successive edges in a path fro 1 to R (when one exists) needs only O(log n) space to store the label of the current node, and to locate where on the input tape the adjacency inforation for the current node is stored. To show NL-hardness, let A NL. Then A is accepted by a nondeterinistic O(log n) space bounded Turing achine M. We prove that A log GAP. Construction: It is easy to odify M to have the properties supposed in the above discussion of IDs and still run in O(log n) space. We ay also code M so that any accepting coputation has a final phase that blanks out all used work tape cells, leaves the input head in cell n + 1, and halts in a special accepting state q a. This ensures that every accepting coputation (if any) ends in the unique ID I t = (q a, λ, n + 1, 1). Now given any x, define G x as above. Let node s be the unique starting ID I s = (q 0, λ, 1, 1), and let node t be I t. Note that the size of G x is polynoial if M runs in k log n space, then the size is O(n k+2 ). Coplexity: We show that the transforation f that takes a string x as input and produces the list of edges in G x as output can be coputed by a achine M f that uses O(log n) space. For each node I = (q, y, i, j) in turn, M f reads the i th sybol of x and then produces an edge (I, J) for each J such that M can ove fro I to J when reading that sybol. The only eory space that M f needs is the space to step through each I in turn, and count up to the i th input position, and produce each J. O(log n) space is sufficient for all of this. 26