Computability and Complexity

Transcription

1 Computability and Complexity Complexity and Turing Machines. P vs NP Problem CAS 705 Ryszard Janicki Department of Computing and Software McMaster University Hamilton, Ontario, Canada janicki@mcmaster.ca Ryszard Janicki Computability and Complexity 1 / 101

2 Big O Notation N = {0, 1, 2,...} Reals + - non-negative real numbers Denition Let f, g : N Reals +. We say: f (n) = O(g(n)) c, n 0 > 0. n n 0. f (n) cg(n) f (n) = O(g(n)): g(n) is an assymptotic upper bound of f (n). g(n) is an upper approximation of f (n). Ryszard Janicki Computability and Complexity 2 / 101

3 Typical O(...): (notation log n = log 2 n) O(log n) O(log(log(n))... O(n) O(n log n) O(n 2 ) O(n k ) O(2 n ) O(k n ) Classication: O(log n) O(n) O(n log n) desired O(n k ) : acceptable O(2 n ) : UNACCEPTABLE Ryszard Janicki Computability and Complexity 3 / 101

4 Fact For every k 0 and every α > 1, there exists n 0 such that for every n > n 0 : n k < α n Another classication: O(n k ) : polynomial, i.e. GOOD O(α n ) : non-polynomial, i.e. BAD Ryszard Janicki Computability and Complexity 4 / 101

5 Helpful Results Lemma 1 O(f (n)) + O(g(n)) = O(f (n) + g(n)) = O(max(f (n), g(n)) 2 O(f (n))o(g(n)) = O(f (n)g(n)) for each polynomial f (n) = a n x n + a n 1 x n a 1 x + a 0, where a n 0, we have: O(f (n)) = O(x n ) O(2 n + n ) = O(2 n ) O(n ) = O(2 n ) O(2 n ) O(n k ) for any k Since log b n = 1 log b log n, for any b we have O(log b n) = O(log n) Ryszard Janicki Computability and Complexity 5 / 101

6 Time Complexity of Turing Machines Let M be a deterministic Turing machine that halts on all inputs. Denition The running time or time complexity of M is the function f : N N, where f (n) is the maximum number of steps that M uses on any input of length n. Important. Turing machines are very inecient but their relationship with RAM programs (i.e. normal algorithms) is polynomial. For Turing machines we are always interested in if time complexity is polynomial or not, not in ecient practical complexity. Ryszard Janicki Computability and Complexity 6 / 101

7 Example L = {a k b k k 0} can be recognized by simple Turing machine that is looking for a and b and then deleting them, in O(n 2 ), or in O(n log n) using more sophisticated algorithm, but it can be proven that it cannot be done in O(n). Example L = {a k b k k 0} can be recognized in O(n) by 2 tapes Turing machine. Ryszard Janicki Computability and Complexity 7 / 101

8 Time Complexity Classes Denition Let t : N Reals +. The time complexity class TIME(t(n)) is dened as follows; TIME(t(n)) = {L L is a language decidable by an O(t(n)) Turing Machine } Example {a k b k k 0} TIME(n) Ryszard Janicki Computability and Complexity 8 / 101

9 Single vs Multitape Turing Machines Theorem Let t : N Reals +, t(n) n. Then every t(n) time multitape Turing machine has equivalent O(t 2 (n)) time single tape Turing machine. Proof. (idea). Simulating each step of the multitape machine uses at most O(t(n)) steps on the single tape machine. Hence, the total time is O(t 2 (n)). Ryszard Janicki Computability and Complexity 9 / 101

10 Non-deterministic Turing Machines Denition Let M be a non-deterministic Turing machine that is a decider. The running time or time complexity of M is the function f : N N, where f (n) is the maximum number of steps that M uses on any branch of its computation on any input of length n. Non-determinism does make sense only for specication purposes. Ryszard Janicki Computability and Complexity 10 / 101

11 Theorem Let t : N Reals +, t(n) n. Then every t(n) time non-deterministic single tape Turing machine has equivalent O(2 t(n) ) time deterministic single tape Turing machine. Proof. (idea). The simulation that uses Breadth First Trees takes O(2 t(n) ) time in the worst case. It can be shown that there are Turing machines where it cannot be done better. Multitape Turing Machines O(t2 (n)) Single Tape Turing Machines. Non-deterministic Turing Machines O(2t(n) ) Deterministic Turing Machines. Ryszard Janicki Computability and Complexity 11 / 101

12 The Class P (Polynomial) Denition The class P, of all polynomial computations, is dened as follows: P = TIME(n k ). k=1 The class P is important because: 1 P is an invariant for all models of computations that are polynomially equivalent to the deterministic single tape Turing machines. 2 P roughly corresponds to the calss of problems that are realistically solvable on computers. (1) = P is mathematically robust, it is independent of the model (2) = P is relevant from the practical viewpoint. Denition (Alternative) P is the class of problems that have algorithmic solutions in polynomial time. More specically, they are problems that can be solved in O(n k ) for some constant k, where n is the size of the input to the problem. Ryszard Janicki Computability and Complexity 12 / 101

13 The class P: Why important and `good'? While time Θ(n 100 ) can reasonable be considered as intractable, there are few practical problems that require time of such degree polynomial. The polynomial time computable problems encountered in practice typically require much less time. Experience has shown that once a polynomial time algorithm for a problem is algorithm is discovered, more ecient algorithms often follow. Relationship between Turing Machines (abstract model of computations) and Random Access Machines (abstract model of executable programs) is polynomial. Relationship between Random Access Machines and programs in high level programming languages as Java, Haskell, etc. is linear. Relationships between all known models of computations (Turing Machines, Post Systems, Recursive Functions, etc.) are polynomial. Ryszard Janicki Computability and Complexity 13 / 101

14 Some problems PATH - Is there a path from vertex s to vertex t in directed graph G? PATH = { G, s, t G is directed graph and has a path from s to t } MAX - Find a maximum element is a sequence S. MAX = { S, x x is a maximal element of S } SORT min - Sort a sequence of numbers from the smallest to the biggest element. SORT min = { S, S sort S sort is S sorted increasingly } Ryszard Janicki Computability and Complexity 14 / 101

15 Theorem PATH, MAX, SORT min P. Proof. PATH has O(n) solution (n - number of nodes). MAX has O(n) solution (n - number of elements). SORT min has O(n log n) solution. Theorem The problem: `for a given context-free grammar G, does a string x belongs to L(G)', is in P. Proof. (idea). A smart version of the algorithm from page 45 of Lecture Notes 5 has complexity O(n 3 ). Ryszard Janicki Computability and Complexity 15 / 101

16 The Class NP: Intuitions In many texts TIME(t(n)) is written DTIME(t(n)), from deterministic Turing machine. Then we have P = DTIME(t(n i )). i=1 For many problems polynomial algorithms could not be found! What can be done about it? Is it our fault or thus the reality? Ryszard Janicki Computability and Complexity 16 / 101

17 A Hamiltonian path in a directed graph G is a path that goes through each node once. Problem: Does a directed graph contains a Hamiltonian path connecting two specied nodes? Exponential algorithm is easy, check all cases. Polynomial algorithm is not found. However, for a given path we can verify (in O(n) time) if it is Hamiltonian! Ryszard Janicki Computability and Complexity 17 / 101

18 Does G have a Hamiltonian path from s to t? - is polynomially veriable. Does G have not a Hamiltonian path from s to t? - is not polynomially veriable. Denition A verier is an algorithm that can verify if a given instance is a solution or not. Denition A verier for a language L is a deterministic Turing machine V, such that L = {w V accepts w, c for some string c } We measure the time of a verier only in terms of the length of w, so a polynomial verier runs in O( w k ), some k. Ryszard Janicki Computability and Complexity 18 / 101

19 Denition A language L (a problem P) is polynomially veriable, if it has a polynomial time verier. Theorem A language L has a polynomial verier if L is decided by a non-deterministic Turing machine in polynomial time. Proof. (sketch). The proof explores the concept of non-determinism. A Turing machine T selects non-deterministically a proper instance and runs verier on it. A verier is a modied part of a Turing machine that decides it. It is just a part after selection of correct instance. Ryszard Janicki Computability and Complexity 19 / 101

20 Equivalent Denitions of NP Denition The time complexity class NTIME(t(n)) is dened as follows; NTIME(t(n)) = {L L is a language decided in O(t(n)) by non-deterministic Turing Machine } Denition (NP) NP is the class of all languages that are decided by non-deterministic Turing machines in polynomial time. Denition (NP) NP is the class of all languages that have polynomial veriers. Denition (NP) NP = NTIME(n k ) k=1 Ryszard Janicki Computability and Complexity 20 / 101

21 Equivalent Denitions of NP Denition (with veriers) The class NP (from Nondeterministic Polynomial) consists of those problems that are veriable in polynomial time. More specically, they are problems that can be veried in O(n k ) for some constant k, where n is the size of the input to the problem. Hamiltonian Path is such a problem! Denition (with nondeterministic algorithms) The class NP (from Nondeterministic Polynomial) consists of those problems that are solvable in polynomial time by nondeterministic algorithms. More specically, they are problems that can be solved in O(n k ) for some constant k. where n is the size of the input to the problem, by nondeterministic algorithms. The idea of Nondeterministic Algorithms is a simple consequence of angelic semantics. Ryszard Janicki Computability and Complexity 21 / 101

22 Veriers vs Nondeterministic Algorithms Nondeterministic algorithm (or nondeterministic Turing machine), if a solution exists, chooses the proper path to follow. Angelic semantics allows it. Every nondeterministic algorithm can be simulated by deterministic one (deterministic Turing machine), we just have to simulate all choices in an appropriate manner. If a solution exists, we will nd it, but it may take at least exponential number of steps (all cases). Proposition Polynomial verier Polynomial Nondeterministic Algorithm. Proof. ( ) Assume we have a polynomial verier. If a solution does exist, we chose a proper choice and the apply verier. ( ) A part of the algorithm that has been used after proper choice is a verier. Ryszard Janicki Computability and Complexity 22 / 101

23 Clearly P NP Open question: P? = NP (Turing medal + one million US $ for a solution) Ryszard Janicki Computability and Complexity 23 / 101

24 P vs NP and NP-completeness Clearly P NP, since every (deterministic) algorithm is also nondeterministic algorithm. The problem if P = NP or not, is an open million US dollars question (one of millennium problems). Informally, a problem is NP-complete, if it is in NP and it is hard as any problem in NP. If P NP, NP-complete problems do not have polynomial solutions. For every problem in NP we have a (deterministic) algorithm, just apply verier to all cases, but its complexity is at least exponential. We will show that if any NP-complete problem has a polynomial solution, then P = NP (Cook-Levin Theorem) In practice, for NP-complete problems we are looking for approximate or good on average algorithms. Ryszard Janicki Computability and Complexity 24 / 101

25 Hamiltonian problem is n NP Other problems that are in NP and maybe nit in P. A clique in an undirected graph is a subgraph where every two nodes are connected by an edge. A k-clique is a clique that contains k-nodes. The 5-clique is given below. The clique problem is to determine whether a graph contains a clique of a specied size. CLIQUE = { G, k G is undirected graph with a k-clique } CLIQUE : Does a graph G has k-clique? Ryszard Janicki Computability and Complexity 25 / 101

26 Theorem CLIQUE NP Proofs. (ideas). Proof 1. Let c be a subgraph of G. To verify if c is a k-clique, we have to check if it has k nodes in G and if all nodes are connected. this can be done in O(n 2 ) time, so this verier works in polynomial time. Proof 2. Non-deterministic Turing machine (procedure etc.) selects a (proper) subset of k nodes of G. The it is tested whether G contains all edges connecting nodes in c. Answer is either yes or no, and it is done in O(n 2 ), so polynomial time. Ryszard Janicki Computability and Complexity 26 / 101

27 P vs PN P = the class of languages/problems that can be decided quickly (i.e. in polynomial time). NP = the class of languages/problems that can be veried quickly (i.e. in polynomial time). Intuition: P NP, but nobody can prove it so far. What if we nd a problem A, which is in NP, and we prove that A P P = NP? We do not know if A P, but if it does then P = NP, if it does not P NP. Any such problem is called NP-complete, but does such a problem exist? Ryszard Janicki Computability and Complexity 27 / 101

28 Reducibility Once Again Denition f : Σ Σ is a polynomial time computable function if some Turing machine computes it in polynomial time. Denition f : Σ Σ is a polynomial time computable function if there is an algorithm that computes it in O(n k ) time, some k 1. Denition A Language A is polynomial time reducible to B, written A P B, if there exists a polynomial time computable function f : Σ Σ, such that for each w Σ : Denition w A f (w) B. A problem A is polynomial time reducible to B, written A P B, if there exists a polynomial time transformation of A into B. Ryszard Janicki Computability and Complexity 28 / 101

29 Theorem If A P B and B P, then A P. Proof. (sketch). First transform and instance of A into B. It takes O(n k ). Then solve this instance of B. It takes O(n m ). Polynomial in both cases and O(n k+m ) altogether. Ryszard Janicki Computability and Complexity 29 / 101

30 Polynomial-time reductions Polynomial-time Desiderata. reductions Suppose we could solve X in polynomial-time. What else could we solve in polynomial time? Desiderata'. Suppose we could solve X in polynomial-time. Denition (Reduction) What else could we solve in polynomial time? Let X and Y be two problems and assume that we already have a Reduction. polynomial Problem time X polynomial-time solution to Y (Cook). Suppose reduces that to problem we havey aif procedure arbitrary that instances transforms of problem any instance X can be αsolved of Xusing: into some instance of β with Polynomial the following number characteristics: of standard computational steps, plus Polynomial 1 Thenumber transformation of calls to oracle takes polynomial that solves problem time. Y. 2 The answers are computational the same. model that supplemented is, theby answer special piece for α is yes if of hardware that solves instances of Y in a single step and only if the answer to β is also yes. We call such a procedure a polynomial-time reduction algorithm. instance I (of X) Algorithm for Y solution S to I Algorithm for X Ryszard Janicki Computability and Complexity 30 / 101

31 Polynomial-time reductions Notation. X P Y (X is reduced to Y ). Note. We pay time for transformation. Caveat. Don't mistake X p Y with Y P X. Ryszard Janicki Computability and Complexity 31 / 101

32 Polynomial-time reductions Design algorithms. If X P Y and Y can be solved in polynomial time, then X can be solved in polynomial time. Establish intractability. If X P Y and X cannot be solved in polynomial time, then Y cannot be solved in polynomial time. Establish equivalence. If both X P Y and Y P X, we use notation X P Y. In this case, X can be solved in polynomial time iff Y can be. Bottom line. Reductions classify problems according to relative difficulty. Ryszard Janicki Computability and Complexity 32 / 101

33 Polynomial-time reductions We will now consider the following well known problems: Independent set Vertex cover Set cover 3-satisability We will show that: 3-satisability P Independent set P Vertex cover P Set cover Ryszard Janicki Computability and Complexity 33 / 101

34 Polynomial-time reductions and intractability NP-completeness is about showing how hard a problem is rather than how easy it is. We will use polynomial time reductions in the opposite way to show that a problem is NP-complete. If X P Y and X cannot be solved in polynomial time, then Y cannot be solved in polynomial time. For NP-completeness, we cannot assume that there is absolutely no polynomial time algorithm for problem X. The proof methodology is similar however, in that we prove that problem Y is NP-complete on the assumption that problem X is also NP-complete. Hence, we will analyze some non trivial polynomial-time reductions in details. Ryszard Janicki Computability and Complexity 34 / 101

35 Independent set Independent Set How to find closest pair with one point in each side? INDEPENDENT-SET. Given a graph Def. Let G = s(v, i be E) the and point an integer the 2 k, δ-strip, is there with a subset the i th smalles of vertices S V such that S k, and for each edge at most one of its endpoints is in S? Claim. If i j 12, then the distance between s i and s j is at least δ. Ex. Is there an independent set of size 6? Ex. Is there an independent Pf. set of size 7? No two points lie in same ½ δ-by-½ δ box. Two points at least 2 rows apart have distance 2 (½ δ). 2 rows Fact. Claim remains true if we replace 12 with 7. i independent set of size 6 11 Ryszard Janicki Computability and Complexity 35 / 101

36 Vertex cover Vertex Cover How to find closest pair with one point in each side? VERTEX-COVER. Given a graph Def. G = Let (V, se) i be and the an point integer the k, is 2 δ-strip, there a with subset the of i th smalles vertices S V such that S k, and for each edge, at least one of its endpoints is in S? Ex. Is there a vertex cover of size 4? Ex. Is there a vertex cover Pf. of size 3? Claim. If i j 12, then the distance between s i and s j is at least δ. No two points lie in same ½ δ-by-½ δ box. Two points at least 2 rows apart have distance 2 (½ δ). 2 rows Fact. Claim remains true if we replace 12 with 7. i independent set of size 6 vertex cover of size 4 12 Ryszard Janicki Computability and Complexity 36 / 101

37 Vertex cover and independent How to set find reduce closest to pair one with another one point in each side? Vertex cover and independent set reduce to one another Theorem. VERTEX-COVER P INDEPENDENT-SET. Def. Let s i be the point in the 2 δ-strip, with the i th smalles Pf. We show S is an independent set of size k iff V S is a vertex cover of size n k. Claim. If i j 12, then the distance between s i and s j is at least δ. Pf. No two points lie in same ½ δ-by-½ δ box. Two points at least 2 rows apart have distance 2 (½ δ). 2 rows Fact. Claim remains true if we replace 12 with 7. i independent set of size 6 vertex cover of size 4 13 Ryszard Janicki Computability and Complexity 37 / 101

38 Vertex cover and independent set reduce to one another Vertex cover and independent set reduce to one another Theorem. VERTEX-COVER P INDEPENDENT-SET. Pf. We show S is an independent set of size k iff V S is a vertex cover of size n k. Let S be any independent set of size k. V S is of size n k. Consider an arbitrary edge (u, v). S independent either u S or v S (or both) either u V S or v V S (or both). Thus, V S covers (u, v). Ryszard Janicki Computability and Complexity 38 / 101

39 Vertex cover and independent set reduce to one another Vertex cover and independent set reduce to one another Theorem. VERTEX-COVER P INDEPENDENT-SET. Pf. We show S is an independent set of size k iff V S is a vertex cover of size n k. Let V S be any vertex cover of size n k. S is of size k. Consider two nodes u S and v S. Observe that (u, v) E since V S is a vertex cover. Thus, no two nodes in S are joined by an edge S independent set. Ryszard Janicki Computability and Complexity 39 / 101

40 Set cover Set Cover SET-COVER. Given a set U of elements, a collection S 1, S 2,, S m of subsets of U, and an integer k, does there exist a collection of k of these sets whose union is equal to U? Sample application. m available pieces of software. Set U of n capabilities that we would like our system to have. The i th piece of software provides the set S i U of capabilities. Goal: achieve all n capabilities using fewest pieces of software. U = { 1, 2, 3, 4, 5, 6, 7 } S 1 = { 3, 7 } S 4 = { 2, 4 } S 2 = { 3, 4, 5, 6 } S 5 = { 5 } S 3 = { 1 } S 6 = { 1, 2, 6, 7 } k = 2 a set cover instance 1 Ryszard Janicki Computability and Complexity 40 / 101

41 Vertex cover reduces to set cover Vertex Cover Reduces to Set Cover Theorem. VERTEX-COVER P SET-COVER. Pf. Given a VERTEX-COVER instance G = (V, E), we construct a SET-COVER instance (U, S) that has a set cover of size k iff G has a vertex cover of size k. Construction. Universe U = E. Include one set for each node v V : S v = {e E : e incident to v }. a b e 7 e 2 e3 e 4 U = { 1, 2, 3, 4, 5, 6, 7 } S a = { 3, 7 } S b = { 2, 4 } f e 6 c S c = { 3, 4, 5, 6 } S d = { 5 } k = 2 e 1 e 5 S e = { 1 } S f = { 1, 2, 6, 7 } e d vertex cover instance (k = 2) set cover instance (k = 2) 17 Ryszard Janicki Computability and Complexity 41 / 101

42 Vertex cover reduces to set cover Vertex Cover Reduces to Set Cover Lemma. G = (V, E) contains a vertex cover of size k iff (U, S) contains a set cover of size k. Pf. Let X V be a vertex cover of size k in G. Then Y = { Sv : v X } is a set cover of size k. a b f e 7 e 2 e 6 e3 e 4 c U = { 1, 2, 3, 4, 5, 6, 7 } S a = { 3, 7 } S b = { 2, 4 } S c = { 3, 4, 5, 6 } S d = { 5 } k = 2 e 1 e d e 5 S e = { 1 } S f = { 1, 2, 6, 7 } vertex cover instance (k = 2) set cover instance (k = 2) 18 Ryszard Janicki Computability and Complexity 42 / 101

43 Vertex cover reduces to set cover Vertex Cover Reduces to Set Cover Lemma. G = (V, E) contains a vertex cover of size k iff (U, S) contains a set cover of size k. Pf. Let Y S be a set cover of size k in (U, S). Then X = { v : Sv Y } is a vertex cover of size k in G. a b f e 7 e 2 e 6 e3 e 4 c U = { 1, 2, 3, 4, 5, 6, 7 } S a = { 3, 7 } S b = { 2, 4 } S c = { 3, 4, 5, 6 } S d = { 5 } k = 2 e 1 e d e 5 S e = { 1 } S f = { 1, 2, 6, 7 } vertex cover instance (k = 2) set cover instance (k = 2) 19 Ryszard Janicki Computability and Complexity 43 / 101

44 Satisability Satisfiability Literal. A boolean variable or its negation. x i or x i Clause. A disjunction of literals. C j = x 1 x 2 x 3 Conjunctive normal form. A propositional formula Φ that is the conjunction of clauses. Φ = C 1 C 2 C 3 C 4 SAT. Given CNF formula Φ, does it have a satisfying truth assignment? 3-SAT. SAT where each clause contains exactly 3 literals (and each literal corresponds to a different variable). Φ = ( x 1 x 2 x 3 ) ( x 1 x 2 x 3 ) ( x 1 x 2 x 4 ) yes instance: x 1 = true, x 2 = true, x 3 = false, x 4 = false Key application. Electronic design automation (EDA). Ryszard Janicki Computability and Complexity 44 / 101

45 3-satisfiability reduces to independent set 3-satisability reduces to independent set Theorem. 3-SAT P INDEPENDENT-SET. Pf. Given an instance Φ of 3-SAT, we construct an instance (G, k) of INDEPENDENT-SET that has an independent set of size k iff Φ is satisfiable. Construction. G contains 3 nodes for each clause, one for each literal. Connect 3 literals in a clause in a triangle. Connect literal to each of its negations. G k = 3 Φ = ( x 1 x 2 x 3 ) ( x 1 x 2 x 3 ) ( x 1 x 2 x 4 ) Ryszard Janicki Computability and Complexity 45 / 101

46 3-satisfiability reduces to independent set 3-satisability reduces to independent set Lemma. G contains independent set of size k = Φ iff Φ is satisfiable. Pf. Let S be independent set of size k. S must contain exactly one node in each triangle. Set these literals to true (and remaining variables consistently). Truth assignment is consistent and all clauses are satisfied. Pf Given satisfying assignment, select one true literal from each triangle. This is an independent set of size k. G k = 3 Φ = ( x 1 x 2 x 3 ) ( x 1 x 2 x 3 ) ( x 1 x 2 x 4 ) Ryszard Janicki Computability and Complexity 46 / 101

47 Review Review Basic reduction strategies. Simple equivalence: INDEPENDENT-SET P VERTEX-COVER. Special case to general case: VERTEX-COVER P SET-COVER. Encoding with gadgets: 3-SAT P INDEPENDENT-SET. Transitivity. If X P Y and Y P Z, then X P Z. Pf idea. Compose the two algorithms. Ex. 3-SAT P INDEPENDENT-SET P VERTEX-COVER P SET-COVER. Ryszard Janicki Computability and Complexity 47 / 101

48 P, NP and EXP P. Decision problems for which there is a poly-time algorithm. NP. Decision problems for which there is a poly-time certier (nondeterministic poly-time algorithm). EXP. Decision problems for which there is an exponential-time algorithm. Claim P NP EXP Proof. Clearly P NP, since every (deterministic) algorithm is also nondeterministic algorithm. Since by considering all possible choices we can simulate every nondeterministic algorithm by deterministic one, but the complexity is exponential so NP EXP. The property NP EXP follows from the observation that not every problem from EXP has a polynomial verier. For example Does G have not a Hamiltonian path from s to t? clearly is in EXP, but it does not have a polynomial verier. Ryszard Janicki Computability and Complexity 48 / 101

49 P vs NP again Does P = NP? [Cook 1971, Edmonds, Levin, Yablonski, Gödel] Is the decision problem as easy as the certification problem? EXP NP EXP P P = NP If P NP If P = NP If yes. Efficient algorithms for 3-SAT, TSP, 3-COLOR, FACTOR, If no. No efficient algorithms possible for 3-SAT, TSP, 3-COLOR, Ryszard Janicki Computability and Complexity 49 / 101

50 NP-completeness Denition A problem Y is NP-complete if it satises the following two conditions: 1 Y NP 2 every X NP is polynomially reducible to Y, i.e. X P Y. Let NPC denote the class of all NP-complete problems. Theorem Suppose Y is NP-complete. Then Y P P = NP. Proof. ( ) If P = NP, then Y P because Y NP. ( ) Suppose Y P. Consider any problem X NP. Since X P Y, we have X P. This implies NP P. We already know P NP. Thus P = NP. Ryszard Janicki Computability and Complexity 50 / 101

51 Equivalent Denitions of NP-Completeness Denition (1) A language/problem B is NP-complete if it satises the following two conditions: 1 B NP 2 every A NP is polynomially reducible to B. Denition (2) A language/problem B is NP-complete i: B P P = NP Theorem Denition 1 Denition 2 Ryszard Janicki Computability and Complexity 51 / 101

52 NP-Completeness: Basic Tool Theorem If X is NP-complete, X NP, and X P Y, then Y is NP-complete. Proof. Let A be any problem from NP, i.e. A NP. Since X is NP-complete, then A P X. Hence we have A P X P Y. This is true for any A NP, so by (2) of the denition of NP-completeness, Y is also NP-complete. Fundamental question. Do there exist natural NP-complete problems? If not, NPC is empty! Ryszard Janicki Computability and Complexity 52 / 101

53 If P (NP-complete) = then P = NP. Does such `red element' exist? Ryszard Janicki Computability and Complexity 53 / 101

54 SAT Problem: Idea An example of a Boolean formula: Φ = (x y) (x z), where x means x, so x = 0 x = 1 and x = 1 x = 0. Denition A Boolean formula Φ is satisable if so some assignment of 0's and 1's to the variables makes the formula to eveluate to 1. (x y) (x z) = 1 if x = 0, y = 1, z = 0. This formula is satisable. (x y) (x z) is never 1, always 0. This formula is not satisable. Ryszard Janicki Computability and Complexity 54 / 101

55 SAT Problem and Cook-Levin Theorem Denition The satisability problem (SAT) is to test whether Boolean formula is satisable. Theorem (Cook-Levin) SAT is NP-complete. Ryszard Janicki Computability and Complexity 55 / 101

56 Theorem (Cook-Levin) SAT is NP-complete. Proof. (idea). We will use Denition (1), i.e. Denition (1) A language/problem B is NP-complete if it satises the following two conditions: 1 B NP 2 every A NP is polynomially reducible to B. The proof consists of two parts. 1 SAT NP. A nondeterministic polynomial time machine can guess an assignment to agiven formula Φ and accept if the assignment satises Φ. 2 The hard part of the proof is showing that any language in NP is polynomial time reducible to SAT. Ryszard Janicki Computability and Complexity 56 / 101

57 Proof. (continuation). We construct a polynomial time reduction for each language L in NP to SAT. The reduction for L takes a string w and produces a Boolean formula Φ that simulates the nondeterministic polynomial (NP) Turing machine for L on input w. If the machine accepts, φ has a satisfying assignment that correspond to the accepting computation. If the machine doesn't accept, no assignment satises Φ. Therefore, w L if and only if Φ is satisable. The proof idea explores the fact that formula is a propositional calculus formula, so it can express w L for any L! Ryszard Janicki Computability and Complexity 57 / 101

58 Why NP-completeness? For most of the problems, it is usually easy to show that B NP. In many cases we cannot nd any polynomial solution, but we are unable to prove that B / P either. Proving that B is NP-complete is in most cases easier (or just possible) than B / P. If B is NP-complete, it is practically considered as non-polynomial. Ecient solution must take particular properties into account. Ryszard Janicki Computability and Complexity 58 / 101

59 Other NP-complete Problems Fact SAT problem is the only problem that has been proven from the denition. All other problems have been proven NP-complete by using polynomial reduction. If A is NP-complete and A P B, i.e. A is polynomially reducible to B, then B is NP-complete. Since so far we only have SAT, the second problem X 2 must be a reduction of SAT, i.e SAT P X 2. Ryszard Janicki Computability and Complexity 59 / 101

60 Conjunctive Normal Form (CNF) Conjunctive Normal Form (CNF): (x y) (y x z) (z x) CNF. ((x y) z) x / CNF Theorem Every Boolean formula Ψ can be polynomially transformed into Φ in CNF such that Ψ Φ. Theorem The satisability problem for Boolean expressions in CNF is NP-complete. Proof. From Theorem above! Ryszard Janicki Computability and Complexity 60 / 101

61 3-Conjunctive Normal Form Theorem 3-Conjunctive Normal Form (3-CNF): (t x y) y x z) 3-CNF. (x y) (y x z t) / 3-CNF. SAT for 3-CNF is NP-complete. Proof. We will polynomially reduce SAT for CNF to SAT for 3-CNF. Let k 0. Replace in CNF each (x 1... x k ) by (x 1 x 2 y 1 ) (x 3 y 1 y 2 ) (x 4 y 2 y 3 )... (x k 2 y k 4 y k 3 ) (x k 1 x k y k 3 ), where y 1,..., y k 3 are new Boolean variables. For example: x 1 x 2 x 3 x 4 (x 1 x 2 y 1 ) (x 3 x 4 y 4 ). Transformation is polynomial. x 1... x k = 1 x i.x i = 1 If x i = 1 then y 1 = y 2 =... = y i 2 = 1, y i 1 = y i =... = y k 3 = 0 guarantees that the replacement has the value 1. Hence SAT for CNF is reduced to SAT for 3-CNF. Ryszard Janicki Computability and Complexity 61 / 101

62 NP-completeness: basic tool Theorem If X is NP-complete, Y NP, and X P Y, then Y is NP-complete. Procedure (Showing NP-completeness of Y ) 1 First show that Y NP. This is usually done by showing that an instance of Y has a polynomial verier. 2 Find a problem X that has been proven to be NP-complete. For example, 3-SAT, VECTOR-COVER, HAMILTON-CYCLE, etc. If Y is a graph problem, try rst X that is also a graph problem. 3 Show X p Y, i.e. X can be polynomially reduced to Y. While the fact that a transformation o X into an instance of Y is polynomial is often almost obvious, always mention it and explain. Ryszard Janicki Computability and Complexity 62 / 101

63 NP-completeness: frequent errors Procedure (Showing NP-completeness of Y ) 1 First show that Y NP. 2 Find a problem X that has been proven to be NP-complete. 3 Show X p Y, i.e. X can be polynomially reduced to Y. Frequent Errors. It is NOT shown that Y NP. Usually can be xed. It is attempted to show that Y P X instead of X P Y. This is the most serious error. It is not argued that transformation of X into an instance of Y is polynomial. Usually can be xed. Ryszard Janicki Computability and Complexity 63 / 101

64 Other NP-complete problems Since: 3-satisability P Independent set P Vertex cover P Set cover and 3-satisability is NP-complete, then Independent set, Vertex cover and Set cover are NP-complete as well! Ryszard Janicki Computability and Complexity 64 / 101

65 Theorem (Clique Problem) Clique problem is NP-complete. Proof. (idea). We will polynomially reduce SAT for 3-CNF to clique problem. Idea of transformation: Φ 1 Φ 2 Φ 3 {}}{{}}{{}}{ Φ = (x 1 x 1 x 2 ) (x 1 x 2 x 2 ) (x 1 x 2 x 2 ) Φ = 1 for x 1 = 0, x 2 = 1, and Φ = Φ 1 Φ 2 Φ 3. No edge between nodes of Φ i No edge between x and x Transformation is polynomial. Φ = Φ 1... Φ k is satisable the graph contains k-clique. Ryszard Janicki Computability and Complexity 65 / 101

66 Search problems Decision problems vs search problems Decision problem. Does there exist a vertex cover of size k? Search problem. Find a vertex cover of size k. Ex. To find a vertex cover of size k : Determine if there exists a vertex cover of size k. Find a vertex v such that G { v } has a vertex cover of size k 1. (any vertex in any vertex cover of size k will have this property) Include v in the vertex cover. Recursively find a vertex cover of size k 1 in G { v }. delete v and all incident edges Bottom line. VERTEX-COVER P FIND-VERTEX-COVER. Ryszard Janicki Computability and Complexity 66 / 101

67 Optimization problems Decision problems, search and optimization problems Decision problem. Does there exist a vertex cover of size k? Search problem. Find a vertex cover of size k. Optimization problem. Find a vertex cover of minimum size. Ex. To find vertex cover of minimum size: (Binary) search for size k* of min vertex cover. Solve corresponding search problem. Bottom line. VERTEX-COVER P FIND-VERTEX-COVER P OPTIMAL-VERTEX-COVER. Ryszard Janicki Computability and Complexity 67 / 101

68 Hamilton cycle NP-complete problems: Hamilton cycle HAM-CYCLE. Given an undirected graph G = (V, E), does there exist a simple cycle Γ that contains every node in V? yes Ryszard Janicki Computability and Complexity 68 / 101

69 NP-complete problems: directed Hamilton cycle Directed hamilton cycle reduces to hamilton cycle DIR-HAM-CYCLE: Given a digraph G = (V, E), does there exist a simple directed cycle Γ that contains every node in V? Theorem. DIR-HAM-CYCLE P HAM-CYCLE. Pf. Given a digraph G = (V, E), construct a graph G' with 3n nodes. a b c v d e a out b out v in v v out d in e in G c out G' Ryszard Janicki Computability and Complexity 69 /

70 NP-complete Directed hamilton problems: cycle reduces directed to hamilton Hamilton cycle cycle Lemma. G has a directed Hamilton cycle iff G' has a Hamilton cycle. Pf. Suppose G has a directed Hamilton cycle Γ. Then G' has an undirected Hamilton cycle (same order). Pf. Suppose G' has an undirected Hamilton cycle Γ'. Γ' must visit nodes in G' using one of following two orders:, B, G, R, B, G, R, B, G, R, B,, B, R, G, B, R, G, B, R, G, B, Blue nodes in Γ' make up directed Hamilton cycle Γ in G, or reverse of one. B = blue, G = green, R = red Ryszard Janicki Computability and Complexity 70 / 101

71 Directed Hamilton Cycle and Hamilton Cycle are NP-Complete Theorem 3-SAT P DIR-HAM-CYCLE. Proof. Idea: Given an instance Φ of 3-SAT, we construct an instance of DIR-HAM-CYCLE that has a Hamilton cycle i Φ is satisable. See Kleinberg-Tardos for details. Corollary 3-SAT P DIR-HAM-CYCLE P HAM-CYCLE Both directed Hamilton cycle and Hamilton cycle are NP-complete. Ryszard Janicki Computability and Complexity 71 / 101

72 NPC-problems: Longest path LONGEST-PATH. Given a directed graph G = (V, E), does there exists a simple path consisting of at least k edges? Theorem HAM-CYCLE P LONGEST-PATH. Corollary 3-SAT P DIR-HAM-CYCLE P HAM-CYCLE P LONGEST-PATH. LONGEST-PATH is NP-complete. Ryszard Janicki Computability and Complexity 72 / 101

73 NPC-problems: Traveling Salesperson Problem (TSP) Hamilton cycle reduces to traveling salesperson problem TSP. Given a set of n cities and a pairwise distance function TSP. d(u, Given v), isa there set of an tour cities of and length a pairwise D? distance function d(u, v), is there a tour of length D? HAM-CYCLE. Given an undirected graph G = (V, E), does there exist a simple cycle Γ that contains every node in V? HAM-CYCLE. Given an undirected graph G = (V, E), does there exist a simple cycle Theorem Γ that contains every node in V? HAM-CYCLE P TSP. Theorem. HAM-CYCLE P TSP. Pf. Proof. Given instance G = (V, E) of HAM-CYCLE, create n cities with distance function 1 if (u, v) E d(u, v) = 2 if (u, v) E TSP instance has tour of length n iff G has a Hamilton cycle. Note that we have triangle inequality d(u, w) d(u, v) + d(v, w). Remark. TSP instance satisfies triangle inequality: d(u, w) d(u, v) + d(v, w). Ryszard Janicki Computability and Complexity 73 / 101

74 3-colorability NPC-problems: 3-colorability 3-COLOR. Given an undirected graph G, can the nodes be colored red, green and blue so that no adjacent nodes have the same color? yes instance Ryszard Janicki Computability and Complexity 74 / 101

75 NPC-problems: 3-colorability Theorem 3-SAT P 3-COLOR. Proof. Given 3-SAT instance Φ, we construct an instance of 3-COLOR that is 3-colorable i Φ is satisable. See Kleinberg-Tardos for details. 3-COLOR is NP-complete. Ryszard Janicki Computability and Complexity 75 / 101

76 NPC-problems: 3-Subset sum Subset sum SUBSET-SUM. Given natural numbers w 1,, w n and an integer W, is there a subset that adds up to exactly W? Ex. { 1, 4, 16, 64, 256, 1040, 1041, 1093, 1284, 1344 }, W = Yes = Remark. With arithmetic problems, input integers are encoded in binary. Subset sum Poly-time reduction must be polynomial in binary encoding. Theorem. 3-SAT P SUBSET-SUM. Pf. Given an instance Φ of 3-SAT, we construct an instance of SUBSET-SUM that has solution iff Φ is satisfiable. SUBSET-SUM is NP-complete. Ryszard Janicki Computability and Complexity 76 / 101

77 Partition NPC-problems: Partition SUBSET-SUM. Given natural numbers w 1,, w n and an integer W, is there a subset that adds up to exactly W? PARTITION. Given natural numbers v 1,, v m, can they be partitioned into two subsets that add up to the same value ½ Σ i v i? Theorem. SUBSET-SUM P PARTITION. Pf. Let W, w 1,, w n be an instance of SUBSET-SUM. Create instance of PARTITION with m = n + 2 elements. v 1 = w 1, v 2 = w 2,, v n = w n, v n+1 = 2 Σ i w i W, v n+2 = Σ i w i + W Lemma: there exists a subset that sums to W iff there exists a partition since elements v n+1 and v n+2 cannot be in the same partition. v n+1 = 2 Σ i w i W W subset A v n+2 = Σ i w i + W Σ i w i W subset B Ryszard Janicki Computability and Complexity 77 / 101

78 Scheduling with release times NPC-problems: Scheduling with release times Scheduling with release times How to find closest pair with one point in each side? SCHEDULE. Given a set of n jobs with processing time t j, release time r j, and Theorem. Scheduling deadline dsubset-sum j, is with possible release P SCHEDULE. to times schedule all jobs on a single machine such that Def. Let s i be the point in the 2 δ-strip, with the i th smalles Pf. job Given j is processed SUBSET-SUM with instance a contiguous w 1,, w n and slot target of t j time W, construct units in the an instance interval [r j, d j ]? of Theorem. SCHEDULE SUBSET-SUM that is feasible P SCHEDULE. iff there exists a subset that sums to exactly W. Claim. If i j 12, then the distance Pf. Ex. Given SUBSET-SUM instance between w 1,, s w n and target W, construct an instance i and s j is at least δ. Construction. of SCHEDULE that is feasible iff there exists a subset that sums to exactly W. Create n jobs with processing time t j = w j, release time r j = 0, Pf. Construction. and no deadline (d j = 1 + Σ j w j ). Create Create job n jobs 0 with with t No two points lie in same ½ δ-by-½ δ box. 0 = processing 1, release time time r 0 t = j W, w j, and release deadline time d r 0 = j W 0, + 1. Two points at least 2 rows apart Lemma: subset that sums to W iff there exists a feasible schedule. 2 rows and no deadline (d j = 1 + Σ j have w j ). distance 2 (½ δ). Create job 0 with t 0 = 1, release time r 0 = W, and deadline d 0 = W + 1. Lemma: subset that sums to W iff there exists a feasible schedule. i Fact. Claim remains true if we replace 12 with must schedule jobs 1 to n either here must schedule jobs 1 to n either here W W+1 must schedule W W+1 job 0 here or here or here 1 + Σj wj Σj wj Ryszard Janicki Computability and Complexity 78 /

79 NP-complete problems constraint satisfaction 3-SAT poly-time reduces to INDEPENDENT-SET 3-SAT INDEPENDENT-SET DIR-HAM-CYCLE GRAPH-3-COLOR SUBSET-SUM VERTEX-COVER HAM-CYCLE PLANAR-3-COLOR SCHEDULING SET-COVER TSP packing and covering sequencing partitioning numerical PLANAR-3-COLOR is just GRAPH-3-COLOR for planar graphs. Ryszard Janicki Computability and Complexity 79 / 101

80 Interval scheduling Interval Scheduling Job j starts at s j and finishes at f j. Two jobs compatible if they don't overlap. Job j starts at s j and nishes at f j. Goal: find maximum subset of mutually compatible jobs. Two jobs compatible if they don't overlap. Goal: nd maximum subset of mutually compatible jobs. a b c d e jobs d and g are incompatible f g h time 9 Ryszard Janicki Computability and Complexity 80 / 101

81 Interval scheduling: earliest-nish-time-rst algorithm EARLIEST-FINISH-TIME-FIRST (n, s1, s2,, sn, f1, f2,, fn) SORT jobs by finish time so that f1 f2 fn A φ set of jobs selected FOR j = 1 TO n IF job j is compatible with A A A { j } RETURN A Claim Time complexity of the part from `FOR' to `RETURN' is O(n). Proof. position. Can implement earliest-finish-time first in O(n log n) time. Keep track Keep of job track j* that of job was j added that was last to added A. last to A (constant time). Job j is compatible Job j is compatible with A iff s j with f j*. A i s j fj (constant time). Sorting by finish time takes O(n log n) time. However time complexity of the Earliest-nish-time-rst algorithm is O(n log n)! WHY? Ryszard Janicki Computability and Complexity 81 /

82 Time complexity of earliest-nish-time-rst algorithm EARLIEST-FINISH-TIME-FIRST (n, s1, s2,, sn, f1, f2,, fn) SORT jobs by finish time so that f1 f2 fn A φ set of jobs selected FOR j = 1 TO n IF job j is compatible with A A A { j } RETURN A Proposition We can implement earliest-nish-time rst (EFTF) in O(n log n) time. Proof. position. Can implement earliest-finish-time first in O(n log n) time. O(EFTF) = O(SORT) + O(A ) + O(FOR...RETURN A) Keep track of job j* that was added last to A. O(SORT) = O(n log n) Job j is compatible with A iff s j f j*. O(A ) = O(1) Sorting O(FOR...RETURN by finish time takes A) = O(n) log n) time. Hence O(EFTF) = O(n log n) + O(1) + O(n) = O(n log n). 12 Ryszard Janicki Computability and Complexity 82 / 101

83 NP-completeness and the class P Interval scheduling Consider Interval Scheduling problem from a class on Greedy Job j starts at s j and finishes at f j. Algorithms. It has a solution in O(n log n)! Two jobs compatible if they don't overlap. Goal: Job find j starts maximum at s j subset and of nishes mutually at compatible f j. jobs. Two jobs compatible if they don't overlap. Goal: nd maximum subset of mutually compatible jobs. a b c d e jobs d and g are incompatible f g h time INTERVAL-SCHEDULING: For any given k, does there exists a subset of mutually compatible jobs k? Ryszard Janicki Computability and Complexity 83 / 101 9

84 NP-completeness and the class P INTERVAL-SCHEDULING: For any given k, does there exists a subset of mutually compatible jobs k? INTERVAL-SCHEDULING has O(n log n) solution: we just nd a maximum subset of mutually compatible jobs in O(n log n) by using greedy algorithm, and then compare if the number of found jobs with k. Hence INTERVAL-SCHEDULING P. Suppose I will give you an assignment question: Which of the below is true: 1 INTERVAL-SCHEDULING P VERTEX-COVER 2 VERTEX-COVER P INTERVAL-SCHEDULING What should be your answer? Ryszard Janicki Computability and Complexity 84 / 101

85 NP-completeness and the class P (1) Is INTERVAL-SCHEDULING P VERTEX-COVER? The answer is YES. Since VERTEX-COVER is NP-complete, then, by denition (see pages of this Lecture Notes), every X NP is polynomially reducible to VERTEX-COVER, i.e. X P VERTEX-COVER. Since INTERVAL-SCHEDULING P and P NP, then INTERVAL-SCHEDULING NP. But this means INTERVAL-SCHEDULING P VERTEX-COVER! Ryszard Janicki Computability and Complexity 85 / 101

86 NP-completeness and the class P (2) Is VERTEX-COVER P INTERVAL-SCHEDULING? The answer is I DO NOT KNOW. Probably NOT. Theorem We have a theorem: Suppose Y is NP-complete. Then Y P P = NP. Since INTERVAL-SCHEDULING P, if VERTEX-COVER P INTERVAL-SCHEDULING then VERTEX-COVER P! But VERTEX-COVER is NP-complete, so VERTEX-COVER P = P = NP. Hence if you could prove VERTEX-COVER P INTERVAL-SCHEDULING, you are one million of US dollars richer! Ryszard Janicki Computability and Complexity 86 / 101

87 Space Complexity Denition Let M be a deterministic Turing machine. Space complexity of M is a function f : N N, where f (n) is the maximum number of tape cells that M uses for any input of length n. Space complexity of M is f (n) M runs in space f (n). SPACE(f (n)) = {L L is a language decided by O(f (n)) space deterministic Turing Machine} NSPACE(f (n)) = {L L is a language decided by O(f (n)) space non-deterministic Turing Machine} PSPACE = k=0 SPACE(nk ) EXPTIME = k=0 TIME(2k ) Ryszard Janicki Computability and Complexity 87 / 101

88 What is the fundamental dierence between time and space from the viewpoint of complexity theory? Ryszard Janicki Computability and Complexity 88 / 101

89 What is the fundamental dierence between time and space from the viewpoint of complexity theory? Space is reusable, time is not! Ryszard Janicki Computability and Complexity 88 / 101

90 What is the fundamental dierence between time and space from the viewpoint of complexity theory? Space is reusable, time is not! Theorem (Savitch) For every function f : N N, where f (n) log n, NSPACE(f (n)) SPACE(f 2 (n)). Ryszard Janicki Computability and Complexity 88 / 101

91 Coping with NP-completeness Question. Suppose I need to solve an NP-complete problem. What should I do? Answer. Theory says you are unlikely to nd poly-time algorithm. But do not panic! We must just sacrice only one of three desired features. Solve problem to optimality. Solve problem in polynomial time. Solve arbitrary instances of the problem. Case 1. Solve some special cases of NP-complete problems. Ryszard Janicki Computability and Complexity 89 / 101

92 Coping with NP-completeness Many special cases of SAT problem can be solve in linear time by using clever heuristics. In reality cases of SAT that cannot be solve in linear time are very rare. SAT solvers are very often used in proving properties of programs, for example in Microsoft C # based specications techniques. It is consider a huge achievement if a given decision or optimization problem can be reduced to SAT problem (for example in concurrency theory), as SAT solvers are very ecient in almost every case. Ryszard Janicki Computability and Complexity 90 / 101

93 Finding small vertex covers Question. VERTEX-COVER is NP-complete. But what if k is small? Brute force: O(kn k+1 ). Try all C(n, k) = O(n k ) subsets of size k. Takes O(kn) time to check whether a subset is a vertex cover. Hence totally O(n k )O(kn) = O(kn k+1 ) Goal. Limit exponential dependency on k, say to O(2 k kn). Example n = 1000, k = 10. Brute. kn k+1 = = infeasible. Better. 2 k kn = 10 7 = feasible. Remark. If k is a constant, then the algorithm is poly-time; if k is a small constant, then it is also practical. Can we nd `Better'? Ryszard Janicki Computability and Complexity 91 / 101

94 Finding small vertex covers Finding small vertex covers: algorithm Claim. Proposition The following algorithm determines if G has a vertex cover of size The k following O(2 algorithm determines if G has a vertex cover of size k in O(2 k kn) time. kn) time. Vertex-Cover(G, k) { if (G contains no edges) return true if (G contains kn edges) return false } let (u, v) be any edge of G a = Vertex-Cover(G - {u}, k-1) b = Vertex-Cover(G - {v}, k-1) return a or b Pf. Correctness follows from previous two claims. Ryszard Janicki Computability and Complexity 92 / 101

95 Coping with NP-completeness Question. Suppose I need to solve an NP-complete problem. What should I do? Answer. Theory says you are unlikely to nd poly-time algorithm. We must sacrice one of three desired features. Solve problem to optimality. Solve problem in polynomial time. Solve arbitrary instances of the problem. ρ-approximation algorithm. Guaranteed to run in polynomial time. Guaranteed to solve arbitrary instance of the problem Guaranteed to nd solution within ratio ρ of true optimum. Ryszard Janicki Computability and Complexity 93 / 101

96 Load Balancing Load balancing Input. m identical machines; n jobs, job j has processing time t j. Input. Job jm must identical run contiguously machines; n jobs, on one job machine. j has processing time t j. Job j must run contiguously on one machine. A machine can process at most one job at a time. A machine can process at most one job at a time. Denition (Load and Makespan) Def. Let J(i) be the subset of jobs assigned to machine i. 1 Let J(i) be the subset of jobs assigned to machine i. The load of machine i is L The load of machine i is L i = i = Σ j J(i) t j. t j. k J(i) Def. The makespan is the maximum load on any machine L = max i L i. 2 The makespan is the maximum load on any machine L = max i {L i }. Load balancing. Assign each job to a machine to minimize makespan. Load balancing. Assign each job to a machine to minimize makespan. machine 1 a dmachine 1 f machine 2 b c Machine e2 g 0 L1 L2 time 4 Ryszard Janicki Computability and Complexity 94 / 101

97 Load Balancing is NP-complete Proposition Load balancing is hard even if only 2 machines. Proof. SUBSET-SUM P LOAD-BALANCE. The proof is not dicult but it is not part of this course. Ryszard Janicki Computability and Complexity 95 / 101

98 gest processing time (LPT). Sort n jobs in descending order of Greedy with Longest Processing Time (LPT). Sort n jobs in descending order of processing time. Consider n jobs in some xed order. Assign job j to machine whose load is smallest so far. cessing time, and then run list scheduling algorithm. LPT-List-Scheduling(m, n, t 1,t 2,,t n ) { Sort jobs so that t 1 t 2 t n for i = 1 to m { } L i 0 J(i) for j = 1 to n { i = argmin k L k J(i) J(i) {j} load on machine i jobs assigned to machine i machine i has smallest load assign job j to machine i } L i L i + t j } return J(1),, J(m) update load of machine i Implementation. O(n log n) because of sorting when priority queue is used to represents loads. Ryszard Janicki Computability and Complexity 96 /