P is the class of problems for which there are algorithms that solve the problem in time O(n k ) for some constant k.

Transcription

1 Complexity Theory Problems are divided into complexity classes. Informally: So far in this course, almost all algorithms had polynomial running time, i.e., on inputs of size n, worst-case running time is O(n k ) for some constant k. Is this always the case? Obviously no (just consider basic FF algorithm: running time depends on f ). Some problems are even insolvable, regardless of how much time is provided, e.g., the famous halting problem: It is not possible to tell, in general, whether a given algorithm will halt for some given input. Also, there are problems that are solvable, but not in time O(n k ) for any constant k (they may require strictly exponential time, or even more). P is the class of problems for which there are algorithms that solve the problem in time O(n k ) for some constant k. NP is the class of problems for which there are algorithms that verify solutions in time O(n k ) for some constant k. Intuitively, solving harder that merely verifying, so maybe problems in NP-P harder than problems in P.... but... is there something in NP-P????!? One of the most famous question in TCS: P = NP or P 6= NP? Complexity 1 Complexity 2 Obviously, P NP. Back to this verification business. Study NP-complete problems. They are in NP, and, in a sense, they are hard among all problems in NP (or, as hard as any other); will be formalised later. Example Hamiltonian cycle. As mentioned, NP-complete, so apparently hard, perhaps no polynomial-time algorithm that can compute a solution (for all instances). Will prove later: if one can show for only one NP-complete problem that it is in fact in P, then P = NP. Seems to be difficult, so far nobody has succeeded ;-) Sometimes slight modifications to some problem make a huge difference. But: verifying a solution/certificate is trivial! Certificate is sequence (v 1,v 2 m...,v V ) (some ordering of vertices), just have to check whether it s a proper cycle. Euler tour vs Hamiltonian cycle. Euler tour: does given directed graph have a cycle that traverses each edge exactly once? Easy, O(E). Hamiltonian cycle: does given directed graph have a simple cycle that contains each vertex? NPcomplete! Techniques for proving NP-completeness differ from usual techniques for designing or analysing algorithms. In the following, a few key concepts. Complexity 3 Complexity 4

2 1) Decision problems vs optimisation problems Many problems are optimisation problems: compute shortest paths, maximum matching, maximum clique, maximum independent set, etc. Concept of NP-completeness does not apply (directly) to those, but to decision problems, where answer is just yes or no Does this given graph have a Hamiltonian cycle? Given this graph, two vertices, and some k, does G contain a path of length at most k from u to v? However, usually close relationship. In a sense, decision is easier than optimisation (or not harder). Example: can solve SP decision problem by solving SP optimisation: just compare length of (computed) shortest path with k. General idea: evidence that decision problem is hard implies evidence, that related optimisation problem is hard. Complexity 5 2) Encodings When we want an algorithm to solve some problem, problem instances must be encoded. For instance, encode natural numbers as strings {0, 1, 10, 11, 100, 101,...} encoding e : IN! {0, 1} with e(i) =(i) 2 Concrete problem: problem whose instance set is the set of binary strings (read: encodings of real instances) We say an alg. solves some concrete decision problem in time O(T (n)) if, when given problem instance i of length n = i, it can produce solution in time O(T (n)). A concrete problem is poly-time solvable, if there is an alg. to solve it in time O(n k ) for some constant k. P is the set of all poly-time solvable concrete decision problems. Complexity 6 Would like to talk about abstract problems rather than concrete ones, independent of any particular encoding. But... efficiency depends heavily on encoding. Example: suppose integer k is input to an alg., running time is (k). If k is given in unary, then then running time is O(n) in length-n inputs, polynomial. If k is given binary, then input length is n = blog kc +1; exponential running time (k) = (2 n ). In practice, rule out expensive encodings like unary. A formal-language framework An alphabet is finite set of sysmbols. A language L over is any set of strings made up of symbols from. Empty string is, empty language is ;. Language of all strings over is Ex: = {0, 1} ) = {, 0, 1, 00, 01, 10, 11, 000,...}, all binary strings. Every language L over is subset of. Union L 1 [L 2 and intersection L1\L 2 just like with ordinary sets, complement L = L. Concatenation of L 1 and L 2 is {xy : x 2 L 1,y 2 L 2 }. Closure of L is { } [ L [ L 2 [ L 3 [ with L k being L concatenated to itself k times. Complexity 7 Complexity 8

3 Set of instances for any decision problem Q is, where = {0, 1}. Encoding: Using formal languages is fine Q is entirely characterised by those instances that produce 1 (yes), so we view Q as language L over = {0, 1} with L = {x 2 : Q(x) =1} Ex: (<something> meaning standard enc. of something) HAMILTON = {< G >: G contains Hamiltonian cycle} SAT = {< F >: formula F is satisfyable} We say function f : {0, 1}! {0, 1} is polynomially computable if there is polynomial-time TM M that, given x 2 {0, 1}, computes f(x). For set I of problem instances, two encodings e 1 and e 2 are polynomially related if there are two poly-time computable functions f 12 and f 21 such that 8i 2 I, f 12 (e 1 (i)) = e 2 (i) and f 21 (e 2 (i)) = e 1 (i) If two encodings of some abtract problem are polynomially related, then whether problem is in P is independent of which encoding we use. Note: the length can increase only by a polynomial factor! It can be shown: the standard inputs (graph as an adjacency matrix,...) are polynomially related. Complexity 9 Complexity 10 Lemma. Let Q be an abstract decision problem on an instance set I, let e 1,e 2, be polynomially related encodings on I. Then, e 1 (Q) 2 P iff e 2 (Q) 2 P. Proof. We show one direction (e 1 (Q) 2 P ) e 2 (Q) 2 P), other is symmetric. Suppose e 1 (Q) 2 P, i.e., e 1 (Q) can be solved in time O(n k ) for some constant k, and that for any instance i, e 1 (i) can be computed from e 2 (i) in time O(n c ) for some constant c, n = e 2 (i). To solve e 2 (Q), on input e 2 (i), first compute e 1 (i), and run alg. for e 1 (Q) on e 1 (i). Time: conversion takes O(n c ), therefore e 1 (i) = O(n c ) (can t write more). Solving e 1 (i) takes O( e 1 (i) k )=O(n ck ), polynomial. Thus e 2 (Q) 2 P. 3) Machine Model Our goal is to say that Problem A can not be solved in polynomial time. But on which machine? A parallel machine with 1000 processors? A modern computer? Can we solve more in polynomial time if the computers get faster? In complecity theory people use a very simple machine model, the so-called Turing machine. One can show that everything that can be solved by a modern computer in polynomial time can also be solved on a TM in polynomial time. Complexity 11 Complexity 12

4 Machine Model A TM (Turing machine) consists of a tape of infinite length (in one direction) and a pointer. The tape consists of cells and every cell can store one symbol. Formally, a Turing Machine M =(Q,, Q is the (finite) set of states.,,q 0,B,F) is defined as follows. The pointer points to one of the memory cells. In the beginning the input is in cell 1,...and the pointer position is cell 1. is the set of tape symbols. is the set of input symbols ( ). The TM has a finite control. In every step the control is in one of a finite amount of states. In every step the TM does the following. It reads the symbol at the actual pointer position B = {N, R, L}. :(Q )! (Q {L, R, N} is the transition function. q 0 2 Q is the initial state. F Q is an end state. It writes a new symbol into the position (which one depends on the state) It can move the pointer one step to the left or to the right. Complexity 13 Complexity 14 State of the TM: 1 q 2 with q 2 Q and 1, 2 2. here = 1 2 is the contents of the tape. The language L accepted by TM M is the set of words from so that M reaches a state in F. To make our life easier we will say in the following that M outputs 1. If now 2 = 0 2 the TM looks up (q, ) =(q0, If M = N the next state will be 1 q If M = R the next state will be 1 q ,M). In all other cases M rejects in input. It can stop in a state not in F and output 0, or M can go into an endless loop. If M = R and 1 = 0 1 the next state will be 1q We write 1 q 2 ) 0 1 q0 0 2 if a one step transition from 1q 2 to 0 1 q0 0 2 exists. We write 1 q 2 ) 0 1 q0 0 2 if a (possibly long) transition from 1q 2 to 0 1 q0 0 2 exists. Examples for TM: see homework. Complexity 15 Complexity 16

5 We say TM M accepts string x 2 {0, 1} if, given input x, M s output is M(x) = 1. The language accepted by M is L = {x 2 : M(x) =1}. Moreover: Theorem. P = {L : L is accepted by a TM in poly-time}. Proof. Clearly decided)accepted, so we need only show accepted)decided. M rejects x if M(x) =0. Important: even if L is accepted by some TM M, M will not necessarily reject x 62 L. A language L is decided by TM M, if L is accepted by M and every string not in L is rejected. L is accepted/decided in polynomial time by TM M, if it is accepted/decided by M in time O(n k ) for some const. k. Let L be accepted by M in time O(n k ), thus there is constant c s.t. M accepts L in at most T = cn k steps. Construct M 0 : for any input string x, M 0 simulates M for T steps. If M has accepted, then M 0 accepts x by outputting 1, otherwise M 0 rejects by outputting 0 q.e.d. Note: proof is nonconstructive, we may not know runtime bound for given L 2 P (but it exists!). Now: alternative definition P = {L {0, 1} : 9 TM M that decides L in poly-time} Complexity 17 Complexity 18 Poly-time verification Look at TMs that verify membership in languages. Ex: for instance <G>of HAMILTON, we are also given ordering c of vertices on cycle. Can easily check whether c is a proper cycle on all vertices of G, so c is certificate that G indeed belongs to HAMILTON. As we will see later, HAMILTON is NP-complete, and thus most likely not in P, but verification is easy. Define verification TM: two argument TM M, one argument is ordinary input string x, other is binary string y called certificate. M verifies input string x if 9 certificate y s.t. M(x, y) =1. The language verified by M is L = {x 2 {0, 1} : 9y 2 {0, 1} s.t. M(x, y) =1} Intuition: M verifies L if 8x 2 L there is y that M can use to prove x 2 L. For any x 62 L, there must be no certificate proving that x 2 L. Ex: If G 62 HAMILTON there must be no permutation of vertices that can fool verifier into believing G is hamiltonian. Def: a language L belongs to NP if and only if 9 two-input poly-time TM M and constant c s.t. L = {x 2 {0, 1} : 9 certificate y with y = O( x c ) such that M(x, y) =1} Historically, NP= non-deterministic poly-time, all the problems that are accepted by a poly-time non-deterministic Turing machine (NTM), which 1. non-deterministically guesses a solution (certificate) if there is one, and 2. deterministically verifies it in poly-time. See the similarity? Facts: NP 6= ; (HAMILTON2 NP) P NP Complexity 19 Complexity 20

6 Open questions: NP-completeness P = NP or P 6= NP? NP closed under complement, i.e., L 2 NP ) L 2 NP? (with co-np = {L : L 2 NP}, equiv. to NP = co-np) Since P is closed under complement, P NP\ co-np. Thus four possibilities: Remember: class of NP-complete problems; property: if one of them is in P, then all of NP. Reducibility P = NP = co NP P NP = co NP Intuition: problem Q can be reduced to Q 0 if any instance of Q can be easily rephrased as instance of Q 0 co NP P = NP co NP NP co NP P = NP P co NP NP We say that language L 1 is poly-time reducible to language L 2, written L 1 apple p L 2, if there exists a poly-time computable function f : {0, 1}! {0, 1} s.t. 8x 2 {0, 1}, TL: NP = co-np and P = NP, most unlikely of the four TR: NP = co-np and P 6= NP BL: NP 6= co-np and P = NP \ co-np BR: NP 6= co-np and P 6= NP \ co-np, most likely Complexity 21 f is called reduction function. x 2 L 1 if and only if f(x) 2 L 2. Complexity 22 NP-completeness II Idea of poly-time reduction from L 1 to L 2 : A language L {0, 1} is called NP-complete if 1. L 2 NP, and 2. L 0 apple p L for every L 0 2 NP. If L satisfies only (2) it is called NP-hard. L 1,L 2 {0, 1} f provides poly-time mapping s.t. if x 2 L 1 then f(x) 2 L 2 if x 62 L 1 then f(x) 62 L 2 Thus f maps any instance x of decision problem represented by L 1 to instance f(x) of problem represented by L 2 Answer to whether f(x) 2 L 2 directly provides answer to whether x 2 L 1 Complexity 23

7 {0,1}* {0,1}* f Proving that some language is in P is easy now: L1 L2 Lemma. If L 1,L 2 {0, 1} with L 1 apple p L 2, then L 2 2 P implies L 1 2 P. Proof. Let M 2 be poly-time TM that decides L 2, let F be poly-time TM that computes reduction function f. Construction of poly-time TM M 1 that decides L 1 : x F f(x) A2 yes, f(x) in L2 yes, x in L1 A1 no, f(x) not in L2 no, x not in L1 Complexity 24 Complexity 25 The set of NP-complete languages is called NPC. Most people believe P 6= NP, thus NP-completeness may helpful for showing P = NP or P 6= NP: Theorem. (i) If any NP-complete problem is in P, then P = NP. Equivalently, (ii) if any problem in NP is not in P, then no NP-complete problem is in P. NP P NPC Proof. (i) Suppose L 2 P and L 2 NPC. For any L 0 2 NP, we have L 0 apple p L by definition of NP-completeness. By last lemma, L 0 2 P, and thus NP = P. (ii) is contrapositive of (i) [a la (A ) B) () ( B ) A)] Anyways, how does one prove NP-completeness of some particular problem? Two ways... By hand (master reduction), show that L 2 NP (easy part) and L 0 apple p L for every L 0 2 P (hard part). Most famous (and first): Cook s proof that SAT2 NPC. or (usually significantly) easier, given that you already know some other problem to be NP-complete: Complexity 26 Complexity 27

8 A helpful lemma: Lemma. (Transitivity) apple p is a transitive relation on languages, i.e., if L 1 apple p L 2 and L 2 apple p L 3, then L 1 apple p L 3. Proof. By definition, there are poly-time functions f and g such that x 2 L 1, f(x) 2 L 2 and y 2 L 2, g(y) 2 L 3, thus x 2 L 1, f(x) 2 L 2, g(f(x)) 2 L 3. Obviously, g(f( )) is poly-time (since f(x) is polynomial in x ). Theorem. If L is a language s.t. L 0 apple p L for some L 0 2 NPC, then L is NPhard. Moreover, if L 2 NP, then L 2 NPC. Proof. L 0 is NP-complete, thus we have L 00 apple p L 0 for all L 00 2 NP. With L 0 apple p L and by transitivity (last lemma) we have L 00 apple p L, so L is NP-hard. If L 2 NP, we have L 2 NPC by definition. NP L L Method for proving that L 2 NPC: prove L 2 NP select known NP-complete L 0 describe alg. F that computes f mapping every instance x 2 {0, 1} of L 0 to instance f(x) of L prove that f satisfies x 2 L 0, f(x) 2 L 8x 2 {0, 1} prove that F runs in poly time Complexity 28 Complexity 29 Formula satisfiability Was first problem to be proven NP-complete. A formula with a satisfying t.a. is called satisfiable. SAT = {< F >: F is a satisfiable boolean formula}. Among most popular problems for reductions. Definition in terms of language SAT. An instance of SAT is a boolean formula F composed of 1. n boolean variables x 1,x 2,...,x n 2. m boolean operators; any boolean function with one or two inputs and one output, such as ^ (AND), _ (OR), (NOT),! (implication), $ (if and only if) 3. parenthesis (WLOG assume no redundant parenthesis) Easy to encode F in length polynomial in n + m. A truth assignment for F is set of values for the variables. A satisfying truth assignment is a t.a. that causes it to evaluate to 1. Complexity 30

9 Example Thus F 2 SAT. F =((x 1! x 2 ) _ (( x 1 $ x 3 ) _ x 4 )) ^ x 2 has satisfying assignment x 1 =0,x 2 =0,x 3 =1,x 4 =1 F = ((0! 0) _ (( 0 $ 1) _ 1)) ^ 0 = (1_ ((1 $ 1) _ 1)) ^ 1 = (1_ (1 _ 1)) ^ 1 = (1_ 1) ^ 1 = 1^ 1 = 1 Note: a b a! b a b a $ b Complexity 31 Theorem. SAT is NP-complete. Proof. (i) SAT 2 NP. Easy, replace each variable with corresponding value, evaluate. Polynomial time. (ii) SAT is NP-hard, i.e., L apple p SAT for all L 2 NP. Not quite so easy, proof not shown. Now: could use SAT to show NP-completeness of other languages. First, consider CNF-SAT and 3-CNF-SAT. A literal is an occurrence of variable (x i ) or its negation ( x i ). A boolean formula is in conjunctive normal form (CNF) if it is an AND of clauses, each of which is OR of literals. Ex: (x 1 _ x 2 _ x 3 ) ^ (x {z } 2 _ x 4 ) ^ (x {z } 2 _ x 3 _ x 4 _ x 5 ) {z } clause clause clause A formula is in 3CNF if each clause contains exactly three, distinct literals. Complexity 32 CNF-SAT = {< F >: F is in CNF and is satisfiable} 3-CNF-SAT = {< F >: F is in 3CNF and is satisfiable} Theorem. CNF-SAT is NP-complete. Proof. Obviously in NP (since SAT 2 NP). NP-hardness can be shown e.g. by reduction from SAT, i.e., SAT apple p CNF-SAT. Proof not shown. Theorem. 3-CNF-SAT is NP-complete. Proof. Obviously in NP (since SAT 2 NP and CNF-SAT 2 NP). NP-hardness by reduction from CNF-SAT. Let F = F 1 ^ F 2 ^ ^ F k be a CNF formula. Suppose clause F i has more than three literals e.g. F i = 1 _ 2 _ _ ` with ` > 3

10 Note: are literals rather than variables. Introduce new variables y 1,y 2,...,y` 3 and replace F i with We want to construct new clauses with exactly three literals each, but same satisfiability property. Fi 0 = ( 1 _ 2 _ y 1 ) ^ ( 3 _ y 1 _ y 2 ) ^ ( 4 _ y 2 _ y 3 ) ^ ^ ( ` 2 _ y` 4 _ y` 3 ) ^ ( ` 1 _ ` _ y` 3 ) Now, an assignment satisfies F i if and only if it (rather, an extension) satisfies F 0 i. ) Suppose F i is satisfied by some assignment. Must be that j =1for some j. Suppose 1 = = j 1 =0and j =1. Then and y 1 = = y j 2 =1 y j 1 = = y` 3 =0 is an extension of the original assignment that satisfies F 0 i (all new clauses in F 0 i are satisfied). Complexity 33 Complexity 34 ( Suppose some assignment satisfies F 0 i. Need to show that for F i, 9j with j =1 when restricting to assignment for s. Suppose that s not true, i.e. i =0for i =1,...,`. Then, by construction, must be that y 1 =1(1st clause). Also, since y 1 =1, y 1 =0, must be that y 2 =1(2nd clause). By induction, must be that y m =18m 2 {1,...,` 3}. What about size 1&2 clauses? ( 1 )! ( 1 _ y 1 _ y 2 ) ^ ( 1 _ y 1 _ y 2 ) ^ ( 1 _ y 1 _ y 2 ) ^ ( 1 _ y 1 _ y 2 )^ If LHS satisfied, then obviously also RHS. Suppose RHS satisfied but 1 =0. Then in each of the four RHS-clauses the y s must take care of the business. But: impossible! But wait, hang on, don t move! Last clause in Fi 0 would be 0, <blink> contradiction! </blink> Similar reasoning. ( 1 _ 2 )! ( 1 _ 2 _ y) ^ ( 1 _ 2 _ y) Size of formula remains polynomial in what it was. Conclusion: each assignm. that satisfies F 0 i also satisfies F i. Time also polynomial. (q.e.d.) Complexity 35 Complexity 36

11 Another NP-complete problem: CLIQUE A clique in an undirected graph G =(V,E) is subset V 0 V, each pair of which is connected by an edge in E, i.e., a complete subgraph. A k-clique is a clique of size k. Optimisation problem: given G, find a clique of maximum size. Decision problem: given G and k 2 IN, does G contain k-clique? CLIQUE = {< G,k>: G is a graph with a k-clique} Naive solution: list all k-subsets, check each one. Running time is k 2 V k (time for checking # of k-subsets). Polynomial if k constant, but super-polynomial if e.g. k V /2. Complexity 37 Theorem. CLIQUE is NP-complete. Proof. To show CLIQUE 2 NP, use V 0 V as certificate. Just check whether (u, v) 2 E for all u, v 2 V 0. Now we show 3-CNF-SAT apple p CLIQUE. Given instance F = C 1 ^ C 2 ^ ^ C k in 3CNF. Each clause C i contains three distinct literals i 1, i 2, i 3. Shall construct G s.t. G contains k-clique if and only if F is satisfiable. For each clause C r =( r 1 _ r 2 _ r 3 ) we place triple of vertices vr 1,vr 2,vr 3 into V. Edge between v r i and vs j if 1. vi r and vs j are in different triples, i.e., r 6= s, and 2. their corresponding literals are consistent: r i is not the negation of s j Complexity 38 Ex: F =(x 1 _ x 2 _ x 3 ) ^ ( x 1 _ x 2 _ x 3 ) ^ (x 1 _ x 2 _ x 3 ) becomes 1st clause $x_1$ $\neg x_2$$\neg x_3$ 2nd clause $\neg x_1$ $x_2$ $x_3$ $x_1$ $x_2$ $x_3$ Obviously, construction possible in polynomial time. Complexity 39 3rd clause ) Suppose F has a satisfying assignment. Then each clause C r contains 1 literal r i =1. Picking one such true literal from each clause corresponds to picking k vertices in G, yielding V 0. Claim: V 0 is a k-clique. For any v r i,vs j 2 V 0, r 6= s, both corresponding literals are true, so cannot be complements. Thus, edge (v r i,vs j ) 2 E. ( Suppose G has k-clique V 0. No inner-triple edges, thus exactly one vertex per triple in V 0. No edges between vertices corresponding to inconsistent literals. Thus, we can assign 1 to each literal r i s.t. vr i 2 V 0. One true literal per clause, thus F satisfied. (q.e.d.) Complexity 40

12 Yet another example: INDEPENDENT-SET An independent set (IS) in an undirected graph G = (V,E) is subset V 0 V, each pair of which is not connected by an edge in E, i.e., an edge-less subgraph. A k-is is an independent set of size k. Why superpolynomial? Let n = V, then k 2 V = (n/2)2 n k n/2 = (n 2 n! /4) (n/2)! (n = (n 2 n! /4) ((n/2)!) 2 n/2)! Optimisation problem: given G, find an IS of maximum size. Decision problem: given G and k 2 IN, does G contain a k-is? INDEPENDENT-SET = {< G,k>: G is a graph with a k-is} Naive solution: list all k-subsets, check each one. Running time is k 2 V k (time for checking # of k-subsets). Polynomial if k constant, but super-polynomial if e.g. k V /2. By Stirling s famous formula x! p 2 x (x/e) x, p k 2 V (n 2 2 n (n/e) n /4) k q2 (n/2) (n/2e) (n/2) 2 p = (n 2 2 n (n/e) n /4) 2 (n/2) (n/2e) p n = (n 2 2 n /4) 2 (n/2) 2n = (n 2 /4) 2p 2 n 2 n 2n =(n 2 2 /4) p 2 n 2 n 2n 2 = 4 p 2 pn 2n = n3/2 2 p 2 2n = (2 n ) Complexity 41 Theorem. INDEPENDENT-SET is NP-complete. Proof. To show INDEPENDENT-SET 2 NP, use V 0 V as certificate. Just check whether (u, v) 62 E for all u, v 2 V 0. Now we show CLIQUE apple p INDEPENDENT-SET. Given instance (G, k). Construct G 0 =(V,E 0 ) with E 0 = V V {(v, v) : v 2 V } {z } self-loops E Claim: G contains k-clique if and only if G 0 contains k-is. Trivially true, since a clique is a subset of vertices with an edge between any pair of vertices, and an IS is a subset of vertices without an edge between any pair of vertices. Construction clearly in P thus... (q.e.d.) Complexity 43