Limits to Approximability: When Algorithms Won't Help You Note: Contents of today s lecture won t be on the exam
Outline Limits to Approximability: basic results Detour: Provers, verifiers, and NP Graph non-isomorphism and coin-flipping verifiers Probabilistically checkable proofs (PCPs) MAX 3SAT is hard to approximate
Limits to Approximability Bad news on two fronts: Sometimes, reductions don t help us get good (e.g., with a constant factor guarantee) approximation algorithms Some problems don t have good approximation algorithms at all (unless P = NP)
Limits to Approximability When reductions don t help
Limits to Approximability When reductions don t help Vertex Cover: Given an undirected graph G and a positive integer k, does G have a vertex cover of size at most k? Independent Set: Given an undirected graph G and a positive integer k, does G have an independent set of size at least k? Min Vertex Cover: Given an undirected graph G, find a vertex cover of G of minimum size Max Independent Set: Given an undirected graph G, find an independent set of G of maximum size
Limits to Approximability When reductions don t help There's an efficient approximation algorithm for Min Vertex Cover with approximation ratio 2 There's an efficient reduction from Independent Set to Vertex Cover Can these be combined to get an efficient approximation algorithm for Max Independent Set? Unfortunately not. Let's look at the details
Limits to Approximability When reductions don t help Let G = (V,E) be an undirected graph U is a vertex cover of G if an only if V U is an independent set U is a min vertex cover if and only if V U is a max independent set So the reduction from Independent Set to Vertex Cover is trivial: I = (G, k) à I' = (G, n-k), where n is the number of nodes of G
Limits to Approximability When reductions don t help Suppose G has 1000 vertices, and let the size of the minimum vertex cover be 490. Our approximation algorithm finds a vertex cover of size 980. The size of the maximum independent set is 1000 490 = 510, but out algorithm gives us an independent set of size just 20 The reduction is not "approximation preserving" (Of course, there might be some other way to approximate Independent Set)
Limits to Approximability When there s no hope
Limits to Approximability When there s no hope Can we show that some problems are "inapproximable"? Yes, assuming that NP P (or using some other complexity-theoretic assumption) An early result along these lines pertains to the Traveling Salesman Problem (Min TSP): Given a list of n cities, and the cost to travel between each pair, find a tour of the cites of minimum cost
Limits to Approximability When there s no hope Claim: There is no poly-time algorithm for Min TSP with constant approximation ratio, unless P = NP Proof: Suppose algorithm k-min-tsp solves Min TSP with approximation ratio k We'll show a polynomial time algorithm for Hamiltonian Circuit problem (HC): does an undirected graph G have a Hamiltonian circuit, i.e., a cycle that includes each node exactly once? HC is NP-complete, so P=NP.
Limits to Approximability When there s no hope From Moore
Limits to Approximability When there s no hope From Moore
Limits to Approximability When there s no hope HC-Alg(G) Construct an instance I of Min TSP as follows: Cities correspond to the nodes of graph Distance between i and j is 1 if {i,j} is in E and is k V otherwise If k-min-tsp(i) k V, output Yes", Else output No" Run time: polynomial in the size of G
Limits to Approximability When there s no hope Claim: G has a Hamiltonian Circuit iff HC-Alg(G) outputs Yes Proof: G has a Hamiltonian circuit I has a tour of cost V k-min-tsp(i) outputs a tour of cost k V HC-Alg(G) outputs "yes"
Limits to Approximability When there s no hope Claim: G has a Hamiltonian Circuit iff HC-Alg(G) outputs Yes Thus the existence of a poly-time algorithm with constant factor approximation ratio for Min TSP would imply that NP=P
Limits to Approximability State of affairs so far No algorithm with constant factor approximation ratio (unless NP = P): Min TSP Algorithms with constant factor approximation ratios: Min Vertex Cover, Max Sat and more. Polynomial time approximation scheme: Knapsack
Limits to Approximability State of affairs so far No algorithm with constant factor approximation ratio (unless NP = P): Min TSP Algorithms with constant factor approximation ratios: Min Vertex Cover, Max Sat and more. Do these problems have efficient approximation schemes? Unfortunately not (unless NP = P) Polynomial time approximation scheme: Knapsack
Detour: Provers, Verifiers, and NP A verifier V for a decision problem D takes both an instance I of D and a proof π, such that: If I is a yes-instance, then for some W, V(I,π) = yes If I is a no-instance, then for all W, V(I,π) = no NP: class of problems with polynomial time verifier algorithms
Graph Isomorphism 1 5 2 4 3 Given two undirected graphs with an equal number of nodes and edges, can the labels of nodes in one graph be permuted to obtain the second graph?
Graph Isomorphism 1 5 2 4 3 Graph Isomorphism is in NP From Moore
Graph Isomorphism 1 1 5 2 5 2 3 4 4 3 Graph Isomorphism is in NP From Moore
Graph Non-Isomorphism How to prove that two graphs are not isomporphic? From Moore
Graph Non-Isomorphism A randomized verifier Input: two graphs G 1, G 2 Repeat, say k times Verifier: Choose one graph, say G i, at random Prover: Randomly permute G i to obtain H and send H to the prover Send either 1 or 2 to the verifier Verifier: Reject if the prover s number is not i Verifier: Accept
Graph Non-Isomorphism A randomized verifier How to prove that two graphs are not isomporphic? From Moore
Graph Non-Isomorphism The verifier uses private coins, i.e., the prover does not see the verifier s random bits The protocol would not be correct if the coins were public It turns out, however, that there is a public-coin verifier for for graph non-isomorphism
PSPACE and Randomized Verifiers In fact, every decision problem D in PSPACE has a public-coin, poly-time verifier V If instance I is a yes instance of D, a prover can convince V to accept with probability 1 If instance I is a no instance, V accepts with low probability, no matter what the prover does V interacts polynomially many times with the prover
Limited Randomized Verifiers: PCPs Much more limited verifiers can recognize all decision problems in NP
Limited Randomized Verifiers: PCPs Given an instance I of decision problem D, where the size of I is n: The verifier V receives a proof π (a binary string) The verifier V can flip O(log n) bits The verifier V can only examine O(1) bits of the proof The bits examined may depend on the random bits, but are examined non-adaptively We ll call a verifier with these properties a probabilistically checkable proof (PCP) verifier
PCPs Let V(I,π) denote the output of verifier V on inputs I, π (Note that V(I,π) is a random variable) V is a probabilistically checkable proof system (PCP) for decision problem D if I is in D π in {0,1} * Pr[V(I,π) = yes] = 1 I is not in D π in {0,1} * Pr[V(I,π) = yes] ½ The class of decision problems that have PCP verifiers is denoted by PCP(log n, 1)
PCPs PCP Theorem: NP = PCP(log n,1)
Max 3SAT is Hard to Approximate We can use the PCP theorem to prove this!
Max 3SAT is Hard to Approximate Max 3SAT: Given a Boolean formula ϕ in 3-conjunctive normal form (i.e., each clause has at most three literals), find the maximum number of clauses that can be simultaneously satisfied Theorem: For some constant c > 1, if there is a polynomial time approximation algorithm for Max 3SAT with approximation ratio c, then P=NP
Max 3SAT is Hard to Approximate Key ideas in proof of Theorem
Max 3SAT is Hard to Approximate Key ideas in proof of Theorem Let D be in NP, let V be a PCP for D Using V, we ll construct a mapping I à ϕ I from instances of D to instances of Max 3SAT, such that for some c > 0 and all I, I is a yes instance ϕ I is satisfiable I is a no instance at most a fraction (1-c) of the clauses of ϕ I are simultaneously satisfiable
Max 3SAT is Hard to Approximate Key ideas in proof of Theorem Let V use q queries, r(n) random bits, and get a proof of length l(n). Fix instance I of D of size n For each string τ of length r(n) let b τ,1, b τ,2,... b τ,q be the positions of the proof that V queries on coin flip sequence τ
Max 3SAT is Hard to Approximate Key ideas in proof of Theorem We can construct a 3CNF formula ϕ τ of constant size (depending on q), with variables corresponding to bits b τ,1, b τ,2,... b τ,q of the proof, such that is ϕ τ satisfiable iff V accepts when its random string is τ and the proof bits The overall 3CNF formula ϕ I is the conjunction of all of the ϕ τ s
Max 3SAT is Hard to Approximate Key ideas in proof of Theorem If I is a yes instance, the bits of the "correct" proof correspond to a truth assignment that satisfies ϕ I If I is a no instance, any proof fails to satisfy at least half of the ϕ τ s and thus some constant fraction of the clauses of ϕ I
Summary Many results on the limits of approximability have resulted from the study of randomized verifiers Hastad showed that if there is a (8/7-ε)- approximation algorithm for Max 3SAT, for any ε > 0, then NP=P (p.s. Contents of today s lecture won t be on the exam.)