7.8 Intractability. Overview. Properties of Algorithms. Exponential Growth. Q. What is an algorithm? A. Definition formalized using Turing machines.

Transcription

1 Overview 7.8 Intractability Q. What is an algorithm? A. Definition formalized using Turing machines. Q. Which problems can be solved on a computer? A. Computability. Q. Which algorithms will be useful in practice? A. Analysis of algorithms. Q. Which problems can be solved in practice? A. Intractability. Introducti on to Computer Sci ence Robert Sedgew i ck and Kevi n Wayne Copyri ght w w.cs.pri nceton.edu/introcs 2 Properties of Algorithms Exponential Growth Q. Which algorithms are useful in practice? Exponential growth dwarfs technological change. Suppose each electron in the universe had power of today's supercomputers And each works for the life of the universe in an effort to solve one TSP problem via brute force. A working definition. [von Neumann 1953, Godel 1956, Cobham 1960, Edmonds 1962] Measure running time as a function of input size N. Efficient = polynomial time for all inputs. Inefficient = "exponential time" for some inputs. Quantity Ex. Dynamic programming algorithm for edit distance takes N2 steps. Ex. Brute force algorithm for TSP takes N steps Age of universe in seconds 1017 Electrons in universe Theory. Definition is broad and robust; huge gulf between polynomial and exponential algorithms. Number Supercomputer instructions per second 1079 Estimated Will not help solve 1,000 city TSP problem via brute force >> >> 1079 " 1017 " 1013 Practice. Exponents and constants of polynomials that arise are small scales to huge problems. 3 4

2 P Extended Church-Turing Thesis Def. P is the set of all yes-no problems solvable in poly-time on a deterministic Turing machine. Problem Description Algorithm Yes No Extended Church-Turing thesis. P = yes-no problems solvable in poly-time in this universe. If computable by a piece of hardware in time T(N) for input of size N, then computable by TM in time (T(N)) k for some constant k. RELPRIME COMPOSITE Are x and y relatively prime? Does x have a factor other than 1 and itself? Euclid (300 BCE) Agarwal-Kayal- Saxena (2002) 34, , Evidence supporting thesis. True for all physical computers. k = 2 for random access machines. [your laptop] EDIT- DISTANCE LSOLVE Is the edit distance between strings x and y less than 5? Is there a vector x that satisfies Ax = b? Dynamic Programming Gauss-Edmonds elimination niether neither # 0 1 1& % ( 2 4 "2 % ( $ % ' (, # 4& % ( % 2 ( $ % 36' ( acgggt ttttta " 1 0 0% $ ' $ ' # $ 0 1 1& ', " 1% $ ' $ 1 ' # $ 1& ' Implication. To make future computers more efficient, only need to focus on improving implementation of existing designs. Law of physics? A new constraint on what is possible? Remark. Algorithm typically also solves the related search problem. Possible counterexample? Quantum computers. like 2nd law of thermodynamics Satisfiability Intractability of Literal. A Boolean variable or its negation. Clause. A disjunction of 3 distinct literals. x i or x i C j = x 1 " x 2 " x 3 Q. How to solve an instance of. A1. Try all 2 n truth assignments. A2. Do something substantially more clever??? Conjunctive normal form. A propositional formula # that is the conjunction of clauses. " = C 1 # C 2 # C 3 # C 4 # C 5 Conjecture. No poly-time algorithm for. "intractable". Given a CNF formula # consisting of k clauses over n literals, does it have a satisfying truth assignment? ( x 1 " x 2 " x 3 ) # ( x 1 " x 2 " x 3 ) # ( x 1 " x 2 " x 3 ) # ( x 1 " x 2 " x 4 ) # ( x 2 " x 3 " x 4 ) x 1 = true, x 2 = true, x 3 = false, x 4 = true a satisfiable formula with 5 clauses and 4 variables 7 8

3 Reductions Planar 3-Color Q. Which problems won't we be able to solve in practice? A. No easy answers, but theory helps. Given a planar map, can it be colored using 3 colors so that no adjacent regions have the same color? Def. Problem X polynomial reduces to problem Y if arbitrary instances of problem X can be solved using: Polynomial number of standard computational steps, plus One call to subroutine for Y as last step. Consequence. If no poly-time algorithm for X, then no poly-time algorithm for Y. your research problem YES instance Planar 3-Color Given a planar map, can it be colored using 3 colors so that no adjacent regions have the same color?. Given a graph (need not be planar), is there a way to color the vertices red, green, and blue so that no adjacent vertices have the same color? yes instance NO instance. Applications. Register allocation, Potts model in physics, 11 12

4 . Given a graph, is there a way to color the vertices red, green, and blue so that no adjacent vertices have the same color? Claim. polynomial reduces to. Pf. Given instance #, we construct an instance of that is 3-colorable iff # is satisfiable. Construction. i. Create one vertex for each literal. ii. Create 3 new vertices T, F, and B; connect them in a triangle, and connect each literal to B. iii. Connect each literal to its negation. iv. For each clause, attach a gadget of 6 vertices and 13 edges. to be described next Claim. Graph is 3-colorable iff # is satisfiable. Claim. Graph is 3-colorable iff # is satisfiable. Pf. Suppose graph is 3-colorable. Consider assignment that sets all T-colored literals to true. (ii) ensures each literal is same color as T or F. (iii) ensures a literal and its negation are opposites. Pf. Suppose graph is 3-colorable. Consider assignment that sets all T-colored literals to true. (ii) ensures each literal is same color as T or F. (iii) ensures a literal and its negation are opposites. (iv) ensures at least one literal in each clause is same color as T. true T false F B B base x 1 x 2 x 3 C i = x 1 V x 2 V x 3 6-node gadget x 1 x x 1 2 x 2 x 3 x 3 x n x n true T F false 15 16

5 Claim. Graph is 3-colorable iff # is satisfiable. Claim. Graph is 3-colorable iff # is satisfiable. Pf. Suppose graph is 3-colorable. Consider assignment that sets all T-colored literals to true. (ii) ensures each literal is same color as T or F. (iii) ensures a literal and its negation are opposites. (iv) ensures at least one literal in each clause is same color as T. Pf. $ Suppose formula # is satisfiable. Color all true literals same color as T. Color vertex below green node F, and vertex below that B. Color remaining middle row vertices B. Color remaining bottom vertices T or F as forced. B not 3-colorable if all are red B a literal set to true in assignment x 1 x 2 x 3 C i = x 1 V x 2 V x 3 x 1 x 2 x 3 C i = x 1 V x 2 V x 3 6-node gadget 6-node gadget contradiction true T F false true T F false More Poly-Time Reductions Still More Hard Computational Problems reduces to Aerospace engineering. Optimal mesh partitioning for finite elements. Biology. Protein folding. Chemical engineering. Heat exchanger network synthesis. Chemistry. Chemical synthesizability. 3DM VERTEX COVER Dick Karp Turing award (1985) Civil engineering. Equilibrium of urban traffic flow. Economics. Computation of arbitrage in financial markets with friction. Electrical engineering. VLSI layout. Environmental engineering. Optimal placement of contaminant sensors. EXACT COVER PLANAR- CLIQUE HAM-CYCLE Financial engineering. Find minimum risk portfolio of given return. Game theory. Find Nash equilibrium that maximizes social welfare. Genomics. Phylogeny reconstruction. Mathematics. Given integer a 1,, a n, compute SUBSET-SUM INDEPENDENT SET TSP HAM-PATH Mechanical engineering. Structure of turbulence in sheared flows. Medicine. Reconstructing 3-D shape from biplane angiocardiogram. Operations research. Traveling salesperson problem. Physics. Partition function of 3-D Ising model in statistical mechanics. PARTITION INTEGER PROGRAMMING Politics. Shapley-Shubik voting power. Pop culture. Minesweeper consistency. Statistics. Optimal experimental design. KNAPSACK BIN-PACKING Conjecture: no poly-time algorithm for. (and hence none of these problems) 6,000+ scientific papers per year

6 Implications of Intractability Cook's Theorem Proving a problem intractable guide scientific inquiry. 1926: Ising introduces simple model for phase transitions. 1944: Onsager find closed form solution to 2D case in tour de force. 19xx: Feynman and other top minds seek 3D solution. a holy grail of statistical physics reduces to 3DM VERTEX COVER Stephen Cook Turing award (1982) 2000: Istrail reduces to 3D version of problem. EXACT COVER PLANAR- CLIQUE HAM-CYCLE search for closed formula appears doomed SUBSET-SUM INDEPENDENT SET TSP HAM-PATH PARTITION INTEGER PROGRAMMING KNAPSACK BIN-PACKING All of these problems (any many more) polynomial reduce to Cook + Karp Complexity Classes reduces to reduces to 3DM VERTEX COVER P. Set of yes-no problems solvable in poly-time on a deterministic TM. EXP. Same as P, but in exponential-time. NP. Same as P, but on non-deterministic Turing machine. Cook-Levin Theorem (1960s). ALL NP problems reduce to. EXACT COVER PLANAR- CLIQUE HAM-CYCLE if we can solve, we can solve any of them NP-complete. All NP problems to which reduces. SUBSET-SUM PARTITION INTEGER PROGRAMMING INDEPENDENT SET TSP HAM-PATH Implications. If efficient algorithm for, then P = NP. if we can solve any of them, we can solve If efficient algorithm for any NP-complete problem, then P = NP. If no efficient algorithm for some NP problem, then none for. KNAPSACK BIN-PACKING All of these problems are different manifestations of one "really hard" problem

7 Certificates Certificates Alternate (and Equivalent) Definition. NP is set of all yes-no problems for which you can check in poly-time that it is a yes instance (given a certificate).. Given a graph, is there a way to color the vertices red, green, and blue so that no adjacent vertices have the same color? Alt. Def. NP is set of all yes-no problems for which you can check in poly-time that it is a yes instance (given a certificate). FACTOR. Given two integers x and y, does x have a factor between 2 and y? x = 2773 y = 50 x = y = instance instance certificate certificate instance certificate The Main Question Implications of NP-Completeness Does P = NP? Is there a poly-time algorithm for SAT? Does nondeterminism help you solve problems faster? Clay $1 million prize. Classify problems according to their computational requirements. NP-complete: SAT, all Karp problems, thousands more. P: RELPRIME, COMPOSITE, LSOLVE. Unclassified: FACTOR is in NP, but unknown if NP-complete or in P. Jack Edmonds, 1962 Computational universality. EXP P NP NPcomplete EXP P = NP All known algorithms for NP-complete problems are exponential. If any NP-complete problem proved exponential, so are rest. If any NP-complete problem proved polynomial, so are rest. If P % NP If P = NP If yes: Efficient algorithms for, TSP, FACTOR,... If no: No efficient algorithms possible for, TSP,... Consensus opinion on P = NP? Probably no. would break modern cryptography and collapse economy Proving a problem is NP-complete can guide scientific inquiry. 1926: Ising introduces simple model for phase transitions. 1944: Onsager solves 2D case in tour de force. 19xx: Feynman and other top minds seek 3D solution. 2000: Istrail proves 3D problem NP-complete

8 Coping With Intractability Coping With Intractability Relax one of desired features. Solve the problem in poly-time. Solve the problem to optimality. Solve arbitrary instances of the problem. Relax one of desired features. Solve the problem in poly-time. Solve the problem to optimality. Solve arbitrary instances of the problem. Complexity theory deals with worst case behavior. Instance(s) you want to solve may be "easy." Concorde algorithm solved 13,509 US city TSP problem. Develop a heuristic, and hope it produces a good solution. No guarantees on quality of solution. Ex: TSP assignment heuristics. Ex: Metropolis algorithm, simulating annealing, genetic algorithms. Design an approximation algorithm. Guarantees to find a nearly-optimal solution. Ex: Euclidean TSP tour guaranteed to be within 1% of optimal. Active area of research, but not always possible Sanjeev Arora (1997) Coping With Intractability Summary Relax one of desired features. Solve the problem in poly-time. Solve the problem to optimality. Solve arbitrary instances of the problem. Exploit intractability. Cryptography. [stay tuned] Keep trying to prove P = NP. can do any 2 of 3 Many fundamental problems are intractable. TSP,,. 3D Ising model. Theory says: we probably won't be able to design efficient algorithms for them. You will encounter such problems in your scientific lives. If you know about intractability, you can identify them and avoid wasting time and energy