10.3: Intractability. Overview. Exponential Growth. Properties of Algorithms. What is an algorithm? Turing machine.

Transcription

1 Overview 10.3: Intractability What is an algorithm? Turing machine. What problems can be solved on a computer? Computability. What ALGORITHMS will be useful in practice? Analysis of algorithms. Which PROBLEMS can be solved in practice? Intractability. Princeton CS Building West Wall, Circa 2001 Introduction to Computer Science Robert Sedgewick and Kevin Wayne 2 Properties of Algorithms Exponential Growth Which ALGORITHMS are useful in practice? A broad and robust definition: (Cobham 1960s, Jack Edmonds, 1962) Efficient = polynomial time for ALL inputs. Ex: mergesort requires at most N log 2 N steps. Ex: insertion sort requires at most N 2 / 2 steps. Exponential growth dwarfs technological change. Suppose each electron in the universe had power of today's supercomputers... And each works for the life of the universe in an effort to solve TSP problem via brute force N! algorithm. Inefficient = "exponential time" for SOME inputs. Ex: brute force TSP takes N! > 2 N steps. In practice, huge gulf between polynomial and exponential algorithms. Quantity Supercomputer instructions per second Age of universe in seconds Number Electrons in universe Estimated Will not help solve 1,000 city TSP problem. 1000! >> >>

2 Exponential Growth P Ex: sociology thesis proposal in Every year since 1950, the number of American children gunned down has doubled. What do YOU think? Which PROBLEMS will we be able to solve in practice? Those with efficient (polynomial-time) algorithms. Definition of P: Set of all decision problems solvable in polynomial time on a deterministic Turing machine. Problem Description Algorithm Yes No RELPRIME Are the integers x and y relatively prime? Euclid (300 BCE) COMPOSITE Does the integer x have a factor other than 1 and x? Agarwal-Kayal- Saxena (2002) EDIT- DISTANCE Is x closer to y or z in terms of edit distance? Dynamic Programming niether neither another niether another neither BACON Is the Kevin Bacon number of x less than 6? Breadth First Search Julia Roberts Akbar Abdi Some problems in P 5 7 Extended Church-Turing Thesis Properties of Problems Extended Church-Turing thesis: P = decision problems solvable in polynomial time on real computers. If function is computable by a piece of hardware in time T(n) for input of size n, then computable by TM in time (T(n)) k for some k. Evidence supporting thesis: True for all physical computers. k = 2 for random access machines. Which PROBLEMS won't we be able to solve in practice? No easy answers, but theory helps. Two hard problems. Factorization: Given an integer, find its prime factorization = Implication: to make future computers more efficient, only need to focus on improving implementation of existing designs. CIRCUIT-SAT: Is there a way to assign inputs to a given combinational circuit that makes its output true? Possible exception: quantum computers. Schor's factoring algorithm runs in poly-time on quantum computer. No algorithm currently known for classical computers. YES NO 8 9

3 More Hard Problems Reduction More hard computational problems. Biology: protein folding. Chemical engineering: heat exchanger network synthesis. Civil engineering: equilibrium of urban traffic flow. Economics: computation of arbitrage in financial markets with friction. Financial engineering: find minimum risk portfolio of given return. Genomics: phylogeny reconstruction, sequence assembly. Electrical engineering: VLSI layout. Mechanical engineering: structure of turbulence in sheared flows. Medicine: reconstructing 3-D shape from biplane angiocardiogram. Operations research: optimal resource allocation. Physics: anti-ferromagnetic Potts model. Politics: Shapley-Shubik voting power. Pop culture: Minesweeper consistency, playing optimal Tetris. Statistics: optimal experimental design. Problem X reduces to problem Y if given an efficient subroutine for Y, you can devise an efficient algorithm for X. Cost of solving X = cost of solving Y + cost of reduction. May call subroutine for Y more than once. Consequences: Classify problems: establish relative difficulty between two problems. Design algorithms: given algorithm for Y, can also solve X. Establish intractability: if X is hard, then so is Y. Example. X = COMPOSITE. Y = Factorization. static boolean iscomposite (int x) { int[] factors = factorize(x); return (factors.length > 1); } How can we justify our belief that these problems are difficult? More Reductions Some Hard Problems CIRCUIT-SAT reduces to 3SAT CIRCUIT-SAT TSP: What is the shortest tour that visits all N cities? 3SAT GRAPH Dick Karp (1972) 3DM VERTEX EXACT PLANAR HAMILTONIAN CIRCUIT CLIQUE SUBSET-SUM TSP INDEPENDENT SET PARTITION INTEGER PROGRAMMING KNAPSACK 12 13

4 Some Hard Problems Some Hard Problems PLANAR-: Given a planar map, can it be colored using 3 colors so that no adjacent regions have the same color? PLANAR-: Given a planar map, can it be colored using 3 colors so that no adjacent regions have the same color? YES instance. NO instance Minesweeper Consistency Problem Minesweeper Consistency Problem Minesweeper. Start: Blank grid of squares, some of which conceal mines. Goal: Find location of all mines without detonating any. Choose a square. if mine underneath, it detonates and you lose otherwise, computer tells you # mines in neighboring squares Repeat. MINESWEEPER: Given a state of what purports to be a N-by-N Minesweeper game, is it logically consistent? YES NO 16 17

5 Reduction Minesweeper Consistency Problem Problem X reduces to problem Y if given an efficient subroutine for Y, you can devise an efficient algorithm for X. Cost of solving X = cost of solving Y + cost of reduction. May call subroutine for Y more than once. Claim. SAT reduces to MINESWEEPER. Build circuit by laying out appropriate minesweeper configurations. Minesweeper game is consistent if and only if circuit is satisfiable. Consequences: Classify problems: establish relative difficulty between two problems. Design algorithms: given algorithm for Y, can also solve X. Establish intractability: if X is hard, then so is Y. A Minesweeper Wire Example. X = CIRCUIT-SAT. Y = MINESWEEPER. Exactly one of x and x' is a bomb. A Minesweeper NOT Gate Minesweeper Consistency Problem One More Hard Problem Claim. SAT reduces to MINESWEEPER. CIRCUIT-SAT Build circuit by laying out appropriate minesweeper configurations. CIRCUIT-SAT reduces to 3SAT Minesweeper game is consistent if and only if circuit is satisfiable. 3SAT MINE- SWEEPER GRAPH 3DM VERTEX EXACT PLANAR HAMILTONIAN CIRCUIT CLIQUE SUBSET-SUM A Minesweeper AND Gate TSP INDEPENDENT SET PARTITION KNAPSACK INTEGER PROGRAMMING 20 21

6 Nondeterminism P and NP Nondeterministic machine. One that can GUESS the right answer and CHECK that the guessed answer is correct. P: Set of decision problems solvable in polynomial time on a deterministic Turing machine. Ex 1: COMPOSITE. Guess a factor p of x. Computer must check that p divides x. Input: x = 437,669 Guess: p = 809 NP: Set of decision problems solvable in polynomial time on a nondeterministic Turing machine. if we can solve SAT, we can solve any of them Ex 2: MINESWEEPER. Guess an assignment of mines to squares. Computer must check that each square adjacent to the required number of mines. Checking seems easy compared to solving from scratch. Is it really? running time = # steps to check Input Guess Cook-Levin Theorem (1960s). ALL problems in NP reduce to SAT. If efficient algorithm for SAT, then P = NP. Intellectual milestone in CS. Proof idea: Given problem X in NP, consider nondeterministic TM that solves X in nondeterministic polynomial time. For each time step t, use Boolean variables to model which symbol occupies cell i, location of read head, state of DFA, etc. Use logic gates to ensure TM makes legal moves, etc. SAT instance is satisfiable if and only if TM outputs YES. Stephen Cook Cook's Theorem: Implications The Main Question CIRCUIT-SAT reduces to 3SAT CIRCUIT-SAT All of these problems reduce to each other. Does P = NP? Can problems like SAT be solved in a polynomial number of steps? 3SAT GRAPH Is the decision problem as easy as checking a proposed solution? Does nondeterminism help you solve problems faster? Most important open problem in computer science. Clay Foundation $1 million prize. 3DM VERTEX EXACT PLANAR Jack Edmonds, 1962 HAMILTONIAN CIRCUIT CLIQUE SUBSET-SUM EXP P NP EXP P = NP TSP INDEPENDENT SET PARTITION INTEGER PROGRAMMING If P NP If P = NP KNAPSACK 24 25

7 Does P = NP? Does P = NP? If yes, then efficient algorithms for, TSP, FACTOR. If no, then can't hope to write efficient algorithm for TSP. What is the consensus opinion on P = NP? would break modern cryptography and collapse ecommerce and banking system Probably no, since thousands of researchers have spent four decades in search of polynomial algorithms without success. But yes is possible since no success in proving P NP either. NP-Complete NP-complete: A problem in NP is NP-complete if SAT reduces to it. Ex: SAT, MINESWEEPER, all Karp problems, thousands more. Note: FACTOR is in NP, but not known to be NP-complete. Links together a huge and diverse number of fundamental problems: Efficient algorithms for all Karp problems or none of them. If efficient algorithm for, then efficient algorithm for: any other NP-complete problem, including all Karp problems any other problem in NP, including FACTOR no one has found a way to reduce SAT to FACTOR Notorious complexity class. All known algorithms for these problems are exponential. Unlikely that they can be solved given limited computing resources. Called intractable Coping With Intractability Coping With Intractability Relax one of desired features. Solve the problem in polynomial time. Solve the problem to optimality. Solve arbitrary instances of the problem. Relax one of desired features. Solve the problem in polynomial time. Solve the problem to optimality. Solve arbitrary instances of the problem. Complexity theory deals with worst case behavior. Instance(s) you want to solve may be "easy." Concorde algorithm solved 13,509 US city TSP problem. Develop a heuristic, and hope it produces a good solution. TSP assignment. Metropolis algorithm, simulating annealing, genetic algorithms. Design an approximation algorithm: algorithm that is guaranteed to find a high-quality solution in polynomial time. Active area of research, but not always possible! Euclidean TSP tour within 1% of optimal. (Cook et. al., 1998) Sanjeev Arora (1997) 33 34

8 Coping With Intractability Summary Relax one of desired features. Solve the problem in polynomial time. Solve the problem to optimality. Solve arbitrary instances of the problem. Exploit intractability. Cryptography. see next lecture Keep trying to prove P = NP. can do any 2 of 3 Many fundamental problems are NP-complete. TSP, CIRCUIT-SAT,. Theory says we probably won't be able to design efficient algorithms for NP-complete problems. You will surely run into these problems in your scientific life. If you know about NP-completeness, you can identify them and avoid wasting time and energy