Theory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death Robbie Hott www.cs.virginia.edu/~jh2jf Department of Computer Science University of Virginia www.cs.virginia.edu/~njb2b/theory
Not finitely describable Not Recognizable The Extended Chomsky Hierarchy Recognizable EXPSPACE-complete =RE EXPTIME-complete Go PSPACE-complete QBF NP-complete SAT 2 S*? H H* H Turing degrees Decidable Presburger arithmetic EXPSPACE EXPTIME PSPACE Context sensitive LBA NP P a n b n c n Context-free ww R Det. CF a n b n Regular a* Finite {a,b}
Computability Complexity Undecidable Decidable Tractable Intractable ~1800s 1960s 1900: Hilbert s Problems 1936: Turing s Computable Numbers 1957: Chomsky s Syntactic Structures (Mostly) Dead field 1960s 2150? 1960s: Hartmanis and Stearns: Complexity class 1971: Cook/Levin, Karp: P=NP? 1976: Knuth s O, Ω, Θ Very Open and Alive
Resource-Bounded Computation Previously: can something be done? Now: how efficiently can it be done? Goal: conserve computational resources: Time, space, other resources?
Resource-Bounded Computation Def: L is decidable within time O(t(n)) if some TM M that decides L always halts on all w * within O(t( w )) steps / time. Def: L is decidable within space O(s(n)) if some TM M that decides L always halts on all w * while never using more than O(s( w )) space / tape cells.
Complexity Classes Def: DTIME(t(n))={L L is decidable within time O(t(n)) by some deterministic TM} Def: NTIME(t(n))={L L is decidable within time O(t(n)) by some non-deterministic TM} Def: DSPACE(s(n))={L L decidable within space O(s(n)) by some deterministic TM} Def: NSPACE(s(n))={L L decidable within space O(s(n)) by some non-deterministic TM}
Examples of Space & Time Usage Let L 1 ={0 n 1 n n>0}: For 1-tape TM s: L 1 DTIME(n 2 ) L 1 DSPACE(n) L 1 DTIME(n log n) For 2-tape TM s: L 1 DTIME(n) L 1 DSPACE(log n)
Examples of Space & Time Usage Let L 2 =S* L 2 DTIME(n) Theorem: every regular language is in DTIME(n) L 2 DSPACE(1) Theorem: every regular language is in DSPACE(1) L 2 DTIME(1) Let L 3 ={w$w w in S*} L 3 DTIME(n 2 ) L 3 DSPACE(n) L 3 DSPACE(log n)
Multi-Tape vs. One-Tape TM Is one more powerful than the other?
Space is Tape Independent Theorem: The space does not depend on the # tapes. Proof: 1 1 0 1 0 1 0 1 1 0 0 1 1 1 1 0 0 Idea: Tapes can be interlaced space-efficiently: Note: This does not asymptotically increase the overall space (but can increase the total time). Theorem: A 1-tape TM can simulate a t(n)-time- bounded k-tape TM in time O(k t 2 (n)).
Multi-Tape vs. One-Tape TM Are there problems that are in TIME(t(n)) for a multi-tape TM, but not in TIME(t(n)) for a one-tape TM?
Copy Input Problem Input: w, a string of N bits Output: ww Obvious multi-tape algorithm that involves 2N steps: N steps: walk over the input, copying it to the second tape N steps: continue to move right, copying the second tape contents onto the input tape after the input Best (?) single-tape algorithm that involves ~2N 2 steps: N iterations: move over the input, marking each symbol N steps: move to the first non-blank square, write that symbol N steps: move back to the rightmost marked input symbol Intuitively is seems impossible to do much better, but hard to prove!
Theory is about Big Questions If little tweaks to our model change the answers, we might as well focus on answering the practical questions for a real system and specific problem instance instead.
Making things Robustier? Find a more robust computing model than a TM Church-Turing thesis says all mechanical models are equivalent (computing power) to a TM But, this doesn t mean there might not be better models for complexity Make the complexity classes bigger Define a complexity class big enough so the little tweaks to TMs do not change the answers
Complexity Class P P = U k DTIME(n k ) P is the class of languages that can be decided in Polynomial Time on a deterministic, singletape Turing machine.
a) TIME(n 2 ) b) TIME(O(n 7 )) c) TIME(O(2 n )) Classes in P d) Class of languages that can be decided in Polynomial Time by a 2-tape TM e) Class of languages that can be decided in Polynomial Time by a nondeterministic TM Unknown! This is the P = NP question. Focus of next class Yes! We can simulate each step of a 2- tape TM by making 2 passes over the whole tape ~ 2(n+t(n)) (See Theorem 7.8)
P Examples What are some examples of problems in P?
Traveling Salesperson Problem You have to visit n cities You want to make the shortest trip How could you do this? What if you had a machine that could guess?
Non-deterministic polynomial time Deterministic Polynomial Time: The TM takes at most O(n c ) steps to accept a string of length n Non-deterministic Polynomial Time: The TM takes at most O(n c ) steps on each computation path to accept a string of length n
Complexity Class NP NP = U k NTIME(n k ) P is the class of languages that can be decided in Polynomial Time on a non-deterministic, single-tape Turing machine.
Properties of NP Complexity Class NP Easy to verify an answer (verifier in P) Hard to compute an answer
Subset Sum Problem NP Example Given a set of integers, determine if a subset sums to 0 Example: {-7, -3, -2, 5, 8, 11, 15} Verify that {-3, -2, 5} is a solution
NP Example? Traveling Salesperson Problem (decision version) Given a list of n cities to visit and a distance k, is there a route to visit all cities with distance less than k?
NP Example Traveling Salesperson Problem (decision version) Even the simpler version is in NP Verifier: given an ordered list of cities, sum up distance and check < k Solver: Non-determinism: at each city, guess the next correctly. After visiting all, verify cost less than k ( linear algorithm ) Deterministic: check all possible routes ( exponential time )
NP Example? Traveling Salesperson Problem (decision version) Given a list of n cities to visit and a distance k, is there a route to visit all cities with distance less than k?
The Class P and the Class NP P = { L L is accepted by a deterministic Turing Machine in polynomial time } NP = { L L is accepted by a non-deterministic Turing Machine in polynomial time } They are sets of languages
P vs NP? Are non-deterministic Turing machines really more powerful (efficient) than deterministic ones? Essence of P vs NP problem
Does Non-Determinism matter? Finite Automata? No! DFA NFA Push Down Automata? Yes! DFA not NFA (PDA)
?? Perelman 2006
Progress P = NP if every NP problem has a deterministic polynomial algorithm We could find an algorithm for every NP problem Seems hard We could use polynomial time reductions to find the hardest problems and just work on those
Real world examples: Reductions Finding your way around the city reduces to reading a map Traveling from Richmond to Cville reduces to driving a car Other suggestions?
Polynomial time reductions PARTITION = { n 1,n 2, n k we can split the integers into two sets which sum to half } SUBSET-SUM = { <n 1,n 2, n k,m> there exists a subset which sums to m } 1) If I can solve SUBSET-SUM, how can I use that to solve an instance of PARTITION? 2) If I can solve PARTITION, how can I use that to solve an instance of SUBSET-SUM?
Polynomial Reductions 1) Partition REDUCES to Subset-Sum Partition < p Subset-Sum 2) Subset-Sum REDUCES to Partition Subset-Sum < p Partition Therefore they are equivalently hard
How long does the reduction take? How could you take advantage of an exponential time reduction?
NP-Completeness How would you define NP-Complete? They are the hardest problems in NP NP-Complete P NP
Definition of NP-Complete Q is an NP-Complete problem if: 1) Q is in NP 2) every other NP problem polynomial time reducible to Q
Getting Started How do you show that EVERY NP problem reduces to Q? One way would be to already have an NP-Complete problem and just reduce from that P 1 P 2 P 3 P 4 Mystery Complete Problem NP- Q
SAT SAT = { f f is a Boolean Formula with a satisfying assignment } Is SAT in NP?
Cook-Levin Theorem (1971) SAT is NP-Complete If you want to see the proof it is Theorem 7.37 in Sipser (assigned reading!) or you can take CS 660 Graduate Theory. You are not responsible for knowing the proof.
3-SAT 3-SAT = { f f is in Conjunctive Normal Form, each clause has exactly 3 literals and f is satisfiable } 3-SAT is NP-Complete (2-SAT is in P)
NP-Complete To prove a problem is NP-Complete show a polynomial time reduction from 3-SAT Other NP-Complete Problems: PARTITION SUBSET-SUM CLIQUE HAMILTONIAN PATH (TSP) GRAPH COLORING MINESWEEPER (and many more)
NP-Completeness Proof Method To show that Q is NP-Complete: 1) Show that Q is in NP 2) Pick an instance, R, of your favorite NP- Complete problem (ex: Φ in 3-SAT) 3) Show a polynomial algorithm to transform R into an instance of Q
Savitch s Theorem Theorem: NSPACE(f(n)) DSPACE (f 2 (n)) Walter Savitch Proof: Simulation: idea is to aggressively conserve and reuse space while sacrificing (lots of) time. Consider a sequence of TM states in one branch of an NSPACE(f(n))-bounded computation: Computation time / length is bounded by c f(n) (why?) We need to simulate this branch and all others too! Q: How can we space-efficiently simulate these?
Verify it is a valid intermediate state by recursively solving both subproblems. Iterate for all possible midpoint states! The recursion stack depth is at most log(c f(n) )=O(f(n)) Each recursion stack frame size is O(f(n)). total space needed is O(f(n)*f(n))=O(f 2 (n)) Note: total time is exponential (but that s OK). Savitch s Theorem Pick a midpoint state along target path: Walter Savitch
Savitch s Theorem Corollary: NPSPACE = PSPACE Walter Savitch Proof: NPSPACE = NSPACE(n k ) k>1 DSPACE(n 2k ) k>1 = DSPACE(n k ) k>1 = PSPACE i.e., polynomial space is invariant with respect to