15-453 TIME COMPLEXITY AND POLYNOMIAL TIME; FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY NON DETERMINISTIC TURING MACHINES AND NP THURSDAY Mar 20 COMPLEXITY THEORY Studies what can and can t be computed under limited resources such as time, space, etc Today: Time complexity MEASURING TIME COMPLEXITY We measure time complexity by counting the elementary steps required or a machine to halt Consider the language A = { 0 k 1 k k 0 } On input o length n: 1. Scan across the tape and reject i the ~n string is not o the orm 0 i 1 j ~n 2 ~n 2. Repeat the ollowing i both 0s and 1s remain on the tape: Scan across the tape, crossing o a single 0 and a single 1 3. I 0s remain ater all 1s have been crossed o, or vice-versa, reject. Otherwise accept. Deinition: Let M be a TM that halts on all inputs. The running time or timecomplexity o M is the unction : N N, where (n) is the maximum number o steps that M uses on any input o length n. ASYMPTOTIC ANALYSIS 5n 3 + 2n 2 + 22n + 6 = O(n 3 ) 1
BIG-O Let and g be two unctions, g : N R +. We say that (n) = O(g(n)) i there exist positive integers c and n 0 so that or every integer n n 0 (n) cg(n) When (n) = O(g(n)), we say that g(n) is an asymptotic upper bound or (n) asymptotically NO MORE THAN g 5n 3 + 2n 2 + 22n + 6 = O(n 3 ) I c = 6 and n 0 = 10, then 5n 3 + 2n 2 + 22n + 6 cn 3 2n 4.1 + 200283n 4 + 2 = 3nlog 2 n + 5n log 2 log 2 n = nlog 10 n 78 = log 10 n = log 2 n / log 2 10 O(nlog 2 n) = O(nlog 10 n) = O(nlog n) A = { 0 k 1 k k 0 } TIME(nlog n) Deinition: TIME(t(n)) = { L L is a language decided by a O(t(n)) time Turing Machine } A = { 0 k 1 k k 0 } TIME(n 2 ) Cross o every other 0 and every other 1. I the # o 0s and 1s let on the tape is odd, reject 00000000000001111111111111 x0x0x0x0x0x0xx1x1x1x1x1x1x xxx0xxx0xxx0xxxx1xxx1xxx1x xxxxxxx0xxxxxxxxxxxx1xxxxx xxxxxxxxxxxxxxxxxxxxxxxxxx We can prove that a TM cannot decide A in less time than O(nlog n) *7.49 Extra Credit. Let (n) = o(nlogn). Then Time((n)) contains only regular languages. Can A = { 0 k 1 k k 0 } be decided in time O(n) with a two-tape TM? Scan all 0s and copy them to the second tape. Scan all 1s, crossing o a 0 rom the second tape or each 1. where (n) = o(g(n)) i lim n-> (n)/g(n) = 0 ie, or all c >0, n 0 such that (n) < cg(n) or all n n 0 asymptotically LESS THAN g 2
Dierent models o computation yield dierent running times or the same language! Theorem: Let t(n) be a unction such that t(n) n. Then every t(n)-time multi-tape TM has an equivalent O(t(n) 2 ) single tape TM Claim: Simulating each step in the multitape machine uses at most O(t(n)) steps on a single-tape machine. Hence total time o simulation is O(t(n) 2 ). Theorem: Every Multitape Turing Machine can be transormed into a single tape Turing Machine Theorem: Every Multitape Turing Machine can be transormed into a single tape Turing Machine 1 0 0 1 0 0 FINITE STATE CONTROL FINITE STATE CONTROL FINITE STATE CONTROL... 1 0 0 # # # FINITE STATE CONTROL... 1 0 0 # # # Theorem: Let t(n) be a unction such that t(n) n. Then every t(n)-time multi-tape TM has an equivalent O(t(n) 2 ) single tape TM Analysis: (Note, k, the # o tapes, is ixed.) Let S be simulator Put S s tape in proper ormat: O(n) steps Two scans to simulate one step, 1. to optain ino or next move O(t(n)) steps, why? 2. to simulate it (may need to shit everything over to right possibly k times): O(t(n)) steps, why? P = TIME(n k ) k N Thereore, O(n) + t(n) O (t(n)) = O(t (n) 2 ) steps in simulation, 3
NON-DETERMINISTIC TURING MACHINES AND NP read write move 0 0, R 0 0, R, R q accept, R 0 0, R q reject Deinition: A Non-Deterministic TM is a 7-tuple T = (Q, Σ, Γ,, q 0, q accept, q reject ), where: Q is a inite set o states Σ is the input alphabet, where Σ Γ is the tape alphabet, where Γ and Σ Γ : Q Γ 2 (Q Γ {L,R}) q 0 Q is the start state q accept Q is the accept state NON-DETERMINISTIC TMs are just like standard TMs, except: 1. The machine may proceed according to several possibilities 2. The machine accepts a string i there exists a path rom start coniguration to an accepting coniguration q reject Q is the reject state, and q reject q accept Deterministic Computation Non-Deterministic Computation Deterministic Computation Non-Deterministic Computation reject reject accept or reject accept accept or reject accept Deinition: Let M be a NTM that is a decider (all branches halt on all inputs). The running time or time-complexity o M is the unction : N N, where (n) is the maximum number o steps that M uses on any branch o its computation on any input o length n. Theorem: Let t(n) be a unction such that t(n) n. Then every t(n)-time nondeterministic single-tape TM has an equivalent 2 O(t(n)) deterministic single tape TM 4
BOOLEAN FORMULAS Deinition: NTIME(t(n)) = { L L is decided by a O(t(n))-time non-deterministic Turing machine } TIME(t(n)) NTIME(t(n)) logical operations = ( x y) z variables parentheses A satisying assignment is a setting o the variables that makes the ormula true x = 1, y = 1, z = 1 is a satisying assignment or A Boolean ormula is satisiable i there exists a satisying assignment or it A 3cn-ormula is o the orm: (x 1 x 2 x 3 ) (x 4 x 2 x 5 ) (x 3 x 2 x 1 ) YES a b c d literals clauses NO (x y) x YES NO (x 1 x 2 x 1 ) (x 3 x 1 ) (x 3 x 2 x 1 ) SAT = { is a satisiable Boolean ormula } NO NO (x 1 x 2 x 3 ) ( x 4 x 2 x 1 ) (x 3 x 1 x 1 ) (x 1 x 2 x 3 ) (x 3 x 2 x 1 ) 3SAT = { is a satisiable 3cn-ormula } 3SAT = { is a satisiable 3cn-ormula } Theorem: 3SAT NTIME(n 2 ) On input : 1. Check i the ormula is in 3cn 2. For each variable, non-deterministically substitute it with 0 or 1 ( x y x ) NP = NTIME(n k ) k N ( 0 y 0 ) ( 1 y 1 ) ( 0 0 0 ) ( 0 1 0 ) 3. Test i the assignment satisies 5
Theorem: L NP i there exists a poly-time Turing machine V(eriier) with L = { x y(witness) y = poly( x ) and V(x,y) accepts } Proo: (1) I L = { x y y = poly( x ) and V(x,y) accepts } then L NP (2) I L NP then L = { x y y = poly( x ) and V(x,y) accepts } Theorem: L NP i there exists a poly-time Turing machine V(eriier) with L = { x y(witness) y = poly( x ) and V(x,y) accepts } Proo: (1) I L = { x y y = poly( x ) and V(x,y) accepts } then L NP Because we can guess y and then run V (2) I L NP then L = { x y y = poly( x ) and V(x,y) accepts } Theorem: L NP i there exists a poly-time Turing machine V(eriier) with L = { x y(witness) y = poly( x ) and V(x,y) accepts } Proo: (1) I L = { x y y = poly( x ) and V(x,y) accepts } then L NP Because we can guess y and then run V (2) I L NP then L = { x y y = poly( x ) and V(x,y) accepts } Let N be a non-deterministic poly-time TM that decides L and deine V(x,y) to accept i y is an accepting computation history o N on x 3SAT = { y such that y is a satisying assignment to and is in 3cn } SAT = { y such that y is a satisying assignment to } HAMILTONIAN PATHS A language is in NP i and only i there exist polynomial-length certiicates* or membership to the language SAT is in NP because a satisying assignment is a polynomial-length certiicate that a ormula is satisiable i b a e c d h g * that can be veriied in poly-time 6
HAMILTONIAN PATHS b e g HAMPATH = { (G,s,t) G is a directed graph with a Hamiltonian path rom s to t } Theorem: HAMPATH NP a d The Hamilton path itsel is a certiicate i c h K-CLIQUES K-CLIQUES a d a d c c b e g b e g CLIQUE = { (G,k) G is an undirected graph with a k-clique } Theorem: CLIQUE NP The k-clique itsel is a certiicate NP = all the problems or which once you have the answer it is easy (i.e. eicient) to veriy 7
POLY-TIME REDUCIBILITY P = NP? $$$ : Σ* Σ* is a polynomial time computable unction i some poly-time Turing machine M, on every input w, halts with just (w) on its tape Language A is polynomial time reducible to language B, written A P B, i there is a polytime computable unction : Σ* Σ* such that: w A (w) B is called a polynomial time reduction o A to B Theorem: I A P B and B P, then A P A B Proo: Let M B be a poly-time (deterministic) TM that decides B and let be a poly-time reduction rom A to B We build a machine M A that decides A as ollows: On input w: 1. Compute (w) 2. Run M B on (w) Suppose B is NP-Complete Deinition: A language B is NP-complete i: 1. B NP 2. Every A in NP is poly-time reducible to B (i.e. B is NP-hard) P B NP So, i B is NP-Complete and B P then NP = P. Why? 8
Theorem (Cook-Levin): SAT is NP-complete Corollary: SAT P i and only i P = NP WWW.FLAC.WS Read Chapter 7.3 o the book or next time 9