Introduction to Complexity Theory Big O Notation Review Linear function: r(n) =O(n). Polynomial function: r(n) =2 O(1) Exponential function: r(n) =2 no(1) Logarithmic function: r(n) = O(log n) Poly-log function: r(n) =log O(1) n Definition 1 (TIME) Let t : ℵ ℵ. Define the time complexity class, TIME(t(n)) to be TIME(t(n)) = {L DTM M which decides L in time O(t(n))}. Definition 2 (NTIME) Let t : ℵ ℵ. Define the time complexity class, NTIME(t(n)) to be NTIME(t(n)) = {L NDTM M which decides L in time O(t(n))}. Example 1 Consider the language A = {0 k 1 k k 0}. Is A TIME(n 2 )? Is A TIME(n log n)? Is A TIME(n)? Could we potentially place A in a smaller complexity class if we consider other computational models? Theorem 1 If t(n) n, then every t(n) time multitape Turing machine has an equivalent O(t 2 (n)) time single-tape turing machine. see Theorem 7.8 in Sipser (pg. 232) Theorem 2 If t(n) n, then every t(n) time RAM machine has an equivalent O(t 3 (n)) time multi-tape turing machine. optional exercise Conclusion: Linear time is model specific; polynomical time is model indepedent. Definition 3 (The Class P ) P = k TIME(n k ) Definition 4 (The Class NP)
NP = k NTIME(n k ) Equivalent Definition of NP NP = {L L has a polynomial time verifier }. A polynomial time verifier for a language A is an algorithm, V,whereA = {w V acepts <w,c> for some string c}. Example 2 Let RELP RIME = {< x,y > gcd(x, y) = 1}. Is RELP RIME NP? Is RELP RIME P? Example 3 PATH = {<G,s,t > G has a directed path from s to t}. IsPATH P? Construct a deterministic polynomial time Turing machine, M, that decides P AT H. M = On Input <G,s,t > where G is a directed graph with nodes s and t. 1. Place a mark on node s. 2. While there exists an unmarked node do 3. Search for all edges (a, b) wherea is marked and b unmarked, and mark such b. 4. If t is marked, accept; otherwise, reject. To show an algorithm runs in polynomial, one must show that each step is executed only a polynomial number of steps as well as each steps executes in polynomial time. Example 4 SAT = {< φ> φ is a satisfiable boolean formula }. IsSAT NP? Construct a nondeterministic polynomial time Turing machine, M, as follows: M = On Input <φ>: 1. Nondeterministically select an assignment of the variables, x. 2. Teset where x satisfies φ. 3. If thest test passes, accept; otherwise, reject. Example 5 VALUE = {< φ,x > φ(x) where φ is a boolean formula and x an assignment of variables }. IsVALUE P? Example 6 RATIONALROOT = {< p> q Q such that p(q) =0where p(x) =a 0 + a 1 x +...+ a k x k is a polynomial of degree k where i =1...k, a i ℵ. Is RATIONALROOT P? Construct a nondeterministic polynomial time Turing machine, M, for RATIONALROOT: M = On Input <p>where p(x) =a 0 + a 1 x +...+ a k x k, a i ℵ.
1. Nondeterministically select q Q. 2. Evaluate p(q). 3. If p(q) = 0, accept; otherwise, reject. Does this run in polynomial time? Cearly, to evaulate p(q) is polynomial in the length of q since multipication can be computed in polynomial time. How big is q though? From the Rational Root Theorem in algebra, any rational root, x, ofp(x) =0isoftheformx = s t where s a 0 and t a k. Since a 0 =log(a 0 ) <nand a k =log(a k ) <n, q = O(n). Thus, step 1 and 2 can be computed in polynomial steps. Example 7 Let SUBSETSUM = {< S,t> {y 1,y 2,...,y n } = Y S = {s 1,s 2,...,s n } such that i y i = t} Construct a nondeterministic polynomial time Turing machine, M, as follows: M = On Input <S,t>: 1. Nondeterministically select a subset c of the numbers in S. 2. Test whether c is a collection of numbers that sum to t. 3. If thest test passes, accept; otherwise, reject. Theorem 3 P NP A O(t(n)) DTM has an equivalent O(t(n)) NDTM. Definition 5 (The Class EXPTIME) EXPTIME = k TIME(2 nk ) Definition 6 (The Class NEXPTIME) NEXPTIME = k NTIME(2 nk ) Relations: P NP EXPTIME NEXPTIME Definition 7 (Space Complexity Classes) SPACE(t(n)) = {L DTM M which decides L in space O(t(n))}. NSPACE(t(n)) = {L NDTM M which decides L in space O(t(n))}. PSPACE = k SPACE(n k ) NPSPACE = k NSPACE(n k )
Definition 8 (Logarithmic Space Classes) L = SPACE(log n) NL = NSPACE(log n) How can our definition of a turing machine use only logarithmic space since the input tape uses linear space? We introduce a new turing machine with two tapes: a read-only input tape and a read/write tape. Example 8 PATH NL Relations between time and space: Theorem 4 TIME(f(n)) SPACE(f(n)) A deterministic turing machine that decides membership in O(f(n)) steps can use at most O(f(n)) space since one TM step requires can only write to one memory cell. Theorem 5 If f(n) log(n), SPACE(f(n)) TIME(k f(n) ) Define a configuration as a snap shot of a turing machine including the position of the heads, the state of the control unit, and the contents of all cells on tape. If our turing machine is restricted to O(f(n)) space, the work tape head can be in only O(f(n)) locations and the input tape head canbeinonlyo(n). The control unit can be in any of c positions were c is a constant representing the number of states in the fsm. If we have k characters that can be written to a cell, then there are O(k f(n) ) possibilities for the content of the tape. Thus, there are O(cnf(n) k f(n) )=O(k f(n) ) configurations. If the turing machine is to halt, there will be no duplicate configuration and thus it must run in O(k f(n) ). Corollary 1 L P Corollary 2 P PSPACE Corollary 3 PSPACE EXPTIME Complements of complexity classes Definition 9 (coc) The complement of a complexity class of decision problems C, denoted coc, is the set of decision problems that are complemnt of decision problems of C. Example 9 cosat = {< φ> φ is NOT a satisfiable boolean formula }. Is cosat NP? Is cosat EXPTIME? IscoSAT PSPACE? Theorem 6 If C is a deterministic time or space complexity class, then C=coC.
Deterministic turing machines are closed under complementation. Is NP = conp? Theorem 7 NP conp {} Let PRIMALITY = {< n> n is prime}. WeshowPRIMALITY NP conp. not quite done. Lemma 1 An integer p>2 is prime iff there is an integer 1 <r<psuch that r p 1 1(modp) and q such that q p 1, r p 1 q 1(modp). not quite done