Comparison of several polynomial and exponential time complexity functions Time complexity function n n 2 n 3 n 5 2 n 3 n Size n 10 20 30 40 50 60.00001.00002.00003.00004.00005.00006 second second second second second second.0001.0004.0009.0016.0025.0036 second second second second second second.001.008.027.064.125.216 second second second second second second.1 3.2 24.3 1.7 5.2 13.0 second second second minutes minutes minutes.001 1.0 17.9 12.7 35.7 366 second second minutes days years centuries.059 58 6.5 3855 2*10 8 1.3 *10 13 second minutes years centuries centuries centuries
Size of Largest Problem Instance Solvable in 1 Hour Time complexity function With present computer With computer 100 times faster With computer 1000 times faster n N 1 100N 1 1000N 1 n 2 N 2 10N 2 31.6N 2 n 3 N 3 4.64N 3 10N 3 n 5 N 4 2.5N 4 3.98N 4 2 n N 5 N 5 +6.64 N 5 +9.97 3 n N 6 N 6 +4.19 N 6 +6.29
Intractable problems rec sets decidable r.e. sets non r.e. sets undecidable note procedure yes not always halts Algorithm yes no always halts
Definition : A problem is said to be intractable if it is so hard that no polynomial time algorithms can possibly solve it. Certain decidable problems may be intractable!!! What are polynomial time algorithms? Why possibly? In what? In which computer?
problem Given I and P, find S satisfying P. problem Generic instance Question A generic instance is specified in terms of various components(parameters) such as sets,graphs,functions,numbers,. A question is asked in terms of the generic instance. Decision problems yes or no answers Optimization problems
Traveling Salesman Instance: A finite set C = {c 1, c 2,., c m } of cities, a distance d(c i, c j ) Z + for each pair of cities c i, c j C, and a bound B Z + (where Z + denote the positive integers ) Question: Is there a tour of all cities in C having total length no more than B. < c π(1), c π(2),., c π(m) > such that m i= 1 1 d yes or no ( i), cπ ( i+ 1) ) + d( cπ ( m), cπ (1 ) B. ( cπ )
Algorithm yes no a TM that always halts An algorithm is a step - by - step porcedure( consisting of a finite sequence of instructions) for solving a problem. Polynomial time algorithms exponential time algorithms
Time Complexity T(n) T(n) time requirements of an algorithm for solving a problem of size n # of steps rather than absolute time Why? Problem length Input Length T(n) = max{ t t is time required by the algorithm to solve a problem instance of size n } Related to an encoding scheme Need two underlying assumptions: (i) The model of computation TM (ii) a reasonable encoding scheme?
How to encode a problem Reasonable encoding schemes What is this? A generally accepted meaning of reasonable includes : (i) Conciseness No unnatural padding (ii) Decodability Existence of a polynomial time algorithm capable of extracting a description of all components of a problem instance from encoded data. Unfortunately, no definite way for satisfying this!!! (even though the existence of a reasonable encoding scheme is intuitively obvious )
v 1 v 2 v 3 v 4 Descriptions of the graph G=(V,E) where V= {V 1, V 2, V 3, V 4 } and E= { {V 1, V 2 },{V 2, V 3 }}, under three different encoding schemes. Encoding Scheme String Length Vertex list, Edge list Neighbor list Adjacency matrix rows V[1]V[2]V[3]V[4](V[1]V[2])(V[2]V[3]) (V[2])(V[1]V[3])(V[2])() 0100/1010/0100/0000 36 24 19 Encoding Scheme Vertex list, Edge list Neighbor list Adjacency matrix rows Lower Bound 4v + 10e 2v + 8e (= 4 2e) v 2 +v - 1 Upper Bound 4v + 10e + (v + 2e)*[log 10 v] 2v + 8e + 2e*[log 10 v] v 2 +v - 1
However, most people would agree on whether or not a particular encoding scheme is reasonable. ( although the absence of a formal way is somewhat uncomfortable ) How can you resolve this difficulty? Setting up a standard encoding scheme
Standard encoding scheme Flexible enough to accommodate any problem with minor modifications. Basic assumption Generic instances of problems consist of basic types of set theoretic objects. Standard encoding scheme f e : D π S, D π = {problem instances } S = { structured strings } Σ* Σ = { 0, 1, -, [, ], (, ), },
structured strings, = { 0, 1, -, [, ], (, ), } (1) The binary representation of an integer k is a structured string representing k. ( put - if k is negative. ) (2) If x is the structured string representing an integer k, then [x] is a structured string that can be used as a name (3) If x 1, x 2,.., x m are structured strings representing the objects X 1, X 2,., X m, then (x 1, x 2,.., x m ) is a structured string representing < X 1, X 2,.., X m >.
Example integers 0, -1, 11, 100, -101, unstructured elements [0],[1],. (vertices, elements, cities,. ) sequences set ( [0],[1],. ) graphs ( x, y ) finite functions ( (x 1,y 1 ), (x 2,y 2 ),.., (x i,y i ),. ) f : X Y X={x 1, x 2,, x n }, Y={y 1, y 2,, y n } rational numbers (p,q) p/q O.k., now, we are comfortable!!! The length of input can be determined.
Input Length Length : Z + D π encoding - independent polynomially related to the input length that we would obtain from a reasonable encoding scheme. String lengths, x 1 and x 2 are polynomially related if there exist two polynomials p 1 and p 2 such that x 1 p 1 [ x 2 ], and x 2 p 2 [ x 1 ]. The function Length is not unrealistic!!! Why?
An encoding scheme e for π is said to be reasonable if Length(I) and x are polynomially related for all x, where x is the string encoding for I under e. D π
Observation The intractability of a problem turns out to be essentially independent of : (1) particular encoding scheme (2) the computer model used for determining time complexity Why?
Independence of computer model Simulated machine B 1-Tape Turing Machine (1TM) Simulating machine A 1TM ktm RAM - O(T(n)) O(T(n)logT(n)) k-tape Turing Machine (KTM) O(T 2 (n)) - O(T(n)logT(n)) Random Access Machine (RAM) O(T 3 (n)) O(T 2 (n)) - Hopcroft, Ullman (1969) Aho,Hopcroft, Ullman (1974)
π D π : a problem : The set of instances of a decision problem D π Y π Y π = { p p D π and the solution of p is yes }
An instance I of a problem π belongs to D π I can be obtained from the generic instance by substituting particular objects of specified types for all the generic components
π : a problem π O : Optimization version π D : Decision version A solution to π O A solution to π D A decision problem is no harder than its corresponding optimization problem!!! If a decision problem is NP-complete, then its corresponding optimization problem is at least as hard!!! Even though the theory of NP-completeness restricts attention to only decision problems, we can extend the implications of the theory to optimization problems as well.
Why restricting attention to only decision problems? These problems have a very natural, formal counterpart which is a suitable object to study a mathematically precise theory of computation!!! What is this counterpart? Language!!!
Language : L Σ a finite set of symbols Σ* the set of all finite strings of symbols from (including empty string ε ) L * L is a language over Decision problem Encoding scheme Language
An encoding scheme e for a problem π provides a way of describing each instance I D π by an appropriate string of symbols over some alphabet!!! e: D π Σ * I D π, e(i) Σ *
The problem π and its encoding scheme e partition Σ* into three class of strings D π * N π Y π π L(π,e)
L(π,e) ={ x Σ* Σ is the alphabet used by e, and x is the encoding under e of an instance I Y π } L(π,e) * L(π,e) is a language!!! Formal theory of language can be applied to decision problem
Observation Decision problem Language Does the language derived from a decision problem depend on encoding schemes? Why? no!!! encoding scheme Reasonable encoding scheme!!! We may assume a reasonable encoding scheme e for a problem π However, if requested, we could specify a particular encoding scheme e
Deterministic Turing machines and the class P Deterministic one-tape Turing machine Finite state control Tape Read-write head A program for a DTM specifies the following information : (1) A finite set Γ of tape symbols including a subset Γof input symbols and a distinguished bank symbol b. (2) a finite set Q of states, including a distinguished start-state q 0 and two distinguished halt-states q Y and q N ; (3) a transition function δ :(Q - {q Y, q N }) Γ Q Γ {-1, +1 }. note -3-2 -1 0 1 2 3 4 M={ Q,, Γ, δ, q 0,b,F} {q Y, q N }
x * initially. 0 1 2. x -1 x x +1. b 1 0. 0 0 b b b x. 0 1 2. x -1 x x +1. b 1 0. 0 0 b b b q j = q N : no q j = q Y : yes read-write header initial state q 0 δ (q i, x i ) (q j, x k, m) ±1
L M { x * A DTM program M accepts x } The DTM program M halts in state q Y when applied to x iff x L M. L M is the language recognized by the DTM program M.
Definition: A DTM program M solves a decision problem π (under encoding scheme e) if : (i) M halts for all x *. (ii) L M = L(π,e) Definition: The time used in the computation of a DTM program M on input x is the number of steps occurring up until a halt state is entered. q N or q Y
Definition: ( time complexity ) Let a DTM program M halt for all x *. Its time complexity function is : T M : Z + Z + such that T M (n) = max{ m M takes m steps on x for x = n. } Why max?
Definition: A DTM program M is said to be a polynomial time DTM program if there exists a polynomial P such that T M (n) p(n) for all n Z + Definition:The class P of languages is defined : P={ L there is a polynomial time DTM program M for which L = L M }
A decision problem π belongs to P ( under encoding scheme e ) if L(π,e) P, i.e., there is a polynomial time DTM program M that solves π ( under encoding scheme e )
Non-deterministic Computation and the class NP Instance: A finite set of cites, C={c 1, c 2,, c n }, a distance d(c i, c j ) Z + for c i, c j C, and B Z +, where Z + is the positive integers. Question: Is there a tour of all cities in C having total length no more than B? No known polynomial time algorithm!!! Suppose that someone claims, for a particular instance of problem, its answer is yes. Can you verify his/her claim? Yes, if he/she provides us a tour having the required properties. Polynomial Verifiability
Polynomial time Solvability Polynomial time verifiability Checking a given S yes if and only if S satisfies P Characterization of an NP class problem
Nondeterministic Program - guessing stage Given I D π,, this stage guesses some structure S - checking stage Given I and S, does S lead this stage to respond yes for I and S?
Observation (i) S does not nec. depend on I D π. S Γ* ( can possibly be ε ) (ii) The checking stage behaves in a normal deterministic manner. (iii) * Y π N π π Y π : halt with yes N π : halt with no π : go forever
An NDTM(program) M can solve a decision program π if the following two properties hold true for I D π : (i) I Y π (ii) I Y π some S that, when guessed for I, will lead the checking stage to respond yes for I and S any S that, when guessed for I, will lead the checking stage to respond yes for I and S.
Nondeterministic One-tape Turing Machine( NDTM ) Guessing module Finite state control Write-only Tape Guessing head Read-write head δ :(Q - {q r, q N }) Γ Q Γ { -1, +1 }. M ={ Q,, Γ, δ, q 0,b,F}
NDTM program (1) initially,. -3-2 -1 0 1 2 3. x (2) guessing, b (3) checking, b b b b... b... w r/w w inactive GM GM... -2-1 0 1 2 3 4...... b In the same fashion as DTM program!!! Γ,,b, Q, q 0, q Y, q N FSC FSC δ :(Q - {q Y, q N }) Γ Q Γ { -1, +1 }. 0 q 0 inactive b b b...
An NDTM program ceases computation for x * when the finite state control enters one of two halt states, q Y and q N. note For each x *, one set of computations Definition : A computation for x * is said to be an accepting computation if it halts in q Y. All the others, halting or not, are classed together as non-accepting computations.
Definition: An NDTM program M accepts x, if one of the computations for x is an accepting computation. L M the language recognized by M L M {x * M accepts x}
Definition: The time required by an NDTM program M to accept the string x L M is defined to be the minimum(over all accepting computations of M for x) number of steps occurring in the guessing and checking stages up until the halt state q y is entered. Informally, assume the right guessing!!! Why?
Definition: The time complexity function T M : Z+ Z+ for an NDTM program M is : T M (n) = max { {1} { m there is an x L M with x =n such that the time to accept x by M is m } } No input of length n are accepted
Definition : An NDTM program M is said to be a polynomial time NDTM program if there exist a polynomial p such that T M (n) P(n), n 1. The class NP is defined : NP = { L there is a polynomial time NDTM program M for which L M = L }
Definition: A decision program π is said to belong to NP( under encoding scheme e ) if L(π,e) NP.
Relationship between P and NP P NP P NP Why? A deterministic algorithm can be used as the checking stage of a non-deterministic algorithm!!! No general methods converting a polynomial time NDTM algorithm to a polynomial time DTM algorithm
Theorem π NP a polynomial P such that π can be solved by a deterministic algorithm having time complexity O(2 p(n) ). The best known result!!! [proof] Take any π in NP. Suppose that A is a polynomial time NDTM program for solving π. q(n) = a polynomial time bound on the time complexity of A. c2 (WLOG, let q(n) c n,c 1, c 2 > 0. ) 1 x L(π,e), there must exist some guessed string s Γ*, s q(n), where x = n. # of possible guesses = k q(n), where Γ = k. Why? q(n) k q(n) K q(n)+ε c 2 p(n)
Is NP = P? Well, P NP why? Is NP P? If yes then P = NP!!! But no hope!!! Most people believe P NP!!! NP NPI P? NP - complete!!!
Polynomial Transformability and NP-Completeness P NP However, nobody knows if NP P. In addition, no hope for proving P = NP. If P NP, then π NP - P This is our main focus!!!
Definition:( polynomial transformation ) A polynomial transformation from a language L 1 1 * to a language L 2 2 * is a function f: 1 * 2 * that satisfies the following conditions Notation (i) There is a polynomial DTM program that computes f. (ii) For all x 1 *, x L 1 f(x) L 2. L 1 L 2
Lemma: If L 1 L 2, then L 2 P L 1 P, and equivalently L 1 P L 2 P. [proof] Suppose that L 1 L 2 and L 2 P. Let 1 and 2 be alphabets of L 1 and L 2, respectively, and let f : 1 * 2 * be a polynomial transform from L 1 to L 2. Such a transform f exists!!! Why? M f = a polynomial time DTM program that computes f. M 2 = a polynomial time DTM program that recognizes L 2. x 1 f(x M 1 ) f M 2 yes no yes no A polynomial DTM program for recognizing L 1 can be constructed by composing M f with M 2!!! Why? (continue)
P f a polynomial function bounding the running time of M f P 2 a polynomial function bounding the running time of M 2 Then, x 1 1 *, f(x 1 ) P f ( x 1 ) O( P f ( x 1 ) + P 2 ( P f ( x 1 ) ) ) M f M 2 x 1 (P M f x 1 ) f M 2 yes no yes no
note L(π 1, e 1 ) L(π 2, e 2 ) π 1 π 2!!! π 1 π 2 if there exists a polynomial transformation f from a decision problem π 1 to a decision problem π 2 f : D π Dπ 1 2 that satisfies the following two conditions: (i) f is computable by a polynomial time algorithm (ii) For all I D, I Y f ( I Y π 1 π ) 1 π 2
A Hamiltonian circuit in G=(V,E) is a simple circuit that includes all vertices of V. HC( Hamiltonian Circuit ) Instance : A Graph G = ( V,E ) Question : Does G contain a Hamiltonian circuit? TS( Traveling Salesman ) Instance : A finite set C = {c 1, c 2,, c π } of cities. A distance d(c i, c j ) Z + for each pair of cities c i, c j C, and a bound B Z +. Question : Is there a tour of all cities in C having total length no more than B? Is HC TS? Yes!!! Why?
C := V (c i := v i ) d(c i, c j ) := 1 if {v i, v j } E 2 otherwise B := V f : D HC D TS (i) f can be computable by a polynomial time algorithm why? (ii) For all I D HC, I Y HC f(i) Y TS why? HC TS
Lemma : If L 1 L 2 and L 2 L 3, then L 1 L 3 [proof] Let 1, 2, and 3 be the alphabets of languages L 1, L 2, and L 3, respectively. f 1 : 1 * 2 * a polynomial transformation from L 1 to L 2. f 2 : 2 * 3 * a polynomial transformation from L 2 to L 3. Then, f 3 : 1 * 3 * is f 2 (f 1 (x)) for all x 1 *. x 1 x P 1 ( x 1 ) P 2 (P 1 ( x 1 ) ) f 1 (x) f 1 f 2 f 2 (f 1 (x)) (f 2 * f 1 )(x) = f 2 (f 1 (x)) f 2 * f 1 = f 3 x 1 *, x L 1 f 1 (x) L 2 f 2 ( f 1 (x) ) L 3 x L 1 f 2 ( f 1 (x) ) L 3 L 1 L 3
Definition: Two language L 1 and L 2 (two decision problems π 1 and π 2 ) are said to be polynomially equivalent if L 1 L 2 and L 2 L 1 ( π 1 π 2 and π 2 π 1 ). NP P NP-complete P and NP-complete give equivalent classes!!!
Definition: A language L is defined to be NP-complete if (i) L NP (ii) L L for all L NP.
Lemma : If L 1 and L 2 belong to NP, L 1 is NP-complete, and L 1 L 2 then L 2 is NP-complete [proof] Exercise. Given an NP-complete problem π and a new problem π, (i) π NP π NP-complete (ii) π π
P : Given I, is X true for I? Co-P : Given I, is X false for I? Given a DTM algorithm A for solving P, A can be used for solving Co-P!!! Co-P P h(x) x h A A' yes no yes no Is the same true for an NDTM algorithm? Well,.
Polynomial verifiability ( for π) Γ* However, to answer π c we need to examine all possible solutions!!!
TSP ( Traveling Salesman ) TSP : Given a set of cities, the intercity distance and a bound B, is it true that there is a tour of all cities having total length no more than B? Co-TSP : -----------------------------------------, Is it true that every tour of all cities has total length more than B? P NP NP P Co-NP
Observation P Co-P Co-P P P = Co-P Suppose that P = NP. Then, NP = Co-NP What if P NP?
Theorem : If there exists an NP-complete problem π such that π c NP, then NP = Co-NP. Why? AA x π' c ( π NP ) hπ' h ( x) f h ( x)) π' π' f π π ( π' π c h π h c polynomial verifiability π ( fπ ( hπ ' ( x))) c π A Yes The NDTM program AA accepts π c π NP NP = Co-NP
SAT( satisfiability ) Instance : A set U of Boolean variables and a collection of clauses C over U. Question : Is there a satisfying truth assignment for C? (u 1 u 2 u 5 ) (u 2 u 3 u 7 u 9 ) (u 4 u 6 ) (u 1 u 7 u 8 u 9 ). wff in CNF U = {u 1, u 2, u 3,.., u 9 } C = {{u 1, u 2, u 5 },{u 2, u 3, u 7, u 9 },{u 4, u 6 },{u 1, u 7, u 8, u 9 } }
Cook s Theorem Theorem: ( Cook s Theorem ) SAT(satisfiability) is NP-complete. [proof] (i) SAT NP Why? L SAT (ii) π NP, π SAT How? Well,.. = L(SAT,e) L NP, L L SAT There are infinitely many problems in NP!!! How to get around this problem?
π NP, π SAT. How? π NP An polynomial time NDTM program M accepting π,i.e., L M =L(π,e) SAT * N π Yπ π (in polynomial time) Describe M s moves using the generic instance of SAT!!! L(π,e) Why? π NP an NDTM program M accepting π in p(n) time, i.e., L M = L(π, e)
f L : * 1 * L(π,e) * L(SAT,e 1 ) 1 * x *, x L(π,e) f L (x) L(SAT,e 1 ) Let L M = L(π,e) for some NDTM M. Since M = (Q,, Γ, δ, q 0,b,F), f L will be derived in terms of P, Q,, Γ, δ, q 0, q N, q Y,!!!
Basic idea U and C are constructed so that an NDTM program M accepts x in * iff f L (x) has a satisfying truth assignment. - A generic instance of SAT consists of U and C -f L (x)? How?
Suppose that x * is accepted by M. There is an accepting computation for M on x such that (i) The number of steps in the checking stage is bounded by some polynomial P(n), n = x. (ii) The number of steps in the guessing stage is bounded by P(n), n = x. (ii) implies that the number of symbols in the guessing string is bounded by P(n), n = x GS FSC -P(n) -2-1 0 1 2 b... b.. Only the tape squares within this range can be involved in the computation q i P(n) b
The status of checking computation at any one time can be specified completely by giving (i) The content of tape squares (ii) The current state (iii) The position of W/R head ID There are at most P(n) + 1 times to be considered!!! Why? Such a computation can be described completely using only (i) a limited number of Boolean variables (ii) a truth assignment to them
Variable Range Intended meaning Q[i,k] 0 i p(n) At time i (upon completion of 0 k r the ith step),m is in state q k. H[i,j] 0 i p(n) At time i,the read-write head is -p(n) j p(n)+1 scanning tape square j. S[i,j,k] 0 i p(n) At time i, the contents of tape -p(n) j p(n)+1 square j is symbol s k. 0 k v U = O(P(n) 2 ) q 0, q 1 = q N, q 2 = q Y, q 3,.., q r r = Q - 1 s 0 = b, s 1, s 2, s 3,.., s v v = Γ - 1
note Initially -1 0 1 2 n..... What if the program M halts before time P(n)? x
x L M there is an accepting computation of M on x with p(n) or fewer steps in its checking stage and with a guessed string w of length at most p(n) there is a satisfying truth assignment for the collection of clauses in f L (x). Now, using previous Boolean variables construct clauses for f L (x). Restricting the behavior of the polynomial NDTM program M!!!
{ Q[i,0], Q[i,1],..., Q[i,r] }, 0 i p(n) (P(n)+1) r { Q[i,j], Q[i,j ] }, 0 i p(n), 0 j <j r (P(n)+1)(r+1)(r/2) at least one state no more than one state Q = r + 1
Clause groups Restriction imposed G 1 At each time i, M is in exactly one state. G 2 At each time i, the read-write head is scanning exactly one tape square. G 3 At each time i, each tape square contains exactly one symbol from Γ. G 4 At each time 0, the computation is in the initial configuration of its checking stage of input x. G 5 By time p(n), M has entered state q Y and hence has accepted x. G 6 For each time i, 0 i < p(n), the configuration of M at time i + 1 follows by a single application of the transition function δ from the configuration at time i. Now, can you see what we are going to do? δ : (Q - {Q N, Q Y }) Γ Q Γ { -1, +1 }
Clause group Clauses in group G 1 { Q[i,0],Q[i,1],.....,Q[i,r] }, 0 i p(n) { Q[i,j],Q[i,j ], 0 i p(n), 0 j <j r G 2 { H[i,-p(n)],H[i,-p(n)+1],...,H[i,p(n)+1] }, 0 i p(n) { H[i,j],H[i,j ]}, 0 i p(n), -p(n) j <j p(n)+1 G 3 { S[i,j,0],S[i,j,1],...,S[i,j,v] }, 0 i p(n), -p(n) j p(n)+1 { S[i,j,k],S[i,j,k ] }, 0 i p(n), p(n) j p(n)+1, 0 k <k v G 4 {Q[0,0]}, {H[0,1]},{S[0,0,0]}, {S[0,1, k 1 ]},{S[0,2, k 2 ]},,{S[0,n, k n ] }, { S[0,n+1,0]}, {S[0,n+2,0]},...,{S[0,p(n)+1,0]}. Where x = S S LS G 5 { Q[p(n),2] } k 1 k 2 k n
G 6 : {S[i,j,l], H(i,j), S[ i+1, j, l] }, 2(p(n)+1) 2 (v+1) 0 i p(n) -p(n) j p(n)+1 0 l v {H[i,j], Q[i,k],S[i,j,l],H[i+1,j+ ] } {H[i,j], Q[i,k],S[i,j,l],Q[i+1,k ] } {H[i,j], Q[i,k],S[i,j,l],S[i+1,j,l ] } 0 i < P(n) -P(n) j P(n)+1 0 k r 0 l v 6(p(n)+1) 2 (r+1)(v+1) q k Q - {q N, q Y } δ(q k, s l ) (q k, s l, )
x L M There is an accepting computation of M on x of steps p(n) or less, and this computation, given the interpretation of variables, imposes the behavior of M so that there exist a truth assignment satisfying all the clauses in C = G l G 2 G 3....... G L Polynomial time for constructing f L (x)!!! Why? U C = O(p(n) 4 ) Check it!!! NP - Hard Exercise