Computability ad computatioal complexity Lecture 6: Relatios betwee complexity classes Io Petre Computer Sciece, Åbo Akademi Uiversity Fall 2015 http://users.abo.fi/ipetre/computability/ 21 May 2018 http://users.abo.fi/ipetre/computability/ 1
Cotet Complexity classes Precise Turig machies The time hierarchy Space vs time LÍNL ÍP ÍNP ÍPSPACE Nodetermiistic space PSPACE=NPSPACE NSPACE=coNSPACE Savitch s theorem Immerma-Szelepscéyi theorem 21 May 2018 http://users.abo.fi/ipetre/computability/ 2
Complexity classes A complexity class is specified by: the uderlyig model of computatio (for us multi-tape Turig machies) mode of computatio (already see the determiistic ad the odetermiistic modes) a resource that will be boud (time, space, ) a boud (f:n N): logarithmic, polyomial, expoetial, The complexity class is the set of all laguages decided by a multitape TM M operatig i the appropriate mode ad such that for ay iput x, M uses at most f( x ) uits of the specified resource 21 May 2018 http://users.abo.fi/ipetre/computability/ 3
Proper complexity fuctios Questio: what kid of fuctios f:n N is reasoable to cosider how about weird fuctios such as 2 if is prime, the th smallest otherwise such uusual fuctios lead to very ivolved cosideratios; perhaps ot useful for atural computatioal problems Defiitio: We say that f:n N is a proper complexity fuctio if f is odecreasig: f(+1)³f(), for all ÎN there is a k-tape TM M f =(K,S,d,s) with iput ad output such that for ay iput x we have that: 1. M f (x)= f( x ), where is a uasi-blak symbol 2. M f halts withi O( x +f( x )) steps 3. M f uses O(f( x )) space besides its iput We oly use proper complexity fuctio as bouds Examples of proper complexity fuctios f(): c,, ëlog û, ëlog 2 û, ë log û, 2, 3 +3, 2, 1/2,! 21 May 2018 http://users.abo.fi/ipetre/computability/ 4
PRECISE TURING MACHINES 21 May 2018 http://users.abo.fi/ipetre/computability/ 5
Precise Turig machies Defiitio. A TM M (with iput/output or ot, determiistic or odetermiistic) is precise if there are fuctios f,g:n N such that for every ad for every iput x of legth ad for every computatio of M o x: M halts after precisely f() steps all of its tapes (except the first ad the last i case of machies with iput ad output) are at haltig of legth precisely g() 21 May 2018 http://users.abo.fi/ipetre/computability/ 6
Simulatig Turig machies with precise Turig machies Propositio. Suppose that a (determiistic or odetermistic) TM M decides a laguage L withi time (or space) f(), where f is a proper fuctio. The there is a precise Turig machie M which decides the same laguage i time (or space, resp.) O(f()). Proof idea. There are 4 cases to cosider: determiistic/odetermiistic, time/space complexity We first eed to compute f( x ) from the defiitio of proper fuctios, there is a TM M f givig output f( x ) Use the output strig of M f as a alarm clock to decide withi either time f( x ) or space f( x ) Simulate M o x for exactly f( x ) steps / withi f( x ) space 21 May 2018 http://users.abo.fi/ipetre/computability/ 7
Time ad space complexity classes Let f:n N be a proper complexity fuctio We defie: TIME(f) (determiistic time) SPACE(f) (determiistic space) NTIME(f) (odetermiistic time) NSPACE(f) (odetermiistic space) 21 May 2018 http://users.abo.fi/ipetre/computability/ 8
Time ad space complexity classes Use a class of parameterized fuctios istead of fuctio f P = TIME( k ) =È j>0 TIME( j ) NP = NTIME( k ) =È j>0 NTIME( j ) PSPACE = SPACE( k ) =È j>0 SPACE( j ) NPSPACE = NSPACE( k ) =È j>0 NSPACE( j ) EXP = TIME(2 k ) =È j>0 TIME(2 j ) L = SPACE(log ) NL = NSPACE(log ) 21 May 2018 http://users.abo.fi/ipetre/computability/ 9
Complemets of complexity classes The complemet of a laguage LÍS* is L=S*-L Recall: a decisio problem ca be idetified with the laguage of its yes istace The complemet of a decisio problem A, deoted A COMPLEMENT, is that problem whose aswer is yes iff the aswer of A o the same iput is o Example: SAT COMPLEMENT give a Boolea formula i CNF, is it usatisfiable? HAMILTON PATH COMPLEMENT give a graph G, decide if G does ot have a Hamiltoia path The complemet of a complexity class C is coc={l: LÎC} 21 May 2018 http://users.abo.fi/ipetre/computability/ 10
Closure uder complemet Observatio: all determiistic time ad space complexity classes are closed uder complemet Chage the yes with the o states of the TM For example, P=coP Later i this lecture we prove that the same is true for odetermiistic space complexity Ope problem: are the odetermiistic time complexity classes closed uder complemet? For example, is NP=coNP? 21 May 2018 http://users.abo.fi/ipetre/computability/ 11
COMPLEXITY CLASSES: HIERARCHIES 21 May 2018 http://users.abo.fi/ipetre/computability/ 12
Hierarchies Addig ew features to the computatioal model ofte leads to more computatioal power Example: Chomsky s hierarchy from fiite automata to pushdow automata, to liear bouded (odetermiistic) TM, to TM Example: the ability of a TM to reject through o-haltig computatios gives the jump from recursive laguages (REC) to RE laguages Discuss ow a uatitative versio of the same observatio: with sufficietly greater time allocatio, TM are able to perform more complex computatioal tasks The proof will employ a uatitative sort of diagoalizatio 21 May 2018 http://users.abo.fi/ipetre/computability/ 13
Upper boud Let f:n N be a proper complexity fuctio. Defie H f ={M;x: M accepts iput x after at most f( x ) steps} i other words: the time-bouded versio of the HALTING laguage H Lemma. H f ÎTIME( f 3 () ) Proof idea Describe a TM U f with 4 tapes decidig H f i f 3 () steps based o: the uiversal TM U, the sigle-tape simulator of multi-tape machies, the liear speed-up machie, ad the machie M f that computes a yardstick of legth exactly f() 21 May 2018 http://users.abo.fi/ipetre/computability/ 14
Proof - cotiued Mode of operatio for U f : use M f o x to iitialize the 4 th tape o the appropriate alarm clock f( x ) so far: O(f( x )) steps, where the costat depeds just o f, ot o M,x copy the descriptio of M to the 3 rd tape covert x o the first tape to >x write o the secod tape the ecodig of state s of M simulate M o x: the iput is maipulated o the first tape, the cofiguratios are kept o the secod tape, descriptio of M o the third if M accepts i at most f( x ) steps, the U f also accepts; if ot, U f rejects Time complexity each step of M is simulated i O(f ( x ) 2 ); ru at most f( x ) steps total time: O(f ( x ) 3 ) it ca be made exactly f ( x ) 3 through the liear speed-up theorem 21 May 2018 http://users.abo.fi/ipetre/computability/ 15
Lower boud Lemma. H f ÏTIME( f(ë/2û) ) Proof idea. Suppose that there is a TM M H f decidig H f i time f(ë/2û) Costruct ow a diagoalizig machie D f with the followig behavior: D f (M): if M H (M;M)= yes, the o, else yes f D f o iput M rus i the same time as M H o iput M;M, i.e., i time f f(ë(2+1)/2û)=f(), where = M Questio: does D f accept its ow descriptio? If D f (D f )= yes, the M H f (D f;d f )= o, i.e., D f does ot halt o D f i at most f( D f ) steps. But D f always rus i at most f() steps for a iput of legth. Coclusio: D f (D f )= o If D f (D f )= o, the M H f (D f;d f )= yes, i.e., o D f o iput D f halts with aswer yes, i.e., D f (D f )= yes 21 May 2018 http://users.abo.fi/ipetre/computability/ 16
The time hierarchy theorem Theorem. If f()³ is a proper complexity fuctio, the TIME(f()) is strictly icluded withi TIME( (f(2+1)) 3 ) Proof Upper boud lemma: H f(2+1) ÎTIME( (f(2+1)) 3 ) Lower boud lemma: H f(2+1) ÏTIME( f(ë(2+1)/2û) )=TIME(f()) Corollary. P Ì EXP Proof. Ay polyomial is evetually smaller tha 2 ad so, PÍTIME(2 )ÍEXP By the theorem: TIME(2 ) Ì TIME( (2 2+1 ) 3 ) Í EXP 21 May 2018 http://users.abo.fi/ipetre/computability/ 17
The space hierarchy theorem Theorem. If f() is a proper complexity fuctio, the SPACE(f()) Ì SPACE( f()log f() ) Proof. Skip it. Similar to the time hierarchy results 21 May 2018 http://users.abo.fi/ipetre/computability/ 18
COMPLEXITY CLASSES: SPACE VS. TIME, DETERMINISTIC VS. NONDETERMINISTIC 21 May 2018 http://users.abo.fi/ipetre/computability/ 20
Comparig classes of differet kids So far: relatioships betwee classes of the same kid (determiistic time, determiistic space) relate to each other similar results hold also for odetermiistic classes Switch to discuss relatioships betwee classes of differet kids P vs. NP is a example here 21 May 2018 http://users.abo.fi/ipetre/computability/ 21
SPACE vs. TIME, DET vs. NONDET Theorem. Let f:n N be a proper complexity fuctio a) SPACE(f()) Í NSPACE(f()) ad TIME(f()) Í NTIME(f()) b) NTIME(f()) Í SPACE(f()) c) NSPACE(f()) Í TIME( c log +f() ) Proof a) A determiistic TM is also a (trivial) odetermiistic TM b) Simulate all computatio paths withi bouded space details below c) The reachability method for simulatig space-bouded machies details below 21 May 2018 http://users.abo.fi/ipetre/computability/ 22
NTIME(f()) Í SPACE(f()) proof Let LÎNTIME(f()) ad a precise odetermiistic TM M decidig L i time f(). Desig a determiistic TM M decidig L i space f() Idea: check all computatio paths; similar to simulatig a odetermiistic machie with a determiistic oe (i exp. time) let d be the degree of odetermiism of M; ay computatio of M is described by a seuece of umbers 0 d-1 of legth f() M starts by geeratig seuece 0 0 of legth f() crucial here that f is proper M simulates M o the computatio described by the curret d-ary seuece of legth f() it ca be doe i space f() because i time f() oly O(f()) characters ca be writte M geerates the ext seuece of legth f(), simulates it i the same space as before, etc. 21 May 2018 http://users.abo.fi/ipetre/computability/ 23
NSPACE(f()) Í TIME( c log +f() ) proof Let L be a laguage decided by a k-tape odetermiistic TM M with iput ad output i space f(); let x be a iput of legth Challege: simulate determiistically the odetermiistic computatio of M o x withi time c log +f(), for some costat c depedig o M Cofiguratio of M: a 2k+1-tuple (,w 1,u 1,,w k,u k ). But M is a machie with iput ad output ad so: w 1 u 1 =x throughout the computatio the output tape w k u k is ot of iterest for a decidig machie its cotet ca be igored all other strigs w i u i are of legth at most f() Coclusio: ay cofiguratio ca be represeted as (,i,w 2,u 2,,w k-1,u k-1 ), where i is the positio of the cursor o the first tape Questio: how may cofiguratios are there? at most K choices for at most +1 choices for i at most S (2k-2)f() choices for the strigs, couted altogether Aswer: for a iput of legth, there are at most c f() 1 =c log +f() 1 cofiguratios, where c 1 is a costat depedig o M 21 May 2018 http://users.abo.fi/ipetre/computability/ 24
NSPACE(f()) Í TIME( k log +f() ) proof cotd. Defie the cofiguratio graph of M odes: all possible cofiguratios of M edges (i,j) if there is a trasitio of M trasformig i ito j Observatio: decidig whether x is i L is the same as decidig whether a state ( yes,i, ) is reachable from the iitial state (s,0,>,1,,>,1) But this is REACHABILITY decidable i uadratic time c 2 m 2 for a iput of size m I other words, we ca aswer the origial uestio i time c 2 (c log +f() 1 ) 2 = c 2 (c 12 ) log +f() The result follows ow with c=c 2 c 1 2 How to hadle the cofiguratio graph of M oe optio: describe its adjacecy matrix i full more elegat optio: for ay two cofiguratios C, C decide o-the-fly whether C ca be yielded from C 21 May 2018 http://users.abo.fi/ipetre/computability/ 25
Class iclusios Corollary: L Í NL Í P Í NP Í PSPACE Í EXP Note: from the space hierarchy theorem we kow L Ì PSPACE Note: from the time hierarchy theorem we kow P Ì EXP Coseuece: at least oe of the iclusios above is proper! Ope uestio: Which oe? Cojecture: all of them! 21 May 2018 http://users.abo.fi/ipetre/computability/ 26
NONDETERMINISTIC SPACE 21 May 2018 http://users.abo.fi/ipetre/computability/ 27
Nodetermiistic space It follows from our previous results that NSPACE(f())ÍSPACE(k log +f() ) Questio: Is there a better way to simulate odetermiistic space? Aswer (to be show i the remaiig of this lecture): YES! Prove it through the reachability method Prove first a result about REACHABILITY Recall our first lecture: REACHABILITYÎSPACE() We ca do better (ad pay the price i time) 21 May 2018 http://users.abo.fi/ipetre/computability/ 28
Savitch s theorem Savitch s theorem. REACHABLITYÎSPACE(log 2 ) Proof (sketch): a middle-first search techiue Let G be a graph with odes, x, y odes of G, i³0 Defie predicate PATH(x,y,i)=true iff there is a path i G from x to y of legth at most 2 i Clearly, we ca solve REACHABILITY if we ca compute PATH(x, y, élog ù) for ay x,yîg Ay o-cyclic path i G has legth at most 21 May 2018 http://users.abo.fi/ipetre/computability/ 29
Savitch s theorem proof (cotiued) Desig a TM with two work tapes, i additio to the the iput tape, decidig PATH(x,y,i) Iput: the adjacecy matrix of G First work tape: x,y,i, all i biary Throughout the computatio, the first work tape will cotai several triples, of which (x,y,i) is the leftmost oe Secod work tape: at most O(log ) space will be eough 21 May 2018 http://users.abo.fi/ipetre/computability/ 30
Savitch s theorem proof (cotiued) Decidig PATH(x,y,i) If i=0, the it is eough to check if x=y or if there is a edge from x to y Otherwise, apply the followig recursive algorithm: For all odes z test whether PATH(x,z,i-1) ad PATH(z,y,i-1) Idea here: ay path of legth at most 2 i is made of two paths, each of legth at most 2 i-1 Space-efficiet implemetatio: Aalysis: geerate all z oe after the other reusig the space; for each ew z, add (x,z,i-1) to the first work strig ad start workig o this problem I case of a egative aswer, erase the triplet ad move to the ext z I case of positive aswer, erase (x,z,i-1), write (z,y,i-1) o the first work strig ad start o this problem I case of egative aswer, erase the triplet ad move to the ext z I case of positive aswer, detect by compariso with (x,y,i) to the left that this is the secod call ad retur yes to PATH(x,y,i) The descriptio above clearly implemets the recursive algorithm First work strig: at most élog ù triplets, each of legth at most 3 log Coclusio: O(log 2 ) space 21 May 2018 http://users.abo.fi/ipetre/computability/ 31
Nodetermiistic space Corollary: NSPACE(f())ÍSPACE(f 2 ()) Proof. To simulate a f()-space bouded odetermiistic TM o a iput of legth, we ru the algorithm i Savitch theorem o the cofiguratio graph of M o iput x Cofiguratio graph has c f() odes, for some costat c Coclusio: O(f 2 ()) is eough for the determiistic space simulatio 21 May 2018 http://users.abo.fi/ipetre/computability/ 32
PSPACE=NPSPACE Coseuece: PSPACE=NPSPACE! I other words, odetermiism is less powerful with respect to space tha it seems to be with respect to time 21 May 2018 http://users.abo.fi/ipetre/computability/ 33
Learig objectives of this lecture Defiitios of major complexity classes: P, NP, PSPACE, NPSPACE, EXP, L, NL The kow relatioships betwee the major complexity classes Nodetermiistic space NPSPACE=PSPACE NSPACE(f())=coNSPACE(f()) REACHABILITYÎSPACE(log 2 ), umber of reachable odes ca computed i NSPACE(log ) 21 May 2018 http://users.abo.fi/ipetre/computability/ 41