Time ad Space CS151 Complexity Theory Lecture 2 April 1, 2004 A motivatig questio: Boolea formula with odes evaluate usig O(log ) space? depth-first traversal requires storig itermediate values idea: short-circuit ANDs ad ORs whe possible April 1, 2004 CS151 Lecture 2 2 Time ad Space Ca we evaluate a ode Boolea circuit usig O(log ) space? Time ad Space Recall: TIME(f()), SPACE(f()) Questios: how are these classes related to each other? how do we defie robust time ad space classes? what problems are cotaied i these classes? complete for these classes? April 1, 2004 CS151 Lecture 2 3 April 1, 2004 CS151 Lecture 2 4 Outlie Why big-oh? Liear Speedup Theorem Hierarchy Theorems Robust Time ad Space Classes Relatioships betwee classes Some complete problems Liear Speedup Theorem: Suppose TM M decides laguage L i time f(). The for ay > 0, there exists TM M that decides L i time f() + + 2. Proof: simple idea: icrease word legth M will have oe more tape tha M m-tuples of symbols of M ew = old old m may more states April 1, 2004 CS151 Lecture 2 5 April 1, 2004 CS151 Lecture 2 6 1
Liear Speedup part 1: compress iput oto fresh tape Liear Speedup part 2: simulate M, m steps at a time 4 (L,R,R,L) steps to read relevat symbols, remember i state 2 (L,R or R,L) to make M s chages April 1, 2004 CS151 Lecture 2 7 April 1, 2004 CS151 Lecture 2 8 Liear Speedup accoutig: part 1 (copyig): + 2 steps part 2 (simulatio): 6 (f()/m) set m = 6/ total: f() + + 2 Theorem: Suppose TM M decides laguage L i space f(). The for ay > 0, there exists TM M that decides L i space f() + 2. Proof: same. Time ad Space Moral: big-oh otatio ecessary give our model of computatio Recall: f() = O(g()) if there exists c such that f() c g() for all sufficietly large. TM model icapable of makig distictios betwee time ad space usage that differs by a costat. I geeral: iterested i course distictios ot affected by model e.g. simulatio of k-strig TM ruig i time f() by sigle-strig TM ruig i time O(f() 2 ) April 1, 2004 CS151 Lecture 2 9 April 1, 2004 CS151 Lecture 2 10 Hierarchy Theorems Does geuiely more time permit us to decide ew laguages? how ca we costruct a laguage L that is ot i TIME(f()) idea: same as HALT udecidable diagoalizatio ad simulatio April 1, 2004 CS151 Lecture 2 11 Recall proof for Haltig Problem Turig Machies H : iputs April 1, 2004 CS151 Lecture 2 12 box (M, x): does M halt o x? The existece of H which tells us yes/o for each box allows us to costruct a TM H that caot be i the table. 2
Turig Machies Time Hierarchy Theorem D : iputs box (M, x): does M accept x i time f()? TM SIM tells us yes/o for each box i time g() rows iclude all of TIME(f()) costruct TM D ruig i time g(2) that is ot i table Time Hierarchy Theorem Theorem (Time Hierarchy Theorem): For every proper complexity fuctio f() : TIME(f())TIME(f(2) 3 ). more o proper complexity fuctios later April 1, 2004 CS151 Lecture 2 13 April 1, 2004 CS151 Lecture 2 14 Proof of Time Hierarchy Theorem Proof: SIM is TM decidig laguage { <M, x> : M accepts x i f( x ) steps } Claim: SIM rus i time g() = f() 3. defie ew TM D: o iput <M> if SIM accepts <M, M>, reject if SIM rejects <M, M>, accept D rus i time g(2) Proof of Time Hierarchy Theorem Proof (cotiued): suppose M i TIME(f()) decides L(D) M(<M>) = SIM(<M, M>) D(<M>) but M(<M>) = D(<M>) cotradictio. April 1, 2004 CS151 Lecture 2 15 April 1, 2004 CS151 Lecture 2 16 Proof of Time Hierarchy Theorem Claim: there is a TM SIM that decides {<M, x> : M accepts x i f( x ) steps} ad rus i time g() = f() 3. Proof sketch: SIM has 4 work tapes cotets ad virtual head positios for M s tapes M s trasitio fuctio ad state f( x ) + s used as a clock scratch space April 1, 2004 CS151 Lecture 2 17 Proof of Time Hierarchy Theorem cotets ad virtual head positios for M s tapes M s trasitio fuctio ad state f( x ) + s used as a clock scratch space iitialize tapes simulate step of M, advace head o tape 3; repeat. ca check ruig time is as claimed. Importat detail: eed to iitialize tape 3 i time O(f() + ) April 1, 2004 CS151 Lecture 2 18 3
Proper Complexity Fuctios Defiitio: f is a proper complexity fuctio if f() f(-1) for all there exists a TM M that outputs exactly f() symbols o iput 1, ad rus i time O(f() + ) ad space O(f()). Proper Complexity Fuctios icludes all reasoable fuctios we will work with log,, 2, 2,!, if f ad g are proper the f + g, fg, f(g), f g, 2 g are all proper ca mostly igore, but be aware it is a geuie cocer: Theorem: o-proper f such that TIME(f()) = TIME(2 f() ). April 1, 2004 CS151 Lecture 2 19 April 1, 2004 CS151 Lecture 2 20 Hierarchy Theorems Does geuiely more space permit us to decide ew laguages? Theorem (Space Hierarchy Theorem): For every proper complexity fuctio f() log : SPACE(f()) SPACE(f()logf()). Proof: same ideas. Robust Time ad Space Classes What is meat by robust class? o formal defiitio reasoable chages to model of computatio should t chage class should allow modular compositio callig subroutie i class (for classes closed uder complemet ) April 1, 2004 CS151 Lecture 2 21 April 1, 2004 CS151 Lecture 2 22 Robust Time ad Space Classes Robust time ad space classes: L = SPACE(log ) PSPACE = k SPACE( k ) P = k TIME( k ) EXP = k TIME(2 k ) Time ad Space Classes Problems i these classes: L : FVAL, iteger multiplicatio, most reductios aucklad sa fracisco pasadea athes davis oaklad PSPACE : geeralized geography, 2-perso games April 1, 2004 CS151 Lecture 2 23 April 1, 2004 CS151 Lecture 2 24 4
Time ad Space Classes P : CVAL, liear programmig, maxflow EXP : SAT, all of NP ad much more Relatioships betwee classes How are these four classes related to each other? Time Hierarchy Theorem implies PEXP P TIME(2 ) TIME(2 (2)3 ) EXP Space Hierarchy Theorem implies LPSPACE L=SPACE(log ) SPACE(log 2 ) PSPACE April 1, 2004 CS151 Lecture 2 25 April 1, 2004 CS151 Lecture 2 26 Relatioships betwee classes Easy: PPSPACE L vs. P, PSPACE vs. EXP? Relatioships betwee classes Useful covetio: Turig Machie cofiguratios. Ay poit i computatio represeted by strig: C = 1 2 i q i+1 i+2 m start cofiguratio for sigle-tape TM o iput x: q start x 1 x 2 x April 1, 2004 CS151 Lecture 2 27 April 1, 2004 CS151 Lecture 2 28 Relatioships betwee classes easy to tell if C yields C i 1 step cofiguratio graph: odes are cofiguratios, edge (C, C ) iff C yields C i oe step # cofiguratios for a 2-tape TM (work tape + read-oly iput) that rus i space t() x t() x Q x f() Relatioships betwee classes if t() = c log, at most x (c log ) x c 0 x c c log 1 k cofiguratios. ca determie if reach q accept or q reject from start cofiguratio by explorig cofig. graph of size k (e.g. by DFS) Coclude: L P April 1, 2004 CS151 Lecture 2 29 April 1, 2004 CS151 Lecture 2 30 5
Relatioships betwee classes if t() = c, at most x c x c 0 x c c 1 2 k cofiguratios. ca determie if reach q accept or q reject from start cofiguratio by explorig cofig. graph of size 2 k (e.g. by DFS) Relatioships betwee classes So far: L P PSPACE EXP believe all cotaimets strict kow LPSPACE, PEXP eve before ay metio of NP, two major usolved problems: Coclude: PSPACE EXP L = P P = PSPACE April 1, 2004 CS151 Lecture 2 31 April 1, 2004 CS151 Lecture 2 32 We do t kow how to prove L P But, ca idetify problems i P least likely to be i L usig P- completeess. eed stroger reductio (why?) yes f yes logspace reductio: f computable by TM that uses O(log ) space deoted L 1 L L 2 If L 2 is P-complete, the L 2 i L implies L = P (homework problem) f L o o 1 L 2 April 1, 2004 CS151 Lecture 2 33 April 1, 2004 CS151 Lecture 2 34 Circuit Value (CVAL): give a variable-free Boolea circuit (gates,,, 0, 1), does it output 1? Theorem: CVAL is P-complete. Proof: already argued i P L arbitrary laguage i P, TM M decides L i k steps Tableau (cofiguratios writte i a array) for machie M o iput w: height = time take = w c width = space used w c April 1, 2004 CS151 Lecture 2 35 April 1, 2004 CS151 Lecture 2 36 6
Importat observatio: cotets of cell i tableau determied by 3 others above it: April 1, 2004 CS151 Lecture 2 37 Ca build Boolea circuit STEP iput (biary ecodig of) 3 cells output (biary ecodig of) 1 cell!"# each output bit is some fuctio of iputs ca build circuit for each size is idepedet of size of tableau April 1, 2004 CS151 Lecture 2 38 Tableau for M o iput w w c copies of STEP compute row i from i-1!"#!"#!"#!"#!"# April 1, 2004 CS151 Lecture 2 39!"#!"#!"#!"#!"#!"#!"#!"#!"#!"#!"#!"#!"#!"#!"# & $$ %% This circuit C M, w has iputs w 1 w 2 w ad C(w) = 1 iff M accepts iput w. logspace reductio Size = O( w 2c ) April 1, 2004 CS151 Lecture 2 40 Aswer to questio Ca we evaluate a ode Boolea circuit usig O(log ) space? NO! (probably) CVAL i P if ad oly if L = P Paddig ad succictess Two cosequeces of measurig ruig time as fuctio of iput legth: paddig suppose L EXP, defie PAD L = { x# N : x L, N = 2 x k } same TM decides L (igore #s) ruig time ow polyomial! April 1, 2004 CS151 Lecture 2 41 April 1, 2004 CS151 Lecture 2 42 7
Paddig ad succictess coverse: succictess suppose L is P-complete ituitively, some iputs are hard -- require full power of P SUCCINCT L = iput ecoded i expoetially shorter form tha L if hard iputs ecodable this way, the cadidate to be EXP-complete April 1, 2004 CS151 Lecture 2 43 A EXP-complete problem succict ecodig for a directed graph G= (V = {1,2,3, }, E): a succict ecodig for a variable-free Boolea circuit: $$ $& &' ' +$ &' April 1, 2004 CS151 Lecture 2 44 $$ ()'* " +$ & ' A EXP-complete problem Succict Circuit Value: give a succictly ecoded variable-free Boolea circuit (gates,,, 0, 1), does it output 1? Theorem: Succict Circuit Value is EXPcomplete. Proof: i EXP (why?) L arbitrary laguage i EXP, TM M decides L i 2 k steps April 1, 2004 CS151 Lecture 2 45 A EXP-complete problem tableau for iput x = x 1 x 2 x 3 x :, Circuit C from CVAL reductio has size O(2 2k ). &) TM M accepts iput x iff circuit outputs 1 April 1, 2004 CS151 Lecture 2 46 A EXP-complete problem Ca ecode C succictly: $$ $& &' +$ & +$ &' if i, j withi sigle STEP circuit, easy to compute output if i, j betwee two STEP circuits, easy to compute output if oe of i, j refers to iput gates, cosult x to compute output April 1, 2004 CS151 Lecture 2 47 ' Summary Remaiig TM details: big-oh ecessary. First complexity classes: L, P, PSPACE, EXP First separatios (via simulatio ad diagoalizatio): P EXP, L PSPACE First major ope questios: L = P P = PSPACE First complete problems: CVAL is P-complete Succict CVAL is EXP-complete April 1, 2004 CS151 Lecture 2 48 8
Summary April 1, 2004 CS151 Lecture 2 49 9