CSE 200 Computability ad Complexity Wedesday, April 3, 2013 Lecture 2: Ucomputability ad the Halig Problem Istructor: Professor Shachar Lovett Scribe: Dogcai She 1 The Uiversal Turig Machie I the last class, we saw that the defiitio of TM is robust: chagig the alphabet, # of tapes, etc. do ot chage its computatio power. Hece, w.l.o.g. assume a TM M is composed of a alphabet Γ, a set of states Q, a trasitio fuctio δ : Q Γ 3 Q Γ 2 {L, S, R} 2, ad 3 tapes: INPUT, WORK, ad OUTPUT. A uiversal TM ca be viewed as a virtual machie. Defiitio 1 (Uiversal TM) A ormal TM: M def = (Γ; Q; δ) ca be treated as a strig i {0, 1}. A TM U is a uiversal machie if U( M, x) = M(x). A aalogy is a compiler. Note that a compiler is of fixed-size, but it ca compile code of ay legth. The iput to U is the descriptio (ecodig) of a TM, for example: Γ Q δ x ( stads for cocateatio i some uambiguous way). U simulates the iput TM. We ca have some covetio C about how to iterpret a arbitrary strig i {0, 1} as a TM. W.l.o.g., if a strig does t make ay sese i our pre-specified covetio C, we just treat it as a special TM that always output othig ad halts immediately after start. C does t map each strig i {0, 1} ito a meaigful TM just i the same way as some real-world programs do t compile. Ecodig C: TM Strigs {0, 1} M M M x The, how to costruct a uiversal TM U? Descriptio of U. U is simply a iterpreter. It has two parts of iput M ecoded as M ad iput x fed to M. U uses a work tape to keep track of the rutime cofiguratio of M. U simulates each step of M(x). At each such step, U scas M s curret cofiguratio o U s work tape ad it scas the trasitio fuctio i M to fid the accordig actio. U performs the correspodig actio by updatig the cofiguratio. Fially, output of M(x) is writte out to the output tape. Theorem 2 (Existece of U) The uiversal TM exists. 2 The Haltig Problem Defiitio 3 (Haltig problem) Halt : {0, 1} {0, 1} {0, 1} Halt( M, x) def = { 0 If M(x) halts. 1 If M(x) loops forever. 2-1
Ituitive implicatio: You ca prove whether some specific program (say, a program for fidig the shortest path) halts or ot evetually. But you caot write a uiversal program that works for all programs. This etails that o automatio for decidig TM haltig exists. Theorem 4 Halt is ot computable. Proof Assume towards cotradictio that Halt is computable by a TM M. Defie g : {0, 1} {0, 1} g(x) def = { 0 Halt(x, x) = 1 loops forever Halt(x, x) = 0 If Halt is computable the so is g. Suppose g is computable by a TM M. What is g( M )? If g( M ) = 0, the by the defiitio of g( ), Halt( M, M ) = 1, which meas that M( M ) loops forever. So g( M ) M( M ). If g( M ) loops forever, the by the defiitio of g( ), Halt( M, M ) = 0, which meas that M( M ) halts. So g( M ) M( M ). Digressio. The proof we saw is by a method called diagoilizatio. It was iveted by Georg Cator to prove that the cardiality of real umbers (R) is bigger tha that of ratioal umbers (Q). 3 Reductios Halt ca be used as the steppig stoe to prove that may other problems are icomputable, too. The basic idea is to obtai cotradictio by reducig Halt to a targeted problem. The assumptio that the target problem is computable will imply that Halt is computable, which is a cotradictio. Defiitio 5 (Busy Beaver) Cosider TM with just oe tape. Alphabet Γ def = {0, 1}. Have o iput. Iitial tape is 00... 0. If they halt, cotet of tape is output. BB() def = maximal umber of 1 s o the tape output by a haltig TM with states. Historical otes. 1 A busy beaver is a TM that attais the maximum umber of steps performed or umber of o-blak symbols fially o the tape amog all TMs i a certai class. The TMs i this class must meet certai desig specificatios ad are required to evetually halt after beig started with a blak tape. A busy beaver fuctio quatifies these upper limits o a give measure, ad is a o-computable fuctio. I fact, a busy beaver fuctio ca be show to grow faster asymptotically tha ay computable fuctio. The cocept was first itroduced by Tibor Radó [2]. For example, let Σ(, 2) deote the largest # of 1 s pritable by a -state, 2-symbol (i.e., {0, 1}) TM started o a iitially blak tape before haltig. Table 1 gives a sese how fast this fuctio skyrockets. Theorem 6 BB is ucomputable. 1 From Busy Beaver s Wiki etry: http://e.wikipedia.org/wiki/busy_beaver. 2-2
Table 1: Values of Σ(, 2) 2-state 3-state 4-state 5-state 6-state 2-symbol 4 6 13 4, 098 3.5 1018267 Proof Assume BB is computable by a TM M with k states. Let DBL be a TM with k states that performs the followig operatio 1111 0 0 2 111111 0 0 Let be large eough to be specific later. Cosider the followig Turig machie M : Writes 1 s o tape : 00 Apply DBL: Apply M: Apply M agai: 11111 0. 2 1111 0 11 1 0. BB(2) 11 11 0. BB(BB(2)) 11 11 0. How may states does M have? The first step takes states; the secod k states; ad the third ad fourth k states. So i total + 2k + k states. We get that BB( + 2k + k ) BB(BB(2)). Note that BB is mootoe icreasig because BB( + 1) BB() for ay N. So BB(2) + 2k + k which is clearly impossible if is large eough (say, > 2k + k ). It turs out that there are o o-trivial computable fuctio of the output of a computatio. Theorem 7 (Rice s Theorem) Let f : {0, 1} {0, 1} {0, 1} be a computable fuctio so that f( M, x) depeds just o M(x). The f must be a costat fuctio. Rice s Theorem is i Exercise 1.12 i [1]. Cojecture 8 (The 3 + 1 Problem, aka Collatz cojecture) Cosider the followig fuctio f : N N. { /2 eve f() = 3 + 1 odd It is cojectured that for ay startig value, if we apply f eough times we will reach 1 evetually (ote that f(f(f(1))) = 1, e.g. 1 is i a cycle). This was verified for may values of, but it is ukow if it is true for all. Defiitio 9 (The Geeralized 3 + 1 Problem) Iput: m, a 0, b 0, a m 1, b m 1 atural umbers. Cosider a fuctio of the form a 0 + b 0 mod m 0 a 1 + b 1 mod m 1 f() =... a m 1 + b m 1 mod m m 1 2-3
For example, lettig m = 2 ad a 0 = 1 2, b 0 = 0, a 1 = 3, a 1 = 1 yields the 3 + 1 problem. Theorem 10 Cosider the followig fuctio F (m, a 0, b 0,..., a m 1, b m 1 ) = the geeralized 3 + 1 problem always reaches 1 from ay startig value is ucomputable. Here is aother example of a ucomputable fuctio. Defiitio 11 (Hilbert s 10 th problem 2 ) Give a multivariate polyomial p(x 1,..., x ) with iteger coefficiets, figure out if p has a iteger solutio. Theorem 12 This is ucomputable. 4 Time Complexity Defiitio 13 (Time complexity) TM M has time complexity T : N N if for all iput x {0, 1}, M(x) always halts; M(x) halts after at most T ( x ) steps, where x deotes the legth of the iput x. f : {0, 1} {0, 1} has time complexity T if there exists a TM M computig f which has time complexity T. Defiitio 14 (TIME) TIME(T ()) def = fuctios computable i time O(T ()). For example, the problem of decidig whether a give graph is bipartite (f : graph Is it bipartite?) is i TIME( 2 ). Be cautious that this is a upperboud istead of a lowerboud. Defiitio 15 (P ad EXP) P def = c 1 TIME( c ). E.g., shortest paths, liear programmig. EXP def = c 1 TIME(2 c ). E def = c 1 TIME(2 c ). E.g., SAT. Remark #1: Why do we care about polyomial-time computatio? Why care about polyomial time ad ot specifically o liear, quadratic, or cubic time? The aswer is, i reality, the degrees of polyomials that upperboud the ruig time of problems i P are usually very small. P captures the ituitio of efficietly computatio. However, i some scearios, people do wat to fid algorithms that ru i a more restricted time boud, say, liear. For example, algorithms for property testig may require sub-liear time, which meas that it caot eve read the whole iput. Remark #2: The defiitio of EXP is robust. Some graph algorithms, for -vertex graph takes 2 time. How do we ecode the graph (say, it s sparse) matters: List of edges: (algorithm rus i O(2 ) steps). Adjacecy matrix: 2 (algorithm rus i O(2 ) steps). EXP is more robust tha E i that a problem caot escape EXP eve its descriptio s size is blow up from to 2. Usually, we take atural ecodig for a problem ad they fit ito EXP. 2 http://e.wikipedia.org/wiki/hilbert s_problems 2-4
Refereces [1] Sajeev Arora ad Boaz Barak. Computatioal Complexity - A Moder Approach. Cambridge Uiversity Press, 2009. [2] Tibor Radó. O o-computable fuctios. Bell System Techical Joural, 41:877 884, 1962. 2-5