Computability Theory

Computability Theory Cristian S. Calude and Nicholas J. Hay May June 2009 Computability Theory 1 / 155

1 Register machines 2 Church-Turing thesis 3 Decidability 4 Reducibility 5 A definition of information 6 Quantum information Computability Theory 2 / 155

Objectives On completion, students will be able to: explain the theoretical limits on computational solutions of undecidable and inherently complex problems, describe concrete examples of computationally undecidable or inherently infeasible problems from different fields, understand formal definitions of machine models, classical and unconventional, prove the undecidability or complexity of a variety of problems, understand the issue of whether there are limits of computability. Computability Theory 3 / 155

Bibliography M. Sipser. Introduction to the Theory of Computation, PWS 1997. (textbook) N. J. Hay. Register machine simulation software, http://www.cs.auckland.ac.nz/~nickjhay/register. Computability Theory 4 / 155

Supplementary bibliography 1 C. S. Calude, Elena Calude, M. J. Dinneen. A new measure of the difficulty of problems, Journal for Multiple-Valued Logic and Soft Computing 12 (2006), 285 307. C. S. Calude, M. J. Dinneen, K. Svozil. Reflections on quantum computing, Complexity 6, 1 (2000), 35 37. C. S. Calude, M. A. Stay. Most programs stop quickly or never halt, Advances in Applied Mathematics, (2007), 40 (2008), 295 308. C. S. Calude. Incompleteness: A Personal Perspective, Google Technical Talk, Mountainview, USA, 4 November 2008, http://www.youtube.com/watch?v=tyjmit422yq. J. P. Dowling. To compute or not to compute? Nature 439, 23 February 2006, 919 920. Computability Theory 5 / 155

Supplementary bibliography 2 Hypercomputation Research Network, http://www.hypercomputation.net. John C. Martin. Introduction to Languages and the Theory of Computation, Mc Graw-Hill, Boston, 2003, third edition. K. Svozil. Quantum algorithmic information theory, J. UCS 2,5 (1996), 311 346. A. Turing. Computing machinery and intelligence, Mind LIX (236) (1950), 433 460, doi:10.1093/mind/lix.236.433. S. Wolfram. The New Kind of Science, Wolfram Media, 2002, http://www.wolframscience.com. Computability Theory 6 / 155

Assignments, test and exam Three assignments worth 20% in total Assignment 3 : 22 May, due 8 June, worth 5% Bonus questions : each worth 1% Test : 30%. Exam : 50%. 13 June. You will be given six questions and asked to choose and answer four of them. Each question is worth 25 marks. Computability Theory 7 / 155

What is computability? What can computers compute? Are there any limitations? If so, exactly what? To answer these questions, we will construct formal models of computers and prove theorems about them. Computers compute using programs. A formal model of a computer is really a formal model of a programming language. To make things easy we want our programming language to be as simple as possible, without sacrificing power. We will present the register machine model. This abstracts from and simplifies modern CPU instruction architectures: we will assume unbounded memory and remove all but the most central instructions. Computability Theory 8 / 155

Example: doubling a register 1 0 A: if (x1!=0) goto B 1 halt 2 B: x1-- 3 y++ 4 y++ 5 if (y!=0) goto A The registers x1 and y are variables which can store natural numbers. If we start running this program with x1 set to 13 and y set to 0, it will halt with y set to 26. Interpret if, ++, and -- as in Java code. halt halts the program. goto L jumps execution to the instruction labelled L. Computability Theory 9 / 155

Example: doubling a register 2 0 A: if (x1!=0) goto B 1 halt 2 B: x1-- 3 y++ 4 y++ 5 if (y!=0) goto A The program starts execution at instruction 0. Since x1 is 13 execution continues at label B. Instructions 2 through 4 decrease x1 by 1 and increase y by 2. Instruction 5 jumps back to instruction 0 since y is nonzero (this is really an unconditional goto), and the loop repeats. The above loops 13 times, until x1 is decremented all the way to 0. At this point instruction 1 runs, and we halt with y equal to 26. Computability Theory 10 / 155

Register machine model 1 A register machine has a number of registers, each of which stores a natural number. We will name registers y, x1, x2, etc. They are similar to variables of type int, except there is no upper bound to their value. y 256 x1 0 x2 2 x3 42 x4 137...... As seen in the previous example, these registers can change value as the program executes. We will assume all registers but the input registers start with the value 0. Computability Theory 11 / 155

Register machine model 2 A program is a list of instructions. There are are four kinds: 1 xi++ This increments the value of register xi. 2 xi-- This decrements the value of register xi. If xi is already zero, however, it stays zero. 3 halt Stops program execution. 4 if (xi!=0) goto L If the value of register xi is not zero, execution continues the instruction labelled L. Otherwise it continues at the next instruction. An instruction prefixed by L: is labelled by the label L. Execution starts at the first instruction. Unless a goto jumps, we execute the next instruction until we run out or hit a halt. Computability Theory 12 / 155

Semantics: computing partial functions 1 Register machine programs can be used to compute partial functions mapping natural numbers to natural numbers. A program computes a partial function f : N N if when given x in the x1 register it outputs f (x) in the y register if f (x) is defined, otherwise the program doesn t halt. We can generalise this to multiple arguments functions, e.g. f (x 1, x 2 ) = 2x 1 + x 2, by placing the inputs in the registers x1, x2, etc. Computability Theory 13 / 155

Semantics: computing partial functions 2 For example, we would implement f (x) = 2x by a program which, when given x in the x1 register outputs 2x in the y register. The program from before implements f (x) = 2x: 0 A: if (x1!=0) goto B 1 halt 2 B: x1- - 3 y++ 4 y++ 5 if (y!=0) goto A Computability Theory 14 / 155

Example: modulo two 0 x2++ 1 A: if (x1!=0) goto B 2 halt 3 B: x1-- 4 if (y!=0) goto C 5 y++ 6 if (x2!=0) goto A 7 C: y-- 8 if (x2!=0) goto A This program computes the function { 0, if x is even, f (x) = 1, if x is odd. The register x2 is incremented on line 0 to make lines 6 and 8 unconditional gotos. Lines 4-8 alternate y between the values 0 and 1. Computability Theory 15 / 155

Semantics: computing sets by deciding them To describe a set S N of natural numbers we can give a method for computing whether x S for any x N. A program decides (or is a decider for) a set S N if when given x in the x1 register it outputs 1 in the y register if x S, and 0 in the y register if x / S. Equivalently, a program decides a set S if it implements its characteristic function χ S. A decidable set is one which has program deciding it. Our modulo two example before decides the set of odd numbers: x modulo two equals 1 if x is odd, and 0 otherwise. Computability Theory 16 / 155

Semantics: computing sets by enumerating them We can also describe a set by listing (or enumerating) its elements. A program enumerates a set S N if S = {f (x) : x N} = ran f where f is the function the program implements. A computably enumerable or Turing-recognisable set is one which has program enumerating it. The program to the right enumerates the set of odd numbers, as it implements the function f (x) = 2x + 1. 0 y++ 1 A: if (x1!=0) goto B 2 halt 3 B: x1-- 4 y++ 5 y++ 6 if (y!=0) goto A Computability Theory 17 / 155

Formal register programs Our previous descriptions of register machine operations were informal. We can give a mathematically precise description, like those for DFAs and Turing machines, although slightly more complex. First we will give a mathematical representation of a register machine program. Next we give a mathematical representation of a register machine running a program: both its state and its transition function. Finally, we can formally define what it means for a register machine program to compute a partial function, decide a set, and enumerate a set. Computability Theory 18 / 155

Formal register programs: encoding programs We can encode instructions by tuples of three numbers. These record instruction type, register number, and jump destination. In detail, each instruction corresponds to an element of N 3 by the translation: xi++ (0, i, 0) xi-- (1, i, 0) halt (2, 0, 0) if (xi!=0) goto L (3, i, #L) where #L is the instruction number corresponding to label L. For example, x2++ will be encoded by (0, 2, 0), and if (x4!=0) goto A by (2, 4, 13) if A labels instruction 13. A register machine program p is a sequence of such tuples i.e. p (N 3 ). Computability Theory 19 / 155

Formal register programs: example program encoding For example, on the left we have a program in our informal notation, on the right our formal representation of it. 0 A: if (x1!=0) goto B (3,1,2) 1 halt (2,0,0) 2 B: x1-- (1,1,0) 3 y++ (0,0,0) 4 y++ (0,0,0) 5 if (y!=0) goto A (3,0,0) Note that register y is register 0, and that the labels A and B correspond to instruction numbers 0 and 2 respectively. Computability Theory 20 / 155

Formal register programs: register machine state The state of a register machine at any point in time is the current instruction and the value of all its registers. We will store the register values as a finite sequence of numbers, assuming all registers not mentioned are zero. For example, the sequence 3, 10, 0, 32 means register y is 3, x1 is 10, x2 is 0, x3 is 32, and all others are 0. Given a sequence of register values r N, we denote the value of ith register to be r(i). For i, v N and a sequence of register values r N, denote by r[i v] the sequence of register values got by taking r and setting the ith register s value to v. Thus, the complete machine state is an element (n, r) of N N, where n is the current instruction number, and r the sequence of register values. Computability Theory 21 / 155

Formal register programs: instruction transition function When a register machine executes an instruction its state (instruction number, register values) changes. For each instruction (x, y, z), we define the instruction transition function δ (x,y,z) which when given the current state (n, r) N N outputs the state (n, r ) after the instruction has executed. (n + 1, r[y v + 1]), if x = 0, v = r[y], (n + 1, r[y v 1]), if x = 1, v = r[y], δ (x,y,z) (n, r) = (n + 1, r), if x = 3, r[y] = 0, where v 1 = max{0, v 1}. (z, r), if x = 3, r[y] 0, (n, r), otherwise, Computability Theory 22 / 155

Formal register programs: program transition function Finally, given a program we can define the register machine s program transition function. At each time step, if the current instruction number is valid the machine looks up the current instruction and executes it, otherwise it does nothing. The program transition function δ p : N N N N is defined from the program p: { δ δ p (n, r) = p(n) (n, r), if n < p, (n, r), otherwise. Computability Theory 23 / 155

Formal register programs: partial functions, sets Define r 0 = 0x (0 and x are the values of registers y and x1), and δ t p(n, r) = δ p (δ p (... δ p (n, r)...)), t times. The program p computes the partial function f when for all x N with if f (x) is defined there exists a t N such δp(0, t r 0 ) = δp t+1 (0, r 0 ) = (n, r) and r(0) = f (x), if f (x) is not defined then there is no t such that δp(0, t r 0 ) = δp t+1 (0, r 0 ). As before we can define when a program decides/enumerates a set. A program p decides a set S if it implements its characteristic function χ S. A program p enumerates a set S if p implements the function f and S = ran f. Computability Theory 24 / 155

More examples: addition 1 0 A: if (x1!=0) goto C 1 B: if (x2!=0) goto D 2 halt 3 C: x1-- 4 y++ 5a x3++ 5b if (x3!=0) goto A 6 D: x2-- 7 y++ 8a x3++ 8b if (x3!=0) goto A This implements f (x 1, x 2 ) = x 1 + x 2. Lines 5a/5b and 8a/8b both use our trick for an unconditional goto: make sure some other register is nonzero then condition on it being nonzero. This would be clearer if we just used goto A. Computability Theory 25 / 155

More examples: addition 2 0 A: if (x1!=0) goto C 1 B: if (x2!=0) goto D 2 halt 3 C: x1-- 4 y++ 5 goto A 6 D: x2-- 7 y++ 8 goto B The instruction goto L moves execution to label L unconditionally. Computability Theory 26 / 155

Macro: goto We define goto L as a macro that expands into: 0 xi++ 1 if (xi!=0) goto L where xi is a register not used by the main program. We didn t add goto as an independent instruction as we didn t need to. This definition is inefficient: it increments xi every time the goto is executed when we only need to do this once. A more efficient definition of goto would have xi incremented once, at the very start of the program. This is more complex, and its definition is not local: we have to add an instruction at the start of the program. Computability Theory 27 / 155

More examples: subtraction Let f (x 1, x 2 ) = x 1 x 2, where x 1 x 2 = max{0, x 1 x 2 }. Modifying only instruction 7 of the addition program gets us subtraction: 0 A: if (x1!=0) goto C 1 B: if (x2!=0) goto D 2 halt 3 C: x1- - 4 y++ 5 goto A 6 D: x2- - 7 y-- 8 goto B Computability Theory 28 / 155

Example: multiplication 1 We can implement multiplication by: 0 A: if (x1!=0) goto C 1 halt 2 C: x1-- 3 addto y x2 4 goto A addto y x2 adds the value of x2 to y without changing the value of x2. How is addto implemented? Computability Theory 29 / 155

Macro: addto addto y xj: 0 C: if (xj!=0) goto D 1 goto E 2 D: xj-- 3 t++ 4 y++ 5 goto C 6 F: t-- 7 xj++ 8 E: if (t!=0) goto F You can verify these instructions will add xj to y without changing the value of xj. The register t is a register not used by the rest of the program. We use it to save the value of xj. Computability Theory 30 / 155

Example: multiplication 2 0 A: if (x1!=0) goto B 1 halt 2 B: x1- - 3a C: if (x2!=0) goto D 3b goto E 3c D: x2-- 3d x3++ 3e y++ 3f goto C 3g F: x3-- 3h x2++ 3i E: if (x3!=0) goto F 4 goto A If we expand the addto macro we get this program. Instructions 0,1,2, and 4 implement a for loop: the block of instructions 3a-3i is repeated x1 times. The block of instructions 3a-3i adds x2 to y. This is just the addto macro expanded. Computability Theory 31 / 155

Closure under composition 1 Function composition is a powerful method for constructing complex functions from simple ones. For example, if we have the function f (x) = x 3 and g(x) = 2x + 1, then their composition gives us (f g)(x) = (2x + 1) 3. Intuitively, we should be able to get a program for f g by sticking together one for f and one for g: [program computing g] move y x1 [program computing f] Computability Theory 32 / 155

Closure under composition 2 Again: [program computing g] move y x1 [program computing f] [program computing g] is shorthand for inserting the code that computes g into the program. move y x1 copies the value in register y, where g places its output, into register x1, where f expected its input. Computability Theory 33 / 155

Closure under composition 3 The basic idea is sound, but there are a couple of technical difficulties. The two programs may use the same label for different code locations. Solution: rename labels e.g. rename f s L label to fl, and g s to gl. The program for g(x) can make arbitrary registers nonzero, e.g. x2, but the program for f (x) may assume these are zero. Solution: clear the registers used by g(x). Computability Theory 34 / 155

Closure under composition 4 This leads to the slightly modified scheme: [program computing g with labels adjusted] move y x1 zero y zero x2... zero xk [program computing f with labels adjusted] zero y zeros the y register. xk is the last register modified by the program for g. Computability Theory 35 / 155

Example: The Fibonacci Function 1 The Fibonacci sequence can be represented by a function f : N N defined by f (0) = 0, f (1) = 1 and f (x + 2) = f (x) + f (x + 1). To implement this we will use registers x2 and x3 to store f (i) and f (i + 1). We will loop over i to compute f (x). Computability Theory 36 / 155

Example: The Fibonacci Function 2 A high-level register program to implement this: x3++ A: if (x1==0) goto F copy x3 x4 addto x2 x3 copy x4 x2 x1-- goto A F: copy x2 y Macros can be written for copy x y and if (x==0) goto L. The next slide expands this program, making a few optimisations. Computability Theory 37 / 155

Example: The Fibonacci Function 3 The program below computes the Fibonacci function. x3++ A: if (x1==0) goto F B: if (x3==0) goto C x3- - x2++ x4++ goto B C: if (x2==0) goto D x2- - x3++ goto C D: if (x4==0) goto E x4- - x2++ goto D E: x1- - goto A F: if (x2==0) goto end x2- - y++ goto F end: halt Computability Theory 38 / 155

String register machines Register machines work with numbers, but DFAs and Turing machines work with strings. A variant register machine computes on strings. We will see that register machines computing on strings are equivalent in power to those computing on natural numbers, and that both are equivalent in power to Turing machines. Computability Theory 39 / 155

Example: reversing a string 1 0 A: pop x1 E B C 1 B: push y 0 2 goto A 1 C: push y 1 2 goto A 5 E: halt The registers now hold binary strings such as 000101, 11, and ɛ, rather than numbers. pop x1 E B C first checks if x1 is empty. If it is, it jumps to label E and halts. Otherwise it removes the last character from x1 and jumps to B if it is 0, and C if it is 1. push y 0 appends a 0 to the end of y. Computability Theory 40 / 155

Example: reversing a string 2 0 A: pop x1 E B C 1 B: push y 0 2 goto A 1 C: push y 1 2 goto A 5 E: halt Characters are repeatedly removed from the end of x1 and placed at the end of y. A moment s thought will confirm this reverses the string originally in x1. Computability Theory 41 / 155

String register machine model 1 String register machines are similar to (numeric) register machines. They have registers, this time storing binary strings. y 00001001 x1 011 x2 x3 111111 x4 101010...... As time goes on these registers can change value. Computability Theory 42 / 155

String register machine model 2 A program is a list of instructions, of the following four types: 1 push xi v Append v (either 0 or 1) to the end of register xi. 2 goto L Continue execution at the instruction labelled by L. 3 pop xi A B C If register xi is empty then goto A. Otherwise, remove the last character from xi: if it s 0 goto B, if it s 1 goto C. 4 halt Halt execution. Computability Theory 43 / 155

Semantics: functions and sets String register programs define functions and sets in the same way as the numerical case. A string register program computes a partial function f : {0, 1} {0, 1} if when given x in the x1 register it outputs f (x) in the y register. A string register program decides a set S {0, 1} if when given x in the x1 register it outputs 1 in the y register if x S, and 0 in the y register if x / S. A string register program enumerates a set S {0, 1} if S = {f (x) : x {0, 1} } = ran f where f is the function the program implements. Computability Theory 44 / 155

Example: doubling a string 1 To implement the doubling function f (x) = xx, we would like to write: copy x1 y copy x1 y where copy x1 y appends the contents of x1 onto y, without reversing it or destroying the original value in x1 (it is easier to reverse and destroy the original value: 5 lines of code suffice). How to implement copy x1 y? Computability Theory 45 / 155

Example: doubling a string 2 Implement the macro copy x y by 0 A: pop x D B C 1 B: push t 0 2 goto A 3 C: push t 1 4 goto A 5 D: pop t G E F 6 E: push x 0 7 push y 0 8 goto D 9 F: push x 1 10 push y 1 11 goto D 12 G: where register t is unused and assumed to be empty. This first moves the string in x into temporary register t, reversing it in the process. It then removes characters from t, restoring them to x and adding them to y. Computability Theory 46 / 155

Example: doubling a string 3 0 A: pop x1 D B C 1 B: push x2 0 2 push x3 0 3 goto A 4 C: push x2 1 5 push x3 1 6 goto A 7 D: pop x2 G E F 8 E: push y 0 9 goto D 10 F: push y 1 11 goto D 12 G: pop x3 X H I 13 H: push y 0 14 goto G 15 I: push y 1 16 goto G 17 X: halt This is what it looks like without the copy macro (optimised a little). First, this makes two reversed copies of x1 in the registers x2 and x3. Finally, it copies x2 and then x3 into y. Computability Theory 47 / 155

Regular languages In previous lectures we introduced DFAs, which recognise exactly the regular languages. Register machines can recognise regular languages too (among many other languages). Recall regular languages can be inductively defined by regular expressions. Starting from the basic one string regular languages {0} and {1} (more generally, {σ} for each character σ Σ) we build up all others by union, complement, concatenation, and Kleene star. Computability Theory 48 / 155

Regular languages: basic languages The regular languages {0} and {1} are easy to recognise. For example {0}: 0 pop x1 R B R 1 B: pop x1 A R R 2 A: push y 1 3 halt 4 R: push y 0 5 halt Computability Theory 49 / 155

Regular languages: union 1 Closure under union is a special case of function composition: given languages L 1 and L 2, the characteristic function of their union L 1 L 2 satisfies: χ L1 L 2 (x) = (χ L1 (x), χ L2 (x)) where (a, b) = 1 if either a = 1 or b = 1, otherwise (a, b) = 0. Computability Theory 50 / 155

Regular languages: union 2 Concretely, L 1 L 2 is decided by: copy x1 t [program computing χ L1 pop y1 C C A C: copy t x1 [program computing χ L2 pop y1 R R A R: empty y push y 0 halt A: empty y push y 1 where empty y empties y. with labels adjusted] with labels adjusted] Computability Theory 51 / 155

Regular languages: concatenation To see whether x L 1 L 2 we need to try all possible splittings x = yz and check whether one has y L 1 and z L 2. copy x1 tleft L: copy tleft x1 [program computing χ L1 pop y1 E E C C: copyrev tright x1 [program computing χ L2 pop y1 E E A E: pop tleft R E0 E1 E0: push lright 0 goto L E1: push lright 1 goto L with labels adjusted] with labels adjusted] Computability Theory 52 / 155

Regular languages: Kleene star Closure under Kleene star is similar in spirit to closure under composition, except we have to allow the string to be divided into many pieces. To see whether x L we need to try all possible splittings x = y 1... y k for all k x and check whether y i L 1 for all i. Each splitting of a string, for example 001010101 into 00,101,0,101, corresponds to a binary string which we mark each cutting point, in this example 010011001. There are, in fact, 2 x 1 of these (ignoring x = 0). One way to decide L, then, is when given x {0, 1} to enumerate all splitting strings as above, then test whether each of them divide x into strings found in L. If some splitting does, accept, if none do, reject. Computability Theory 53 / 155

Beyond regular languages String register machines can recognise languages DFAs cannot. For example, the language of strings with the same number of 0 s as 1 s. One way to implement this decider. First sort the 0 s and 1 s of the input into separate registers. Then remove elements from both registers one by one to check whether there are the same number of 0 s and 1 s. Computability Theory 54 / 155

Beyond regular languages String register machines can recognise languages DFAs cannot. For example, the language of strings with the same number of 0 s as 1 s. One way to implement this decider. First sort the 0 s and 1 s of the input into separate registers. Then remove elements from both registers one by one to check whether there are the same number of 0 s and 1 s. Computability Theory 55 / 155

Example: equal 0 s and 1 s 0 S: pop x1 C M0 M1 1 M0: push x2 0 2 goto S 3 M1: push x3 1 4 goto S 5 C: pop x2 E NE NE 6 NE: pop x3 R C C 7 E: pop x3 A R R 8 A: push y 1 9 halt 10 R: push y 0 11 halt Lines 0-4 sort the characters of x1: all the 0 s into x2, all the 1 s into x3. Lines 5-7 check to see whether x2 and x3 have the same number of elements by first trying to remove an element of x2 then trying to remove one from x3. Labels A and R accept and reject the input, respectively. Computability Theory 56 / 155

String and numeric register machines equivalent 1 We described two different register machines, working with numbers and strings. How are they related? They are equivalent: programs from one can be translated into equivalent programs for the other. The easier direction is showing numerical register machine programs can be translated into string register programs. We ll do that. The key ingredient for the translation is some way of translating numbers into binary strings. We use the obvious plan: write the number in binary. For example, 7 translates to 111, 12 to 1100, and 0 to ɛ. (For translating binary strings into numbers, think about the sequence ɛ, 0, 1, 00, 01, 10, 11, 100,... ) Computability Theory 57 / 155

String and numeric register machines equivalent 2 We need only translate each of the 4 numerical instructions into string register machine code. We don t need to do anything for halt, so that leaves 3: xi++ xi-- if (xi!=0) goto L Computability Theory 58 / 155

String and numeric register machines equivalent 3 x++. The following routine will add 1, in binary, to the contents of the register x using temporary register temp: loop: pop x1 nocarry nocarry carry carry: push temp 0 goto loop nocarry push x1 1 move: pop temp end w0 w1 w0: push x1 0 goto move w1: push x1 1 goto move end: halt Computability Theory 59 / 155

String and numeric register machines equivalent 4 x-- can be implemented by using the same trick as in ones complement subtraction (left as an exercise). if (x!=0) goto L can be similarly translated (left as an exercise). Computability Theory 60 / 155

String register machines and TMs equivalent One can translate Turing machines into equivalent string register programs. This is simply a coding exercise. Basically, use one register to store the contents of the tape of the left of the tape head, another to store the contents under and after it (in reverse). Another register stores the current tape state. For each pair (q, γ) Q Γ implement code to modify the tape registers (write the new symbol and move left/right), and the state register (write the next state). Implement a central loop that jumps to the right code location depending on the current machine state. Computability Theory 61 / 155

Universal register machines One can implement a universal string register machine program. This, when given an encoding of a string register program in one of its register, and an input for it in another, will simulate the program and write its output to y. Outline of construction: prefix-free encoding of strings (x 1 0x 2 0... x n 1 is convenient for local modifications), prefix-free encoding of numbers (n coded as 0 n 1), prefix-free encoding of instructions (000 i 1 for goto i, 01v0 x 1 for push x v, 100 r 10 i 10 j 10 k 1 for pop r i j k, 11 for halt). One register pair holds the program (a pair storing left and right halves as the program reads it), another pair holds the simulated registers (the values concatenated together), another the program counter. Computability Theory 62 / 155

Equivalence between various models We have presented the register machine model and the Turing machine model and have shown them to be equivalent in power. There are several variants of the Turing machine model, generative grammars, λ-calculus, etc. that share the essential feature of having unrestricted access to unlimited memory. Remarkably, all models with that feature turn out to be equivalent in computational power. Computability Theory 63 / 155

Church-Turing thesis 1 The Church-Turing thesis (formerly commonly known simply as Church s thesis) says that every intuitively computable function is computable by some Turing machine. There are conflicting points of view about the Church-Turing thesis... Computability Theory 64 / 155

Church-Turing thesis 2 (1) There has never been a proof for Church-Turing thesis; in fact it cannot be proven. The evidence for its validity comes from the fact that every realistic model of computation, yet discovered, has been shown to be equivalent. (2) A device which could answer questions beyond those that a Turing machine can answer would disprove the Church-Turing thesis. No such convincing device has been found (yet). Hypercomputation is searching for such a device. Computability Theory 65 / 155

Physical Church-Turing thesis or Here are two formulations out of a much larger set: every real-world computation can be translated into an equivalent computation involving a Turing machine any physical process can be simulated by some Turing machine. Both formulations would be falsified by the existence of genuine random processes. A. Turing: a machine with a random element can do more than a Turing Machine (see Turing 1950, pp. 438 439). What is more? Computability Theory 66 / 155

Wolfram s principle of computational equivalence Almost all processes that are not obviously simple can be viewed as computations of equivalent sophistication. In detail, the principle of computational equivalence says that: the systems found in nature can perform computations up to a maximal ( universal ) level of computational power, and that most systems do in fact attain this maximal level of computational power, so most systems are computationally equivalent. For example, the workings of the human brain or the evolution of weather systems can, in principle, compute the same things as a computer. Computability Theory 67 / 155

Encodings 1 We have shown how to code register machine programs. In a similar way one can encode Turing machines by strings over a (large) alphabet Σ, and then, using a bijection between Σ and {0, 1}, by binary strings. There are various possibilities to design such encodings. A Turing machine can always translate one such encoding into another. In what follows we will just fix one encoding and denote by M the string coding/representing the Turing machine M. If we have several Turing machines M 1, M 2,..., M k, we denote their encoding into a single string M 1, M 2,..., M k. Computability Theory 68 / 155

Encodings 2 We will work with a prefix-free encoding. The prefix-code of a string x 1 x 2... x n 1 x n is the string x 1 0x 2 0... x n 1 0x n 1. For example, the prefix-code of the empty string is 1; the prefix-code of the string 101 is 100011. The length of the prefix-code of a string of length n is 2n. A Turing machine can compute from the prefix-free code M 1, M 2,..., M k each of its components M 1, M 2,..., M k. Computability Theory 69 / 155

Decidable languages 1 In what follows we will work with high-level descriptions of Turing machines, i.e. we will use English prose to describe algorithms (and hence we ignore the implementation details). Recall that a language Turing-recognisable or computably enumerable if some Turing machine recognises it. Call a language Turing-decidable or simply decidable, or recursive, or computable if some Turing machine decides it. Computability Theory 70 / 155

Decidable languages 2 A DFA = { B, w : B is a DFA that accepts the input string w}. Theorem 4.1 A DFA is decidable. Proof. A TM that decides A DFA : M = On input B, w, where B is a DFA and w is a string: 1. Simulate B on input w. 2. If the simulation ends in an accept state, accept. If it ends in a nonaccept state, reject. Computability Theory 71 / 155

Decidable languages 3 A NFA = { B, w : B is an NFA that accepts the input string w}. Theorem 4.3 A NFA is decidable. A REX = { R, w : R is a regular expression that accepts the Theorem 4.4 A REX is decidable. input string w}. Computability Theory 72 / 155

Decidable languages 4 E DFA = { A : A is a DFA and L(A) = }. Theorem 4.4 E DFA is decidable. EQ DFA = { A, B : A and B are DFAs and L(A) = L(B)}. Theorem 4.5 EQ DFA is decidable. Computability Theory 73 / 155

The halting problem 1 We prove one of the most philosophically important theorems of the theory of computation: There is a specific problem that is algorithmically unsolvable. What sort of problems are unsolvable by computer? Are they esoteric, dwelling only in the minds of theoreticians? Even some ordinary problems that people want to solve turn out to be computationally unsolvable. For example, the general problem of software verification is unsolvable! Computability Theory 74 / 155

The halting problem 2 We establish the undecidability of the acceptance problem, i.e. the problem of determining whether a Turing machine accepts a given input string. Formally: A TM = { M, w : M is a TM and M accepts w}. Theorem 4.11 A TM is undecidable. Computability Theory 75 / 155

The halting problem 3 Intermediate step: A TM Turing-recognisable. Proof. A TM is Turing-recognisable by the TM U: U = On input M, w, where M is a TM and w is a string: 1. Simulate M on input w. 2. If M ever enters its accept state, accept; if M ever enters its reject state, reject. U loops on input M, w if M loops on w, so U does not decide A TM! But is there a TM deciding A TM? U is interesting in its own right: it is a universal Turing machine because it is capable of simulating any other Turing machine from the description of that machine. Computability Theory 76 / 155

The halting problem 4 Proof of Theorem 4.11. Assume, to obtain a contradiction, that A TM = { M, w : M is a TM and M accepts w}. is decidable, say, by decider H. On input M, w, where M is a TM and w is a string: H halts and accepts if M accepts w, H halts and rejects if M does not accept w, i.e. if 1 M halts and does not accept w, or 2 M does not halt on w. Computability Theory 77 / 155

The halting problem 5 Formally, H( M, w ) = { accept, if M accepts w, reject, if M does not accept w. accept, if M accepts w, = reject, if M halts and does not accept w, reject, if M does not halt on w. Computability Theory 78 / 155

The halting problem 6 Construct a new TM D with H as a subroutine: D calls H to determine what M does on its own description w = M and D rejects if M accepts and D accepts if M does not accept. D = On input M, where M is a TM: 1. Run H on input M, M. 2. If H accepts, reject; if H rejects, accept. D( M ) = D( D ) = a contradiction. { accept, if M does not accept M, reject, if M accept M. { accept, if D does not accept D, reject, if D accept D, Computability Theory 79 / 155

The halting problem 7 We prove the undecidability of the most (in)famous problem in computer science: the problem of determining whether a Turing machine halts on a given input string. Formally: HALT TM = { M, w : M is a TM and M halts on input w}. Theorem 5.1 HALT TM is undecidable. Proof. The proof is by contradiction: we assume that HALT TM is decidable and use that assumption to show that A TM is decidable, contradicting Theorem 4.11. The key idea is to show that A TM is reducible to HALT TM. Computability Theory 80 / 155

The halting problem 7 Proof of Theorem 5.1 continued. We assume that TM R decides HALT TM and we construct TM S to decide A TM. Here is how S operates: U = On input M, w, where M is a TM and w is a string: 1. Run TM R on input M, w. 2. If R rejects, reject. 3. If R accepts, simulate M on w until it halts. 4. If M has accepted, accept; if M has rejected, reject. If R decides HALT TM then S decides A TM, which is undecidable (Theorem 4.11). Hence HALT TM is undecidable. Computability Theory 81 / 155

Can humans solve the halting problem? It might seem like humans could solve the halting problem. After all, a programmer can often look at a program and tell whether it will halt. Let s understand why this cannot be true in general. To solve the halting problem means to be able to look and study any program and decide in a finite amount of time whether it halts. It is not enough to be able to look at some programs and decide. Even for short programs, it isn t clear that humans can always tell whether they halt. Computability Theory 82 / 155

Solving problems by testing the halting problem 1 Solving the halting problem would offer solutions to many open question in mathematics and computer science: The twin prime conjecture (Euclid around 300 B.C.), there are infinitely many primes p such that p + 2 is also prime. Goldbach s conjecture (1742), every even integer greater than two can be expressed as the sum of two primes. The Riemann hypothesis (1859), the real part of any non-trivial zero of the Riemann zeta function is 1/2. The correctness problem in software engineering, proving that a given program meets a given specification. Computability Theory 83 / 155

Solving problems by testing the halting problem 2 A conjecture is finitely refutable if verifying a finite number of instances suffices to disprove it. A systematic enumeration (of the problem s search domain) will find a counter-example if one exists. If the search stops, the conjecture is false; if the search does not halt the conjecture is true. For a finitely refutable problem Π we can construct a program C Π such that Π is false iff C Π halts. Π = Goldbach s conjecture is finitely refutable and one can construct a program C Π of 3,484 bits. Computability Theory 84 / 155

Solving problems by testing the halting problem 3 For example, Π = the Riemann hypothesis is finitely refutable and one can construct a program C Π of 7,780 bits. The twin prime conjecture n { p [p > n & p prime & p + 2 prime]} is not finitely refutable but can still be solved by testing the halting status of a small program. The stronger twin prime conjecture is finitely refutable: n { p [n < p < 2 n+4 & p prime & p + 2 prime]}. Computability Theory 85 / 155

Solving problems by testing the halting problem 4 Collatz function is defined by { x/2, if x is even, T (x) = 3x + 1, if x is odd, The Collatz conjecture states that for every seed a > 0, there is an iteration N such that T N (a) = 1. There is a non-constructive way to prove that there exists a program C Collatz which never stops iff the conjecture is true. Open question: Can you find/write such a program? Computability Theory 86 / 155

A Turing-unrecognisable language 1 In the proof of Theorem 4.11 we showed that: A TM is Turing-recognisable, A TM is not decidable. Recall that the complement of a language is the language consisting of all strings that are not in the language. We say that a language is co-turing-recognisable if it is the complement of a Turing-Recognisable language. We will show that A TM is not co-turing-recognisable. Computability Theory 87 / 155

A Turing-unrecognisable language 2 Theorem 4.22 A language is decidable iff it is Turing-recognisable and co-turing-recognisable. Proof. We have two directions to prove. Any decidable language is Turing-recognisable, and the complement of a decidable language also is decidable. For the other direction, assume A and A are Turing-recognisable. Let M 1 be the recogniser for A and M 2 be the recogniser for A. Computability Theory 88 / 155

A Turing-unrecognisable language 2 The following TM is a decider for A: D = On input string w: 1. Run both M 1 and M 2 on input w in parallel. 2. If M 1 accepts, accept; if M 2 accepts, reject. We show that D decides A: Every string w is either in A or A, but not in both. Therefore either M 1 or M 2 must accept w. Because D halts whenever M 1 or M 2 accepts, D always halts and so it is a decider. Furthermore, D accepts all strings in A and rejects all strings not in A, so it is a decider for A. Computability Theory 89 / 155

A Turing-unrecognisable language 3 Corollary 4.23 A TM is not Turing-recognisable. Proof. By the proof of Theorem 4.11, A TM is Turing-recognisable. If A TM also were Turing-recognisable, A TM would be decidable. Theorem 4.11 tells us that A TM is not decidable, so A TM must not be Turing-recognisable. Computability Theory 90 / 155

Undecidable problems 1 The strategy used in the proof of undecidability of HALT TM was the following: we assumed that HALT TM is decidable and use that assumption to show that A TM is decidable, contradicting Theorem 4.11. We will use this method to prove the undecidability of other problems. Computability Theory 91 / 155

Undecidable problems 2 E TM = { M : M is a TM and L(M) = }. Theorem 5.2 E TM is undecidable. Proof. First we construct the auxiliary TM M 1 based on a TM M and string w: M 1 = On input string x: 1. If x w, reject. 2. If x = w, run M on input w and accept if M 1 does. Computability Theory 92 / 155

Undecidable problems 3 Proof of Theorem 5.2 continued. Assume that TM R decides E TM and construct TM S that decides A TM : S = On input M, w, an encoding of a TM M and a string w: 1. Use the description of M and w to construct the TM M 1 described before. 2. Run R on input M 1. 3. If R accepts, reject; if R rejects, accept. If R were a decider for E TM, S would be a decider for A TM, a contradiction. Computability Theory 93 / 155

Undecidable problems 4 EQ TM = { M 1, M 2 : M 1 and M 2 are TMs and L(M 1 ) = L(M 2 )}. Theorem 5.4 EQ TM is undecidable. Proof. Let R decide EQ TM and construct TM S to decide E TM : S = On input M, where M is a TM: 1. Run R on input M, M 1, where M 1 is a TM that rejects all inputs. 2. If R accepts, accept; if R rejects, reject. If R decides EQ TM, S decides E TM, which by Theorem 5.2 is a contradiction. Computability Theory 94 / 155

Mapping reducibility 1 The notion of reducing one problem to another may be defined formally in one of several ways. Our choice is a simple type of reducibility called mapping reducibility or many-one reducibility. A problem A can be reduced to a problem B by using a mapping reducibility if there is a computable function that converts instances of problem A to instances of problem B. If we have such a conversion function we can solve A with a solver for B. Reason: any instance of A can be solved by first using the reduction to convert it to an instance of B and then applying the solver for B. Computability Theory 95 / 155

Mapping reducibility 2 A function f : Σ Σ is called computable if some TM M, on every input w, halts with f (w) on its tape. All usual arithmetic operations on integers are computable functions. All usual string operations are computable functions. Computable functions may be transformations of TM descriptions. For example, the function that takes an input string w and returns 1 if w = M is an encoding of a TM or returns 0 in the opposite case, is computable. Computability Theory 96 / 155

Formal definition of mapping reducibility 1 Language A is mapping reducible to language B, written A m B, if there is a computable function f : Σ Σ, where for every w Σ w A f (w) B. The function f is called the reduction of A to B. To test whether w A we use the reduction f to map w to f (w) and test whether f (w) B. Computability Theory 97 / 155

Formal definition of mapping reducibility 2 Theorem 5.22 If A m B and B is decidable, then A is decidable. Proof. Let M be the decider for B and f be the reduction from A to B. Here is a decider N for A: N = On input w: 1. Compute f (w). 2. Run M on input f (w) and output whatever M outputs. So, w A iff f (w) B because f is a reduction from A to B. Computability Theory 98 / 155

Formal definition of mapping reducibility 3 Corollary If A m B and A is undecidable, then B is undecidable. Proof. Assume by contradiction that B is decidable. Then, by Theorem 5.22, A is decidable, a contradiction. Computability Theory 99 / 155

Formal definition of mapping reducibility 4 Theorem 5.28 If A m B and B is Turing-recognisable, then A is Turing-recognisable. Proof. The proof is the same as that of Theorem 5.22, except that M and N are recognisers instead of deciders. Let M be the recogniser for B and f be the reduction from A to B. The recogniser N for A is: N = On input w: 1. Compute f (w). 2. Run M on input f (w) and output whatever M outputs. So, w A iff f (w) B because f is a reduction from A to B. Computability Theory 100 / 155

Formal definition of mapping reducibility 5 Corollary 5.29 If A m B and A is not Turing-recognisable, then B is not Turing-recognisable. Proof. Assume A m B. If A is not Turing-recognisable then B cannot be Turing-recognisable because by Theorem 5.28 A would be Turing-recognisable. Computability Theory 101 / 155

Example 1 Example 5.24 A TM m HALT TM. Proof. We will construct a computable function f that takes input of the form M, w and returns output of the form M, w, where M, w A TM M, w HALT TM. The following TM F computes f. F = On input M, w : 1. Construct the following machine M. M = On input x: 1. Run M on x. 2. If M accepts, accept. 3. If M rejects, enter a loop. 2. Output M, w. Computability Theory 102 / 155

Example continued 1 Improperly formed input strings i.e. inputs not of the form M, w (where M is a TM and w is a string) can be detected and in such a case F outputs a string not in HALT TM. How does the proof compare with the proof of Theorem 5.1? Computability Theory 103 / 155

Example 2 Example 5.26 E TM m EQ TM. Proof. The reduction f maps the input M to the output M, M 1, where M 1 is the TM that rejects all inputs (revisit the proof of Theorem 5.4). Computability Theory 104 / 155

Example 3 Example 5.27 A TM m E TM. Proof. From the reduction in the proof of Theorem 5.2, we construct a function f that takes input M, w and produces output M 1, where M 1 is the TM described in that proof. As M accepts w iff L(M 1 ), f is a mapping reduction from A TM to E TM. Computability Theory 105 / 155

Example 4 Example A If A TM m B, then B is not Turing-recognisable. Proof. As A TM m B, we have A TM m B = B, so by Corollary 4.23 and Corollary 5.29, B is not Turing-recognisable. Example B E TM is not Turing-recognisable. Proof. Use Example A and Example 5.27. Computability Theory 106 / 155

Example 5 Example C If A m B, A m B, and B is Turing-recognisable, then A is decidable. Proof. First, from A m B and B is Turing-recognisable we deduce, using Theorem 5.28, that A is Turing-recognisable. If A m B, then A m B = B and B is Turing-recognisable we deduce, using Theorem 5.28, that A is Turing-recognisable. Since A and A are Turing-recognisable, A is decidable (Theorem 4.22). Computability Theory 107 / 155

Example 6 Example D A TM m E TM. Proof. In Example C we put: A = A TM, B = E TM. By Example 5.27, A TM m E TM. Since the language E TM is Turing-recognisable, if A TM m E TM, then by Example C, A TM is decidable, a contradiction (Theorem 4.9). Computability Theory 108 / 155

The remarkable language EQ TM 1 Recall that EQ TM = { M 1, M 2 : M 1 and M 2 are TMs and L(M 1 ) = L(M 2 )}. Theorem 5.30 EQ TM is not Turing-recognisable nor co-turing-recognisable. Proof. First we show that EQ TM is not Turing-recognisable by showing that A TM m EQ TM. Computability Theory 109 / 155

The remarkable language EQ TM 2 The reducing function f works as follows. F = On input M, w where M is a TM and w a string: 1. Construct the following two machines M 1 and M 2. M 1 = On any input: 1. Reject. M 2 = On any input: 1. Run M on w. If it accepts, accept. 2. Output M 1, M 2. Here, M 1 accepts nothing. If M accepts w, M 2 accepts everything, and so the two TMs are not equivalent. Conversely, if M doesn t accept w, M 2 accepts nothing, and the TMs are equivalent. Computability Theory 110 / 155

The remarkable language EQ TM 3 To show that EQ TM is not Turing-Recognisable we give a reduction from A TM to the complement of EQ TM which is simply EQ TM. Hence we show that A TM m EQ TM. The following TM G computes the reducing function g: G = The input is M, w where M is a TM and w a string: 1. Construct the following two machines M 1 and M 2. M 1 = On any input: 1. Accept. M 2 = On any input: 1. Run M on w. If it accepts, accept. 2. Output M 1, M 2. Computability Theory 111 / 155

The remarkable language EQ TM 4 The only difference between f and g is in the construction of the TM machine M 1. In f, machine M 1 always rejects, whereas in g it always accepts. In both f and g, M accepts w iff M 2 always accepts. In g, M accepts w iff L(M 1 ) = L(M 2 ) showing that A TM m EQ TM. Computability Theory 112 / 155

Rice s theorem 1 Rice s theorem (Problem 5.28) Let P be a language consisting of TM descriptions where P fulfils two conditions. First, P is nontrivial it contains some, but not all, TM descriptions. Second, P is a property of the TM s language whenever L(M 1 ) = L(M 2 ), we have M 1 P iff M 2 P. Here M 1 and M 2 are any TMs. Then, P is undecidable. Proof. Assume for the sake of a contradiction that P is decidable and let R P be a TM that decides P. We will show how to use R P to decide A TM obtaining a contradiction (Theorem 4.11). Computability Theory 113 / 155

Rice s theorem 2 Proof of Rice s theorem continued. First let T be a TM such that L(T ) =. You may assume that T P without loss of generality (otherwise proceed with P instead of P). Secondly, because P is not trivial, there exists a TM T with T P. Computability Theory 114 / 155

Rice s theorem 2 Proof of Rice s theorem continued. The following TM S decides A TM. S = On input M, w : 1. Use M and w to construct the following TM M w. M w = On input x: 1. Simulate M on w. If it halts and rejects, reject. If it accepts, proceed to stage 2. 2. Simulate T on x. If it accepts, accept. 2. Use TM R P to determine whether M w P. If YES, accept. If NO, reject. Computability Theory 115 / 155

Rice s theorem 3 Proof of Rice s theorem continued. TM M w simulates T if M accepts w, hence { L(T ), if M accepts w, L(M w ) =, if M does not accept w, { L(T ), if M accepts w, = L(T ), if M does not accept w. Therefore M w P iff M accepts w. Computability Theory 116 / 155

Rice s theorem 4 Corollary to Rice s Theorem The following languages 1 { M : L(M) = }, 2 { M : L(M) = infinite}, 3 { M : L(M) = finite}, 4 { M : L(M) = regular}, 5 { M : 10 L(M)}, 6 { M : L(M) = Σ }. are undecidable. Proof. Each property specified in the languages above is a nontrivial property of the TM s language so it satisfies Rice s Theorem. Computability Theory 117 / 155