Computability Theory


 Steven Horn
 11 months ago
 Views:
Transcription
1 Computability Theory Cristian S. Calude and Nicholas J. Hay May June 2009 Computability Theory 1 / 155
2 1 Register machines 2 ChurchTuring thesis 3 Decidability 4 Reducibility 5 A definition of information 6 Quantum information Computability Theory 2 / 155
3 Objectives On completion, students will be able to: explain the theoretical limits on computational solutions of undecidable and inherently complex problems, describe concrete examples of computationally undecidable or inherently infeasible problems from different fields, understand formal definitions of machine models, classical and unconventional, prove the undecidability or complexity of a variety of problems, understand the issue of whether there are limits of computability. Computability Theory 3 / 155
4 Bibliography M. Sipser. Introduction to the Theory of Computation, PWS (textbook) N. J. Hay. Register machine simulation software, Computability Theory 4 / 155
5 Supplementary bibliography 1 C. S. Calude, Elena Calude, M. J. Dinneen. A new measure of the difficulty of problems, Journal for MultipleValued Logic and Soft Computing 12 (2006), C. S. Calude, M. J. Dinneen, K. Svozil. Reflections on quantum computing, Complexity 6, 1 (2000), C. S. Calude, M. A. Stay. Most programs stop quickly or never halt, Advances in Applied Mathematics, (2007), 40 (2008), C. S. Calude. Incompleteness: A Personal Perspective, Google Technical Talk, Mountainview, USA, 4 November 2008, J. P. Dowling. To compute or not to compute? Nature 439, 23 February 2006, Computability Theory 5 / 155
6 Supplementary bibliography 2 Hypercomputation Research Network, John C. Martin. Introduction to Languages and the Theory of Computation, Mc GrawHill, Boston, 2003, third edition. K. Svozil. Quantum algorithmic information theory, J. UCS 2,5 (1996), A. Turing. Computing machinery and intelligence, Mind LIX (236) (1950), , doi: /mind/lix S. Wolfram. The New Kind of Science, Wolfram Media, 2002, Computability Theory 6 / 155
7 Assignments, test and exam Three assignments worth 20% in total Assignment 3 : 22 May, due 8 June, worth 5% Bonus questions : each worth 1% Test : 30%. Exam : 50%. 13 June. You will be given six questions and asked to choose and answer four of them. Each question is worth 25 marks. Computability Theory 7 / 155
8 What is computability? What can computers compute? Are there any limitations? If so, exactly what? To answer these questions, we will construct formal models of computers and prove theorems about them. Computers compute using programs. A formal model of a computer is really a formal model of a programming language. To make things easy we want our programming language to be as simple as possible, without sacrificing power. We will present the register machine model. This abstracts from and simplifies modern CPU instruction architectures: we will assume unbounded memory and remove all but the most central instructions. Computability Theory 8 / 155
9 Example: doubling a register 1 0 A: if (x1!=0) goto B 1 halt 2 B: x1 3 y++ 4 y++ 5 if (y!=0) goto A The registers x1 and y are variables which can store natural numbers. If we start running this program with x1 set to 13 and y set to 0, it will halt with y set to 26. Interpret if, ++, and  as in Java code. halt halts the program. goto L jumps execution to the instruction labelled L. Computability Theory 9 / 155
10 Example: doubling a register 2 0 A: if (x1!=0) goto B 1 halt 2 B: x1 3 y++ 4 y++ 5 if (y!=0) goto A The program starts execution at instruction 0. Since x1 is 13 execution continues at label B. Instructions 2 through 4 decrease x1 by 1 and increase y by 2. Instruction 5 jumps back to instruction 0 since y is nonzero (this is really an unconditional goto), and the loop repeats. The above loops 13 times, until x1 is decremented all the way to 0. At this point instruction 1 runs, and we halt with y equal to 26. Computability Theory 10 / 155
11 Register machine model 1 A register machine has a number of registers, each of which stores a natural number. We will name registers y, x1, x2, etc. They are similar to variables of type int, except there is no upper bound to their value. y 256 x1 0 x2 2 x3 42 x As seen in the previous example, these registers can change value as the program executes. We will assume all registers but the input registers start with the value 0. Computability Theory 11 / 155
12 Register machine model 2 A program is a list of instructions. There are are four kinds: 1 xi++ This increments the value of register xi. 2 xi This decrements the value of register xi. If xi is already zero, however, it stays zero. 3 halt Stops program execution. 4 if (xi!=0) goto L If the value of register xi is not zero, execution continues the instruction labelled L. Otherwise it continues at the next instruction. An instruction prefixed by L: is labelled by the label L. Execution starts at the first instruction. Unless a goto jumps, we execute the next instruction until we run out or hit a halt. Computability Theory 12 / 155
13 Semantics: computing partial functions 1 Register machine programs can be used to compute partial functions mapping natural numbers to natural numbers. A program computes a partial function f : N N if when given x in the x1 register it outputs f (x) in the y register if f (x) is defined, otherwise the program doesn t halt. We can generalise this to multiple arguments functions, e.g. f (x 1, x 2 ) = 2x 1 + x 2, by placing the inputs in the registers x1, x2, etc. Computability Theory 13 / 155
14 Semantics: computing partial functions 2 For example, we would implement f (x) = 2x by a program which, when given x in the x1 register outputs 2x in the y register. The program from before implements f (x) = 2x: 0 A: if (x1!=0) goto B 1 halt 2 B: x y++ 4 y++ 5 if (y!=0) goto A Computability Theory 14 / 155
15 Example: modulo two 0 x2++ 1 A: if (x1!=0) goto B 2 halt 3 B: x1 4 if (y!=0) goto C 5 y++ 6 if (x2!=0) goto A 7 C: y 8 if (x2!=0) goto A This program computes the function { 0, if x is even, f (x) = 1, if x is odd. The register x2 is incremented on line 0 to make lines 6 and 8 unconditional gotos. Lines 48 alternate y between the values 0 and 1. Computability Theory 15 / 155
16 Semantics: computing sets by deciding them To describe a set S N of natural numbers we can give a method for computing whether x S for any x N. A program decides (or is a decider for) a set S N if when given x in the x1 register it outputs 1 in the y register if x S, and 0 in the y register if x / S. Equivalently, a program decides a set S if it implements its characteristic function χ S. A decidable set is one which has program deciding it. Our modulo two example before decides the set of odd numbers: x modulo two equals 1 if x is odd, and 0 otherwise. Computability Theory 16 / 155
17 Semantics: computing sets by enumerating them We can also describe a set by listing (or enumerating) its elements. A program enumerates a set S N if S = {f (x) : x N} = ran f where f is the function the program implements. A computably enumerable or Turingrecognisable set is one which has program enumerating it. The program to the right enumerates the set of odd numbers, as it implements the function f (x) = 2x y++ 1 A: if (x1!=0) goto B 2 halt 3 B: x1 4 y++ 5 y++ 6 if (y!=0) goto A Computability Theory 17 / 155
18 Formal register programs Our previous descriptions of register machine operations were informal. We can give a mathematically precise description, like those for DFAs and Turing machines, although slightly more complex. First we will give a mathematical representation of a register machine program. Next we give a mathematical representation of a register machine running a program: both its state and its transition function. Finally, we can formally define what it means for a register machine program to compute a partial function, decide a set, and enumerate a set. Computability Theory 18 / 155
19 Formal register programs: encoding programs We can encode instructions by tuples of three numbers. These record instruction type, register number, and jump destination. In detail, each instruction corresponds to an element of N 3 by the translation: xi++ (0, i, 0) xi (1, i, 0) halt (2, 0, 0) if (xi!=0) goto L (3, i, #L) where #L is the instruction number corresponding to label L. For example, x2++ will be encoded by (0, 2, 0), and if (x4!=0) goto A by (2, 4, 13) if A labels instruction 13. A register machine program p is a sequence of such tuples i.e. p (N 3 ). Computability Theory 19 / 155
20 Formal register programs: example program encoding For example, on the left we have a program in our informal notation, on the right our formal representation of it. 0 A: if (x1!=0) goto B (3,1,2) 1 halt (2,0,0) 2 B: x1 (1,1,0) 3 y++ (0,0,0) 4 y++ (0,0,0) 5 if (y!=0) goto A (3,0,0) Note that register y is register 0, and that the labels A and B correspond to instruction numbers 0 and 2 respectively. Computability Theory 20 / 155
21 Formal register programs: register machine state The state of a register machine at any point in time is the current instruction and the value of all its registers. We will store the register values as a finite sequence of numbers, assuming all registers not mentioned are zero. For example, the sequence 3, 10, 0, 32 means register y is 3, x1 is 10, x2 is 0, x3 is 32, and all others are 0. Given a sequence of register values r N, we denote the value of ith register to be r(i). For i, v N and a sequence of register values r N, denote by r[i v] the sequence of register values got by taking r and setting the ith register s value to v. Thus, the complete machine state is an element (n, r) of N N, where n is the current instruction number, and r the sequence of register values. Computability Theory 21 / 155
22 Formal register programs: instruction transition function When a register machine executes an instruction its state (instruction number, register values) changes. For each instruction (x, y, z), we define the instruction transition function δ (x,y,z) which when given the current state (n, r) N N outputs the state (n, r ) after the instruction has executed. (n + 1, r[y v + 1]), if x = 0, v = r[y], (n + 1, r[y v 1]), if x = 1, v = r[y], δ (x,y,z) (n, r) = (n + 1, r), if x = 3, r[y] = 0, where v 1 = max{0, v 1}. (z, r), if x = 3, r[y] 0, (n, r), otherwise, Computability Theory 22 / 155
23 Formal register programs: program transition function Finally, given a program we can define the register machine s program transition function. At each time step, if the current instruction number is valid the machine looks up the current instruction and executes it, otherwise it does nothing. The program transition function δ p : N N N N is defined from the program p: { δ δ p (n, r) = p(n) (n, r), if n < p, (n, r), otherwise. Computability Theory 23 / 155
24 Formal register programs: partial functions, sets Define r 0 = 0x (0 and x are the values of registers y and x1), and δ t p(n, r) = δ p (δ p (... δ p (n, r)...)), t times. The program p computes the partial function f when for all x N with if f (x) is defined there exists a t N such δp(0, t r 0 ) = δp t+1 (0, r 0 ) = (n, r) and r(0) = f (x), if f (x) is not defined then there is no t such that δp(0, t r 0 ) = δp t+1 (0, r 0 ). As before we can define when a program decides/enumerates a set. A program p decides a set S if it implements its characteristic function χ S. A program p enumerates a set S if p implements the function f and S = ran f. Computability Theory 24 / 155
25 More examples: addition 1 0 A: if (x1!=0) goto C 1 B: if (x2!=0) goto D 2 halt 3 C: x1 4 y++ 5a x3++ 5b if (x3!=0) goto A 6 D: x2 7 y++ 8a x3++ 8b if (x3!=0) goto A This implements f (x 1, x 2 ) = x 1 + x 2. Lines 5a/5b and 8a/8b both use our trick for an unconditional goto: make sure some other register is nonzero then condition on it being nonzero. This would be clearer if we just used goto A. Computability Theory 25 / 155
26 More examples: addition 2 0 A: if (x1!=0) goto C 1 B: if (x2!=0) goto D 2 halt 3 C: x1 4 y++ 5 goto A 6 D: x2 7 y++ 8 goto B The instruction goto L moves execution to label L unconditionally. Computability Theory 26 / 155
27 Macro: goto We define goto L as a macro that expands into: 0 xi++ 1 if (xi!=0) goto L where xi is a register not used by the main program. We didn t add goto as an independent instruction as we didn t need to. This definition is inefficient: it increments xi every time the goto is executed when we only need to do this once. A more efficient definition of goto would have xi incremented once, at the very start of the program. This is more complex, and its definition is not local: we have to add an instruction at the start of the program. Computability Theory 27 / 155
28 More examples: subtraction Let f (x 1, x 2 ) = x 1 x 2, where x 1 x 2 = max{0, x 1 x 2 }. Modifying only instruction 7 of the addition program gets us subtraction: 0 A: if (x1!=0) goto C 1 B: if (x2!=0) goto D 2 halt 3 C: x y++ 5 goto A 6 D: x y 8 goto B Computability Theory 28 / 155
29 Example: multiplication 1 We can implement multiplication by: 0 A: if (x1!=0) goto C 1 halt 2 C: x1 3 addto y x2 4 goto A addto y x2 adds the value of x2 to y without changing the value of x2. How is addto implemented? Computability Theory 29 / 155
30 Macro: addto addto y xj: 0 C: if (xj!=0) goto D 1 goto E 2 D: xj 3 t++ 4 y++ 5 goto C 6 F: t 7 xj++ 8 E: if (t!=0) goto F You can verify these instructions will add xj to y without changing the value of xj. The register t is a register not used by the rest of the program. We use it to save the value of xj. Computability Theory 30 / 155
31 Example: multiplication 2 0 A: if (x1!=0) goto B 1 halt 2 B: x1  3a C: if (x2!=0) goto D 3b goto E 3c D: x2 3d x3++ 3e y++ 3f goto C 3g F: x3 3h x2++ 3i E: if (x3!=0) goto F 4 goto A If we expand the addto macro we get this program. Instructions 0,1,2, and 4 implement a for loop: the block of instructions 3a3i is repeated x1 times. The block of instructions 3a3i adds x2 to y. This is just the addto macro expanded. Computability Theory 31 / 155
32 Closure under composition 1 Function composition is a powerful method for constructing complex functions from simple ones. For example, if we have the function f (x) = x 3 and g(x) = 2x + 1, then their composition gives us (f g)(x) = (2x + 1) 3. Intuitively, we should be able to get a program for f g by sticking together one for f and one for g: [program computing g] move y x1 [program computing f] Computability Theory 32 / 155
33 Closure under composition 2 Again: [program computing g] move y x1 [program computing f] [program computing g] is shorthand for inserting the code that computes g into the program. move y x1 copies the value in register y, where g places its output, into register x1, where f expected its input. Computability Theory 33 / 155
34 Closure under composition 3 The basic idea is sound, but there are a couple of technical difficulties. The two programs may use the same label for different code locations. Solution: rename labels e.g. rename f s L label to fl, and g s to gl. The program for g(x) can make arbitrary registers nonzero, e.g. x2, but the program for f (x) may assume these are zero. Solution: clear the registers used by g(x). Computability Theory 34 / 155
35 Closure under composition 4 This leads to the slightly modified scheme: [program computing g with labels adjusted] move y x1 zero y zero x2... zero xk [program computing f with labels adjusted] zero y zeros the y register. xk is the last register modified by the program for g. Computability Theory 35 / 155
36 Example: The Fibonacci Function 1 The Fibonacci sequence can be represented by a function f : N N defined by f (0) = 0, f (1) = 1 and f (x + 2) = f (x) + f (x + 1). To implement this we will use registers x2 and x3 to store f (i) and f (i + 1). We will loop over i to compute f (x). Computability Theory 36 / 155
37 Example: The Fibonacci Function 2 A highlevel register program to implement this: x3++ A: if (x1==0) goto F copy x3 x4 addto x2 x3 copy x4 x2 x1 goto A F: copy x2 y Macros can be written for copy x y and if (x==0) goto L. The next slide expands this program, making a few optimisations. Computability Theory 37 / 155
38 Example: The Fibonacci Function 3 The program below computes the Fibonacci function. x3++ A: if (x1==0) goto F B: if (x3==0) goto C x3  x2++ x4++ goto B C: if (x2==0) goto D x2  x3++ goto C D: if (x4==0) goto E x4  x2++ goto D E: x1  goto A F: if (x2==0) goto end x2  y++ goto F end: halt Computability Theory 38 / 155
39 String register machines Register machines work with numbers, but DFAs and Turing machines work with strings. A variant register machine computes on strings. We will see that register machines computing on strings are equivalent in power to those computing on natural numbers, and that both are equivalent in power to Turing machines. Computability Theory 39 / 155
40 Example: reversing a string 1 0 A: pop x1 E B C 1 B: push y 0 2 goto A 1 C: push y 1 2 goto A 5 E: halt The registers now hold binary strings such as , 11, and ɛ, rather than numbers. pop x1 E B C first checks if x1 is empty. If it is, it jumps to label E and halts. Otherwise it removes the last character from x1 and jumps to B if it is 0, and C if it is 1. push y 0 appends a 0 to the end of y. Computability Theory 40 / 155
41 Example: reversing a string 2 0 A: pop x1 E B C 1 B: push y 0 2 goto A 1 C: push y 1 2 goto A 5 E: halt Characters are repeatedly removed from the end of x1 and placed at the end of y. A moment s thought will confirm this reverses the string originally in x1. Computability Theory 41 / 155
42 String register machine model 1 String register machines are similar to (numeric) register machines. They have registers, this time storing binary strings. y x1 011 x2 x x As time goes on these registers can change value. Computability Theory 42 / 155
43 String register machine model 2 A program is a list of instructions, of the following four types: 1 push xi v Append v (either 0 or 1) to the end of register xi. 2 goto L Continue execution at the instruction labelled by L. 3 pop xi A B C If register xi is empty then goto A. Otherwise, remove the last character from xi: if it s 0 goto B, if it s 1 goto C. 4 halt Halt execution. Computability Theory 43 / 155
44 Semantics: functions and sets String register programs define functions and sets in the same way as the numerical case. A string register program computes a partial function f : {0, 1} {0, 1} if when given x in the x1 register it outputs f (x) in the y register. A string register program decides a set S {0, 1} if when given x in the x1 register it outputs 1 in the y register if x S, and 0 in the y register if x / S. A string register program enumerates a set S {0, 1} if S = {f (x) : x {0, 1} } = ran f where f is the function the program implements. Computability Theory 44 / 155
45 Example: doubling a string 1 To implement the doubling function f (x) = xx, we would like to write: copy x1 y copy x1 y where copy x1 y appends the contents of x1 onto y, without reversing it or destroying the original value in x1 (it is easier to reverse and destroy the original value: 5 lines of code suffice). How to implement copy x1 y? Computability Theory 45 / 155
46 Example: doubling a string 2 Implement the macro copy x y by 0 A: pop x D B C 1 B: push t 0 2 goto A 3 C: push t 1 4 goto A 5 D: pop t G E F 6 E: push x 0 7 push y 0 8 goto D 9 F: push x 1 10 push y 1 11 goto D 12 G: where register t is unused and assumed to be empty. This first moves the string in x into temporary register t, reversing it in the process. It then removes characters from t, restoring them to x and adding them to y. Computability Theory 46 / 155
47 Example: doubling a string 3 0 A: pop x1 D B C 1 B: push x2 0 2 push x3 0 3 goto A 4 C: push x2 1 5 push x3 1 6 goto A 7 D: pop x2 G E F 8 E: push y 0 9 goto D 10 F: push y 1 11 goto D 12 G: pop x3 X H I 13 H: push y 0 14 goto G 15 I: push y 1 16 goto G 17 X: halt This is what it looks like without the copy macro (optimised a little). First, this makes two reversed copies of x1 in the registers x2 and x3. Finally, it copies x2 and then x3 into y. Computability Theory 47 / 155
48 Regular languages In previous lectures we introduced DFAs, which recognise exactly the regular languages. Register machines can recognise regular languages too (among many other languages). Recall regular languages can be inductively defined by regular expressions. Starting from the basic one string regular languages {0} and {1} (more generally, {σ} for each character σ Σ) we build up all others by union, complement, concatenation, and Kleene star. Computability Theory 48 / 155
49 Regular languages: basic languages The regular languages {0} and {1} are easy to recognise. For example {0}: 0 pop x1 R B R 1 B: pop x1 A R R 2 A: push y 1 3 halt 4 R: push y 0 5 halt Computability Theory 49 / 155
50 Regular languages: union 1 Closure under union is a special case of function composition: given languages L 1 and L 2, the characteristic function of their union L 1 L 2 satisfies: χ L1 L 2 (x) = (χ L1 (x), χ L2 (x)) where (a, b) = 1 if either a = 1 or b = 1, otherwise (a, b) = 0. Computability Theory 50 / 155
51 Regular languages: union 2 Concretely, L 1 L 2 is decided by: copy x1 t [program computing χ L1 pop y1 C C A C: copy t x1 [program computing χ L2 pop y1 R R A R: empty y push y 0 halt A: empty y push y 1 where empty y empties y. with labels adjusted] with labels adjusted] Computability Theory 51 / 155
52 Regular languages: concatenation To see whether x L 1 L 2 we need to try all possible splittings x = yz and check whether one has y L 1 and z L 2. copy x1 tleft L: copy tleft x1 [program computing χ L1 pop y1 E E C C: copyrev tright x1 [program computing χ L2 pop y1 E E A E: pop tleft R E0 E1 E0: push lright 0 goto L E1: push lright 1 goto L with labels adjusted] with labels adjusted] Computability Theory 52 / 155
53 Regular languages: Kleene star Closure under Kleene star is similar in spirit to closure under composition, except we have to allow the string to be divided into many pieces. To see whether x L we need to try all possible splittings x = y 1... y k for all k x and check whether y i L 1 for all i. Each splitting of a string, for example into 00,101,0,101, corresponds to a binary string which we mark each cutting point, in this example There are, in fact, 2 x 1 of these (ignoring x = 0). One way to decide L, then, is when given x {0, 1} to enumerate all splitting strings as above, then test whether each of them divide x into strings found in L. If some splitting does, accept, if none do, reject. Computability Theory 53 / 155
54 Beyond regular languages String register machines can recognise languages DFAs cannot. For example, the language of strings with the same number of 0 s as 1 s. One way to implement this decider. First sort the 0 s and 1 s of the input into separate registers. Then remove elements from both registers one by one to check whether there are the same number of 0 s and 1 s. Computability Theory 54 / 155
55 Beyond regular languages String register machines can recognise languages DFAs cannot. For example, the language of strings with the same number of 0 s as 1 s. One way to implement this decider. First sort the 0 s and 1 s of the input into separate registers. Then remove elements from both registers one by one to check whether there are the same number of 0 s and 1 s. Computability Theory 55 / 155
56 Example: equal 0 s and 1 s 0 S: pop x1 C M0 M1 1 M0: push x2 0 2 goto S 3 M1: push x3 1 4 goto S 5 C: pop x2 E NE NE 6 NE: pop x3 R C C 7 E: pop x3 A R R 8 A: push y 1 9 halt 10 R: push y 0 11 halt Lines 04 sort the characters of x1: all the 0 s into x2, all the 1 s into x3. Lines 57 check to see whether x2 and x3 have the same number of elements by first trying to remove an element of x2 then trying to remove one from x3. Labels A and R accept and reject the input, respectively. Computability Theory 56 / 155
57 String and numeric register machines equivalent 1 We described two different register machines, working with numbers and strings. How are they related? They are equivalent: programs from one can be translated into equivalent programs for the other. The easier direction is showing numerical register machine programs can be translated into string register programs. We ll do that. The key ingredient for the translation is some way of translating numbers into binary strings. We use the obvious plan: write the number in binary. For example, 7 translates to 111, 12 to 1100, and 0 to ɛ. (For translating binary strings into numbers, think about the sequence ɛ, 0, 1, 00, 01, 10, 11, 100,... ) Computability Theory 57 / 155
58 String and numeric register machines equivalent 2 We need only translate each of the 4 numerical instructions into string register machine code. We don t need to do anything for halt, so that leaves 3: xi++ xi if (xi!=0) goto L Computability Theory 58 / 155
59 String and numeric register machines equivalent 3 x++. The following routine will add 1, in binary, to the contents of the register x using temporary register temp: loop: pop x1 nocarry nocarry carry carry: push temp 0 goto loop nocarry push x1 1 move: pop temp end w0 w1 w0: push x1 0 goto move w1: push x1 1 goto move end: halt Computability Theory 59 / 155
60 String and numeric register machines equivalent 4 x can be implemented by using the same trick as in ones complement subtraction (left as an exercise). if (x!=0) goto L can be similarly translated (left as an exercise). Computability Theory 60 / 155
61 String register machines and TMs equivalent One can translate Turing machines into equivalent string register programs. This is simply a coding exercise. Basically, use one register to store the contents of the tape of the left of the tape head, another to store the contents under and after it (in reverse). Another register stores the current tape state. For each pair (q, γ) Q Γ implement code to modify the tape registers (write the new symbol and move left/right), and the state register (write the next state). Implement a central loop that jumps to the right code location depending on the current machine state. Computability Theory 61 / 155
62 Universal register machines One can implement a universal string register machine program. This, when given an encoding of a string register program in one of its register, and an input for it in another, will simulate the program and write its output to y. Outline of construction: prefixfree encoding of strings (x 1 0x x n 1 is convenient for local modifications), prefixfree encoding of numbers (n coded as 0 n 1), prefixfree encoding of instructions (000 i 1 for goto i, 01v0 x 1 for push x v, 100 r 10 i 10 j 10 k 1 for pop r i j k, 11 for halt). One register pair holds the program (a pair storing left and right halves as the program reads it), another pair holds the simulated registers (the values concatenated together), another the program counter. Computability Theory 62 / 155
63 Equivalence between various models We have presented the register machine model and the Turing machine model and have shown them to be equivalent in power. There are several variants of the Turing machine model, generative grammars, λcalculus, etc. that share the essential feature of having unrestricted access to unlimited memory. Remarkably, all models with that feature turn out to be equivalent in computational power. Computability Theory 63 / 155
64 ChurchTuring thesis 1 The ChurchTuring thesis (formerly commonly known simply as Church s thesis) says that every intuitively computable function is computable by some Turing machine. There are conflicting points of view about the ChurchTuring thesis... Computability Theory 64 / 155
65 ChurchTuring thesis 2 (1) There has never been a proof for ChurchTuring thesis; in fact it cannot be proven. The evidence for its validity comes from the fact that every realistic model of computation, yet discovered, has been shown to be equivalent. (2) A device which could answer questions beyond those that a Turing machine can answer would disprove the ChurchTuring thesis. No such convincing device has been found (yet). Hypercomputation is searching for such a device. Computability Theory 65 / 155
66 Physical ChurchTuring thesis or Here are two formulations out of a much larger set: every realworld computation can be translated into an equivalent computation involving a Turing machine any physical process can be simulated by some Turing machine. Both formulations would be falsified by the existence of genuine random processes. A. Turing: a machine with a random element can do more than a Turing Machine (see Turing 1950, pp ). What is more? Computability Theory 66 / 155
67 Wolfram s principle of computational equivalence Almost all processes that are not obviously simple can be viewed as computations of equivalent sophistication. In detail, the principle of computational equivalence says that: the systems found in nature can perform computations up to a maximal ( universal ) level of computational power, and that most systems do in fact attain this maximal level of computational power, so most systems are computationally equivalent. For example, the workings of the human brain or the evolution of weather systems can, in principle, compute the same things as a computer. Computability Theory 67 / 155
68 Encodings 1 We have shown how to code register machine programs. In a similar way one can encode Turing machines by strings over a (large) alphabet Σ, and then, using a bijection between Σ and {0, 1}, by binary strings. There are various possibilities to design such encodings. A Turing machine can always translate one such encoding into another. In what follows we will just fix one encoding and denote by M the string coding/representing the Turing machine M. If we have several Turing machines M 1, M 2,..., M k, we denote their encoding into a single string M 1, M 2,..., M k. Computability Theory 68 / 155
69 Encodings 2 We will work with a prefixfree encoding. The prefixcode of a string x 1 x 2... x n 1 x n is the string x 1 0x x n 1 0x n 1. For example, the prefixcode of the empty string is 1; the prefixcode of the string 101 is The length of the prefixcode of a string of length n is 2n. A Turing machine can compute from the prefixfree code M 1, M 2,..., M k each of its components M 1, M 2,..., M k. Computability Theory 69 / 155
70 Decidable languages 1 In what follows we will work with highlevel descriptions of Turing machines, i.e. we will use English prose to describe algorithms (and hence we ignore the implementation details). Recall that a language Turingrecognisable or computably enumerable if some Turing machine recognises it. Call a language Turingdecidable or simply decidable, or recursive, or computable if some Turing machine decides it. Computability Theory 70 / 155
71 Decidable languages 2 A DFA = { B, w : B is a DFA that accepts the input string w}. Theorem 4.1 A DFA is decidable. Proof. A TM that decides A DFA : M = On input B, w, where B is a DFA and w is a string: 1. Simulate B on input w. 2. If the simulation ends in an accept state, accept. If it ends in a nonaccept state, reject. Computability Theory 71 / 155
72 Decidable languages 3 A NFA = { B, w : B is an NFA that accepts the input string w}. Theorem 4.3 A NFA is decidable. A REX = { R, w : R is a regular expression that accepts the Theorem 4.4 A REX is decidable. input string w}. Computability Theory 72 / 155
73 Decidable languages 4 E DFA = { A : A is a DFA and L(A) = }. Theorem 4.4 E DFA is decidable. EQ DFA = { A, B : A and B are DFAs and L(A) = L(B)}. Theorem 4.5 EQ DFA is decidable. Computability Theory 73 / 155
74 The halting problem 1 We prove one of the most philosophically important theorems of the theory of computation: There is a specific problem that is algorithmically unsolvable. What sort of problems are unsolvable by computer? Are they esoteric, dwelling only in the minds of theoreticians? Even some ordinary problems that people want to solve turn out to be computationally unsolvable. For example, the general problem of software verification is unsolvable! Computability Theory 74 / 155
75 The halting problem 2 We establish the undecidability of the acceptance problem, i.e. the problem of determining whether a Turing machine accepts a given input string. Formally: A TM = { M, w : M is a TM and M accepts w}. Theorem 4.11 A TM is undecidable. Computability Theory 75 / 155
76 The halting problem 3 Intermediate step: A TM Turingrecognisable. Proof. A TM is Turingrecognisable by the TM U: U = On input M, w, where M is a TM and w is a string: 1. Simulate M on input w. 2. If M ever enters its accept state, accept; if M ever enters its reject state, reject. U loops on input M, w if M loops on w, so U does not decide A TM! But is there a TM deciding A TM? U is interesting in its own right: it is a universal Turing machine because it is capable of simulating any other Turing machine from the description of that machine. Computability Theory 76 / 155
77 The halting problem 4 Proof of Theorem Assume, to obtain a contradiction, that A TM = { M, w : M is a TM and M accepts w}. is decidable, say, by decider H. On input M, w, where M is a TM and w is a string: H halts and accepts if M accepts w, H halts and rejects if M does not accept w, i.e. if 1 M halts and does not accept w, or 2 M does not halt on w. Computability Theory 77 / 155
78 The halting problem 5 Formally, H( M, w ) = { accept, if M accepts w, reject, if M does not accept w. accept, if M accepts w, = reject, if M halts and does not accept w, reject, if M does not halt on w. Computability Theory 78 / 155
79 The halting problem 6 Construct a new TM D with H as a subroutine: D calls H to determine what M does on its own description w = M and D rejects if M accepts and D accepts if M does not accept. D = On input M, where M is a TM: 1. Run H on input M, M. 2. If H accepts, reject; if H rejects, accept. D( M ) = D( D ) = a contradiction. { accept, if M does not accept M, reject, if M accept M. { accept, if D does not accept D, reject, if D accept D, Computability Theory 79 / 155
80 The halting problem 7 We prove the undecidability of the most (in)famous problem in computer science: the problem of determining whether a Turing machine halts on a given input string. Formally: HALT TM = { M, w : M is a TM and M halts on input w}. Theorem 5.1 HALT TM is undecidable. Proof. The proof is by contradiction: we assume that HALT TM is decidable and use that assumption to show that A TM is decidable, contradicting Theorem The key idea is to show that A TM is reducible to HALT TM. Computability Theory 80 / 155
81 The halting problem 7 Proof of Theorem 5.1 continued. We assume that TM R decides HALT TM and we construct TM S to decide A TM. Here is how S operates: U = On input M, w, where M is a TM and w is a string: 1. Run TM R on input M, w. 2. If R rejects, reject. 3. If R accepts, simulate M on w until it halts. 4. If M has accepted, accept; if M has rejected, reject. If R decides HALT TM then S decides A TM, which is undecidable (Theorem 4.11). Hence HALT TM is undecidable. Computability Theory 81 / 155
82 Can humans solve the halting problem? It might seem like humans could solve the halting problem. After all, a programmer can often look at a program and tell whether it will halt. Let s understand why this cannot be true in general. To solve the halting problem means to be able to look and study any program and decide in a finite amount of time whether it halts. It is not enough to be able to look at some programs and decide. Even for short programs, it isn t clear that humans can always tell whether they halt. Computability Theory 82 / 155
83 Solving problems by testing the halting problem 1 Solving the halting problem would offer solutions to many open question in mathematics and computer science: The twin prime conjecture (Euclid around 300 B.C.), there are infinitely many primes p such that p + 2 is also prime. Goldbach s conjecture (1742), every even integer greater than two can be expressed as the sum of two primes. The Riemann hypothesis (1859), the real part of any nontrivial zero of the Riemann zeta function is 1/2. The correctness problem in software engineering, proving that a given program meets a given specification. Computability Theory 83 / 155
84 Solving problems by testing the halting problem 2 A conjecture is finitely refutable if verifying a finite number of instances suffices to disprove it. A systematic enumeration (of the problem s search domain) will find a counterexample if one exists. If the search stops, the conjecture is false; if the search does not halt the conjecture is true. For a finitely refutable problem Π we can construct a program C Π such that Π is false iff C Π halts. Π = Goldbach s conjecture is finitely refutable and one can construct a program C Π of 3,484 bits. Computability Theory 84 / 155
85 Solving problems by testing the halting problem 3 For example, Π = the Riemann hypothesis is finitely refutable and one can construct a program C Π of 7,780 bits. The twin prime conjecture n { p [p > n & p prime & p + 2 prime]} is not finitely refutable but can still be solved by testing the halting status of a small program. The stronger twin prime conjecture is finitely refutable: n { p [n < p < 2 n+4 & p prime & p + 2 prime]}. Computability Theory 85 / 155
86 Solving problems by testing the halting problem 4 Collatz function is defined by { x/2, if x is even, T (x) = 3x + 1, if x is odd, The Collatz conjecture states that for every seed a > 0, there is an iteration N such that T N (a) = 1. There is a nonconstructive way to prove that there exists a program C Collatz which never stops iff the conjecture is true. Open question: Can you find/write such a program? Computability Theory 86 / 155
87 A Turingunrecognisable language 1 In the proof of Theorem 4.11 we showed that: A TM is Turingrecognisable, A TM is not decidable. Recall that the complement of a language is the language consisting of all strings that are not in the language. We say that a language is coturingrecognisable if it is the complement of a TuringRecognisable language. We will show that A TM is not coturingrecognisable. Computability Theory 87 / 155
88 A Turingunrecognisable language 2 Theorem 4.22 A language is decidable iff it is Turingrecognisable and coturingrecognisable. Proof. We have two directions to prove. Any decidable language is Turingrecognisable, and the complement of a decidable language also is decidable. For the other direction, assume A and A are Turingrecognisable. Let M 1 be the recogniser for A and M 2 be the recogniser for A. Computability Theory 88 / 155
89 A Turingunrecognisable language 2 The following TM is a decider for A: D = On input string w: 1. Run both M 1 and M 2 on input w in parallel. 2. If M 1 accepts, accept; if M 2 accepts, reject. We show that D decides A: Every string w is either in A or A, but not in both. Therefore either M 1 or M 2 must accept w. Because D halts whenever M 1 or M 2 accepts, D always halts and so it is a decider. Furthermore, D accepts all strings in A and rejects all strings not in A, so it is a decider for A. Computability Theory 89 / 155
90 A Turingunrecognisable language 3 Corollary 4.23 A TM is not Turingrecognisable. Proof. By the proof of Theorem 4.11, A TM is Turingrecognisable. If A TM also were Turingrecognisable, A TM would be decidable. Theorem 4.11 tells us that A TM is not decidable, so A TM must not be Turingrecognisable. Computability Theory 90 / 155
91 Undecidable problems 1 The strategy used in the proof of undecidability of HALT TM was the following: we assumed that HALT TM is decidable and use that assumption to show that A TM is decidable, contradicting Theorem We will use this method to prove the undecidability of other problems. Computability Theory 91 / 155
92 Undecidable problems 2 E TM = { M : M is a TM and L(M) = }. Theorem 5.2 E TM is undecidable. Proof. First we construct the auxiliary TM M 1 based on a TM M and string w: M 1 = On input string x: 1. If x w, reject. 2. If x = w, run M on input w and accept if M 1 does. Computability Theory 92 / 155
93 Undecidable problems 3 Proof of Theorem 5.2 continued. Assume that TM R decides E TM and construct TM S that decides A TM : S = On input M, w, an encoding of a TM M and a string w: 1. Use the description of M and w to construct the TM M 1 described before. 2. Run R on input M If R accepts, reject; if R rejects, accept. If R were a decider for E TM, S would be a decider for A TM, a contradiction. Computability Theory 93 / 155
94 Undecidable problems 4 EQ TM = { M 1, M 2 : M 1 and M 2 are TMs and L(M 1 ) = L(M 2 )}. Theorem 5.4 EQ TM is undecidable. Proof. Let R decide EQ TM and construct TM S to decide E TM : S = On input M, where M is a TM: 1. Run R on input M, M 1, where M 1 is a TM that rejects all inputs. 2. If R accepts, accept; if R rejects, reject. If R decides EQ TM, S decides E TM, which by Theorem 5.2 is a contradiction. Computability Theory 94 / 155
95 Mapping reducibility 1 The notion of reducing one problem to another may be defined formally in one of several ways. Our choice is a simple type of reducibility called mapping reducibility or manyone reducibility. A problem A can be reduced to a problem B by using a mapping reducibility if there is a computable function that converts instances of problem A to instances of problem B. If we have such a conversion function we can solve A with a solver for B. Reason: any instance of A can be solved by first using the reduction to convert it to an instance of B and then applying the solver for B. Computability Theory 95 / 155
96 Mapping reducibility 2 A function f : Σ Σ is called computable if some TM M, on every input w, halts with f (w) on its tape. All usual arithmetic operations on integers are computable functions. All usual string operations are computable functions. Computable functions may be transformations of TM descriptions. For example, the function that takes an input string w and returns 1 if w = M is an encoding of a TM or returns 0 in the opposite case, is computable. Computability Theory 96 / 155
97 Formal definition of mapping reducibility 1 Language A is mapping reducible to language B, written A m B, if there is a computable function f : Σ Σ, where for every w Σ w A f (w) B. The function f is called the reduction of A to B. To test whether w A we use the reduction f to map w to f (w) and test whether f (w) B. Computability Theory 97 / 155
98 Formal definition of mapping reducibility 2 Theorem 5.22 If A m B and B is decidable, then A is decidable. Proof. Let M be the decider for B and f be the reduction from A to B. Here is a decider N for A: N = On input w: 1. Compute f (w). 2. Run M on input f (w) and output whatever M outputs. So, w A iff f (w) B because f is a reduction from A to B. Computability Theory 98 / 155
99 Formal definition of mapping reducibility 3 Corollary If A m B and A is undecidable, then B is undecidable. Proof. Assume by contradiction that B is decidable. Then, by Theorem 5.22, A is decidable, a contradiction. Computability Theory 99 / 155
100 Formal definition of mapping reducibility 4 Theorem 5.28 If A m B and B is Turingrecognisable, then A is Turingrecognisable. Proof. The proof is the same as that of Theorem 5.22, except that M and N are recognisers instead of deciders. Let M be the recogniser for B and f be the reduction from A to B. The recogniser N for A is: N = On input w: 1. Compute f (w). 2. Run M on input f (w) and output whatever M outputs. So, w A iff f (w) B because f is a reduction from A to B. Computability Theory 100 / 155
101 Formal definition of mapping reducibility 5 Corollary 5.29 If A m B and A is not Turingrecognisable, then B is not Turingrecognisable. Proof. Assume A m B. If A is not Turingrecognisable then B cannot be Turingrecognisable because by Theorem 5.28 A would be Turingrecognisable. Computability Theory 101 / 155
102 Example 1 Example 5.24 A TM m HALT TM. Proof. We will construct a computable function f that takes input of the form M, w and returns output of the form M, w, where M, w A TM M, w HALT TM. The following TM F computes f. F = On input M, w : 1. Construct the following machine M. M = On input x: 1. Run M on x. 2. If M accepts, accept. 3. If M rejects, enter a loop. 2. Output M, w. Computability Theory 102 / 155
103 Example continued 1 Improperly formed input strings i.e. inputs not of the form M, w (where M is a TM and w is a string) can be detected and in such a case F outputs a string not in HALT TM. How does the proof compare with the proof of Theorem 5.1? Computability Theory 103 / 155
104 Example 2 Example 5.26 E TM m EQ TM. Proof. The reduction f maps the input M to the output M, M 1, where M 1 is the TM that rejects all inputs (revisit the proof of Theorem 5.4). Computability Theory 104 / 155
105 Example 3 Example 5.27 A TM m E TM. Proof. From the reduction in the proof of Theorem 5.2, we construct a function f that takes input M, w and produces output M 1, where M 1 is the TM described in that proof. As M accepts w iff L(M 1 ), f is a mapping reduction from A TM to E TM. Computability Theory 105 / 155
106 Example 4 Example A If A TM m B, then B is not Turingrecognisable. Proof. As A TM m B, we have A TM m B = B, so by Corollary 4.23 and Corollary 5.29, B is not Turingrecognisable. Example B E TM is not Turingrecognisable. Proof. Use Example A and Example Computability Theory 106 / 155
107 Example 5 Example C If A m B, A m B, and B is Turingrecognisable, then A is decidable. Proof. First, from A m B and B is Turingrecognisable we deduce, using Theorem 5.28, that A is Turingrecognisable. If A m B, then A m B = B and B is Turingrecognisable we deduce, using Theorem 5.28, that A is Turingrecognisable. Since A and A are Turingrecognisable, A is decidable (Theorem 4.22). Computability Theory 107 / 155
108 Example 6 Example D A TM m E TM. Proof. In Example C we put: A = A TM, B = E TM. By Example 5.27, A TM m E TM. Since the language E TM is Turingrecognisable, if A TM m E TM, then by Example C, A TM is decidable, a contradiction (Theorem 4.9). Computability Theory 108 / 155
109 The remarkable language EQ TM 1 Recall that EQ TM = { M 1, M 2 : M 1 and M 2 are TMs and L(M 1 ) = L(M 2 )}. Theorem 5.30 EQ TM is not Turingrecognisable nor coturingrecognisable. Proof. First we show that EQ TM is not Turingrecognisable by showing that A TM m EQ TM. Computability Theory 109 / 155
110 The remarkable language EQ TM 2 The reducing function f works as follows. F = On input M, w where M is a TM and w a string: 1. Construct the following two machines M 1 and M 2. M 1 = On any input: 1. Reject. M 2 = On any input: 1. Run M on w. If it accepts, accept. 2. Output M 1, M 2. Here, M 1 accepts nothing. If M accepts w, M 2 accepts everything, and so the two TMs are not equivalent. Conversely, if M doesn t accept w, M 2 accepts nothing, and the TMs are equivalent. Computability Theory 110 / 155
111 The remarkable language EQ TM 3 To show that EQ TM is not TuringRecognisable we give a reduction from A TM to the complement of EQ TM which is simply EQ TM. Hence we show that A TM m EQ TM. The following TM G computes the reducing function g: G = The input is M, w where M is a TM and w a string: 1. Construct the following two machines M 1 and M 2. M 1 = On any input: 1. Accept. M 2 = On any input: 1. Run M on w. If it accepts, accept. 2. Output M 1, M 2. Computability Theory 111 / 155
112 The remarkable language EQ TM 4 The only difference between f and g is in the construction of the TM machine M 1. In f, machine M 1 always rejects, whereas in g it always accepts. In both f and g, M accepts w iff M 2 always accepts. In g, M accepts w iff L(M 1 ) = L(M 2 ) showing that A TM m EQ TM. Computability Theory 112 / 155
113 Rice s theorem 1 Rice s theorem (Problem 5.28) Let P be a language consisting of TM descriptions where P fulfils two conditions. First, P is nontrivial it contains some, but not all, TM descriptions. Second, P is a property of the TM s language whenever L(M 1 ) = L(M 2 ), we have M 1 P iff M 2 P. Here M 1 and M 2 are any TMs. Then, P is undecidable. Proof. Assume for the sake of a contradiction that P is decidable and let R P be a TM that decides P. We will show how to use R P to decide A TM obtaining a contradiction (Theorem 4.11). Computability Theory 113 / 155
114 Rice s theorem 2 Proof of Rice s theorem continued. First let T be a TM such that L(T ) =. You may assume that T P without loss of generality (otherwise proceed with P instead of P). Secondly, because P is not trivial, there exists a TM T with T P. Computability Theory 114 / 155
115 Rice s theorem 2 Proof of Rice s theorem continued. The following TM S decides A TM. S = On input M, w : 1. Use M and w to construct the following TM M w. M w = On input x: 1. Simulate M on w. If it halts and rejects, reject. If it accepts, proceed to stage Simulate T on x. If it accepts, accept. 2. Use TM R P to determine whether M w P. If YES, accept. If NO, reject. Computability Theory 115 / 155
116 Rice s theorem 3 Proof of Rice s theorem continued. TM M w simulates T if M accepts w, hence { L(T ), if M accepts w, L(M w ) =, if M does not accept w, { L(T ), if M accepts w, = L(T ), if M does not accept w. Therefore M w P iff M accepts w. Computability Theory 116 / 155
117 Rice s theorem 4 Corollary to Rice s Theorem The following languages 1 { M : L(M) = }, 2 { M : L(M) = infinite}, 3 { M : L(M) = finite}, 4 { M : L(M) = regular}, 5 { M : 10 L(M)}, 6 { M : L(M) = Σ }. are undecidable. Proof. Each property specified in the languages above is a nontrivial property of the TM s language so it satisfies Rice s Theorem. Computability Theory 117 / 155
Computability Theory
Computability Theory Cristian S. Calude May 2012 Computability Theory 1 / 1 Bibliography M. Sipser. Introduction to the Theory of Computation, PWS 1997. (textbook) Computability Theory 2 / 1 Supplementary
More informationIntroduction to Turing Machines. Reading: Chapters 8 & 9
Introduction to Turing Machines Reading: Chapters 8 & 9 1 Turing Machines (TM) Generalize the class of CFLs: Recursively Enumerable Languages Recursive Languages ContextFree Languages Regular Languages
More informationThe Turing machine model of computation
The Turing machine model of computation For most of the remainder of the course we will study the Turing machine model of computation, named after Alan Turing (1912 1954) who proposed the model in 1936.
More informationFinal Exam Comments. UVa  cs302: Theory of Computation Spring < Total
UVa  cs302: Theory of Computation Spring 2008 Final Exam Comments < 50 50 59 60 69 70 79 80 89 90 94 95102 Total 2 6 8 22 16 16 12 Problem 1: Short Answers. (20) For each question, provide a correct,
More informationCSCE 551: ChinTser Huang. University of South Carolina
CSCE 551: Theory of Computation ChinTser Huang huangct@cse.sc.edu University of South Carolina ChurchTuring Thesis The definition of the algorithm came in the 1936 papers of Alonzo Church h and Alan
More informationCS 361 Meeting 26 11/10/17
CS 361 Meeting 26 11/10/17 1. Homework 8 due Announcements A Recognizable, but Undecidable Language 1. Last class, I presented a brief, somewhat inscrutable proof that the language A BT M = { M w M is
More informationTuring Machines Part III
Turing Machines Part III Announcements Problem Set 6 due now. Problem Set 7 out, due Monday, March 4. Play around with Turing machines, their powers, and their limits. Some problems require Wednesday's
More information(a) Definition of TMs. First Problem of URMs
Sec. 4: Turing Machines First Problem of URMs (a) Definition of the Turing Machine. (b) URM computable functions are Turing computable. (c) Undecidability of the Turing Halting Problem That incrementing
More informationDecidability and Undecidability
Decidability and Undecidability Major Ideas from Last Time Every TM can be converted into a string representation of itself. The encoding of M is denoted M. The universal Turing machine U TM accepts an
More informationComputational Models Lecture 9, Spring 2009
Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. p. 1 Computational Models Lecture 9, Spring 2009 Reducibility among languages Mapping reductions More undecidable
More informationCSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION "Winter" 2018 http://cseweb.ucsd.edu/classes/wi18/cse105ab/ Today's learning goals Sipser Ch 4.2 Trace highlevel descriptions of algorithms for computational problems. Use
More informationMore About Turing Machines. Programming Tricks Restrictions Extensions Closure Properties
More About Turing Machines Programming Tricks Restrictions Extensions Closure Properties 1 Overview At first, the TM doesn t look very powerful. Can it really do anything a computer can? We ll discuss
More informationComputability Theory. CS215, Lecture 6,
Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David
More informationNondeterministic finite automata
Lecture 3 Nondeterministic finite automata This lecture is focused on the nondeterministic finite automata (NFA) model and its relationship to the DFA model. Nondeterminism is an important concept in the
More information16.1 Countability. CS125 Lecture 16 Fall 2014
CS125 Lecture 16 Fall 2014 16.1 Countability Proving the nonexistence of algorithms for computational problems can be very difficult. Indeed, we do not know how to prove P NP. So a natural question is
More informationDecision Problems with TM s. Lecture 31: Halting Problem. Universe of discourse. Semidecidable. Look at following sets: CSCI 81 Spring, 2012
Decision Problems with TM s Look at following sets: Lecture 31: Halting Problem CSCI 81 Spring, 2012 Kim Bruce A TM = { M,w M is a TM and w L(M)} H TM = { M,w M is a TM which halts on input w} TOTAL TM
More informationTheory of Computation Lecture Notes. Problems and Algorithms. Class Information
Theory of Computation Lecture Notes Prof. YuhDauh Lyuu Dept. Computer Science & Information Engineering and Department of Finance National Taiwan University Problems and Algorithms c 2004 Prof. YuhDauh
More informationCpSc 421 Homework 9 Solution
CpSc 421 Homework 9 Solution Attempt any three of the six problems below. The homework is graded on a scale of 100 points, even though you can attempt fewer or more points than that. Your recorded grade
More informationCS154, Lecture 10: Rice s Theorem, Oracle Machines
CS154, Lecture 10: Rice s Theorem, Oracle Machines Moral: Analyzing Programs is Really, Really Hard But can we more easily tell when some program analysis problem is undecidable? Problem 1 Undecidable
More informationTuring Machines Part II
Turing Machines Part II Problem Set Set Five Five due due in in the the box box up up front front using using a late late day. day. Hello Hello Condensed Slide Slide Readers! Readers! This This lecture
More informationOn Rice s theorem. Hans Hüttel. October 2001
On Rice s theorem Hans Hüttel October 2001 We have seen that there exist languages that are Turingacceptable but not Turingdecidable. An important example of such a language was the language of the Halting
More informationTuring machines and linear bounded automata
and linear bounded automata Informatics 2A: Lecture 29 John Longley School of Informatics University of Edinburgh jrl@inf.ed.ac.uk 25 November, 2011 1 / 13 1 The Chomsky hierarchy: summary 2 3 4 2 / 13
More informationLecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009
CS 373: Theory of Computation Sariel HarPeled and Madhusudan Parthasarathy Lecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009 This lecture covers Rice s theorem, as well as
More informationAutomata and Computability. Solutions to Exercises
Automata and Computability Solutions to Exercises Spring 27 Alexis Maciel Department of Computer Science Clarkson University Copyright c 27 Alexis Maciel ii Contents Preface vii Introduction 2 Finite Automata
More informationFurther discussion of Turing machines
Further discussion of Turing machines In this lecture we will discuss various aspects of decidable and Turingrecognizable languages that were not mentioned in previous lectures. In particular, we will
More informationLecture Notes: The Halting Problem; Reductions
Lecture Notes: The Halting Problem; Reductions COMS W3261 Columbia University 20 Mar 2012 1 Review Key point. Turing machines can be encoded as strings, and other Turing machines can read those strings
More informationUndecidable Problems and Reducibility
University of Georgia Fall 2014 Reducibility We show a problem decidable/undecidable by reducing it to another problem. One type of reduction: mapping reduction. Definition Let A, B be languages over Σ.
More informationTuring Machines (TM) The Turing machine is the ultimate model of computation.
TURING MACHINES Turing Machines (TM) The Turing machine is the ultimate model of computation. Alan Turing (92 954), British mathematician/engineer and one of the most influential scientists of the last
More informationPS2  Comments. University of Virginia  cs3102: Theory of Computation Spring 2010
University of Virginia  cs3102: Theory of Computation Spring 2010 PS2  Comments Average: 77.4 (full credit for each question is 100 points) Distribution (of 54 submissions): 90, 12; 80 89, 11; 7079,
More informationCSCI3390Assignment 2 Solutions
CSCI3390Assignment 2 Solutions due February 3, 2016 1 TMs for Deciding Languages Write the specification of a Turing machine recognizing one of the following three languages. Do one of these problems.
More information6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, Class 10 Nancy Lynch
6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, 2010 Class 10 Nancy Lynch Today Final topic in computability theory: SelfReference and the Recursion
More informationHomework 8. a b b a b a b. twoway, read/write
Homework 8 309 Homework 8 1. Describe a TM that accepts the set {a n n is a power of 2}. Your description should be at the level of the descriptions in Lecture 29 of the TM that accepts {ww w Σ } and the
More informationTheory of computation: initial remarks (Chapter 11)
Theory of computation: initial remarks (Chapter 11) For many purposes, computation is elegantly modeled with simple mathematical objects: Turing machines, finite automata, pushdown automata, and such.
More informationTuring Machines Part II
Turing Machines Part II Hello Hello Condensed Slide Slide Readers! Readers! This This lecture lecture is is almost almost entirely entirely animations that that show show how how each each Turing Turing
More informationTheory of computation: initial remarks (Chapter 11)
Theory of computation: initial remarks (Chapter 11) For many purposes, computation is elegantly modeled with simple mathematical objects: Turing machines, finite automata, pushdown automata, and such.
More informationModels of Computation, Recall Register Machines. A register machine (sometimes abbreviated to RM) is specified by:
Models of Computation, 2010 1 Definition Recall Register Machines A register machine (sometimes abbreviated M) is specified by: Slide 1 finitely many registers R 0, R 1,..., R n, each capable of storing
More informationDRAFT. Algebraic computation models. Chapter 14
Chapter 14 Algebraic computation models Somewhat rough We think of numerical algorithms rootfinding, gaussian elimination etc. as operating over R or C, even though the underlying representation of the
More informationFinite Automata and Regular languages
Finite Automata and Regular languages Huan Long Shanghai Jiao Tong University Acknowledgements Part of the slides comes from a similar course in Fudan University given by Prof. Yijia Chen. http://basics.sjtu.edu.cn/
More informationContextfree grammars and languages
Contextfree grammars and languages The next class of languages we will study in the course is the class of contextfree languages. They are defined by the notion of a contextfree grammar, or a CFG for
More informationCS243, Logic and Computation Nondeterministic finite automata
CS243, Prof. Alvarez NONDETERMINISTIC FINITE AUTOMATA (NFA) Prof. Sergio A. Alvarez http://www.cs.bc.edu/ alvarez/ Maloney Hall, room 569 alvarez@cs.bc.edu Computer Science Department voice: (67) 5524333
More informationTheory of Computation
Theory of Computation Lecture #2 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 1 Lecture 2: Overview Recall some basic definitions from Automata Theory.
More informationPeano Arithmetic. CSC 438F/2404F Notes (S. Cook) Fall, Goals Now
CSC 438F/2404F Notes (S. Cook) Fall, 2008 Peano Arithmetic Goals Now 1) We will introduce a standard set of axioms for the language L A. The theory generated by these axioms is denoted PA and called Peano
More information1 Acceptance, Rejection, and I/O for Turing Machines
1 Acceptance, Rejection, and I/O for Turing Machines Definition 1.1 (Initial Configuration) If M = (K,Σ,δ,s,H) is a Turing machine and w (Σ {, }) then the initial configuration of M on input w is (s, w).
More informationLecture 13: Foundations of Math and Kolmogorov Complexity
6.045 Lecture 13: Foundations of Math and Kolmogorov Complexity 1 SelfReference and the Recursion Theorem 2 Lemma: There is a computable function q : Σ* Σ* such that for every string w, q(w) is the description
More information3 The language of proof
3 The language of proof After working through this section, you should be able to: (a) understand what is asserted by various types of mathematical statements, in particular implications and equivalences;
More informationFORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY Chomsky Normal Form and TURING MACHINES TUESDAY Feb 4 CHOMSKY NORMAL FORM A contextfree grammar is in Chomsky normal form if every rule is of the form:
More informationDefinition: Alternating time and space Game Semantics: State of machine determines who
CMPSCI 601: Recall From Last Time Lecture Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins
More information,
Kolmogorov Complexity Carleton College, CS 254, Fall 2013, Prof. Joshua R. Davis based on Sipser, Introduction to the Theory of Computation 1. Introduction Kolmogorov complexity is a theory of lossless
More information1 More finite deterministic automata
CS 125 Section #6 Finite automata October 18, 2016 1 More finite deterministic automata Exercise. Consider the following game with two players: Repeatedly flip a coin. On heads, player 1 gets a point.
More informationTheory of Computation
Theory of Computation Lecture #10 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 43 Lecture 10: Overview Linear Bounded Automata Acceptance Problem for LBAs
More information7.1 The Origin of Computer Science
CS125 Lecture 7 Fall 2016 7.1 The Origin of Computer Science Alan Mathison Turing (1912 1954) turing.jpg 170!201 pixels On Computable Numbers, with an Application to the Entscheidungsproblem 1936 1936:
More informationAutomata Theory CS S12 Turing Machine Modifications
Automata Theory CS4112015S12 Turing Machine Modifications David Galles Department of Computer Science University of San Francisco 120: Extending Turing Machines When we added a stack to NFA to get a
More informationModels. Models of Computation, Turing Machines, and the Limits of Turing Computation. Effective Calculability. Motivation for Models of Computation
Turing Computation /0/ Models of Computation, Turing Machines, and the Limits of Turing Computation Bruce MacLennan Models A model is a tool intended to address a class of questions about some domain of
More informationAnnouncements. Problem Set 6 due next Monday, February 25, at 12:50PM. Midterm graded, will be returned at end of lecture.
Turing Machines Hello Hello Condensed Slide Slide Readers! Readers! This This lecture lecture is is almost almost entirely entirely animations that that show show how how each each Turing Turing machine
More informationFORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY REVIEW for MIDTERM 1 THURSDAY Feb 6 Midterm 1 will cover everything we have seen so far The PROBLEMS will be from Sipser, Chapters 1, 2, 3 It will be
More informationAn Algebraic Characterization of the Halting Probability
CDMTCS Research Report Series An Algebraic Characterization of the Halting Probability Gregory Chaitin IBM T. J. Watson Research Center, USA CDMTCS305 April 2007 Centre for Discrete Mathematics and Theoretical
More informationBefore we show how languages can be proven not regular, first, how would we show a language is regular?
CS35 Proving Languages not to be Regular Before we show how languages can be proven not regular, first, how would we show a language is regular? Although regular languages and automata are quite powerful
More informationIntroduction to the Theory of Computation. Automata 1VO + 1PS. Lecturer: Dr. Ana Sokolova.
Introduction to the Theory of Computation Automata 1VO + 1PS Lecturer: Dr. Ana Sokolova http://cs.unisalzburg.at/~anas/ Setup and Dates Lectures and Instructions 23.10. 3.11. 17.11. 24.11. 1.12. 11.12.
More information1 Computational Problems
Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation
More informationTheory of Computation p.1/?? Theory of Computation p.2/?? Unknown: Implicitly a Boolean variable: true if a word is
Abstraction of Problems Data: abstracted as a word in a given alphabet. Σ: alphabet, a finite, nonempty set of symbols. Σ : all the words of finite length built up using Σ: Conditions: abstracted as a
More informationCSE 105 Theory of Computation
CSE 105 Theory of Computation http://www.jflap.org/jflaptmp/ Professor Jeanne Ferrante 1 Undecidability Today s Agenda Review and More Problems A NonTR Language Reminders and announcements: HW 7 (Last!!)
More informationTuring machines and computable functions
Turing machines and computable functions In this chapter we address the question: how can we define the class of discrete functions that are computable? Probably, most of you are familiar with a few programming
More informationLecture 23 : Nondeterministic Finite Automata DRAFT Connection between Regular Expressions and Finite Automata
CS/Math 24: Introduction to Discrete Mathematics 4/2/2 Lecture 23 : Nondeterministic Finite Automata Instructor: Dieter van Melkebeek Scribe: Dalibor Zelený DRAFT Last time we designed finite state automata
More informationEssential facts about NPcompleteness:
CMPSCI611: NP Completeness Lecture 17 Essential facts about NPcompleteness: Any NPcomplete problem can be solved by a simple, but exponentially slow algorithm. We don t have polynomialtime solutions
More informationComputer Science. Questions for discussion Part II. Computer Science COMPUTER SCIENCE. Section 4.2.
COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I I : A L G O R I T H M S, T H E O R Y, A N D M A C H I N E S Computer Science Computer Science An Interdisciplinary Approach Section 4.2 ROBERT SEDGEWICK
More information6.8 The Post Correspondence Problem
6.8. THE POST CORRESPONDENCE PROBLEM 423 6.8 The Post Correspondence Problem The Post correspondence problem (due to Emil Post) is another undecidable problem that turns out to be a very helpful tool for
More information1 Computational problems
80240233: Computational Complexity Lecture 1 ITCS, Tsinghua Univesity, Fall 2007 9 October 2007 Instructor: Andrej Bogdanov Notes by: Andrej Bogdanov The aim of computational complexity theory is to study
More informationVerifying whether OneTape Turing Machines Run in Linear Time
Electronic Colloquium on Computational Complexity, Report No. 36 (2015) Verifying whether OneTape Turing Machines Run in Linear Time David Gajser IMFM, Jadranska 19, 1000 Ljubljana, Slovenija david.gajser@fmf.unilj.si
More informationChomsky Normal Form and TURING MACHINES. TUESDAY Feb 4
Chomsky Normal Form and TURING MACHINES TUESDAY Feb 4 CHOMSKY NORMAL FORM A contextfree grammar is in Chomsky normal form if every rule is of the form: A BC A a S ε B and C aren t start variables a is
More informationECS 120: Theory of Computation UC Davis Phillip Rogaway February 16, Midterm Exam
ECS 120: Theory of Computation Handout MT UC Davis Phillip Rogaway February 16, 2012 Midterm Exam Instructions: The exam has six pages, including this cover page, printed out twosided (no more wasted
More informationacs07: Decidability Decidability Andreas Karwath und Malte Helmert Informatik Theorie II (A) WS2009/10
Decidability Andreas Karwath und Malte Helmert 1 Overview An investigation into the solvable/decidable Decidable languages The halting problem (undecidable) 2 Decidable problems? Acceptance problem : decide
More informationChapter 2. Mathematical Reasoning. 2.1 Mathematical Models
Contents Mathematical Reasoning 3.1 Mathematical Models........................... 3. Mathematical Proof............................ 4..1 Structure of Proofs........................ 4.. Direct Method..........................
More informationLecture 12: Mapping Reductions
Lecture 12: Mapping Reductions October 18, 2016 CS 1010 Theory of Computation Topics Covered 1. The Language EQ T M 2. Mapping Reducibility 3. The Post Correspondence Problem 1 The Language EQ T M The
More informationState Machines. Example FSM: Roboant
page 1 State Machines 1) State Machine Design 2) How can we improve on FSMs? 3) Turing Machines 4) Computability Oh genie, will you now tell me what it means to compute? Doctor, I think you ve built a
More informationRussell s logicism. Jeff Speaks. September 26, 2007
Russell s logicism Jeff Speaks September 26, 2007 1 Russell s definition of number............................ 2 2 The idea of reducing one theory to another.................... 4 2.1 Axioms and theories.............................
More informationCS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression
CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression Rosencrantz & Guildenstern Are Dead (Tom Stoppard) Rigged Lottery? And the winning numbers are: 1, 2, 3, 4, 5, 6 But is
More informationIntroduction to Metalogic
Philosophy 135 Spring 2008 Tony Martin Introduction to Metalogic 1 The semantics of sentential logic. The language L of sentential logic. Symbols of L: Remarks: (i) sentence letters p 0, p 1, p 2,... (ii)
More informationHOW TO CREATE A PROOF. Writing proofs is typically not a straightforward, algorithmic process such as calculating
HOW TO CREATE A PROOF ALLAN YASHINSKI Abstract We discuss how to structure a proof based on the statement being proved Writing proofs is typically not a straightforward, algorithmic process such as calculating
More informationAutomata and Computability
Automata and Computability Fall 207 Alexis Maciel Department of Computer Science Clarkson University Copyright c 207 Alexis Maciel ii Contents Preface vii Introduction 2 Finite Automata 5 2. Turing Machines...............................
More informationTheory of Computation
Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation / Lecture 3: Overview Decidability of Logical Theories Presburger arithmetic Decidability of Presburger Arithmetic Dr.
More informationWe are going to discuss what it means for a sequence to converge in three stages: First, we define what it means for a sequence to converge to zero
Chapter Limits of Sequences Calculus Student: lim s n = 0 means the s n are getting closer and closer to zero but never gets there. Instructor: ARGHHHHH! Exercise. Think of a better response for the instructor.
More informationIntroduction to the Theory of Computation. Automata 1VO + 1PS. Lecturer: Dr. Ana Sokolova.
Introduction to the Theory of Computation Automata 1VO + 1PS Lecturer: Dr. Ana Sokolova http://cs.unisalzburg.at/~anas/ Setup and Dates Lectures Tuesday 10:45 pm  12:15 pm Instructions Tuesday 12:30
More informationFinish KComplexity, Start Time Complexity
6.045 Finish KComplexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts
More informationNotes on Complexity Theory Last updated: November, Lecture 10
Notes on Complexity Theory Last updated: November, 2015 Lecture 10 Notes by Jonathan Katz, lightly edited by Dov Gordon. 1 Randomized Time Complexity 1.1 How Large is BPP? We know that P ZPP = RP corp
More informationVariants of Turing Machine (intro)
CHAPTER 3 The ChurchTuring Thesis Contents Turing Machines definitions, examples, Turingrecognizable and Turingdecidable languages Variants of Turing Machine Multitape Turing machines, nondeterministic
More information6.841/18.405J: Advanced Complexity Wednesday, February 12, Lecture Lecture 3
6.841/18.405J: Advanced Complexity Wednesday, February 12, 2003 Lecture Lecture 3 Instructor: Madhu Sudan Scribe: Bobby Kleinberg 1 The language MinDNF At the end of the last lecture, we introduced the
More informationCSC : Homework #3
CSC 707001: Homework #3 William J. Cook wjcook@math.ncsu.edu Monday, March 15, 2004 1 Exercise 4.13 on page 118 (3Register RAM) Exercise 4.13 Verify that a RAM need not have an unbounded number of registers.
More informationDecidability. William Chan
Decidability William Chan Preface : In 1928, David Hilbert gave a challenge known as the Entscheidungsproblem, which is German for Decision Problem. Hilbert s problem asked for some purely mechanical procedure
More informationCSC 5170: Theory of Computational Complexity Lecture 5 The Chinese University of Hong Kong 8 February 2010
CSC 5170: Theory of Computational Complexity Lecture 5 The Chinese University of Hong Kong 8 February 2010 So far our notion of realistic computation has been completely deterministic: The Turing Machine
More informationKybernetika. Daniel Reidenbach; Markus L. Schmid Automata with modulo counters and nondeterministic counter bounds
Kybernetika Daniel Reidenbach; Markus L. Schmid Automata with modulo counters and nondeterministic counter bounds Kybernetika, Vol. 50 (2014), No. 1, 66 94 Persistent URL: http://dml.cz/dmlcz/143764 Terms
More informationcse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska
cse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska LECTURE 13 CHAPTER 4 TURING MACHINES 1. The definition of Turing machine 2. Computing with Turing machines 3. Extensions of Turing
More informationLecture 20: conp and Friends, Oracles in Complexity Theory
6.045 Lecture 20: conp and Friends, Oracles in Complexity Theory 1 Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode:
More informationMA554 Assessment 1 Cosets and Lagrange s theorem
MA554 Assessment 1 Cosets and Lagrange s theorem These are notes on cosets and Lagrange s theorem; they go over some material from the lectures again, and they have some new material it is all examinable,
More informationBusch Complexity Lectures: Turing s Thesis. Costas Busch  LSU 1
Busch Complexity Lectures: Turing s Thesis Costas Busch  LSU 1 Turing s thesis (1930): Any computation carried out by mechanical means can be performed by a Turing Machine Costas Busch  LSU 2 Algorithm:
More informationFORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY KOLMOGOROVCHAITIN (descriptive) COMPLEXITY TUESDAY, MAR 18 CAN WE QUANTIFY HOW MUCH INFORMATION IS IN A STRING? A = 01010101010101010101010101010101
More information1 Deterministic Turing Machines
Time and Space Classes Exposition by William Gasarch 1 Deterministic Turing Machines Turing machines are a model of computation. It is believed that anything that can be computed can be computed by a Turing
More informationCS 21 Decidability and Tractability Winter Solution Set 3
CS 21 Decidability and Tractability Winter 2018 Posted: January 31 Solution Set 3 If you have not yet turned in the Problem Set, you should not consult these solutions. 1. (a) A 2NPDA is a 7tuple (Q,,
More informationAutomata: a short introduction
ILIAS, University of Luxembourg Discrete Mathematics II May 2012 What is a computer? Real computers are complicated; We abstract up to an essential model of computation; We begin with the simplest possible
More informationCPS 220 Theory of Computation REGULAR LANGUAGES
CPS 22 Theory of Computation REGULAR LANGUAGES Introduction Model (def) a miniature representation of a thing; sometimes a facsimile Iraq village mockup for the Marines Scientific modelling  the process
More informationDefinition: Alternating time and space Game Semantics: State of machine determines who
CMPSCI 601: Recall From Last Time Lecture 3 Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins
More information