# (a) Definition of TMs. First Problem of URMs

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Sec. 4: Turing Machines First Problem of URMs (a) Definition of the Turing Machine. (b) URM computable functions are Turing computable. (c) Undecidability of the Turing Halting Problem That incrementing a number by one takes arbitrarily many steps happens on a real computer as well: If we want to represent arbitrary big numbers on the computer, we have to represent them by multiple machine integers Then incrementing a number by one will correspond to arbitrarily many machine instructions (although usually only a few). However, often in complexity theory this problem is ignored because the effect is marginal in real applications. The exception are applications in which very big integers occur, e.g. tests for primality. There this effect cannot be ignored any more. 6 Computability Theory, Lent Term 2008, Sect (a) Definition of TMs There are two problems with the model of a URM: Execution of a single URM instruction might take arbitrarily long: Consider succ(n). If R n contains in binary 111 } {{ 111}, this instruction k times replaces it by 1} 000 {{ 000}. k times We have to replace k symbols 1 by 0. k is arbitrary this single step might take arbitrarily long time. First Problem of URMs If one takes this effect into account, one needs in many examples to multiply the running time by a factor of ln(n), where n is the largest number occurring. Therefore URMs unsuitable as a basis for defining the precise complexity of algorithms. However, there are theorems linking complexity of URMs to actual complexities of algorithms. 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-2

2 Second Problem of URMs Idea of a Turing Machine We aim at a notion of computability, which covers all possible ways of computing something, independently of any concrete machine. URMs are a model of computation which covers current standard computers. However, there might be completely different notions of computability, based on symbolic manipulations of a sequence of characters, where it might be more complicated to see directly that all such computations can be simulated by a URM. It is more easy to see that such notions are covered by the Turing machine model of computation. Steps in this formulation: Algorithm should be deterministic. The agent will use only finitely many symbols, put at discrete positions on the paper = Computability Theory, Lent Term 2008, Sec. 4 (a) 4-5 Idea of a Turing Machine Idea of a Turing machine ( TM): Analysis of a computation carried out by a human being ( agent) on a piece of paper = Idea of a Turing Machine We can replace a two-dimensional piece of paper by one potentially infinite tape, by using a special symbol for a line break. Each entry on this tape is called a cell: = CR 1 5 CR 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-6

3 Steps in Formalising TMs Steps in Formalising TMs In the real situation, agent can look at several cells at the same time, but bounded by his physical capability. Can be simulated by looking at one cell only at any time, and moving around in order to get information about neighbouring cells. In the real situation, an agent can make arbitrary jumps, but bounded by the physical ability of the agent. Each such jump can be simulated by finitely many one-step jumps. Restriction to one-step movements = CR 1 5 CR Head = CR 1 5 CR Head 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-9 Steps in Formalising TMs In the real situation, an agent can make arbitrary jumps, but bounded by the physical ability of the agent. Each such jump can be simulated by finitely many one-step jumps. Restriction to one-step movements. Steps in Formalising TMs In the real situation, an agent can make arbitrary jumps, but bounded by the physical ability of the agent. Each such jump can be simulated by finitely many one-step jumps. Restriction to one-step movements = CR 1 5 CR Head 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-10

4 Steps in Formalising TMs Steps in Formalising TMs In the real situation, an agent can make arbitrary jumps, but bounded by the physical ability of the agent. Each such jump can be simulated by finitely many one-step jumps. Restriction to one-step movements. Agent operates purely mechanistically: Reads a symbol, and depending on it changes it and makes a movement. Agent himself will have only finite memory. There is a finite state of the agent, and, depending on the state and the symbol at the head, a next state, a new symbol, and a movement is chosen = CR 1 5 CR Head 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-10 Steps in Formalising TMs In the real situation, an agent can make arbitrary jumps, but bounded by the physical ability of the agent. Each such jump can be simulated by finitely many one-step jumps. Restriction to one-step movements = CR 1 5 CR Head Steps in Formalising TMs Agent operates purely mechanistically: Reads a symbol, and depending on it changes it and makes a movement. Agent himself will have only finite memory. There is a finite state of the agent, and, depending on the state and the symbol at the head, a next state, a new symbol, and a movement is chosen = CR 1 5 CR s 0 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-10

5 Steps in Formalising TMs Steps in Formalising TMs Agent operates purely mechanistically: Reads a symbol, and depending on it changes it and makes a movement. Agent himself will have only finite memory. There is a finite state of the agent, and, depending on the state and the symbol at the head, a next state, a new symbol, and a movement is chosen = CR 1 5 CR s 1 Agent operates purely mechanistically: Reads a symbol, and depending on it changes it and makes a movement. Agent himself will have only finite memory. There is a finite state of the agent, and, depending on the state and the symbol at the head, a next state, a new symbol, and a movement is chosen = CR 1 5 CR s 0 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-11 Steps in Formalising TMs Agent operates purely mechanistically: Reads a symbol, and depending on it changes it and makes a movement. Agent himself will have only finite memory. There is a finite state of the agent, and, depending on the state and the symbol at the head, a next state, a new symbol, and a movement is chosen = CR 1 5 CR Definition of TMs = CR 1 5 CR s 0 A Turing machine is a quintuple (Σ, S, I,,s 0 ), where Σ is a finite set of symbols, called the alphabet of the Turing machine. On the tape, the symbols in Σ will be written. S is a finite set of states. s 2 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-11

6 Definition of TMs Meaning of Instructions = CR 1 5 CR s 0 I is a finite sets of quintuples (s,a,s,a,d), where s,s S, a,a Σ D {L, R}, s.t. for every s S, a Σ there is at most one s,a,d s.t. (s,a,s,a,d) S. The elements of I are called instructions. Σ (a symbol for blank). s 0 S (the initial state). A instruction (s,a,s,a,d) I means the following: Example: (s 0, 1,s 1, 0, R) (s 1, 6,s 2, 7, L) If the Turing machine is in state s, and the symbol at position of the head is a, then the state is changed to s, the symbol at this position is changed to a, if D = L, the head moves left, if D = R, the head moves right. 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-13 Meaning of Instructions A instruction (s,a,s,a,d) I means the following: If the Turing machine is in state s, and the symbol at position of the head is a, then the state is changed to s, the symbol at this position is changed to a, if D = L, the head moves left, if D = R, the head moves right. Meaning of Instructions A instruction (s,a,s,a,d) I means the following: If the Turing machine is in state s, and the symbol at position of the head is a, then the state is changed to s, the symbol at this position is changed to a, if D = L, the head moves left, if D = R, the head moves right. Example: (s 0, 1,s 1, 0, R) (s 1, 6,s 2, 7, L) = CR 1 5 CR s 0 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-14

7 Meaning of Instructions Meaning of Instructions A instruction (s,a,s,a,d) I means the following: Example: (s 0, 1,s 1, 0, R) (s 1, 6,s 2, 7, L) If the Turing machine is in state s, and the symbol at position of the head is a, then the state is changed to s, the symbol at this position is changed to a, if D = L, the head moves left, if D = R, the head moves right = CR 1 5 CR s 1 Note that for the above it is important that for every s S, a Σ there is at most one s,a,d s.t. (s,a,s,a,d) S. Without this condition, there might be more than one choice of selecting a new tape symbol, next state and direction. If we omit this definition, we obtain a non-deterministic TM. In this case the machine selects in each step one of the possible choices (provided there exist one) at random. If the Turing machine is in a state s and reads symbol a at his head, and there are no s,a,d s.t. (s,a,s,a,d) I, then the Turing machine stops. 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-14 Meaning of Instructions A instruction (s,a,s,a,d) I means the following: Example: (s 0, 1,s 1, 0, R) (s 1, 6,s 2, 7, L) If the Turing machine is in state s, and the symbol at position of the head is a, then the state is changed to s, the symbol at this position is changed to a, if D = L, the head moves left, if D = R, the head moves right = CR 1 5 CR s 2 Visualisation of TMs A TM (Σ, S, I,,s 0 ) can be visualised by a labelled graph as follows: Vertices: states (i.e. S). Edges: If (s,a,t,b,d) I, then there is an edge s a/b, D Furthermore we write an arrow to the initial state coming from nowhere. If there are several vertices from s to s, one draws only one arrow with one label for each vertex. t 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-14

8 Example Example of a TM The Turing machine with initial state s 0 and instructions {(s 0, 0,s 0, 0, R), (s 0, 1,s 0, 0, R), (s 0,,s 1,, L), (s 1, 0,s 1, 0, L), (s 1,,s 2,, R)} is visualised as follows (we write B instead of ): B/B, L B/B, R s0 0/0, R 0/0, L 1/0, R 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-17 Example The TM on the previous slide sets the binary number the head is pointing to to zero, provided to the left of the head there are is a blank. Exercise: This example assumes that the TM points to the left most digit of a binary number. Modify this TM, so that it works as well if the TM points initially to any digit of a binary number. Development of a TM with Σ = {0, 1, }, where is the symbol for the blank entry. Functionality of the TM: Assume initially the following: The tape contains binary number, The rest of the tape contains. The head points to any digit of the number. The TM in state s 0. Then the TM stops after finitely many steps and then the tape contains the original number incremented by one, the rest of tape contains, the head points to most significant bit. Example Initially s 0 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-18

9 Example Step 1 Initially s 0 Finally s 3 Initially, move head to least significant bit. I.e. as long as symbol at head is 0 or 1, move right, leave symbol as it is. If symbol is, move head left, leave symbol again as it is. Achieved by the following instructions: (s 0, 0,s 0, 0, R) (s 0, 1,s 0, 1, R) (s 0,,s 1,, L) At the end TM is in state s 1. s0 0/0, R 1/1, R B/B, L 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-20 Construction of the TM TM is ({0, 1, }, S, I,,s 0 ). States S and instructions I developed in the following. Step 1 Initially, move head to least significant bit. I.e. as long as symbol at head is 0 or 1, move right, leave symbol as it is. If symbol is, move head left, leave symbol again as it is. Achieved by the following instructions: (s 0, 0,s 0, 0, R) (s 0, 1,s 0, 1, R) (s 0,,s 1,, L) At the end TM is in state s 1. s0 0/0, R 1/1, R B/B, L Computability Theory, Lent Term 2008, Sec. 4 (a) 4-21 s 0

12 Step 1 Step 2 Initially, move head to least significant bit. I.e. as long as symbol at head is 0 or 1, move right, leave symbol as it is. If symbol is, move head left, leave symbol again as it is. Achieved by the following instructions: (s 0, 0,s 0, 0, R) (s 0, 1,s 0, 1, R) (s 0,,s 1,, L) At the end TM is in state s 1. s0 0/0, R 1/1, R B/B, L Increasing a binary number b done as follows: Case number consists of 1 only: I.e. b = (111 } {{ 111} ) 2. k times b + 1 = (1} 000 {{ 000} ) 2. k times Obtained by replacing all ones by zeros and then replacing the first blank symbol by s 0 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-22 Step 1 Initially, move head to least significant bit. I.e. as long as symbol at head is 0 or 1, move right, leave symbol as it is. If symbol is, move head left, leave symbol again as it is. Achieved by the following instructions: (s 0, 0,s 0, 0, R) (s 0, 1,s 0, 1, R) (s 0,,s 1,, L) At the end TM is in state s 1. s0 0/0, R 1/1, R B/B, L s 1 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-22 Step 2 Otherwise: Then the representation of the number contains at the end one 0 followed by ones only. Includes case where the least significant digit is 0. Example 1: b = ( ) 2, one 0 followed by 3 ones. Example 2: b = ( ) 2, least significant digit is 0. Let b = (b 0 b 1 b k 0} 111 {{ 111} ) 2. l times b + 1 obtained by replacing the final block of ones by 0 and the 0 by 1: b + 1 = (b 0 b 1 b k 1} 000 {{ 000} ) 2. l times

13 Step 2 General Situation Step 2 General Situation We have to replace, as long as we find ones, the ones by zeros, and move left, until we encounter a 0 or a, which is replaced by a 1. So we need a new state s 2, and the following instructions (s 1, 1,s 1, 0, L) (s 1, 0,s 2, 1, L) (s 1,,s 2, 1, L) 1/0, L 0/1, L B/1, L At the end the head will be one field to the left of the 1 written, and the state will be s 2. We have to replace, as long as we find ones, the ones by zeros, and move left, until we encounter a 0 or a, which is replaced by a 1. So we need a new state s 2, and the following instructions (s 1, 1,s 1, 0, L) (s 1, 0,s 2, 1, L) (s 1,,s 2, 1, L) 1/0, L 0/1, L B/1, L At the end the head will be one field to the left of the 1 written, and the state will be s Computability Theory, Lent Term 2008, Sec. 4 (a) 4-25 s 1 Step 2 General Situation We have to replace, as long as we find ones, the ones by zeros, and move left, until we encounter a 0 or a, which is replaced by a 1. So we need a new state s 2, and the following instructions (s 1, 1,s 1, 0, L) (s 1, 0,s 2, 1, L) (s 1,,s 2, 1, L) 1/0, L 0/1, L B/1, L At the end the head will be one field to the left of the 1 written, and the state will be s Step 2 General Situation We have to replace, as long as we find ones, the ones by zeros, and move left, until we encounter a 0 or a, which is replaced by a 1. So we need a new state s 2, and the following instructions (s 1, 1,s 1, 0, L) (s 1, 0,s 2, 1, L) (s 1,,s 2, 1, L) 1/0, L 0/1, L B/1, L At the end the head will be one field to the left of the 1 written, and the state will be s Computability Theory, Lent Term 2008, Sec. 4 (a) 4-25 s 1 s 1

14 Step 2 General Situation Step 3 We have to replace, as long as we find ones, the ones by zeros, and move left, until we encounter a 0 or a, which is replaced by a 1. So we need a new state s 2, and the following instructions (s 1, 1,s 1, 0, L) (s 1, 0,s 2, 1, L) (s 1,,s 2, 1, L) 0/1, L B/1, L 1/0, L At the end the head will be one field to the left of the 1 written, and the state will be s 2. Finally, we have to move the most significant bit, which is done as follows (s 2, 0,s 2, 0, L) (s 2, 1,s 2, 1, L) (s 2,,s 3,, R) 0/0, L 1/1, L The program terminates in state s 3. B/B, R s Computability Theory, Lent Term 2008, Sec. 4 (a) 4-25 Step 2 General Situation We have to replace, as long as we find ones, the ones by zeros, and move left, until we encounter a 0 or a, which is replaced by a 1. So we need a new state s 2, and the following instructions (s 1, 1,s 1, 0, L) 0/1, L (s 1, 0,s 2, 1, L) B/1, L (s 1,,s 2, 1, L) 1/0, L At the end the head will be one field to the left of the 1 written, and the state will be s Computability Theory, Lent Term 2008, Sec. 4 (a) 4-25 s 2 s 1 Step 3 Finally, we have to move the most significant bit, which is done as follows (s 2, 0,s 2, 0, L) (s 2, 1,s 2, 1, L) (s 2,,s 3,, R) 0/0, L 1/1, L The program terminates in state s 3. B/B, R s 2 s3

15 Step 3 Step 3 Finally, we have to move the most significant bit, which is done as follows Finally, we have to move the most significant bit, which is done as follows (s 2, 0,s 2, 0, L) (s 2, 1,s 2, 1, L) (s 2,,s 3,, R) 0/0, L 1/1, L B/B, R s3 (s 2, 0,s 2, 0, L) (s 2, 1,s 2, 1, L) (s 2,,s 3,, R) 0/0, L 1/1, L B/B, R s3 The program terminates in state s 3. The program terminates in state s s s 2 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-26 Step 3 Finally, we have to move the most significant bit, which is done as follows Step 3 Finally, we have to move the most significant bit, which is done as follows (s 2, 0,s 2, 0, L) (s 2, 1,s 2, 1, L) (s 2,,s 3,, R) 0/0, L 1/1, L B/B, R s3 (s 2, 0,s 2, 0, L) (s 2, 1,s 2, 1, L) (s 2,,s 3,, R) 0/0, L 1/1, L B/B, R s3 The program terminates in state s 3. The program terminates in state s s s 2 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-26

16 Step 3 Step 3 Finally, we have to move the most significant bit, which is done as follows Finally, we have to move the most significant bit, which is done as follows (s 2, 0,s 2, 0, L) (s 2, 1,s 2, 1, L) (s 2,,s 3,, R) 0/0, L 1/1, L B/B, R s3 (s 2, 0,s 2, 0, L) (s 2, 1,s 2, 1, L) (s 2,,s 3,, R) 0/0, L 1/1, L B/B, R s3 The program terminates in state s 3. The program terminates in state s s s 2 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-26 Step 3 Finally, we have to move the most significant bit, which is done as follows Step 3 Finally, we have to move the most significant bit, which is done as follows (s 2, 0,s 2, 0, L) (s 2, 1,s 2, 1, L) (s 2,,s 3,, R) 0/0, L 1/1, L B/B, R s3 (s 2, 0,s 2, 0, L) (s 2, 1,s 2, 1, L) (s 2,,s 3,, R) 0/0, L 1/1, L B/B, R s3 The program terminates in state s 3. The program terminates in state s s s 3 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-26

17 Complete TM Notation: bin The complete TM is as follows: ({0, 1, }, {s 0,s 1,s 2,s 3 }, {(s 0, 0,s 0, 0, R), (s 0, 1,s 0, 1, R), (s 0,,s 1,, L), (s 1, 1,s 1, 0, L), (s 1, 0,s 2, 1, L), (s 1,,s 2, 1, L), (s 2, 0,s 2, 0, L), (s 2, 1,s 2, 1, L), (s 2,,s 3,, R)},s 0, ) TMs usually operate on binary numbers. Therefore we define for a natural number bin(n) as the sequence of digits representing the unique standard binary representation of n. So bin(n) has no leading zeros, except for bin(0) := 0. Examples: bin(0) = 0, bin(1) = 1, bin(2) = 10, bin(3) = 11, bin(4) = 100, etc. 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-27 Complete TM Notation: bin s0 B/B, L 0/1, L B/1, L B/B, R s3 In order to read off the final result, we need to interpret an arbitrary finite sequence of 0, 1 as a binary number, even if it has leading zeros. We define bin(n) as one of the possible binary representations of n, allowing leading 0. 0/0, R 1/1, R 1/0, L 0/0, L 1/1, L So bin(1) can be 1, 01, 001, etc. In the special case 0 we treat the empty string as one of the possible representations, so bin(0) can be, 0, 00, 000, etc. 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-28

18 Notation: bin When carrying out intermediate calculations, it is easier to refer to bin(n) rather than bin(n) E.g. we can set a number on the tape easily to an element of bin(0) by overwriting it with 0s. In order to set it to bin(0) one would need to make sure that exactly one 0 remains. Then one usually has to shift left the content of the tape to the right of the original number. Definition 4.1 Output: Case 1: The TM stops. Only finitely many cells are non-blank. Let tape, starting from the head-position, contain b 0 b 1 b k 1 c where b i {0, 1} and c {0, 1}. (k might be 0). Let a = (b 0,...,b k 1 ) 2. (in case k = 0, a = 0). This means b 0 b k 1 is one of the choices for bin(a). Then T (k) (a 0,...,a k 1 ) a. 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-31 Definition 4.1 Let T = (Σ, S, I,,s 0 ) be a Turing machine with {0, 1} Σ. Define for every k N T (k) : N k N, where T (k) (a 0,...,a k 1 ) is computed as follows: Initialisation: We write on the tape bin(a 0 ) bin(a 1 ) bin(a k 1 ). E.g. if k = 3, a 0 = 0, a 1 = 3, a 2 = 2 then we write All other cells contain. The head is at the left most bit of the arguments written on the tape. The state is set to s 0. Iteration: Run the TM, until it stops. 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-32 Definition 4.1 Example: Let Σ = {0, 1,a,b, } where 0, 1,a,b, are different. If the tape starting with the head is as follows: or 01010a, output is (01010) 2 = (10) 10. If tape starting with the head is as follows: ab or a, or, the output is 0. Case 2: Otherwise. Then T (k) (a 0,...,a k 1 ).

19 Definition 4.2 (b) URM Comput. TM Comput. f : N k N is Turing-computable, in short TM-computable, if f = T (k) for some TM T, the alphabet of which contains {0, 1}. Theorem 4.3 If f : N n N is URM-computable, then it is as well Turing-computable by a TM with alphabet {0, 1, }. Example: That succ : N N and zero : N N are Turingcomputable was shown above. 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-35 Remark If the tape of the Turing machine initially contains only finitely many cells which are not blank, then at any step during the execution of the TM only finitely many cells are non blank. Follows since in each step at most one cell can be modified to become non-blank. So in finitely many steps only finitely many cells can be converted from blank to non-blank. Proof of Theorem 4.3 Notation The tape of a TM contains a 0,...,a l means: Starting from the head position, the cells of the tape contain a 0,...,a l. All other cells contain. 6 Computability Theory, Lent Term 2008, Sec. 4 (a) 4-36

20 Proof of Theorem 4.3 Example Assume f = U (n), U refers only to R 0,...,R l 1 and l > n, We define a TM T, which simulates U. Done as follows: That the registers R 0,...,R l 1 contain a 0,...,a l 1 is simulated by the tape containing bin(a 0 ) bin(a l 1 ). An instruction I j will be simulated by states s j,0,...,s j,i with instructions for those states. Assume the URM is about to execute instruction I 4 = pred(2) (i.e. PC = 4), with register contents R 0 R 1 R Then the URM will end with PC = 5 and register contents R 0 R 1 R Computability Theory, Lent Term 2008, Sec. 4 (b) 4-39 Conditions on the Simulation Assume the URM U is in a state s.t. R 0,...,R l 1 contain a 0,...,a l 1, the URM is about to execute I j. Assume after executing I j, the URM is in a state where R 0,...,R l 1 contain b 0,...,b l 1, the PC contains k. Then we want that, if configuration of the TM T is, s.t. the tape contains bin(a 0 ) bin(a 1 ) bin(a l 1 ), and the TM is in state s j,0, then the TM reaches a configuration s.t. the tape contains bin(b 0 ) bin(b 1 ) bin(b l 1 ), the TM is in state s k,0. Example Then we want that, if the simulating TM is in state s 4,0, with tape content bin(2) bin(1) bin(3) it should reach state s 5,0 with tape content bin(2) bin(1) bin(2) 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-40

21 Proof of Theorem 4.3 Proof of Theorem 4.3 Furthermore, we need initial states s init,0,...,s init,j and corresponding instructions, s.t. if the TM initially contains bin(b 0 ) bin(b 1 ) bin(b n 1 ) it will reach state s 0,0 with the tape containing bin(b 0 ) bin(b 1 ) bin(b n 1 ) 0 0 }{{ 0} l n times Then the corresponding TM will successively reach the following configurations: State s init,0 s 0,0 Tape contains bin(a0 ) bin(a 1 ) bin(a n 1 ) bin(a0 ) bin(a 1 ) bin(a n 1 ) bin(0) bin(0 = = bin(a 0,0 ) bin(a 0,1 ) bin(a 0,l 1 ) bin(a 1,0 ) bin(a 1,1 ) bin(a 1,l 1 ) bin(a 2,0 ) bin(a 2,1 ) bin(a 2,l 1 ) s k0,0 s k1,0 s k2,0 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-43 Proof of Theorem 4.3 Assume the run of the URM, starting with R i containing a 0,i = a i i = 0,...,n 1, and a 0,i = 0 for i = n,...,l 1 is as follows: Instruction R 0 R 1 R n 1 R n R l 1 I 0 a 0 a 1 a n = = = = = = I k0 a 0,0 a 0,1 a 0,n 1 a 0,n a 0,l 1 I k1 a 1,0 a 1,1 a 1,n 1 a 1,n a 1,l 1 I k2 a 2,0 a 2,1 a 2,n 1 a 2,n a 2,l 1 Example Consider the URM program U (which was discussed already in the section on URMs): I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) U (1) (a) 0. 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-44

22 Example Example I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Instruction R 0 R 1 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-47 Example I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Example I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Instruction R 0 R 1 I Computability Theory, Lent Term 2008, Sec. 4 (b) 4-47

23 Example Example I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Instruction R 0 R 1 I I I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Instruction R 0 R 1 I I I I Computability Theory, Lent Term 2008, Sec. 4 (b) 4-47 Example I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Instruction R 0 R 1 I I I Example I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Instruction R 0 R 1 I I I I I Computability Theory, Lent Term 2008, Sec. 4 (b) 4-47

24 Example Example I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Instruction R 0 R 1 I I I I I I I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Instruction R 0 R 1 I I I I I I I I Computability Theory, Lent Term 2008, Sec. 4 (b) 4-47 Example I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Instruction R 0 R 1 I I I I I I I Computability Theory, Lent Term 2008, Sec. 4 (b) 4-47 Example I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) A run of U (1) (2) is as follows: Instruction R 0 R 1 I I I I I I I I URM Stops

25 Corresponding TM Simulation Proof of Theorem 4.3 I 0 = ifzero(0, 3) I 1 = pred(0) I 2 = ifzero(1, 0) Instruction R 0 R 1 State of TM Content of Tape s init,0 bin(2) I s 0,0 bin(2) bin(0) I s 1,0 bin(2) bin(0) I s 2,0 bin(1) bin(0) I s 0,0 bin(1) bin(0) I s 1,0 bin(1) bin(0) I s 2,0 bin(0) bin(0) I s 0,0 bin(0) bin(0) I s 3,0 bin(0) bin(0) URM Stops TM Stops 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-48 Proof of Theorem 4.3 If we have defined this we have If U n (a 0,...,a n 1 ), U n (a 0,...,a n 1 ) c, then U eventually stops with R i containing some values b i, where b 0 = c. Then, the TM T starting with If U n (a 0,...,a n 1 ), the URM U will loop and the TM T will carry out the same steps as the URM and loop as well. Therefore T n (a 0,...,a n 1 ), again U n (a 0,...,a n 1 ) T n (a 0,...,a n 1 ). Proof of Theorem 4.3 It follows U (n) = T (n), and the proof is complete, if the simulation has been introduced. The following slides contain a detailed proof, which will not be presented in the lecture this year. Jump over remaining proof. bin(a 0 ) bin(a n 1 ) will eventually terminate in a configuration bin(b 0 ) bin(b k 1 ). for some k n. Therefore T n (a 0,...,a n 1 ) b 0 = c. 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-49

26 Proof of Theorem 4.3 Proof of Theorem 4.3 Informal description of the simulation of URM instructions. Initialisation. Initially, the tape contains bin(a 0 ) bin(a n 1 ). We need to obtain configuration: bin(a 0 ) bin(a n 1 ) bin(0) bin(0). } {{ } l n times Achieved by moving head to the end of the initial configuration inserting, starting from the next blank, l n-times 0, then moving back to the beginning. 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-52 Proof of Theorem 4.3 Simulation of URM instructions. Simulation of instruction I k = succ(j). Need to increase (j + 1)st binary number by 1 Initial configuration: bin(c 0 ) bin(c 1 ) bin(c j ) bin(c l ) s k,0 First move to the (j + 1)st blank to the right. Then we are at the end of the (j + 1)st binary number. bin(c 0 ) bin(c 1 ) bin(c j ) bin(c l ) bin(c 0 ) bin(c 1 ) bin(c j ) bin(c l Now perform the operation for increasing by 1 as above. At the end we obtain: bin(c 0 ) bin(c 1 ) bin(c j + 1) b It might be that we needed to write over the separating blank a 1, in which case we have: bin(c 0 ) bin(c 1 ) bin(cj 1 ) bin(cj + 1) Proof of Theorem 4.3 In the latter case, shift all symbols to the left once left, in order to obtain a separating between the lth and l 1st entry. We obtain bin(c 0 ) bin(c 1 ) bin(cj 1 ) bin(c j + 1) Otherwise, move the head to the left, until we reach the (j + 1)st blank to the left, and then move it once to the right. We obtain bin(c 0 ) bin(c 1 ) bin(c j + 1) b 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-53

27 Proof of Theorem 4.3 Proof of Theorem 4.3 Simulation of instruction I k = pred(j). Assume the configuration at the beginning is : bin(c 0 ) bin(c 1 ) bin(cj ) bin(c l ) We want to achieve bin(c 0 ) bin(c 1 ) bin(cj 1) bin(c l ) Done as follows: Initially: bin(c0 ) bin(cj ) bin( Finally: bin(c0 ) bin(c j 1) bin( We have achieved bin(c 0 ) bin(c 1 ) bin(cj 1) Move back to the beginning: bin(c 0 ) bin(c 1 ) bin(cj 1) 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-56 Proof of Theorem 4.3 Initially: bin(c0 ) bin(cj ) bin(c l ) Finally: bin(c0 ) bin(c j 1) bin(c l ) Move to end of the (j + 1)st number. Check, if the number consists only of zeros or not. If it consists only of zeros, pred(j) doesn t change anything. Otherwise, number is of the form b 0 b k 1 } 00 {{ 0}. l times Replace it by b 0 b k 0 } 11 {{ 1}. l times Done as for succ. 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-57 Proof of Theorem 4.3 Simulation of instruction I k = ifzero(j,k ). Move to j + 1st binary number on the tape. Check whether it contains only zeros. If yes, switch to state s k,0. Otherwise switch to state s k+1,0. This completes the simulation of the URM U.

28 Remark Extension to Arbitrary Alphabets We will later show that all TM-computable functions are URM-computable. This will be done by showing that all TM-computable functions are partial recursive, all partial recursive functions are URM-computable. This will be easier than showing directly that TM-computable functions are URM-computable. Therefore the set of TM-computable functions and the set of URM-computable functions coincide. Notion is modulo encoding of A into N equivalent to the notion of Turing-computability on N. However, when considering complexity bounds, this notation might be more appropriate. Avoids encoding/decoding into N. 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-60 Extension to Arbitrary Alphabets Let A be a finite alphabet s.t. A, and B := A. To a Turing machine T = (Σ, S, I,,s 0 ) with A Σ corresponds a partial function T (A,n) : B n B, where T (A,n) (a 0,...,a n 1 ) is computed as follows: Initially write a 0 a n 1 on the tape, otherwise. Start in state s 0 on the left most position of a 0. Iterate TM as before. In case of termination, the output of the function is c 0 c l 1, if the tape contains, starting with the head position c 0 c l 1 d with c i A, d A. Otherwise, the function value is undefined. Turing-Computable Predicates A predicate A is Turing-decidable, iff χ A is Turing-computable. Instead of simulating χ A means to write the output of χ A (a binary number 0 or 1) on the tape it is more convenient, to take TM with two additional special states s true and s false corresponding to truth and falsity of the predicate. 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-61

29 Turing-Computable Predicates History of Computability Theory Then a predicate is Turing decidable, if, when we write initially the inputs as before on the tape and start executing the TM, it always terminates in s true or s false, and it terminates in s true, iff the predicate holds for the inputs, and in s false, otherwise. The latter notion is equivalent to the first notion. Usually the latter one is taken as basis for complexity considerations. Alan Mathison Turing ( ) Introduced the Turing machine. Proved the undecidability of the Turing-Halting problem. 6 Computability Theory, Lent Term 2008, Sec. 4 (b) 4-64 (c) Undecid. of the Halting Problem Undecidability of the Halting Problem first proved 1936 by Alan Turing. In this Section, we will identify computable with Turing-computable. This will later be justified by the Church-Turing thesis. CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c) Definition 4.4 (a) A problem is an n-ary predicate M( x) of natural numbers, i.e. a property of n-tuples of natural numbers. (b) A problem (or predicate) M is (Turing-) decidable, if the characteristic function χ M of M is (Turing-)computable. 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-65 CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c)

30 Characteristic function Encoding of Turing Machines Reminder: χ M ( x) := { 1 if M( x) holds, 0 otherwise If we treat true as 1 and false as 0, then the characteristic function is nothing but the Boolean valued function which decides whether M( x) holds or not: χ M ( x) = { true if M( x) holds, false otherwise A Turing Machine is a quintuple (Σ, S, I,,s 0 ). We can assume that, each symbol of the alphabet, and each state can be represented by a string of letters and numbers. Then this quintuple can be written as a string of ASCII-symbols. Turing machines can be represented as elements of A, where A = set of ASCII-symbols. Turing machines can be encoded as natural numbers. Of course more efficient encoding exist. 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-68 Example of Decidable Problems The binary predicate Multiple(x,y) : x is a multiple of y is a predicate and therefore a problem. χ Multiple (x,y) decides, whether Multiple(x,y) holds (then it returns 1 for yes), or not: χ Multiple (x,y) = { 1 if x is a multiple of y, 0 if x is not a multiple of y. CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c) Encoding of Turing Machines Let for a Turing machine T, encode(t) N be its code. It is intuitively decidable, whether a string of ASCII symbols is a Turing machine. One can show that this can be decided by a Turing machine. It is intuitively decidable, whether n = encode(t) for a Turing machine T. χ Multiple is intuitively computable, therefore Multiple is decidable. 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-69 CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c)

31 {e} k (n) Stephen Cole Kleene Assume e N. We define a partial function {e} k : N k N, by m if e = encode(t) for some Turing machine T {e} k ( x) and T (k) ( x) m, otherwise. So if e = encode(t), {e} k = T (k). Roughly speaking, {e} k is the function computed by the eth Turing machine. So for every computable (more precisely Turing-computable) function f : N k N there exists an e s.t. f = {e} k. Stephen Cole Kleene ( ) Probably the most influential computability theorist up to now. Introduced the partial recursive functions. 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-72 {e} k The notation {e} k is due to Stephen Kleene. {} are called Kleene-Brackets. We write {e} for {e} 1. CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c) The Halting Problem Definition 4.5 Problem is the following binary predicate: The Halting Halt(e, n) : {e}(n) We will show that Halt is undecdiable. 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-73 CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c)

32 Example Question Let e = encode(t), where T is the Turing machine T I 0 = ifzero(0, 0) If input is > 0, the program terminates immediately, and R 0 remains unchanged, so What is WeakHalt(e,n), where e is a code for the Turing machine I 0 = ifzero(0, 0)? for k > 0. {e}(k) T (1) (k) k If input is = 0, the program loops for ever. Therefore {e}(0) T (1) (0). Therefore Halt(e,k) holds for k > 0 and does not hold for k = 0. 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-76 Remark Below we will see: Halt is undecideable. However, the following function WeakHalt is computable: { 1 if {e}(n) WeakHalt(e, n) : otherwise Computed as follows: First check whether e = encode(t) for some Turing machine T. If not, enter an infinite loop. Otherwise, simulate T with input n. If simulation stops, output 1, otherwise the program loops for ever. CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c) Theorem 4.6 Theorem 4.6 The halting problem is undecidable. Proof: Assume the Halting problem is decidable i.e. assume that we can decide using a Turing machine whether {e}(n) holds. We will define below a computable function f : N N, s.t. f {e}. Therefore f cannot be computed by the Turing machine with code e for any e, i.e. f is noncomputable. Therefore we obtain a contradiction. 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-77 CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c)

33 Proof of Theorem 4.6 Proof of Theorem 4.6 We argue similarly as in the proof of N P(N) We define f(e) in such a way that f = {e} is violated by having f(e) {e}(e). If {e}(e), then we let f(e). If {e}(e), we let f(e), e.g. by defining f(e) 0 (any other defined result would be appropriate as well). So we define f(e) {, if {e}(e) 0, if {e}(e) The complete proof on one slide is as follows: Assume Halt were decidable. Define f(e) {, if {e}(e) 0, if {e}(e) By Halt decidable, we obtain f is computable, so f = {e} for some e. But then f(e) Def of f {e}(e) f={e} f(e) 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-80 CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c) Proof of Theorem 4.6 f(e) { We obtain f(e) {e}(e)., if {e}(e) 0, if {e}(e) Since Halt is decidable, f is computable (Exercise: show that f is computable by a Turing machine, assuming a Turing machine for Halt). Therefore f = {e} for some e. But then by ( ) f(e) ( ) {e}(e) f={e} f(e) ( ) Remark The above proof can easily be adapted to any reasonable programming language, in which one can define all intuitively computable functions. Such programming languages are called Turing-complete languages. Babbage s machine was, if one removes the restriction to finite memory, Turing-complete, since it had a conditional jump. For standard Turing complete languages, the unsolvability of the Turing-halting problem means: it is not possible to write a program, which checks, whether a program on given input terminates. a contradiction. 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-81 CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c)

34 Halting Problem with no Inputs Halting Problem with no Inputs Theorem 4.7: It is undecidable, whether a Turing machine started with a blank tape terminates. Proof: Let Halt (e) : e is a code for a Turing machinet and T started with a blank tape terminates V, run with blank tape, terminates iff {e}(n). Let encode(v) = e. Then Halt (e ) Halt(e,n) Therefore using the decidability of Halt we can decide Halt(e, n). So we have decided Halt, a contradiction. Assume Halt were decidable. 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-84 CS 226 Computability Theory, Lent Term 2008, Sec. 4 (c) Halting Problem with no Inputs Then we can decide Halt(e,n) as follows: Assume inputs e, n. If e is not a code for a Turing machine, we return 0. Otherwise, let encode(t) = e. Define a Turing machine V as follows: V first writes bin(n) on the tape and moves head to the left most bit of bin(n). Then it executes the Turing machine T. We have V, run with blank tape, terminates iff T run with tape containing bin(n) terminates iff T (1) (n) iff {e}(n). 6 Computability Theory, Lent Term 2008, Sec. 4 (c) 4-85

### Introduction to Turing Machines. Reading: Chapters 8 & 9

Introduction to Turing Machines Reading: Chapters 8 & 9 1 Turing Machines (TM) Generalize the class of CFLs: Recursively Enumerable Languages Recursive Languages Context-Free Languages Regular Languages

### The Turing machine model of computation

The Turing machine model of computation For most of the remainder of the course we will study the Turing machine model of computation, named after Alan Turing (1912 1954) who proposed the model in 1936.

### Lecture Notes: The Halting Problem; Reductions

Lecture Notes: The Halting Problem; Reductions COMS W3261 Columbia University 20 Mar 2012 1 Review Key point. Turing machines can be encoded as strings, and other Turing machines can read those strings

### ECS 120 Lesson 15 Turing Machines, Pt. 1

ECS 120 Lesson 15 Turing Machines, Pt. 1 Oliver Kreylos Wednesday, May 2nd, 2001 Before we can start investigating the really interesting problems in theoretical computer science, we have to introduce

### Further discussion of Turing machines

Further discussion of Turing machines In this lecture we will discuss various aspects of decidable and Turing-recognizable languages that were not mentioned in previous lectures. In particular, we will

### Turing machines COMS Ashley Montanaro 21 March Department of Computer Science, University of Bristol Bristol, UK

COMS11700 Turing machines Department of Computer Science, University of Bristol Bristol, UK 21 March 2014 COMS11700: Turing machines Slide 1/15 Introduction We have seen two models of computation: finite

### Solution to CS375 Homework Assignment 11 (40 points) Due date: 4/26/2017

Solution to CS375 Homework Assignment 11 (40 points) Due date: 4/26/2017 1. Find a Greibach normal form for the following given grammar. (10 points) S bab A BAa a B bb Ʌ Solution: (1) Since S does not

### FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY Chomsky Normal Form and TURING MACHINES TUESDAY Feb 4 CHOMSKY NORMAL FORM A context-free grammar is in Chomsky normal form if every rule is of the form:

### Chomsky Normal Form and TURING MACHINES. TUESDAY Feb 4

Chomsky Normal Form and TURING MACHINES TUESDAY Feb 4 CHOMSKY NORMAL FORM A context-free grammar is in Chomsky normal form if every rule is of the form: A BC A a S ε B and C aren t start variables a is

### Theory of Computation

Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation / Lecture 3: Overview Decidability of Logical Theories Presburger arithmetic Decidability of Presburger Arithmetic Dr.

### The tape of M. Figure 3: Simulation of a Turing machine with doubly infinite tape

UG3 Computability and Intractability (2009-2010): Note 4 4. Bells and whistles. In defining a formal model of computation we inevitably make a number of essentially arbitrary design decisions. These decisions

### HIS LEGACY. 100 Years Turing celebration. Gordana Dodig Crnkovic, IDT Open Seminar. Computer Science and Network Department Mälardalen University

IDT Open Seminar AAN TUING AND HIS EGACY 00 Years Turing celebration http://www.mrtc.mdh.se/~gdc/work/turingcentenary.pdf http://www.mrtc.mdh.se/ mdh se/~gdc/work/turingmachine.pdf Gordana Dodig Crnkovic,

### Chapter 6: Turing Machines

Chapter 6: Turing Machines 6.1 The Turing Machine Definition A deterministic Turing machine (DTM) M is specified by a sextuple (Q, Σ, Γ, δ, s, f), where Q is a finite set of states; Σ is an alphabet of

### Turing Machines Part II

Turing Machines Part II Hello Hello Condensed Slide Slide Readers! Readers! This This lecture lecture is is almost almost entirely entirely animations that that show show how how each each Turing Turing

### Busch Complexity Lectures: Turing s Thesis. Costas Busch - LSU 1

Busch Complexity Lectures: Turing s Thesis Costas Busch - LSU 1 Turing s thesis (1930): Any computation carried out by mechanical means can be performed by a Turing Machine Costas Busch - LSU 2 Algorithm:

### Theory of Computation

Theory of Computation Lecture #10 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 43 Lecture 10: Overview Linear Bounded Automata Acceptance Problem for LBAs

### More About Turing Machines. Programming Tricks Restrictions Extensions Closure Properties

More About Turing Machines Programming Tricks Restrictions Extensions Closure Properties 1 Overview At first, the TM doesn t look very powerful. Can it really do anything a computer can? We ll discuss

### ,

Kolmogorov Complexity Carleton College, CS 254, Fall 2013, Prof. Joshua R. Davis based on Sipser, Introduction to the Theory of Computation 1. Introduction Kolmogorov complexity is a theory of lossless

### acs-07: Decidability Decidability Andreas Karwath und Malte Helmert Informatik Theorie II (A) WS2009/10

Decidability Andreas Karwath und Malte Helmert 1 Overview An investigation into the solvable/decidable Decidable languages The halting problem (undecidable) 2 Decidable problems? Acceptance problem : decide

### Lecture 23 : Nondeterministic Finite Automata DRAFT Connection between Regular Expressions and Finite Automata

CS/Math 24: Introduction to Discrete Mathematics 4/2/2 Lecture 23 : Nondeterministic Finite Automata Instructor: Dieter van Melkebeek Scribe: Dalibor Zelený DRAFT Last time we designed finite state automata

### Lecture 14: IP = PSPACE

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 14: IP = PSPACE David Mix Barrington and Alexis Maciel August 3, 2000 1. Overview We

### Recovery Based on Kolmogorov Complexity in Underdetermined Systems of Linear Equations

Recovery Based on Kolmogorov Complexity in Underdetermined Systems of Linear Equations David Donoho Department of Statistics Stanford University Email: donoho@stanfordedu Hossein Kakavand, James Mammen

### Space Complexity. The space complexity of a program is how much memory it uses.

Space Complexity The space complexity of a program is how much memory it uses. Measuring Space When we compute the space used by a TM, we do not count the input (think of input as readonly). We say that

### cse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska

cse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska LECTURE 14 SMALL REVIEW FOR FINAL SOME Y/N QUESTIONS Q1 Given Σ =, there is L over Σ Yes: = {e} and L = {e} Σ Q2 There are uncountably

### State Space Search Problems

State Space Search Problems Lecture Heuristic Search Stefan Edelkamp 1 Overview Different state space formalisms, including labelled, implicit and explicit weighted graph representations Alternative formalisms:

### Advanced Topics in Theoretical Computer Science

Advanced Topics in Theoretical Computer Science Part 5: Complexity (Part II) 30.01.2014 Viorica Sofronie-Stokkermans Universität Koblenz-Landau e-mail: sofronie@uni-koblenz.de 1 Contents Recall: Turing

### Computer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms

Computer Science 385 Analysis of Algorithms Siena College Spring 2011 Topic Notes: Limitations of Algorithms We conclude with a discussion of the limitations of the power of algorithms. That is, what kinds

### Institute for Applied Information Processing and Communications (IAIK) Secure & Correct Systems. Decidability

Decidability and the Undecidability of Predicate Logic IAIK Graz University of Technology georg.hofferek@iaik.tugraz.at 1 Fork of ways Brainteaser: Labyrinth Guards One to salvation One to perdition Two

### 198:538 Lecture Given on February 23rd, 1998

198:538 Lecture Given on February 23rd, 1998 Lecturer: Professor Eric Allender Scribe: Sunny Daniels November 25, 1998 1 A Result From theorem 5 of the lecture on 16th February 1998, together with the

### 10. Finite Automata and Turing Machines

. Finite Automata and Turing Machines Frank Stephan March 2, 24 Introduction Alan Turing s th Birthday Alan Turing was a completely original thinker who shaped the modern world, but many people have never

### KOLMOGOROV COMPLEXITY AND ALGORITHMIC RANDOMNESS

KOLMOGOROV COMPLEXITY AND ALGORITHMIC RANDOMNESS HENRY STEINITZ Abstract. This paper aims to provide a minimal introduction to algorithmic randomness. In particular, we cover the equivalent 1-randomness

### Nondeterministic finite automata

Lecture 3 Nondeterministic finite automata This lecture is focused on the nondeterministic finite automata (NFA) model and its relationship to the DFA model. Nondeterminism is an important concept in the

### Machine Characterizations of the Classes of the W-Hierarchy

Machine Characterizations of the Classes of the W-Hierarchy Yijia Chen and Jörg Flum Abteilung für Mathematische Logik, Universität Freiburg, Eckerstr. 1, 79104 Freiburg, Germany. chen@zermelo.mathematik.uni-freiburg.de

### P systems based on tag operations

Computer Science Journal of Moldova, vol.20, no.3(60), 2012 P systems based on tag operations Yurii Rogozhin Sergey Verlan Abstract In this article we introduce P systems using Post s tag operation on

### Kolmogorov complexity and its applications

CS860, Winter, 2010 Kolmogorov complexity and its applications Ming Li School of Computer Science University of Waterloo http://www.cs.uwaterloo.ca/~mli/cs860.html We live in an information society. Information

### P, NP, NP-Complete, and NPhard

P, NP, NP-Complete, and NPhard Problems Zhenjiang Li 21/09/2011 Outline Algorithm time complicity P and NP problems NP-Complete and NP-Hard problems Algorithm time complicity Outline What is this course

### An example of a decidable language that is not a CFL Implementation-level description of a TM State diagram of TM

Turing Machines Review An example of a decidable language that is not a CFL Implementation-level description of a TM State diagram of TM Varieties of TMs Multi-Tape TMs Nondeterministic TMs String Enumerators

### The Complexity of Optimization Problems

The Complexity of Optimization Problems Summary Lecture 1 - Complexity of algorithms and problems - Complexity classes: P and NP - Reducibility - Karp reducibility - Turing reducibility Uniform and logarithmic

### De Morgan s a Laws. De Morgan s laws say that. (φ 1 φ 2 ) = φ 1 φ 2, (φ 1 φ 2 ) = φ 1 φ 2.

De Morgan s a Laws De Morgan s laws say that (φ 1 φ 2 ) = φ 1 φ 2, (φ 1 φ 2 ) = φ 1 φ 2. Here is a proof for the first law: φ 1 φ 2 (φ 1 φ 2 ) φ 1 φ 2 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0 0 a Augustus DeMorgan

### DRAFT. Algebraic computation models. Chapter 14

Chapter 14 Algebraic computation models Somewhat rough We think of numerical algorithms root-finding, gaussian elimination etc. as operating over R or C, even though the underlying representation of the

### Kolmogorov complexity and its applications

Spring, 2009 Kolmogorov complexity and its applications Paul Vitanyi Computer Science University of Amsterdam http://www.cwi.nl/~paulv/course-kc We live in an information society. Information science is

### Theory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death

Theory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death Robbie Hott www.cs.virginia.edu/~jh2jf Department of Computer Science University of Virginia

### Notes on Complexity Theory Last updated: October, Lecture 6

Notes on Complexity Theory Last updated: October, 2015 Lecture 6 Notes by Jonathan Katz, lightly edited by Dov Gordon 1 PSPACE and PSPACE-Completeness As in our previous study of N P, it is useful to identify

### Randomized Complexity Classes; RP

Randomized Complexity Classes; RP Let N be a polynomial-time precise NTM that runs in time p(n) and has 2 nondeterministic choices at each step. N is a polynomial Monte Carlo Turing machine for a language

### Theory of Computation 8 Deterministic Membership Testing

Theory of Computation 8 Deterministic Membership Testing Frank Stephan Department of Computer Science Department of Mathematics National University of Singapore fstephan@comp.nus.edu.sg Theory of Computation

### Min/Max-Poly Weighting Schemes and the NL vs UL Problem

Min/Max-Poly Weighting Schemes and the NL vs UL Problem Anant Dhayal Jayalal Sarma Saurabh Sawlani May 3, 2016 Abstract For a graph G(V, E) ( V = n) and a vertex s V, a weighting scheme (w : E N) is called

### satisfiability (sat) Satisfiability unsatisfiability (unsat or sat complement) and validity Any Expression φ Can Be Converted into CNFs and DNFs

Any Expression φ Can Be Converted into CNFs and DNFs φ = x j : This is trivially true. φ = φ 1 and a CNF is sought: Turn φ 1 into a DNF and apply de Morgan s laws to make a CNF for φ. φ = φ 1 and a DNF

### 6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch

6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch Today: More Complexity Theory Polynomial-time reducibility, NP-completeness, and the Satisfiability (SAT) problem Topics: Introduction

### CS21 Decidability and Tractability

CS21 Decidability and Tractability Lecture 3 January 9, 2017 January 9, 2017 CS21 Lecture 3 1 Outline NFA, FA equivalence Regular Expressions FA and Regular Expressions January 9, 2017 CS21 Lecture 3 2

### Verification of String Manipulating Programs Using Multi-Track Automata

Verification of String Manipulating Programs Using Multi-Track Automata Fang Yu University of California, Santa Barbara yuf@cs.ucsb.edu Tevfik Bultan University of California, Santa Barbara bultan@cs.ucsb.edu

### 1 Reductions and Expressiveness

15-451/651: Design & Analysis of Algorithms November 3, 2015 Lecture #17 last changed: October 30, 2015 In the past few lectures we have looked at increasingly more expressive problems solvable using efficient

### Logics with Counting. Ian Pratt-Hartmann School of Computer Science University of Manchester Manchester M13 9PL, UK

Logics with Counting Ian Pratt-Hartmann School of Computer Science University of Manchester Manchester M13 9PL, UK 2 Chapter 1 Introduction It is well-known that first-order logic is able to express facts

### 9. PSPACE 9. PSPACE. PSPACE complexity class quantified satisfiability planning problem PSPACE-complete

Geography game Geography. Alice names capital city c of country she is in. Bob names a capital city c' that starts with the letter on which c ends. Alice and Bob repeat this game until one player is unable

### Chapter 1 - Time and Space Complexity. deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE

Chapter 1 - Time and Space Complexity deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE 1 / 41 Deterministic Turing machines Definition 1.1 A (deterministic

### 9. PSPACE. PSPACE complexity class quantified satisfiability planning problem PSPACE-complete

9. PSPACE PSPACE complexity class quantified satisfiability planning problem PSPACE-complete Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley Copyright 2013 Kevin Wayne http://www.cs.princeton.edu/~wayne/kleinberg-tardos

### Review. Principles of Programming Languages. Equality. The Diamond Property. The Church-Rosser Theorem. Corollaries. CSE 230: Winter 2007

CSE 230: Winter 2007 Principles of Programming Languages Lecture 12: The λ-calculus Ranjit Jhala UC San Diego Review The lambda calculus is a calculus of functions: e := x λx. e e 1 e 2 Several evaluation

### CS2742 midterm test 2 study sheet. Boolean circuits: Predicate logic:

x NOT ~x x y AND x /\ y x y OR x \/ y Figure 1: Types of gates in a digital circuit. CS2742 midterm test 2 study sheet Boolean circuits: Boolean circuits is a generalization of Boolean formulas in which

### Computation in Networks of Passively Mobile Finite-State Sensors

Distributed Computing manuscript No. (will be inserted by the editor) Dana Angluin James Aspnes Zoë Diamadi Michael J. Fischer René Peralta Computation in Networks of Passively Mobile Finite-State Sensors

### The Polynomial Hierarchy

The Polynomial Hierarchy Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Motivation..synthesizing circuits is exceedingly difficulty. It is even

### Notes for Comp 497 (Comp 454) Week 10 4/5/05

Notes for Comp 497 (Comp 454) Week 10 4/5/05 Today look at the last two chapters in Part II. Cohen presents some results concerning context-free languages (CFL) and regular languages (RL) also some decidability

### Introduction to Kolmogorov Complexity

Introduction to Kolmogorov Complexity Marcus Hutter Canberra, ACT, 0200, Australia http://www.hutter1.net/ ANU Marcus Hutter - 2 - Introduction to Kolmogorov Complexity Abstract In this talk I will give

### Busch Complexity Lectures: Turing Machines. Prof. Busch - LSU 1

Busch Complexity ectures: Turing Machines Prof. Busch - SU 1 The anguage Hierarchy a n b n c n? ww? Context-Free anguages n b n a ww egular anguages a* a *b* Prof. Busch - SU 2 a n b anguages accepted

### Introduction to Divide and Conquer

Introduction to Divide and Conquer Sorting with O(n log n) comparisons and integer multiplication faster than O(n 2 ) Periklis A. Papakonstantinou York University Consider a problem that admits a straightforward

### 3 The language of proof

3 The language of proof After working through this section, you should be able to: (a) understand what is asserted by various types of mathematical statements, in particular implications and equivalences;

### Symposium on Theoretical Aspects of Computer Science 2008 (Bordeaux), pp

Symposium on Theoretical Aspects of Computer Science 2008 (Bordeaux), pp. 373-384 www.stacs-conf.org COMPLEXITY OF SOLUTIONS OF EQUATIONS OVER SETS OF NATURAL NUMBERS ARTUR JEŻ 1 AND ALEXANDER OKHOTIN

### The Matrix Vector Product and the Matrix Product

The Matrix Vector Product and the Matrix Product As we have seen a matrix is just a rectangular array of scalars (real numbers) The size of a matrix indicates its number of rows and columns A matrix with

### 3 Propositional Logic

3 Propositional Logic 3.1 Syntax 3.2 Semantics 3.3 Equivalence and Normal Forms 3.4 Proof Procedures 3.5 Properties Propositional Logic (25th October 2007) 1 3.1 Syntax Definition 3.0 An alphabet Σ consists

### Section 20: Arrow Diagrams on the Integers

Section 0: Arrow Diagrams on the Integers Most of the material we have discussed so far concerns the idea and representations of functions. A function is a relationship between a set of inputs (the leave

### CS 6820 Fall 2014 Lectures, October 3-20, 2014

Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 3-20, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given

### On the Accepting Power of 2-Tape Büchi Automata

On the Accepting Power of 2-Tape Büchi Automata Equipe de Logique Mathématique Université Paris 7 STACS 2006 Acceptance of infinite words In the sixties, Acceptance of infinite words by finite automata

### CMPSCI 601: Tarski s Truth Definition Lecture 15. where

@ CMPSCI 601: Tarski s Truth Definition Lecture 15! "\$#&%(') *+,-!".#/%0'!12 43 5 6 7 8:9 4; 9 9 < = 9 = or 5 6?>A@B!9 2 D for all C @B 9 CFE where ) CGE @B-HI LJKK MKK )HG if H ; C if H @ 1 > > > Fitch

### Nondeterministic Streaming String Transducers

Nondeterministic Streaming String Transducers Rajeev Alur and Jyotirmoy V. Deshmukh Dept. of Computer and Information Science, University of Pennsylvania. {alur,djy}@cis.upenn.edu Abstract. We introduce

### /633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programming II Date: 10/12/17

601.433/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programming II Date: 10/12/17 12.1 Introduction Today we re going to do a couple more examples of dynamic programming. While

### On String Languages Generated by Numerical P Systems

ROMANIAN JOURNAL OF INFORMATION SCIENCE AND TECHNOLOGY Volume 8, Number 3, 205, 273 295 On String Languages Generated by Numerical P Systems Zhiqiang ZHANG, Tingfang WU, Linqiang PAN, Gheorghe PĂUN2 Key

### Notes on State Minimization

U.C. Berkeley CS172: Automata, Computability and Complexity Handout 1 Professor Luca Trevisan 2/3/2015 Notes on State Minimization These notes present a technique to prove a lower bound on the number of

### The Post Correspondence Problem; Applications to Undecidability Results

Chapter 8 The Post Correspondence Problem; Applications to Undecidability Results 8.1 The Post Correspondence Problem The Post correspondence problem (due to Emil Post) is another undecidable problem that

### Automata-Theoretic LTL Model-Checking

Automata-Theoretic LTL Model-Checking Arie Gurfinkel arie@cmu.edu SEI/CMU Automata-Theoretic LTL Model-Checking p.1 LTL - Linear Time Logic (Pn 77) Determines Patterns on Infinite Traces Atomic Propositions

### Formal Definition of a Finite Automaton. August 26, 2013

August 26, 2013 Why a formal definition? A formal definition is precise: - It resolves any uncertainties about what is allowed in a finite automaton such as the number of accept states and number of transitions

### Automata, Logic and Games: Theory and Application

Automata, Logic and Games: Theory and Application 1. Büchi Automata and S1S Luke Ong University of Oxford TACL Summer School University of Salerno, 14-19 June 2015 Luke Ong Büchi Automata & S1S 14-19 June

### Equivalence Checking of Sequential Circuits

Equivalence Checking of Sequential Circuits Sanjit Seshia EECS UC Berkeley With thanks to K. Keutzer, R. Rutenbar 1 Today s Lecture What we know: How to check two combinational circuits for equivalence

### Describe an algorithm for finding the smallest integer in a finite sequence of natural numbers.

Homework #4 Problem One (2.1.2) Determine which characteristics o f an algorithm the following procedures have and which they lack. a) procedure double(n: positive integer) while n > 0 n := 2n Since n

### Ackermann s Function

Ackermann s Function George Tourlakis February 18, 2008 1 What The Ackermann function was proposed, naturally, by Ackermann. The version here is a simplification offered by Robert Ritchie. What the function

### Infinite and Finite Model Theory Part II

Infinite and Finite Model Theory Part II Anuj Dawar Computer Laboratory University of Cambridge Lent 2002 3/2002 0 Finite Model Theory Finite Model Theory motivated by computational issues; relationship

### Definition: Alternating time and space Game Semantics: State of machine determines who

CMPSCI 601: Recall From Last Time Lecture Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins

### Analysis of Algorithms

October 1, 2015 Analysis of Algorithms CS 141, Fall 2015 1 Analysis of Algorithms: Issues Correctness/Optimality Running time ( time complexity ) Memory requirements ( space complexity ) Power I/O utilization

### Theory of Computation

Thomas Zeugmann Hokkaido University Laboratory for Algorithmics http://www-alg.ist.hokudai.ac.jp/ thomas/toc/ Lecture 1: Introducing Formal Languages Motivation I This course is about the study of a fascinating

### THEORY OF COMPUTATION

SHARIF UNIVERSITY OF TECHNOLOGY THEORY OF COMPUTATION FIRST WEEK February 13, 2016 1 COURSE DESCRIPTION Session 1 1 Course Description Welcome to "Theory of Computation" course. In this course we will

### Descriptive Complexity and Some Unusual Quantifiers

MASSACHUSETTS INSTITUTE OF TECHNOLOGY 18.405J / 6.841J, ADVANCED COMPLEXITY THEORY Descriptive Complexity and Some Unusual Quantifiers Chelsea Voss May 6, 2016 Abstract. It is possible to define and operators

### Lecture 20: PSPACE. November 15, 2016 CS 1010 Theory of Computation

Lecture 20: PSPACE November 15, 2016 CS 1010 Theory of Computation Recall that PSPACE = k=1 SPACE(nk ). We will see that a relationship between time and space complexity is given by: P NP PSPACE = NPSPACE

### Kvantifikator för en Dag

Philosophical Communications, Web Series, No. 35, pp. 155-162 Dept. of Philosophy, Göteborg University, Sweden ISSN 1652-0459 Kvantifikator för en Dag Essays dedicated to Dag Westerståhl on his sixtieth

### Module 1: Analyzing the Efficiency of Algorithms

Module 1: Analyzing the Efficiency of Algorithms Dr. Natarajan Meghanathan Associate Professor of Computer Science Jackson State University Jackson, MS 39217 E-mail: natarajan.meghanathan@jsums.edu Based

### THE RSA ENCRYPTION SCHEME

THE RSA ENCRYPTION SCHEME Contents 1. The RSA Encryption Scheme 2 1.1. Advantages over traditional coding methods 3 1.2. Proof of the decoding procedure 4 1.3. Security of the RSA Scheme 4 1.4. Finding

### Unitary Dynamics and Quantum Circuits

qitd323 Unitary Dynamics and Quantum Circuits Robert B. Griffiths Version of 20 January 2014 Contents 1 Unitary Dynamics 1 1.1 Time development operator T.................................... 1 1.2 Particular

### Conjunctive Normal Form and SAT

Notes on Satisfiability-Based Problem Solving Conjunctive Normal Form and SAT David Mitchell mitchell@cs.sfu.ca September 19, 2013 This is a preliminary draft of these notes. Please do not distribute without

### 10 Is every function computable?

10 Is every function computable? Learning Objectives: See a fundamental result in computer science and mathematics: the existence of uncomputable functions. See the canonical example for an uncomputable

### Lecture 3: Nondeterministic Finite Automata

Lecture 3: Nondeterministic Finite Automata September 5, 206 CS 00 Theory of Computation As a recap of last lecture, recall that a deterministic finite automaton (DFA) consists of (Q, Σ, δ, q 0, F ) where

### Cosets and Lagrange s theorem

Cosets and Lagrange s theorem These are notes on cosets and Lagrange s theorem some of which may already have been lecturer. There are some questions for you included in the text. You should write the