Theory of Computation Lecture Notes. Problems and Algorithms. Class Information

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Theory of Computation Lecture Notes. Problems and Algorithms. Class Information"

Transcription

1 Theory of Computation Lecture Notes Prof. Yuh-Dauh Lyuu Dept. Computer Science & Information Engineering and Department of Finance National Taiwan University Problems and Algorithms c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 1 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 3 Class Information Papadimitriou. Computational Complexity. 2nd printing. Addison-Wesley Check I have never done anything useful. Godfrey Harold Hardy ( ), A Mathematician s Apology (1940) for lecture notes. c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 2 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 4

2 Tractability and intractability What This Course Is All About Computability: What can be computed? What is computation anyway? There are well-defined problems that cannot be computed. In fact, most problems cannot be computed. Polynomial in terms of the input size n defines tractability. n, n log n, n 2, n 90. Time, space, circuit size, number of random bits, etc. It results in a fruitful and practical theory of complexity. Few practical, tractable problems require a large degree. Exponential-time or superpolynomial-time algorithms are usually impractical. n log n, 2 n, 2 n, n! 2πn (n/e) n. c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 5 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 7 What This Course Is All About (concluded) Complexity: What is a computable problem s inherent complexity? Some computable problems require at least exponential time and/or space; they are intractable. Some practical problems require superpolynomial resources unless certain conjectures are disproved. Other resource limits besides time and space? Program size, circuit size (growth), number of random bits, etc. Growth of Factorials n n! n n! , ,628, ,916, ,001, ,227,020, ,178,291, ,307,674,368, ,922,789,888,000 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 6 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 8

3 Turing Machines a A Turing machine (TM) is a quadruple M = (K, Σ, δ, s). K is a finite set of states. s K is the initial state. Turing Machines Σ is a finite set of symbols (disjoint from K). Σ includes (blank) and (first symbol). δ : K Σ (K {h, yes, no }) Σ {,, } is a transition function. (left), (right), and (stay) signify cursor movements. a Turing (1936). c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 9 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 11 What Is Computation? A TM Schema That can be coded in an algorithm. An algorithm is a detailed step-by-step method for solving a problem. The Euclidean algorithm for the greatest common divisor is an algorithm. Let s be the least upper bound of compact set A is not an algorithm. Let s be a smallest element of a finite-sized array can be solved by an algorithm δ c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 10 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 12

4 The Operations of TMs Physical Interpretations The tape: computer memory and registers. δ: program. K: instruction numbers. s: main() in C. Σ: alphabet much like the ASCII code. Initially the state is s. The string on the tape is initialized to a, followed by a finite-length string x (Σ { }). x is the input of the TM. The input must not contain s (why?)! The cursor is pointing to the first symbol, always a. The TM takes each step according to δ. The cursor may overwrite to make the string longer during the computation. c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 13 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 15 More about δ The program has the halting state (h), the accepting state ( yes ), and the rejecting state ( no ). Given current state q K and current symbol σ Σ, δ(q, σ) = (p, ρ, D). It specifies the next state p, the symbol ρ to be written over σ, and the direction D the cursor will move afterwards. We require δ(q, ) = (p,, ) so that the cursor never falls off the left end of the string. The Halting of a TM A TM M may halt in three cases. yes : M accepts its input x, and M(x) = yes. no : M rejects its input x, and M(x) = no. h: M(x) = y, where the string consists of a, followed by a finite string y, whose last symbol is not, followed by a string of s. y is the output of the computation. y may be empty denoted by ɛ. If M never halts on x, then write M(x) =. c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 14 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 16

5 Why TMs? Because of the simplicity of the TM, the model has the advantage when it comes to complexity issues. One can develop a complexity theory based on C++ or Java, say. But the added complexity does not yield additional fundamental insights. We will describe TMs in pseudocode. Configurations (concluded) A configuration is a triple (q, w, u): q K. w Σ is the string to the left of the cursor (inclusive). u Σ is the string to the right of the cursor. Note that (w, u) describes both the string and the cursor position. c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 17 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 19 The Concept of Configuration A configuration is a complete description of the current state of the computation. The specification of a configuration is sufficient for the computation to continue as if it had not been stopped. What does your PC save before it sleeps? Enough for it to resume work later w = u = c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 18 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 20

6 Fix a TM M. Yielding Configuration (q,w, u) yields configuration (q, w, u ) in one step, (q, w, u) M (q, w, u ), if a step of M from configuration (q, w, u) results in configuration (q, w, u ). (q,w, u) Mk (q, w, u ): Configuration (q,w, u) yields configuration (q, w, u ) in k N steps. (q,w, u) M (q, w, u ): Configuration (q,w, u) yields configuration (q, w, u ). Palindromes A string is a palindrome if it reads the same forwards and backwards (e.g., ). A TM program can be written to recognize palindromes: It matches the first character with the last character. It matches the second character with the next to last character, etc. (see next page). yes for palindromes and no for nonpalindromes. This program takes O(n 2 ) steps. Can we do better? c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 21 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 23 Example: How to Insert a Symbol We want to compute f(x) = ax. The TM moves the last symbol of x to the right by one position. It then moves the next to last symbol to the right, and so on. The TM finally writes a in the first position The total number of steps is O(n), where n is the length of x. c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 22 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 24

7 Decidability and Recursive Languages Let L (Σ { }) be a language, i.e., a set of strings of symbols with a finite length. For example, {0, 01, 10, 210, 1010,...}. Let M be a TM such that for any string x: If x L, then M(x) = yes. If x L, then M(x) = no. We say M decides L. If L is decided by some TM, then L is recursive. Palindromes over {0, 1} are recursive. Acceptability and Recursively Enumerable Languages (concluded) If L is accepted by some TM, then L is a recursively enumerable language. A recursively enumerable language can be generated by a TM, thus the name. That is, there is an algorithm such that for every x L, it will be printed out eventually. c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 25 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 27 Recursive and Recursively Enumerable Languages Acceptability and Recursively Enumerable Languages Let L (Σ { }) be a language. Let M be a TM such that for any string x: If x L, then M(x) = yes. If x L, then M(x) =. We say M accepts L. Proposition 1 If L is recursive, then it is recursively enumerable. We need to design a TM that accepts L. Let TM M decide L. We next modify M s program to obtain M that accepts L. M is identical to M except that when M is about to halt with a no state, M goes into an infinite loop. M accepts L. c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 26 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 28

8 Turing-Computable Functions Let f : (Σ { }) Σ. Optimization problems, root finding problems, etc. Let M be a TM with alphabet Σ. M computes f if for any string x (Σ { }), M(x) = f(x). We call f a recursive function a if such an M exists. a Gödel (1931). Extended Church s Thesis All reasonably succinct encodings of problems are polynomially related. Representations of a graph as an adjacency matrix and as a linked list are both succinct. The unary representation of numbers is not succinct. The binary representation of numbers is succinct vs All numbers for TMs will be binary from now on. c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 29 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 31 Church s Thesis or the Church-Turing Thesis What is computable is Turing-computable; TMs are algorithms (Kleene 1953). Many other computation models have been proposed. Recursive function (Gödel), λ calculus (Church), formal language (Post), assembly language-like RAM (Shepherdson & Sturgis), boolean circuits (Shannon), extensions of the Turing machine (more strings, two-dimensional strings, and so on), etc. All have been proved to be equivalent. No intuitively computable problems have been shown not to be Turing-computable (yet). Turing Machines with Multiple Strings A k-string Turing machine (TM) is a quadruple M = (K, Σ, δ, s). K, Σ, s are as before. δ : K Σ k (K {h, yes, no }) (Σ {,, }) k. All strings start with a. The first string contains the input. Decidability and acceptability are the same as before. When TMs compute functions, the output is on the last (kth) string. c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 30 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 32

9 A 2-String TM δ δ ababbaabbaabbaabbaba ababbaabbaabbaabbaba c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 33 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 35 palindrome Revisited A 2-string TM can decide palindrome in O(n) steps. It copies the input to the second string. The cursor of the first string is positioned at the first symbol of the input. The cursor of the second string is positioned at the last symbol of the input. The two cursors are then moved in opposite directions until the ends are reached. The machine accepts if and only if the symbols under the two cursors are identical at all steps. Configurations and Yielding The concept of configuration and yielding is the same as before except that a configuration is a (2k + 1)-triple w i u i is the ith string. (q, w 1, u 1, w 2, u 2,...,w k, u k ). The ith cursor is reading the last symbol of w i. Recall that is each w i s first symbol. The k-string TM s initial configuration is 2k {}}{ (s,, x,, ɛ,, ɛ,...,, ɛ). c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 34 c 2004 Prof. Yuh-Dauh Lyuu, National Taiwan University Page 36

Computability Theory. CS215, Lecture 6,

Computability Theory. CS215, Lecture 6, Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David

More information

satisfiability (sat) Satisfiability unsatisfiability (unsat or sat complement) and validity Any Expression φ Can Be Converted into CNFs and DNFs

satisfiability (sat) Satisfiability unsatisfiability (unsat or sat complement) and validity Any Expression φ Can Be Converted into CNFs and DNFs Any Expression φ Can Be Converted into CNFs and DNFs φ = x j : This is trivially true. φ = φ 1 and a CNF is sought: Turn φ 1 into a DNF and apply de Morgan s laws to make a CNF for φ. φ = φ 1 and a DNF

More information

Randomized Complexity Classes; RP

Randomized Complexity Classes; RP Randomized Complexity Classes; RP Let N be a polynomial-time precise NTM that runs in time p(n) and has 2 nondeterministic choices at each step. N is a polynomial Monte Carlo Turing machine for a language

More information

Turing s thesis: (1930) Any computation carried out by mechanical means can be performed by a Turing Machine

Turing s thesis: (1930) Any computation carried out by mechanical means can be performed by a Turing Machine Turing s thesis: (1930) Any computation carried out by mechanical means can be performed by a Turing Machine There is no known model of computation more powerful than Turing Machines Definition of Algorithm:

More information

Turing Machines Part III

Turing Machines Part III Turing Machines Part III Announcements Problem Set 6 due now. Problem Set 7 out, due Monday, March 4. Play around with Turing machines, their powers, and their limits. Some problems require Wednesday's

More information

(a) Definition of TMs. First Problem of URMs

(a) Definition of TMs. First Problem of URMs Sec. 4: Turing Machines First Problem of URMs (a) Definition of the Turing Machine. (b) URM computable functions are Turing computable. (c) Undecidability of the Turing Halting Problem That incrementing

More information

Computability and Complexity Theory

Computability and Complexity Theory Discrete Math for Bioinformatics WS 09/10:, by A Bockmayr/K Reinert, January 27, 2010, 18:39 9001 Computability and Complexity Theory Computability and complexity Computability theory What problems can

More information

Lecture 13: Foundations of Math and Kolmogorov Complexity

Lecture 13: Foundations of Math and Kolmogorov Complexity 6.045 Lecture 13: Foundations of Math and Kolmogorov Complexity 1 Self-Reference and the Recursion Theorem 2 Lemma: There is a computable function q : Σ* Σ* such that for every string w, q(w) is the description

More information

cse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska

cse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska cse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska LECTURE 13 CHAPTER 4 TURING MACHINES 1. The definition of Turing machine 2. Computing with Turing machines 3. Extensions of Turing

More information

7.1 The Origin of Computer Science

7.1 The Origin of Computer Science CS125 Lecture 7 Fall 2016 7.1 The Origin of Computer Science Alan Mathison Turing (1912 1954) turing.jpg 170!201 pixels On Computable Numbers, with an Application to the Entscheidungsproblem 1936 1936:

More information

Lecture 14: Recursive Languages

Lecture 14: Recursive Languages Lecture 14: Recursive Languages Instructor: Ketan Mulmuley Scriber: Yuan Li February 24, 2015 1 Recursive Languages Definition 1.1. A language L Σ is called recursively enumerable (r. e.) or computably

More information

De Morgan s a Laws. De Morgan s laws say that. (φ 1 φ 2 ) = φ 1 φ 2, (φ 1 φ 2 ) = φ 1 φ 2.

De Morgan s a Laws. De Morgan s laws say that. (φ 1 φ 2 ) = φ 1 φ 2, (φ 1 φ 2 ) = φ 1 φ 2. De Morgan s a Laws De Morgan s laws say that (φ 1 φ 2 ) = φ 1 φ 2, (φ 1 φ 2 ) = φ 1 φ 2. Here is a proof for the first law: φ 1 φ 2 (φ 1 φ 2 ) φ 1 φ 2 0 0 1 1 0 1 1 1 1 0 1 1 1 1 0 0 a Augustus DeMorgan

More information

The Turing Machine. Computability. The Church-Turing Thesis (1936) Theory Hall of Fame. Theory Hall of Fame. Undecidability

The Turing Machine. Computability. The Church-Turing Thesis (1936) Theory Hall of Fame. Theory Hall of Fame. Undecidability The Turing Machine Computability Motivating idea Build a theoretical a human computer Likened to a human with a paper and pencil that can solve problems in an algorithmic way The theoretical provides a

More information

Turing machines and computable functions

Turing machines and computable functions Turing machines and computable functions In this chapter we address the question: how can we define the class of discrete functions that are computable? Probably, most of you are familiar with a few programming

More information

Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar

Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar Turing Machine A Turing machine is an abstract representation of a computing device. It consists of a read/write

More information

Introduction to Turing Machines. Reading: Chapters 8 & 9

Introduction to Turing Machines. Reading: Chapters 8 & 9 Introduction to Turing Machines Reading: Chapters 8 & 9 1 Turing Machines (TM) Generalize the class of CFLs: Recursively Enumerable Languages Recursive Languages Context-Free Languages Regular Languages

More information

1 Computational problems

1 Computational problems 80240233: Computational Complexity Lecture 1 ITCS, Tsinghua Univesity, Fall 2007 9 October 2007 Instructor: Andrej Bogdanov Notes by: Andrej Bogdanov The aim of computational complexity theory is to study

More information

Chapter 2 Algorithms and Computation

Chapter 2 Algorithms and Computation Chapter 2 Algorithms and Computation In this chapter, we first discuss the principles of algorithm and computation in general framework, common both in classical and quantum computers, then we go to the

More information

Turing machines COMS Ashley Montanaro 21 March Department of Computer Science, University of Bristol Bristol, UK

Turing machines COMS Ashley Montanaro 21 March Department of Computer Science, University of Bristol Bristol, UK COMS11700 Turing machines Department of Computer Science, University of Bristol Bristol, UK 21 March 2014 COMS11700: Turing machines Slide 1/15 Introduction We have seen two models of computation: finite

More information

1 Acceptance, Rejection, and I/O for Turing Machines

1 Acceptance, Rejection, and I/O for Turing Machines 1 Acceptance, Rejection, and I/O for Turing Machines Definition 1.1 (Initial Configuration) If M = (K,Σ,δ,s,H) is a Turing machine and w (Σ {, }) then the initial configuration of M on input w is (s, w).

More information

MTAT Complexity Theory October 13th-14th, Lecture 6

MTAT Complexity Theory October 13th-14th, Lecture 6 MTAT.07.004 Complexity Theory October 13th-14th, 2011 Lecturer: Peeter Laud Lecture 6 Scribe(s): Riivo Talviste 1 Logarithmic memory Turing machines working in logarithmic space become interesting when

More information

About the relationship between formal logic and complexity classes

About the relationship between formal logic and complexity classes About the relationship between formal logic and complexity classes Working paper Comments welcome; my email: armandobcm@yahoo.com Armando B. Matos October 20, 2013 1 Introduction We analyze a particular

More information

Computer Science. Questions for discussion Part II. Computer Science COMPUTER SCIENCE. Section 4.2.

Computer Science. Questions for discussion Part II. Computer Science COMPUTER SCIENCE. Section 4.2. COMPUTER SCIENCE S E D G E W I C K / W A Y N E PA R T I I : A L G O R I T H M S, T H E O R Y, A N D M A C H I N E S Computer Science Computer Science An Interdisciplinary Approach Section 4.2 ROBERT SEDGEWICK

More information

CSCI3390-Assignment 2 Solutions

CSCI3390-Assignment 2 Solutions CSCI3390-Assignment 2 Solutions due February 3, 2016 1 TMs for Deciding Languages Write the specification of a Turing machine recognizing one of the following three languages. Do one of these problems.

More information

FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY 15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY THURSDAY APRIL 3 REVIEW for Midterm TUESDAY April 8 Definition: A Turing Machine is a 7-tuple T = (Q, Σ, Γ, δ, q, q accept, q reject ), where: Q is a

More information

Turing Machines Part Two

Turing Machines Part Two Turing Machines Part Two Recap from Last Time Our First Turing Machine q acc a start q 0 q 1 a This This is is the the Turing Turing machine s machine s finiteisttiteiconntont. finiteisttiteiconntont.

More information

CpSc 421 Homework 9 Solution

CpSc 421 Homework 9 Solution CpSc 421 Homework 9 Solution Attempt any three of the six problems below. The homework is graded on a scale of 100 points, even though you can attempt fewer or more points than that. Your recorded grade

More information

CS154, Lecture 10: Rice s Theorem, Oracle Machines

CS154, Lecture 10: Rice s Theorem, Oracle Machines CS154, Lecture 10: Rice s Theorem, Oracle Machines Moral: Analyzing Programs is Really, Really Hard But can we more easily tell when some program analysis problem is undecidable? Problem 1 Undecidable

More information

CSCE 551: Chin-Tser Huang. University of South Carolina

CSCE 551: Chin-Tser Huang. University of South Carolina CSCE 551: Theory of Computation Chin-Tser Huang huangct@cse.sc.edu University of South Carolina Church-Turing Thesis The definition of the algorithm came in the 1936 papers of Alonzo Church h and Alan

More information

Variants of Turing Machine (intro)

Variants of Turing Machine (intro) CHAPTER 3 The Church-Turing Thesis Contents Turing Machines definitions, examples, Turing-recognizable and Turing-decidable languages Variants of Turing Machine Multi-tape Turing machines, non-deterministic

More information

1 Computational Problems

1 Computational Problems Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

More information

Lecture Notes: The Halting Problem; Reductions

Lecture Notes: The Halting Problem; Reductions Lecture Notes: The Halting Problem; Reductions COMS W3261 Columbia University 20 Mar 2012 1 Review Key point. Turing machines can be encoded as strings, and other Turing machines can read those strings

More information

Lecture 12: Mapping Reductions

Lecture 12: Mapping Reductions Lecture 12: Mapping Reductions October 18, 2016 CS 1010 Theory of Computation Topics Covered 1. The Language EQ T M 2. Mapping Reducibility 3. The Post Correspondence Problem 1 The Language EQ T M The

More information

The Turing machine model of computation

The Turing machine model of computation The Turing machine model of computation For most of the remainder of the course we will study the Turing machine model of computation, named after Alan Turing (1912 1954) who proposed the model in 1936.

More information

1 Two-Way Deterministic Finite Automata

1 Two-Way Deterministic Finite Automata 1 Two-Way Deterministic Finite Automata 1.1 Introduction Hing Leung 1 In 1943, McCulloch and Pitts [4] published a pioneering work on a model for studying the behavior of the nervous systems. Following

More information

Turing machines Finite automaton no storage Pushdown automaton storage is a stack What if we give the automaton a more flexible storage?

Turing machines Finite automaton no storage Pushdown automaton storage is a stack What if we give the automaton a more flexible storage? Turing machines Finite automaton no storage Pushdown automaton storage is a stack What if we give the automaton a more flexible storage? What is the most powerful of automata? In this lecture we will introduce

More information

Finish K-Complexity, Start Time Complexity

Finish K-Complexity, Start Time Complexity 6.045 Finish K-Complexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts

More information

Decision Problems with TM s. Lecture 31: Halting Problem. Universe of discourse. Semi-decidable. Look at following sets: CSCI 81 Spring, 2012

Decision Problems with TM s. Lecture 31: Halting Problem. Universe of discourse. Semi-decidable. Look at following sets: CSCI 81 Spring, 2012 Decision Problems with TM s Look at following sets: Lecture 31: Halting Problem CSCI 81 Spring, 2012 Kim Bruce A TM = { M,w M is a TM and w L(M)} H TM = { M,w M is a TM which halts on input w} TOTAL TM

More information

Busch Complexity Lectures: Turing s Thesis. Costas Busch - LSU 1

Busch Complexity Lectures: Turing s Thesis. Costas Busch - LSU 1 Busch Complexity Lectures: Turing s Thesis Costas Busch - LSU 1 Turing s thesis (1930): Any computation carried out by mechanical means can be performed by a Turing Machine Costas Busch - LSU 2 Algorithm:

More information

16.1 Countability. CS125 Lecture 16 Fall 2014

16.1 Countability. CS125 Lecture 16 Fall 2014 CS125 Lecture 16 Fall 2014 16.1 Countability Proving the non-existence of algorithms for computational problems can be very difficult. Indeed, we do not know how to prove P NP. So a natural question is

More information

Boolean circuits. Lecture Definitions

Boolean circuits. Lecture Definitions Lecture 20 Boolean circuits In this lecture we will discuss the Boolean circuit model of computation and its connection to the Turing machine model. Although the Boolean circuit model is fundamentally

More information

Computability Theory

Computability Theory Computability Theory Cristian S. Calude and Nicholas J. Hay May June 2009 Computability Theory 1 / 155 1 Register machines 2 Church-Turing thesis 3 Decidability 4 Reducibility 5 A definition of information

More information

Introduction to Computer Science

Introduction to Computer Science Introduction to Computer Science CSCI 109 China Tianhe-2 Andrew Goodney Fall 2017 Lecture 11: Abstract Machines and Theory Nov. 13th, 2017 Schedule 1 Abstract machines and theory uwhat is theoretical computer

More information

CS 361 Meeting 26 11/10/17

CS 361 Meeting 26 11/10/17 CS 361 Meeting 26 11/10/17 1. Homework 8 due Announcements A Recognizable, but Undecidable Language 1. Last class, I presented a brief, somewhat inscrutable proof that the language A BT M = { M w M is

More information

Decidability and Undecidability

Decidability and Undecidability Decidability and Undecidability Major Ideas from Last Time Every TM can be converted into a string representation of itself. The encoding of M is denoted M. The universal Turing machine U TM accepts an

More information

Automata Theory CS S-12 Turing Machine Modifications

Automata Theory CS S-12 Turing Machine Modifications Automata Theory CS411-2015S-12 Turing Machine Modifications David Galles Department of Computer Science University of San Francisco 12-0: Extending Turing Machines When we added a stack to NFA to get a

More information

CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression

CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression Rosencrantz & Guildenstern Are Dead (Tom Stoppard) Rigged Lottery? And the winning numbers are: 1, 2, 3, 4, 5, 6 But is

More information

ECS 120 Lesson 15 Turing Machines, Pt. 1

ECS 120 Lesson 15 Turing Machines, Pt. 1 ECS 120 Lesson 15 Turing Machines, Pt. 1 Oliver Kreylos Wednesday, May 2nd, 2001 Before we can start investigating the really interesting problems in theoretical computer science, we have to introduce

More information

Notes for Lecture 3... x 4

Notes for Lecture 3... x 4 Stanford University CS254: Computational Complexity Notes 3 Luca Trevisan January 18, 2012 Notes for Lecture 3 In this lecture we introduce the computational model of boolean circuits and prove that polynomial

More information

Turing Machines (TM) The Turing machine is the ultimate model of computation.

Turing Machines (TM) The Turing machine is the ultimate model of computation. TURING MACHINES Turing Machines (TM) The Turing machine is the ultimate model of computation. Alan Turing (92 954), British mathematician/engineer and one of the most influential scientists of the last

More information

Turing machines and linear bounded automata

Turing machines and linear bounded automata and linear bounded automata Informatics 2A: Lecture 29 John Longley School of Informatics University of Edinburgh jrl@inf.ed.ac.uk 25 November, 2011 1 / 13 1 The Chomsky hierarchy: summary 2 3 4 2 / 13

More information

Computational Models - Lecture 6

Computational Models - Lecture 6 Computational Models - Lecture 6 Turing Machines Alternative Models of Computers Multitape TMs, RAMs, Non Deterministic TMs The Church-Turing Thesis The language classes R = RE core David Hilbert s Tenth

More information

Mm7 Intro to distributed computing (jmp) Mm8 Backtracking, 2-player games, genetic algorithms (hps) Mm9 Complex Problems in Network Planning (JMP)

Mm7 Intro to distributed computing (jmp) Mm8 Backtracking, 2-player games, genetic algorithms (hps) Mm9 Complex Problems in Network Planning (JMP) Algorithms and Architectures II H-P Schwefel, Jens M. Pedersen Mm6 Advanced Graph Algorithms (hps) Mm7 Intro to distributed computing (jmp) Mm8 Backtracking, 2-player games, genetic algorithms (hps) Mm9

More information

CS 21 Decidability and Tractability Winter Solution Set 3

CS 21 Decidability and Tractability Winter Solution Set 3 CS 21 Decidability and Tractability Winter 2018 Posted: January 31 Solution Set 3 If you have not yet turned in the Problem Set, you should not consult these solutions. 1. (a) A 2-NPDA is a 7-tuple (Q,,

More information

Turing Machine Variants

Turing Machine Variants CS311 Computational Structures Turing Machine Variants Lecture 12 Andrew Black Andrew Tolmach 1 The Church-Turing Thesis The problems that can be decided by an algorithm are exactly those that can be decided

More information

FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY 15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY Chomsky Normal Form and TURING MACHINES TUESDAY Feb 4 CHOMSKY NORMAL FORM A context-free grammar is in Chomsky normal form if every rule is of the form:

More information

Sample Project: Simulation of Turing Machines by Machines with only Two Tape Symbols

Sample Project: Simulation of Turing Machines by Machines with only Two Tape Symbols Sample Project: Simulation of Turing Machines by Machines with only Two Tape Symbols The purpose of this document is to illustrate what a completed project should look like. I have chosen a problem that

More information

Models. Models of Computation, Turing Machines, and the Limits of Turing Computation. Effective Calculability. Motivation for Models of Computation

Models. Models of Computation, Turing Machines, and the Limits of Turing Computation. Effective Calculability. Motivation for Models of Computation Turing Computation /0/ Models of Computation, Turing Machines, and the Limits of Turing Computation Bruce MacLennan Models A model is a tool intended to address a class of questions about some domain of

More information

Lecture 2. 1 More N P-Compete Languages. Notes on Complexity Theory: Fall 2005 Last updated: September, Jonathan Katz

Lecture 2. 1 More N P-Compete Languages. Notes on Complexity Theory: Fall 2005 Last updated: September, Jonathan Katz Notes on Complexity Theory: Fall 2005 Last updated: September, 2005 Jonathan Katz Lecture 2 1 More N P-Compete Languages It will be nice to find more natural N P-complete languages. To that end, we ine

More information

,

, Kolmogorov Complexity Carleton College, CS 254, Fall 2013, Prof. Joshua R. Davis based on Sipser, Introduction to the Theory of Computation 1. Introduction Kolmogorov complexity is a theory of lossless

More information

Chapter 1 - Time and Space Complexity. deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE

Chapter 1 - Time and Space Complexity. deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE Chapter 1 - Time and Space Complexity deterministic and non-deterministic Turing machine time and space complexity classes P, NP, PSPACE, NPSPACE 1 / 41 Deterministic Turing machines Definition 1.1 A (deterministic

More information

Lecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009

Lecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009 CS 373: Theory of Computation Sariel Har-Peled and Madhusudan Parthasarathy Lecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009 This lecture covers Rice s theorem, as well as

More information

Theory of Computation

Theory of Computation Theory of Computation Lecture #2 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 1 Lecture 2: Overview Recall some basic definitions from Automata Theory.

More information

Turing Machines. The Language Hierarchy. Context-Free Languages. Regular Languages. Courtesy Costas Busch - RPI 1

Turing Machines. The Language Hierarchy. Context-Free Languages. Regular Languages. Courtesy Costas Busch - RPI 1 Turing Machines a n b n c The anguage Hierarchy n? ww? Context-Free anguages a n b n egular anguages a * a *b* ww Courtesy Costas Busch - PI a n b n c n Turing Machines anguages accepted by Turing Machines

More information

Further discussion of Turing machines

Further discussion of Turing machines Further discussion of Turing machines In this lecture we will discuss various aspects of decidable and Turing-recognizable languages that were not mentioned in previous lectures. In particular, we will

More information

Theory of Computation

Theory of Computation Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation / Lecture 3: Overview Decidability of Logical Theories Presburger arithmetic Decidability of Presburger Arithmetic Dr.

More information

Automata Theory CS S-FR2 Final Review

Automata Theory CS S-FR2 Final Review Automata Theory CS411-2015S-FR2 Final Review David Galles Department of Computer Science University of San Francisco FR2-0: Halting Problem e(m) e(w) Halting Machine takes as input an encoding of a Turing

More information

Lecture 16: Time Complexity and P vs NP

Lecture 16: Time Complexity and P vs NP 6.045 Lecture 16: Time Complexity and P vs NP 1 Time-Bounded Complexity Classes Definition: TIME(t(n)) = { L there is a Turing machine M with time complexity O(t(n)) so that L = L(M) } = { L L is a language

More information

Verifying whether One-Tape Turing Machines Run in Linear Time

Verifying whether One-Tape Turing Machines Run in Linear Time Electronic Colloquium on Computational Complexity, Report No. 36 (2015) Verifying whether One-Tape Turing Machines Run in Linear Time David Gajser IMFM, Jadranska 19, 1000 Ljubljana, Slovenija david.gajser@fmf.uni-lj.si

More information

Chomsky Normal Form and TURING MACHINES. TUESDAY Feb 4

Chomsky Normal Form and TURING MACHINES. TUESDAY Feb 4 Chomsky Normal Form and TURING MACHINES TUESDAY Feb 4 CHOMSKY NORMAL FORM A context-free grammar is in Chomsky normal form if every rule is of the form: A BC A a S ε B and C aren t start variables a is

More information

10. Finite Automata and Turing Machines

10. Finite Automata and Turing Machines . Finite Automata and Turing Machines Frank Stephan March 2, 24 Introduction Alan Turing s th Birthday Alan Turing was a completely original thinker who shaped the modern world, but many people have never

More information

Lecture 20: conp and Friends, Oracles in Complexity Theory

Lecture 20: conp and Friends, Oracles in Complexity Theory 6.045 Lecture 20: conp and Friends, Oracles in Complexity Theory 1 Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode:

More information

Time Complexity (1) CSCI Spring Original Slides were written by Dr. Frederick W Maier. CSCI 2670 Time Complexity (1)

Time Complexity (1) CSCI Spring Original Slides were written by Dr. Frederick W Maier. CSCI 2670 Time Complexity (1) Time Complexity (1) CSCI 2670 Original Slides were written by Dr. Frederick W Maier Spring 2014 Time Complexity So far we ve dealt with determining whether or not a problem is decidable. But even if it

More information

Theory of computation: initial remarks (Chapter 11)

Theory of computation: initial remarks (Chapter 11) Theory of computation: initial remarks (Chapter 11) For many purposes, computation is elegantly modeled with simple mathematical objects: Turing machines, finite automata, pushdown automata, and such.

More information

On Rice s theorem. Hans Hüttel. October 2001

On Rice s theorem. Hans Hüttel. October 2001 On Rice s theorem Hans Hüttel October 2001 We have seen that there exist languages that are Turing-acceptable but not Turing-decidable. An important example of such a language was the language of the Halting

More information

198:538 Lecture Given on February 23rd, 1998

198:538 Lecture Given on February 23rd, 1998 198:538 Lecture Given on February 23rd, 1998 Lecturer: Professor Eric Allender Scribe: Sunny Daniels November 25, 1998 1 A Result From theorem 5 of the lecture on 16th February 1998, together with the

More information

17.1 The Halting Problem

17.1 The Halting Problem CS125 Lecture 17 Fall 2016 17.1 The Halting Problem Consider the HALTING PROBLEM (HALT TM ): Given a TM M and w, does M halt on input w? Theorem 17.1 HALT TM is undecidable. Suppose HALT TM = { M,w : M

More information

Advanced Undecidability Proofs

Advanced Undecidability Proofs 17 Advanced Undecidability Proofs In this chapter, we will discuss Rice s Theorem in Section 17.1, and the computational history method in Section 17.3. As discussed in Chapter 16, these are two additional

More information

Introduction to Computability

Introduction to Computability Chapter 2 Introduction to Computability This subject is primarily concerned with the limitations of computing. As one of the highlights of this study, we will learn several specific problems that computers

More information

ECS 120 Lesson 20 The Post Correspondence Problem

ECS 120 Lesson 20 The Post Correspondence Problem ECS 120 Lesson 20 The Post Correspondence Problem Oliver Kreylos Wednesday, May 16th, 2001 Today we will continue yesterday s discussion of reduction techniques by looking at another common strategy, reduction

More information

An example of a decidable language that is not a CFL Implementation-level description of a TM State diagram of TM

An example of a decidable language that is not a CFL Implementation-level description of a TM State diagram of TM Turing Machines Review An example of a decidable language that is not a CFL Implementation-level description of a TM State diagram of TM Varieties of TMs Multi-Tape TMs Nondeterministic TMs String Enumerators

More information

Turing Machines Part II

Turing Machines Part II Turing Machines Part II Problem Set Set Five Five due due in in the the box box up up front front using using a late late day. day. Hello Hello Condensed Slide Slide Readers! Readers! This This lecture

More information

Turing Machines Part II

Turing Machines Part II Turing Machines Part II Hello Hello Condensed Slide Slide Readers! Readers! This This lecture lecture is is almost almost entirely entirely animations that that show show how how each each Turing Turing

More information

Machine Characterizations of the Classes of the W-Hierarchy

Machine Characterizations of the Classes of the W-Hierarchy Machine Characterizations of the Classes of the W-Hierarchy Yijia Chen and Jörg Flum Abteilung für Mathematische Logik, Universität Freiburg, Eckerstr. 1, 79104 Freiburg, Germany. chen@zermelo.mathematik.uni-freiburg.de

More information

Busch Complexity Lectures: Turing Machines. Prof. Busch - LSU 1

Busch Complexity Lectures: Turing Machines. Prof. Busch - LSU 1 Busch Complexity ectures: Turing Machines Prof. Busch - SU 1 The anguage Hierarchy a n b n c n? ww? Context-Free anguages n b n a ww egular anguages a* a *b* Prof. Busch - SU 2 a n b anguages accepted

More information

Final Exam Comments. UVa - cs302: Theory of Computation Spring < Total

Final Exam Comments. UVa - cs302: Theory of Computation Spring < Total UVa - cs302: Theory of Computation Spring 2008 Final Exam Comments < 50 50 59 60 69 70 79 80 89 90 94 95-102 Total 2 6 8 22 16 16 12 Problem 1: Short Answers. (20) For each question, provide a correct,

More information

ENEE 459E/CMSC 498R In-class exercise February 10, 2015

ENEE 459E/CMSC 498R In-class exercise February 10, 2015 ENEE 459E/CMSC 498R In-class exercise February 10, 2015 In this in-class exercise, we will explore what it means for a problem to be intractable (i.e. it cannot be solved by an efficient algorithm). There

More information

Partition of Integers into Distinct Summands with Upper Bounds. Partition of Integers into Even Summands. An Example

Partition of Integers into Distinct Summands with Upper Bounds. Partition of Integers into Even Summands. An Example Partition of Integers into Even Summands We ask for the number of partitions of m Z + into positive even integers The desired number is the coefficient of x m in + x + x 4 + ) + x 4 + x 8 + ) + x 6 + x

More information

Randomness and non-uniformity

Randomness and non-uniformity Randomness and non-uniformity JASS 2006 Course 1: Proofs and Computers Felix Weninger TU München April 2006 Outline Randomized computation 1 Randomized computation 2 Computation with advice Non-uniform

More information

6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, Class 10 Nancy Lynch

6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, Class 10 Nancy Lynch 6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, 2010 Class 10 Nancy Lynch Today Final topic in computability theory: Self-Reference and the Recursion

More information

Lecture 17: Cook-Levin Theorem, NP-Complete Problems

Lecture 17: Cook-Levin Theorem, NP-Complete Problems 6.045 Lecture 17: Cook-Levin Theorem, NP-Complete Problems 1 Is SAT solvable in O(n) time on a multitape TM? Logic circuits of 6n gates for SAT? If yes, then not only is P=NP, but there would be a dream

More information

Homework 8. a b b a b a b. two-way, read/write

Homework 8. a b b a b a b. two-way, read/write Homework 8 309 Homework 8 1. Describe a TM that accepts the set {a n n is a power of 2}. Your description should be at the level of the descriptions in Lecture 29 of the TM that accepts {ww w Σ } and the

More information

DRAFT. Diagonalization. Chapter 4

DRAFT. Diagonalization. Chapter 4 Chapter 4 Diagonalization..the relativized P =?NP question has a positive answer for some oracles and a negative answer for other oracles. We feel that this is further evidence of the difficulty of the

More information

Definition: Alternating time and space Game Semantics: State of machine determines who

Definition: Alternating time and space Game Semantics: State of machine determines who CMPSCI 601: Recall From Last Time Lecture Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins

More information

Computer Sciences Department

Computer Sciences Department Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y

More information

CSC : Homework #3

CSC : Homework #3 CSC 707-001: Homework #3 William J. Cook wjcook@math.ncsu.edu Monday, March 15, 2004 1 Exercise 4.13 on page 118 (3-Register RAM) Exercise 4.13 Verify that a RAM need not have an unbounded number of registers.

More information

An Algebraic Characterization of the Halting Probability

An Algebraic Characterization of the Halting Probability CDMTCS Research Report Series An Algebraic Characterization of the Halting Probability Gregory Chaitin IBM T. J. Watson Research Center, USA CDMTCS-305 April 2007 Centre for Discrete Mathematics and Theoretical

More information

Peter Wood. Department of Computer Science and Information Systems Birkbeck, University of London Automata and Formal Languages

Peter Wood. Department of Computer Science and Information Systems Birkbeck, University of London Automata and Formal Languages and and Department of Computer Science and Information Systems Birkbeck, University of London ptw@dcs.bbk.ac.uk Outline and Doing and analysing problems/languages computability/solvability/decidability

More information

Computability Theory

Computability Theory Computability Theory Cristian S. Calude May 2012 Computability Theory 1 / 1 Bibliography M. Sipser. Introduction to the Theory of Computation, PWS 1997. (textbook) Computability Theory 2 / 1 Supplementary

More information

TAKE-HOME: OUT 02/21 NOON, DUE 02/25 NOON

TAKE-HOME: OUT 02/21 NOON, DUE 02/25 NOON CSE 555 MIDTERM EXAMINATION SOLUTIONS TAKE-HOME: OUT 02/21 NOON, DUE 02/25 NOON The completed test is to be submitted electronically to colbourn@asu.edu before the deadline, or time-stamped by the main

More information