# ,

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Kolmogorov Complexity Carleton College, CS 254, Fall 2013, Prof. Joshua R. Davis based on Sipser, Introduction to the Theory of Computation 1. Introduction Kolmogorov complexity is a theory of lossless data compression. It ponders the existence of compression/decompression schemes, in which long strings of data are compressed into short strings, that can be decompressed back into their longer versions. Kolmogorov complexity is also a theory of information. Intuitively, the length of a compressed string is a measure of how much information the uncompressed string contains. To get an idea of what we mean, consider the uncompressed strings , Both strings have length 70. The first string can be described succinctly: repeat times. In contrast, the second string does not have any apparent pattern. There is no obvious way to describe it, other than just laying out the whole string. It seems to contain more information than the first string does. We begin with a minimal assumption: The process of decompressing a compressed string into its uncompressed version should be algorithmic. Therefore, we define a decompression scheme to be a deterministic Turing machine F that halts on all inputs. This F takes in an input string w, which we regard as the compressed string, and outputs the corresponding uncompressed string F (w) (as the contents of its tape upon halting). For any x, any string w such that F (w) = x is called a description of x. Let d F (x) be the minimal description of x, according to lexicographic order. The Kolmogorov complexity K F (x) is the length of the minimal description of x. Throughout this tutorial, all strings are taken over the alphabet {0, 1}. Whenever we need to represent a non-negative integer m as a string m, we do so in binary. In particular, m log 2 m + 1 for all m 1. These choices have no serious effect on the theory. Theorem 1.1. Under any scheme F, for any string length m, there are incompressible strings meaning, strings x such that K F (x) x. Proof. Let m 1 be any positive string length. There are 2 m 1 strings of length less than m, which, when fed to F, can produce at most 2 m 1 decompressed strings. But there are 2 m strings of length m. Therefore, at least one string of length m is incompressible. 2. The default scheme Let s define a particular decompression scheme F. This F takes as input M, w, where M is a Turing machine and w is an input for M, all encoded into bits according to some pre-arranged 1

2 system. This F runs M on w until it halts (if ever). Then F halts, with its tape containing the same contents as M s final tape. It is conventional to assume that the encoding is such that w is given explicitly at the end of M, w. That is, M, w consists of some encoding of M, followed by some separator mark, followed by w. In particular, M, w = M, + w. This scheme will be our default. When we write d and K without any subscript, we mean this scheme. Example 2.1. Let M be the Turing machine that, on input w, produces 35 concatenated copies of w on its tape, and then halts. Then M, 01 is a description of the first 70-bit string given above. The length of M, w is M, + 2. Our next example expresses the idea that d(x) should never be much longer than x itself, because here is the string x should always be a description of x. Example 2.2. There exists a constant c such that for all x, K(x) c + x. Proof. Let M be a Turing machine that immediately halts. Let c = M,. Then, for any string x, M, x is a description of x, of length M, x = M, + x = c + x. Thus the minimal description of x can be no longer than c + x, and K(x) c + x. This next example says that xx should not require much more description than x. Example 2.3. There exists a constant c such that for all x, K(xx) c + x. Proof. Let M be a Turing machine that repeats its input twice on its tape and then halts. Let c = M,. Then, for any string x, M, x is a description of xx, of length c + x. Now that we have a taste for how compression and decompression work, let s prove a result that says that our default scheme is about as good as any other. Theorem 2.4. For any scheme F there exists a constant c such that K(x) c + K F (x). Proof. Let c = F,. Then F, d F (x) is a description of x in the default scheme. Its length is F, + d F (x) = c + K F (x). 3. Intermediate Results Henceforth we shall work only with our default scheme. For any c 0, we say that a string x is incompressible by c if K(x) > x c. The notion of incompressibility introduced earlier is incompressibility by 1. description of x, should itself be incompressible. This next theorem gets at the idea that d(x), being the minimal Theorem 3.1. There exists a constant c such that for all x, d(x) is incompressible by c. Proof. Let N be a Turing machine that, on input M, w, does the following steps. 2

3 (1) Run M on w. (2) If the output of M is not of the form P, y, then reject. (3) If the output is of the form P, y, then run P on y and halt with that output. Let c = N, + 1. Now suppose, for the sake of contradiction, that x is a string such that d(x) is compressible by c. Then d(d(x)) d(x) c. But N, d(d(x)) is a description of x, and its length is N, d(d(x)) = N, + d(d(x)) (c 1) + d(x) c = d(x) 1. Therefore K(x) d(x) 1, which contradicts the definition of K(x) = d(x). Recall that this whole theory is founded on a mild assumption: that decompression should be algorithmic. Under that assumption, this next theorem shows that optimal compression cannot be algorithmic. (Perhaps we should place more constraints on decompression, to arrive at a theory in which decompression and optimal compression are both algorithmic?) Theorem 3.2. The Kolmogorov complexity is not computable. In other words, there does not exist a Turing machine M that, given any input x, halts with K(x) on its tape. Proof. Suppose, for the sake of contradiction, that such an M exists. Build a decider N that, on input m, outputs some string x satisfying K(x) m. (N tries all strings x of length m, using M to compute K(x), until it finds an x such that K(x) m. Our first theorem guarantees that such an x will be found.) Now let m be a number large enough that m log 2 m 1 > N,, and let x be the output of N on input m. Then N, m is a description of x, of length N, m = N, + m < (m log 2 m 1) + (log 2 m + 1) = m. So K(x) < m. On the other hand, K(x) m, by the definition of N. This contradiction implies that K is not computable. 4. Random Strings In this section, we explain the notion introduced earlier, that a random string has no pattern and hence should not be compressible. A property of strings over {0, 1} is a function f : {0, 1} {True, False}. A property f is said to hold for almost all strings if #{x : x = n, f(x) = False} lim = 0. n #{x : x = n} Intuitively, f is True for typical strings x and False for special cases of x. If you select a string x randomly, then f(x) = True with high probability. As n, the probability that a randomly chosen string x of length n will have f(x) = True goes to 1. Examples of such f include x contains at least 40% 0s and at least 40% 1s. 3

4 the longest run of 0s in x has length between 0.5 log 2 x and 1.5 log 2 x. This notion allows us to investigate properties of random strings without really doing any probability theory. The following purely mathematical lemma shows that we can replace = with in certain parts of the above definition. Sipser uses this fact without proof. You may want to skip the proof on a first reading. Lemma 4.1. Let f be a property that holds for almost all strings. Then lim = 0. n #{x : x n} Proof. Let ɛ > 0. We wish to show that there exists N such that for all n N #{x : x n} For the sake of brevity, let L n = #{x : x = n, f(x) = False}. Because f holds for almost all strings, there exists an M such that for all n > M, #{x : x = n, f(x) = False} #{x : x = n} That is, L n < ɛ 2 2n for all n > M. Pick N large enough so that Then for all n N L i < ɛ ( 2 N+1 1 ). 2 = < L i + L i + < ɛ. < ɛ 2. n i=m+1 n i=m+1 L i ɛ 2 2i < ɛ ( 2 N+1 1 ) + ɛ 2 2 (2n+1 1) ɛ ( 2 n+1 1 ) = ɛ #{ x n}. This proves the lemma. The following theorem says, roughly, that long incompressible strings have every property that holds for almost all strings. In this sense, they are random. Theorem 4.2. Let f be a computable property that holds for almost all strings. Let c 1. Then there exists an N such that f(x) = True for all x such that x N and x is incompressible by c. 4

5 Proof. If f is False on only finitely many strings, then f is true for all longer strings, and the theorem is obviously true. Henceforth assume that f is False on infinitely many strings. Denote these strings s 0, s 1, s 2,... in lexicographic order. For any string x in the sequence s 0, s 1, s 2,..., let i x be its index in the list. That is, i x is the unique number such that s ix = x. Let M be a Turing machine that on input i outputs s i. (How would you design M, using the fact that f is computable?) Then M, i x is a description of x. Fix c 1. By the lemma, there exists a large N so that for all n N #{x : x n} Using the fact that #{x : x n} = 2 n+1 1, we have < 1 2 c+ M, n+1 < 2 c+ M, +2 = 2n c M, 1. If x is any string of length n N such that f(x) = False, then and Therefore i x < 2 n c M, 1 i x n c M,. K(x) M, i x M, + n c M, = n c. So x is compressible by c. In other words, any x of length at least N that is incompressible by c satisfies f(x) = True. 5

### Theory of Computation

Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation 1 / 33 Lecture 20: Overview Incompressible strings Minimal Length Descriptions Descriptive Complexity Dr. Sarmad Abbasi

### Computability Theory

Computability Theory Cristian S. Calude May 2012 Computability Theory 1 / 1 Bibliography M. Sipser. Introduction to the Theory of Computation, PWS 1997. (textbook) Computability Theory 2 / 1 Supplementary

### FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY KOLMOGOROV-CHAITIN (descriptive) COMPLEXITY TUESDAY, MAR 18 CAN WE QUANTIFY HOW MUCH INFORMATION IS IN A STRING? A = 01010101010101010101010101010101

### CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression

CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression Rosencrantz & Guildenstern Are Dead (Tom Stoppard) Rigged Lottery? And the winning numbers are: 1, 2, 3, 4, 5, 6 But is

### Lecture 13: Foundations of Math and Kolmogorov Complexity

6.045 Lecture 13: Foundations of Math and Kolmogorov Complexity 1 Self-Reference and the Recursion Theorem 2 Lemma: There is a computable function q : Σ* Σ* such that for every string w, q(w) is the description

### FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY THURSDAY APRIL 3 REVIEW for Midterm TUESDAY April 8 Definition: A Turing Machine is a 7-tuple T = (Q, Σ, Γ, δ, q, q accept, q reject ), where: Q is a

### COS597D: Information Theory in Computer Science October 19, Lecture 10

COS597D: Information Theory in Computer Science October 9, 20 Lecture 0 Lecturer: Mark Braverman Scribe: Andrej Risteski Kolmogorov Complexity In the previous lectures, we became acquainted with the concept

### CISC 876: Kolmogorov Complexity

March 27, 2007 Outline 1 Introduction 2 Definition Incompressibility and Randomness 3 Prefix Complexity Resource-Bounded K-Complexity 4 Incompressibility Method Gödel s Incompleteness Theorem 5 Outline

### Computability Theory. CS215, Lecture 6,

Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David

### Turing Machines Part III

Turing Machines Part III Announcements Problem Set 6 due now. Problem Set 7 out, due Monday, March 4. Play around with Turing machines, their powers, and their limits. Some problems require Wednesday's

### Finish K-Complexity, Start Time Complexity

6.045 Finish K-Complexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts

### Introduction to Turing Machines. Reading: Chapters 8 & 9

Introduction to Turing Machines Reading: Chapters 8 & 9 1 Turing Machines (TM) Generalize the class of CFLs: Recursively Enumerable Languages Recursive Languages Context-Free Languages Regular Languages

### CS 361 Meeting 26 11/10/17

CS 361 Meeting 26 11/10/17 1. Homework 8 due Announcements A Recognizable, but Undecidable Language 1. Last class, I presented a brief, somewhat inscrutable proof that the language A BT M = { M w M is

### Further discussion of Turing machines

Further discussion of Turing machines In this lecture we will discuss various aspects of decidable and Turing-recognizable languages that were not mentioned in previous lectures. In particular, we will

### The Turing Machine. Computability. The Church-Turing Thesis (1936) Theory Hall of Fame. Theory Hall of Fame. Undecidability

The Turing Machine Computability Motivating idea Build a theoretical a human computer Likened to a human with a paper and pencil that can solve problems in an algorithmic way The theoretical provides a

### Lecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009

CS 373: Theory of Computation Sariel Har-Peled and Madhusudan Parthasarathy Lecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009 This lecture covers Rice s theorem, as well as

### Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar

IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar Turing Machine A Turing machine is an abstract representation of a computing device. It consists of a read/write

### CpSc 421 Homework 9 Solution

CpSc 421 Homework 9 Solution Attempt any three of the six problems below. The homework is graded on a scale of 100 points, even though you can attempt fewer or more points than that. Your recorded grade

### Decision Problems with TM s. Lecture 31: Halting Problem. Universe of discourse. Semi-decidable. Look at following sets: CSCI 81 Spring, 2012

Decision Problems with TM s Look at following sets: Lecture 31: Halting Problem CSCI 81 Spring, 2012 Kim Bruce A TM = { M,w M is a TM and w L(M)} H TM = { M,w M is a TM which halts on input w} TOTAL TM

### CS 125 Section #10 (Un)decidability and Probability November 1, 2016

CS 125 Section #10 (Un)decidability and Probability November 1, 2016 1 Countability Recall that a set S is countable (either finite or countably infinite) if and only if there exists a surjective mapping

### Lecture Notes: The Halting Problem; Reductions

Lecture Notes: The Halting Problem; Reductions COMS W3261 Columbia University 20 Mar 2012 1 Review Key point. Turing machines can be encoded as strings, and other Turing machines can read those strings

### Lecture 12: Mapping Reductions

Lecture 12: Mapping Reductions October 18, 2016 CS 1010 Theory of Computation Topics Covered 1. The Language EQ T M 2. Mapping Reducibility 3. The Post Correspondence Problem 1 The Language EQ T M The

### Theory of Computation

Theory of Computation Lecture #2 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 1 Lecture 2: Overview Recall some basic definitions from Automata Theory.

### Sample Project: Simulation of Turing Machines by Machines with only Two Tape Symbols

Sample Project: Simulation of Turing Machines by Machines with only Two Tape Symbols The purpose of this document is to illustrate what a completed project should look like. I have chosen a problem that

### Decidability and Undecidability

Decidability and Undecidability Major Ideas from Last Time Every TM can be converted into a string representation of itself. The encoding of M is denoted M. The universal Turing machine U TM accepts an

### Kolmogorov Complexity in Randomness Extraction

LIPIcs Leibniz International Proceedings in Informatics Kolmogorov Complexity in Randomness Extraction John M. Hitchcock, A. Pavan 2, N. V. Vinodchandran 3 Department of Computer Science, University of

### 1 Computational Problems

Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

### Turing machines Finite automaton no storage Pushdown automaton storage is a stack What if we give the automaton a more flexible storage?

Turing machines Finite automaton no storage Pushdown automaton storage is a stack What if we give the automaton a more flexible storage? What is the most powerful of automata? In this lecture we will introduce

### Lecture 23 : Nondeterministic Finite Automata DRAFT Connection between Regular Expressions and Finite Automata

CS/Math 24: Introduction to Discrete Mathematics 4/2/2 Lecture 23 : Nondeterministic Finite Automata Instructor: Dieter van Melkebeek Scribe: Dalibor Zelený DRAFT Last time we designed finite state automata

### Turing s thesis: (1930) Any computation carried out by mechanical means can be performed by a Turing Machine

Turing s thesis: (1930) Any computation carried out by mechanical means can be performed by a Turing Machine There is no known model of computation more powerful than Turing Machines Definition of Algorithm:

### Undecidable Problems and Reducibility

University of Georgia Fall 2014 Reducibility We show a problem decidable/undecidable by reducing it to another problem. One type of reduction: mapping reduction. Definition Let A, B be languages over Σ.

### MTAT Complexity Theory October 13th-14th, Lecture 6

MTAT.07.004 Complexity Theory October 13th-14th, 2011 Lecturer: Peeter Laud Lecture 6 Scribe(s): Riivo Talviste 1 Logarithmic memory Turing machines working in logarithmic space become interesting when

### How to Pop a Deep PDA Matters

How to Pop a Deep PDA Matters Peter Leupold Department of Mathematics, Faculty of Science Kyoto Sangyo University Kyoto 603-8555, Japan email:leupold@cc.kyoto-su.ac.jp Abstract Deep PDA are push-down automata

### TAKE-HOME: OUT 02/21 NOON, DUE 02/25 NOON

CSE 555 MIDTERM EXAMINATION SOLUTIONS TAKE-HOME: OUT 02/21 NOON, DUE 02/25 NOON The completed test is to be submitted electronically to colbourn@asu.edu before the deadline, or time-stamped by the main

### Busch Complexity Lectures: Turing Machines. Prof. Busch - LSU 1

Busch Complexity ectures: Turing Machines Prof. Busch - SU 1 The anguage Hierarchy a n b n c n? ww? Context-Free anguages n b n a ww egular anguages a* a *b* Prof. Busch - SU 2 a n b anguages accepted

### Theory of Computation Lecture Notes. Problems and Algorithms. Class Information

Theory of Computation Lecture Notes Prof. Yuh-Dauh Lyuu Dept. Computer Science & Information Engineering and Department of Finance National Taiwan University Problems and Algorithms c 2004 Prof. Yuh-Dauh

### CS 21 Decidability and Tractability Winter Solution Set 3

CS 21 Decidability and Tractability Winter 2018 Posted: January 31 Solution Set 3 If you have not yet turned in the Problem Set, you should not consult these solutions. 1. (a) A 2-NPDA is a 7-tuple (Q,,

### Turing Machines. The Language Hierarchy. Context-Free Languages. Regular Languages. Courtesy Costas Busch - RPI 1

Turing Machines a n b n c The anguage Hierarchy n? ww? Context-Free anguages a n b n egular anguages a * a *b* ww Courtesy Costas Busch - PI a n b n c n Turing Machines anguages accepted by Turing Machines

### Recovery Based on Kolmogorov Complexity in Underdetermined Systems of Linear Equations

Recovery Based on Kolmogorov Complexity in Underdetermined Systems of Linear Equations David Donoho Department of Statistics Stanford University Email: donoho@stanfordedu Hossein Kakavand, James Mammen

### CSE 105 Theory of Computation

CSE 105 Theory of Computation http://www.jflap.org/jflaptmp/ Professor Jeanne Ferrante 1 Undecidability Today s Agenda Review and More Problems A Non-TR Language Reminders and announcements: HW 7 (Last!!)

### Lecture 20: conp and Friends, Oracles in Complexity Theory

6.045 Lecture 20: conp and Friends, Oracles in Complexity Theory 1 Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode:

### Randomized Complexity Classes; RP

Randomized Complexity Classes; RP Let N be a polynomial-time precise NTM that runs in time p(n) and has 2 nondeterministic choices at each step. N is a polynomial Monte Carlo Turing machine for a language

### 1 Deterministic Turing Machines

Time and Space Classes Exposition by William Gasarch 1 Deterministic Turing Machines Turing machines are a model of computation. It is believed that anything that can be computed can be computed by a Turing

### A version of for which ZFC can not predict a single bit Robert M. Solovay May 16, Introduction In [2], Chaitin introd

CDMTCS Research Report Series A Version of for which ZFC can not Predict a Single Bit Robert M. Solovay University of California at Berkeley CDMTCS-104 May 1999 Centre for Discrete Mathematics and Theoretical

### Notes on State Minimization

U.C. Berkeley CS172: Automata, Computability and Complexity Handout 1 Professor Luca Trevisan 2/3/2015 Notes on State Minimization These notes present a technique to prove a lower bound on the number of

### CS154, Lecture 10: Rice s Theorem, Oracle Machines

CS154, Lecture 10: Rice s Theorem, Oracle Machines Moral: Analyzing Programs is Really, Really Hard But can we more easily tell when some program analysis problem is undecidable? Problem 1 Undecidable

### 198:538 Lecture Given on February 23rd, 1998

198:538 Lecture Given on February 23rd, 1998 Lecturer: Professor Eric Allender Scribe: Sunny Daniels November 25, 1998 1 A Result From theorem 5 of the lecture on 16th February 1998, together with the

### Lecture Examples of problems which have randomized algorithms

6.841 Advanced Complexity Theory March 9, 2009 Lecture 10 Lecturer: Madhu Sudan Scribe: Asilata Bapat Meeting to talk about final projects on Wednesday, 11 March 2009, from 5pm to 7pm. Location: TBA. Includes

### KOLMOGOROV COMPLEXITY AND ALGORITHMIC RANDOMNESS

KOLMOGOROV COMPLEXITY AND ALGORITHMIC RANDOMNESS HENRY STEINITZ Abstract. This paper aims to provide a minimal introduction to algorithmic randomness. In particular, we cover the equivalent 1-randomness

### 1 Randomized complexity

80240233: Complexity of Computation Lecture 6 ITCS, Tsinghua Univesity, Fall 2007 23 October 2007 Instructor: Elad Verbin Notes by: Zhang Zhiqiang and Yu Wei 1 Randomized complexity So far our notion of

### On Rice s theorem. Hans Hüttel. October 2001

On Rice s theorem Hans Hüttel October 2001 We have seen that there exist languages that are Turing-acceptable but not Turing-decidable. An important example of such a language was the language of the Halting

### DRAFT. Diagonalization. Chapter 4

Chapter 4 Diagonalization..the relativized P =?NP question has a positive answer for some oracles and a negative answer for other oracles. We feel that this is further evidence of the difficulty of the

### Turing Machine Variants

CS311 Computational Structures Turing Machine Variants Lecture 12 Andrew Black Andrew Tolmach 1 The Church-Turing Thesis The problems that can be decided by an algorithm are exactly those that can be decided

### About the relationship between formal logic and complexity classes

About the relationship between formal logic and complexity classes Working paper Comments welcome; my email: armandobcm@yahoo.com Armando B. Matos October 20, 2013 1 Introduction We analyze a particular

17 Advanced Undecidability Proofs In this chapter, we will discuss Rice s Theorem in Section 17.1, and the computational history method in Section 17.3. As discussed in Chapter 16, these are two additional

### Final Exam Comments. UVa - cs302: Theory of Computation Spring < Total

UVa - cs302: Theory of Computation Spring 2008 Final Exam Comments < 50 50 59 60 69 70 79 80 89 90 94 95-102 Total 2 6 8 22 16 16 12 Problem 1: Short Answers. (20) For each question, provide a correct,

### CSE 105 THEORY OF COMPUTATION

CSE 105 THEORY OF COMPUTATION "Winter" 2018 http://cseweb.ucsd.edu/classes/wi18/cse105-ab/ Today's learning goals Sipser Ch 4.2 Trace high-level descriptions of algorithms for computational problems. Use

### Kolmogorov structure functions for automatic complexity

Kolmogorov structure functions for automatic complexity Bjørn Kjos-Hanssen June 16, 2015 Varieties of Algorithmic Information, University of Heidelberg Internationales Wissenschaftssentrum History 1936:

### 17.1 The Halting Problem

CS125 Lecture 17 Fall 2016 17.1 The Halting Problem Consider the HALTING PROBLEM (HALT TM ): Given a TM M and w, does M halt on input w? Theorem 17.1 HALT TM is undecidable. Suppose HALT TM = { M,w : M

### cse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska

cse303 ELEMENTS OF THE THEORY OF COMPUTATION Professor Anita Wasilewska LECTURE 13 CHAPTER 4 TURING MACHINES 1. The definition of Turing machine 2. Computing with Turing machines 3. Extensions of Turing

### CSCE 551: Chin-Tser Huang. University of South Carolina

CSCE 551: Theory of Computation Chin-Tser Huang huangct@cse.sc.edu University of South Carolina Church-Turing Thesis The definition of the algorithm came in the 1936 papers of Alonzo Church h and Alan

### (a) Definition of TMs. First Problem of URMs

Sec. 4: Turing Machines First Problem of URMs (a) Definition of the Turing Machine. (b) URM computable functions are Turing computable. (c) Undecidability of the Turing Halting Problem That incrementing

### Computational Models Lecture 9, Spring 2009

Slides modified by Benny Chor, based on original slides by Maurice Herlihy, Brown University. p. 1 Computational Models Lecture 9, Spring 2009 Reducibility among languages Mapping reductions More undecidable

### NP-Completeness. Until now we have been designing algorithms for specific problems

NP-Completeness 1 Introduction Until now we have been designing algorithms for specific problems We have seen running times O(log n), O(n), O(n log n), O(n 2 ), O(n 3 )... We have also discussed lower

### Turing machines COMS Ashley Montanaro 21 March Department of Computer Science, University of Bristol Bristol, UK

COMS11700 Turing machines Department of Computer Science, University of Bristol Bristol, UK 21 March 2014 COMS11700: Turing machines Slide 1/15 Introduction We have seen two models of computation: finite

### A Turing Machine Distance Hierarchy

A Turing Machine Distance Hierarchy Stanislav Žák and Jiří Šíma Institute of Computer Science, Academy of Sciences of the Czech Republic, P. O. Box 5, 18207 Prague 8, Czech Republic, {stan sima}@cs.cas.cz

### 6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch

6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch Today: More Complexity Theory Polynomial-time reducibility, NP-completeness, and the Satisfiability (SAT) problem Topics: Introduction

### Kolmogorov complexity and its applications

CS860, Winter, 2010 Kolmogorov complexity and its applications Ming Li School of Computer Science University of Waterloo http://www.cs.uwaterloo.ca/~mli/cs860.html We live in an information society. Information

### Sophistication Revisited

Sophistication Revisited Luís Antunes Lance Fortnow August 30, 007 Abstract Kolmogorov complexity measures the ammount of information in a string as the size of the shortest program that computes the string.

### Verifying whether One-Tape Turing Machines Run in Linear Time

Electronic Colloquium on Computational Complexity, Report No. 36 (2015) Verifying whether One-Tape Turing Machines Run in Linear Time David Gajser IMFM, Jadranska 19, 1000 Ljubljana, Slovenija david.gajser@fmf.uni-lj.si

### Variants of Turing Machine (intro)

CHAPTER 3 The Church-Turing Thesis Contents Turing Machines definitions, examples, Turing-recognizable and Turing-decidable languages Variants of Turing Machine Multi-tape Turing machines, non-deterministic

### Min/Max-Poly Weighting Schemes and the NL vs UL Problem

Min/Max-Poly Weighting Schemes and the NL vs UL Problem Anant Dhayal Jayalal Sarma Saurabh Sawlani May 3, 2016 Abstract For a graph G(V, E) ( V = n) and a vertex s V, a weighting scheme (w : E N) is called

### Math Real Analysis

1 / 28 Math 370 - Real Analysis G.Pugh Sep 3 2013 Real Analysis 2 / 28 3 / 28 What is Real Analysis? Wikipedia: Real analysis... has its beginnings in the rigorous formulation of calculus. It is a branch

### Boolean circuits. Lecture Definitions

Lecture 20 Boolean circuits In this lecture we will discuss the Boolean circuit model of computation and its connection to the Turing machine model. Although the Boolean circuit model is fundamentally

### Automata Theory CS Complexity Theory I: Polynomial Time

Automata Theory CS411-2015-17 Complexity Theory I: Polynomial Time David Galles Department of Computer Science University of San Francisco 17-0: Tractable vs. Intractable If a problem is recursive, then

### ECS 120 Lesson 20 The Post Correspondence Problem

ECS 120 Lesson 20 The Post Correspondence Problem Oliver Kreylos Wednesday, May 16th, 2001 Today we will continue yesterday s discussion of reduction techniques by looking at another common strategy, reduction

### CSE 555 HW 5 SAMPLE SOLUTION. Question 1.

CSE 555 HW 5 SAMPLE SOLUTION Question 1. Show that if L is PSPACE-complete, then L is NP-hard. Show that the converse is not true. If L is PSPACE-complete, then for all A PSPACE, A P L. We know SAT PSPACE

### Turing machines and linear bounded automata

and linear bounded automata Informatics 2A: Lecture 29 John Longley School of Informatics University of Edinburgh jrl@inf.ed.ac.uk 25 November, 2011 1 / 13 1 The Chomsky hierarchy: summary 2 3 4 2 / 13

### Filters in Analysis and Topology

Filters in Analysis and Topology David MacIver July 1, 2004 Abstract The study of filters is a very natural way to talk about convergence in an arbitrary topological space, and carries over nicely into

### Decidability. William Chan

Decidability William Chan Preface : In 1928, David Hilbert gave a challenge known as the Entscheidungsproblem, which is German for Decision Problem. Hilbert s problem asked for some purely mechanical procedure

### PS2 - Comments. University of Virginia - cs3102: Theory of Computation Spring 2010

University of Virginia - cs3102: Theory of Computation Spring 2010 PS2 - Comments Average: 77.4 (full credit for each question is 100 points) Distribution (of 54 submissions): 90, 12; 80 89, 11; 70-79,

### Lecture 3: Reductions and Completeness

CS 710: Complexity Theory 9/13/2011 Lecture 3: Reductions and Completeness Instructor: Dieter van Melkebeek Scribe: Brian Nixon Last lecture we introduced the notion of a universal Turing machine for deterministic

### Stochasticity in Algorithmic Statistics for Polynomial Time

Stochasticity in Algorithmic Statistics for Polynomial Time Alexey Milovanov 1 and Nikolay Vereshchagin 2 1 National Research University Higher School of Economics, Moscow, Russia almas239@gmail.com 2

### Chapter 2 Algorithms and Computation

Chapter 2 Algorithms and Computation In this chapter, we first discuss the principles of algorithm and computation in general framework, common both in classical and quantum computers, then we go to the

### Theory of Computation

Thomas Zeugmann Hokkaido University Laboratory for Algorithmics http://www-alg.ist.hokudai.ac.jp/ thomas/toc/ Lecture 3: Finite State Automata Motivation In the previous lecture we learned how to formalize

### Turing machines and computable functions

Turing machines and computable functions In this chapter we address the question: how can we define the class of discrete functions that are computable? Probably, most of you are familiar with a few programming

### Lecture 24: Randomized Complexity, Course Summary

6.045 Lecture 24: Randomized Complexity, Course Summary 1 1/4 1/16 1/4 1/4 1/32 1/16 1/32 Probabilistic TMs 1/16 A probabilistic TM M is a nondeterministic TM where: Each nondeterministic step is called

### Theory of Computer Science. Theory of Computer Science. D7.1 Introduction. D7.2 Turing Machines as Words. D7.3 Special Halting Problem

Theory of Computer Science May 2, 2018 D7. Halting Problem and Reductions Theory of Computer Science D7. Halting Problem and Reductions Gabriele Röger University of Basel May 2, 2018 D7.1 Introduction

### 1 More finite deterministic automata

CS 125 Section #6 Finite automata October 18, 2016 1 More finite deterministic automata Exercise. Consider the following game with two players: Repeatedly flip a coin. On heads, player 1 gets a point.

### S 1 S 2 L S 1 L S 2 S 2 R

Turing Machines Thomas Wieting Reed College, 2011 1 Verify/Prove 2 Turing Machines 3 Problems 4 Computable Functions 5 Gödel Numbers 6 Universal Turing Machines 7 The Halting Problem 8 The Busy Beaver

### Lecture 14: IP = PSPACE

IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Basic Course on Computational Complexity Lecture 14: IP = PSPACE David Mix Barrington and Alexis Maciel August 3, 2000 1. Overview We

### 1 Acceptance, Rejection, and I/O for Turing Machines

1 Acceptance, Rejection, and I/O for Turing Machines Definition 1.1 (Initial Configuration) If M = (K,Σ,δ,s,H) is a Turing machine and w (Σ {, }) then the initial configuration of M on input w is (s, w).

### Automata and Computability. Solutions to Exercises

Automata and Computability Solutions to Exercises Spring 27 Alexis Maciel Department of Computer Science Clarkson University Copyright c 27 Alexis Maciel ii Contents Preface vii Introduction 2 Finite Automata

### Theory of Computation

Theory of Computation (Feodor F. Dragan) Department of Computer Science Kent State University Spring, 2018 Theory of Computation, Feodor F. Dragan, Kent State University 1 Before we go into details, what

### Homework 5 Solution CSCI 2670 October 6, c) New text q 1 1##1 xq 3 ##1 x#q 5 #1 q reject. Old text. ~q 3 ##1 ~#q 5 #1

Homework 5 Solution CSCI 2670 October 6, 2005 3.2 c) New text q 1 1##1 xq 3 ##1 x#q 5 #1 q reject Old text q 1 1##1 ~q 3 ##1 ~#q 5 #1 q reject 3.2 e) New text q 1 10#10 xq 3 0#10 x0q 3 #10 x0#q 5 10 x0q

### Turing Machines and Time Complexity

Turing Machines and Time Complexity Turing Machines Turing Machines (Infinitely long) Tape of 1 s and 0 s Turing Machines (Infinitely long) Tape of 1 s and 0 s Able to read and write the tape, and move

### Theory of Computation

Theory of Computation Lecture #10 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 43 Lecture 10: Overview Linear Bounded Automata Acceptance Problem for LBAs

### Computability Theory

Computability Theory Cristian S. Calude and Nicholas J. Hay May June 2009 Computability Theory 1 / 155 1 Register machines 2 Church-Turing thesis 3 Decidability 4 Reducibility 5 A definition of information