# Theory of Computation

Save this PDF as:

Size: px
Start display at page:

## Transcription

1 Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation 1 / 33

2 Lecture 20: Overview Incompressible strings Minimal Length Descriptions Descriptive Complexity Dr. Sarmad Abbasi () Theory of Computation 2 / 33

3 Incompressible strings Recall that a string, x, is b-compressible if K(x) x b 1 A string is b-incompressible if it is not b-compressible. 2 A string is incompressible if it is not 1-compressible. Incompressible strings cannot be compressed. Their description is at least as long as the string itself. Dr. Sarmad Abbasi () Theory of Computation 3 / 33

4 Incompressible strings 1 Incompressible strings look like random strings. 2 This can be made precise in many ways. Many properties of random strings are also true for incompressible strings. For example Dr. Sarmad Abbasi () Theory of Computation 4 / 33

5 Incompressible strings We proved two theorems: Theorem For every n there is a incompressible string of length n. Theorem The number of b-incompressible strings of length b is at least The proof is simple counting! 2 n 2 n b Dr. Sarmad Abbasi () Theory of Computation 5 / 33

6 Incompressible strings 1 Incompressible strings have roughly the same number of 0 s and 1 s. 2 The longest run of 0 is of size O(log n). Let us prove a theorem which is central in this theory. Dr. Sarmad Abbasi () Theory of Computation 6 / 33

7 Incompressible strings Theorem A computable property that holds for almost all strings also holds for incompressible strings. Let us formalize this statement. A property p is a mapping from set of all strings to {TRUE, FALSE}. Thus p : {0, 1} {true, false} Dr. Sarmad Abbasi () Theory of Computation 7 / 33

8 Incompressible strings We say that p holds for almost all strings if {x {0, 1} n : p(x) = false } 2 n approaches 0 as n approaches infinity. Dr. Sarmad Abbasi () Theory of Computation 8 / 33

9 Incompressible strings Theorem Let p be a computable property. If p holds for almost all strings the for any b > 0, the property p is false on only finitely many strings that are incompressible by b. Let M be the following algorithm: 1 On input i (in binary) 2 Find the ith string s such that p(s) = False. Considering the strings lexicographically. 3 Output s. Dr. Sarmad Abbasi () Theory of Computation 9 / 33

10 Incompressible strings We can use M to obtain short descriptions of string that do not have property p. Let x be a string that does not have property p. Let i x be the index of x in the list of all strings that do not have property p ordered lexicographically. Then M, i x is a description of x. The length of this description is i x + c Dr. Sarmad Abbasi () Theory of Computation 10 / 33

11 Incompressible strings Now we count. For any give b > 0 select n such that at most 1 2 b+c fraction of the strings of length n do not have property p. Note that since p holds for almost all strings, this is true for all sufficiently large n. Dr. Sarmad Abbasi () Theory of Computation 11 / 33

12 Incompressible strings Then we have i x 2n 2 b+c Therefore, i x n b c. Hence the length of M i x is at most n b c + c = n b. Thus K(x) n b which means x is b-compressible. Hence, only finitely many strings that fail p can be incompressible by b. Dr. Sarmad Abbasi () Theory of Computation 12 / 33

13 Incompressible strings We can ask a very interesting question now: Is it possible to compress a string that is already compressed? The answer to this question seems to be no! Let us prove something that is quite close to this. Dr. Sarmad Abbasi () Theory of Computation 13 / 33

14 Incompressible strings Theorem There is a constant b such that for every string x the minimal description d(x) is incompressible by b. Roughly the idea of the proof is that If d(x) were compressible we could use the compressed version of d(x) to obtain a smaller description of x than d(x). Dr. Sarmad Abbasi () Theory of Computation 14 / 33

15 Incompressible strings Consider the following TM M: 1 On input R, y 2 Run R on y and reject if its output is not of the form S, z 3 Run S on z and halt with its output on the tape. This machine is decoding twice. Let b = 2 M + 3. We now show that b satisfies the statement of the theorem. Suppose on the contrary that d(x) is b compressible for some string. The d(d(x)) d(x) b. But what happens if we run M on d(d(x)). Note that it outputs x Dr. Sarmad Abbasi () Theory of Computation 15 / 33

16 Incompressible strings Thus < M > d(d(x)) is a description of x. How long is this description? 2 M d(d(x)) = (b 1) + ( d(x) b) = d(x) 1 This contradicts the fact that d(x) is the minimal description of x. Dr. Sarmad Abbasi () Theory of Computation 16 / 33

17 Complexity Theory Now we will look at resource bounded computations. Two of the most important resources are going to be 1 Time 2 Space Dr. Sarmad Abbasi () Theory of Computation 17 / 33

18 Complexity Theory Let A = {0 k 1 k : k 0} Consider the following TM that accepts A 1 Scan across the tape to check if all 0 s are before 1 s if not reject. 2 Scan across the tape crossing of a 0 and a 1 each time. 3 If 0 s remain and 1 s finished reject. 4 if 1 s remain and 0 s are finished reject. 5 if both 0 s and 1 s are finished accept. 6 Goto Step 2 We want to analyze this algorithm and talk about how many steps (time) does it take. Dr. Sarmad Abbasi () Theory of Computation 18 / 33

19 Complexity Theory A step of the Turing machine is assumed to take unit time. Lets start with a definition. Let M be a TM that halts on all inputs. The running time or the time complexity of M is a function f : N N such that f (n) is the maximum number of steps that M uses on any input of length n. 1 We say that M runs in time f (n). 2 M is f (n)-time TM. Dr. Sarmad Abbasi () Theory of Computation 19 / 33

20 Big O notation We want to understand how much time a TM takes in the worst case. The exact running time t(n) is going to be very complicated expression. In some cases we will not be able to compute it. But what we are interested in is getting a rough idea of how t(n) grows for large inputs. So we will use asymptotic notation. Dr. Sarmad Abbasi () Theory of Computation 20 / 33

21 Big O notation Lets look at an example let f (n) = 5n n 2 6n + 48 This function has four terms. He most important term is 5n 3 which will be the bulk if n is large. Dr. Sarmad Abbasi () Theory of Computation 21 / 33

22 Big O notation In fact 5n 3 the constant 5 does not contribute to the growth of this function. Thus we will say that f (n) is order n 3. We will denote this by Let us state this precisely: f (n) = O(n 3 ). Dr. Sarmad Abbasi () Theory of Computation 22 / 33

23 Big O notation Let f, g : N R +. We say that f (n) = O(g(n)) if there are positive integers c, n 0 such that for all n n 0 f (n) cg(n) When f (n) = O(g(n)) we say that g is an asymptotic upper-bound on f. Sometime we will just say that g is an upper bound on f. Dr. Sarmad Abbasi () Theory of Computation 23 / 33

24 Big O notation Intuitively if f (n) = O(g(n)) you can think that f is less than or equal to some multiple of g. For most functions the highest order term is is an obvious candidate to be g. Dr. Sarmad Abbasi () Theory of Computation 24 / 33

25 Big O notation Let Then Furthermore, But it is not true that f (n) = 13n 4 + 7n f (n) = O(n 4 ) f (n) = O(n 5 ) f (n) = O(n 3 ) Dr. Sarmad Abbasi () Theory of Computation 25 / 33

26 Big O notation When we say that f (n) = O(g(n)) we are only claiming that g(n) is an asymptotic upper bound on f. It may not be the best one. Lets look at other examples. Dr. Sarmad Abbasi () Theory of Computation 26 / 33

27 Big O notation Suppose that where b is a constant. Note that f (n) = log b n log b n = log 2 n log 2 b. Hence we can say Let then f (n) = O(log n) f 2 (n) = 5n log n + 200n log log n + 17 f 2 (n) = O(n log n) Dr. Sarmad Abbasi () Theory of Computation 27 / 33

28 Big O notation We can use O notation in exponents also. Thus 2 O(n) stands for a function that is upper bounded by 2 cn for some constant c. We have to be careful though For example O(log n) 2 Well remember that O(log n) has a constant attached to it. Thus this is upper bounded by 2 c log n by some constant c. Now, we get 2 c log n = (2 log n ) c = n c Dr. Sarmad Abbasi () Theory of Computation 28 / 33

29 Big O notation Bounds of the form n c are called polynomial bounds. Where as bounds of the form 2 nc are called exponential bounds. Dr. Sarmad Abbasi () Theory of Computation 29 / 33

30 small o notation The Big O notation roughly says that f is upper bounded by g. Now we want to say f grows strictly slower that g. Let f, g : N R +. We say that f (n) = o(g(n)) if f (n) lim n g(n) = 0 Thus the ratio of f and g becomes closer and closer to 0. Dr. Sarmad Abbasi () Theory of Computation 30 / 33

31 small o notation 1 n = o(n) 2 n = o(n log log n) 3 n log log n = o(n log n) 4 n log n = o(n 2 ) 5 n 2 = o(n 3 ). Dr. Sarmad Abbasi () Theory of Computation 31 / 33

32 small o notation Lets look at a new TM that recognizes A = {0 k 1 k : k 0} 1 Scan the tape and make sure all the 0s are before 1s. 2 Repeat till there are 0s and 1 s on the tape. 3 Scan the tape and if the total number of 0 s and 1 s is odd reject. 4 Scan and cross of every other 0 and do the same with every other 1. 5 If all 0 s and 1 s are crossed accept. Thus if we have 13 0 s in the next stage there would be 6. Dr. Sarmad Abbasi () Theory of Computation 32 / 33

33 small o notation How much time does this TM take? O(n log n) Thus the time taken by this TM is o(n 2 ). Hence, this reflects a real algorithmic improvement. Dr. Sarmad Abbasi () Theory of Computation 33 / 33

### ,

Kolmogorov Complexity Carleton College, CS 254, Fall 2013, Prof. Joshua R. Davis based on Sipser, Introduction to the Theory of Computation 1. Introduction Kolmogorov complexity is a theory of lossless

### Finish K-Complexity, Start Time Complexity

6.045 Finish K-Complexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts

### FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY KOLMOGOROV-CHAITIN (descriptive) COMPLEXITY TUESDAY, MAR 18 CAN WE QUANTIFY HOW MUCH INFORMATION IS IN A STRING? A = 01010101010101010101010101010101

### Theory of Computation

Theory of Computation Lecture #2 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 1 Lecture 2: Overview Recall some basic definitions from Automata Theory.

### CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression

CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression Rosencrantz & Guildenstern Are Dead (Tom Stoppard) Rigged Lottery? And the winning numbers are: 1, 2, 3, 4, 5, 6 But is

### Lecture 13: Foundations of Math and Kolmogorov Complexity

6.045 Lecture 13: Foundations of Math and Kolmogorov Complexity 1 Self-Reference and the Recursion Theorem 2 Lemma: There is a computable function q : Σ* Σ* such that for every string w, q(w) is the description

### Lecture Notes: The Halting Problem; Reductions

Lecture Notes: The Halting Problem; Reductions COMS W3261 Columbia University 20 Mar 2012 1 Review Key point. Turing machines can be encoded as strings, and other Turing machines can read those strings

### Theory of Computation

Theory of Computation Lecture #10 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 43 Lecture 10: Overview Linear Bounded Automata Acceptance Problem for LBAs

### TIME COMPLEXITY AND POLYNOMIAL TIME; NON DETERMINISTIC TURING MACHINES AND NP. THURSDAY Mar 20

TIME COMPLEXITY AND POLYNOMIAL TIME; NON DETERMINISTIC TURING MACHINES AND NP THURSDAY Mar 20 COMPLEXITY THEORY Studies what can and can t be computed under limited resources such as time, space, etc Today:

### Data Structures and Algorithms. Asymptotic notation

Data Structures and Algorithms Asymptotic notation Estimating Running Time Algorithm arraymax executes 7n 1 primitive operations in the worst case. Define: a = Time taken by the fastest primitive operation

### Space Complexity. The space complexity of a program is how much memory it uses.

Space Complexity The space complexity of a program is how much memory it uses. Measuring Space When we compute the space used by a TM, we do not count the input (think of input as readonly). We say that

### Growth of Functions (CLRS 2.3,3)

Growth of Functions (CLRS 2.3,3) 1 Review Last time we discussed running time of algorithms and introduced the RAM model of computation. Best-case running time: the shortest running time for any input

### Big O 2/14/13. Administrative. Does it terminate? David Kauchak cs302 Spring 2013

/4/3 Administrative Big O David Kauchak cs3 Spring 3 l Assignment : how d it go? l Assignment : out soon l CLRS code? l Videos Insertion-sort Insertion-sort Does it terminate? /4/3 Insertion-sort Loop

### Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort

Analysis of Algorithms I: Asymptotic Notation, Induction, and MergeSort Xi Chen Columbia University We continue with two more asymptotic notation: o( ) and ω( ). Let f (n) and g(n) are functions that map

### Computability Theory. CS215, Lecture 6,

Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David

### Sample Project: Simulation of Turing Machines by Machines with only Two Tape Symbols

Sample Project: Simulation of Turing Machines by Machines with only Two Tape Symbols The purpose of this document is to illustrate what a completed project should look like. I have chosen a problem that

### NP-Completeness. Until now we have been designing algorithms for specific problems

NP-Completeness 1 Introduction Until now we have been designing algorithms for specific problems We have seen running times O(log n), O(n), O(n log n), O(n 2 ), O(n 3 )... We have also discussed lower

### Theory of Computation

Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation / Lecture 3: Overview Decidability of Logical Theories Presburger arithmetic Decidability of Presburger Arithmetic Dr.

### Lecture 16: Time Complexity and P vs NP

6.045 Lecture 16: Time Complexity and P vs NP 1 Time-Bounded Complexity Classes Definition: TIME(t(n)) = { L there is a Turing machine M with time complexity O(t(n)) so that L = L(M) } = { L L is a language

### MA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 2

MA008 p.1/36 MA008/MIIZ01 Design and Analysis of Algorithms Lecture Notes 2 Dr. Markus Hagenbuchner markus@uow.edu.au. MA008 p.2/36 Content of lecture 2 Examples Review data structures Data types vs. data

### 198:538 Lecture Given on February 23rd, 1998

198:538 Lecture Given on February 23rd, 1998 Lecturer: Professor Eric Allender Scribe: Sunny Daniels November 25, 1998 1 A Result From theorem 5 of the lecture on 16th February 1998, together with the

### P, NP, NP-Complete, and NPhard

P, NP, NP-Complete, and NPhard Problems Zhenjiang Li 21/09/2011 Outline Algorithm time complicity P and NP problems NP-Complete and NP-Hard problems Algorithm time complicity Outline What is this course

### Computability Theory

Computability Theory Cristian S. Calude May 2012 Computability Theory 1 / 1 Bibliography M. Sipser. Introduction to the Theory of Computation, PWS 1997. (textbook) Computability Theory 2 / 1 Supplementary

### Further discussion of Turing machines

Further discussion of Turing machines In this lecture we will discuss various aspects of decidable and Turing-recognizable languages that were not mentioned in previous lectures. In particular, we will

### CpSc 421 Homework 9 Solution

CpSc 421 Homework 9 Solution Attempt any three of the six problems below. The homework is graded on a scale of 100 points, even though you can attempt fewer or more points than that. Your recorded grade

### Boolean circuits. Lecture Definitions

Lecture 20 Boolean circuits In this lecture we will discuss the Boolean circuit model of computation and its connection to the Turing machine model. Although the Boolean circuit model is fundamentally

### CS 361 Meeting 26 11/10/17

CS 361 Meeting 26 11/10/17 1. Homework 8 due Announcements A Recognizable, but Undecidable Language 1. Last class, I presented a brief, somewhat inscrutable proof that the language A BT M = { M w M is

### Lecture 3: Reductions and Completeness

CS 710: Complexity Theory 9/13/2011 Lecture 3: Reductions and Completeness Instructor: Dieter van Melkebeek Scribe: Brian Nixon Last lecture we introduced the notion of a universal Turing machine for deterministic

### Exponential Functions Dr. Laura J. Pyzdrowski

1 Names: (4 communication points) About this Laboratory An exponential function is an example of a function that is not an algebraic combination of polynomials. Such functions are called trancendental

### 1 Computational Problems

Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

### Computer Sciences Department

Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y

### MTAT Complexity Theory October 13th-14th, Lecture 6

MTAT.07.004 Complexity Theory October 13th-14th, 2011 Lecturer: Peeter Laud Lecture 6 Scribe(s): Riivo Talviste 1 Logarithmic memory Turing machines working in logarithmic space become interesting when

### Decision Problems with TM s. Lecture 31: Halting Problem. Universe of discourse. Semi-decidable. Look at following sets: CSCI 81 Spring, 2012

Decision Problems with TM s Look at following sets: Lecture 31: Halting Problem CSCI 81 Spring, 2012 Kim Bruce A TM = { M,w M is a TM and w L(M)} H TM = { M,w M is a TM which halts on input w} TOTAL TM

### For use only in [the name of your school] 2014 S4 Note. S4 Notes (Edexcel)

s (Edexcel) Copyright www.pgmaths.co.uk - For AS, A2 notes and IGCSE / GCSE worksheets 1 Copyright www.pgmaths.co.uk - For AS, A2 notes and IGCSE / GCSE worksheets 2 Copyright www.pgmaths.co.uk - For AS,

### CSC : Homework #3

CSC 707-001: Homework #3 William J. Cook wjcook@math.ncsu.edu Monday, March 15, 2004 1 Exercise 4.13 on page 118 (3-Register RAM) Exercise 4.13 Verify that a RAM need not have an unbounded number of registers.

### DRAFT. Diagonalization. Chapter 4

Chapter 4 Diagonalization..the relativized P =?NP question has a positive answer for some oracles and a negative answer for other oracles. We feel that this is further evidence of the difficulty of the

### Computational Complexity: A Modern Approach. Draft of a book: Dated January 2007 Comments welcome!

i Computational Complexity: A Modern Approach Draft of a book: Dated January 2007 Comments welcome! Sanjeev Arora and Boaz Barak Princeton University complexitybook@gmail.com Not to be reproduced or distributed

### The Polynomial Hierarchy

The Polynomial Hierarchy Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Motivation..synthesizing circuits is exceedingly difficulty. It is even

### Computational complexity and some Graph Theory

Graph Theory Lars Hellström January 22, 2014 Contents of todays lecture An important concern when choosing the algorithm to use for something (after basic requirements such as correctness and stability)

### CS154, Lecture 10: Rice s Theorem, Oracle Machines

CS154, Lecture 10: Rice s Theorem, Oracle Machines Moral: Analyzing Programs is Really, Really Hard But can we more easily tell when some program analysis problem is undecidable? Problem 1 Undecidable

### b + O(n d ) where a 1, b > 1, then O(n d log n) if a = b d d ) if a < b d O(n log b a ) if a > b d

CS161, Lecture 4 Median, Selection, and the Substitution Method Scribe: Albert Chen and Juliana Cook (2015), Sam Kim (2016), Gregory Valiant (2017) Date: January 23, 2017 1 Introduction Last lecture, we

### 2. Two binary operations (addition, denoted + and multiplication, denoted

Chapter 2 The Structure of R The purpose of this chapter is to explain to the reader why the set of real numbers is so special. By the end of this chapter, the reader should understand the difference between

### Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar

IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar Turing Machine A Turing machine is an abstract representation of a computing device. It consists of a read/write

### CSE 20. Lecture 4: Introduction to Boolean algebra. CSE 20: Lecture4

CSE 20 Lecture 4: Introduction to Boolean algebra Reminder First quiz will be on Friday (17th January) in class. It is a paper quiz. Syllabus is all that has been done till Wednesday. If you want you may

### CIS 121 Data Structures and Algorithms with Java Spring Big-Oh Notation Monday, January 22/Tuesday, January 23

CIS 11 Data Structures and Algorithms with Java Spring 018 Big-Oh Notation Monday, January /Tuesday, January 3 Learning Goals Review Big-Oh and learn big/small omega/theta notations Discuss running time

### Exercise Sheet #1 Solutions, Computer Science, A M. Hallett, K. Smith a k n b k = n b + k. a k n b k c 1 n b k. a k n b.

Exercise Sheet #1 Solutions, Computer Science, 308-251A M. Hallett, K. Smith 2002 Question 1.1: Prove (n + a) b = Θ(n b ). Binomial Expansion: Show O(n b ): (n + a) b = For any c 1 > b = O(n b ). Show

### /633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programming II Date: 10/12/17

601.433/633 Introduction to Algorithms Lecturer: Michael Dinitz Topic: Dynamic Programming II Date: 10/12/17 12.1 Introduction Today we re going to do a couple more examples of dynamic programming. While

### 1 Computational problems

80240233: Computational Complexity Lecture 1 ITCS, Tsinghua Univesity, Fall 2007 9 October 2007 Instructor: Andrej Bogdanov Notes by: Andrej Bogdanov The aim of computational complexity theory is to study

### Chapter 2: The Basics. slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens

Chapter 2: The Basics slides 2017, David Doty ECS 220: Theory of Computation based on The Nature of Computation by Moore and Mertens Problem instances vs. decision problems vs. search problems Decision

### CS 6820 Fall 2014 Lectures, October 3-20, 2014

Analysis of Algorithms Linear Programming Notes CS 6820 Fall 2014 Lectures, October 3-20, 2014 1 Linear programming The linear programming (LP) problem is the following optimization problem. We are given

### AQA Level 2 Further mathematics Further algebra. Section 4: Proof and sequences

AQA Level 2 Further mathematics Further algebra Section 4: Proof and sequences Notes and Examples These notes contain subsections on Algebraic proof Sequences The limit of a sequence Algebraic proof Proof

### k-protected VERTICES IN BINARY SEARCH TREES

k-protected VERTICES IN BINARY SEARCH TREES MIKLÓS BÓNA Abstract. We show that for every k, the probability that a randomly selected vertex of a random binary search tree on n nodes is at distance k from

### Lecture 12: Mapping Reductions

Lecture 12: Mapping Reductions October 18, 2016 CS 1010 Theory of Computation Topics Covered 1. The Language EQ T M 2. Mapping Reducibility 3. The Post Correspondence Problem 1 The Language EQ T M The

### CISC 876: Kolmogorov Complexity

March 27, 2007 Outline 1 Introduction 2 Definition Incompressibility and Randomness 3 Prefix Complexity Resource-Bounded K-Complexity 4 Incompressibility Method Gödel s Incompleteness Theorem 5 Outline

### Time Complexity. Definition. Let t : n n be a function. NTIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM}

Time Complexity Definition Let t : n n be a function. TIME(t(n)) = {L L is a language decidable by a O(t(n)) deterministic TM} NTIME(t(n)) = {L L is a language decidable by a O(t(n)) non-deterministic

### Lecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009

CS 373: Theory of Computation Sariel Har-Peled and Madhusudan Parthasarathy Lecture 23: Rice Theorem and Turing machine behavior properties 21 April 2009 This lecture covers Rice s theorem, as well as

### Running Time Evaluation

Running Time Evaluation Quadratic Vs. Linear Time Lecturer: Georgy Gimel farb COMPSCI 220 Algorithms and Data Structures 1 / 19 1 Running time 2 Examples 3 Big-Oh, Big-Omega, and Big-Theta Tools 4 Time

### Problem Set 1 Solutions

Introduction to Algorithms September 24, 2004 Massachusetts Institute of Technology 6.046J/18.410J Professors Piotr Indyk and Charles E. Leiserson Handout 7 Problem Set 1 Solutions Exercise 1-1. Do Exercise

### The space complexity of a standard Turing machine. The space complexity of a nondeterministic Turing machine

298 8. Space Complexity The space complexity of a standard Turing machine M = (Q,,,, q 0, accept, reject) on input w is space M (w) = max{ uav : q 0 w M u q av, q Q, u, a, v * } The space complexity of

### i=1 i B[i] B[i] + A[i, j]; c n for j n downto i + 1 do c n i=1 (n i) C[i] C[i] + A[i, j]; c n

Fundamental Algorithms Homework #1 Set on June 25, 2009 Due on July 2, 2009 Problem 1. [15 pts] Analyze the worst-case time complexity of the following algorithms,and give tight bounds using the Theta

### Kolmogorov complexity and its applications

CS860, Winter, 2010 Kolmogorov complexity and its applications Ming Li School of Computer Science University of Waterloo http://www.cs.uwaterloo.ca/~mli/cs860.html We live in an information society. Information

### Kolmogorov complexity and its applications

Spring, 2009 Kolmogorov complexity and its applications Paul Vitanyi Computer Science University of Amsterdam http://www.cwi.nl/~paulv/course-kc We live in an information society. Information science is

### Decidability and Undecidability

Decidability and Undecidability Major Ideas from Last Time Every TM can be converted into a string representation of itself. The encoding of M is denoted M. The universal Turing machine U TM accepts an

### Lecture 2. 1 More N P-Compete Languages. Notes on Complexity Theory: Fall 2005 Last updated: September, Jonathan Katz

Notes on Complexity Theory: Fall 2005 Last updated: September, 2005 Jonathan Katz Lecture 2 1 More N P-Compete Languages It will be nice to find more natural N P-complete languages. To that end, we ine

### Introduction to Turing Machines. Reading: Chapters 8 & 9

Introduction to Turing Machines Reading: Chapters 8 & 9 1 Turing Machines (TM) Generalize the class of CFLs: Recursively Enumerable Languages Recursive Languages Context-Free Languages Regular Languages

### Topic 17. Analysis of Algorithms

Topic 17 Analysis of Algorithms Analysis of Algorithms- Review Efficiency of an algorithm can be measured in terms of : Time complexity: a measure of the amount of time required to execute an algorithm

### ECS 120 Lesson 15 Turing Machines, Pt. 1

ECS 120 Lesson 15 Turing Machines, Pt. 1 Oliver Kreylos Wednesday, May 2nd, 2001 Before we can start investigating the really interesting problems in theoretical computer science, we have to introduce

### CSCI3390-Assignment 2 Solutions

CSCI3390-Assignment 2 Solutions due February 3, 2016 1 TMs for Deciding Languages Write the specification of a Turing machine recognizing one of the following three languages. Do one of these problems.

### Min/Max-Poly Weighting Schemes and the NL vs UL Problem

Min/Max-Poly Weighting Schemes and the NL vs UL Problem Anant Dhayal Jayalal Sarma Saurabh Sawlani May 3, 2016 Abstract For a graph G(V, E) ( V = n) and a vertex s V, a weighting scheme (w : E N) is called

### An Optimal Lower Bound for Nonregular Languages

An Optimal Lower Bound for Nonregular Languages Alberto Bertoni Carlo Mereghetti Giovanni Pighizzini Dipartimento di Scienze dell Informazione Università degli Studi di Milano via Comelico, 39 2035 Milano

### In other words, we are interested in what is happening to the y values as we get really large x values and as we get really small x values.

Polynomial functions: End behavior Solutions NAME: In this lab, we are looking at the end behavior of polynomial graphs, i.e. what is happening to the y values at the (left and right) ends of the graph.

### 6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, Class 10 Nancy Lynch

6.045: Automata, Computability, and Complexity Or, Great Ideas in Theoretical Computer Science Spring, 2010 Class 10 Nancy Lynch Today Final topic in computability theory: Self-Reference and the Recursion

### Cpt S 223. School of EECS, WSU

Algorithm Analysis 1 Purpose Why bother analyzing code; isn t getting it to work enough? Estimate time and memory in the average case and worst case Identify bottlenecks, i.e., where to reduce time Compare

### COS597D: Information Theory in Computer Science October 19, Lecture 10

COS597D: Information Theory in Computer Science October 9, 20 Lecture 0 Lecturer: Mark Braverman Scribe: Andrej Risteski Kolmogorov Complexity In the previous lectures, we became acquainted with the concept

### Busch Complexity Lectures: Turing s Thesis. Costas Busch - LSU 1

Busch Complexity Lectures: Turing s Thesis Costas Busch - LSU 1 Turing s thesis (1930): Any computation carried out by mechanical means can be performed by a Turing Machine Costas Busch - LSU 2 Algorithm:

### Stochasticity in Algorithmic Statistics for Polynomial Time

Stochasticity in Algorithmic Statistics for Polynomial Time Alexey Milovanov 1 and Nikolay Vereshchagin 2 1 National Research University Higher School of Economics, Moscow, Russia almas239@gmail.com 2

### CS 360, Winter Morphology of Proof: An introduction to rigorous proof techniques

CS 30, Winter 2011 Morphology of Proof: An introduction to rigorous proof techniques 1 Methodology of Proof An example Deep down, all theorems are of the form If A then B, though they may be expressed

### Turing Machines Part II

Turing Machines Part II Problem Set Set Five Five due due in in the the box box up up front front using using a late late day. day. Hello Hello Condensed Slide Slide Readers! Readers! This This lecture

### means is a subset of. So we say A B for sets A and B if x A we have x B holds. BY CONTRAST, a S means that a is a member of S.

1 Notation For those unfamiliar, we have := means equal by definition, N := {0, 1,... } or {1, 2,... } depending on context. (i.e. N is the set or collection of counting numbers.) In addition, means for

### CSE 105 Theory of Computation

CSE 105 Theory of Computation http://www.jflap.org/jflaptmp/ Professor Jeanne Ferrante 1 Undecidability Today s Agenda Review and More Problems A Non-TR Language Reminders and announcements: HW 7 (Last!!)

### Nondeterministic Finite Automata

Nondeterministic Finite Automata Mahesh Viswanathan Introducing Nondeterminism Consider the machine shown in Figure. Like a DFA it has finitely many states and transitions labeled by symbols from an input

### Data Structures and Algorithms Chapter 2

1 Data Structures and Algorithms Chapter 2 Werner Nutt 2 Acknowledgments The course follows the book Introduction to Algorithms, by Cormen, Leiserson, Rivest and Stein, MIT Press [CLRST]. Many examples

### Turing Machines Part III

Turing Machines Part III Announcements Problem Set 6 due now. Problem Set 7 out, due Monday, March 4. Play around with Turing machines, their powers, and their limits. Some problems require Wednesday's

### DRAFT. Algebraic computation models. Chapter 14

Chapter 14 Algebraic computation models Somewhat rough We think of numerical algorithms root-finding, gaussian elimination etc. as operating over R or C, even though the underlying representation of the

### Lecture 22: PSPACE

6.045 Lecture 22: PSPACE 1 VOTE VOTE VOTE For your favorite course on automata and complexity Please complete the online subject evaluation for 6.045 2 Final Exam Information Who: You On What: Everything

### Definition: conp = { L L NP } What does a conp computation look like?

Space Complexity 28 Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode: Guess string y of x k length and the machine accepts

### Complexity and NP-completeness

Lecture 17 Complexity and NP-completeness Supplemental reading in CLRS: Chapter 34 As an engineer or computer scientist, it is important not only to be able to solve problems, but also to know which problems

### Turing Machines Part II

Turing Machines Part II Hello Hello Condensed Slide Slide Readers! Readers! This This lecture lecture is is almost almost entirely entirely animations that that show show how how each each Turing Turing

### We introduce one more operation on sets, perhaps the most important

11. The power set Please accept my resignation. I don t want to belong to any club that will accept me as a member. Groucho Marx We introduce one more operation on sets, perhaps the most important one:

### Analysis of Algorithms

October 1, 2015 Analysis of Algorithms CS 141, Fall 2015 1 Analysis of Algorithms: Issues Correctness/Optimality Running time ( time complexity ) Memory requirements ( space complexity ) Power I/O utilization

### Advanced Topics in Theoretical Computer Science

Advanced Topics in Theoretical Computer Science Part 5: Complexity (Part II) 30.01.2014 Viorica Sofronie-Stokkermans Universität Koblenz-Landau e-mail: sofronie@uni-koblenz.de 1 Contents Recall: Turing

### Laver Tables A Direct Approach

Laver Tables A Direct Approach Aurel Tell Adler June 6, 016 Contents 1 Introduction 3 Introduction to Laver Tables 4.1 Basic Definitions............................... 4. Simple Facts.................................

### Omega notation. Transitivity etc.

Omega notation Big-Omega: Lecture 2, Sept. 25, 2014 f () n (()) g n const cn, s.t. n n : cg() n f () n Small-omega: 0 0 0 f () n (()) g n const c, n s.t. n n : cg() n f () n 0 0 0 Intuition (works most

### Lecture 17. In this lecture, we will continue our discussion on randomization.

ITCS:CCT09 : Computational Complexity Theory May 11, 2009 Lecturer: Jayalal Sarma M.N. Lecture 17 Scribe: Hao Song In this lecture, we will continue our discussion on randomization. 1 BPP and the Polynomial

### PRIMES Math Problem Set

PRIMES Math Problem Set PRIMES 017 Due December 1, 01 Dear PRIMES applicant: This is the PRIMES 017 Math Problem Set. Please send us your solutions as part of your PRIMES application by December 1, 01.

### Compilers. Lexical analysis. Yannis Smaragdakis, U. Athens (original slides by Sam

Compilers Lecture 3 Lexical analysis Yannis Smaragdakis, U. Athens (original slides by Sam Guyer@Tufts) Big picture Source code Front End IR Back End Machine code Errors Front end responsibilities Check

### Analysis of Algorithms

September 29, 2017 Analysis of Algorithms CS 141, Fall 2017 1 Analysis of Algorithms: Issues Correctness/Optimality Running time ( time complexity ) Memory requirements ( space complexity ) Power I/O utilization

### 1 Recommended Reading 1. 2 Public Key/Private Key Cryptography Overview RSA Algorithm... 2

Contents 1 Recommended Reading 1 2 Public Key/Private Key Cryptography 1 2.1 Overview............................................. 1 2.2 RSA Algorithm.......................................... 2 3 A Number