Complexity 6: AIT. Outline. Dusko Pavlovic. Kolmogorov. Solomonoff. Chaitin: The number of wisdom RHUL Spring Complexity 6: AIT.
|
|
- Lilian Lang
- 5 years ago
- Views:
Transcription
1 Outline Complexity Theory Part 6: did we achieve? Algorithmic information and logical depth : Algorithmic information : Algorithmic probability : The number of wisdom RHUL Spring 2012 : Logical depth Outline did we achieve? did we achieve? : Algorithmic information : Algorithmic probability : The number of wisdom : Logical depth did we achieve? Why are there complex phenomena? The question was: Where does the complexity come from?
2 Why are there complex phenomena? Why are there complex phenomena? Why does the evolution increase complexity? Why are we so complex? is complexity? Which strings are complex? Did complexity theory bring us closer to an answer? digits digits digits is complexity? is complexity? How do you know which strings are complex? (01) digits digits digits
3 is complexity? is complexity? (01) (01) 50 do i=1..50 write 01 od do i=1..50 write 01 od {{ 2 0 i 1 i i=1; do until length= write 0 i 1 i ;i=i+1od is complexity? Complexity = length of description (01) 50 do i=1..50 write 01 od Idea {{ 2 0 i 1 i i=1; do until length= write 0 i 1 i ;i=i+1od Fix a language and define for any x C(x) = length of the simplest description x print Complexity = length of description is a description? C( ) < C(010...) < C( )
4 is a description? is a description? β = x β = x The length of the expression " x" is< 20 is a description? Berry Paradox β = x The length of the expression " x" is< 20 Therefore β does not exist. β = x The length of the expression " Therefore β does not exist. x" is< 20 Therefore all x satisfy C(x) 20 Berry Paradox Moral We need a formal definition of a description. β = x The length of the expression " x" is< 20 Therefore β does not exist. Therefore all x satisfy C(x) 20 " x" is not a valid description
5 Outline Idea: descriptions = programs did we achieve? : Algorithmic information M { N : Algorithmic probability ( ) u c : The number of wisdom N N N N : Logical depth Assume programs with a "length" measure Assume programs with a "length" measure M { N l M { N l ( ) u c ( ) u c N N N N N N N N l(p) =acode length of p l(p) =acode length of p (e.g. the number of symbols in p) Complexity = length of program Escape from Berry Paradox Idea complexity of x is the length of the simplest program that outputs x Fact The predicate K (x) y K (x) = {p()=x l(p) is not decidable, or else β = K (x)>20 x would be a program.
6 Consequence Definition Komogorov complexity does not satisfy the axioms of complexity measures. For a given 2-tape Turing Machine M, them-description distance from a string x to a string y is K M (y x) = l(p) M(p,x)=y The machine M is called the interpreter. Definition For a given 2-tape Turing Machine M, them-description distance from a string x to a string y is 1 Notation For any pair of functions f, g : N N we write K M (y x) = l(p) M(p,x)=y f + g c n. f (n) g(n)+c f + = g f + g g + f The machine M is called the interpreter. 1 The definition of infimum implies =. Proof. If U is universal machine for all 2-tape machines, i.e. Proposition 1 (The Invariance Theorem) U ( M, x, y) = M(x, y) There is a universal interpreter: a 2-tape Turing Machine Usuchthatforall2-tapeTuringMachinesMholds then define U (J(x, y), z) = U(x, y, z) K U (y x) + K M (y x) where J is Cantor s pairing function, so that U (J( M, p), x) = M(p, x) and thus K U (y x) K M (y x)+l ( M ) + 6
7 Scholium For any two universal interpreters U, V holds K U (y x) + = K V (y x) Definition The description distance from a string x to a string y (or complexity of y relative to x) isthe description distance with respect to a universal interpreter K (y x) = l(p) {p(x)=y Properties of description complexity Definition The description complexity (or complexity) of astringy is the description distance from the empty string to y K (y) = l(p) {p()=y #{x {0, 1 l K (x) < l m 2 l m m(x) K (x) l(x) for m(x) = x y K (y) lim x m(x) = φ RPF. lim x φ(x) = m(x) < φ(x) χ RTF. lim t χ(t, x) =K (x) h. K (x + h) K (x) < 2h Compression and randomness Compression and randomness Question Temperature is the average of kinetic energies. is the entropy average of? Theorem (Schack 1997) complexity can be defined with optimal encoding and then H(q) K (q i ) i I
8 Compression and randomness Compression and randomness Upshot Upshot The most informative strings cannot be compressed: The most informative strings cannot be compressed: K (x) = l(x) K (x) = l(x) This is s definition of randomness. Outline Entropy encoding: more likely shorter did we achieve? Shannon s Noiseless Coding : Algorithmic information Let µ : Σ [0, 1] be the frequency distribution of the alphabet Σ. Thentheoptimalwordlengthfor representing a symbol s Σ is : Algorithmic probability l(s) = log µ(s) : The number of wisdom : Logical depth Entropy encoding: more likely shorter Occam s Razor: shorter more likely Shannon s Noiseless Coding s Algorithmic Probability Let µ : Σ [0, 1] be the frequency distribution of the alphabet Σ. Thentheoptimalwordlengthfor representing a symbol s Σ is Let U be a self-delimiting universal Turing machine, i.e. U(p) = q. U(p :: q) l(s) = log µ(s) Then the aprioriprobability distribution of data from a dataset Σ is Remark µ(y) = U(p)=y 2 l(p) The Shannon entropy is the average word length.
9 Occam s Razor: shorter more likely Occam s Razor: shorter more likely s Algorithmic Probability s Algorithmic Probability Let U be a self-delimiting universal Turing machine, i.e. Let U be a self-delimiting universal Turing machine, i.e. U(p) = q. U(p :: q) U(p, x) = q. U(p :: q, x) Then the aprioriprobability distribution of data from a dataset Σ is Then the aprioriconditional probability distribution of data from a dataset Σ is µ(y) = 2 l(p) µ(y x) = 2 l(p) U(p)=y U(p,x)=y Remark Remark Self-delimiting = y Σ µ(y) 1. Self-delimiting = y Σ µ(y) 1. Inductive inference Inductive inference s model of theory formation Inferring the most effective explanations phenomena are modeled as bitstrings d, h... {0, 1 The goal is to maximize d observeddata h hypotheticdata Pr(h d) = Pr(d h) Pr(h) Pr(d) {p (d) =h explanationofcausality Pr(h d) = is the best {p(d)=h 2 l(p) thesimplestexplanation Inductive inference Inductive inference Inferring the most effective explanations The goal is to maximize Pr(h d) = Pr(d h) Pr(h) Pr(d) Minimum Description Length Principle (MDLP) Given an observation d, thebestexplanationminimizes l(p)+l(q) Since Pr(x) =2 K (x),thisisequivalenttominimizing where {p () = h {q (h) =d K (h d) = K (d h)+k (h) K (d)
10 Outline did we achieve? The halting number : Algorithmic information : Algorithmic probability κ =.κ 1 κ 2 κ 3 κ p 1 if U(p) κ p = 0 otherwise : The number of wisdom : Logical depth The halting number κ =.κ 1 κ 2 κ 3 κ p 1 if U(p) κ p = 0 otherwise Fact κ is highly compressible! Remark The value of κ depend on a fixed Universal Turing Machine U. Its crucial properties do not depend on the choice of U. Definition K t M (y) = {p()=y time(p,y) t( y ) l(p) Proposition (Barzdin) Suppose that Y = {i N y i = 1 is recursively enumerable. Then for any c > 0thereisatotalrecursive function t such that K t M (y) c log y
11 Proposition 2 Question For every n there is a program of length 2lognthat outputs the first n digits of κ. might this compressed κ look like? s number s number Ω = µ(s) = s N U(p) 2 l(p) Ω = µ(s) = s N U(p) 2 l(p) Interpretations probability that a hypothesis will be formed a priori probability that a randomly chosen program will halt Proposition 3 Proposition 4 Ω is incompressible. Ω 1..n decides halting of all programs of length up to n.
12 Interpretation Interpretation Many open problems can be formulated as the questions whether certain search programs will halt: Riemann Hypothesis P = NP, AP = ANP, one-way,trapdoor,ddh... Twin Primes... Many open problems can be formulated as the questions whether certain search programs will halt: Riemann Hypothesis P = NP, AP = ANP, one-way,trapdoor,ddh... Twin Primes... These programs are shorter than 5000 characters. Knowing Ω would resolve most of the open problems of mathematics. Outline Time bounded algorithmic probability did we achieve? : Algorithmic information Definition : Algorithmic probability : The number of wisdom µ t (y) = {p()=y time(p,y) t( y ) 2 l(p) : Logical depth Logical depth Logical depth Definition (, Adleman... ) Definition (, Adleman... ) L ɛ (y) = { t REC µ t (y) µ(y) ɛ L ɛ (y) = { t REC µ t (y) µ(y) ɛ Interpretation The shortest time in which that one of the minimal programs that output y will halt, with the conditional probability ɛ.
13 Logical depth Logical depth Proposition 5 (Adleman) Proposition 6 () The predicate L ɛ (y) O(n k ) is decidable in polynomial time. Deep strings cannot be quickly computed from shallow ones. Logical depth Logical mutual information Levin s Law of Conservation of Information s interpretation Let "A structure is deep if most of its algorithmic probability is contributed by slow-running processes." d be a stream of randomized observations, and h astreamgeneratedbyadeterministic mathematical model. Then the probability that the mutual information is I(D : H) > m is less than 2 m. Logical depth Logical depth Levin s comment Levin s interpretation "Following Church s Thesis, Theorem 3 precludes the increase of information through computational processes. Theorem 2 precludes the increase of information through acombinationofcomputationalandrandomized processes." "Our results contradict the assertion of some mathematicians that the truth of any valid proposition can be verified in the course of scientific progress through informal methods. (To do so by formal methods has been proven impossible by Gödel.)"
CISC 876: Kolmogorov Complexity
March 27, 2007 Outline 1 Introduction 2 Definition Incompressibility and Randomness 3 Prefix Complexity Resource-Bounded K-Complexity 4 Incompressibility Method Gödel s Incompleteness Theorem 5 Outline
More informationCS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression
CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression Rosencrantz & Guildenstern Are Dead (Tom Stoppard) Rigged Lottery? And the winning numbers are: 1, 2, 3, 4, 5, 6 But is
More informationComplexity 1: Motivation. Outline. Dusko Pavlovic. Introductions. Algorithms. Introductions. Complexity 1: Motivation. Contact.
Outline Complexity Theory Part 1: Defining complexity Complexity of algorithms RHUL Spring 2012 Outline Contact Defining complexity email: dusko.pavlovic@rhul.ac.uk phone: (01784 44 30 81) Complexity of
More informationLecture 13: Foundations of Math and Kolmogorov Complexity
6.045 Lecture 13: Foundations of Math and Kolmogorov Complexity 1 Self-Reference and the Recursion Theorem 2 Lemma: There is a computable function q : Σ* Σ* such that for every string w, q(w) is the description
More informationUnderstanding Computation
Understanding Computation 1 Mathematics & Computation -Mathematics has been around for a long time as a method of computing. -Efforts to find canonical way of computations. - Machines have helped with
More information2 Plain Kolmogorov Complexity
2 Plain Kolmogorov Complexity In this section, we introduce plain Kolmogorov Complexity, prove the invariance theorem - that is, the complexity of a string does not depend crucially on the particular model
More informationPREDICATE LOGIC: UNDECIDABILITY AND INCOMPLETENESS HUTH AND RYAN 2.5, SUPPLEMENTARY NOTES 2
PREDICATE LOGIC: UNDECIDABILITY AND INCOMPLETENESS HUTH AND RYAN 2.5, SUPPLEMENTARY NOTES 2 Neil D. Jones DIKU 2005 14 September, 2005 Some slides today new, some based on logic 2004 (Nils Andersen) OUTLINE,
More informationComputability Theory. CS215, Lecture 6,
Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David
More informationTuring s thesis: (1930) Any computation carried out by mechanical means can be performed by a Turing Machine
Turing s thesis: (1930) Any computation carried out by mechanical means can be performed by a Turing Machine There is no known model of computation more powerful than Turing Machines Definition of Algorithm:
More informationCS21 Decidability and Tractability
CS21 Decidability and Tractability Lecture 14 February 7, 2018 February 7, 2018 CS21 Lecture 14 1 Outline Gödel Incompleteness Theorem February 7, 2018 CS21 Lecture 14 2 Background Hilbert s program (1920
More informationDecidability: Church-Turing Thesis
Decidability: Church-Turing Thesis While there are a countably infinite number of languages that are described by TMs over some alphabet Σ, there are an uncountably infinite number that are not Are there
More informationKolmogorov complexity and its applications
Spring, 2009 Kolmogorov complexity and its applications Paul Vitanyi Computer Science University of Amsterdam http://www.cwi.nl/~paulv/course-kc We live in an information society. Information science is
More informationTheory of Computation Lecture Notes. Problems and Algorithms. Class Information
Theory of Computation Lecture Notes Prof. Yuh-Dauh Lyuu Dept. Computer Science & Information Engineering and Department of Finance National Taiwan University Problems and Algorithms c 2004 Prof. Yuh-Dauh
More informationLimits of Computability
Limits of Computability Wolfgang Schreiner Wolfgang.Schreiner@risc.jku.at Research Institute for Symbolic Computation (RISC) Johannes Kepler University, Linz, Austria http://www.risc.jku.at Wolfgang Schreiner
More informationKolmogorov complexity
Kolmogorov complexity In this section we study how we can define the amount of information in a bitstring. Consider the following strings: 00000000000000000000000000000000000 0000000000000000000000000000000000000000
More informationAlgorithmic Information Theory
Algorithmic Information Theory Peter D. Grünwald CWI, P.O. Box 94079 NL-1090 GB Amsterdam, The Netherlands E-mail: pdg@cwi.nl Paul M.B. Vitányi CWI, P.O. Box 94079 NL-1090 GB Amsterdam The Netherlands
More informationPROOFS IN PREDICATE LOGIC AND COMPLETENESS; WHAT DECIDABILITY MEANS HUTH AND RYAN 2.3, SUPPLEMENTARY NOTES 2
PROOFS IN PREDICATE LOGIC AND COMPLETENESS; WHAT DECIDABILITY MEANS HUTH AND RYAN 2.3, SUPPLEMENTARY NOTES 2 Neil D. Jones DIKU 2005 12 September, 2005 Some slides today new, some based on logic 2004 (Nils
More informationFORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY KOLMOGOROV-CHAITIN (descriptive) COMPLEXITY TUESDAY, MAR 18 CAN WE QUANTIFY HOW MUCH INFORMATION IS IN A STRING? A = 01010101010101010101010101010101
More informationIntroduction to Turing Machines
Introduction to Turing Machines Deepak D Souza Department of Computer Science and Automation Indian Institute of Science, Bangalore. 12 November 2015 Outline 1 Turing Machines 2 Formal definitions 3 Computability
More informationLarge Numbers, Busy Beavers, Noncomputability and Incompleteness
Large Numbers, Busy Beavers, Noncomputability and Incompleteness Food For Thought November 1, 2007 Sam Buss Department of Mathematics U.C. San Diego PART I Large Numbers, Busy Beavers, and Undecidability
More informationInformal Statement Calculus
FOUNDATIONS OF MATHEMATICS Branches of Logic 1. Theory of Computations (i.e. Recursion Theory). 2. Proof Theory. 3. Model Theory. 4. Set Theory. Informal Statement Calculus STATEMENTS AND CONNECTIVES Example
More informationMost General computer?
Turing Machines Most General computer? DFAs are simple model of computation. Accept only the regular languages. Is there a kind of computer that can accept any language, or compute any function? Recall
More informationTheory of Computation Lecture Notes
Theory of Computation Lecture Notes Prof. Yuh-Dauh Lyuu Dept. Computer Science & Information Engineering and Department of Finance National Taiwan University c 2008 Prof. Yuh-Dauh Lyuu, National Taiwan
More informationFORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY THURSDAY APRIL 3 REVIEW for Midterm TUESDAY April 8 Definition: A Turing Machine is a 7-tuple T = (Q, Σ, Γ, δ, q, q accept, q reject ), where: Q is a
More informationCompression Complexity
Compression Complexity Stephen Fenner University of South Carolina Lance Fortnow Georgia Institute of Technology February 15, 2017 Abstract The Kolmogorov complexity of x, denoted C(x), is the length of
More information1 Showing Recognizability
CSCC63 Worksheet Recognizability and Decidability 1 1 Showing Recognizability 1.1 An Example - take 1 Let Σ be an alphabet. L = { M M is a T M and L(M) }, i.e., that M accepts some string from Σ. Prove
More informationKolmogorov complexity and its applications
CS860, Winter, 2010 Kolmogorov complexity and its applications Ming Li School of Computer Science University of Waterloo http://www.cs.uwaterloo.ca/~mli/cs860.html We live in an information society. Information
More informationDecidability. Linz 6 th, Chapter 12: Limits of Algorithmic Computation, page 309ff
Decidability Linz 6 th, Chapter 12: Limits of Algorithmic Computation, page 309ff 1 A property P of strings is said to be decidable if the set of all strings having property P is a recursive set; that
More informationComplex Systems Methods 2. Conditional mutual information, entropy rate and algorithmic complexity
Complex Systems Methods 2. Conditional mutual information, entropy rate and algorithmic complexity Eckehard Olbrich MPI MiS Leipzig Potsdam WS 2007/08 Olbrich (Leipzig) 26.10.2007 1 / 18 Overview 1 Summary
More informationTuring Machines A Turing Machine is a 7-tuple, (Q, Σ, Γ, δ, q0, qaccept, qreject), where Q, Σ, Γ are all finite
The Church-Turing Thesis CS60001: Foundations of Computing Science Professor, Dept. of Computer Sc. & Engg., Turing Machines A Turing Machine is a 7-tuple, (Q, Σ, Γ, δ, q 0, q accept, q reject ), where
More informationKolmogorov Complexity
Kolmogorov Complexity Davide Basilio Bartolini University of Illinois at Chicago Politecnico di Milano dbarto3@uic.edu davide.bartolini@mail.polimi.it 1 Abstract What follows is a survey of the field of
More informationFinish K-Complexity, Start Time Complexity
6.045 Finish K-Complexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts
More information23.1 Gödel Numberings and Diagonalization
Applied Logic Lecture 23: Unsolvable Problems in Logic CS 4860 Spring 2009 Tuesday, April 14, 2009 The fact that Peano Arithmetic is expressive enough to represent all computable functions means that some
More informationComputer Sciences Department
Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y
More informationFinal Exam Comments. UVa - cs302: Theory of Computation Spring < Total
UVa - cs302: Theory of Computation Spring 2008 Final Exam Comments < 50 50 59 60 69 70 79 80 89 90 94 95-102 Total 2 6 8 22 16 16 12 Problem 1: Short Answers. (20) For each question, provide a correct,
More informationModels. Models of Computation, Turing Machines, and the Limits of Turing Computation. Effective Calculability. Motivation for Models of Computation
Turing Computation /0/ Models of Computation, Turing Machines, and the Limits of Turing Computation Bruce MacLennan Models A model is a tool intended to address a class of questions about some domain of
More informationOpleiding Informatica
Opleiding Informatica Tape-quantifying Turing machines in the arithmetical hierarchy Simon Heijungs Supervisors: H.J. Hoogeboom & R. van Vliet BACHELOR THESIS Leiden Institute of Advanced Computer Science
More informationPart I: Definitions and Properties
Turing Machines Part I: Definitions and Properties Finite State Automata Deterministic Automata (DFSA) M = {Q, Σ, δ, q 0, F} -- Σ = Symbols -- Q = States -- q 0 = Initial State -- F = Accepting States
More informationCSCE 551: Chin-Tser Huang. University of South Carolina
CSCE 551: Theory of Computation Chin-Tser Huang huangct@cse.sc.edu University of South Carolina Church-Turing Thesis The definition of the algorithm came in the 1936 papers of Alonzo Church h and Alan
More informationCS294: Pseudorandomness and Combinatorial Constructions September 13, Notes for Lecture 5
UC Berkeley Handout N5 CS94: Pseudorandomness and Combinatorial Constructions September 3, 005 Professor Luca Trevisan Scribe: Gatis Midrijanis Notes for Lecture 5 In the few lectures we are going to look
More informationInstitute for Applied Information Processing and Communications (IAIK) Secure & Correct Systems. Decidability
Decidability and the Undecidability of Predicate Logic IAIK Graz University of Technology georg.hofferek@iaik.tugraz.at 1 Fork of ways Brainteaser: Labyrinth Guards One to salvation One to perdition Two
More informationIs there an Elegant Universal Theory of Prediction?
Is there an Elegant Universal Theory of Prediction? Shane Legg Dalle Molle Institute for Artificial Intelligence Manno-Lugano Switzerland 17th International Conference on Algorithmic Learning Theory Is
More informationPeano Arithmetic. CSC 438F/2404F Notes (S. Cook) Fall, Goals Now
CSC 438F/2404F Notes (S. Cook) Fall, 2008 Peano Arithmetic Goals Now 1) We will introduce a standard set of axioms for the language L A. The theory generated by these axioms is denoted PA and called Peano
More information6-1 Computational Complexity
6-1 Computational Complexity 6. Computational Complexity Computational models Turing Machines Time complexity Non-determinism, witnesses, and short proofs. Complexity classes: P, NP, conp Polynomial-time
More informationComplexity Theory Part I
Complexity Theory Part I Problem Problem Set Set 77 due due right right now now using using a late late period period The Limits of Computability EQ TM EQ TM co-re R RE L D ADD L D HALT A TM HALT A TM
More informationGreat Theoretical Ideas
15-251 Great Theoretical Ideas in Computer Science Gödel s Legacy: Proofs and Their Limitations Lecture 25 (November 16, 2010) The Halting Problem A Quick Recap of the Previous Lecture Is there a program
More informationComputability and Complexity
Computability and Complexity Lecture 5 Reductions Undecidable problems from language theory Linear bounded automata given by Jiri Srba Lecture 5 Computability and Complexity 1/14 Reduction Informal Definition
More informationHilbert s problems, Gödel, and the limits of computation
Hilbert s problems, Gödel, and the limits of computation Logan Axon Gonzaga University April 6, 2011 Hilbert at the ICM At the 1900 International Congress of Mathematicians in Paris, David Hilbert gave
More informationDecidability: Reduction Proofs
Decidability: Reduction Proofs Basic technique for proving a language is (semi)decidable is reduction Based on the following principle: Have problem A that needs to be solved If there exists a problem
More informationTURING MAHINES
15-453 TURING MAHINES TURING MACHINE FINITE STATE q 10 CONTROL AI N P U T INFINITE TAPE read write move 0 0, R, R q accept, R q reject 0 0, R 0 0, R, L read write move 0 0, R, R q accept, R 0 0, R 0 0,
More informationHilbert s problems, Gödel, and the limits of computation
Hilbert s problems, Gödel, and the limits of computation Logan Axon Gonzaga University November 14, 2013 Hilbert at the ICM At the 1900 International Congress of Mathematicians in Paris, David Hilbert
More information3 Self-Delimiting Kolmogorov complexity
3 Self-Delimiting Kolmogorov complexity 3. Prefix codes A set is called a prefix-free set (or a prefix set) if no string in the set is the proper prefix of another string in it. A prefix set cannot therefore
More informationTheory of Computation
Theory of Computation Lecture #6 Sarmad Abbasi Virtual University Sarmad Abbasi (Virtual University) Theory of Computation 1 / 39 Lecture 6: Overview Prove the equivalence of enumerators and TMs. Dovetailing
More informationVictoria Gitman and Thomas Johnstone. New York City College of Technology, CUNY
Gödel s Proof Victoria Gitman and Thomas Johnstone New York City College of Technology, CUNY vgitman@nylogic.org http://websupport1.citytech.cuny.edu/faculty/vgitman tjohnstone@citytech.cuny.edu March
More informationGödel s Incompleteness Theorem. Overview. Computability and Logic
Gödel s Incompleteness Theorem Overview Computability and Logic Recap Remember what we set out to do in this course: Trying to find a systematic method (algorithm, procedure) which we can use to decide,
More informationCOS597D: Information Theory in Computer Science October 19, Lecture 10
COS597D: Information Theory in Computer Science October 9, 20 Lecture 0 Lecturer: Mark Braverman Scribe: Andrej Risteski Kolmogorov Complexity In the previous lectures, we became acquainted with the concept
More informationChapter 2 Algorithms and Computation
Chapter 2 Algorithms and Computation In this chapter, we first discuss the principles of algorithm and computation in general framework, common both in classical and quantum computers, then we go to the
More informationHandbook of Logic and Proof Techniques for Computer Science
Steven G. Krantz Handbook of Logic and Proof Techniques for Computer Science With 16 Figures BIRKHAUSER SPRINGER BOSTON * NEW YORK Preface xvii 1 Notation and First-Order Logic 1 1.1 The Use of Connectives
More informationDecision Problems with TM s. Lecture 31: Halting Problem. Universe of discourse. Semi-decidable. Look at following sets: CSCI 81 Spring, 2012
Decision Problems with TM s Look at following sets: Lecture 31: Halting Problem CSCI 81 Spring, 2012 Kim Bruce A TM = { M,w M is a TM and w L(M)} H TM = { M,w M is a TM which halts on input w} TOTAL TM
More informationExample. Lemma. Proof Sketch. 1 let A be a formula that expresses that node t is reachable from s
Summary Summary Last Lecture Computational Logic Π 1 Γ, x : σ M : τ Γ λxm : σ τ Γ (λxm)n : τ Π 2 Γ N : τ = Π 1 [x\π 2 ] Γ M[x := N] Georg Moser Institute of Computer Science @ UIBK Winter 2012 the proof
More informationPredicate Logic - Undecidability
CS402, Spring 2016 Undecidable Problems Does the following program halts? (1) N : n, total, x, y, z (2) n GetUserInput() (3) total 3 (4) while true (5) for x 1 to total 2 (6) for y 1 to total x 1 (7) z
More informationCS154, Lecture 10: Rice s Theorem, Oracle Machines
CS154, Lecture 10: Rice s Theorem, Oracle Machines Moral: Analyzing Programs is Really, Really Hard But can we more easily tell when some program analysis problem is undecidable? Problem 1 Undecidable
More informationAlgorithmic Probability
Algorithmic Probability From Scholarpedia From Scholarpedia, the free peer-reviewed encyclopedia p.19046 Curator: Marcus Hutter, Australian National University Curator: Shane Legg, Dalle Molle Institute
More informationUndecidability. Andreas Klappenecker. [based on slides by Prof. Welch]
Undecidability Andreas Klappenecker [based on slides by Prof. Welch] 1 Sources Theory of Computing, A Gentle Introduction, by E. Kinber and C. Smith, Prentice-Hall, 2001 Automata Theory, Languages and
More informationTuring Machines. Lecture 8
Turing Machines Lecture 8 1 Course Trajectory We will see algorithms, what can be done. But what cannot be done? 2 Computation Problem: To compute a function F that maps each input (a string) to an output
More informationDecidable Languages - relationship with other classes.
CSE2001, Fall 2006 1 Last time we saw some examples of decidable languages (or, solvable problems). Today we will start by looking at the relationship between the decidable languages, and the regular and
More informationTutorial on Axiomatic Set Theory. Javier R. Movellan
Tutorial on Axiomatic Set Theory Javier R. Movellan Intuitively we think of sets as collections of elements. The crucial part of this intuitive concept is that we are willing to treat sets as entities
More information10-704: Information Processing and Learning Fall Lecture 10: Oct 3
0-704: Information Processing and Learning Fall 206 Lecturer: Aarti Singh Lecture 0: Oct 3 Note: These notes are based on scribed notes from Spring5 offering of this course. LaTeX template courtesy of
More information6.080 / Great Ideas in Theoretical Computer Science Spring 2008
MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationPreface These notes were prepared on the occasion of giving a guest lecture in David Harel's class on Advanced Topics in Computability. David's reques
Two Lectures on Advanced Topics in Computability Oded Goldreich Department of Computer Science Weizmann Institute of Science Rehovot, Israel. oded@wisdom.weizmann.ac.il Spring 2002 Abstract This text consists
More informationComputable Functions
Computable Functions Part I: Non Computable Functions Computable and Partially Computable Functions Computable Function Exists a Turing Machine M -- M Halts on All Input -- M(x) = f (x) Partially Computable
More information1 Computational Problems
Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation
More informationArtificial Intelligence. 3 Problem Complexity. Prof. Dr. Jana Koehler Fall 2016 HSLU - JK
Artificial Intelligence 3 Problem Complexity Prof. Dr. Jana Koehler Fall 2016 Agenda Computability and Turing Machines Tractable and Intractable Problems P vs. NP Decision Problems Optimization problems
More informationTuring Machine Recap
Turing Machine Recap DFA with (infinite) tape. One move: read, write, move, change state. High-level Points Church-Turing thesis: TMs are the most general computing devices. So far no counter example Every
More informationIntroduction to Information Theory
Introduction to Information Theory Impressive slide presentations Radu Trîmbiţaş UBB October 2012 Radu Trîmbiţaş (UBB) Introduction to Information Theory October 2012 1 / 19 Transmission of information
More informationCS 301. Lecture 17 Church Turing thesis. Stephen Checkoway. March 19, 2018
CS 301 Lecture 17 Church Turing thesis Stephen Checkoway March 19, 2018 1 / 17 An abridged modern history of formalizing algorithms An algorithm is a finite, unambiguous sequence of steps for solving a
More informationExam Computability and Complexity
Total number of points:... Number of extra sheets of paper:... Exam Computability and Complexity by Jiri Srba, January 2009 Student s full name CPR number Study number Before you start, fill in the three
More informationCOMP/MATH 300 Topics for Spring 2017 June 5, Review and Regular Languages
COMP/MATH 300 Topics for Spring 2017 June 5, 2017 Review and Regular Languages Exam I I. Introductory and review information from Chapter 0 II. Problems and Languages A. Computable problems can be expressed
More informationCSCC63 Worksheet Turing Machines
1 An Example CSCC63 Worksheet Turing Machines Goal. Design a turing machine, M that accepts only strings of the form {w#w w {0, 1} }. Idea. Describe in words how the machine would work. Read first symbol
More informationIntroduction to Kolmogorov Complexity
Introduction to Kolmogorov Complexity Marcus Hutter Canberra, ACT, 0200, Australia http://www.hutter1.net/ ANU Marcus Hutter - 2 - Introduction to Kolmogorov Complexity Abstract In this talk I will give
More informationNP-Completeness. A language B is NP-complete iff B NP. This property means B is NP hard
NP-Completeness A language B is NP-complete iff B NP A NP A P B This property means B is NP hard 1 3SAT is NP-complete 2 Result Idea: B is known to be NP complete Use it to prove NP-Completeness of C IF
More informationLecture 11: Measuring the Complexity of Proofs
IAS/PCMI Summer Session 2000 Clay Mathematics Undergraduate Program Advanced Course on Computational Complexity Lecture 11: Measuring the Complexity of Proofs David Mix Barrington and Alexis Maciel July
More informationInformation Theory and Statistics Lecture 2: Source coding
Information Theory and Statistics Lecture 2: Source coding Łukasz Dębowski ldebowsk@ipipan.waw.pl Ph. D. Programme 2013/2014 Injections and codes Definition (injection) Function f is called an injection
More informationNotes for Lecture Notes 2
Stanford University CS254: Computational Complexity Notes 2 Luca Trevisan January 11, 2012 Notes for Lecture Notes 2 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation
More informationOn some Metatheorems about FOL
On some Metatheorems about FOL February 25, 2014 Here I sketch a number of results and their proofs as a kind of abstract of the same items that are scattered in chapters 5 and 6 in the textbook. You notice
More informationCS154, Lecture 13: P vs NP
CS154, Lecture 13: P vs NP The EXTENDED Church-Turing Thesis Everyone s Intuitive Notion of Efficient Algorithms Polynomial-Time Turing Machines More generally: TM can simulate every reasonable model of
More informationAutomata Theory. Definition. Computational Complexity Theory. Computability Theory
Outline THEORY OF COMPUTATION CS363, SJTU What is Theory of Computation? History of Computation Branches and Development Xiaofeng Gao Dept. of Computer Science Shanghai Jiao Tong University 2 The Essential
More informationAcknowledgments 2. Part 0: Overview 17
Contents Acknowledgments 2 Preface for instructors 11 Which theory course are we talking about?.... 12 The features that might make this book appealing. 13 What s in and what s out............... 14 Possible
More information6.045J/18.400J: Automata, Computability and Complexity. Quiz 2. March 30, Please write your name in the upper corner of each page.
6.045J/18.400J: Automata, Computability and Complexity March 30, 2005 Quiz 2 Prof. Nancy Lynch Please write your name in the upper corner of each page. Problem Score 1 2 3 4 5 6 Total Q2-1 Problem 1: True
More informationCSCI3390-Lecture 6: An Undecidable Problem
CSCI3390-Lecture 6: An Undecidable Problem September 21, 2018 1 Summary The language L T M recognized by the universal Turing machine is not decidable. Thus there is no algorithm that determines, yes or
More informationDynamic Semantics. Dynamic Semantics. Operational Semantics Axiomatic Semantics Denotational Semantic. Operational Semantics
Dynamic Semantics Operational Semantics Denotational Semantic Dynamic Semantics Operational Semantics Operational Semantics Describe meaning by executing program on machine Machine can be actual or simulated
More informationSection 14.1 Computability then else
Section 14.1 Computability Some problems cannot be solved by any machine/algorithm. To prove such statements we need to effectively describe all possible algorithms. Example (Turing machines). Associate
More informationTowards a Theory of Information Flow in the Finitary Process Soup
Towards a Theory of in the Finitary Process Department of Computer Science and Complexity Sciences Center University of California at Davis June 1, 2010 Goals Analyze model of evolutionary self-organization
More information258 Handbook of Discrete and Combinatorial Mathematics
258 Handbook of Discrete and Combinatorial Mathematics 16.3 COMPUTABILITY Most of the material presented here is presented in far more detail in the texts of Rogers [R], Odifreddi [O], and Soare [S]. In
More informationThe following techniques for methods of proofs are discussed in our text: - Vacuous proof - Trivial proof
Ch. 1.6 Introduction to Proofs The following techniques for methods of proofs are discussed in our text - Vacuous proof - Trivial proof - Direct proof - Indirect proof (our book calls this by contraposition)
More informationThe Cook-Levin Theorem
An Exposition Sandip Sinha Anamay Chaturvedi Indian Institute of Science, Bangalore 14th November 14 Introduction Deciding a Language Let L {0, 1} be a language, and let M be a Turing machine. We say M
More informationCSE 555 Homework Three Sample Solutions
CSE 555 Homework Three Sample Solutions Question 1 Let Σ be a fixed alphabet. A PDA M with n states and stack alphabet of size σ may recognize the empty language; if its language is non-empty then denote
More informationUndecidability COMS Ashley Montanaro 4 April Department of Computer Science, University of Bristol Bristol, UK
COMS11700 Undecidability Department of Computer Science, University of Bristol Bristol, UK 4 April 2014 COMS11700: Undecidability Slide 1/29 Decidability We are particularly interested in Turing machines
More informationMathematical Logic. Reasoning in First Order Logic. Chiara Ghidini. FBK-IRST, Trento, Italy
Reasoning in First Order Logic FBK-IRST, Trento, Italy April 12, 2013 Reasoning tasks in FOL Model checking Question: Is φ true in the interpretation I with the assignment a? Answer: Yes if I = φ[a]. No
More informationIntroduction to Computational Complexity
Introduction to Computational Complexity Jiyou Li lijiyou@sjtu.edu.cn Department of Mathematics, Shanghai Jiao Tong University Sep. 24th, 2013 Computation is everywhere Mathematics: addition; multiplication;
More information