# Time, Space, and Energy in Reversible Computing. Paul Vitanyi, CWI, University of Amsterdam, National ICT Australia

Save this PDF as:

Size: px
Start display at page:

Download "Time, Space, and Energy in Reversible Computing. Paul Vitanyi, CWI, University of Amsterdam, National ICT Australia"

## Transcription

1 Time, Space, and Energy in Reversible Computing Paul Vitanyi, CWI, University of Amsterdam, National ICT Australia

2 Thermodynamics of Computing Heat Dissipation Input Compute Output Physical Law: 1 kt ln 2 is needed to irreversibly process 1 bit (Von Neumann, Landauer) Reversible computation is free. A B A AND B B AND NOT A A AND NOT B A AND B Input A billiard ball computer Output

3 Trend Graph follows Moore s Law: eV kt = ca 3 x 10 J Even at kt, 20 C, 100 Gigahertz, 10 Tera gates/cm Wat

4 Relevance: Mobilization Battery improvement (20% in 10 years) Miniaturization computing devices Improved speed + higher density (linear+square) More heat = dissipation /cm2 (cubic) Matter migration of circuits Flaky performance requiring reduced clock speed.

5 Reversible Universal Computer We use Turing Machines for convenience It is Reversible if for every operation there is an inverse operation that undoes the effect of the former operation. We can effectively enumerate a family of reversible TMs. There is a Universal Reversible TM that simulates all others (Lecerf 63, Bennett 73)

6 Energy If we compute x y reversibly, then actually we do p,x y,q Program garbage Garbage Program What is a program at one end, is garbage at the other end. We need to supply the program bits and erase the garbage bits. The best way to do this is to compact them as much as possible (like the garbage collection truck does).

7 Andrey Nikolaevich Kolmogorov ( , Tambov, Russia) Measure Theory Probability Analysis Intuitionistic Logic Cohomology Dynamical Systems Hydrodynamics Kolmogorov complexity

8 Kolmogorov Complexity Defines the ultimate data compression limit: gzip, bzip2, PPMZ... KC. Therefore it gives us the thermodynamically cheapest way to get rid of data garbage. Getting rid of (n ``1 s) costs n kt ln 2 joule; by first compressing it reversibly to log n bits, erasing costs only (log n)kt ln 2 Joule.

9 Kolmogorov Complexity Solomonoff (1960)-Kolmogorov (1965)-Chaitin (1969): The amount of information in a string is the size of the smallest program generating that string. { } ( ) min : ( ) K x = p U p = x p Invariance Theorem: It does not matter which universal Turing machine U we choose. I.e. all encoding methods are ok.

10 Kolmogorov complexity K(x)= length of shortest description of x K(x y)=length of shortest description of x given y. A string is (effectively) random if it is already compressed to the limit, that is, if K(x) x. To erase such a string costs the maximal entropy/energy x kt ln 2.

11 GENERALLY: The ultimate thermodynamic cost of erasing x is reached by: Reversibly compress x to x* Then erase x*. Cost ~K(x) bits. The longer you compute, the less heat dissipation. Thermodynamic cost of converting: x y E(x,y) = min { p : U(x,p) = y, U(y,p)=x }. Theorem. E(x,y) ) = max{ K(x y), K(y x) } (Bennett, Gacs,, Li, Vitanyi, Zurek STOC 91)

12 Cost of Erasure Trivially, E(x,ε) = K(x) (i.e., erasing x) t t Theorem. E(x, ε) K(x) Cost of erasing x using t time Length shortest program Computing x in t time Question: Is there a hierarchy for growing t?

13 Energy-Time Tradeoff Hierarchy For each large n and m= n /2, there is an x of length n, and t_1(n)<t_2(n) <... t_m(n) such that t_1 t_2 t_m E (x,ε) > E (x,ε) >... > E (x,ε) Note: t_i (n) = 2 in

14 General Reversible Simulation To do all (a priori unknown) computing tasks reversibly, we need reversibly compile irreversible programs into reversible ones, and then run a reversible computation. This implies general reversible simulation of irreversible computations.

15 Space Hungry (Bennett 1973) Irreversible Computation: Input Time 0 Reversible Simulation: ID Output Time T Instantaneous description = complete computer configuration ID description α Auxiliary Record: α_1, α_2,..., α_t Exists α -1

16 Algorithm: Input x: Reversibly Compute x f(x) Reversibly Copy f(x) Reversibly ``Uncompute x f(x) Erasing α_1, α_2,..., α_t History Output <x,f(x)>

17 Complexity Each Reversible Computation is 1:1 Reversible Simulation x f(x) using T time and S space is x <x,f(x)> with T = Θ (T) and S = Θ(T+S)

18 Space Parsimonious (Bennett 89) Compute 1 Copy 2 ID_1 ID_2 ID_2 Uncompute History Uncompute, erasing History 3 4

19 Checkpoint Levels: Can be modelled by ``Reversible Pebble Game Time 0

20 Reversible Pebble Game Pebble = ID 1. Initially: 1 2 T... Positions n free pebbles o 2. Each step: o OR 3. Win if 1) T gets pebbled, and 2) All n pebbles eventually free

21 Theorem: There is a winning strategy using n pebbles for T=2^n T o o o o o o..... o o o.... o. o... o o. o... o.. o o o..... o o o.... o. o.... o. o o... o. o.... o o o.... o o.. T n n pebbles T = 2-1 in 2 Log 3 T = T steps

22 Theorem (Li, Tromp, Vitanyi 98) No reversible pebble game can do better. (Proof is subtle) We use too much space!! In the worst case S= (log T)^2 which is pretty bad for large T. What if we allow erasures?

23 Theorem (Space-Erasure LVT 98) There is a winning strategy with n+2 pebbles and m-1 erasures for pebble games with T= m2^n for all m 1 with simulation time T = 2m^{1 log 3} T^{log 3}. Corollary: We can use n log (E-1) pebbles and E erasures instead of n pebbles.

24 Using Less space can only be done by simulations that are not reversible pebble games (LVT98). Lange, Mackenzie, Tapp 00: No extra space but exponential time, by reversible cycling through the configuration tree of the TM computation (connected component of fixed size IDs including the start ID), using a method due to Sipser 90.

25 Lemma (LMT00): Reversible simulation time is O(2^S) and reversible simulation space is S+ log T Reversibly cycle through All ID s of irreversible TM with ID < S (space) output Irreversible computatio input

26 Time-Space Tradeoff, Buhrman Tromp Vitanyi 00,01, Williams 00. Reversible Pebbling with long intervals for pebble moves being bridged by LMT simulations.

27 Algorithm: Reversible pebbling m SLMT Interval, Time = O(2^{O(m)}S) m SLMT

28 Using k pebbles and m=t/2^k Lemma: (BTV01) Simulation time is T ' = 3 Simulation space is: 2 k O ( T / 2 ) k S S = S(1+O(k))

29 Corollary 1: T ' = S3 S '/ S 2 O( T / 2 S '/ S )

30 Some special values: k=log log T: T' = S' = S(logT ) log3 2 O( T /logt ) S loglogt S logs k= log T: T ' = S 3 log T 2 O ( T / 2 log T ) S ' = S log T S S That is: Both Subexponential Time and Subquadratic Space!

31 Unknown Computation Time T: Simulate first t steps of computation for all values t=2,2^2,2^3,... Consecutively (and if T<t reversibly undo the simualtion and start with t 2t. Total time used is T 2T T '' 2 logt t = 1 S3 S '/ S 2 O(2 t S '/ S ) 2T '

32 Lower Bound (BTV01) Lemma: If T,S,T,S are average-case, then every reversible simulation of an Irreversible computation has: T T and S n+ log T Input length, e.g., n=s This is sharp, since (S)LMT reaches bound on S from above.

33 Big Question: Is there a polynomial time * space reversible simulation? We actually want about linear time And linear space -

### a logically reversible way says nothing about whether or not the computation dissipates energy. It merely means that the laws of physics do not requir

Reversible Simulation of Irreversible Computation Ming Li Paul Vitanyi y Computer Science Department CWI University of Waterloo Kruislaan 413 Waterloo, Ontario 1098 SJ Amsterdam Canada N2L 3G1 The Netherlands

### Kolmogorov complexity and its applications

CS860, Winter, 2010 Kolmogorov complexity and its applications Ming Li School of Computer Science University of Waterloo http://www.cs.uwaterloo.ca/~mli/cs860.html We live in an information society. Information

### Kolmogorov complexity and its applications

Spring, 2009 Kolmogorov complexity and its applications Paul Vitanyi Computer Science University of Amsterdam http://www.cwi.nl/~paulv/course-kc We live in an information society. Information science is

### 2.2 Classical circuit model of computation

Chapter 2 Classical Circuit Models for Computation 2. Introduction A computation is ultimately performed by a physical device. Thus it is a natural question to ask what are the fundamental limitations

### CISC 876: Kolmogorov Complexity

March 27, 2007 Outline 1 Introduction 2 Definition Incompressibility and Randomness 3 Prefix Complexity Resource-Bounded K-Complexity 4 Incompressibility Method Gödel s Incompleteness Theorem 5 Outline

### Sophistication Revisited

Sophistication Revisited Luís Antunes Lance Fortnow August 30, 007 Abstract Kolmogorov complexity measures the ammount of information in a string as the size of the shortest program that computes the string.

### Computability Theory

Computability Theory Cristian S. Calude May 2012 Computability Theory 1 / 1 Bibliography M. Sipser. Introduction to the Theory of Computation, PWS 1997. (textbook) Computability Theory 2 / 1 Supplementary

### Algorithmic Information Theory

Algorithmic Information Theory Peter D. Grünwald CWI, P.O. Box 94079 NL-1090 GB Amsterdam, The Netherlands E-mail: pdg@cwi.nl Paul M.B. Vitányi CWI, P.O. Box 94079 NL-1090 GB Amsterdam The Netherlands

### Finish K-Complexity, Start Time Complexity

6.045 Finish K-Complexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts

### Derandomizing from Random Strings

Derandomizing from Random Strings Harry Buhrman CWI and University of Amsterdam buhrman@cwi.nl Lance Fortnow Northwestern University fortnow@northwestern.edu Michal Koucký Institute of Mathematics, AS

### SAMPLE CHAPTERS UNESCO EOLSS QUANTUM COMPUTING. Paul Vitányi CWI, Kruislaan, Amsterdam, The Netherlands

QUANTUM COMPUTING Paul Vitányi CWI, Kruislaan, Amsterdam, The Netherlands Keywords: Quantum computing, entanglement, coherent quantum superposition, quantum interference, reversible computing; miniaturization

### Computer Sciences Department

Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y

### Kolmogorov Complexity with Error

Kolmogorov Complexity with Error Lance Fortnow 1, Troy Lee 2, and Nikolai Vereshchagin 3, 1 University of Chicago, 1100 E. 58th Street, Chicago, IL, USA 60637. fortnow@cs.uchicago.edu, http://people.cs.uchicago.edu/~fortnow

### FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY THURSDAY APRIL 3 REVIEW for Midterm TUESDAY April 8 Definition: A Turing Machine is a 7-tuple T = (Q, Σ, Γ, δ, q, q accept, q reject ), where: Q is a

### Lecture Notes for Ph219/CS219: Quantum Information Chapter 5. John Preskill California Institute of Technology

Lecture Notes for Ph219/CS219: Quantum Information Chapter 5 John Preskill California Institute of Technology Updated July 2015 Contents 5 Classical and quantum circuits 3 5.1 Classical Circuits 3 5.1.1

### CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression

CS154, Lecture 12: Kolmogorov Complexity: A Universal Theory of Data Compression Rosencrantz & Guildenstern Are Dead (Tom Stoppard) Rigged Lottery? And the winning numbers are: 1, 2, 3, 4, 5, 6 But is

### 1 Randomized complexity

80240233: Complexity of Computation Lecture 6 ITCS, Tsinghua Univesity, Fall 2007 23 October 2007 Instructor: Elad Verbin Notes by: Zhang Zhiqiang and Yu Wei 1 Randomized complexity So far our notion of

### FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY

15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY KOLMOGOROV-CHAITIN (descriptive) COMPLEXITY TUESDAY, MAR 18 CAN WE QUANTIFY HOW MUCH INFORMATION IS IN A STRING? A = 01010101010101010101010101010101

### Notes for Lecture 3... x 4

Stanford University CS254: Computational Complexity Notes 3 Luca Trevisan January 18, 2012 Notes for Lecture 3 In this lecture we introduce the computational model of boolean circuits and prove that polynomial

### Information in Biology

Information in Biology CRI - Centre de Recherches Interdisciplinaires, Paris May 2012 Information processing is an essential part of Life. Thinking about it in quantitative terms may is useful. 1 Living

### Maxwell s Demon. Kirk T. McDonald Joseph Henry Laboratories, Princeton University, Princeton, NJ (October 3, 2004)

1 Problem Maxwell s Demon Kirk T. McDonald Joseph Henry Laboratories, Princeton University, Princeton, NJ 08544 (October 3, 2004) This problem is prob. 2 of [1]. One of the earliest conceptual supercomputers

### A statistical mechanical interpretation of algorithmic information theory

A statistical mechanical interpretation of algorithmic information theory Kohtaro Tadaki Research and Development Initiative, Chuo University 1 13 27 Kasuga, Bunkyo-ku, Tokyo 112-8551, Japan. E-mail: tadaki@kc.chuo-u.ac.jp

### Computability Theory. CS215, Lecture 6,

Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David

### Measures of relative complexity

Measures of relative complexity George Barmpalias Institute of Software Chinese Academy of Sciences and Visiting Fellow at the Isaac Newton Institute for the Mathematical Sciences Newton Institute, January

### The physics of information: from Maxwell s demon to Landauer. Eric Lutz University of Erlangen-Nürnberg

The physics of information: from Maxwell s demon to Landauer Eric Lutz University of Erlangen-Nürnberg Outline 1 Information and physics Information gain: Maxwell and Szilard Information erasure: Landauer

### Chapter 2 Algorithms and Computation

Chapter 2 Algorithms and Computation In this chapter, we first discuss the principles of algorithm and computation in general framework, common both in classical and quantum computers, then we go to the

### Lecture 3: Reductions and Completeness

CS 710: Complexity Theory 9/13/2011 Lecture 3: Reductions and Completeness Instructor: Dieter van Melkebeek Scribe: Brian Nixon Last lecture we introduced the notion of a universal Turing machine for deterministic

### ,

Kolmogorov Complexity Carleton College, CS 254, Fall 2013, Prof. Joshua R. Davis based on Sipser, Introduction to the Theory of Computation 1. Introduction Kolmogorov complexity is a theory of lossless

### Average Case Analysis of QuickSort and Insertion Tree Height using Incompressibility

Average Case Analysis of QuickSort and Insertion Tree Height using Incompressibility Tao Jiang, Ming Li, Brendan Lucier September 26, 2005 Abstract In this paper we study the Kolmogorov Complexity of a

### Algorithmic Probability

Algorithmic Probability From Scholarpedia From Scholarpedia, the free peer-reviewed encyclopedia p.19046 Curator: Marcus Hutter, Australian National University Curator: Shane Legg, Dalle Molle Institute

### Robustness of PSPACE-complete sets

Robustness of PSPACE-complete sets A. Pavan a,1 and Fengming Wang b,1 a Department of Computer Science, Iowa State University. pavan@cs.iastate.edu b Department of Computer Science, Rutgers University.

### Kolmogorov complexity ; induction, prediction and compression

Kolmogorov complexity ; induction, prediction and compression Contents 1 Motivation for Kolmogorov complexity 1 2 Formal Definition 2 3 Trying to compute Kolmogorov complexity 3 4 Standard upper bounds

### Cellular Automata: Tutorial

Cellular Automata: Tutorial Jarkko Kari Department of Mathematics, University of Turku, Finland TUCS(Turku Centre for Computer Science), Turku, Finland Cellular Automata: examples A Cellular Automaton

### NP-Completeness. Andreas Klappenecker. [based on slides by Prof. Welch]

NP-Completeness Andreas Klappenecker [based on slides by Prof. Welch] 1 Prelude: Informal Discussion (Incidentally, we will never get very formal in this course) 2 Polynomial Time Algorithms Most of the

### Three Approaches to the Quantitative Definition of Information in an Individual Pure Quantum State

Three Approaches to the Quantitative Definition of Information in an Individual Pure Quantum State Paul Vitanyi CWI and University of Amsterdam Abstract In analogy of classical Kolmogorov complexity we

### 6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch

6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch Today: More Complexity Theory Polynomial-time reducibility, NP-completeness, and the Satisfiability (SAT) problem Topics: Introduction

### Separating NE from Some Nonuniform Nondeterministic Complexity Classes

Separating NE from Some Nonuniform Nondeterministic Complexity Classes Bin Fu 1, Angsheng Li 2, and Liyu Zhang 3 1 Dept. of Computer Science, University of Texas - Pan American TX 78539, USA. binfu@cs.panam.edu

### ON BROWNIAN COMPUTATION

ON BROWNIAN COMPUTATION JOHN D. NORTON Department of History and Philosophy of Science Center for Philosophy of Science University of Pittsburgh Pittsburgh PA USA 15260 jdnorton@pitt.edu Draft: October10,

### Time-Bounded Kolmogorov Complexity and Solovay Functions

Time-Bounded Kolmogorov Complexity and Solovay Functions Rupert Hölzl, Thorsten Kräling, and Wolfgang Merkle Institut für Informatik, Ruprecht-Karls-Universität, Heidelberg, Germany Abstract. A Solovay

### Computer Science 385 Analysis of Algorithms Siena College Spring Topic Notes: Limitations of Algorithms

Computer Science 385 Analysis of Algorithms Siena College Spring 2011 Topic Notes: Limitations of Algorithms We conclude with a discussion of the limitations of the power of algorithms. That is, what kinds

### Algorithmic Information Theory and Kolmogorov Complexity

Algorithmic Information Theory and Kolmogorov Complexity Alexander Shen, Uppsala University, Computing Science Department, Independent University of Moscow, shen@mccme.ru November 19, 2007 Abstract This

### 1 Randomized Computation

CS 6743 Lecture 17 1 Fall 2007 1 Randomized Computation Why is randomness useful? Imagine you have a stack of bank notes, with very few counterfeit ones. You want to choose a genuine bank note to pay at

### 1 Computational Problems

Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

### From fully quantum thermodynamical identities to a second law equality

From fully quantum thermodynamical identities to a second law equality Alvaro Alhambra, Lluis Masanes, Jonathan Oppenheim, Chris Perry Fluctuating States Phys. Rev. X 6, 041016 (2016) Fluctuating Work

### BBM402-Lecture 11: The Class NP

BBM402-Lecture 11: The Class NP Lecturer: Lale Özkahya Resources for the presentation: http://ocw.mit.edu/courses/electrical-engineering-andcomputer-science/6-045j-automata-computability-andcomplexity-spring-2011/syllabus/

### 04. Information and Maxwell's Demon. I. Dilemma for Information-Theoretic Exorcisms.

04. Information and Maxwell's Demon. I. Dilemma for Information-Theoretic Exorcisms. Two Options: (S) (Sound). The combination of object system and demon forms a canonical thermal system. (P) (Profound).

### Theory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death

Theory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death Robbie Hott www.cs.virginia.edu/~jh2jf Department of Computer Science University of Virginia

### A better lower bound for quantum algorithms searching an ordered list

A better lower bound for quantum algorithms searching an ordered list Andris Ambainis Computer Science Division University of California Berkeley, CA 94720, e-mail: ambainis@cs.berkeley.edu Abstract We

### Theory of Computation

Theory of Computation Dr. Sarmad Abbasi Dr. Sarmad Abbasi () Theory of Computation 1 / 33 Lecture 20: Overview Incompressible strings Minimal Length Descriptions Descriptive Complexity Dr. Sarmad Abbasi

### Kolmogorov Complexity in Randomness Extraction

LIPIcs Leibniz International Proceedings in Informatics Kolmogorov Complexity in Randomness Extraction John M. Hitchcock, A. Pavan 2, N. V. Vinodchandran 3 Department of Computer Science, University of

### Computational Complexity

p. 1/24 Computational Complexity The most sharp distinction in the theory of computation is between computable and noncomputable functions; that is, between possible and impossible. From the example of

### Degrees of Streams. Jörg Endrullis Dimitri Hendriks Jan Willem Klop. Streams Seminar Nijmegen, 20th April Vrije Universiteit Amsterdam

Degrees of Streams Jörg Endrullis Dimitri Hendriks Jan Willem Klop Vrije Universiteit Amsterdam Streams Seminar Nijmegen, 20th April 2010 Complexity of streams Complexity measures for infinite streams:

### Announcements. Friday Four Square! Problem Set 8 due right now. Problem Set 9 out, due next Friday at 2:15PM. Did you lose a phone in my office?

N P NP Completeness Announcements Friday Four Square! Today at 4:15PM, outside Gates. Problem Set 8 due right now. Problem Set 9 out, due next Friday at 2:15PM. Explore P, NP, and their connection. Did

### Chapter 1. Introduction

Chapter 1 Introduction Symbolical artificial intelligence is a field of computer science that is highly related to quantum computation. At first glance, this statement appears to be a contradiction. However,

### Polynomial Identity Testing and Circuit Lower Bounds

Polynomial Identity Testing and Circuit Lower Bounds Robert Špalek, CWI based on papers by Nisan & Wigderson, 1994 Kabanets & Impagliazzo, 2003 1 Randomised algorithms For some problems (polynomial identity

### Randomized Complexity Classes; RP

Randomized Complexity Classes; RP Let N be a polynomial-time precise NTM that runs in time p(n) and has 2 nondeterministic choices at each step. N is a polynomial Monte Carlo Turing machine for a language

### Adiabatic quantum computation a tutorial for computer scientists

Adiabatic quantum computation a tutorial for computer scientists Itay Hen Dept. of Physics, UCSC Advanced Machine Learning class UCSC June 6 th 2012 Outline introduction I: what is a quantum computer?

### Mm7 Intro to distributed computing (jmp) Mm8 Backtracking, 2-player games, genetic algorithms (hps) Mm9 Complex Problems in Network Planning (JMP)

Algorithms and Architectures II H-P Schwefel, Jens M. Pedersen Mm6 Advanced Graph Algorithms (hps) Mm7 Intro to distributed computing (jmp) Mm8 Backtracking, 2-player games, genetic algorithms (hps) Mm9

### ISSN Article

Entropy 0, 3, 595-6; doi:0.3390/e3030595 OPEN ACCESS entropy ISSN 099-4300 www.mdpi.com/journal/entropy Article Entropy Measures vs. Kolmogorov Compleity Andreia Teieira,,, Armando Matos,3, André Souto,

### Kolmogorov Complexity

Kolmogorov Complexity Davide Basilio Bartolini University of Illinois at Chicago Politecnico di Milano dbarto3@uic.edu davide.bartolini@mail.polimi.it 1 Abstract What follows is a survey of the field of

### COE 202: Digital Logic Design Sequential Circuits Part 3. Dr. Ahmad Almulhem ahmadsm AT kfupm Phone: Office:

COE 202: Digital Logic Design Sequential Circuits Part 3 Dr. Ahmad Almulhem Email: ahmadsm AT kfupm Phone: 860-7554 Office: 22-324 Objectives Important Design Concepts State Reduction and Assignment Design

### Does the Polynomial Hierarchy Collapse if Onto Functions are Invertible?

Does the Polynomial Hierarchy Collapse if Onto Functions are Invertible? Harry Buhrman Lance Fortnow Michal Koucký John D. Rogers Nikolay Vereshchagin December 23, 2008 Abstract The class TFNP, defined

### Theory of Computation Lecture Notes. Problems and Algorithms. Class Information

Theory of Computation Lecture Notes Prof. Yuh-Dauh Lyuu Dept. Computer Science & Information Engineering and Department of Finance National Taiwan University Problems and Algorithms c 2004 Prof. Yuh-Dauh

### Algorithms and Theory of Computation. Lecture 1: Introduction, Basics of Algorithms

Algorithms and Theory of Computation Lecture 1: Introduction, Basics of Algorithms Xiaohui Bei MAS 714 August 14, 2017 Nanyang Technological University MAS 714 August 14, 2017 1 / 23 Administration Lectures

### NP-Complete problems

NP-Complete problems NP-complete problems (NPC): A subset of NP. If any NP-complete problem can be solved in polynomial time, then every problem in NP has a polynomial time solution. NP-complete languages

### When Abandoned Bits Bite Back Reversibility and Quantum Computing

Area Exam February 4, 00 When Abandoned Bits Bite Back Reversibility and Quantum Computing Paul Fitzpatrick AI Lab, MIT, Cambridge, USA paulfitz@ai.mit.edu Abstract. Reversible logic was originally proposed

### CS446: Machine Learning Fall Final Exam. December 6 th, 2016

CS446: Machine Learning Fall 2016 Final Exam December 6 th, 2016 This is a closed book exam. Everything you need in order to solve the problems is supplied in the body of this exam. This exam booklet contains

### Lecture 16: Time Complexity and P vs NP

6.045 Lecture 16: Time Complexity and P vs NP 1 Time-Bounded Complexity Classes Definition: TIME(t(n)) = { L there is a Turing machine M with time complexity O(t(n)) so that L = L(M) } = { L L is a language

### Indefinite survival through backup copies

Indefinite survival through backup copies Anders Sandberg and Stuart Armstrong June 06 202 Abstract If an individual entity endures a fixed probability µ < of disappearing ( dying ) in a given fixed time

### A Tale of Two Cultures: Phase Transitions in Physics and Computer Science. Cristopher Moore University of New Mexico and the Santa Fe Institute

A Tale of Two Cultures: Phase Transitions in Physics and Computer Science Cristopher Moore University of New Mexico and the Santa Fe Institute Computational Complexity Why are some problems qualitatively

### Definition: Alternating time and space Game Semantics: State of machine determines who

CMPSCI 601: Recall From Last Time Lecture Definition: Alternating time and space Game Semantics: State of machine determines who controls, White wants it to accept, Black wants it to reject. White wins

### Kernelization by matroids: Odd Cycle Transversal

Lecture 8 (10.05.2013) Scribe: Tomasz Kociumaka Lecturer: Marek Cygan Kernelization by matroids: Odd Cycle Transversal 1 Introduction The main aim of this lecture is to give a polynomial kernel for the

### CpSc 421 Homework 9 Solution

CpSc 421 Homework 9 Solution Attempt any three of the six problems below. The homework is graded on a scale of 100 points, even though you can attempt fewer or more points than that. Your recorded grade

### 3 Self-Delimiting Kolmogorov complexity

3 Self-Delimiting Kolmogorov complexity 3. Prefix codes A set is called a prefix-free set (or a prefix set) if no string in the set is the proper prefix of another string in it. A prefix set cannot therefore

### Q = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar

IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar Turing Machine A Turing machine is an abstract representation of a computing device. It consists of a read/write

### Limitations of Algorithm Power

Limitations of Algorithm Power Objectives We now move into the third and final major theme for this course. 1. Tools for analyzing algorithms. 2. Design strategies for designing algorithms. 3. Identifying

### Lecture 27: Entropy and Information Prof. WAN, Xin

General Pysis I Leture 27: Entropy and Information Prof. WAN, Xin xinwan@zju.edu.n ttp://zimp.zju.edu.n/~xinwan/ 1st & 2nd Laws of ermodynamis e 1st law speifies tat we annot get more energy out of a yli

### A version of for which ZFC can not predict a single bit Robert M. Solovay May 16, Introduction In [2], Chaitin introd

CDMTCS Research Report Series A Version of for which ZFC can not Predict a Single Bit Robert M. Solovay University of California at Berkeley CDMTCS-104 May 1999 Centre for Discrete Mathematics and Theoretical

### Kernel-Size Lower Bounds. The Evidence from Complexity Theory

: The Evidence from Complexity Theory IAS Worker 2013, Warsaw Part 2/3 Note These slides are a slightly revised version of a 3-part tutorial given at the 2013 Workshop on Kernelization ( Worker ) at the

### Introduction to Kolmogorov Complexity

Introduction to Kolmogorov Complexity Marcus Hutter Canberra, ACT, 0200, Australia http://www.hutter1.net/ ANU Marcus Hutter - 2 - Introduction to Kolmogorov Complexity Abstract In this talk I will give

### Notes for Lecture 25

U.C. Berkeley CS278: Computational Complexity Handout N25 ofessor Luca Trevisan 12/1/2004 Notes for Lecture 25 Circuit Lower Bounds for Parity Using Polynomials In this lecture we prove a lower bound on

### Lecture 2: From Classical to Quantum Model of Computation

CS 880: Quantum Information Processing 9/7/10 Lecture : From Classical to Quantum Model of Computation Instructor: Dieter van Melkebeek Scribe: Tyson Williams Last class we introduced two models for deterministic

### arxiv:math/ v1 [math.nt] 17 Feb 2003

arxiv:math/0302183v1 [math.nt] 17 Feb 2003 REPRESENTATIONS OF Ω IN NUMBER THEORY: FINITUDE VERSUS PARITY TOBY ORD AND TIEN D. KIEU Abstract. We present a new method for expressing Chaitin s random real,

### THE LANDAUER LIMIT AND THERMODYNAMICS OF BIOLOGICAL SYSTEMS

THE LANDAUER LIMIT AND THERMODYNAMICS OF BIOLOGICAL SYSTEMS Daid H. Wolpert Santa Fe Institute, MIT ttp://daidwolpert.weebly.com Jos Grocow Santa Fe Institute HEAT OF ERASING A BIT a) b) c) R 1 2 2R 1

### CSC 5170: Theory of Computational Complexity Lecture 5 The Chinese University of Hong Kong 8 February 2010

CSC 5170: Theory of Computational Complexity Lecture 5 The Chinese University of Hong Kong 8 February 2010 So far our notion of realistic computation has been completely deterministic: The Turing Machine

### University of Connecticut Department of Mathematics

University of Connecticut Department of Mathematics Math 1131 Sample Exam 1 Fall 2013 Name: This sample exam is just a guide to prepare for the actual exam. Questions on the actual exam may or may not

### Demon Dynamics: Deterministic Chaos, the Szilard Map, & the Intelligence of Thermodynamic Systems.

Demon Dynamics: Deterministic Chaos, the Szilard Map, & the Intelligence of Thermodynamic Systems http://csc.ucdavis.edu/~cmg/ Jim Crutchfield Complexity Sciences Center Physics Department University of

### DESIGN OF A COMPACT REVERSIBLE READ- ONLY-MEMORY WITH MOS TRANSISTORS

DESIGN OF A COMPACT REVERSIBLE READ- ONLY-MEMORY WITH MOS TRANSISTORS Sadia Nowrin, Papiya Nazneen and Lafifa Jamal Department of Computer Science and Engineering, University of Dhaka, Bangladesh ABSTRACT

### arxiv: v2 [quant-ph] 27 Dec 2010

Reversible circuit synthesis using a cycle-based approach arxiv:1004.4320v2 [quant-ph] 27 Dec 2010 Mehdi Saeedi, Morteza Saheb Zamani, Mehdi Sedighi, Zahra Sasanian Quantum Design Automation Lab Department

### Fourier Sampling & Simon s Algorithm

Chapter 4 Fourier Sampling & Simon s Algorithm 4.1 Reversible Computation A quantum circuit acting on n qubits is described by an n n unitary operator U. Since U is unitary, UU = U U = I. This implies

### k-protected VERTICES IN BINARY SEARCH TREES

k-protected VERTICES IN BINARY SEARCH TREES MIKLÓS BÓNA Abstract. We show that for every k, the probability that a randomly selected vertex of a random binary search tree on n nodes is at distance k from

### Lecture 17. In this lecture, we will continue our discussion on randomization.

ITCS:CCT09 : Computational Complexity Theory May 11, 2009 Lecturer: Jayalal Sarma M.N. Lecture 17 Scribe: Hao Song In this lecture, we will continue our discussion on randomization. 1 BPP and the Polynomial

### Synchronous Logic. These must be the slings and arrows of outrageous fortune. Comp 411 Fall /18/06. L14 Synchronous Logic 1

Synchronous Logic 1) Sequential Logic 2) Synchronous Design 3) Synchronous Timing Analysis 4) Single Clock Design 5) Finite State Machines 6) Turing Machines 7) What it means to be Computable These must

### Introduction to Turing Machines. Reading: Chapters 8 & 9

Introduction to Turing Machines Reading: Chapters 8 & 9 1 Turing Machines (TM) Generalize the class of CFLs: Recursively Enumerable Languages Recursive Languages Context-Free Languages Regular Languages

### THE INFORMATION REQUIREMENTS OF COMPLEX BIOLOGICAL AND ECONOMIC SYSTEMS WITH ALGORITHMIC INFORMATION THEORY

S. Devine Int. J. of Design & Nature and Ecodynamics. Vol. 12, No. 3 (2017) 367-376 THE INFORMATION REQUIREMENTS OF COMPLEX BIOLOGICAL AND ECONOMIC SYSTEMS WITH ALGORITHMIC INFORMATION THEORY S. DEVINE

### Real Numbers and Computational Complexity

Real Numbers and Computational Complexity Norbert Müller mueller@uni-trier.de FB IV Informatik Universität Trier Norbert Müller (Trier) Darmstadt, 2012-10-05 1 / 34 Introduction 1 Introduction 2 Computational

### Lecture 13: Foundations of Math and Kolmogorov Complexity

6.045 Lecture 13: Foundations of Math and Kolmogorov Complexity 1 Self-Reference and the Recursion Theorem 2 Lemma: There is a computable function q : Σ* Σ* such that for every string w, q(w) is the description

### P, NP, NP-Complete, and NPhard

P, NP, NP-Complete, and NPhard Problems Zhenjiang Li 21/09/2011 Outline Algorithm time complicity P and NP problems NP-Complete and NP-Hard problems Algorithm time complicity Outline What is this course