CS 591 Complex Adaptive Systems Spring 2009 Measures of Complexity. Melanie Moses 1/28/09
|
|
- Bathsheba Thornton
- 5 years ago
- Views:
Transcription
1 CS 591 Complex Adaptive Systems Spring 2009 Measures of Complexity Melanie Moses 1/28/09
2 Topics Why do we need formal definitions of complexity? Complex Adaptive Systems Definitions of Complexity Flake & Gell Mann Computational Definitions of complexity Running time complexity Gell Mann s Effective Complexity Kolmogorov (or Algorithmic) complexity Mutual Information Logical Depth Others Per Bak Stuart Kauffman John Holland Wolfram and Langton s complexity of CAs
3 Why define complexity? To estimate how long a particular system will take to solve a problem To estimate difficulty in engineering complex systems To understand the limits of prediction, approximation, and simulation To answer fundamental scientific questions Does (why does?) complexity increase through evolution? How can complexity arise when entropy should always increase? As evolution proceeded on the surface of the earth, there has been a progressive increase in size and complexity J. T. Bonner Evolution of Complexity. Evolution tends to push systems towards the edge of chaos, where complex, interesting behaviors such as life can occur Chris Langton The universe, the biosphere, the econosphere have all become more complex Stuart Kauffmann Are these statements correct? If so, how do we quantify increases in complexity over time?
4 What are Complex Adaptive Systems? Collections of agents: Molecules, cells, animals, planets, economic agents. Agents interact (locally) with one another and with their environment: No central controller. Interaction rules may be trivial or nontrivial (i.e. nonlinear); deterministic or nondeterministic Chemical reactions, cellular interactions, mating, buy/sell decisions. Unanticipated properties often result from the interactions: Immune system responses, flocks of animals, settlement patterns, earthquakes, speculative bubbles and crashes. Agents adapt their behavior to other agents and environmental constraints: Imitation, adaptation, learning. System behavior evolves over time: Rules change, unfolding of complex behavior.
5 Example Complex Adaptive Systems Biological systems Cells Organs Immune systems Organisms Ecosystems Food webs Symbioses Colonies: ants, bees, termites, wasps; bacteria, corals Social/Economic Systems Nations Firms Households Cities Universities Technological Systems The Internet Peer to peer networks Botnets Computers? Which is most complex?
6 Characteristics of Complex Systems What makes a system complex? Nonlinear interactions among components. Multi-scale phenomena and hierarchy. Evolution of underlying components and environments. Is complexity inherent in the system or in our understanding of it? Ex 1: quantum computation and factoring Ex 2: Differential equations vs simulations to model disease spread How to measure a system s complexity? By its unpredictability? By how difficult it is to describe? Length of most concise description. No single model adequate to describe system---the more models that are required, the more complex the system. (Lee Segel) The length of time before it halts or repeats itself, if ever? Entropy? Multiple levels of organization? Number of interdependencies?
7 Caveat Some believe that there is no general science of complex systems: It s becoming apparent that a theory of complexity in some sense of a great and transforming body of knowledge that applies to a whole range of cases may be untenable. Sir Robert May (2001)
8 G.W. Flake s Definition of Complexity Flake s scientific view An intermediate view between Reductionism and Holism The interactions of agents bind one level of understanding to the next Nature is Frugal: Occam s Razor CAS characterized by Emergence, self-similarity, unpredictability, self-organizing Collections/Multiplicity/Parallelism Iteration, Recursion, Feedback Adaptation, Learning, Evolution Conversion of the sciences facilitated by computation
9 Representative Measures of Complexity Computational complexity (Cook): How long a program runs (or how much memory it uses). Asymptotic. Language complexity: Classes of languages that can be computed (recognized) by different kinds of abstract machines. Decidability, computability. Information-theoretic approaches: Algorithmic Complexity (Solomonoff, Komogorov, and Chaitin): Length of the shortest program that can produce the phenomenon. Mutual information (many authors) Logical depth (Bennett) Effective complexity (Gell-Mann and Lloyd)
10 Computational Complexity Introduced by Steve Cook (~1970). Asymptotic running time, and/or memory consumption of an algorithm. Worst-case versus average-case. Important computational complexity classes: P problems can be solved in polynomial time on a deterministic Turing Machine for some polynomial function p: Cn k where c and K are constants independent of n which is the size (length) of the input f ( n) is O( g( n)) iff k,c ( n > k), f ( n) c g( n) Polynomial time algorithms P are O (p(n)) NP (nondeterministic polynomial) problems can be verified in O(p(n)) or solved on a non-deterministic (parallel) Turing Machine: subset sum {6,-1,-4, -2, 5} NP hard: an algorithm for solving it can be translated to solve any NP problem, e.g. traveling salesman NP complete problems are both NP and NP hard
11 Turing Machines 1. a tape 2. a head that can r/w and move l/r 3. instruction table 4. state register tape is infinite, all else is finite and discrete
12 Topics Why do we need formal definitions of complexity? Complex Adaptive Systems Definitions of Complexity Flake & Gell Mann Computational Definitions of complexity Running time complexity Gell Mann s Effective Complexity Kolmogorov (or Algorithmic) complexity Mutual Information Logical Depth Others Per Bak Stuart Kauffman John Holland Wolfram and Langton s complexity of CAs
13 Quicksort (P) vs Traveling Salesman (NP hard) O(N 2 ) or O(Nlog(N)) O(N!) or O(2 N )
14 Computational Complexity Drawbacks Constants can matter Special cases vs worst cases Approximations may suffice Context dependent: what machine, what is already known.. Benefit: Allows objective comparison of how hard problems are as input size increases
15 Murray Gell-Mann: Effective Complexity Complexity is highest for things that are neither strictly regular nor strictly random Kolmogorov or Algorithmic Information Complexity is based on the length of a concise description of a set. AIC is a measure of randomness NOT a good measure of complexity Effective complexity is the length of a concise description of a set s regularities Regularities can be found using Mutual Information What are AIC, entropy and Mutual Information?
16 Algorithmic Complexity (AC) (also known as Kolmogorov-Chaitin complexity) The Kolomogorov-Chaitin complexity K(x) is the length, in bits, of the smallest program that when run on a Universal Turing Machine outputs (prints) x and then halts. Example: What is K(x) where x is the first 10 even natural numbers? Where x is the first 5 million even natural numbers? Possible representations: 0, 2, 4, 6, 8, 10, 12, 14, 16, 18, (2n - 2) for (j = 0; j < n: j++) printf( %d\n, j * 2); How many bits? Alternative 1: O(n log n) Alternative 2: K(x) = O(log n) Two problems: Calculation of K(x) depends on the machine we have available (e.g., what if we have a machine with an instruction print the first 10 even natural numbers?) In general, it is an uncomputable problem to determine K(x) for arbitrary x. How long is the program to do x?
17 Algorithmic Complexity cont. AC formalizes what it means for a set of numbers to be compressible and incompressible. Data that are redundant can be more easily described and have lower AC. Data that have no clear pattern and no easy algorithmic description have high AC. What about random numbers? If a string is random, then it possesses no regularities: K(x) = Print(x) The shortest program to produce x is to input to the computer a copy of x and say print this. Implication: The more random a system, the greater its AC. Gell Mann: Shakespeare vs monkeys AC is related to entropy: The entropy rate of a symbolic sequence measures the unpredictability (in bits per symbol) of the sequence. For a sequence of n random variables, how does the entropy of the sequence grow with n?
18 Shannon Information Shannon Entropy H to measure basic information capacity: For a random variable X with a probability mass function p(x), the entropy of X is defined as: Entropy is measured in bits. H measures the average uncertainty in the random variable. Example 1: H ( X ) = p( x)log2 p( x) Consider a random variable with uniform distribution over 32 outcomes. To identify an outcome, we need a label that takes on 32 different values, e.g., 5-bit strings. H ( X ) = p( i)log p( i) = i= 1 i= log 1 32 = log32 = 5 bits
19 Entropy is the uncertainty of a single random variable. The entropy of a random variable is a lower bound on the average number of bits required to represent the variable. The uncertainty (complexity) of a random variable can be extended to define the descriptive complexity of a single string. How much entropy is in the string, or how much information is required to describe the string? Kolmogorov (or algorithmic) complexity is the length of the shortest computer program that prints out the string. Algorithmic information theory (AIT) is the result of putting Shannon's information theory and Turing's computability theory into a cocktail shaker and shaking vigorously. The basic idea is to measure the complexity of an object by the size in bits of the smallest program for computing it. (G. J. Chaitin)
20 Murray Gell-Mann: Effective Complexity Effective complexity is the AIC of a set s regularities Mutual Information can be used to determine regularities
21 Mutual Information Measures the amount of information that one random variable contains about another random variable. Mutual information is a measure of reduction of uncertainty due to another random variable. That is, mutual information measures the dependence between two random variables. It is symmetric in X and Y, and is always non-negative. Recall: Entropy of a random variable X is H(X). Conditional entropy of a random variable X given another random variable Y = H(X Y). The mutual information of two random variables X and Y is: I( X, Y ) = H ( X ) H ( X Y ) = p( x, y)log p( x, y) x, y p( x) p( y)
22 Recap: Effective Complexity The shortest description of the regularities in an entity (and their probabilities) Regularities are found by the mutual information Effective complexity is NOT AIC itself (random strings have high AIC but low complexity) or Mutual information ( has high MI but low complexity) but AIC and MI are used to determine Effective complexity Exercise: regularities in spoken language Give the most concise description you can of regularities in verb tenses and usage English Spanish Hindi (?) Chinese (dialect?) Mutual Information to determine regularities? Context?
23 English Spanish Chinese Erdu Tenses Subject-verb agreement Past--ed, present, future--will, conditional would, Would have Gerunds Subjunctive barely used 1 st, 2 nd, 3 rd person, singular, plural 3 major tenses Defining and asking are different Classes Characters Ar er ir 7 temporal, 6 subject tenses 2 situational tenses Drawn, constructed, char Pronunciation differences
24 Other Points from Gell-Mann Schema are representations of the regularities: a package of information describing the regularities in a system, its environment and their interactions The schemata evolve by competing to produce the most predictive description How long does the schema take to make predictions? Logical Depth The Logical depth of x is the run time of the shortest program that will cause a UTM to produce x and then halt. Must distinguish between effective complexity and logical depth The universe has high effective complexity despite simple physical rules because of probabilities & alternative histories Frozen accidents produce mutual information or regularities Most of the effective complexity of the universe lies in the AIC of a description of those frozen accidents and their consequences
25 an individual human being, owes its existence not only to the simple fundamental law of physics and the boundary condition on the early universe but also to the outcomes of an inconceivably long sequence of probabilistic events, each of which could have turned out differently
26 Simplicity: algorithmic regularity, statistical regularity, compressible Flake, chapter 9 Algorithmic Complexity Randomness
27 Other Concepts of Complexity Per Bak: Sandpile model and the edge of chaos 1. A powerlaw size distribution of extinction events: many small and few large 2. The powerlaw lifetime distribution of species and genera. 3. The powerlaw distribution of species per genus, genera per family, and so on. Stuart Kauffmann: The Adjacent possible Autonomous agents have to live the most complex game that they can There is a tendency for self-constructing biospheres to enlarge their workspace, the dimensionality of their adjacent possible John Holland CAS: multiple diverse agents, stimulus response rules and adaptation Match environmental conditions at various time scales Stimulus causes an immediate change in action (response) Cumulative stimulus causes changes in rules (adaptation, learning, evolution)
28 Summary of Complexity Measures Computational complexity: How much time (how many resources) does it take to compute something? The language/machine hierarchy: How complex a machine is needed to compute a function? Information-theoretic methods: Entropy Algorithmic complexity Mutual information Gell Mann s Effective Complexity: The shortest description of the regularities Logical depth: Running time of the shortest program Combines computational complexity and AIC Asymptotic behavior of dynamical systems: Fixed points, limit cycles, chaos. Wolfram s CA classification: the outcome of complex CA can not be predicted any faster than it can be simulated.
29 Neither random nor regular Complex systems occur at a transition point between two extremes Often the only way to predict a future state of a complex system is to simulate it It cannot be described by a tractable formula or short program It cannot be described statistically Some cellular automata are complex: the results of running them are neither regular nor random Langton s lambda parameter and Wolfram s CA classification schemes go from regular to chaotic, with complex in the middle
30 Suggested References Computational Complexity by Papadimitriou. Addison-Wesley (1994). Elements of Information Theory by Cover and Thomas. Wiley (1991). Kaufmann, At Home in the Universe (1996) and Investigations (2002). Per Bak, How Nature Works: The Science of Self-Organized Criticality (1988) Gell-Mann, The Quark and the Jaguar (1994) Holland, Adaptation in Natural and Artificial Systems, MIT Press; 1992
Computer Sciences Department
Computer Sciences Department 1 Reference Book: INTRODUCTION TO THE THEORY OF COMPUTATION, SECOND EDITION, by: MICHAEL SIPSER Computer Sciences Department 3 ADVANCED TOPICS IN C O M P U T A B I L I T Y
More informationCISC 876: Kolmogorov Complexity
March 27, 2007 Outline 1 Introduction 2 Definition Incompressibility and Randomness 3 Prefix Complexity Resource-Bounded K-Complexity 4 Incompressibility Method Gödel s Incompleteness Theorem 5 Outline
More informationCellular Automata CS 591 Complex Adaptive Systems Spring Professor: Melanie Moses 2/02/09
Cellular Automata CS 591 Complex Adaptive Systems Spring 2009 Professor: Melanie Moses 2/02/09 Introduction to Cellular Automata (CA) Invented by John von Neumann (circa~1950). A cellular automata consists
More informationAdvanced topic: Space complexity
Advanced topic: Space complexity CSCI 3130 Formal Languages and Automata Theory Siu On CHAN Chinese University of Hong Kong Fall 2016 1/28 Review: time complexity We have looked at how long it takes to
More informationbiologically-inspired computing lecture 5 Informatics luis rocha 2015 biologically Inspired computing INDIANA UNIVERSITY
lecture 5 -inspired Sections I485/H400 course outlook Assignments: 35% Students will complete 4/5 assignments based on algorithms presented in class Lab meets in I1 (West) 109 on Lab Wednesdays Lab 0 :
More information6.080 / Great Ideas in Theoretical Computer Science Spring 2008
MIT OpenCourseWare http://ocw.mit.edu 6.080 / 6.089 Great Ideas in Theoretical Computer Science Spring 2008 For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.
More informationNotes for Lecture Notes 2
Stanford University CS254: Computational Complexity Notes 2 Luca Trevisan January 11, 2012 Notes for Lecture Notes 2 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation
More informationComputational Complexity and Intractability: An Introduction to the Theory of NP. Chapter 9
1 Computational Complexity and Intractability: An Introduction to the Theory of NP Chapter 9 2 Objectives Classify problems as tractable or intractable Define decision problems Define the class P Define
More informationToward a Better Understanding of Complexity
Toward a Better Understanding of Complexity Definitions of Complexity, Cellular Automata as Models of Complexity, Random Boolean Networks Christian Jacob jacob@cpsc.ucalgary.ca Department of Computer Science
More informationFinish K-Complexity, Start Time Complexity
6.045 Finish K-Complexity, Start Time Complexity 1 Kolmogorov Complexity Definition: The shortest description of x, denoted as d(x), is the lexicographically shortest string such that M(w) halts
More information1 Computational Problems
Stanford University CS254: Computational Complexity Handout 2 Luca Trevisan March 31, 2010 Last revised 4/29/2010 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation
More informationComplexity Theory Part I
Complexity Theory Part I Outline for Today Recap from Last Time Reviewing Verifiers Nondeterministic Turing Machines What does nondeterminism mean in the context of TMs? And just how powerful are NTMs?
More informationTuring Machines and Time Complexity
Turing Machines and Time Complexity Turing Machines Turing Machines (Infinitely long) Tape of 1 s and 0 s Turing Machines (Infinitely long) Tape of 1 s and 0 s Able to read and write the tape, and move
More informationThe Polynomial Hierarchy
The Polynomial Hierarchy Slides based on S.Aurora, B.Barak. Complexity Theory: A Modern Approach. Ahto Buldas Ahto.Buldas@ut.ee Motivation..synthesizing circuits is exceedingly difficulty. It is even
More informationThe P-vs-NP problem. Andrés E. Caicedo. September 10, 2011
The P-vs-NP problem Andrés E. Caicedo September 10, 2011 This note is based on lecture notes for the Caltech course Math 6c, prepared with A. Kechris and M. Shulman. 1 Decision problems Consider a finite
More informationComplexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler
Complexity Theory Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität Wien 15 May, 2018 Reinhard
More informationOutline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181.
Complexity Theory Complexity Theory Outline Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität
More informationAcknowledgments 2. Part 0: Overview 17
Contents Acknowledgments 2 Preface for instructors 11 Which theory course are we talking about?.... 12 The features that might make this book appealing. 13 What s in and what s out............... 14 Possible
More informationCOMP/MATH 300 Topics for Spring 2017 June 5, Review and Regular Languages
COMP/MATH 300 Topics for Spring 2017 June 5, 2017 Review and Regular Languages Exam I I. Introductory and review information from Chapter 0 II. Problems and Languages A. Computable problems can be expressed
More informationCellular automata are idealized models of complex systems Large network of simple components Limited communication among components No central
Cellular automata are idealized models of complex systems Large network of simple components Limited communication among components No central control Complex dynamics from simple rules Capability of information
More informationComputability Theory
Computability Theory Cristian S. Calude May 2012 Computability Theory 1 / 1 Bibliography M. Sipser. Introduction to the Theory of Computation, PWS 1997. (textbook) Computability Theory 2 / 1 Supplementary
More informationAlgorithmic Probability
Algorithmic Probability From Scholarpedia From Scholarpedia, the free peer-reviewed encyclopedia p.19046 Curator: Marcus Hutter, Australian National University Curator: Shane Legg, Dalle Molle Institute
More informationTheory of Computation Lecture Notes. Problems and Algorithms. Class Information
Theory of Computation Lecture Notes Prof. Yuh-Dauh Lyuu Dept. Computer Science & Information Engineering and Department of Finance National Taiwan University Problems and Algorithms c 2004 Prof. Yuh-Dauh
More informationArtificial Intelligence. 3 Problem Complexity. Prof. Dr. Jana Koehler Fall 2016 HSLU - JK
Artificial Intelligence 3 Problem Complexity Prof. Dr. Jana Koehler Fall 2016 Agenda Computability and Turing Machines Tractable and Intractable Problems P vs. NP Decision Problems Optimization problems
More informationII. Spatial Systems. A. Cellular Automata. Structure. Cellular Automata (CAs) Example: Conway s Game of Life. State Transition Rule
II. Spatial Systems A. Cellular Automata B. Pattern Formation C. Slime Mold D. Excitable Media A. Cellular Automata 1/18/17 1 1/18/17 2 Cellular Automata (CAs) Invented by von Neumann in 1940s to study
More information6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch
6.045: Automata, Computability, and Complexity (GITCS) Class 15 Nancy Lynch Today: More Complexity Theory Polynomial-time reducibility, NP-completeness, and the Satisfiability (SAT) problem Topics: Introduction
More informationLecture 3: Reductions and Completeness
CS 710: Complexity Theory 9/13/2011 Lecture 3: Reductions and Completeness Instructor: Dieter van Melkebeek Scribe: Brian Nixon Last lecture we introduced the notion of a universal Turing machine for deterministic
More informationComplex Systems Methods 2. Conditional mutual information, entropy rate and algorithmic complexity
Complex Systems Methods 2. Conditional mutual information, entropy rate and algorithmic complexity Eckehard Olbrich MPI MiS Leipzig Potsdam WS 2007/08 Olbrich (Leipzig) 26.10.2007 1 / 18 Overview 1 Summary
More informationCSE 200 Lecture Notes Turing machine vs. RAM machine vs. circuits
CSE 200 Lecture Notes Turing machine vs. RAM machine vs. circuits Chris Calabro January 13, 2016 1 RAM model There are many possible, roughly equivalent RAM models. Below we will define one in the fashion
More informationIntroduction to Languages and Computation
Introduction to Languages and Computation George Voutsadakis 1 1 Mathematics and Computer Science Lake Superior State University LSSU Math 400 George Voutsadakis (LSSU) Languages and Computation July 2014
More informationBBM402-Lecture 11: The Class NP
BBM402-Lecture 11: The Class NP Lecturer: Lale Özkahya Resources for the presentation: http://ocw.mit.edu/courses/electrical-engineering-andcomputer-science/6-045j-automata-computability-andcomplexity-spring-2011/syllabus/
More informationVariations of the Turing Machine
Variations of the Turing Machine 1 The Standard Model Infinite Tape a a b a b b c a c a Read-Write Head (Left or Right) Control Unit Deterministic 2 Variations of the Standard Model Turing machines with:
More informationLimits of Feasibility. Example. Complexity Relationships among Models. 1. Complexity Relationships among Models
Limits of Feasibility Wolfgang Schreiner Wolfgang.Schreiner@risc.jku.at Research Institute for Symbolic Computation (RISC) Johannes Kepler University, Linz, Austria http://www.risc.jku.at 1. Complexity
More informationLecture 25: Cook s Theorem (1997) Steven Skiena. skiena
Lecture 25: Cook s Theorem (1997) Steven Skiena Department of Computer Science State University of New York Stony Brook, NY 11794 4400 http://www.cs.sunysb.edu/ skiena Prove that Hamiltonian Path is NP
More informationbiologically-inspired computing lecture 6 Informatics luis rocha 2015 INDIANA UNIVERSITY biologically Inspired computing
lecture 6 -inspired Sections I485/H400 course outlook Assignments: 35% Students will complete 4/5 assignments based on algorithms presented in class Lab meets in I1 (West) 109 on Lab Wednesdays Lab 0 :
More informationLecture 20: conp and Friends, Oracles in Complexity Theory
6.045 Lecture 20: conp and Friends, Oracles in Complexity Theory 1 Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode:
More informationComputational Complexity
p. 1/24 Computational Complexity The most sharp distinction in the theory of computation is between computable and noncomputable functions; that is, between possible and impossible. From the example of
More informationSpace Complexity. The space complexity of a program is how much memory it uses.
Space Complexity The space complexity of a program is how much memory it uses. Measuring Space When we compute the space used by a TM, we do not count the input (think of input as readonly). We say that
More informationUniverse as quantum computer
Universe as quantum computer Seth Lloyd d Arbeloff Laboratory for Information Systems and Technology MIT 3-160, Cambridge, Mass. 02139 and The Santa Fe Institute slloyd@mit.edu Abstract: This paper shows
More informationIs there an Elegant Universal Theory of Prediction?
Is there an Elegant Universal Theory of Prediction? Shane Legg Dalle Molle Institute for Artificial Intelligence Manno-Lugano Switzerland 17th International Conference on Algorithmic Learning Theory Is
More informationComputability and Complexity Theory: An Introduction
Computability and Complexity Theory: An Introduction meena@imsc.res.in http://www.imsc.res.in/ meena IMI-IISc, 20 July 2006 p. 1 Understanding Computation Kinds of questions we seek answers to: Is a given
More informationPeter Wood. Department of Computer Science and Information Systems Birkbeck, University of London Automata and Formal Languages
and and Department of Computer Science and Information Systems Birkbeck, University of London ptw@dcs.bbk.ac.uk Outline and Doing and analysing problems/languages computability/solvability/decidability
More information20.1 2SAT. CS125 Lecture 20 Fall 2016
CS125 Lecture 20 Fall 2016 20.1 2SAT We show yet another possible way to solve the 2SAT problem. Recall that the input to 2SAT is a logical expression that is the conunction (AND) of a set of clauses,
More informationCS154, Lecture 17: conp, Oracles again, Space Complexity
CS154, Lecture 17: conp, Oracles again, Space Complexity Definition: conp = { L L NP } What does a conp computation look like? In NP algorithms, we can use a guess instruction in pseudocode: Guess string
More informationII. Spatial Systems A. Cellular Automata 8/24/08 1
II. Spatial Systems A. Cellular Automata 8/24/08 1 Cellular Automata (CAs) Invented by von Neumann in 1940s to study reproduction He succeeded in constructing a self-reproducing CA Have been used as: massively
More informationDRAFT. Diagonalization. Chapter 4
Chapter 4 Diagonalization..the relativized P =?NP question has a positive answer for some oracles and a negative answer for other oracles. We feel that this is further evidence of the difficulty of the
More informationComparison of several polynomial and exponential time complexity functions. Size n
Comparison of several polynomial and exponential time complexity functions Time complexity function n n 2 n 3 n 5 2 n 3 n Size n 10 20 30 40 50 60.00001.00002.00003.00004.00005.00006 second second second
More informationComputability and Complexity Theory
Discrete Math for Bioinformatics WS 09/10:, by A Bockmayr/K Reinert, January 27, 2010, 18:39 9001 Computability and Complexity Theory Computability and complexity Computability theory What problems can
More informationUndecidable Problems and Reducibility
University of Georgia Fall 2014 Reducibility We show a problem decidable/undecidable by reducing it to another problem. One type of reduction: mapping reduction. Definition Let A, B be languages over Σ.
More informationQ = Set of states, IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar
IE661: Scheduling Theory (Fall 2003) Primer to Complexity Theory Satyaki Ghosh Dastidar Turing Machine A Turing machine is an abstract representation of a computing device. It consists of a read/write
More informationCSE 4111/5111/6111 Computability Jeff Edmonds Assignment 3: Diagonalization & Halting Problem Due: One week after shown in slides
CSE 4111/5111/6111 Computability Jeff Edmonds Assignment 3: Diagonalization & Halting Problem Due: One week after shown in slides First Person: Second Person: Family Name: Family Name: Given Name: Given
More informationCS 350 Algorithms and Complexity
CS 350 Algorithms and Complexity Winter 2019 Lecture 15: Limitations of Algorithmic Power Introduction to complexity theory Andrew P. Black Department of Computer Science Portland State University Lower
More informationShannon Information (very briefly!) Lecture 4. Maximum and Minimum Entropy. Entropy. Entropy of Transition Rules. Entropy Examples
Lecture 4 9/4/07 1 Shannon Information (very briefly!) Information varies directly with surprise Information varies inversely with probability Information is additive The information content of a message
More informationIntroduction to Complexity Theory
Introduction to Complexity Theory Read K & S Chapter 6. Most computational problems you will face your life are solvable (decidable). We have yet to address whether a problem is easy or hard. Complexity
More informationMore Turing Machines. CS154 Chris Pollett Mar 15, 2006.
More Turing Machines CS154 Chris Pollett Mar 15, 2006. Outline Multitape Turing Machines Nondeterministic Turing Machines Enumerators Introduction There have been many different proposals for what it means
More informationbiologically-inspired computing lecture 12 Informatics luis rocha 2015 INDIANA UNIVERSITY biologically Inspired computing
lecture 12 -inspired Sections I485/H400 course outlook Assignments: 35% Students will complete 4/5 assignments based on algorithms presented in class Lab meets in I1 (West) 109 on Lab Wednesdays Lab 0
More informationCS 350 Algorithms and Complexity
1 CS 350 Algorithms and Complexity Fall 2015 Lecture 15: Limitations of Algorithmic Power Introduction to complexity theory Andrew P. Black Department of Computer Science Portland State University Lower
More informationCSC 1700 Analysis of Algorithms: P and NP Problems
CSC 1700 Analysis of Algorithms: P and NP Problems Professor Henry Carter Fall 2016 Recap Algorithmic power is broad but limited Lower bounds determine whether an algorithm can be improved by more than
More informationECS 120 Lesson 24 The Class N P, N P-complete Problems
ECS 120 Lesson 24 The Class N P, N P-complete Problems Oliver Kreylos Friday, May 25th, 2001 Last time, we defined the class P as the class of all problems that can be decided by deterministic Turing Machines
More informationFORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY
15-453 FORMAL LANGUAGES, AUTOMATA AND COMPUTABILITY KOLMOGOROV-CHAITIN (descriptive) COMPLEXITY TUESDAY, MAR 18 CAN WE QUANTIFY HOW MUCH INFORMATION IS IN A STRING? A = 01010101010101010101010101010101
More information6-1 Computational Complexity
6-1 Computational Complexity 6. Computational Complexity Computational models Turing Machines Time complexity Non-determinism, witnesses, and short proofs. Complexity classes: P, NP, conp Polynomial-time
More informationTuring Machines Part III
Turing Machines Part III Announcements Problem Set 6 due now. Problem Set 7 out, due Monday, March 4. Play around with Turing machines, their powers, and their limits. Some problems require Wednesday's
More informationLimitations of Algorithm Power
Limitations of Algorithm Power Objectives We now move into the third and final major theme for this course. 1. Tools for analyzing algorithms. 2. Design strategies for designing algorithms. 3. Identifying
More information1 Computational problems
80240233: Computational Complexity Lecture 1 ITCS, Tsinghua Univesity, Fall 2007 9 October 2007 Instructor: Andrej Bogdanov Notes by: Andrej Bogdanov The aim of computational complexity theory is to study
More informationA Tutorial on Computational Learning Theory Presented at Genetic Programming 1997 Stanford University, July 1997
A Tutorial on Computational Learning Theory Presented at Genetic Programming 1997 Stanford University, July 1997 Vasant Honavar Artificial Intelligence Research Laboratory Department of Computer Science
More informationan efficient procedure for the decision problem. We illustrate this phenomenon for the Satisfiability problem.
1 More on NP In this set of lecture notes, we examine the class NP in more detail. We give a characterization of NP which justifies the guess and verify paradigm, and study the complexity of solving search
More informationKolmogorov complexity and its applications
CS860, Winter, 2010 Kolmogorov complexity and its applications Ming Li School of Computer Science University of Waterloo http://www.cs.uwaterloo.ca/~mli/cs860.html We live in an information society. Information
More informationNotes for Lecture 3... x 4
Stanford University CS254: Computational Complexity Notes 3 Luca Trevisan January 18, 2012 Notes for Lecture 3 In this lecture we introduce the computational model of boolean circuits and prove that polynomial
More informationTheory of Computation. Theory of Computation
Theory of Computation Theory of Computation What is possible to compute? We can prove that there are some problems computers cannot solve There are some problems computers can theoretically solve, but
More informationIntelligent Agents. Formal Characteristics of Planning. Ute Schmid. Cognitive Systems, Applied Computer Science, Bamberg University
Intelligent Agents Formal Characteristics of Planning Ute Schmid Cognitive Systems, Applied Computer Science, Bamberg University Extensions to the slides for chapter 3 of Dana Nau with contributions by
More informationTheory of Computation CS3102 Spring 2014 A tale of computers, math, problem solving, life, love and tragic death
Theory of Computation CS3102 Spring 2014 A tale of computers, math, problem solving, life, love and tragic death Nathan Brunelle Department of Computer Science University of Virginia www.cs.virginia.edu/~njb2b/theory
More informationRecap DFA,NFA, DTM. Slides by Prof. Debasis Mitra, FIT.
Recap DFA,NFA, DTM Slides by Prof. Debasis Mitra, FIT. 1 Formal Language Finite set of alphabets Σ: e.g., {0, 1}, {a, b, c}, { {, } } Language L is a subset of strings on Σ, e.g., {00, 110, 01} a finite
More informationCellular Automata. and beyond. The World of Simple Programs. Christian Jacob
Cellular Automata and beyond The World of Simple Programs Christian Jacob Department of Computer Science Department of Biochemistry & Molecular Biology University of Calgary CPSC / MDSC 605 Fall 2003 Cellular
More informationCS Lecture 29 P, NP, and NP-Completeness. k ) for all k. Fall The class P. The class NP
CS 301 - Lecture 29 P, NP, and NP-Completeness Fall 2008 Review Languages and Grammars Alphabets, strings, languages Regular Languages Deterministic Finite and Nondeterministic Automata Equivalence of
More informationModelling with cellular automata
Modelling with cellular automata Shan He School for Computational Science University of Birmingham Module 06-23836: Computational Modelling with MATLAB Outline Outline of Topics Concepts about cellular
More informationPreface These notes were prepared on the occasion of giving a guest lecture in David Harel's class on Advanced Topics in Computability. David's reques
Two Lectures on Advanced Topics in Computability Oded Goldreich Department of Computer Science Weizmann Institute of Science Rehovot, Israel. oded@wisdom.weizmann.ac.il Spring 2002 Abstract This text consists
More informationChapter 2 Algorithms and Computation
Chapter 2 Algorithms and Computation In this chapter, we first discuss the principles of algorithm and computation in general framework, common both in classical and quantum computers, then we go to the
More informationPart I: Definitions and Properties
Turing Machines Part I: Definitions and Properties Finite State Automata Deterministic Automata (DFSA) M = {Q, Σ, δ, q 0, F} -- Σ = Symbols -- Q = States -- q 0 = Initial State -- F = Accepting States
More informationU.C. Berkeley CS278: Computational Complexity Professor Luca Trevisan August 30, Notes for Lecture 1
U.C. Berkeley CS278: Computational Complexity Handout N1 Professor Luca Trevisan August 30, 2004 Notes for Lecture 1 This course assumes CS170, or equivalent, as a prerequisite. We will assume that the
More informationTractability. Some problems are intractable: as they grow large, we are unable to solve them in reasonable time What constitutes reasonable time?
Tractability Some problems are intractable: as they grow large, we are unable to solve them in reasonable time What constitutes reasonable time?» Standard working definition: polynomial time» On an input
More informationInformation Theory and Coding Techniques: Chapter 1.1. What is Information Theory? Why you should take this course?
Information Theory and Coding Techniques: Chapter 1.1 What is Information Theory? Why you should take this course? 1 What is Information Theory? Information Theory answers two fundamental questions in
More informationComputational Complexity
Computational Complexity Problems, instances and algorithms Running time vs. computational complexity General description of the theory of NP-completeness Problem samples 1 Computational Complexity What
More informationPOLYNOMIAL SPACE QSAT. Games. Polynomial space cont d
T-79.5103 / Autumn 2008 Polynomial Space 1 T-79.5103 / Autumn 2008 Polynomial Space 3 POLYNOMIAL SPACE Polynomial space cont d Polynomial space-bounded computation has a variety of alternative characterizations
More informationEvolutionary Games and Computer Simulations
Evolutionary Games and Computer Simulations Bernardo A. Huberman and Natalie S. Glance Dynamics of Computation Group Xerox Palo Alto Research Center Palo Alto, CA 94304 Abstract The prisoner s dilemma
More informationComputability and Complexity
Computability and Complexity Lecture 5 Reductions Undecidable problems from language theory Linear bounded automata given by Jiri Srba Lecture 5 Computability and Complexity 1/14 Reduction Informal Definition
More informationAre chaotic systems dynamically random?
Are chaotic systems dynamically random? Karl Svozil Institute for Theoretical Physics, Technical University Vienna, Karlsplatz 13, A 1040 Vienna, Austria. November 18, 2002 Abstract Physical systems can
More informationIntroduction to Complexity Theory. Bernhard Häupler. May 2, 2006
Introduction to Complexity Theory Bernhard Häupler May 2, 2006 Abstract This paper is a short repetition of the basic topics in complexity theory. It is not intended to be a complete step by step introduction
More informationKolmogorov structure functions for automatic complexity
Kolmogorov structure functions for automatic complexity Bjørn Kjos-Hanssen June 16, 2015 Varieties of Algorithmic Information, University of Heidelberg Internationales Wissenschaftssentrum History 1936:
More informationCSCI FOUNDATIONS OF COMPUTER SCIENCE
1 CSCI- 2200 FOUNDATIONS OF COMPUTER SCIENCE Spring 2015 May 11, 2015 2 Announcements The bonus quiz points will be posted on LMS today. You can earn up to 3 bonus points on the final exam. I will have
More informationIntroduction to Turing Machines. Reading: Chapters 8 & 9
Introduction to Turing Machines Reading: Chapters 8 & 9 1 Turing Machines (TM) Generalize the class of CFLs: Recursively Enumerable Languages Recursive Languages Context-Free Languages Regular Languages
More informationCS 125 Section #10 (Un)decidability and Probability November 1, 2016
CS 125 Section #10 (Un)decidability and Probability November 1, 2016 1 Countability Recall that a set S is countable (either finite or countably infinite) if and only if there exists a surjective mapping
More informationTheory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death
Theory of Computation CS3102 Spring 2015 A tale of computers, math, problem solving, life, love and tragic death Robbie Hott www.cs.virginia.edu/~jh2jf Department of Computer Science University of Virginia
More informationLecture Notes 4. Issued 8 March 2018
CM30073 Advanced Algorithms and Complexity 1. Structure of the class NP Lecture Notes 4 Issued 8 March 2018 Recall that it is not known whether or not P = NP, the widely accepted hypothesis being that
More informationCSE 105 THEORY OF COMPUTATION
CSE 105 THEORY OF COMPUTATION Spring 2017 http://cseweb.ucsd.edu/classes/sp17/cse105-ab/ Today's learning goals Summarize key concepts, ideas, themes from CSE 105. Approach your final exam studying with
More informationconp, Oracles, Space Complexity
conp, Oracles, Space Complexity 1 What s next? A few possibilities CS161 Design and Analysis of Algorithms CS254 Complexity Theory (next year) CS354 Topics in Circuit Complexity For your favorite course
More informationThe purpose here is to classify computational problems according to their complexity. For that purpose we need first to agree on a computational
1 The purpose here is to classify computational problems according to their complexity. For that purpose we need first to agree on a computational model. We'll remind you what a Turing machine is --- you
More informationCHAPTER 3 FUNDAMENTALS OF COMPUTATIONAL COMPLEXITY. E. Amaldi Foundations of Operations Research Politecnico di Milano 1
CHAPTER 3 FUNDAMENTALS OF COMPUTATIONAL COMPLEXITY E. Amaldi Foundations of Operations Research Politecnico di Milano 1 Goal: Evaluate the computational requirements (this course s focus: time) to solve
More informationComputational Tasks and Models
1 Computational Tasks and Models Overview: We assume that the reader is familiar with computing devices but may associate the notion of computation with specific incarnations of it. Our first goal is to
More informationComputability Theory. CS215, Lecture 6,
Computability Theory CS215, Lecture 6, 2000 1 The Birth of Turing Machines At the end of the 19th century, Gottlob Frege conjectured that mathematics could be built from fundamental logic In 1900 David
More informationComplexity 6: AIT. Outline. Dusko Pavlovic. Kolmogorov. Solomonoff. Chaitin: The number of wisdom RHUL Spring Complexity 6: AIT.
Outline Complexity Theory Part 6: did we achieve? Algorithmic information and logical depth : Algorithmic information : Algorithmic probability : The number of wisdom RHUL Spring 2012 : Logical depth Outline
More information