Bio-inspired Models of Computation Seminar Università di Pisa 16 October 2009
Outline Introduction Motivation History Cellular Systems Wolfram Classes Variants and Extensions Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion
Introduction Motivation Part I Introduction
Motivation Evolution has rediscovered several times multicellularity as a way to build complex living systems: Introduction Motivation multicellular systems are composed by many copies of a unique fundamental unit, i.e. the cell; the local interaction between cells influences the fate and the behavior of each cell; the result is a heterogeneous system composed by differentiated cells that act as specialized units, even if they all contain the same genetic material and have essentially the same structure.
Fields of Application Introduction Motivation The concept of many simple systems with geometrically structured local interaction is relevant to: Artificial Life and Evolutionary Experiments, where it allows the definition of arbitrary synthetic universes. Computer Science and Technology for the implementation of parallel computing engines and the study of the rules of emergent computation.
Fields of Application Introduction Motivation Physics, Biology, and other sciences, for the modeling and simulation of complex biological, natural, and physical systems and phenomena, and research on the rules of structure and pattern formation: more generally, the study of complex systems, i.e., systems composed by many simple units that interact non-linearly. Mathematics, for the definition and exploration of complex space-time dynamics and of the behavior of dynamical systems.
Modeling Complex Phenomena Many complex phenomena are the result of the collective dynamics of a very large number of parts obeying simple rules. Introduction Motivation Unexpected global behaviors and patterns can emerge from the interaction of many systems that communicate only locally.
Part II History Cellular Systems Wolfram Classes Variants and Extensions
Historical Notes The best-known way in which cellular automata were introduced (and which eventually led to their name) was through work by John von Neumann in trying to develop an abstract model of self-reproduction in biology. Around 1947 - perhaps based on chemical engineering - von Neumann began by thinking about models based on 3D factories described by partial differential equations. History Cellular Systems Wolfram Classes Variants and Extensions
Historical Notes And following a 1951 suggestion from Stanislaw Ulam (who may have already independently considered the problem) he simplified his model and ended up with a 2D cellular automaton. Stanislaw Ulam and others used computers at Los Alamos to produce a handful of examples of what they called recursively defined geometrical objects - essentially the results of evolving generalized 2D cellular automata from single black cells. History Cellular Systems Wolfram Classes Variants and Extensions
Historical Notes In the 1970s a two-state, two-dimensional cellular automaton named Game of Life became very widely known, particularly among the early computing community. Invented by John Conway, and popularized by Martin Gardner in a Scientific American article. History Cellular Systems Wolfram Classes Variants and Extensions
Historical Notes In 1983 Stephen Wolfram published the first of a series of papers systematically investigating a very basic but essentially unknown class of cellular automata, which he terms elementary cellular automata. In 2002 he published his results in the 1280-page text A New Kind of Science, which extensively argued that the discoveries about cellular automata are not isolated facts but are robust and have significance for all disciplines of science. History Cellular Systems Wolfram Classes Variants and Extensions
Definitions Cell: a single element of a cellular space, the smallest unit of the space. Cellular Space: a lattice space made up of cells, each of which is in one of several predefined states. : a structure built in a cellular space, an automaton built out of cells. Local Rule: the rule governing the transition between states. The definition of a cell s finite state machine. It s called local, because it only uses the Neighborhood as it s input. Neighborhood: the cells surrounding a cell, that influence it s next state. The choice of neighborhood influences the behaviour of the cellular space. Configuration: a snapshot of all cell states, representing a single point in time. It s usually the starting point or a result of running a cellular space. History Cellular Systems Wolfram Classes Variants and Extensions
Modeling Cellular Systems We want to define the simplest nontrivial model of a cellular system. We base our model on the following concepts: Cell and cellular space; History Cellular Systems Wolfram Classes Variants and Extensions Neighborhood (local interaction); Cell state; Transition rule.
Modeling Cellular Systems We do not model all the details and characteristics of biological multicellular organisms but we obtain simple models where many interesting phenomena can still be observed. There are many kinds of cellular system models based on these concepts. The simplest model is called (CA). History Cellular Systems Wolfram Classes Variants and Extensions
Cellular Space History Cellular Systems Wolfram Classes Variants and Extensions
Neighborhood Informally, it is the set of cells that can influence directly a given cell. In homogeneous cellular models it has the same shape for all cells. History Cellular Systems Wolfram Classes Variants and Extensions
Examples of Classic Neighborhood History Cellular Systems Wolfram Classes Variants and Extensions
State Set and Transition Rule The value of the state of each cell belongs to a finite set, whose elements we can assume as being numbers. The value of the state is often represented by cell colors. There can be a special quiescent state s 0. History Cellular Systems Wolfram Classes Variants and Extensions
State Set and Transition Rule The transition rule is the fundamental element of the CA. It specifies the new state corresponding to each possible configuration of states of the cells in the neighborhood. The transition rule can be represented as a transition table, although this becomes rapidly impractical. History Cellular Systems Wolfram Classes Variants and Extensions
Boundary Conditions If the cellular space has a boundary, cells on the boundary may lack the cells required to form the prescribed neighborhood. Boundary conditions specify how to build a virtual neighborhood for boundary cells. History Cellular Systems Wolfram Classes Variants and Extensions
Initial Conditions History Cellular Systems Wolfram Classes Variants and Extensions
Example: Modeling Traffic We construct an elementary model of car motion in a single lane, based only on the local traffic conditions. The cars advance at discrete time steps and at discrete space intervals. A car can advance (and must advance) only if the destination interval is free. History Cellular Systems Wolfram Classes Variants and Extensions
Example: Traffic Jam Running the traffic CA with a high-density random initial distribution of cars we observe a phenomenon of backward propagation of a region of extreme traffic congestion (traffic jam). History Cellular Systems Wolfram Classes Variants and Extensions
In Practice To implement and run a CA experiment: 1. assign the geometry of the CA space. 2. assign the geometry of the neighborhood. 3. define the set of states of the cells. 4. assign the transition rule. 5. assign the boundary conditions. 6. assign the initial conditions of the CA. 7. repeatedly update all the cells of the CA, until some stopping condition is met: a pre-assigned number of steps is attained; the CA is in a quiescent state, or cycles in a loop. History Cellular Systems Wolfram Classes Variants and Extensions
Informal Definition of CA A is: a geometrically structured and discrete collection of identical (simple) systems called cells that interact only locally with each cell having a local state (memory) that can take a finite number of values and a (simple) rule used to update the state of all cells at discrete time steps and synchronously for all the cells of the automaton (global signal ) History Cellular Systems Wolfram Classes Variants and Extensions
Formal Definition of CA A is: an n-dimensional lattice of identical and synchronous finite state machines whose state s is updated (synchronously) following a transition function (or transition rule) φ that takes into account the state of the machines belonging to a neighborhood N of the machine, and whose geometry is the same for all machines History Cellular Systems Wolfram Classes Variants and Extensions s i (t + 1) = φ(s j (t)), s j N i
Special Rules The transition table of a generic CA can have an enormous number of entries. Special rules can have more compact definitions. A rule is totalistic if the new value of the state depends only on the sum of the values of the states of the cells in the neighborhood: s i (t + 1) = φ j s j (t), s j N i History Cellular Systems Wolfram Classes Variants and Extensions A rule is outer totalistic if the new value of the state depends on the value of the state of the updated cell and on the sum of the values of the states of the other cells in the neighborhood s i (t + 1) = φ s i (t), j s j (t), s j N i, j i
Rules for 1D CA History Cellular Systems Wolfram Classes Variants and Extensions k states, range (or radius) r; k k 2r+1 possible rules; e.g., k = 2, r = 1 256; k (2r+1)(k 1)+1 totalistic rules: e.g., k = 2, r = 1 16.
Wolfram Classes Wolfram defined four classes into which cellular automata can be divided depending on their behavior. In order of complexity the classes are: 1 Nearly all initial patterns evolve quickly into a stable, homogeneous state. Any randomness in the initial pattern disappears. 2 Nearly all initial patterns evolve quickly into stable or oscillating structures. Some of the randomness in the initial pattern may filtered out, but some remains. Local changes to the initial pattern tend to remain local. History Cellular Systems Wolfram Classes Variants and Extensions
Wolfram Classes 3 Nearly all initial patterns evolve in a pseudo-random or chaotic manner. Any stable structures that appear are quickly destroyed by the surrounding noise. Local changes to the initial pattern tend to spread indefinitely. 4 Nearly all initial patterns evolve into structures that interact in complex and interesting ways. Class 2 type stable or oscillating structures may be the eventual outcome, but the number of steps required to reach this state may be very large, even when the initial pattern is relatively simple. Local changes to the initial pattern may spread indefinitely. History Cellular Systems Wolfram Classes Variants and Extensions
Rule Code for Elementary CA Elementary CA: 256 1D binary CA (k = 2) with minimal range (r=1). Wolfram s Rule Code (here blue=0, yellow=1) History Cellular Systems Wolfram Classes Variants and Extensions
Examples of Elementary CA History Cellular Systems Wolfram Classes Variants and Extensions As said before, there are four qualitative behavioral classes: 1. uniform final state; 2. simple stable or periodic final state; 3. chaotic, random, non-periodic patterns; 4. complex, localized, propagating structures.
Examples of Application: RNG Rule 30 is used by Mathematica as its Random Number Generator (RNG are ubiquitous in bio-inspired experiments). History Cellular Systems Wolfram Classes Variants and Extensions
Variants and Extensions The basic CA is: discrete in space, time and state; updates all its cells synchronously; History Cellular Systems Wolfram Classes Variants and Extensions uses the same neighborhood geometry and transition rule for all cells. We can relinquish some of these prescriptions and obtain: Asynchronous CA (for example, mobile automata, where only one cell is active at each time step, and the transition rule specifies the fate of the activation); one problem for hardware implementation of cellular spaces is the synchronous operation that can limit the size of spaces that can be simulated.
Variants and Extensions Continuous-state CA (Coupled Map Lattices); replacing the discrete states with a continuous representation, we get a very interesting construction that can simulate processes in gases and fluids. Non-homogeneous (or non-uniform) CA; different rules for every cell; different rules for different regions (e.g., boundary cells); the rules could be determined according to some data external to the cellular space. History Cellular Systems Wolfram Classes Variants and Extensions
Probabilistic CA So far we have considered only deterministic CA. To model many phenomena it is useful to transition rules that depending on some externally assigned probability. Example: The forest fire model: Each cell contains a tree (green), a burning tree (yellow), or is empty (black). A burning tree becomes an empty cell. History Cellular Systems Wolfram Classes Variants and Extensions A green tree with at least a burning neighbor becomes a burning tree. A green tree without burning neighbors becomes a burning tree with probability f (probability of lightning). An empty cell grows a green tree with probability g (probability of growth). The parameters can be varied in a continuous range and introduce some continuity in the discrete world of CA models.
Probabilistic CA History Cellular Systems Wolfram Classes Variants and Extensions
Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Part III Conclusion Extended Topics
Garden of Eden Extended Topics A Garden of Eden configuration is a configuration which cannot appear on the lattice after one time step, no matter what the initial configuration. Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion Each Garden of Eden configuration contains at least one finite pattern with no predecessor: such a pattern is called an orphan.
Garden of Eden Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility By definition, a cellular automaton is surjective (i.e., its global mapping is onto), if and only if it has no Garden of Eden configuration. Conclusion The Garden of Eden theorem, due to Edward F. Moore and John Myhill, states that the class of surjective cellular automata and those which are injective over finite configurations coincide.
Garden of Eden Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility A cellular automaton has a Garden of Eden, if and only if it has two different finite configurations that evolve into the same configuration in one step. Conclusion As a corollary, every injective cellular automaton (i.e., one with one-to-one global mapping) is surjective, hence also bijective. However, note that surjective cellular automata do not need to be injective.
Garden of Eden Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion G F denotes the functions restricted to finite configurations, G P those restricted to the periodic ones and GoE the Garden of Eden configurations.
Garden of Eden Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility If there are two configurations which cannot be distinguished, there are Garden of Eden configurations. Conclusion For the converse proposition, suppose if possible that every pair of configurations cannot be distinguished, and that there exists a square Garden of Eden configuration G of side n.
Garden of Eden Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion If A is the number of states, ν the number of non GoE configuration: A (kn 2)2 ν (A n2 1) k2 which, for large k, contradicts Moore s inequality: (A n2 1) k2 < A (kn 2)2
Firing Squad Problem Proposed by John Myhill in 1957 and published (with a solution) in 1962 by Edward Moore: Consider a finite but arbitrary number of identical finite state machines (soldiers) arranged in a line; At time t = 0, each soldier is initialized to the quiescent (idle) state, except for the soldier on the far left (the general). Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion
Firing Squad Problem Extended Topics The state of each soldier at each discrete time-step t > 0 is dependent on its state and the state of its two neighbors at time t 1 (except for the two soldiers at either end, each of whose state depends only on itself and its sole neighbor). Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion In addition, if a soldier and its neighbors are in the quiescent state, then the soldier will remain quiescent at the next time-step. The problem is to define a finite set of states and state transition rules for the soldiers such that all soldiers enter a distinguished state (fire) at the same time and for the very first time.
Solution A solution using a minimal number of states was introduced by Jacques Mazoyer in 1988, whose solution uses only six states. Robert Balzer had earlier proven that no four-state solution exists. Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion Umeo, Yun es, and Kamikawa found 4-state FSSP solutions based on Wolfram s algebraic rule 60 and 150, respectively. The solutions can synchronize an infinite set of arrays, but not all. A solution using a minimal amount of time was later found by Professor E. Goto at MIT, whose solution uses thousands of states and requires exactly 2n 2 units of time for n soldiers. It is proven that a solution using a smaller amount of time cannot exist.
General Solution Extended Topics A general solution to the FSSP involves propagating two waves down the line of soldiers: a fast wave and a slow wave moving three times as slow. Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion The fast wave bounces of the other end of the line and meets the slow wave in the center. The two waves then split into four waves, a fast and slow wave moving in either direction from the cent-re, effectively splitting the line into two equal parts. This process continues, subdividing the line until each division is of length 1. At this moment, every soldier fires. This solution requires 3n units of time for n soldiers.
General Solution Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion
A Time-Optimal Solution The general sends to the right signals S 1, S 2, S 3,..., S i at speeds 1, 1 3, 1 7,..., 1 (2 i 1 1). The signal S 1 reflects at the right end of the line (like the fast wave in the preceding section), and meets signal S i (i > 2) at cell n 2 i 1. When S 1 reflects, it also creates a new general at the right end. Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion Signals S i are constructed using auxiliary signals, which propagate to the left. Every second time a signal moves (to the right), it sends an auxiliary signal to the left. S 1 moves on its own at speed 1 while each of the slower signals moves only when it gets an auxiliary signal.
General Solution Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion
General Solution Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion
Comparison of FSS Algorithms Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion
The Growth of Complexity Usually a machine produces machines less complex than itself: can we prove formally that there exist machines which can produce more complex machines? von Neumann s approach: a machine capable of self-reproduction would produce machines of equal complexity; if the self-reproduction process could tolerate some error (robust self-reproduction) then some of the resulting machines might have greater complexity than the original one. Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion
Self-Replicating Machine A self-replicating machine is an artificial construct that is theoretically capable of autonomously manufacturing a copy of itself using raw materials taken from its environment A detailed conceptual proposal for a physical non-biological self-replicating system was first put forward by John von Neumann in lectures delivered in 1948 and 1949, when he proposed a kinematic self-reproducing automaton model. Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion
Self-Reproducing Automata von Neumann defined an automaton composed by a universal constructor UC and a description D(M) of the machine to be generated. Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion
Self-Reproducing Automata Extended Topics Garden of Eden von Neumann automata is quite complex (29 states per cell, and about 200.000 active cells). states constituting means of signal carriage and logical operation, and acting upon signals represented as bit streams. Firing Squad Synchronization Problem Self-Reproducibility Conclusion A tape of cells encodes the sequence of actions to be performed by the machine. Using a writing head (termed a construction arm) the machine can print out (construct) a new pattern of cells, allowing it to make a complete copy of itself, and the tape.
von Neumann s Automaton Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion
Quine Extended Topics Garden of Eden A quine is a computer program which produces a copy of its own source code as its only output. A quine is a fixed point of an execution environment, when the execution environment is viewed as a function. Firing Squad Synchronization Problem Self-Reproducibility Conclusion Quines are possible in any programming language that has the ability to output any computable string, as a direct consequence of Kleene s recursion theorem. A quine takes no input: allowing input would permit the source code to be fed to the program via the keyboard, opening the source file of the program, and similar mechanisms.
Quine We want the quine to be correct C code, so it has to begin something like this: #include <stdio.h> int main (void) { Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion The first thing we want to do is print all what precedes. We could write: printf("#include <stdio.h>\n\nint\nmain (void)\n{\n"); Then we need to print this line itself: printf("printf(\"#include <stdio.h>\\n\\nint\\nmain (void)\\n{\\n\");\n"); And so on. It should be obvious that this is not going to work.
Quine Extended Topics Garden of Eden The problem is that we need to print something, so we use a character string (s) to print it, and then we need to print s itself, so we use another character string, and so on... Firing Squad Synchronization Problem Self-Reproducibility Conclusion If we intend to print s, we don t need another string: we can use s itself. So let s give it another try: char *s="#include <stdio.h>\n\nint\nmain (void)\n{\n"; printf(s); printf("char *s=\"%s\";\n",s); Central idea: it is possible to reuse the data used to represent the code to print the data themselves.
Quine Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Now we re using s as it stands, but that won t work because it contains some backslashes. Conclusion Since we are writing in C, we can exploit some properties of the printf function: char *s1="#include <stdio.h>%c%cint%cmain (void)%c{%c"; char *s2=" char *s1=%c%s%c;%c char *s2=%c%s%c;%c"; char n= \n, q= " ; printf(s1,n,n,n,n,n); printf(s2,q,s1,q,n,q,s2,q,n)
Quine A complete example: #include <stdio.h> int main (void) { char *s1="#include <stdio.h>%c%cint%cmain (void)%c{%c"; char *s2=" char *s%c=%c%s%c;%c char *s%c=%c%s%c;%c"; char *s3=" char n= %cn, q= %c, b= %c%c ;%c"; char *sp=" printf("; char *s4="%ss1,n,n,n,n,n);%c"; char *s5="%ss2, 1,q,s1,q,n, 2,q,s2,q,n);%ss2, 3,q,s3,q,n, p,q,sp,q,n);%c"; char *s6="%ss2, 4,q,s4,q,n, 5,q,s5,q,n);%ss2, 6,q,s6,q,n, 7,q,s7,q,n);%c"; char *s7="%ss2, 8,q,s8,q,n, 9,q,s9,q,n);%ss2, 0,q,s0,q,n, x,q,sx,q,n);%c"; char *s8="%ss3,b,q,b,b,n);%ss4,sp,n);%ss5,sp,sp,n);%c"; char *s9="%ss6,sp,sp,n);%ss7,sp,sp,n);%ss8,sp,sp,sp,n);%c"; char *s0="%ss9,sp,sp,sp,n);%ss0,sp,sp,n,n,n);%c return 0;%c}%c"; char *sx="--- This is an intron. ---"; char n= \n, q= ", b= \\ ; printf(s1,n,n,n,n,n); printf(s2, 1,q,s1,q,n, 2,q,s2,q,n); printf(s2, 3,q,s3,q,n, p,q,sp,q,n); printf(s2, 4,q,s4,q,n, 5,q,s5,q,n); printf(s2, 6,q,s6,q,n, 7,q,s7,q,n); printf(s2, 8,q,s8,q,n, 9,q,s9,q,n); printf(s2, 0,q,s0,q,n, x,q,sx,q,n); printf(s3,b,q,b,b,n); printf(s4,sp,n); printf(s5,sp,sp,n); printf(s6,sp,sp,n); printf(s7,sp,sp,n); printf(s8,sp,sp,sp,n); printf(s9,sp,sp,sp,n); printf(s0,sp,sp,n,n,n); return 0; } Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion > gcc quine.c -o quine >./quine > quine2.c > diff quine.c quine2.c >
References [1] Harald Niesche Introduction to [2] Marianne Delorme Cellular Automta: a parallel device, Mathematics and Its Application, Kluwer: 5-49, 1998. Extended Topics Garden of Eden Firing Squad Synchronization Problem Self-Reproducibility Conclusion [3] Palash Sarkar A Brief History of ACM Computing Surveys (CSUR), Volume 32, Issue 1 (March 2000), pp. 80-107 [4] Stephen Wolfram Universality and complexity in cellular automata Physica D: Nonlinear Phenomena, Vol. 10, No. 1-2. (January 1984), pp. 1-35. [5] Kojiro Kobayashi and Darin Goldstei On Formulations of Firing Squad Synchronization Problems LCNS 3699, pp. 157-168 [6] Umeo, Hiroshi and Yanagihara, Takashi A Small Five-State Non-Optimum-Time Solution to the Firing Squad Synchronization Problem: A Geometrical Approach Fundamenta Informaticae, Volume 91, Issue 1 (January 2009) [7] Myhill, J. The converse of Moore s Garden-of-Eden theorem Proceedings of the American Mathematical Society 14: 685-686.