Towards a Theory of in the Finitary Process Department of Computer Science and Complexity Sciences Center University of California at Davis June 1, 2010
Goals Analyze model of evolutionary self-organization in terms of information flow. Measure channel capacity of elementary. Develop functional partitioning based on this and language properties.
Outline 1 2 3 4 5 6
ǫ-machines ǫ-machines Defined T = {S,T } S is a set of causal states T is the set of transitions between them: T (s) ij, s A ǫ-machine Properties All of their recurrent states form a single strongly connected component. Transitions are deterministic. S is minimal: an ǫ-machine is the smallest causal representation of the transformation it implements.
Transducers We interpret the symbols labeling the transitions in the alphabet A as consisting of two parts: an input symbol that determines which transition to take from a state and an output symbol which is emitted on taking that transition. Transducers implement functions: Character to character Input string to output string Map sets to sets (languages)
Set of Machines 15 0 0 0 1 1 0 1 1 14 0 1 1 0 1 1 13 0 0 1 0 1 1 12 1 0 1 1 11 0 0 0 1 1 1 10 0 1 1 1 9 0 0 1 1 8 1 1 7 0 0 0 1 1 0 6 0 1 1 0 5 0 0 1 0 4 1 0 3 0 0 0 1 2 0 1 1 0 0
Transducer Composition Create new mapping, input language of one machine becomes input language of another Not commutative Possible exponential growth in number of states.
Interaction Network Interaction Matrix G (k) G (k) ij = { 1 iftk = T j T i 0 otherwise
Transition Matrix T 1 T 2 T 3 T 0 T 1 T 2 T 3 T 0 T 1 T 2 T 3 T 0 T 1 T 2 T 3 T 0 T 0 T 0 T 1 T 1 T 1 T 1 T 2 T 2 T 2 T 2 T 3 T 3 T 3 T 3 T 1 T 2 T 3 T 1 T 1 T 3 T 3 T 2 T 3 T 2 T 3 T 3 T 3 T 3 T 3 T 4 T 8 T 12 T 0 T 4 T 8 T 12 T 0 T 4 T 8 T 12 T 0 T 4 T 8 T 12 T 5 T 10 T 15 T 0 T 5 T 10 T 15 T 0 T 5 T 10 T 15 T 0 T 5 T 10 T 15 T 4 T 8 T 12 T 1 T 5 T 9 T 13 T 2 T 6 T 10 T 14 T 3 T 7 T 11 T 15 T 5 T 10 T 15 T 1 T 5 T 11 T 15 T 2 T 7 T 10 T 15 T 3 T 7 T 11 T 15 T 0 T 0 T 0 T 4 T 4 T 4 T 4 T 8 T 8 T 8 T 8 T 12 T 12 T 12 T 12 T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 T 11 T 12 T 13 T 14 T 15 T 0 T 0 T 0 T 5 T 5 T 5 T 5 T 10 T 10 T 10 T 10 T 15 T 15 T 15 T 15 T 1 T 2 T 3 T 5 T 5 T 7 T 7 T 10 T 11 T 10 T 11 T 15 T 15 T 15 T 15 T 4 T 8 T 12 T 4 T 4 T 12 T 12 T 8 T 12 T 8 T 12 T 12 T 12 T 12 T 12 T 5 T 10 T 15 T 4 T 5 T 14 T 15 T 8 T 13 T 10 T 15 T 12 T 13 T 14 T 15 T 4 T 8 T 12 T 5 T 5 T 13 T 13 T 10 T 14 T 10 T 14 T 15 T 15 T 15 T 15 T 5 T 10 T 15 T 5 T 5 T 15 T 15 T 10 T 15 T 10 T 15 T 15 T 15 T 15 T 15
Interaction Network Interaction Graph G Nodes correspond to ǫ-transducers If T C = T B T A, place directed edge connecting node T A, to T C, labeled with the transforming machine T B.
T_12 T_8 T_14 T_6 T_1 T_4 T_13 T_15 T_3 T_7 T_11 T_1 T_4 T_5 T_2 T_8T_9T_10T_11 T_15 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_2 T_8 T_10 T_3 T_1 T_1T_5T_9T_13 T_10 T_4 T_5 T_6 T_7 T_2 T_2 T_4 T_3 T_8 T_5 T_12 T_5 T_2 T_7 T_9T_13 T_6 T_14 T_3 T_1 T_7 T_11 T_15 T_1 T_3 T_5 T_6 T_7 T_11 T_14 T_9T_11 T_7 T_11 T_12 T_13 T_14 T_15 T_2 T_9 T_10 T_10 T_3 T_7 T_11 T_12 T_13 T_14 T_15 T_1 T_4 T_5 T_3 T_7 T_11 T_15 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_4 T_15 T_8 T_10 T_15 T_9 T_6 T_6 T_15 T_13 T_6 T_14 T_5 T_10 T_9 T_7 T_2 T_13 T_6 T_9T_13 T_11 T_14 T_6 T_7 T_14 T_9T_11 T_10 T_8 T_10 T_4 T_3 T_4 T_13 T_14 T_10 T_8T_9T_10T_11 T_15 T_12 T_1 T_1T_5 T_12 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_8 T_12 T_2 T_8 T_10 T_13 T_14 T_15 T_14 T_5 T_1T_5T_9T_13 T_6 T_7 T_4 T_2 T_6 T_10 T_4 T_5 T_2 T_12 T_12 T_4 T_6 T_10 T_14 T_5 T_3 T_7 T_11 T_15 T_1T_5T_9T_13 T_3 T_4 T_5 T_4 T_5 T_6 T_7 T_1 T_2 T_8 T_8T_9T_10T_11 T_8 T_2 T_12 T_13 T_14 T_15 T_8 T_10 T_1 T_2 Interaction Network G
Population Population P N individuals
Population A single replication is determined through compositions and replacements in a two-step sequence: 1 Construct ǫ-machine T C by forming the composition T C = T B T A from T A and T B randomly selected from the population and minimizing. 2 Replace a randomly selected ǫ-machine, T D, with T C. Note that there is no imposed notion of fitness nor spatial component.
Relaxation to Steady State ( of Size 100,000) 0.25 T 15 0.2 p 0.15 T 3, T 5, T 10, T 12 0.1 T 1, T 2, T 4, T 8 0.05 0 T 7, T 11, T 13, T 14 T 6, T 9 0 2 4 6 8 10 t/n
Communication Discrete [Cover and Thomas, 2006] (X,p(y x), Y): (X and Y) are finite sets p(y x) are probability mass functions
Cover and Thomas [Cover and Thomas, 2006] define the channel capacity of a discrete memoryless channel as: C = max I(X;Y ) (1) p(x) This capacity specifies the highest rate, in bits, at which information may be reliably transmitted through the channel, and Shannon s second theorem states that his rate is achievable in practice.
Mutual Remember that mutual information is the reduction in uncertainty in one random variable due to knowledge of another. I(X;Y ) = H(Y ) H(Y X) (2)
Discrete Noiseless Discrete Noiseless One to one correspondence between input and output symbols
Languages based on language L L in = Σ L out = Σ union of these possibility for positive channel capacity
Capacities 1 bits 0.5 0 1 0.5 0 1 0.5 0 0 0.5 1 0 0.5 1 0 0.5 1 0 0.5 1 0 0.5 1 P(X)
Machines of Maximal 1 bits 0.5 0 0 0.5 1 P(X)
Partially Noisy s 1 bits 0.5 0 0 0.5 1 P(X)
Partially Noisy s 1 bits 0.5 0 0 0.5 1 P(X)
Capacities Most are zero, except for... Machine Number P(X=1) 6 1.0 0.5 7 0.32 0.6 9 1.0 0.5 11 0.32 0.6 13 0.32 0.4 14 0.32 0.4
Meta-machines Meta-machine A set of machines that is closed and self-maintained under composition Ω P is a meta-machine if and only if (i) T i T j Ω, for all T i,t j Ω and (ii) For all T k Ω, there exists T i,t j Ω, such that T k = T i T j. Captures notion of invariant set: Ω = G Ω
T_12 T_8 T_14 T_6 T_1 T_4 T_13 T_15 T_3 T_7 T_11 T_1 T_4 T_5 T_2 T_8T_9T_10T_11 T_15 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_2 T_8 T_10 T_3 T_1 T_1T_5T_9T_13 T_10 T_4 T_5 T_6 T_7 T_2 T_2 T_4 T_3 T_8 T_5 T_12 T_5 T_2 T_7 T_9T_13 T_6 T_14 T_3 T_1 T_7 T_11 T_15 T_1 T_3 T_6 T_7 T_11 T_14 T_9T_11 T_7 T_11 T_12 T_13 T_14 T_15 T_2 T_9 T_10 T_10 T_3 T_7 T_11 T_12 T_13 T_14 T_15 T_1 T_4 T_5 T_3 T_7 T_11 T_15 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_5 T_4 T_15 T_8 T_10 T_15 T_9 T_6 T_6 T_15 T_13 T_6 T_14 T_5 T_10 T_9 T_7 T_2 T_13 T_6 T_9T_13 T_11 T_14T_6 T_7 T_14 T_9T_11 T_10 T_8 T_10 T_4 T_3 T_4 T_13 T_14 T_10 T_8T_9T_10T_11 T_15 T_12 T_1 T_1T_5 T_12 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_8 T_12 T_2 T_8 T_10 T_13 T_14 T_15 T_14 T_5 T_6 T_7 T_1T_5T_9T_13 T_4 T_2 T_6 T_10 T_4 T_5 T_2 T_12 T_12 T_4 T_6 T_10 T_14 T_5 T_1T_5T_9T_13 T_3 T_3 T_7 T_11 T_15 T_4 T_5 T_4 T_5 T_6 T_7 T_1 T_2 T_8 T_8T_9T_10T_11 T_8 T_2 T_12 T_13 T_14 T_15 T_8 T_10 T_1 T_2
T A T D T E T C T O T O T L T C T A (0.125) T A T E T T L T O T J T H B T E (0.125) TCTO TA TD T E T D T E T B T J T O (0.225) T O T L T J TH T J (0.125) T B T J TH T A T D T E T C T O T O T L T C T C (0.125) T T L T O T J T H B T A T E T D (0.069) T D T E T B T J T H (0.069) T J T H T O T L T C T J T H T J (0.069) T D T E T J T B T B (0.069) T H T J [Crutchfield, 2006]
Summary capacity is a function of machine structure. High channel capacity does not necessarily lead to persistence. All of the transducers in the single state meta-machine have zero channel capacity. Composition never increases channel capacity.
For Further Reading T. M. Cover and J. A. Thomas Elements of Theory Wiley-Interscience, 2006 J. P. Crutchfield and O. Görnerup Objects That Make Objects: The Population of Structural Complexity Journal of the Royal Society Interface, 3 (2006) 345-349.