Towards a Theory of Information Flow in the Finitary Process Soup

Transcription

1 Towards a Theory of in the Finitary Process Department of Computer Science and Complexity Sciences Center University of California at Davis June 1, 2010

2 Goals Analyze model of evolutionary self-organization in terms of information flow. Measure channel capacity of elementary. Develop functional partitioning based on this and language properties.

3 Outline

4 ǫ-machines ǫ-machines Defined T = {S,T } S is a set of causal states T is the set of transitions between them: T (s) ij, s A ǫ-machine Properties All of their recurrent states form a single strongly connected component. Transitions are deterministic. S is minimal: an ǫ-machine is the smallest causal representation of the transformation it implements.

5 Transducers We interpret the symbols labeling the transitions in the alphabet A as consisting of two parts: an input symbol that determines which transition to take from a state and an output symbol which is emitted on taking that transition. Transducers implement functions: Character to character Input string to output string Map sets to sets (languages)

6 Set of Machines

7 Transducer Composition Create new mapping, input language of one machine becomes input language of another Not commutative Possible exponential growth in number of states.

8 Interaction Network Interaction Matrix G (k) G (k) ij = { 1 iftk = T j T i 0 otherwise

9 Transition Matrix T 1 T 2 T 3 T 0 T 1 T 2 T 3 T 0 T 1 T 2 T 3 T 0 T 1 T 2 T 3 T 0 T 0 T 0 T 1 T 1 T 1 T 1 T 2 T 2 T 2 T 2 T 3 T 3 T 3 T 3 T 1 T 2 T 3 T 1 T 1 T 3 T 3 T 2 T 3 T 2 T 3 T 3 T 3 T 3 T 3 T 4 T 8 T 12 T 0 T 4 T 8 T 12 T 0 T 4 T 8 T 12 T 0 T 4 T 8 T 12 T 5 T 10 T 15 T 0 T 5 T 10 T 15 T 0 T 5 T 10 T 15 T 0 T 5 T 10 T 15 T 4 T 8 T 12 T 1 T 5 T 9 T 13 T 2 T 6 T 10 T 14 T 3 T 7 T 11 T 15 T 5 T 10 T 15 T 1 T 5 T 11 T 15 T 2 T 7 T 10 T 15 T 3 T 7 T 11 T 15 T 0 T 0 T 0 T 4 T 4 T 4 T 4 T 8 T 8 T 8 T 8 T 12 T 12 T 12 T 12 T 1 T 2 T 3 T 4 T 5 T 6 T 7 T 8 T 9 T 10 T 11 T 12 T 13 T 14 T 15 T 0 T 0 T 0 T 5 T 5 T 5 T 5 T 10 T 10 T 10 T 10 T 15 T 15 T 15 T 15 T 1 T 2 T 3 T 5 T 5 T 7 T 7 T 10 T 11 T 10 T 11 T 15 T 15 T 15 T 15 T 4 T 8 T 12 T 4 T 4 T 12 T 12 T 8 T 12 T 8 T 12 T 12 T 12 T 12 T 12 T 5 T 10 T 15 T 4 T 5 T 14 T 15 T 8 T 13 T 10 T 15 T 12 T 13 T 14 T 15 T 4 T 8 T 12 T 5 T 5 T 13 T 13 T 10 T 14 T 10 T 14 T 15 T 15 T 15 T 15 T 5 T 10 T 15 T 5 T 5 T 15 T 15 T 10 T 15 T 10 T 15 T 15 T 15 T 15 T 15

10 Interaction Network Interaction Graph G Nodes correspond to ǫ-transducers If T C = T B T A, place directed edge connecting node T A, to T C, labeled with the transforming machine T B.

11 T_12 T_8 T_14 T_6 T_1 T_4 T_13 T_15 T_3 T_7 T_11 T_1 T_4 T_5 T_2 T_8T_9T_10T_11 T_15 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_2 T_8 T_10 T_3 T_1 T_1T_5T_9T_13 T_10 T_4 T_5 T_6 T_7 T_2 T_2 T_4 T_3 T_8 T_5 T_12 T_5 T_2 T_7 T_9T_13 T_6 T_14 T_3 T_1 T_7 T_11 T_15 T_1 T_3 T_5 T_6 T_7 T_11 T_14 T_9T_11 T_7 T_11 T_12 T_13 T_14 T_15 T_2 T_9 T_10 T_10 T_3 T_7 T_11 T_12 T_13 T_14 T_15 T_1 T_4 T_5 T_3 T_7 T_11 T_15 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_4 T_15 T_8 T_10 T_15 T_9 T_6 T_6 T_15 T_13 T_6 T_14 T_5 T_10 T_9 T_7 T_2 T_13 T_6 T_9T_13 T_11 T_14 T_6 T_7 T_14 T_9T_11 T_10 T_8 T_10 T_4 T_3 T_4 T_13 T_14 T_10 T_8T_9T_10T_11 T_15 T_12 T_1 T_1T_5 T_12 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_8 T_12 T_2 T_8 T_10 T_13 T_14 T_15 T_14 T_5 T_1T_5T_9T_13 T_6 T_7 T_4 T_2 T_6 T_10 T_4 T_5 T_2 T_12 T_12 T_4 T_6 T_10 T_14 T_5 T_3 T_7 T_11 T_15 T_1T_5T_9T_13 T_3 T_4 T_5 T_4 T_5 T_6 T_7 T_1 T_2 T_8 T_8T_9T_10T_11 T_8 T_2 T_12 T_13 T_14 T_15 T_8 T_10 T_1 T_2 Interaction Network G

12 Population Population P N individuals

13 Population A single replication is determined through compositions and replacements in a two-step sequence: 1 Construct ǫ-machine T C by forming the composition T C = T B T A from T A and T B randomly selected from the population and minimizing. 2 Replace a randomly selected ǫ-machine, T D, with T C. Note that there is no imposed notion of fitness nor spatial component.

14 Relaxation to Steady State ( of Size 100,000) 0.25 T p 0.15 T 3, T 5, T 10, T T 1, T 2, T 4, T T 7, T 11, T 13, T 14 T 6, T t/n

15 Communication Discrete [Cover and Thomas, 2006] (X,p(y x), Y): (X and Y) are finite sets p(y x) are probability mass functions

16 Cover and Thomas [Cover and Thomas, 2006] define the channel capacity of a discrete memoryless channel as: C = max I(X;Y ) (1) p(x) This capacity specifies the highest rate, in bits, at which information may be reliably transmitted through the channel, and Shannon s second theorem states that his rate is achievable in practice.

17 Mutual Remember that mutual information is the reduction in uncertainty in one random variable due to knowledge of another. I(X;Y ) = H(Y ) H(Y X) (2)

18 Discrete Noiseless Discrete Noiseless One to one correspondence between input and output symbols

19 Languages based on language L L in = Σ L out = Σ union of these possibility for positive channel capacity

20 Capacities 1 bits P(X)

21 Machines of Maximal 1 bits P(X)

22 Partially Noisy s 1 bits P(X)

23 Partially Noisy s 1 bits P(X)

24 Capacities Most are zero, except for... Machine Number P(X=1)

25 Meta-machines Meta-machine A set of machines that is closed and self-maintained under composition Ω P is a meta-machine if and only if (i) T i T j Ω, for all T i,t j Ω and (ii) For all T k Ω, there exists T i,t j Ω, such that T k = T i T j. Captures notion of invariant set: Ω = G Ω

26 T_12 T_8 T_14 T_6 T_1 T_4 T_13 T_15 T_3 T_7 T_11 T_1 T_4 T_5 T_2 T_8T_9T_10T_11 T_15 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_2 T_8 T_10 T_3 T_1 T_1T_5T_9T_13 T_10 T_4 T_5 T_6 T_7 T_2 T_2 T_4 T_3 T_8 T_5 T_12 T_5 T_2 T_7 T_9T_13 T_6 T_14 T_3 T_1 T_7 T_11 T_15 T_1 T_3 T_6 T_7 T_11 T_14 T_9T_11 T_7 T_11 T_12 T_13 T_14 T_15 T_2 T_9 T_10 T_10 T_3 T_7 T_11 T_12 T_13 T_14 T_15 T_1 T_4 T_5 T_3 T_7 T_11 T_15 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_5 T_4 T_15 T_8 T_10 T_15 T_9 T_6 T_6 T_15 T_13 T_6 T_14 T_5 T_10 T_9 T_7 T_2 T_13 T_6 T_9T_13 T_11 T_14T_6 T_7 T_14 T_9T_11 T_10 T_8 T_10 T_4 T_3 T_4 T_13 T_14 T_10 T_8T_9T_10T_11 T_15 T_12 T_1 T_1T_5 T_12 T_3T_6T_7T_9T_11T_12T_13T_14T_15 T_8 T_12 T_2 T_8 T_10 T_13 T_14 T_15 T_14 T_5 T_6 T_7 T_1T_5T_9T_13 T_4 T_2 T_6 T_10 T_4 T_5 T_2 T_12 T_12 T_4 T_6 T_10 T_14 T_5 T_1T_5T_9T_13 T_3 T_3 T_7 T_11 T_15 T_4 T_5 T_4 T_5 T_6 T_7 T_1 T_2 T_8 T_8T_9T_10T_11 T_8 T_2 T_12 T_13 T_14 T_15 T_8 T_10 T_1 T_2

27 T A T D T E T C T O T O T L T C T A (0.125) T A T E T T L T O T J T H B T E (0.125) TCTO TA TD T E T D T E T B T J T O (0.225) T O T L T J TH T J (0.125) T B T J TH T A T D T E T C T O T O T L T C T C (0.125) T T L T O T J T H B T A T E T D (0.069) T D T E T B T J T H (0.069) T J T H T O T L T C T J T H T J (0.069) T D T E T J T B T B (0.069) T H T J [Crutchfield, 2006]

28 Summary capacity is a function of machine structure. High channel capacity does not necessarily lead to persistence. All of the transducers in the single state meta-machine have zero channel capacity. Composition never increases channel capacity.

29 For Further Reading T. M. Cover and J. A. Thomas Elements of Theory Wiley-Interscience, 2006 J. P. Crutchfield and O. Görnerup Objects That Make Objects: The Population of Structural Complexity Journal of the Royal Society Interface, 3 (2006)