THE MEASURE-THEORETIC ENTROPY OF LINEAR CELLULAR AUTOMATA WITH RESPECT TO A MARKOV MEASURE. 1. Introduction

Transcription

1 THE MEASURE-THEORETIC ENTROPY OF LINEAR CELLULAR AUTOMATA WITH RESPECT TO A MARKOV MEASURE HASAN AKIN Abstract. The purpose of this short paper is to compute the measure-theoretic entropy of the onedimensional linear cellular automata defined by bipermutative local rules on the ring Z m, m, with respect to a Markov measure generated by a stochastic matrix P and a probability vector π such that πp = π. Mathematics Subject Classification: Primary 8D0; Secondary 37A35, 37B40. Key words: Cellular automata, measure entropy, Markov measure.. Introduction Cellular automata (CAs for short), begun by Stanislaw Ulam and John von Neumann, are discrete models studied extensively in mathematics, physics, theoretical biology, computability theory and microstructure modelling with different purposes. Hedlund s paper [] started investigation of current problems in symbolic dynamics. CAs from the point of view of the ergodic theory have received remarkable attention in the last few years ([], [8], [3]). Shereshevsky [4] has defined nth iteration of a permutative CA and shown that if the local rule f is right (left) permutative, then its nth iteration also is right (left) permutative. For the definition and some properties of one-dimensional linear cellular automata (-D LCA) we refer to [0, ]. Mass and Martinez [3] have studied the dynamics of Markov measures by a particular linear cellular automata (LCA). They have reviewed some results on the evolution of probability measures under CA acting on a fullshift. The concept of entropy has been studied in computer science, mathematics, physics, chemistry, information theory and social sciences. In the last decades, a lot of works are devoted to this subject (see []-[7] and [9, ]). This notion first arose in thermodynamics as a measure of the heat absorbed (or emitted), when external work is done on a system. In mathematics, the notion of entropy of a dynamical system has been given by Kolmogorov []. This entropy is known as measure-theoretic entropy, Kolmogorov entropy, Kolmogorov-Sinai entropy, or just KS entropy. In this paper, we deal with KS entropy. In [], the author has computed the measure-theoretic entropy with respect to uniform Bernoulli measure for the case λ i =, for all i Z m. In [5], the author has computed the measure entropy of the -D LCA defined by a bipermutatibe local rule with respect to arbitrary Bernoulli measure. In this paper, we generalize the result obtained in [5] to any Markov measure generated by a stochastic matrix P = (p ij ) and probability vector π = (p i ). We compute the measure-theoretic entropy of the -D LCA T f[ l,r] defined by Date: March 7, 00.

2 HASAN AKIN a bipermutative local rule with respect to a Markov measure µ πp generated by a stochastic matrix P and a probability vector π such that πp = π. We show that if the local rule f is bipermutative, then we have h µπp (T f[ l,r] ) = (l + r) π i p ij log p ij.. Preliminaries and definitions.. The -D linear cellular automata. Let Z m = {0,,..., m } (m ) be a ring of the integers modulo m and Z Z m be the space of all doubly-infinite sequences x = (x n ) n= Z Z m and x n Z m. A CA can be defined as a homomorphism Z Z m with product topology. A CA is a continuous map T : Z Z m Z Z m defined by (T x) i = f(x i l,..., x i+r ), where f : Z r+l+ m Z m is a given local rule or map. Favati et al. [0] have stated that a local rule f is linear (additive) if and only if it can be written as (.) f(x l,..., x r ) = where at least one between λ l and λ r is nonzero. r λ i x i (mod m), i= l In this paper, we consider -D LCA T f[ l,r] defined by the local rule f in (.): (.) (T x) = (y n ) n=, y n = f(x n l,..., x n+r ) = where λ l,..., λ r Z m. r λ i x n+i (mod m), i= l We are going to use the notation T f[ l,r] for LCA-map defined in (.) to emphasize the local rule f and the numbers l and r. Definition.. The local rule defined by (.) is permutative in x j, l j r, iff for any given finite sequence x l,..., x j, x j+,..., x r Z l+r m we have {f(x l,..., x j, x j, x j+,..., x r ) : x j Z m } = Z m. The linear local rule f defined by (.) is permutative in the jth variable if and only if gcd(λ j, m) =, where gcd denotes the greatest common divisor. A local rule f is said to be right (respectively, left) permutative, if gcd(λ r, m) = (respectively, gcd(λ l, m) = ). It is said that f is bipermutative if it is both left and right permutative (see [] for a formal definition of permutivity)... Measure-theoretic entropy. Recall necessary definitions and Theorems (see [9] and [5] for details). Let X be a set and B be a σ-algebra of subsets of X. The pair (X, B) is called a measurable space. If the function µ : B [0, ] is a finite measure on the space (X, B), then (X, B, µ) is called probability space (or measure space). Let T : X X be a measure preserving transformation, then (X, B, µ, T ) is called measure theoretical dynamical system.

3 THE ENTROPY OF LCA W.C.T MARKOV MEASURE 3 If ξ = {A,..., A n } and β = {B,..., B m } are two measurable partitions of X, their refinement ξ β is the partition {A i B j : i =,..., n; j =,..., m}. Also, T ξ is the partition {T A,..., T A n }. Definition.. Let (X, B, µ, T ) be a measure-theoretical dynamical system and ξ be a measurable partition of X. The partition ξ is called a strong generator if T k ξ = B. Definition.3. Let ξ be a measurable partition of X. The quantity H µ (ξ) = A ξµ(a) log µ(a) is called the entropy of the partition ξ. Let ξ be a partition with finite entropy, then the quantity h µ (ξ, T ) = lim n is called the entropy of ξ with respect to T. The quantity n n H µ( T i ξ) i=0 h µ (T ) = sup{h µ (ξ, T ) : ξ is a partition with H µ (ξ) < } ξ is called the measure-theoretic entropy of (X, B, µ, T ), the entropy of T (with respect to µ). Theorem.4 ([5], Theorem 4.8). If T is a measure-preserving transformation (but not necessary invertible) of the probability space (X, B, µ) and if ξ is a finite partition of B with h µ (T ) = h µ (ξ, T ). T k ξ = B then Theorem.4 is known as Kolmogorov-Sinai theorem. They proved that if a partition ξ is a strong generator for T then h µ (T ) = h µ (T, ξ). 3. The main result In this section we study the measure-theoretic entropy of the -D LCA given in Eq. (.) with respect to Markov measure. In order to compute the entropy of the -D LCA over ring Z m (m ) we must define σ-algebra and the Markov measure. In symbolic dynamical system, it is well known that this σ-algebra B is generated by thin cylinder sets C = a [j 0, j,..., j s ] s+a = {x Z Z m : x a = j 0,..., x a+s = j s }, where j 0, j,..., j s Z m. Recall that a subshift of finite type σ : X X defined on a space (3.) X = {x Z Z m : M (xn,x n+) =, n Z} for some m m irreducible matrix M with entries either zero or unity. Let P = (p (i,j) ) denote a m m stochastic matrix (p (i,j) 0, j=0 p (i,j) = ) with entries p (i,j) = 0 iff M (i,j) = 0, let π = {π 0, π,..., π }

4 4 HASAN AKIN be its left eigenvector such that πp = π. Put a measure determined on cylinder sets by an m-dimensional probability vector π (the initial distribution) and a stochastic transition matrix P such that πp = π on the space Z Z m of sequences on an alphabet Z m with m symbols. A pair (π, P ) defines a set function µ πp on the cylinders of Z Z m. Recall that the associated Markov measure is defined as follows: µ πp ( 0 [i 0,..., i k ] k ) = π i0 p (i0,i )... p (ik,i k ) = π i0 p i0i... p ik i k, where the (i, j)-th element of the stochastic matrix P can be accepted as the transition probability from the state i to the state j. See [9], [3] and [5] for the properties of the Markov measure. We take definitions given in [, 5] again for the sake of the integrity of expression. Let us consider a particular case. Assume that the local rule f is bipermutative, so, we have the following Lemma. Lemma 3.. ([5], Lemma ) Suppose that f(x l,..., x r ) = r i= l λ i x i (mod m) is a bipermutative local rule, and ξ = { 0 [i] : 0 i < m} is a partition of Z Z m, then the partition ξ is a strong generator for -D LCA defined by the local rule f. From Lemma 3., one can show that the partition (l + r)n + : n T k f[ l,r] (ξ) consists of all cylinder sets of length ln[j ln, j ln+,..., j rn ] rn = {x Z Z m : x ln = j ln,..., x rn = j rn }, j ln,..., j rn Z m. The following Theorem is the main result of this short paper. Our main purpose here is to compute the measure entropy of a bipermutative LCA with respect to an arbitrary Markov measure. Theorem 3.. Let µ πp be a Markov measure given by the stochastic matrix P = (p ij ) and the probability vector π = (p i ). Assume that l and r are positive integers and gcd(λ l, m) =, gcd(λ r, m) =. Let T f[ l,r] be -D LCA defined by bipermutative local rule f. Then we have h µπp (T f[ l,r] ) = (l + r) π i p ij log p ij. Proof. Now we can calculate the measure entropy of the -D LCA by means of the Kolmogorov-Sinai Theorem ([5], p. 95), namely, h µπp (T f[ l,r] ) = h µπp (T f[ l,r], ξ). Let ξ be the zero-time partition of Z Z m: ξ = { 0 [i] : 0 i < m}, where 0 [i] = {x Z Z m : x 0 = i} is a cylinder set for all i, 0 i < m. So, we can state the partition ξ as follows: ξ = { 0 [0], 0 [],..., 0 [m ]}.

5 THE ENTROPY OF LCA W.C.T MARKOV MEASURE 5 From the definition of entropy we have So, we have H µπp (ξ) = µ πp ( 0 [i]) log µ πp ( 0 [i]) = π i log π i <. i=0 i=0 = = = where j=0 H µπp ( n T k f[ l,r] ξ) nl, nl+,...,nl=0 nl, nl+,...,nl=0 nl=0 π i nl p i nl i nl+ p inr i nr log π i nl p i nl i nl+ p inr i nr π i nl p i nl i nl+ p inr i nr [logπ i nl + logp i nl i nl+ + + log p inr i nr ] π nl logπ nl ((n(l + r) + ) p ij = and i=0 π i p ij = π j. π i p ij log p ij ), From Theorem.4 and Lemma 3., we have n h µπp (T f[ l,r] ) = h µπp (T f[ l,r], ξ) = lim n n H µ πp ( Therefore the proof is completed. = (l + r) π i p ij log p ij. Example 3.3. Let us consider stochastic matrix P = 3 3 T k f[ l,r] ξ). To compute π 0, π we write the equation πp = π. So, the probability vector corresponding to the stochastic matrix P can be obtained as π = ( 4 7, 3 7 ). Now, define local rule f as follows: f(x, x 0, x ) = x + x 0 + x (mod ). Let µ πp on Z Z be the Markov measure defined by stochastic matrix P and probability vector π associated to the matrix P. Then we have h µπp (T f[,] ) = π i p ij log p ij = log 7 4. In probability theory, it constitutes a measure of the uncertainty. In the information theory, it is known that this measure carries the amount of information of the system. Therefore, the value log bits is the amount of information that can be carried (or the amount of information obtained).

6 6 HASAN AKIN Example 3.4. Let f(x, x, x 0, x, x ) = x + 3x + x 0 + 3x (mod ). It is clear that this local rule is bipermutative. Let us consider the stochastic matrix (3.) P = To compute π 0, π, π, π 3 we write the equations πp = π:. (3.3) π 0 + π 8 + π 3 π 0 + π 6 7π 8 + π 6 7π 8 + π 3 = π 0 = π = π = π 3. From (3.3) we obtain π = 60 3 π 0, π = 56 3 π 0 and π 3 = 98 3 π 0. Since π 0 + π + π + π 3 =, we have π 0 = 3 37, π = 60 37, π = 56 37, π 3 = So, the probability vector corresponding to the matrix P given in (3.) is π = ( 3 37, 60 37, 56 37, ). From Theorem 3., we have h µπp (T f[,] ) = (l + r) 3 π i p ij log p ij = 4 3 π i i=0 j=0 3 p ij log p ij = 4[ log (654 log 03 log 7)] = 848, 3 log 7. Therefore, we conclude that the value 848, 3 log 7 is the amount of information that can be carried by the Markov measure µ πp over Z Z 4 defined by the stochastic matrix P and the probability vector π. The number (848, 3 log 7) can be interpreted as the average uncertainty about what symbol will appear next, given the one it is seen at present. A Markov measure on Z Z m is uniform, if measure of any one-dimensional cylinder is equal to m, where m is the cardinality of Z m. A doubly stochastic matrix is a matrix P such that P and P tr (transpose) are both stochastic. If a matrix P is a doubly stochastic then corresponding Markov measure is a uniform measure. Cardinality of Z m is equal to m, so that any doubly stochastic matrix P of m m size will generate uniform Markov measure.

7 THE ENTROPY OF LCA W.C.T MARKOV MEASURE 7 Corollary 3.5. Let µ πp be the uniform Markov measure on Z Z m and f(x l,..., x r ) = r i= l λ i x i (mod m), where f[ l, r] is bipermutative. Then measure-theoretic entropy of the -D LCA T f[ l,r] with respect to µ πp is equal to (l + r) log m. 4. Conclusion This paper contains the following results: We have found a generating partition for the -D LCA defined by a bipermutative local rule (Lemma 3.). We have calculated the measure-theoretic entropy of the -D LCA with respect to any Markov measure (Theorem 3.). We have illustrated the result by means of examples. This is the first step toward arbitrary Markov measure classification of multi-dimensional CA defined on alphabets of composite cardinality (or different ring [7]). In [], the author has computed the measure-theoretic entropy with respect to uniform Bernoulli measure for the case λ i =, for all i Z m. The author proved that the uniform Bernoulli measure is the maximal measure for these LCA. He also posed the question whether the maximal measure is unique. Thus, here a question raises: exist another Markov measure of maximal entropy there? Let f(x, x, x 0, x, x ) = x + 3x + x 0 + 3x + x (mod ). It is clear that this local rule is not bipermutative. Consider the stochastic matrix in (3.), can the measure entropy of the -D LCA T f[,] with respect to Markov measure µ πp be computed? This is an open problem. acknowledgements The author would like to thank the anonymous referees for their careful reading and useful comments which greatly improved the writing and presentation of the paper. References [] H. Akın, On the measure entropy of additive CA f, Entropy 5 (003) [] H. Akın, On the directional entropy of Z -actions generated by additive cellular automata, Appl. Math. Computation 70 () (005) [3] H. Akın, The topological entropy of nth iteration of an additive cellular automata, Appl. Math. Computation 74 () (006) [4] H. Akın, On directional entropy of A Z -action, Int. J. of Appl. Math. and Mech. () (006) [5] H. Akın, The entropy of a linear cellular automata with respect to any Bernoulli measure, Complex Systems 8 (009) [6] H. Akın, On the topological directional entropy, J. Computation and Appl. Math. 5 () (009) [7] H. Akın, I. Siap, On cellular automata over Galois rings, Inform. Process. Letters 03 () (007) 4-7. [8] G. Cattaneo, E. Formenti, G. Manzini, L. Margara, Ergodicity, transitivity, and regularity for linear cellular automata over Z m, Theoret. Comput. Sci. 33 (000) [9] M. Denker, C. Grillenberger and K. Sigmund, Ergodic theory on compact space, Springer Lecture Notes in Math. 57, 976. [0] P. Favati, G. Lotti, L. Margara, Additive one-dimensional cellular automata are chaotic according to Devaney s definition of chaos, Theor. Comput. Sci. 74 (997)

8 8 HASAN AKIN [] G.A. Hedlund, Endomorphisms and automorphisms of full shift dynamical system, Math. Syst. Theor. 3 (969) [] A.N. Kolmogorov, A new metric invariant of transitive dynamical systems and automorphisms of Lebesgue spaces, Dokl. Akad. Nauk SSSR 9 (958), [3] A. Maass and S. Martinez, Evolution of probability measures by cellular automata on algebraic topological Markov chains, Biol Res. 35 (003) 3-8. [4] M.A. Shereshevsky, Ergodic properties of certain surjective cellular automata, Mh. Math. 4 (99) [5] P. Walters, An Introduction to Ergodic Theory, Springer Graduate Texts in Math. 79 New York, 98. Hasan Akın, Department of Mathematics, Education Faculty, Zirve University, 760, Gaziantep, Turkey address: hasanakin69@gmail.com