Effective Dimensions and Relative Frequencies

Transcription

1 Effective Dimensions and Relative Frequencies Xiaoyang Gu and Jack H Lutz Department of Computer Science, Iowa State University, Ames, IA 500 USA xiaoyang,lutz}@csiastateedu Abstract Consider the problem of calculating the fractal dimension of a set X consisting of all infinite sequences S over a finite alphabet Σ that satisfy some given condition P on the asymptotic frequencies with which various symbols from Σ appear in S Solutions to this problem are known in cases where (i) the fractal dimension is classical (Hausdorff or packing dimension), or (ii) the fractal dimension is effective (even finite-state) and the condition P completely specifies an empirical distribution π over Σ, ie, a limiting frequency of occurrence for every symbol in Σ In this paper we show how to calculate the finite-state dimension (equivalently, the finite-state compressibility) of such a set X when the condition P only imposes partial constraints on the limiting frequencies of symbols Our results automatically extend to less restrictive effective fractal dimensions (eg, polynomial-time, computable, and constructive dimensions), and they have the classical results (i) as immediate corollaries Our methods are nevertheless elementary and, in most cases, simpler than those by which the classical results were obtained Keywords: effective fractal dimensions, empirical frequencies, finite-state dimension, randomness, saturated sets This author s research was supported in part by National Science Foundation Grants , and , and by Spanish Government MEC Projects TIC C03-03 and TIN C03-02 Part of the results were announced (without proceedings) at the special session on Randomness in Computation in Fall 2005 Central Section Meeting of the American Mathematical Society corresponding author

2 Introduction The most fundamental statistics used in the analysis of data for purposes of compression or prediction are the empirical frequencies with which various symbols appear When every symbol has a frequency that is known and stable throughout the data, the problems of compression and prediction are well understood, with the main insights now over a half-century old [5, 6, 5] However, when only partial constraints on the empirical frequencies eg, the relative frequencies of some of the symbols are known, these problems become more challenging This paper shows how to calculate the finite-state dimension (equivalently, the compressibility or predictability by finite-state machines [6, 4]) of a set X of infinite sequences over a finite alphabet Σ when membership of a sequence S in X is determined by some given condition P on the asymptotic frequencies with which various symbols from Σ appear in S Our results hold even when P only imposes partial constraints on the limiting frequencies of symbols, and they automatically extend to less restrictive effective dimensions, such as polynomialtime, computable, and constructive dimensions In order to explain our results and their significance, we briefly review four lines of research that are precursors of our work Classical fractal dimensions In 99, Hausdorff [3] developed a rigorous way of assigning a dimension to every subset of an arbitrary metric space His definition agrees with the intuitive notion of dimension for smooth sets (eg, smooth curves have dimension ; smooth surfaces have dimension 2), but assigns non-integer dimensions to some more exotic sets, and hence came to be called a fractal dimension In 949, Eggleston [7], building on work of Besicovitch [3] and Good [0], proved that, for any probability measure π on a finite alphabet Σ, the set of all sequences in which each symbol a Σ has asymptotic frequency π(a) has Hausdorff dimension H Σ (π), the Shannon entropy of π, normalized to range over [0, ] In retrospect, Hausdorff dimension is an information-theoretic concept [27], but these developments essentially all took place prior to Shannon s development of information theory [28] In the early 980 s, another fractal dimension called packing dimension, was introduced [30, 29] Packing dimension agrees with Hausdorff on regular sets, but is larger on some sets [9] 2 Shannon information theory In 948, Shannon [28] developed a probabilistic theory of information (Shannon entropy) that has been enormously productive and is the setting in which most work on compression and prediction has been carried out [5]

3 3 Effective fractal dimensions In 2000, Lutz [8, 9] proved a new characterization of Hausdorff dimension in terms of betting strategies and used this characterization to formulate effective fractal dimensions ranging from polynomial-time and polynomial-space dimensions to computable and constructive dimensions Pushing this effort further, finite-state dimension was introduced the following year [6], and is now known to characterize the compressibility [6] and predictability [4] of sequences over finite alphabets In [], packing dimension was shown to have a betting-strategy characterization that is exactly dual to that of Hausdorff dimension, thereby giving dual strong dimensions at each of the levels of effectivity for which dimensions had been defined Each of the papers mentioned here extended the above-mentioned result of Eggleston to the effective dimension(s) introduced Hence, as indicated in our first paragraph, the compression and prediction problems are well understood, even at the finite-state level, when a set X of sequences is defined in terms of given, well-defined, asymptotic frequencies of all symbols 4 Classical dimensions of saturated sets In 2002, Barreira, Saussol, and Schmeling [2] considered the classical fractal dimensions of sets of sequences defined in terms of conditions placing (typically partial) constraints on the frequencies and relative frequencies of symbols The example by which they introduced their work was the set X of all sequences over the alphabet 0,, 2, 3} in which there are asymptotically five times as many 0 s as s (No constraint is placed on the frequency of any individual symbol) Using sophisticated techniques (multifractal analysis and ergodic theory), they showed how to compute the classical Hausdorff dimensions of sets of this kind As it turns out, Volkmann [3] and his student Cajar [4] had previously defined a set X of sequences to be saturated if membership in it is completely determined by the asymptotic behaviors (not necessarily convergent) of the frequencies of symbols and investigated the Hausdorff dimensions of many saturated sets Olsen [2 24] and Olsen and Winter [25, 26] also used multifractal analysis to study such sets 5 Our results We show how to calculate the finite-state dimensions of saturated sets We give a pointwise characterization of the dimensions of such sets, and we prove a general correspondence principle stating that, if X is any saturated set, then the finite-state dimension of X is exactly its classical Hausdorff dimension, and the finite-state strong dimension of X is exactly its classical packing dimension We also give completely elementary methods (no multifractal analysis or ergodic theory) for computing the finite-state dimensions of various types of saturated sets By our correspondence principle, this yields elementary proofs that these results also hold for classical fractal dimensions and less restrictive effective fractal dimensions

4 The rest of this paper is organized as follows Section 2 lists the basic definitions and conventions we use in this paper Section 3 reviews the definitions of Hausdorff dimension, packing dimension, finite-state dimension, and finite-state strong dimension We give a few example of calculating the dimensions of exotic saturated sets in Section 4 In Section 5, we discuss finite-state dimensions of saturated sets in detail and give insight into why a maximum entropy principle holds 2 Preliminaries Let m 2 be an integer We work with the m-ary alphabet Σ m = 0,,, m } Σm is the set of all (finite) strings on Σ m including the empty string λ C m = Σm is the of all (infinite) m-ary sequences C = C 2 is the Cantor space (Σ m ) is the set of all probability measures on Σ m Let i be an integer such that 0 i m The symbol counting function # i : (C m Σm) N N is defined such that for every string or sequence S and n N, # i (S, n) is the number of occurrences of i in the first n bits of S The symbol frequency function π i : (C m Σm) N [0, ] is defined such that π i (S, n) = # i (S, n)/n The empirical measure function π : (C m Σm) N (Σ m ) is defined such that π(s, n) = (π 0 (S, n),, π m (S, n)) Intuitively, π extracts empirical probability measures from the first n bits of a string or a sequence based on the actual frequencies of digits 3 The Four Dimensions Hausdorff dimension and packing dimension are important tools in mathematics used to study the size of sets and the properties of dynamic systems All countable sets have 0 for both of these dimensions In order to study relative size of countable sets from the eyes of computers with different resources, Lutz generalized Hausdorff dimension to effective dimensions by using his gale characterization of Hausdorff dimension [8] Athreya, Hitchcock, Lutz, and Mayordomo then gave a dual gale characterization of packing dimension, with which, they generalized packing dimension to effective strong dimensions [] We first review the definitions related to gales Note that Σ m is an alphabet with m symbols and m 2 Definition Let s [0, ) An s-supergale is a function d : Σm [0, ) such that for all w Σm m s d(w) a Σ m d(wa) The success set of an s-supergale d is S [d] = S C lim sup d(s[0n ]) = } The strong success set of d is Sstr[d] = S C lim inf d(s[0n ]) = } Now we conveniently give the gale characterizations of Hausdorff and packing dimensions as definitions Please refer to Falconer [8] for classical definitions

5 Definition ([8, ]) Let X C m The Hausdorff dimension of X is dim H (X) = inf s [0, ) X S [d] for some s-supergale d } The packing dimension of X is dim P (X) = inf s [0, ) X S str[d] for some s-supergale d} Finite-state dimension and strong dimension are finite-state counterparts of classical Hausdorff dimension [3] and packing dimension [20, 29] introduced by Dai, Lathrop, Lutz, and Mayordomo [6] and Athreya, Hitchcock, Lutz, and Mayordomo [] in the Cantor space C Finite-state dimensions are defined by using the gale characterizations of the Hausdorff dimension [8] and the packing dimension [] and restricting the gales to the ones whose underlying betting strategies can be carried out by finite-state gamblers In this section, we give the definitions of the finite-state dimensions for space C m and review their basic properties Now, we define finite-state gamblers on alphabet Σ m Definition ([6]) A finite-state gambler (FSG) is a 5-tuple G = (Q, Σ m, δ, β, q 0 ) such that Q is a non-empty finite set of states; Σ m is the input alphabet; δ : Q Σ m Q is the state transition function; β : Q (Σ m ) is the betting function; q 0 Q is the initial state The extended transition function δ : Q Σm Q is defined such that δ q if w = a = λ, (q, wa) = δ(δ (q, w), a) if w λ We use δ for δ and δ(w) for δ(q 0, w) for convenience The betting function β i : Q (Σ m ) specifies the bets the FSG places on each input symbol in Σ m with respect to a state q Q Definition ([6]) Let G = (Q, Σ m, δ, β, q 0 ) be an FSG The s-gale of G is the function d G : Σm [0, ) defined by the recursion if w = b = λ, d G (wb) = m s d G (w)β i (δ(w))(b) if b λ, for all w Σ m and b Σ m λ} For s [0, ), a function d : Σ m [0, ) is a finite-state s-gale if it is the s-gale of some finite-state gambler Note that in the original definition of a finite-state gambler the range of the betting function β is (0, }) Q 2 [6, ] In the following observation, we show that allowing the range of β to have irrational probability measures does not change the notions of finite-state dimension and strong dimension Observation 3 Let G = (Q, Σ m, δ, β, q 0 ) be an FSG For each ɛ > 0, there exists an FSG G = (Q, Σ m, δ, β, q 0 ) with β : Q (Σ m ) Q m such that for all s [0, ), S [d (s) G ] S [d (s+ɛ) G ] and S str[d (s) G ] S str[d (s+ɛ) G ]

6 In this paper, we allow the finite-state gamblers to place irrational bets Definition ([6, ]) Let X C m The finite-state dimension of X is dim FS (X) = inf s [0, ) X S [d] for some finite-state s-gale d} and the finite-state strong dimension of X is Dim FS (X) = inf s [0, ) X S str[d] for some finite-state s-gale d} We will use the following basic properties of the Hausdorff, packing, finitestate, strong finite-state dimensions Theorem 32 ([6, ]) Let X, Y, X i Σ m for i N 0 dim H (X) dim FS (X), 0 dim P (X) Dim FS (X) 2 dim H (X) dim P (X), dim FS (X) Dim FS (X) 3 If X Y, then the dimension of X is at most that same dimension of Y 4 dim FS (X Y ) = maxdim FS (X), dim FS (Y )} and Dim FS (X Y ) = max Dim FS (X), Dim FS (Y )} 5 dim H ( X i) = sup i N dim H (X i ), dim P ( X i) = sup i N dim P (X i ) 4 Relative Frequencies of Digits As we have mentioned in Section, Besicovitch in 934 and Eggleston in 949 proved the following two identities respectively Theorem 4 dim H (FREQ β ) = H 2 ((β, β)) [3] and dim H (FREQ β ) = H 2 ((β, β)) [7] In this section, we will calculate the finite-state dimension of some more exotic sets that contain m-adic sequences that satisfy certain conditions placed on the frequencies of digits The proofs in this section use straightforward constructions of finite-state gamblers Both the constructions and analysis use completely elementary techniques α βα Let H β,m (α) = (α log m α + βα log m βα + ( α βα) log m m 2 ) Let Note that H β,m (α (β)) = α m x < (x) = otherwise x +x+(m 2)x x+ sup α [0, +β ] H β,m (α) = if β <, log m (m 2 + +β ) otherwise β β+

7 Theorem 42 Let β β 0 Let X = S lim inf π (S, n) π (S, n) β and lim sup π 0 (S, n) π 0 (S, n) β Then dim H (X) = dim FS (X) = H β,m(α (β )) and dim P (X) = Dim FS (X) = H β,m (α (β)) Corollary 43 (Theorem 2 [2]) Let β 0 Let X = S lim π (S, n) π 0 (S, n) = β Let β = maxβ, /β} Then ( ) dim H (X) = H β,m (α (β )) = log m m β } β β β + Note that dim P (X), dim FS (X), and Dim FS (X) all takes the value of dim H (X), which were not proven in [2] Proof We prove the case } where β = β The other case is similar Let Y = π S lim inf (S,n) π β 0(S,n) Let lim π 0(S, n) = α (β), lim π (S, n) = βα } (β), Z = S and ( i > ) lim π i(s, n) = α (β) βα (β) m 2 By Eggleston s theorem, dim H (Z) = H β,m (α (β)) Since Z X Y, it follows immediately from Theorem 42 that dim H (X) = H β,m (α (β)) } 5 Saturated Sets and Maximum Entropy Principle In Section 4, we calculated the finite-state dimensions of many sets defined using properties on asymptotic frequencies of digits They are all saturated sets Now we formally define saturated sets and investigate their collective properties Let Π n (S) = π(s, m) m n} for all n N Let Π n (S) = Π n (S), ie, Π n (S) is the closure of Π n (S) Define Π : C m P( (Σ m )) such that for all S C m, Π(S) = n N Π n (S) Definition Let X C m We say that X is saturated if for all S, S C m, Π(S) = Π(S ) [S X S X] When we determine an upper bound on the finite-state dimensions of a set X C m, it is in general not possible to use a single probability measure as the betting strategy even when X is saturated However, when certain conditions

8 are true, a simple -state finite-state gambler may win on a huge set of sequences with different empirical digit distribution probability measures In the following, we formalize such a condition and reveal some relationship between betting and the Kullback-Leibler distance (relative entropy) [5] Note that m-dimensional Kullback-Leibler distance D m ( β α) is defined as D m ( β α) = E β log m β α Definition Let α, β (Σ m ) We say that α ɛ-dominates β, denoted as α >> ɛ β, if H m ( α) H m ( β) + D m ( β α) ɛ We say that α dominates β, denoted as α >> β, if α >> 0 β Note that H m ( β) + D m ( β α) = E β log m β + E β log m β α = E β log m α, where E β log β m α = m β β i log i m α i It is very easy to see that the uniform probability measure dominates all probability measures Observation 5 Let α = ( m,, m ) Let β (Σ m ) Then α >> β Here, we give a few interesting properties of the domination relation Theorem 52 Let α = (α 0,, α k ) (Σ k ) Let β = (β 0,, β k ) (Σ k ) be such that β j =, where j = arg maxα 0,, α k } Then α >> β and H k ( β) = 0 Theorem 53 Let α, β (Σ k ), ɛ 0, and r [0, ] If α >> ɛ α >> ɛ r α + ( r) β β, then Theorem 54 Let µ = ( m,, m ) (Σ m) be the uniform probability measure Let β (Σ m ) Let s [0, ] Let α = s µ + ( s) β Then α >> β The following theorem relates the domination relation to finite-state dimensions Theorem 55 Let α (Σ k ) and X Σ k If α >> ɛ π(s, n) for infinitely many n for every ɛ > 0 and every S X, then dim FS (X) H k ( α) 2 If α >> ɛ π(s, n) for all but finitely many n for every ɛ > 0 and every S X, then Dim FS (X) H k ( α) Theorem 55 tells us that if we can find a single dominating probability measure for X C m, then a simple -state FSG may be used to assess the dimension of X However, in the following, we will see that the domination relationship is not even transitive Theorem 56 Domination relation defined above is not transitive

9 Fix α (Σ m ) with H m ( α), the hyperplane H in R m defined by H m ( α) = m x i log m α i divides the simplex (Σ m ) into two halves A and B with A B H Suppose ( m,, m ) B, then A = β (Σ m ) α >> β} So it is not always possible to find a single probability measure that dominates all the empirical probability measures of sequences in X C m Nevertheless, we take advantage of the compactness of (Σ m ) and give a general solution for finding the dimensions of X C m, when X is saturated The following theorem is our pointwise maximum entropy principle for saturated sets It says that the dimension of a saturated set is the maximum pointwise asymptotic entropy of the empirical digit distribution measure Theorem 57 Let X C m be saturated Let H = sup S X lim inf H m( π(s, n)) and P = sup S X lim sup H m ( π(s, n)) Then dim FS (X) = dim H (X) = H and Dim FS (X) = dim P (X) = P This theorem automatically gives a solution for finding an upper bounds for dimensions of arbitrary X Corollary 58 Let X C m and let H and P be defined as in Theorem 57 Then dim FS (X) H and Dim FS (X) P In the following, we derive the dimensions of a few interesting saturated sets using Theorem 57 We will give more examples in the full version of this paper Let H α,m = log m [α α ( α m )α ] Theorem 59 Let α, ᾱ [0, ] such that /m < α ᾱ and let M α,ᾱ k = S Σm lim inf π k(s, n) = α and lim sup π k (S, n) = ᾱ} Then dim H (M α,ᾱ k ) = Hᾱ,m and dim P (M α,ᾱ k ) = H α,m Proof It is easy to check that M α,ᾱ k is saturated, that Hᾱ,m = inf α [α,ᾱ] H α,m, and that H α,m = sup α [α,ᾱ] H α,m The theorem follows from Theorem 57 Corollary 50 Let α, ᾱ [0, ] such that α ᾱ and let M α,ᾱ k = S C m lim inf π k(s, n) = α and lim sup π k (S, n) = ᾱ} Then dim H (M α,ᾱ k ) = inf α [α,ᾱ] H α,m = min(h α,m, Hᾱ,m ) and dim P (M α,ᾱ k ) = sup H α,m = α [α,ᾱ] Proof If α /m ᾱ, then for some S M α,ᾱ k α /m ᾱ, max(h α,m, Hᾱ,m ) otherwise, lim sup H m ( π(s, n)) = Corollary 5 (Theorem 7 [2]) Let α k, ᾱ k [0, ] for k Σ m Let M R = m k=0 M α k,ᾱ k k Then dim FS (M R ) = dim H (M R ) = min m k=0 dim H(M α,ᾱ k ) and Dim FS (M R ) = dim P (M R ) = min m k=0 dim P(M α,ᾱ k )

10 Corollary 52 (Theorem [2]) Let k Σ m and let M k = S C m lim inf π k(x, n) < lim sup π k (x, n)} Then dim H ( m k=0 M k) = Theorem 53 Let A be a d m matrix and b = (b,, b d ) R d Let K io (A, b) = S C m ( k n } N) lim k n = and lim A( π(s, k n)) T = b} and let K(A, b) = S C m lim A( π(s, n))t = b} Then dim FS (K io (A, b)) = dim H (K io (A, b)) = sup α (Σm) H m ( α), dim P (K io (A, b)) =, and A α T =b dim H (K(A, b)) = Dim FS (K(A, b)) = sup α (Σm ) A α T =b H m ( α) Proof It is easy to check that K io (A, b) and K(A, b) are both saturated 6 Conclusion A general saturated set usually has an uncountable decomposition in which, the dimension of each element is easy to determine, while the dimension of the whole set, which is the uncountable union of all the element sets, is very difficult to determine and requires advanced techniques in multifractal analysis and ergodic theory By using finite-state gambler and gale characterizations of dimensions, we are able to obtain very general results calculating the classical dimensions and finite-state dimensions of saturated sets using completely elementary analysis This indicates that gale characterizations will play a more important role in dimension-theoretic analysis and that finite-state gambler is very powerful References K B Athreya, J M Hitchcock, J H Lutz, and E Mayordomo Effective strong dimension, algorithmic information, and computational complexity SIAM Journal on Computing, 37:67 705, L Barreira, B Saussol, and J Schmeling Distribution of frequencies of digits via multifractal analyais Journal of Number Theory, 97(2):40 438, A S Besicovitch On the sum of digits of real numbers represented in the dyadic system Mathematische Annalen, 0:32 330, H Cajar Billingsley dimension in probability spaces Lecture notes in mathematics, 892, 98 5 T M Cover and J A Thomas Elements of Information Theory John Wiley & Sons, Inc, New York, NY, 99 6 J J Dai, J I Lathrop, J H Lutz, and E Mayordomo Finite-state dimension Theoretical Computer Science, 30: 33, H Eggleston The fractional dimension of a set defined by decimal properties Quarterly Journal of Mathematics, Oxford Series 20:3 36, K Falconer The Geometry of Fractal Sets Cambridge University Press, K Falconer Fractal Geometry: Mathematical Foundations and Applications Wiley, second edition, 2003

11 0 I J Good The fractional dimensional theory of continued fractions Proceedings of the Cambridge Philosophical Society, 37:99 228, 94 X Gu A note on dimensions of polynomial size circuits Theoretical Computer Science, 359(-3):76 87, X Gu, J H Lutz, and P Moser Dimensions of Copeland-Erdős sequences Information and Computation, 205(9):37 333, F Hausdorff Dimension und äusseres Mass Mathematische Annalen, 79:57 79, 99 4 J M Hitchcock Fractal dimension and logarithmic loss unpredictability Theoretical Computer Science, 304( 3):43 44, D A Huffman A method for the construction of minimum redundancy codes Proc IRE, 40:098 0, J Kelly A new interpretation of information rate Bell Systems Technical Journal, 35:97 926, J H Lutz Gales and the constructive dimension of individual sequences In Proceedings of the 27th International Colloquium on Automata, Languages, and Programming, pages , 2000 Revised as [9] 8 J H Lutz Dimension in complexity classes SIAM Journal on Computing, 32: , 2003 Preliminary version appeared in Proceedings of the Fifteenth Annual IEEE Conference on Computational Complexity, pages 58 69, J H Lutz The dimensions of individual strings and sequences Information and Computation, 87:49 79, 2003 Preliminary version appeared as [7] 20 C T McMullen Hausdorff dimension of general Sierpinski carpets Nagoya Mathematical Journal, 96: 9, L Olsen Multifractal analysis of divergence points of deformed measure theoretical Birkhoff averages Journal de Mathématiques Pures et Appliquées Neuvième Série, 82(2):59 649, L Olsen Applications of multifractal divergence points to some sets of d-tuples of numbers defined by their n-adic expansion Bulletin des Sciences Mathématiques, 28(4): , L Olsen Applications of divergence points to local dimension functions of subsets of R d Proceedings of the Edinburgh Mathematical Society, 48:2328, L Olsen Multifractal analysis of divergence points of the deformed measure theoretical Birkhoff averages III Aequationes Mathematicae, 7(-2):29 53, L Olsen and S Winter Multifractal analysis of divergence points of the deformed measure theoretical Birkhoff averages II 200 preprint 26 L Olsen and S Winter Normal and non-normal points of self-similar sets and divergence points of self-similar measures Journal of the London Mathematical Society (Second Series), 67():03 22, B Ryabko, J Suzuki, and F Topsoe Hausdorff dimension as a new dimension in source coding and predicting In 999 IEEE Information Theory Workshop, pages 66 68, C E Shannon A mathematical theory of communication Bell System Technical Journal, 27: , , D Sullivan Entropy, Hausdorff measures old and new, and limit sets of geometrically finite Kleinian groups Acta Mathematica, 53: , C Tricot Two definitions of fractional dimension Mathematical Proceedings of the Cambridge Philosophical Society, 9:57 74, B Volkmann Über Hausdorffsche Dimensionen von Mengen, die durch Zifferneigenschaften charakterisiert sind VI Mathematische Zeitschrift, 68: , 958

12 A Appendix for Section 3 Proof (Proof of Observation 3) Let δ = min q Q min m (β i (q) βi(q) 2 ) ɛ For all q Q, let β i (q) [β i(q) δ, β i (q)] [0, ] Q, if i 0 Otherwise, let β 0(q) = m i= β i (q) Note that for all q Q and all i 0,, m }, β i(q) β i (q) δ 0 (A) and that β maps states to rational bets For w Σ m, let Note that #q i (w) = n 0 n w, δ(w[0n ]) = q and w[n] = i} w = q Q Now we have that for all w Σ m, By the choice of δ, m d (s+ɛ) G (w) = c 0m (s+ɛ) w q Q d (s+ɛ) G (w) c 0m s w q Q m #q i (w) (A2) β i(q) #q i(w) (A) c 0 m m (s+ɛ) w (β i (q) δ) #qi(w) q Q = (A2) c 0 m m s w [(β i (q) δ)2 ɛ ] #qi(w) q Q = c 0 m s w q Q = d (s) G (w) Thus, for all S S [d (s) G ], lim sup and for all S S str[d (s) G ], lim inf d (s+ɛ) G d(s+ɛ) G m m [(β i (q) (β i (q) βi(q) 2 ɛ ))2 ɛ ] #qi(w) β i (q) #q i(w) (S) lim sup (S) lim inf d(s) G d (s) G (S) =, (S) =

13 Therefore, and S [d (s) G ] S [d (s+ɛ) G ] Sstr[d (s) G ] S str[d (s+ɛ) G ] B Appendix for Section 4 Proof (Proof of Theorem 42) We assume that β β, since when either of these values are less than, the proof is essentially looking at the subset of X where their values are replaced by First, we prove the lower bounds for the dimensions When S is clear from the context, let α n = π 0 (S, n) and β n = π (S, n) Let α = α (β ) and let α = α (β) For Hausdorff dimension and finite-state dimension, let Y = S lim α n = α, lim β n = β α, and ( i > ) lim π i(s, n) = α β α m 2 By Eggleston s theorem, we have dim H (Y ) = H β,m(α (β )) Since β β and Y X, dim FS (X) dim H (X) dim H (Y ) = H β,m(α (β )) For packing dimension and finite-state strong dimension, let Z = S lim α n = α, lim β n = βα, and ( i > ) lim π i(s, n) = α βα m 2 Now we construct from Z a set Z X by interpolating the sequences in Z First let l 0 = 2 and for every i N, l i+ = 2 l i Define f 0 : Σm Σm be such that f 0 (w) = w for all w Σm Let ρ = αβ αβ+ For each n > 0, define f n : Σm Σm such that for every w Σm, f n (w) = w and for every i < w, f n (w)[i] i l n w[i] i ρl n and i > l n f n (w)[i] = i > ρl n and i l n w[i] i > l n Define f : Σ m Σ m such that for all w Σ m f(w) = f n(w) (w), where n(w) = min n N l n w } Also, extend f to f : Σm Σm such that for all S Σm, f(s) = lim f(s[0n ]) } }

14 Let Z = f(z) By the construction of f and choice of ρ, it is clear that f is a dilation and for all n N, Col(f, S[0 ρl n ]) log l n Thus for all ɛ > 0, there are infinitely many n such that Col(f, S[0n ]) < ɛn (B) Note that by Eggleston s theorem, dim H (Z) = H β,m (α (β)) Then by Supergale Dilation Theorem [] and (B), dim P (Z ) H β,m (α (β)) It is easy to verify that for every S Z, lim inf β n β n β and lim sup β α n α n So Z X Therefore, Dim FS (X) dim P (X) H β,m (α (β)) Now, we prove that H β,m(α (β )) is an upper bound for dim H (X) and dim FS (X) If β <, then H β,m(α (β )) = and the upper bound holds trivially So assume β Let α = α (β ) Let s > H β,m(α (β )) Define m s αd(w) b = 0 d(wb) = m s β αd(w) b = m s α β α m 2 d(w) b 2 It is clear that d is a finite-state s-gale Let B = β Let β β + ɛ = s H β,m(α (β )) 2 log m B Let S X and let δ > 0 be such that δ min(ɛβ 2 /2, /2) Since β n lim sup β, α n there exists an infinite set J N such that for all n J By the choice of δ, for all n J ie, β n α n β δ α n β n β δ = β + δ (β δ)β β + ɛ, α n + β n β + β β n + ɛ (B2)

15 Now, note that m s B ɛ = ( + β + (m 2)B)B ɛ, (B3) since m s B ɛ = m s B s log m (m 2+ 2 log m B = B +log B ms log m m = B + 2 log m m = B + log m m = B + s log m (m 2+ +β B ) s logm (m 2+ +β B ) 2 log m B s logm m s +log m (m 2+ +β B ) 2 log m B s +logm (m 2+ +β B ) 2 log m B = B +ɛ+log +β B (m 2+ B ) +β B )+2 log +β m (m 2+ B ) 2 log m B For all n J, ( d(s[0n ]) = m sn α nα n (β α) nβ n α β α m 2 [ m s β β n B α n β n ] n = + β + (m 2)B (B2) ms β β n B β + n β β n ɛ + β + (m 2)B [ m s B ɛ ] n = + β + (m 2)B Since J is an infinite set, = (B3) B ɛn lim sup d(s[0n ]) =, ) n( αn β n ) ie, S S [d] Since s > H β,m(α (β )) is arbitrary and d is finite-state s-gale, dim H (X) dim FS (X) H β,m(α (β )) An essentially identical argument gives us dim P (X) Dim FS (X) H β,m (α (β)) Theorem B Let α /m Let X = } S lim inf π 0(S, n) α Then S dim P (X) = dim H (X) = dim P (Y ) = dim H (Y ) = log m [ } lim π 0(S, n) = α and Y = ( ) ] α α α α m

16 Proof (Proof of Theorem B) The results are clear for α = /m, so we assume that α > /m [ ( ) ] α Let H α,m = log m α α α m We first show that dim P (Y ) H α,m For s > H α,m, define m s αd(w) b = 0 d(wb) = m s α m d(w) b 0 It is clear that d is an s-gale Let ɛ = s H α,m (B4) α(m ) 2 log m α Note that α(m ) α > Let S Y, ie, lim inf π 0(S, n) α So there exists J N such that N \ J < and for every n J, π 0 (S, n) α ɛ [ ( ) ] π0 (S,n) n α d(s[0n ]) = m s α π0(s,n) m [ (α(m ) 2ɛ ( ) α ( ) ] π0(s,n) n ) α = (B4) α α α π 0(S,n) α α m m [ (α(m ) 2ɛ ( ) ] α π0(s,n) n ) α = α π0(s,n) α α m [ (α(m ) 2ɛ ( ) ] π0 (S,n) α n ) α(m ) = α α [ (α(m ) ] 2ɛ+π0(S,n) α n ) = α Then for every n J, [ ] ɛn α(m ) d(s[0n ]) α Since α(m ) α >, S S str[d] and dim H (Y ) dim P (Y ) H α,m Note taht X Y, so dim H (X) dim P (X) H α,m Now it suffices to show that dim H (X) H α,m Let Z = S lim π 0(S[0n ]) = α and ( i > 0) lim π i(s[0n ]) = α m By Eggleston s theorem, dim H (Z) = H α,m Since Z X Y, dim H (Y ) dim H (X) H α,m }

17 Theorem B2 (Corollary 3 in [2]) Let Σ m be the m-ary alphabet Let k < m Let α 0, α,, α k [0, ] be such that α = k α i Let Then dim H (X) is X = S } lim π i(s, n) = α i, 0 i k ) [ H m (α 0,, α k, α m k,, α m k = log m α α0 0 α α k k ( α m k ) ( α) ] and dim FS (X) = Dim FS (X) = dim P (X) = dim H (X) Proof (Proof of Theorem B2) We insist that 0 0 = and 0/0 = in the proof Let For s > H, define ( H = H m α 0, α,, α k, α m k,, α ) m k d(wb) = m s d(w)α b m s d(w) α m k b < k otherwise It is clear that d is a finite-state s-gale Let For S X, δ = s H 2 log m (α 0 α k α m k ) lim π i(s, n) = α i, 0 i k So there exists n 0 N such that for all n n 0 π i (S, n) α i < δ for all i < k and that k α π i (S, n) < δ

18 Then for all n n 0, [ ( α d(s[0n ]) = m s m k [ ( α = m s H m H m k [ = = [ ) k π i(s,n) k m s H α α0 0 α α k k α πi(s,n) i ) k π i(s,n) k ( α m k ) α k π i(s,n) k ( α m s H m k ( [m s H α α 0 α k m k = m s H 2 n ]n α πi(s,n) i ]n ) ( α) ( α m k ) δ ] n = α πi(s,n) αi i [m s H m H s 2 ]n ) k π i(s,n) k So S Sstr[d] and thus dim FS (X) Dim FS (X) H Let Z = S ( i < k) lim π i(s, n) = α i and ( i k) lim π i(s, n) = α } m k By Eggleston s theorem, dim H (Z) = H The theorem then follows from the monotonicity of dimensions C Appendix for Section 5 Proof (Proof of Theorem 52) It is easy to see that H k (β) = 0 It suffices to show that H k ( α) E β log k α k E β log k α = β i log k = β j log α k i α j k = log k α i log α k j α i = H k ( α) Proof (Proof of Theorem 53) Assume α >> ɛ β, it suffices to show that H k ( α) E r α+( r) β log k α ɛ ] n α πi(s,n) i ]n

19 k E r α+( r) β log k α ɛ = (rα i + ( r)β i ) log k ɛ α i k = k rα i log k + α i ( r)β i log k α i ɛ = rh k ( α) + ( r)e β log k ( r)ɛ rɛ α H k ( α) Proof (Proof of Theorem 54) Let A = i µ i β i } and let B = i µ i < β i } Then A B = and A B = [0m ] Note that for any i A, µ i = m β i and log m sµ i +( s)β i and for any i B, µ i = m < β i and i B s(µ i β i ) log m sµ i +( s)β i < Since m s(µ i β i ) = 0, i A s(µ i β i ) = i B s(µ i β i ) E α log m α E β log m α = E s( µ β) log m s µ + ( s) β = m = i A s(µ i β i ) log m sµ i + ( s)β i s(µ i β i ) log m + s(µ i β i ) log sµ i + ( s)β m i sµ i + ( s)β i i B s(µ i β i ) + s(µ i β i ) i A i B 0 Therefore, E α log m α E β log m α, ie, α >> β Proof (Proof of Theorem 55) Let G = (Q, δ, β, q 0, ) be an FSG such that Q = q 0 }, δ(q 0, b) = q 0 for all b Σ k, and β(q 0 ) = α Let s > H k ( α) + ɛ The s-gale d (s) G d (s) G (wb) = of G is defined by the following recursion, w = b = λ k s d (s) G (w)α b otherwise,

20 for all w Σ k and b Σ k Let S X Then k d (s) G (S[0n ]) = ksn α nπ i(s,n) i = k sn k n k πi(s,n) log k αi ( ) = k s E n π(s,n) log k α Thus S S [d (s) G ] and dim FS(S) s, when the domination condition holds for infinitely many n Similarly, S Sstr[d (s) G ] and Dim FS(S) s, when the domination condition holds for all but finitely many n The theorem then follows, since ɛ can be arbitrarily small Proof (Proof of Theorem 56) We prove this by giving a counterexample with Σ 3 This counterexample can be extended to larger alphabets very easily Let α = ( , , ), β = ( 300, , ) And we have and H(α) , H(β) , and β H(α) E β log 3 α So α >> β Note that fix α (0,, 2}), for γ (0,, 2}), α >> γ if ie, H(α) E γ log 3 α, H(α) γ 0 log 3 α 0 + γ log 3 α + γ 2 log 3 α 2 It is clear that α determines a hyperplane that separate the space of all probability measures Since we only consider the cases where γ 0 + γ + γ 2 =, the above inequality simplifies to H(α) γ 0 log 3 α 0 + γ log 3 α + ( γ 0 γ ) log 3 α 0 α Let γ 0 = 0, we may solve the above inequality and obtain the boundary point for α at γ 0 = 0 is γ = 9 25 Similarly, the boundary point for β at γ 0 = 0 is approximately γ = Let γ = (0, 037, 063) H(α) E γ log 3 α

21 and H(β) E γ log 3 β Thus β dominates γ but α does not dominate γ Lemma C ([2]) For every n m 2 and every partition a = (a 0,, a m ) of n, there are more than m nh m( a n ) (m+) log m n strings u of length n and #(i, u) = a i for each i Σ m Theorem C2 ([6]) Let d be an s-supergale, where s [0, ) Then for all w Σm, l N, and 0 < α R, there are fewer than ml α strings u Σl m for which d(wu) > αm (s )l d(w) Proof (Proof of Theorem 57) First we prove dim H (X) H It suffices to show that for all s < H, dim H (X) s Let s < H Let d be an arbitrary s-supergale Let s = (H + s)/2 Let n 0 N be such that m < n 0 (H s ) and m s n 0 (m+) log m n 0 > 2 sn0+ Fix an S X such that lim inf H m( π(s, n)) > s For each i n 0, let β i,,, β i,ci } (Σ m ) be such that for each j [c i ], β i,j = k n for some k n and H m( β i,j ) > s ; for all β F (S) there exists j [c i ] such that β i,j β < /i; for all j [c i ], there exists β F (S) such that β i,j β < /i; for all j [c i ], β i,j β i,j+ < i ; for all i n 0, β i+,0 β i,ci < i+ Now, we first construct a sequence S Σm by building its prefixes inductively Let w 0 be such that w 0 = 2 n 0 Note that the choice of w 0 does not affect the argument, since w 0 does not change the asymptotic behavior of the sequence Without loss of generality, assume π(w 0, w 0 ) = β n0, For all n > 0, assume w n is already constructed Let w n,0 = w n We construct inductively w n,,, w n,cn and then let w n = w n,cn For j > 0, assume w n,j is already constructed Let l = n 0 + n For each l, j, let B l,j = u Σ l m π(u, l) = β l,j } For each l n 0 and w Σm, let W l,w = u Σm l d(wu) } m d(w) Since d is an s-supergale, by Theorem C2, for all w Σ m, there are fewer than m sl+ strings u Σ l m for which d(wu) > m d(w) By the choice of n 0, β l,j, and Lemma C, B l,j > m sl+,

22 ie, W l,w B l,j Let u W l,w B l,j For all i [22 wn,j ], let u i W l,wu u i B l,j Let w n,j = w n,j u u 2 w n,j Let S = lim w n Note that when w n is being constructed, l log m w n,j It is then easy to verify that S / S [d] Now we verify that Π(S) = Π(S ) Then S X, since X is defined by asymptotic frequency Let β Π(S) be arbitrary For each l = n 0 + n, there exists some j l such that β β l,jl < l Then by the construction, So it is clear that Thus π(w l,jl, w l,jl ) β l,jl < m 2 w l,jl < l π(w l,jl, w l,jl ) β < 2 m l lim π(w l,j l, w l,jl ) = β l Since w l,jl S for all l = n 0 + n So we have for all n N, β Π n (S ), hence β Π(S ) Therefore Π(S) Π(S ) Now, let β / Π(S) Since Π(S) is closed, there exists δ > 0 such that for all β Π(S), β β > δ Let n be such that l = n 0 + n > 8m δ By construction, for all l l, all j [c l ], and all w l,j k w l,j, π(w l,j, w l,j ) π(w l,j, k) < 2 m l Also, for all l l and all j [c l ], there exists β Π(S) such that π(w l,j, w l,j ) β < 2 m l Thus for all k > w l,, there exists β Π(S) such that π(s, k) β < 4m l Therefore, for all k > w l, Thus for all sufficiently large k, π(s, k) β < 4m l < δ 2 π(s, k) β > δ 2

23 So there exists n 2 N such that for all n n 2, β / Π n, ie, β / Π(S ) Now we have that S X Since S / S [d], s < H is arbitrary, and d is an arbitrary s-supergale, dim H (X) H By a similar construction, we may prove that dim P (X) P In the following, we prove the finite-state dimension upper bounds Given α (Σ m ), define B( α, r) as B( α, r) = (Σ m ) } β R m ( i)[β i < α i m r and β i > α i m r ] Let Let ɛ > 0 Let F (X) = α (Σ m ) H( α) = H } C = B( α, ɛ 2 ) α F (X)} It is clear that C is an open cover of F (X) Since F (X) is compact, there exists C (Σ m ) such that C < and F (X) B( α, ɛ 2 ) α C Let S X Then lim inf H m( π(s, n)) H By Theorem 54, there exists α F (X) such that α >> ɛ π(s, n) for infinitely many n N Then by the construction of C, there exists α C such that α B( α, ɛ 2 ) Now, we have that for infinitely many n N, By the definition of B(α, ɛ 2 ), H m ( α) = H m ( α ) E π(s,n) log m α ɛ 2 = E π(s,n) log m α + E α π(s,n) log m α ɛ 2 H m ( α) E π(s,n) log m α ɛ, ie, α >> ɛ π(s, n) for infinitely many n N Since S X is arbitrary, we may partition X as X = α C X α such that for every α C, X α = S X α >> ɛ π(s, n) for infinitely many n N} Since ɛ > 0 is arbitrary, thus by Theorem 55, dim FS (X α ) H m ( α) = H for every α C Since C <, by Theorem 32, dim FS (X) H Similarly, Dim FS (X) P