Dimension of stablesets and scrambled sets in positive finite entropy systems

University of Massachusetts Amherst From the SelectedWorks of Pengfei Zhang 202 Dimension of stablesets and scrambled sets in positive finite entropy systems Pengfei Zhang Available at: https://works.bepress.com/pengfei_zhang/5/

Ergod. Th. & Dynam. Sys. (202), 32, 599 628 c Cambridge University Press, 20 doi:0.07/s04338570000982 Dimensions of stable sets and scrambled sets in positive finite entropy systems CHUN FANG, WEN HUANG, YINGFEI YI and PENGFEI ZHANG Department of Mathematics, University of Science and Technology of China, Hefei, Anhui 230026, PR China (e-mail: Fangchun@mail.ustc.edu.cn, wenh@mail.ustc.edu.cn, pfzhang5@mail.ustc.edu.cn) School of Mathematics, Jilin University, Changchun, 3002, PR China and School of Mathematics, Georgia Institute of Technology, Atlanta, GA 30332, USA (e-mail: yi@math.gatech.edu) (Received September 200 and accepted in revised form 3 December 200) In memory of Dan Rudolph Abstract. We study the dimensions of stable sets and scrambled sets of a dynamical system with positive finite entropy. We show that there is a measure-theoretically large set containing points whose sets of hyperbolic points (i.e. points lying in the intersections of the closures of the stable and unstable sets) admit positive Bowen dimension entropies; under the continuum hypothesis, this set also contains a scrambled set with positive Bowen dimension entropies. For several kinds of specific invertible dynamical systems, the lower bounds of the Hausdorff dimension of these sets are estimated. In particular, for a diffeomorphism on a smooth Riemannian manifold with positive entropy, such a lower bound is given in terms of the metric entropy and Lyapunov exponent.. Introduction Throughout the paper, by a topologically dynamical system (, T ) (TDS) we mean a compact metric space (, d) with a continuous map T from into itself, where d denotes the metric on. If T is a homeomorphism, then we say that the TDS (, T ) is invertible. For a TDS (, T ), the stable set of a point x is defined as { } W s (x, T ) = y : lim d(t n x, T n y) = 0. If (, T ) is invertible, then we can also define the unstable set of a point x as { } W u (x, T ) = y : lim d(t n x, T n y) = 0.

600 C. Fang et al A pair of points x, y is said to be a Li Yorke pair with modulus δ if lim sup d(t n x, T n y) = δ > 0 and lim inf d(t n x, T n y) = 0. A subset S is called scrambled if any pair of distinct points x, y S forms a Li Yorke pair. If contains an uncountable scrambled set, then the TDS (, T ) is said to be chaotic in the sense of Li Yorke [20]. Blanchard et al [2] showed that any general TDS with positive entropy always contains an uncountable scrambled set, hence it is chaotic in the sense of Li Yorke. In fact, Blanchard and Huang [3] further showed that such a TDS even admits a certain amount of weak mixing dynamics a stronger chaos than that defined by Li and Yorke. For an Anosov diffeomorphism T on a compact manifold, it is well known that points belonging to the stable set tend to diverge under T, but pairs belonging to the unstable set behave in the opposite way. A natural question concerning a general TDS with finite positive entropy is then whether it can retain a faint flavor of such dynamical behavior. Recently, Blanchard et al [] showed that for any invertible TDS (, T ) with positive entropy, the stable sets for T are not stable for T, i.e. if a T -invariant ergodic measure µ has positive entropy, then there is δ > 0 such that for µ-a.e. x, one can find an uncountable subset F x of W s (x, T ) such that for any y F x \{x}, {x, y} forms a Li Yorke pair for T with modulus δ. For a C 2 diffeomorphism f on a closed smooth manifold M, it was further shown by Sumi [29] that if the metric entropy with respect to an f -invariant ergodic probability measure µ is then positive for µ-a.e. x M, both the closure W s (x, f ) of the stable set and the closure W u (x, f ) of the unstable set contain uncountable scrambled sets. Generalizing this to a TDS, it was shown by the second author in [4] that in any (invertible) TDS with positive entropy there is a measure-theoretically rather big set such that the closure of the stable (or unstable) sets of points in the set contains a weak mixing set. With these known descriptions and characterizations, it is of fundamental importance to know how big these stable sets or scrambled sets can be in a positive entropy system. Usually, one refers to a set E in a TDS (, T ) as big if one of the following properties holds. (a) E is of a positive measure with respect to some T -invariant measure on. (b) E has a non-empty interior or is a dense G δ set in. (c) htop B (T E), the Bowen dimension entropy of E, is positive. (d) H d (E), the Hausdorff dimension of E with respect to d, is positive. It is not hard to see that the properties (a) and (b) do not hold in general for a scrambled set or the closure of a stable set. This leaves the properties (c) and (d) as possible criteria for describing the bigness. The purpose of this paper is to investigate the Bowen dimension entropy and Hausdorff dimension of the stable sets and scrambled sets in a TDS with positive finite entropy. Our main results are as follows. THEOREM.. Let (, T ) be a TDS with a metric d and h top (T ) <. If µ is a T - invariant ergodic measure on with h µ (T ) > 0, then the following hold. () (Bowen dimension) h B top (T W s (x, T )) h µ (T ) for µ-a.e. x, and, moreover,

Dimensions of stable sets and scrambled sets 60 under the continuum hypothesis, for µ-a.e. x there exists a scrambled set S x W s (x, T ) for T satisfying h B top (T S x) h µ (T ). (2) (Hausdorff dimension) If T is a Lipschitz continuous self-map with Lipschitz constant L >, then H d (W s (x, T )) h µ (T )/log L for µ-a.e. x, and, moreover, under the continuum hypothesis, for µ-a.e. x there exists a scrambled set S x W s (x, T ) for T satisfying H d (S x ) h µ (T )/log L. THEOREM.2. Let (, T ) be an invertible TDS with a metric d and h top (T ) <. If µ is a T -invariant ergodic measure on with h µ (T ) > 0, then the following hold. () (Bowen dimension) For µ-a.e. x, h B top (T W s (x, T ) W u (x, T )) h µ (T ) and h B top (T W s (x, T ) W u (x, T )) h µ (T ). Moreover, under the continuum hypothesis, for µ-a.e. x, there exists S x W s (x, T ) W u (x, T ) which is a scrambled set for both T and T such that h B top (T S x) h µ (T ) and h B top (T S x ) h µ (T ). (2) (Hausdorff dimension) If T is Lipschitz continuous with Lipschitz constant L >, then H d (W s (x, T ) W u (x, T )) h µ(t ) for µ-a.e. x, log L and, moreover, under the continuum hypothesis, for µ-a.e. x there exists S x W s (x, T ) W u (x, T ) which is a scrambled set for both T and T such that H d (S x ) h µ (T )/log L. Using the variational principle of entropy, the following result is a direct consequence of Theorems. and.2. COROLLARY. Let (, T ) be a TDS with a metric d. We assume that h top (T ) < and T is a Lipschitz continuous self-map with Lipschitz constant L >. Then the following hold. (a) sup x H d (W s (x, T )) h top (T )/log L, and under the continuum hypothesis, (b) If T is invertible, then sup{h d (S) : S is a scrambled set for T } h top(t ) log L. and under the continuum hypothesis, sup H d (W s (x, T ) W u (x, T )) h top(t ) x log L, sup{h d (S) : S is a scrambled set for T, T } h top(t ) log L.

602 C. Fang et al For a differentiable self-map on a smooth Riemannian manifold, a lower bound of the Hausdorff dimension of the stable set and the scrambled set with respect to an ergodic measure µ can be estimated in terms of the metric entropy and the top Lyapunov exponent χµ (see 6 for details). THEOREM.3. Let f be a C self-map on a smooth Riemannian manifold M and ρ be the metric on M induced by the Riemannian structure. If µ is an f -invariant ergodic measure with a compact support such that h µ ( f ) > 0, then the following hold. () H ρ (W s (x, f )) h µ ( f )/χµ for µ-a.e. x, and under the continuum hypothesis, for µ-a.e. x there exists a scrambled set S x W s (x, f ) for f satisfying H ρ (S x ) h µ ( f )/χµ. (2) If f is a diffeomorphsim, then H ρ (W s (x, f ) W u (x, f )) h µ( f ) χ µ for µ-a.e. x, and under the continuum hypothesis, for µ-a.e. x there exists a scrambled set S x W s (x, f ) W u (x, f ) for both f and f satisfying H ρ (S x ) h µ ( f )/χ µ. Remark. We conjecture that Theorems. and.2 still hold without assuming h top (T ) <. We also conjecture that the continuum hypothesis contained in all of the Theorems..3 can be removed. The paper is organized as follows. Section 2 is a preliminary section in which we review some notions of ergodic theory and TDS. In 3, we discuss some basic properties of Bowen dimension entropy for non-compact sets which will be used in later sections. In 4, we prove parts () of Theorems. and.2 by estimating a lower bound of the Bowen dimension entropy for stable sets as well as for scrambled sets in a positive finite entropy system. In 5, we prove parts (2) of Theorems. and.2 by estimating a lower bound of the Hausdorff dimension for stable sets as well as for scrambled sets in a Lipschitz continuous TDS with positive finite entropy. We prove Theorem.3 in 6. 2. Preliminaries Given a TDS (, T ), we denote by B the σ -algebra of Borel subsets of. A cover of is a family of Borel subsets of whose union is. An open cover is one that consists of open sets. A partition of is a cover of consisting of pairwise disjoint sets. Given a partition α of and x, we denote by α(x) the atom of α containing x. We denote the set of finite partitions, finite covers and finite open covers of by P, C and C o, respectively. Given two covers U, V of, U is said to be finer than V (denoted by U V) if each element of U is contained in some element of V. Let U V = {U V : U U, V V}. It is clear that U V U and U V V. Given integers M, N with M N and U C, we use UM N to denote N n=m T n U. For U C, we define N(U) as the minimum among the cardinalities of the subcovers of U. Then the topological entropy of U with respect to T is defined by h top (T, U) = lim N + N log N(U N 0 ) = inf N N N log N(U N 0 ).

Dimensions of stable sets and scrambled sets 603 The topological entropy of (, T ) is defined by h top (T ) = sup h top (T, U). U C o We sometimes write h top (T, ) to emphasize the dependence of the entropy on the space. Let K be a non-empty closed subset of. For ɛ > 0, a subset F of is called an (n, ɛ)-spanning set of K if, for any x K, there exists y F with d n (x, y) ɛ, where d n (x, y) = max 0 i n d(t i x, T i y); a subset E of K is called an (n, ɛ)-separated set of K if x, y E, x y implies d n (x, y) > ɛ. Let r n (d, T, ɛ, K ) denote the smallest cardinality among all (n, ɛ)-spanning sets of K and s n (d, T, ɛ, K ) denote the largest cardinality among all (n, ɛ)-separated subsets of K. We define r(d, T, ɛ, K ) = lim sup s(d, T, ɛ, K ) = lim sup n log r n(d, T, ɛ, K ), n log s n(d, T, ɛ, K ). Obviously, r(d, T, ɛ, K ) and s(d, T, ɛ, K ) are monotonically increasing when ɛ 0. Let h (d, T, K ) = lim r(d, T, ɛ, K ) and ɛ 0+ h (d, T, K ) = lim s(t, d, ɛ, K ). ɛ 0+ It is well known that h (d, T, K ) = h (d, T, K ), which is independent of the choice of a compatible metric d on, so we simply denote it by h(t, K ). When K =, h(t, ) = h top (T ). Given U C, define { N(U K ) = min the cardinality of F F U, } F K, F F and h(t, U K ) = lim sup ( n n log N i=0 ) T i U K. It is easy to see that h(t, K ) = sup U C o h(t, U K ). Let M(), M(, T ) and M e (, T ), respectively, be the sets of all Borel probability measures, T -invariant Borel probability measures and T -invariant ergodic Borel probability measures on. Then M() and M(, T ) are all convex, compact metric spaces when endowed with the weak -topology. For any given α P, µ M() and any sub-σ -algebra C B µ, where B µ is the completion of B under µ, the conditional informational function of α relevant to C is defined by I µ (α C)(x) := A α A (x) log E( A C)(x), where E( A C) is the conditional expectation of A with respect to C. Let H µ (α C) = I µ (α C)(x) dµ(x) = E( A C) log E( A C) dµ. A α Then H µ (α C) increases with respect to α and decreases with respect to C.

604 C. Fang et al When µ M(, T ) and C is T -invariant (i.e. T C = C), it is not hard to see that H µ (α n 0 C) is a non-negative and subadditive sequence for a given α P. Thus h µ (T, α C) = lim n H µ(α n 0 C) = inf n n H µ(α n 0 C) is well defined. It is well known that ) h µ (T, α C) = H µ (α T n α C. If C = {, }(mod µ), we denote H µ (α C) and h µ (T, α C) by H µ (α) and h µ (T, α), respectively. The measure-theoretical entropy of µ is defined by If, in addition, T is invertible, then n= h µ (T ) = sup α P h µ (T, α). h µ (T, α C) = h µ (T, α C) for any α P and h µ (T ) = h µ (T ). It is also well known that for α P, h µ (T, α) = h µ (T, α P µ (T )) H µ (α P µ (T )), where P µ (T ) is the Pinsker σ -algebra of (, B µ, µ, T ). The following result is a conditional version of the Shanon McMillan Breiman theorem. Its proof is completely similar to the proof of the Shanon McMillan Breiman theorem (see [3], for example). THEOREM 2.. Let (, T ) be an invertible TDS, µ M(, T ), α P and C B µ a T -invariant sub-σ -algebra. Then there exists a T -invariant function f L (µ) such that I µ (α n 0 C)(x) f (x) dµ(x) = h µ (T, α C) and lim = f (x) for µ-a.e. x, n as well as in the sense of L (µ). Moreover, if µ is ergodic, then f (x) = h µ (T, α C) for µ-a.e. x. 3. Bowen dimension entropy of non-compact sets The notion of Bowen dimension entropy for non-compact sets in a TDS was first introduced by Bowen in [7] (see Pesin and Pitskel [27] for some later developments of the relevant theory). Let (, T ) be a TDS and U C. For E, we write E U if E is contained in some element of U. Let n T,U (E) be the biggest non-negative integer such that T k E U for every k {0,,..., n T,U (E) }; n T,U (E) = 0 if E U and n T,U (E) = + if T k E U for any k Z +. Let Y. For each s 0 and k N, denote { m k (T, s, U Y ) = inf e sn T,U (U i ) : } U i Y and n T,U (U i ) k for each i N. i N i= Since m k (T, s, U Y ) is increasing with respect to k N, m(t, s, U Y ) =: lim m k(t, s, U Y ) k +

Dimensions of stable sets and scrambled sets 605 is well defined. It is clear that m(t, s, U Y ) m(t, s, U Y ) if s s 0 and m(t, s, U Y ) / {0, + } for at most one point s 0. We define the Bowen dimension entropy of Y relative to U by htop B (T, U Y ) = inf{s 0 : m(t, s, U Y ) = 0} = sup{s 0 : m(t, s, U Y ) = + }, and define the Bowen dimension entropy of Y by htop B (T Y ) = sup h U C o top B (T, U Y ). When Y =, we omit the restriction on and simply denote htop B (T, U ), h top B (T ) by htop B (T, U), h top B (T ), respectively. It is well known that h top B (T ) = h top(t ) (see [7]). We now give an equivalent definition of the Bowen dimension entropy of a non-compact set. For k N, x and r > 0, let B k (x, r, T ) = {x : d k (x, x ) < r} be the Bowen ball. We define B (x, r, T ) = {x} when x and r > 0. Let E and ɛ > 0. For any n N and s 0, denote { } M n (T, s, ɛ E) = inf e n i s : B ni (x i, ɛ, T ) E and n i n for i N. i= i= Since M n (T, s, ɛ Y ) is increasing with respect to n N, M(T, s, ɛ E) =: lim M n(t, s, ɛ E) is well defined. It is clear that M(T, s, ɛ E) M(T, s, ɛ E) if s s 0 and M(T, s, ɛ E) / {0, + } for at most one point s 0. Let htop B (T, ɛ E) = inf{s 0 : M(T, s, ɛ E) = 0} = sup{s 0 : M(T, s, ɛ E) = + }. The following result is well known (for example, see [26, Remark, p. 74]). PROPOSITION 3.. Let (, T ) be a TDS and E. Then htop B (T E) = lim h top B (T, ɛ E). ɛ 0 The Bowen dimension entropy is a monotonic function of sets, i.e. if E F, then htop B (T E) h top B (T F). Moreover, if {E n} n is a countable family of subsets of, then ( htop B ) T E n = sup htop B (T E n). n n= Hence if E is countable, then h B top (T E) = 0. Let (, T ) and (Y, S) be two TDSs. A continuous map π : (, T ) (Y, S) is called a homomorphism between (, T ) and (Y, S) if it is onto and πt = Sπ. In this case, we say (, T ) is an extension of (Y, S) or (Y, S) is a factor of (, T ). Such a map is often referred to as a factor map or a semi-conjugacy. The following results are elementary (for example, see [7, Proposition 2]).

606 C. Fang et al PROPOSITION 3.2. Let (, T ) and (Y, S) be two TDSs and π : (, T ) (Y, S) be a factor map. Then for any E : () htop B (T E) = h top B (T T (E)); (2) htop B (T E) h top B (S π(e)); (3) htop B (T k E) = khtop B (T E), k N. THEOREM 3.3. Let π : (, T ) (Y, S) be a factor map between two TDSs. Then for any E, htop B (T E) h top B (S π(e)) + sup h(t, π (y)). (3.) Proof. We follow the argument in [5, Proof of Theorem 7]. Let d be a compatible metric on and ρ a compatible metric on Y. If sup y Y h(t, π (y)) =, (3.) is clear. In the following, we assume that a := sup y Y h(t, π (y)) <. Fix ɛ > 0 and τ > 0. For each y Y, choose m(y) N such that m(y) log r m(y)(d, T, ɛ, π (y)) h(t, π (y)) + τ a + τ. Let E y be a (m(y), ɛ)-spanning set of π (y) with E y = r m(y) (d, T, ɛ, π (y)). Denote y Y U y = {u : there exists z E y such that d m(y) (u, z) < 2ɛ}. Then U y is an open neighborhood of π (y) and (\U y ) γ >0 π (B γ (y)) =, where B γ (y) = {y Y : ρ(y, y) < γ }. By the finite intersection property of compact sets, there is a W y = B γy (y) for some γ y > 0 such that U y π (W y ). Since Y is compact, there exist y,..., y r such that W y,..., W yr cover Y. Let δ > 0 be a Lebesgue number of open cover {W y,..., W yr } with respect to ρ, and denote δ = δ /2, M = max{m(y ),..., m(y r )}. Let y Y and m N. We claim that there exist l(y) > 0 and v (y), v 2 (y),..., v l(y) (y) such that where and l(y) e (a+τ)(m+m) and l(y) i= B m (v i (y), 4ɛ, T ) π (B m (y, δ, S)), B m (y, δ, S) = {y Y : ρ m (y, y ) < δ} B m (v i (y), 4ɛ, T ) = {x : d m (v i (y), x ) < 4ɛ}. For each 0 j < m, we choose y ( j) {y,..., y r } such that B δ (S j (y)) W y ( j). Define the sequence t 0,..., t q depending on y recursively such that t 0 (y) = 0 and t s+ (y) = t s (y) + m( y (t s (y))) until one gets a t q+ (y) m. For z 0 E y (t 0 (y)), z E y (t (y)),..., z q(y) E y (t q(y) (y)), we let V (y; z 0,..., z q(y) ) = {u : d(t t+t s(y) (u), T t (z s )) < 2ε for all 0 t < m( y (t s (y))) and 0 s q(y)}.

It is not hard to see that Dimensions of stable sets and scrambled sets 607 z 0 E y (t0 (y)),...,z q(y) E y (tq(y) (y)) For z 0 E y (t 0 (y)),..., z q(y) E y (t q(y) (y)), V (y; z 0,..., z q(y) ). It is clear that Moreover, by (3.2), we have z 0 E y (t0 (y)),...,z q(y) E y (tq(y) (y)) Let Clearly, V (y; z 0,..., z q(y) ) π (B m (y, δ, S)). (3.2) we pick any v(z 0,..., z q(y) ) from B m (v(z 0,..., z q(y) ), 4ɛ, T ) V (y; z 0,..., z q (y)). s=0 B m (v(z 0,..., z q(y) ), 4ε, T ) π (B m (y, δ, S)). (3.3) q(y) q(y) l(y) = E y (t s (y) = r m( y (t s (y)))(d, T, ɛ, π ( y (t s (y)))). s=0 q(y) l(y) = e s=0 log r m( y (ts (y)))(d,t,ɛ,π ( y (t s (y)))) e (a+τ) q(y) s=0 m( y(t s (y))) = e (a+τ)t q(y)+(y) e (a+τ)(m+m). Since the number of permissible (z 0,..., z q(y) ) is l(y), we may let v (y), v 2 (y),..., v l(y) (y) be an enumeration of Then by (3.3), {v(z 0,..., z q(y) ) : z 0 E y (t 0 (y)),..., z q(y) E y (t q(y) (y))}. l(y) i= This proves the claim. For any n N and s a + τ, we show that B m (v i (y), 4ɛ, T ) π (B m (y, δ, S)). M n (T, s, 4ɛ E) M n (S, s (a + τ), δ π(e))e (a+τ)m. Let {B n j (w j, δ, S)} j= be a cover of π(e) satisfying n j n for each j N. By the above claim, for each B n j (w j, δ, S) there exist l(w j ) > 0 and v (w j ), v 2 (w j ),..., v l(w j )(w j ) such that Then l(w j ) e (a+τ)(n j +M) Now j= l(w j ) i= M n (T, s, 4ɛ E) and B n j (v i (w j ), 4ε, T ) j= l(w j ) i= l(w j ) i= e sn j B n j (v i (w j ), 4ɛ, T ) π (B n j (w j, δ, S)). π (B n j (w j, δ, S)) π (π(e)) E. j= e sn j l(w j ) e (a+τ)m j= j= e (s (a+τ))n j.

608 C. Fang et al Since the above inequality is true for any {B n j (w j, δ, S)} j=, we have Let n + ; we have This implies that M n (T, s, 4ɛ E) M n (S, s (a + τ), δ π(e))e (a+τ)m. M(T, s, 4ɛ E) M(S, s (a + τ), δ π(e))e (a+τ)m. h top (T, 4ɛ E) h top (S, δ π(e)) + a + τ h top (S π(e)) + a + τ (see Proposition 3.). Finally, let ɛ 0 and τ 0; we have h top (T E) h top (S π(e)) + a. This completes the proof. For a TDS (, T ) with a metric d and surjective map T, we consider its associated natural extension or inverse limit (, T ) (, T ), where = {(x, x 2,...) : T (x i+ ) = x i, x i, i N} is a subspace of the product space N = i= endowed with the compatible metric d T : d(x i, y i ) d T ((x, x 2,...), (y, y 2,...)) = 2 i. T : is the shift homeomorphism defined by T (x, x 2,...) = (T (x ), x, x 2,...). For each i N, let π i : be the projection onto the ith coordinate. Clearly, each π i : (, T ) (, T ) is a factor map. LEMMA 3.4. Let (, T ) be a TDS with a metric d and a surjective map T, (, T ) be the natural extension of (, T ) and π : be the projection onto the first coordinate. Then h B top ( T K ) = h B top (T π (K )) for any subset K of. Proof. By Theorem 3.3, we only need to prove that h( T, π (x)) = 0 for any x. Fix a x. For any ɛ > 0, take an N N large enough such that i=n diam()/2 i < ɛ. Let E N π (x) be a finite (N, ɛ)-spanning set of π (x). We want to show that E N is also an (n, ɛ)-spanning set of π (x) for n > N. Let n N with n > N. For any ỹ π (x), since E N is an (N, ɛ)-spanning set of π (x), there exists a x E N such that d T ( T i x, T i ỹ) < ɛ for all i = 0,,..., N. Since for any k {N, N +,..., n }, π j ( T k x ) = π j ( T k ỹ)) = T k j+ (x) for all j =,..., k, k +, we have d T ( T k x, T k d(π j ( T k x ), π j ( T k x )) d(π j ( T k x ), π j ( T k x )) ỹ) = 2 j = 2 j j= j=k+2 diam() 2 j j=n i= j=k+2 diam() 2 j < ɛ. It follows that (d T ) n ( x, ỹ) < ɛ. Hence E N is also an (n, ɛ)-spanning set of π (x) for n > N. Therefore r(d T, T, ɛ, π (x)) = lim sup n n log r n(d T, T, ɛ, π (x)) lim sup n n log E N = 0. By taking ɛ 0, we have h( T, π (x)) = 0. This completes the proof.

Dimensions of stable sets and scrambled sets 609 4. Bowen dimension entropy of stable sets and scrambled sets Let (, T ) be an invertible TDS, µ M(, T ), and B µ be the completion of B with respect to µ. Then (, B µ, µ, T ) turns out to be a Lebesgue system. If {α i } i I is a countable family of finite partitions of, then the partition α = i I α i is called a measurable partition. The sets A B µ, which are unions of atoms of α, form a sub-σ - algebra of B µ, denoted by α, or simply by α if there is no confusion. Every sub-σ -algebra of B µ coincides with a σ -algebra constructed in this way (mod µ). For a given measurable partition α, we define α = n= T n α and α T = + n= T n α. In the same way, we can define F and F T, where F is a sub-σ -algebra of B µ. It is clear that for a measurable partition α of, α = ( α) and α T = ( α) T (mod µ). The Pinsker σ -algebra P µ (T ) of (, B µ, µ, T ) is defined as the smallest sub-σ - algebra of B µ containing the collection {ξ P : h µ (T, ξ) = 0}. It is well known that P µ (T ) = P µ (T ) and P µ (T ) is T -invariant, i.e. T P µ (T ) = P µ (T ). Let γ be a measurable partition of with γ = P µ (T ) (mod µ). Then µ can be disintegrated over P µ (T ) as µ = µ x dµ(x), where µ x M() and µ x (γ (x)) = for µ-a.e. x. The disintegration is characterized by properties (4.) and (4.2) below: for every f L (, B, µ), f L (, B, µ x ) for µ-a.e. x, and the map x f (y) dµ x (y) is in L (, P µ (T ), µ); (4.) for every f L (, B, µ), E µ ( f P µ (T ))(x) = For any f L (, B, µ), we also have ( ) f dµ x dµ(x) = Define for µ-a.e. x the set Ɣ x = {y : µ x = µ y }. x. f dµ x for µ-a.e. x. (4.2) f dµ. Then µ x (Ɣ x ) = for µ-a.e. LEMMA 4.. Let (, T ) be an invertible TDS and µ M(, T ). Then there exist a sequence of partitions W i P and a sequence of integers 0 = k < k 2 < satisfying: () lim i + diam(w i ) = 0; (2) lim k + H µ (P k P ) = h µ (T ), where P k = k i= T k i W i and P = k= P k ; (3) n=0 T n P = P µ (T ). Proof. The lemma follows directly from [, Proof of Lemma 4]. For the sake of completeness, we outline the construction of {W i } i= P and 0 = k < k 2 < below. Let {W i } i= be an increasing sequence of finite partitions such that lim i + diam(w i ) = 0. Take k = 0 and inductively define k, k 2,... such that ( ) H µ (P k P q ) H µ(p k Pq ) <, k =, 2,..., q, k 2q k for each q 2, where P j = j i= T k i W i. It is not hard to check that () (3) are satisfied (for example, see the proof in [3] or [25]).

60 C. Fang et al Remark 4.2. Since lim i + diam(w i ) = 0, it is easy to see that (T n P )(x) W s (x, T ) for each n N {0} and x, where (T n P )(x) is the atom of T n P containing x. Let be a compact metric space and µ M(). For K, the outer measure of K for µ is defined by µ (K ) = inf{µ(a) : A B, K A}. LEMMA 4.3. Let be a compact metric space and µ M(). If G is a Borel set with µ µ(g) =, then under the continuum hypothesis there exists a set K with µ (K ) = and K K \ G, where = {(x, x) : x }. Proof. This result is proved in [30] (see also [4, 9]). For the sake of completeness, we give a proof below. Let G = {(x, y) : (y, x) G} and set G = G G. Since µ µ(g ) =, we may assume without loss of generality that G = G. By Fubini s theorem, there exists a µ-measurable set E such that µ(e) = and µ(g y ) = for any y E, where G y = {x : (x, y) G}. Using the continuum hypothesis, we let ω be the first uncountable ordinal and let B 0, B,..., B α,... (α < ω) be the collection of all closed subsets B of with µ(b) > 0. Since µ(e) =, µ(b 0 E) = µ(b 0 ) > 0. Thus we can choose x 0 B 0 E. Since µ(b E G x0 ) = µ(b ) > 0, we can choose x B E G x0. Similarly, we can choose x 2 B 2 E G x0 G x. Suppose that for all β < α < ω, we have chosen x β B β E γ <β G x γ. To find x α, consider the set S α = B α E γ <α G xγ. As a countable intersection of full measure sets, µ(e γ <α G x γ ) =. Hence µ(s α ) = µ(b α ) > 0 and we can choose x α S α. Let K = {x α : α < ω}. The fact that K intersects every closed subset B of with µ(b) > 0 implies that µ (K ) =. Indeed, suppose otherwise, i.e. µ (K ) <. Then there exists a µ-measurable subset F of such that K F and µ(\f) > 0. Since the measure µ is regular, there exists an α 0 < ω such that B α0 \F. Then K B α0 K (\F) =, which contradicts x α0 K B α0. Now, for any (x, y) K K \, we have x = x α and y = x β for some α β. If α < β, then x β G xα and (x α, x β ) G. If α > β, then we similarly have (x β, x α ) G and (x α, x β ) G by the symmetry of G. In any case, we have (x, y) G. Remark 4.4. We conjecture that Lemma 4.3 holds without the continuum hypothesis. If this conjecture is true, then conclusions (2) in our main Theorems..3 will be true without the continuum hypothesis. Let (, T ) be a TDS. A pair {x, y} is said to be a strong Li Yorke pair for T if it is a Li Yorke pair and recurrent (meaning that (x, y) lies in the closure of {(T n x, T n y) : n }); a subset S of is called a strong scrambled set for T if for any x, y S with x y, {x, y} is a strong Li Yorke pair for T. It is clear that a

Dimensions of stable sets and scrambled sets 6 strong scrambled set for T is a scrambled set for T. Recall that a TDS (, T ) is transitive if for each pair of non-empty open subsets U and V of, there exists n 0 such that U T n V. A point x is said to be transitive if {T x, T 2 x,...} is dense in. If (, T ) is transitive, then it is well known that the set of transitive points forms a dense G δ set of (denoted by trans (T )). For ν M(, T ), the set of generic points of ν with respect to T is defined by { G ν = x : lim n } φ(t i x) = φ dν holds for any φ C(; R). (4.3) n i=0 We note that if ν is ergodic, then (supp(ν), T ) is transitive and ν(g ν ) = by the Birkhoff pointwise ergodic theorem. PROPOSITION 4.5. Let (, T ) be a zero-dimensional invertible TDS. If µ M e (, T ) with h µ (T ) > 0 and E B with µ(e) =, then the following hold. () For µ-a.e. x, h B top (T W s (x, T ) W u (x, T ) E) h µ (T ) and h B top (T W s (x, T ) W u (x, T ) E) h µ (T ). (2) Under the continuum hypothesis, for µ-a.e. x, there exists a set S x W s (x, T ) W u (x, T ) E such that: (a) S x is a strongly scrambled set for T, T ; (b) htop B (T S x) h µ (T ) and htop B (T S x ) h µ (T ). Proof. Let B µ be the completion of B with respect to µ, P µ (T ) be the Pinsker σ -algebra of (, B µ, µ, T ) and γ be the measurable partition of with γ = P µ (T ) (mod µ). Then µ can be disintegrated over P µ (T ), as µ = µ x dµ(x), where µ x M() and µ x (γ (x)) = for µ-a.e. x. Define for µ-a.e. x the set Ɣ x = {y : µ x = µ y }. Then for µ-a.e. x, µ x (Ɣ x ) =. CLAIM. supp(µ x ) W s (x, T ) W u (x, T ) for µ-a.e. x. This has already been proved in [5] (Step of the proof of Theorem 4.6). For completeness, we include the proof below. Since P µ (T ) is also the Pinsker σ -algebra of (, B µ, µ, T ) and W s (x, T ) = W u (x, T ), by symmetry it remains to prove that for µ-a.e. x, supp(µ x ) W s (x, T ). By Lemma 4., there exist {W i } i= P and 0 = k < k 2 < satisfying: () lim i + diam(w i ) = 0; (2) lim k + H µ (P k P ) = h µ (T ), where P k = k i= T k i W i and P = k= P k ; (3) n=0 T n P = P µ (T ). It is clear that P (x) W s (x, T ) for x. Let µ = µ n,x dµ(x) be the disintegration of µ over T n P for n N. Then for n N, µ n,x ((T n P )(x)) = for µ-a.e. x. Moreover, since (T n P )(x) W s (x, T ) for each x, supp(µ n,x ) W s (x, T ) for µ-a.e. x.

62 C. Fang et al Let { f i } i= be a dense subset of C(; R) with respect to the supremum norm. For each i N, by the martingale theorem, for µ-a.e. x, lim f i (y) dµ n,x (y) = lim E( f i T n P )(x) = E( f i P µ (T ))(x) = f i (y) dµ x (y). Hence there exists a measurable subset 0 with µ( 0 ) = such that for each x 0 and i N, lim f i (y) dµ n,x (y) = f i (y) dµ x (y). By a simple approximation argument, we have for each f C(; R), lim f (y) dµ n,x (y) = f (y) dµ x (y) for each x 0, i.e. lim µ n,x = µ x for x 0 under the weak -topology. For µ-a.e. x, since supp(µ n,x ) W s (x, T ) for all n N, we have supp(µ x ) W s (x, T ). This proves the claim. Since is zero dimensional, there exists a sequence of finite clopen partitions {α j } j= (i.e. each element in α j is closed and open) of such that lim j + diam(α j ) = 0. By Theorem 2., for each j N, we have ( n ) lim n I µ T i α j P µ (T ) (x) = h µ (T, α j P µ (T )) = h µ (T, α j ) for µ-a.e. x. i=0 In the above, we have used the fact that h µ (T, α P µ (T )) = h µ (T, α) for any α P. Now, since for µ-a.e. x, ( n ) I µ T i α j P µ (T ) (x) = A (x) log E( A P µ (T ))(x) i=0 A n i=0 T i α j = A (x) log µ x (A) A n i=0 T i α j (( n ) = log µ x T i α j )(x), we have log µ x (( n i=0 lim T i α j )(x)) = h µ (T, α j ) for µ-a.e. x. n Thus we easily find a Borel subset of satisfying µ( ) = and for all j N, x, log µ x (( n i=0 lim T i α j )(x)) = h µ (T, α j ). (4.4) n Since µ( ) =, µ x ( ) = for µ-a.e. x. Moreover, µ x (Ɣ x ) = for µ- a.e. x. Thus there exists a Borel set 2 of with µ( 2 ) = satisfying µ x (Ɣ x ) =. i=0

Dimensions of stable sets and scrambled sets 63 CLAIM 2. If x 2 and B x with µ x (B x) > 0, then h B top (T B x) h µ (T ) and h B top (T B x ) h µ (T ). Let x 2 and B x with µ x (B x) > 0. Put D x = B x Ɣ x. Since x 2, µ x (Ɣ x ) = and so µ x (B x) = µ x (D x) > 0. By the symmetry of T, T, h µ (T ) = h µ (T ) and D x B x. It remains to prove that h B top (T D x) h µ (T ). Since h B top (T D x) = it is sufficient to show that lim h top B (T, α j D x ) and h µ (T ) = lim h µ(t, α j ), j + j + h B top (T, α j D x ) h µ (T, α j ) for all j N. Fix j N. Without loss of generality, we suppose that h µ (T, α j ) > 0. For any ɛ (0, h µ (T, α j )) and k N, it follows from the fact that µ x = µ y for all y D x Ɣ x that D x (k, ɛ) =: = (( n {y D x : µ y i=0 (( n {y D x : µ x i=0 ) } T i α j )(y) e n(h µ(t,α j ) ɛ) for all n k ) } T i α j )(y) e n(h µ(t,α j ) ɛ) for all n k. Since D x, we have by (4.4) that k= D x (k, ɛ) = D x for any ɛ > 0. The fact µ x (D x) = implies that there is an N N such that µ x (D x(n, ɛ)) > 0. Let n N with n N and {U i : i =, 2,...} be a countable cover of D x (N, ɛ) with n T,α j (U i ) n for any i N. For each U i, there exists a B i n T,α j (U i ) l=0 T l α j such that U i B i. Hence if U i D x (N, ɛ), then B i D x (N, ɛ) U i D x (N, ɛ). Taking x i B i D x (N, ɛ), then Using the fact that we have ((n T,α j (U i ) µ x (B i ) = µ x l=0 i N:U i D x (N,ɛ) B i e n T,α j (U i )(h µ (T,α j ) ɛ) i= Since {U i : i =, 2,...} is arbitrary, ) T l α j )(x i ) e n T,α j (U i )(h µ (T,α j ) ɛ). i N:U i D x (N,ɛ) U i D x (N, ɛ), e n T,α j (U i )(h µ (T,α j ) ɛ) i N:U i D x (N,ɛ) i N:U i D x (N,ɛ) µ x (B i ) µ x (D x(n, ɛ)). m n (T, h µ (T, α j ) ɛ, α j D x (N, ɛ)) µ x (D x(n, ɛ) > 0

64 C. Fang et al for all n N, which, when passing to the limit n +, yields m(t, h µ (T, α j ) ɛ, α j D x (N, ɛ)) µ x (D x(n, ɛ)) > 0. This implies that h B top (T, α j D x (N, ɛ)) h µ (T, α j ) ɛ and therefore h B top (T, α j D x ) h B top (T, α j D x (N, ɛ)) h µ (T, α j ) ɛ. Now, by taking ɛ 0, we have h B top (T, α j D x ) h µ (T, α j ), and the claim is proved. To prove (i), we note that by Claim, and the fact that µ(e) =, there exists a Borel subset 3 of with µ( 3 ) = such that µ x (W s (x, T ) W u (x, T ) E) = µ x (E) = for all x 3. For each x 2 3, since µ x (W s (x, T ) W u (x, T ) E) =, we have by Claim 2 that h B top (T W s (x, T ) W u (x, T ) E) h µ (T ) and h B top (T W s (x, T ) W u (x, T ) E) h µ (T ). This proves (i) since µ( 2 3 ) =. To prove (ii), we note that µ( 2 ) =. By Claim 2, it is sufficient to show under the continuum hypothesis that for µ-a.e. x there exists a strong scrambled set S x W s (x, T ) W u (x, T ) E for T, T with µ x (S x) =. Define a measure λ(µ) on 2 by λ(µ) = µ x µ x dµ(x). It is well known (see [2, 5], for example) that µ x is non-atomic for µ-a.e. x and λ(µ) is a T T -invariant ergodic measure on. Let W = supp(λ(µ)). Since λ(µ) is an ergodic measure for T T, both (W, T T ) and (W, (T T ) ) are transitive. Since µ(e) =, µ x (E) = for µ-a.e. x. Hence λ(µ)(e E) = µ x µ x (E E) dµ(x) =. Let G + be the set of generic points of λ(µ) for T T and G be the set of generic points of λ(µ) for (T T ). Then λ(µ)(g + G (E E)) = and G + G (E E) W trans (T T ) W trans ((T T ) ). Since = λ(µ)(g + G (E E)) = µ x µ x (G + G (E E)) dµ(x), and µ x is non-atomic for µ-a.e. x, there exists a subset 4 with µ( 4 ) = such that µ x µ x (G + G (E E)) =, supp(µ x ) W s (x, T ) W u (x, T ), µ x (Ɣ x ) =, and µ x is non-atomic for x 4. For each x 4, let C x = W s (x, T ) W u (x, T ) E. Since µ x µ x (G + G (E E)) = and supp(µ x ) W s (x, T ) W u (x, T ), we have µ x (C x ) = and µ x µ x (G + G (C x C x )) =.

Dimensions of stable sets and scrambled sets 65 By Lemma 4.3 there exists S x such that µ x (S x) = and S x S x \ G + G (C x C x ). This implies that S x C x. Since µ x is non-atomic, S x must be uncountable. Next we show that for each x 4, S x is a strong scrambled set for T, T. Let (x, x 2 ) W trans (T T ) W trans ((T T ) ). On the one hand, since {(z, z) : z supp(µ)} W, we have lim inf d(t n x, T n x 2 ) = 0 and lim inf d(t n x, T n x 2 ) = 0. On the other hand, since µ x is non-atomic for µ-a.e. x, we have W. This implies that x x 2. Hence {x, x 2 } is a strong Li Yorke pair for T, T. Since S x S x \ W trans (T T ) W trans ((T T ) ), S x is a strong scrambled set for T, T. Definition 4.6. As in [8], an extension π : (Z, R) (, T ) between two TDSs is said to be a principal extension if h ν (R) = h πν (T ) for every ν M(Z, R). LEMMA 4.7. [8] Every invertible TDS (, T ) with h top (T ) < has a zero-dimensional principal extension (Z, R) with R being invertible. Proof. See [8, Proposition 7.8]. Remark 4.8. For an invertible TDS (, T ), Lindenstrauss and Weiss [22] introduced the mean dimension mdim(, T ) (an idea suggested by Gromov). It is well known that for an invertible TDS (, T ), if h top (T ) < or the topological dimension of is finite, then mdim(, T ) = 0 (see [22, Definition 2.6, Theorem 4.2]). In general, one can show that for an invertible TDS (, T ), if mdim(, T ) = 0, then (, T ) has a zero-dimensional principal extension (Z, R), with R being invertible. Indeed, let (Y, S) be an irrational rotation on the circle. Then ( Y, T S) admits a nonperiodic minimal factor (Y, S) and mdim( Y, T S) = 0. Hence ( Y, T S) has the so-called small-boundary property [2, Theorem 6.2], which implies the existence of a basis of the topology consisting of sets whose boundaries have measure zero for every invariant measure. With these results, it is easy to construct a refining sequence of smallboundary partitions for ( Y, T S), where the partitions have small boundaries if their boundaries have measure zero for all µ M( Y, T S). Then by a standard construction (see [8]), associated with this sequence there exists a zero-dimensional principal extension (Z, R) of ( Y, T S), with R being invertible. Finally, noting that ( Y, T S) is a principal extension of (, T ), we know that (Z, R) is also a zero-dimensional principal extension of (, T ) since the composition of two principal extensions is still a principal extension. The following theorem implies Theorem.2(). THEOREM 4.9. Let (, T ) be an invertible TDS with h top (T ) <. If µ M e (, T ), with h µ (T ) > 0 and E B with µ(e) =, then the following hold. () For µ-a.e. x, and h B top (T W s (x, T ) W u (x, T ) E) h µ (T ) h B top (T W s (x, T ) W u (x, T ) E) h µ (T ).

66 C. Fang et al (2) Under the continuum hypothesis, for µ-a.e. x, there exists a set S x W s (x, T ) W u (x, T ) E such that: (a) S x is a strong scrambled set for T, T ; (b) htop B (T S x) h µ (T ) and htop B (T S x ) h µ (T ). Proof. We only show (2) as the proof of () is similar to that of (2). By Lemma 4.7, there exists a principal extension π : (Z, R) (, T ), where Z is zero dimensional and R is invertible. Take a ν M e (Z, R) such that πν = µ. Since π (E) B Z with ν(π (E)) =, Proposition 4.5 implies that there exists a Borel set Z 0 Z with ν(z 0 ) = such that for each z Z 0 there exists a strong scrambled set S z W s (z, R) W u (z, R) π (E) for both R and R, and h B top (R S z) h ν (R) and h B top (R S z ) h ν (R). Let 0 = π(z 0 ). Then µ( 0 ) =. For x 0, we take z Z 0 with π(z) = x and define S x = π(s z ). It is clear that S x W s (x, T ) W u (x, T ) E and S x is a strong scrambled set for T, T. Since h top (T ) <, by the variational principle of condition entropy (see [, 6, 9]), we have sup x h(r, π (x)) = sup θ M(Z,R) (h θ (R) h πθ (T )) = 0 and sup x h(r, π (x)) = sup θ M(Z,R ) (h θ (R ) h πθ (T )) = 0. Now, we have by Theorem 3.3 that htop B (T S x) = htop B (R S z) h ν (R) = h µ (T ). Similarly, htop B (T S x ) h µ (T ). This proves (2). The following theorem implies part () of Theorem.. THEOREM 4.0. Let (, T ) be a TDS with h top (T ) <. If µ M e (, T ) with h µ (T ) > 0 and E B with µ(e) =, then the following hold. () h B top (T W s (x, T ) E) h µ (T ) for µ-a.e. x. (2) Under the continuum hypothesis, for µ-a.e. x, there exists a scrambled set S x W s (x, T ) E for T satisfying h B top (T S x) h µ (T ). Proof. We only prove (2) since the proof of () is similar to that of (2). Without loss of generality, we assume that T is surjective because otherwise we can replace with supp(µ). Let π : (, T ) (, T ) be the natural extension. Then T is a homeomorphism and h top ( T ) = h top (T ) <. Take ν M e (, T ) with π ν = µ. By Theorem 4.9, under the continuum hypothesis there exists a Borel subset 0 of with µ( 0 ) = such that for x 0 there exists a strong scrambled set S x W s ( x, T ) for T satisfying h B top ( T S x ) h ν ( T ). Let 0 = π ( 0 ). Obviously, µ( 0 ) =. For any x 0, take x 0 with x = π ( x ) and let S x = π (S x ). Then S x W s (x, T ), and by Lemma 3.4 h B top (T S x) = h B top ( T S x ) h ν ( T ) = h µ (T ). Now, since {ỹ, z} is a Li Yorke pair for T if and only if {π (ỹ), π ( z)} is a Li Yorke pair for T, S x is a scrambled set for T. This proves (2).

Dimensions of stable sets and scrambled sets 67 Let (, T ) be a TDS with a compatible metric d. Given ɛ > 0, the ɛ-stable set of x under T is the set of points whose forward orbit ɛ-shadows that of x: W s ɛ (x, T ) = {y : d(t n x, T n y) ɛ for all n = 0,,...}. Definition 4.. As in Bowen [6], a TDS (, T ) is called h-expansive if there exists an ɛ > 0 such that sup h(t, Wɛ s (x, T )) = 0, x and, as in Misiurewicz [23], (, T ) is called asymptotically h-expansive if lim sup h(t, Wɛ s (x, T )) = 0. ɛ 0 x It was shown by Bowen [6] that all expansive systems, expansive homeomorphisms, endomorphisms of a compact Lie group, and Axiom A diffeomorphisms are h-expansive, by Misiurewicz [23] that every continuous endomorphism of a compact metric group is asymptotically h-expansive if its entropy is finite, and by Buzzi [0] that each C diffeomorphism on a compact manifold is asymptotically h-expansive. In [23], Misiurewicz showed that for an asymptotically h-expansive system (, T ), the entropy map ν M(, T ) h ν (T ) R + is upper semi-continuous. Hence for an asymptotically h-expansive system (, T ), there always exists a µ M e (, T ) such that h µ (T ) = h top (T ) < (an asymptotically h-expansive system always has finite topological entropy). COROLLARY 4.2. Let (, T ) be an asymptotically h-expansive, invertible TDS. Then there exists x for which the following hold. () h B top (T W s (x, T ) W u (x, T )) = h top (T ) and h B top (T W s (x, T ) W u (x, T )) = h top (T ). (2) Under the continuum hypothesis, there exists a scrambled set S x W s (x, T ) W u (x, T ) satisfying h B top (T S x) = h top (T ) and h B top (T S x ) = h top (T ). Proof. We only show (2) since the proof of () is similar to that of (2). Since (, T ) is an asymptotically h-expansive system, we always have a µ M e (, T ) such that h µ (T ) = h top (T ) <. Under the continuum hypothesis, we have by part () of Theorem.2 that for µ-a.e. x there exists a scrambled set S x W s (x, T ) W u (x, T ) satisfying htop B (T S x) h µ (T ) = h top (T ) and htop B (T S x ) h µ (T ) = h top (T ). Since htop B (T E) h top (T ) and htop B (T E) h top (T ) for any E, the proof is complete. COROLLARY 4.3. Let (, T ) be an asymptotically h-expansive TDS. Then there exists x for which the following hold. () h B top (T W s (x, T )) = h top (T ). (2) Under the continuum hypothesis, there exists a scrambled set S x W s (x, T ) satisfying h B top (T S x) = h top (T ). Proof. Using Theorem 4.0, the proof is the same as that of Corollary 4.2.

68 C. Fang et al 5. Hausdorff dimension of stable sets and scrambled sets Let (, d) be a metric space. We first recall the definition of the Hausdorff dimension of a set. Fix t 0. For each δ > 0 and subset A, define { + H t,δ d (A) = inf diam(u i ) }, t i= where the infimum is taken over all countable covers {U i : i =, 2,...} of A of diameter not exceeding δ. This definition induces an outer measure on, i.e. a function defined on all subsets of taking values in [0, + ] satisfying H t,δ t,δ t,δ d ( ) = 0, Hd (A) Hd (B) if A B and ( ) H t,δ d A n H t,δ d (A n) n= n= for any countable family {A n : n =, 2,...} of subsets of. Since H t,δ d (A) increases as δ decreases for any A, we can define Hd t (A) = lim H t,δ δ 0 d (A) = sup δ>0 H t,δ d (A). The case Hd t t,δ (A) = + is not excluded. Since all Hd ( ) are outer measures, H d t ( ) is also an outer measure. It is well known that H t d (A B) = H t d (A) + H t d (B) for each pair of positively separated sets A, B, i.e. d(a, B) = inf{d(x, y) : x A, y B} > 0. The metric outer measure Hd t is called the Hausdorff outer measure associated with t. Its restriction to the σ -algebra of Hd t -measurable sets, which includes all the Borel sets, is called the Hausdorff measure associated with t. Fix A. Since for every 0 < δ the function t H t,δ d (A) is non-increasing, so is the function t Hd t (A). Moreover, if 0 < s < t, then for every δ > 0, H s,δ d (A) δs t H t,δ d (A), which implies that if Hd t (A) > 0, then H d s (A) = +. Thus there is a unique value H d (A) [0, + ], which is called the Hausdorff dimension of A with respect to the metric d on, such that { + if 0 t < Hd t (A) = Hd (A), 0 if H d (A) < t <. The Hausdorff dimension is a monotonic function of sets, i.e. if A B, then H d (A) H d (B). Moreover, if {A n } n is a countable family of subsets of, then ( ) H d A n = sup H d (A n ). n= n Hence if E is countable, then H d (E) = 0. In the following, we investigate the interrelation between the Hausdorff dimension and the Bowen entropy of a set in some specific TDSs. Let (, T ) be a TDS with a metric d. We assume that T is Lipschitz continuous, with the Lipschitz constant L, i.e. d(t x, T y) Ld(x, y) for any x, y.

Dimensions of stable sets and scrambled sets 69 It is easy to see that if h top (T ) > 0, then L >. Indeed, if L, then the metric d n = d, which results in h top (T ) = 0, a contradiction. The following result is just Theorem in [24]. LEMMA 5.. Let (, T ) be a Lipschitz continuous TDS with Lipschitz constant L > associated with a metric d. Then H d (Y ) h top B (T Y ) log L for any subset Y. Remark 5.2. Let (, T ) be as in Lemma 5.. Then h top (T ) H d () log L. Hence when H d () <, we always have h top (T ) <. When H d () =, the following example shows that h top (T ) = can happen. Example 5.3. Let = [0, ] N and be endowed with product topology. Then the compact space is metrizable, and a compatible metric on can be chosen as d(x i, y i ) d(x, y) = 2 i, for any x = (x 0, x,...), y = (y 0, y,...). With the shift map T : : i=0 T (x, x 2,...) = (x 2, x 3,...), (x, x 2,...), it is clear that (, T ) is a Lipschitz continuous TDS with Lipschitz constant L = 2, and, moreover, H d () =. It is not hard to see that h top (T ) = as well. LEMMA 5.4. Let (, T ) be a TDS with a metric d. If there exist ɛ > 0 and L > such that d(t x, T y) Ld(x, y) whenever d(x, y) < ɛ, then for any subset Y. H d (Y ) h top B (T Y ) log L Proof. Let Y be given and U be a finite open cover of with diam(u) < ɛ/2 for any U U. It is sufficient to show that H d (Y ) htop B (T, U Y )/log L. Fix k N. For any A with n T,U (A) k, it is obvious that diam(t i (A)) < ɛ for i = 0,,..., n T,U (A). Since d(t x, T y) Ld(x, y) when d(x, y) < ɛ, we have Moreover, diam(a) L n T,U (A)+ diam(t n T,U (A) A) L n T,U (A)+ ɛ L k+ ɛ. (5.) e sn T,U (A) C s,ɛ,l (diam(a)) s/log L (5.2) for any s 0, where C s,ɛ,l = (Lɛ) s/log L. Let A = {A i } i= be any cover of Y satisfying n T,U (A i ) k. Then A is a L k+ ɛ-cover of Y by (5.). By (5.2), we have e sn T,U (A i ) C s,ɛ,l i= Since A is arbitrary, we have i= (diam(a i )) s/log L C s,ɛ,l H s/log L,L k+ ɛ d (Y ), s 0. m k (T, s, U Y ) C s,ɛ,l H s/log L,L k+ ɛ d (Y ), s 0.

620 C. Fang et al Taking the limit k + yields m(t, s, U Y ) C s,ɛ,l H s/log L d (Y ), s 0. This implies that H d (Y ) htop B (T, U Y )/log L. We are now ready to prove part (2) of Theorem. and part (2) of Theorem.2. Proof of part (2) of Theorem.2. Let µ be a T -invariant ergodic measure with h µ (T ) > 0. Under the continuum hypothesis, we have by Theorem 4.9 that for µ-a.e. x there exists a scrambled set S x W s (x, T ) W u (x, T ) for both T and T such that h B top (T S x) h µ (T ). Moreover, since d(t x, T y) Ld(x, y) for any x, y, we have by Lemma 5. that H d (S x ) h B top (T S x) log L h µ(t ) log L for µ-a.e. x. The proof of the remaining part of (2) uses a similar argument. Proof of part (2) of Theorem.. By using Theorem 4.0 and Lemma 5., the proof is the same as that of part (2) of Theorem.2. Using Remark 5.2 and part (2) of Theorem., we have the following result. THEOREM 5.5. Let (, T ) be a TDS with a metric d such that H d () < and T be a Lipschitz continuous self-map with Lipschitz constant L >. If µ is a T -invariant ergodic measure with h µ (T ) > 0, then the following hold. () H d (W s (x, T )) h µ (T )/log L for µ-a.e. x. (2) Under the continuum hypothesis, for µ-a.e. x there exists a scrambled set S x W s (x, T ) for T such that H d (S x ) h µ (T )/log L. As a direct consequence of the above theorem, we have the following results. COROLLARY 5.6. Let (, T ) be a TDS with a metric d such that h top (T ) < and T be a Lipschitz continuous self-map with Lipschitz constant L >. If there exists µ M e (, T ) such that h µ (T ) = h top (T ), then the following hold. () H d (W s (x, T )) h top (T )/log L for µ-a.e. x. (2) Under the continuum hypothesis, for µ-a.e. x there exists a scrambled set S x W s (x, T ) for T satisfying H d (S x ) h top (T )/log L. COROLLARY 5.7. Let (, T ) be an asymptotically h-expansive TDS with a metric d and T be a Lipschitz continuous self-map with Lipschitz constant L >. Then there exists x for which the following hold. () H d (W s (x, T )) h top (T )/log L. (2) Under the continuum hypothesis, there exists a scrambled set S x W s (x, T ) for T satisfying H d (S x ) h top (T )/log L. Proof. Since (, T ) is an asymptotically h-expansive system, there exists a µ M e (, T ) such that h µ (T ) = h top (T ) <. Hence the corollary follows from Corollary 5.6.

Dimensions of stable sets and scrambled sets 62 Let A = {0,,..., N } for some integer N 2 endowed with the discrete metric d, and + (N) be the space of one-sided sequences in A endowed with the product topology. Then + (N) is metrizable, and a compatible metric ρ on (N) can be chosen as ρ(x, y) = i=0 d(x i, y i ) N i, x = (x 0, x,...), y = (y 0, y,...) + (N). We consider the shift map σ : + (N) + (N): σ (x) i = x i+, i = 0,,.... If is a σ -invariant non-empty closed subset of + (N), then we say that (, σ ) is a subshift of ( + (N), σ ). THEOREM 5.8. Let (, σ ) be a subshift of ( + (N), σ ) with a metric ρ as above. Then there exists µ M e (, σ ) with h µ (σ ) = h top (σ, ) for which the following hold. () H ρ (W s (x, σ )) = h top (σ, )/log N for µ-a.e. x. (2) Under the continuum hypothesis, for µ-a.e. x there exists a scrambled set S x W s (x, σ ) for σ satisfying H ρ (S x ) = h top (σ, )/log N. Proof. Since the entropy map θ M(, T ) h θ (σ, ) is upper semi-continuous, there exists µ M e (, T ) such that h µ (σ ) = h top (σ, ). To finish the proof, we only need to show (2) since the proof of () is similar to that of (2). Since ρ(σ x, σ y) Nρ(x, y) for any x, y, it follows from Theorem 5.5 that for µ-a.e. x there exists a scrambled set S x W s (x, σ ) for σ satisfying H ρ (S x ) h µ(σ ) log N = h top(σ ) log N. Since ρ(σ x, σ y) Nρ(x, y) for any x, y with ρ(x, y) < /N, Lemma 5.4 implies that H ρ (S x ) h top(σ S x ) log N h top(σ ) log N for µ-a.e. x. Hence H ρ (S x ) = h top (σ, )/log N for µ-a.e. x. Remark 5.9. () In [2], Furstenberg proved that for a subshift (, σ ) of ( + (N), σ ), H ρ () = h top (σ, )/log N. Hence for any E, we always have H ρ (E) h top (σ, )/log N (see also Lemma 5.4). (2) Let (, σ ) be a subshift of ( + (N), σ ). For any x, it is clear that W s (x, σ ) is a countable set. Hence htop B (σ W s (x, σ )) = 0 and H ρ (W s (x, σ )) = 0 (in fact, the former is true for any TDS (, T )). Thus taking closures of the stable sets in the statement of Theorems. and.2 is necessary. We end this section by posing the following questions. Question 5.0. Let (, T ) be an invertible TDS and µ be a T -invariant ergodic measure on with h µ (T ) > 0. Do the following statements hold for µ-a.e. x? () h B top (T W u (x, T )) h µ (T ) and h B top (T W s (x, T )) h µ (T ). (2) (W s (x, T ) W u (x, T ))\{x}.