C. FANG, W. HUANG, Y. YI, AND P. ZHANG

Transcription

1 DIMENSION OF STABLE SETS AND SCRAMBLED SETS IN POSITIVE FINITE ENTROPY SYSTEMS C. FANG, W. HUANG, Y. YI, AND P. ZHANG Abstract. We study dimension of stable sets and scrambled sets of a dynamical system with positive finite entropy. We show that there is a measure-theoretically large set containing points whose sets of hyperbolic points (i.e., points lying in the intersections of the closures of the stable and unstable sets) admit positive Bowen dimension entropies, which, under a continuum hypothesis, also contains a scrambled set with positive Bowen dimension entropy. For several kinds of specific invertible dynamical systems, lower bounds of Hausdorff dimension of these sets are estimated. In particular, for a diffeomorphism on a smooth Riemannian manifold with positive entropy, such a lower bound is given in terms of the metric entropy and Lyapunov exponent.. Introduction Throughout of the paper, by a topologically dynamical system (, T) (TDS for short) we mean a compact metric space (,d) with a continuous map T from into itself, where d denotes the metric on. If T is a homeomorphism, then we say that the TDS (,T) is invertible. For a TDS (,T), the stable set of a point x is defined as W s (x,t) = {y : lim d(t n x,t n y) = 0}. If (,T) is invertible, then we can also define the unstable set of a point x as W u (x,t) = {y : lim d(t n x,t n y) = 0}. A pair of points x,y is said to be a Li-Yorke pair with modulus δ if lim supd(t n x,t n y) = δ > 0 and lim inf d(t n x,t n y) = 0. A subset S is called scrambled if any pair of distinct points x,y S forms a Li-Yorke pair. If contains an uncountable scrambled set, then the TDS (,T) is said to be chaotic in the sense of Li-Yorke [20]. For a general TDS with positive entropy, it was shown by Blanchard et al [2] that any TDS with positive entropy always contains an uncountable scrambled set, hence it is chaotic in the sense of Li-Yorke. In fact, Blanchard and Huang [3] further showed that such a TDS even admits a certain amount of weak mixing dynamics a stronger chaos than that defined by Li and Yorke. For an Anosov diffeomorphism T on a compact manifold, it is well-known that points belonging to the stable set are asymptotic under T and tend to diverge under T, while pairs belonging to the unstable set behave in the opposite way. A natural question concerning Dedicated to the memory of Professor Dan Rudolph 99 Mathematics Subject Classification. 37B05, 54H20. The second author was partially supported by NSFC, Fok Ying Tung Education Foundation, FANEDD (Grant ) and the Fundamental Research Funds for the Central Universities. The Third author was partially supported by NSF grant DMS and a Scholarship from Jilin University.

2 2 C. FANG, W. HUANG, Y. YI, AND P. ZHANG a general TDS with finite positive entropy is then whether it can retains a faint flavor of such a dynamical behavior. Recently, Blanchard, Host and Ruette [] showed for any invertible TDS (,T) with positive entropy that the stable sets for T are not stable for T, i.e., if a T-invariant ergodic measure µ has positive entropy, then there is δ > 0 such that for µ-a.e. x, one can find an uncountable subset F x of W s (x,t) such that for any y F x \ {x}, {x,y} forms a Li-Yorke pair for T with modulus δ. For a C 2 diffeomorphism f on a closed smooth manifold M, it was further shown by Sumi [29] that if the metric entropy with respect to a f-invariant ergodic probability measure µ is positive, then, for µ-a.e. x M, both the stable set W s (x,f) and the unstable set W u (x,f) contain uncountable scrambled sets. Generalizing this to a TDS, it was shown by the second author in [4] that in any (invertible) TDS with positive entropy there is a measure-theoretically rather big set such that the closure of the stable (or unstable) sets of points in the set contains a weak mixing set. With these known descriptions and characterizations, it is of fundamental importance to know how big these stable sets or scrambled sets can be in a positive entropy system. Usually one refers a set E in a TDS (,T) as big if one of the following properties holds: a) E is of a positive measure with respect to some T-invariant measure on ; b) E has a non-empty interior or is a dense G δ set in ; c) h B top(t E), the Bowen dimension entropy of E, is positive; d) H d (E), the Hausdorff dimension of E with respect to d, is positive. It is not hard to see that, the properties a) and b) do not hold in general for a scrambled set or the closure of a stable set. This leave the properties c) and d) as possible criteria to describe the bigness. The purpose of this paper is to investigate the Bowen dimension entropy and Hausdorff dimension of the stable sets and scrambled sets in a TDS with positive finite entropy. Our main results state as follows. Theorem. Let (,T) be a TDS with metric d and h top (T) <. If µ is a T-invariant ergodic measure on with h µ (T) > 0, then the following holds. ) (Bowen dimension) h B top (T W s (x,t)) h µ (T) for µ-a.e x, and moreover, under the continuum hypothesis, for µ-a.e x there exists a scrambled set S x W s (x,t) for T satisfying h B top(t S x ) h µ (T). 2) (Hausdorff dimension) If T is a Lipschitz continuous self-map with Lipschitz constant L >, then H d (W s (x,t)) hµ(t) for µ-a.e x, and moreover, under the continuum hypothesis, for µ-a.e x there exists a scrambled set S x W s (x,t) for T satisfying H d (S x ) hµ(t). Theorem 2. Let (,T) be an invertible TDS with metric d and h top (T) <. If µ is a T-invariant ergodic measure on with h µ (T) > 0, then the following holds. ) (Bowen dimension) For µ-a.e x, h B top (T W s (x,t) W u (x,t)) h µ (T) and h B top (T W s (x,t) W u (x,t)) h µ (T).

3 DIMENSION OF STABLE SETS AND SCRAMBLED SETS 3 Moreover, under the continuum hypothesis, for µ-a.e x there exists S x W s (x,t) W u (x,t) which is a scrambled set for both T and T such that h B top (T S x) h µ (T) and h B top (T S x ) h µ (T). 2) (Hausdorff dimension) If T is Lipschitz continuous with Lipschitz constant L >, then H d (W s (x,t) W u (x,t)) hµ(t) for µ-a.e x, and moreover, under the continuum hypothesis, for µ-a.e x there exists S x W s (x,t) W u (x,t) which is a scrambled set for both T and T such that H d (S x ) hµ(t). Using the variational principle of entropy, the following result is a direct consequence of Theorems, 2. Corollary. Let (,T) be a TDS with metric d. We assume that h top (T) < and T is a Lipschitz continuous self-map with Lipschitz constant L >. Then the following holds. a) sup x H d (W s (x,t)) htop(t) ; and under the continuum hypothesis, sup{h d (S) : S is a scrambled set for T } h top(t). b) If T is invertible, then sup x H d (W s (x,t) W u (x,t)) htop(t) ; and under the continuum hypothesis, sup{h d (S) : S is a scrambled set for T,T } h top(t). For a differentiable self-map on a smooth Riemannian manifold, a lower bound of the Hausdorff dimension of the stable set and the scrambled set with respect to an ergodic measure µ can be estimated in terms of the metric entropy and the top Lyapunov exponent χ µ (see Section 6 for detail). Theorem 3. Let f be a C self-map on a smooth Riemannian manifold M and ρ be the metric on M induced by the Riemannian structure. If µ is a f-invariant ergodic measure with a compact support Λ such that h µ (f) > 0, then the following holds. ) H ρ (W s (x,f)) hµ(f) χ for µ-a.e. x Λ, and under the continuum hypothesis, for µ- µ a.e. x Λ there exists a scrambled set S x W s (x,f) for f satisfying H ρ (S x ) hµ(f). χ µ 2) If f is a diffeomorphsim, then H ρ (W s (x,f) W u (x,f)) hµ(f) for µ-a.e. x Λ, χ µ and under the continuum hypothesis, for µ-a.e. x Λ there exists a scrambled set S x W s (x,f) W u (x,f) for both f and f satisfying H ρ (S x ) hµ(f) χ. µ Remark. We conjecture that Theorems, 2 still hold without assuming h top (T) <. We also conjecture that the continuum hypothesis contained in all Theorems -3 should be removed. The paper is organized as follows. Section 2 is a preliminary section in which we review some notions of ergodic theory and TDS. In Section 3, we discuss some basic properties of Bowen dimension entropy for non-compact sets which will be used in later sections. In Section 4, we prove parts ) of Theorems,2 by estimating a lower bound of Bowen dimension entropy for stable sets as well as for scrambled sets in a positive finite entropy system. In Section 5, we prove parts 2) of Theorems, 2 by estimating a lower bound of Hausdorff dimension for

4 4 C. FANG, W. HUANG, Y. YI, AND P. ZHANG stable sets as well as for scrambled sets in a Lipschitz continuous TDS with positive finite entropy. We prove Theorem 3 in Section Preliminary Given a TDS (,T), we denote by B the σ-algebra of Borel subsets of. A cover of is a family of Borel subsets of whose union is. An open cover is one that consists of open sets. A partition of is a cover of consisting of pairwise disjoint sets. Given a partition α of and x, we denote by α(x) the atom of α containing x. We denote the set of finite partitions, finite covers and finite open covers, of, respectively, by P, C and C o, respectively. Given two covers U, V of, U is said to be finer than V (denote by U V) if each element of U is contained in some element of V. Let U V = {U V : U U,V V}. It is clear that U V U and U V V. Given integers M,N with M N and U C, we use UM N to denote N n=m T n U. For U C, we define N(U) as the minimum among the cardinalities of the subcovers of U. Then the topological entropy of U with respect to T is defined by h top (T, U) = lim N + log N(UN 0 ) = inf N N N The topological entropy of (,T) is defined by h top (T) = sup h top (T, U). U C o log N(UN 0 ). N We sometimes write h top (T,) to emphasis the dependence of the entropy on the space. Let K be a non-empty closed subset of. For ǫ > 0, a subset F of is called (n,ǫ)- spanning set of K, if for any x K, there exists y F with d n (x,y) ǫ, where d n (x,y) = max 0 i n d(t i x,t i y); a subset E of K is called (n,ǫ)-separated set of K if x,y E,x y implies d n (x,y) > ǫ. Let r n (d,t,ǫ,k) denote the smallest cardinality among all (n,ǫ)- spanning sets of K and s n (d,t,ǫ,k) denote the largest cardinality among all (n,ǫ)-separated subsets of K. We define r(d,t,ǫ,k) = lim sup s(d,t,ǫ,k) = lim sup n log r n(d,t,ǫ,k), n log s n(d,t,ǫ,k). Obviously, r(d, T, ǫ, K) and s(d, T, ǫ, K) are monotonically increasing when ǫ ց 0. Let h (d,t,k) = lim ǫ 0+ r(d,t,ǫ,k) and h (d,t,k) = lim ǫ 0+ s(t,d,ǫ,k). It is well-known that h (d,t,k) = h (d,t,k) which is independent of the choice of a compatible metric d on, so we simply denote it by h(t,k). When K =, h(t,) = h top (T). and Given U C, define N(U K) = min{the cardinality of F F U, h(t, U K) = lim sup It is easy to see that h(t,k) = sup U C o h(t, U K). F F n n log N( T i U K). i=0 F K},

5 DIMENSION OF STABLE SETS AND SCRAMBLED SETS 5 Let M(), M(,T), and M e (,T), respectively, be the set of all Borel probability measures, T-invariant Borel probability measures, and T-invariant ergodic Borel probability measures on, respectively. Then M() and M(, T) are all convex, compact metric spaces when endowed with the weak -topology. For any given α P, µ M() and any sub-σ-algebra C B µ, where B µ is the completion of B under µ, the conditional informational function of α relevant to C is defined by I µ (α C)(x) := A α A (x)log E( A C)(x), where E( A C) is the conditional expectation of A with respect to C. Let H µ (α C) = I µ (α C)(x)dµ(x) = E( A C)log E( A C)dµ. A α Then H µ (α C) increases with respect to α and decreases with respect to C. When µ M(,T) and C is T-invariant (i.e. T C = C), it is not hard to see that H µ (α0 n C) is a non-negative and sub-additive sequence for a given α P. Thus h µ (T,α C) = lim n H µ(α n 0 C) = inf n n H µ(α n 0 C) is well defined. It is well-known that h µ (T,α C) = H µ (α n= T n α C). If C = {,}(mod µ), we write H µ (α C) and h µ (T,α C) by H µ (α) and h µ (T,α) respectively. The measuretheoretic entropy of µ is defined by If, in addition, T is invertible, then h µ (T) = sup α P h µ (T,α). h µ (T,α C) = h µ (T,α C) for any α P and h µ (T) = h µ (T ). It is also well-known that for α P, h µ (T,α) = h µ (T,α P µ (T)) H µ (α P µ (T)), where P µ (T) is the Pinsker σ-algebra of (, B µ,µ,t). The following result is a conditional version of Shanon-McMillan-Breiman Theorem. Its proof is completely similar to the proof of Shanon-McMillan-Breiman Theorem (see e.g. [3]). Theorem 2.. Let (,T) be an invertible TDS, µ M(,T), α P and C B µ a T-invariant sub-σ-algebra. Then there exists a T-invariant function f L (µ) such that f(x)dµ(x) = h µ (T,α C) and lim I µ (α n 0 C)(x) = f(x) for µ-a.e. x n as well as in the sense of L (µ). Moreover, if µ is ergodic then f(x) = h µ (T,α C) for µ-a.e. x. 3. Bowen dimension entropy of non-compact set The notion Bowen dimension entropy for non-compact sets in a TDS was introduced by Bowen in [7] and Pesin and Pitskel in [27]. Let (,T) be a TDS and U C. For E we write E U if E is contained in some element of U. Let n T,U (E) be the biggest non-negative integer such that T k E U for every k {0,,,n T,U (E) }; n T,U (E) = 0 if E U and n T,U (E) = + if T k E U for any k Z +.

6 6 C. FANG, W. HUANG, Y. YI, AND P. ZHANG Let Y. For each s 0 and k N, denote m k (T,s, U Y ) = inf{ e sn T,U(U i ) : i N U i Y and n T,U (U i ) k for each i N }. Since m k (T,s, U Y ) is increasing with respect to k N, m(t,s, U Y ) =: lim k + m k(t,s, U Y ) is well defined. It is clear that m(t,s, U Y ) m(t,s, U Y ) if s s 0 and m(t,s, U Y ) / {0,+ } for at most one point s 0. We define the Bowen dimension entropy of Y relative to U by h B top (T, U Y ) = inf{s 0 : m(t,s, U Y ) = 0} = sup{s 0 : m(t,s, U Y ) = + }, and define the Bowen dimension entropy of Y by h B top(t Y ) = sup h B top(t, U Y ). U C o When Y =, we omit the restriction on and simply denote h B top (T, U ), hb top (T ) by h B top (T, U), hb top (T), respectively. It is well-known that hb top (T) = h top(t) (see [7]). We now give an equivalent definition of Bowen dimension entropy of non-compact set. For k N, x and r > 0, let B k (x,r,t) = {x : d k (x,x ) < r} be the Bowen ball. We define B (x,r,t) = {x} when x and r > 0. Let E and ǫ > 0. For any n N and s 0, denote M n (T,s,ǫ E) = inf{ e nis : B n i (x i,ǫ,t) E and n i n for i N}. Since M n (T,s,ǫ Y ) is increasing with respect to n N, M(T,s,ǫ E) =: lim M n(t,s,ǫ E) is well defined. It is clear that M(T,s,ǫ E) M(T,s,ǫ E) if s s 0 and M(T,s,ǫ E) / {0,+ } for at most one point s 0. Let h B top (T,ǫ E) = inf{s 0 : M(T,s,ǫ E) = 0} = sup{s 0 : M(T,s,ǫ E) = + }. The following result is well-known (See e.g. [26], Remark (), pp.74). Proposition 3.. Let (,T) be a TDS and E. Then h B top(t E) = lim ǫ 0 h B top(t,ǫ E). The Bowen dimension entropy is a monotonic function of sets, i.e., if E F then h B top(t E) h B top(t F). Moreover if {E n } n is a countable family of subsets of then h B top (T n= Hence if E is countable, then h B top (T E) = 0. E n ) = suph B top (T E n). n

7 DIMENSION OF STABLE SETS AND SCRAMBLED SETS 7 Let (,T) and (Y,S) be two TDSs. A continuous map π : (,T) (Y,S) called a homomorphism or a factor map between (,T) and (Y,S) if it is onto and πt = Sπ. In this case we say (,T) is an extension of (Y,S) or (Y,S) is a factor of (,T). The following results are elementary (See e.g., [7, Proposition 2]). Proposition 3.2. Let (,T) and (Y,S) be two TDSs and π : (,T) (Y,S) be a factor map. Then for any E, () h B top(t E) = h B top(t T(E)); (2) h B top(t E) h B top(s π(e)); (3) h B top(t k E) = kh B top(t E), k N. Theorem 3.3. Let π : (,T) (Y,S) be a factor map between two TDSs. Then for any E, (3.) h B top(t E) h B top(s π(e)) + sup h(t,π (y)). y Y Proof. We follow the argument in the proof of Theorem 7 in [5]. Let d be a compatible metric on and ρ a compatible metric on Y. If sup y Y h(t,π (y)) =, (3.) is clear. In the following we assume that a := sup y Y h(t,π (y)) <. Fix ǫ > 0 and τ > 0. For each y Y, choose m(y) N such that m(y) log r m(y)(d,t,ǫ,π (y)) h(t,π (y)) + τ a + τ. Let E y be a (m(y),ǫ)-spanning set of π (y) with E y = r m(y) (d,t,ǫ,π (y)). Denote U y = {u : there exists z E y such that d m(y) (u,z) < 2ǫ}. Then U y is an open neighborhood of π (y) and ( \ U y ) γ>0 π (B γ (y)) =, where B γ (y) = {y Y : ρ(y,y) < γ}. By the finite intersection property of compact sets, there is a W y = B γy (y) for some γ y > 0 such that U y π (W y ). Since Y is compact, there exist y,,y r such that W y,,w yr cover Y. Let δ > 0 be a Lebesgue number of open cover {W y,,w yr } with respect to ρ, and denote δ = δ 2, M = max{m(y ),...,m(y r )}. Let y Y and m N. We claim that there exist l(y) > 0 and v (y),v 2 (y),,v l(y) (y) such that l(y) e (a+τ)(m+m) and l(y) B m(v i (y),4ǫ,t) π (B m (y,δ,s)), where B m (y,δ,s) = {y Y : ρ m (y,y ) < δ} and B m (v i (y),4ǫ,t) = {x : d m (v i (y),x ) < 4ǫ}. For each 0 j < m, we choose y (j) {y,...,y r } such that B δ (S j (y)) W y(j). Define the sequence t 0,,t q depending on y recursively such that t 0 (y) = 0 and t s+ (y) = t s (y) + m( y (t s (y))) until one gets a t q+ (y) m. For z 0 E y(t 0 (y)),z E y(t (y)),,z q(y) E y(t q(y) (y)), we let It is not hard to see that (3.2) V (y;z 0,...,z q(y) ) = {u : d(t t+ts(y) (u),t t (z s )) < 2ε for all 0 t < m( y (t s (y))) and 0 s q(y)}. z 0 E y(t0 (y)),,z q(y) E y(tq(y) (y)) V (y;z 0,...,z q(y) ) π (B m (y,δ,s)).

8 8 C. FANG, W. HUANG, Y. YI, AND P. ZHANG For z 0 E y(t 0 (y)),...,z q(y) E y(t q(y) (y)), we pick any v(z 0,...,z q(y) ) from V (y;z 0,...,z q(y) ). It is clear that B m (v(z 0,...,z q(y) ),4ǫ,T) V (y;z 0,...,z q (y)). Moreover, by (3.2), we have (3.3) B m (v(z 0,...,z q(y) ),4ε,T) π (B m (y,δ,s)). z 0 E y(t0 (y)),,z q(y) E y(tq(y) (y)) Let l(y) = q(y) s=0 E y(t s(y) = q(y) s=0 r m( y(t s(y)))(d,t,ǫ,π ( y (t s (y)))). Clearly, l(y) = e q(y) log r m( y(ts(y))) (d,t,ǫ,π ( y(t s(y)))) s=0 q(y) e (a+τ) m( y(t s(y))) s=0 = e (a+τ)t q(y)+ (y) e (a+τ)(m+m). Since the number of permissible (z 0,...,z q(y) ) is l(y), we may let v (y),v 2 (y),,v l(y) (y) be an enumeration of {v(z 0,...,z q(y) ) : z 0 E y(t 0 (y)),...,z q(y) E y(t q(y) (y))}. Then by (3.3), l(y) B m(v i (y),4ǫ,t) π (B m (y,δ,s)). This proves the claim. For any n N and s a + τ, we are to show that M n (T,s,4ǫ E) M n (S,s (a + τ),δ π(e))e (a+τ)m. Let {B nj (w j,δ,s)} j= be a cover of π(e) satisfying n j n for each j N. By the above claim, for each B nj (w j,δ,s) there exist l(w j ) > 0 and v (w j ),v 2 (w j ),,v l(wj )(w j ) such that l(w j ) e (a+τ)(nj+m) and l(w j ) Now j= B nj (v i (w j ),4ǫ,T) π (B nj (w j,δ,s)). l(wj ) B nj (v i (w j ),4ε,T) j= π (B nj (w j,δ,s)) π (π(e)) E. Then M n (T,s,4ǫ E) j= l(w j ) e sn j e sn j l(w j ) e (a+τ)m e (s (a+τ))n j. Since the above inequality is true for any {B nj (w j,δ,s)} j=, we have j= j= M n (T,s,4ǫ E) M n (S,s (a + τ),δ π(e))e (a+τ)m. Let n +, we have M(T,s,4ǫ E) M(S,s (a + τ),δ π(e))e (a+τ)m. This implies that h top (T,4ǫ E) h top (S,δ π(e)) + a + τ h top (S π(e)) + a + τ (see Proposition 3.). Finally, let ǫ ց 0 and τ ց 0, we have h top (T E) h top (S π(e)) + a. This completes the proof. For a TDS (,T) with metric d and surjective map T, we define a natural extension (, T) (,T), where = {(x,x 2, ) : T(x i+ ) = x i,x i,i N} is a subspace of the product space N = Π endowed with the compatible metric d T: d T ((x,x 2, ),(y,y 2, )) = d(x i,y i ) 2 i, T : is the shift homeomorphism defined by T(x,x 2, ) = (T(x ),x,x 2, ). For each i N, let π i : be the projection onto the i-th coordinate. Clearly, each π i : (, T) (,T) is a factor map. Lemma 3.4. Let (,T) be a TDS with a metric d and a surjective map T, (, T) be the natural extension of (,T) and π : be the projection onto the first coordinate. Then h B top ( T K) = h B top (T π (K)) for any subset K of.

9 DIMENSION OF STABLE SETS AND SCRAMBLED SETS 9 Proof. By Theorem 3.3, we only need to prove that h( T,π (x)) = 0 for any x. Fix a x. For any ǫ > 0, take an N N large enough such that diam() i=n < ǫ. 2 i Let E N π (x) be a finite (N,ǫ)-spanning set of π (x). We want to show that E N is also an (n,ǫ)-spanning set of π (x) for n > N. Let n N with n > N. For any ỹ π (x), since E N is an (N,ǫ)-spanning set of π (x) there exists a x E N such that d T ( T i x, T i ỹ) < ǫ for all i = 0,,,N. Since for any k {N,N +,,n }, π j ( T k x) = π j ( T k ỹ)) = T k j+ (x) for all j =,,k,k +, we have d T ( T k x, T k ỹ) = j= j=k+2 d(π j ( T k x),π j ( T k x)) 2 j = diam() 2 j j=n j=k+2 diam() 2 j < ǫ. d(π j ( T k x),π j ( T k x)) 2 j It follows that (d T ) n ( x,ỹ) < ǫ. Hence E N is also an (n,ǫ)-spanning set of π (x) for n > N. Therefore, r(d T, T,ǫ,π (x)) = lim sup n n log r n(d T, T,ǫ,π (x)) lim sup n n log E N = 0. By taking ǫ 0, we have h( T,π (x)) = 0. This completes the proof. 4. Bowen dimension entropy of stable sets and scrambled sets Let (,T) be an invertible TDS, µ M(,T), and B µ be the completion of B with respect to µ. Then (, B µ,µ,t) turns out to be a Lebesgue system. If {α i } i I is a countable family of finite partitions of, then the partition α = i I α i is called a measurable partition. The sets A B µ, which are unions of atoms of α, form a sub-σ-algebra of B µ, denoted by α or simply by α if there is no confusion. Every sub-σ-algebra of B µ coincides with a σ- algebra constructed in this way (mod µ). For a given measurable partition α, we define α = n= T n α and α T = + n= T n α. In the same way we can define F and F T where F is a sub-σ-algebra of B µ. It is clear that for a measurable partition α of, α = ( α) and α T = ( α) T (mod µ). The Pinsker σ-algebra P µ (T) of (, B µ,µ,t) is defined as the smallest sub-σ-algebra of B µ containing the collection {ξ P : h µ (T,ξ) = 0}. It is well-known that P µ (T) = P µ (T ) and P µ (T) is T-invariant, i.e. T P µ (T) = P µ (T). Let γ be a measurable partition of with γ = P µ (T) (mod µ). Then µ can be disintegrated over P µ (T) as µ = µ xdµ(x) where µ x M() and µ x (γ(x)) = for µ-a.e. x. The disintegration is characterized by properties (4.) and (4.2) below: (4.) for every f L (, B,µ),f L (, B,µ x ) for µ-a.e. x, and the map x f(y)dµ x(y) is in L (,P µ (T),µ); (4.2) for every f L (, B,µ),E µ (f P µ (T))(x) = f dµ x for µ-a.e. x.

10 0 C. FANG, W. HUANG, Y. YI, AND P. ZHANG For any f L (, B,µ), we also have ( ) f dµ x dµ(x) = f dµ. Define for µ-a.e. x the set Γ x = {y : µ x = µ y }. Then µ x (Γ x ) = for µ-a.e. x. Lemma 4.. Let (,T) be an invertible TDS and µ M(,T). Then there exist a sequence of partitions W i P and a sequence of integers 0 = k < k 2 < satisfying: () lim i + diam(w i) = 0, (2) lim H µ(p k P ) = h µ (T), where P k = k k + (3) T n P = P µ (T). n=0 T k i W i and P = k= P k, Proof. The lemma follows directly from the proof of Lemma 4 in []. For the sake of completeness, we outline the construction of {W i } P and 0 = k < k 2 < below. Let {W i } be an increasing sequence of finite partitions such that lim diam(w i) = 0. i + Take k = 0 and inductively define k,k 2, such that H µ (P k Pq ) H µ(p k Pq ) < ( k ), k =,2,,q, 2q k for each q 2, where P j = j T k i W i. It is not hard to check that ()-(3) are satisfied (see e.g., the proof in [3] or [25]). Remark 4.2. Since lim i + diam(w i ) = 0, it is easy to see that (T n P )(x) W s (x,t) for each n N {0} and x, where (T n P )(x) is the atom of T n P containing x. Let be a compact metric space and µ M(). For K, the outer measure of K for µ is defined by µ (K) = inf{ µ(a n ) : A n B,K A n }. n N n N Lemma 4.3. Let be a compact metric spaces and µ M(). If G is a Borel set with µ µ(g) =, then under the continuum hypothesis there exists a set K with µ (K) = and K K \ G, where = {(x,x) : x }. Proof. This result is proved in [30] (see also [9] or [4, Lemma 52]). For the sake of completeness, we give a proof below. Let G = {(x,y) : (y,x) G} and set G = G G. Since µ µ(g ) =, we may assume without loss of generality that G = G. By Fubini s Theorem, there exists a µ-measurable set E such that µ(e) = and µ(g y ) = for any y E, where G y = {x : (x,y) G}. Using the Continuum Hypothesis, we let ω be the first uncountable ordinal and let B 0,B,,B α, (α < ω) be the collection of all closed subsets B of with µ(b) > 0. Since µ(e) =, µ(b 0 E) = µ(b 0 ) > 0. Thus we can choose a x 0 B 0 E. Since µ(b E G x0 ) = µ(b ) > 0, we can choose a x B E G x0. Similarly we can

11 DIMENSION OF STABLE SETS AND SCRAMBLED SETS choose a x 2 B 2 E G x0 G x. Suppose that for all β < α < ω, we have chosen x β B β E γ<β G x γ. To find x α, consider the set S α = B α E G xγ. As a countable intersection of full measure sets, µ(e γ<α G x γ ) =. Hence µ(s α ) = µ(b α ) > 0 and we can choose a x α S α. Let K = {x α : α < ω}. The fact that K intersects every closed subset B of with µ(b) > 0 implies that µ (K) =. Indeed, suppose for otherwise µ (K) <. Then there exists a µ-measurable subset F of such that K F and µ( \F) > 0. Since the measure µ is regular, there exists an α 0 < ω such that B α0 \F. Then K B α0 K ( \ F) =, which is a contradiction to the fact that x α0 K B α0. Now for any (x,y) K K \, we have x = x α and y = x β for some α β. If α < β, then x β G xα and (x α,x β ) G. If α > β, then we similarly have (x β,x α ) G and (x α,x β ) G by the symmetry of G. In any case, we have (x,y) G. Remark 4.4. We conjecture that Lemma 4.3 holds without the continuum hypothesis. If this conjecture is true, then conclusions (2) in our main Theorems -3 will be true without the continuum hypothesis. Let (,T) be a TDS. A pair {x,y} is said to be a strong Li-Yorke pair for T if it is a Li-Yorke pair and recurrent (meaning that (x,y) lies in the closure of {(T n x,t n y) : n }); a subset S of is called a strong scrambled set for T if for any x,y S with x y, {x,y} is a strong Li-Yorke pair for T. It is clear that a strong scrambled set for T is a scrambled set for T. Recall that a TDS (,T) is transitive if for each pair of non-empty open subsets U and V of, there exists n 0 such that U T n V. A point x is said to be a transitive point if orb(x,t) = {Tx,T 2 x,...} is dense in. If (,T) is transitive then it is well-known that the set of transitive points forms a dense G δ set of (denoted by trans (T)). For ν M(,T), the set of generic points of ν with respect T is defined by n (4.3) G ν = {x : lim φ(t i x) = φdν holds for any φ C(; R)}. n i=0 We note that if ν is ergodic, then (supp(ν),t) is transitive and ν(g ν ) = by Birkhoff Pointwise Ergodic Theorem. Proposition 4.5. Let (,T) be a zero-dimensional invertible TDS. If µ M e (,T) with h µ (T) > 0 and E B with µ(e) =, then the following holds. () For µ-a.e x, γ<α h B top (T W s (x,t) W u (x,t) E) h µ (T) and h B top(t W s (x,t) W u (x,t) E) h µ (T). (2) Under the continuum hypothesis, for µ-a.e x there exists a set S x W s (x,t) W u (x,t) E such that (a) S x is a strongly scrambled set for T, T, (b) h B top (T S x) h µ (T) and h B top (T S x ) h µ (T).

12 2 C. FANG, W. HUANG, Y. YI, AND P. ZHANG Proof. Let B µ be the completion of B with respect to µ, P µ (T) be the Pinsker σ-algebra of (, B µ,µ,t), and γ be the measurable partition of with γ = P µ (T) (mod µ). Then µ can be disintegrated over P µ (T) as µ = µ xdµ(x) where µ x M() and µ x (γ(x)) = for µ-a.e. x. Define for µ-a.e. x the set Γ x = {y : µ x = µ y }. Then for µ-a.e. x, µ x (Γ x ) =. Claim : supp(µ x ) W s (x,t) W u (x,t) for µ-a.e. x. This is already proved in [5] (Step of the proof of Theorem 4.6). For completion, we include the proof below. Since P µ (T) is also the Pinsker σ-algebra of (, B µ,µ,t ) and W s (x,t ) = W u (x,t), by symmetry it remains to prove that for µ-a.e. x, supp(µ x ) W s (x,t). By Lemma 4., there exist {W i } P and 0 = k < k 2 < satisfying () lim i + diam(w i ) = 0, (2) lim k + H µ (P k P ) = h µ (T), where P k = k T k i W i and P = k= P k, (3) n=0 T n P = P µ (T). It is clear that P (x) W s (x,t) for x. Let µ = µ n,xdµ(x) be the disintegration of µ over T n P for n N. Then for n N, µ n,x ((T n P )(x)) = for µ-a.e. x. Moreover, since (T n P )(x) W s (x,t) for each x, supp(µ n,x ) W s (x,t) for µ-a.e. x. Let {f i } be a dense subset of C(; R) with respect to the supremum norm. For each i N, by Martingale Theorem for µ-a.e. x lim f i (y)dµ n,x (y) = lim E(f i T n P )(x) = E(f i P µ (T))(x) = f i (y)dµ x (y). Hence there exists a measurable subset 0 with µ( 0 ) = such that for each x 0 and i N, lim f i (y)dµ n,x (y) = f i (y)dµ x (y). By a simple approximation argument, we have for each f C(; R), f(y)dµ n,x (y) = f(y)dµ x (y) for each x 0, lim i.e., lim µ n,x = µ x for x 0 under the weak -topology. For µ-a.e. x, since supp(µ n,x ) W s (x,t) for all n N, we have supp(µ x ) W s (x,t). This proves the Claim. Since is zero dimensional, there exists a sequence of finite clopen partitions {α j } j= (i.e. each element in α j is closed and open) of such that lim j + diam(α j ) = 0. By Theorem 2., for each j N we have lim n I µ( n i=0 T i α j P µ (T))(x) = h µ (T,α j P µ (T)) = h µ (T,α j ) for µ-a.e. x. In the above, we have used the fact that h µ (T,α P µ (T)) = h µ (T,α) for any α P.

13 Now since for µ-a.e. x n I µ ( T i α j P µ (T))(x) = i=0 we have lim DIMENSION OF STABLE SETS AND SCRAMBLED SETS 3 = A n i=0 T i α j A (x)log E( A P µ (T))(x) A n i=0 T i α j A (x)log µ x (A) = log µ x (( n i=0 log µ x (( n i=0 T i α j )(x)) = h µ (T,α j ) for µ-a.e. x. n T i α j )(x)), Thus we easily find a Borel subset of satisfying µ( ) = and for all j N, x (4.4) lim log µ x (( n i=0 T i α j )(x)) = h µ (T,α j ). n Since µ( ) =, µ x ( ) = for µ-a.e. x. Moreover µ x (Γ x ) = for µ-a.e. x. Thus there exists a Borel set 2 of with µ( 2 ) = satisfying µ x (Γ x ) =. Claim 2. If x 2 and B x with µ x (B x) > 0, then h B top(t B x ) h µ (T) and h B top(t B x ) h µ (T). Let x 2 and B x with µ x(b x ) > 0. Put D x = B x Γ x. Since x 2, µ x (Γ x ) = and so µ x(b x ) = µ x(d x ) > 0. By symmetry of T, T, h µ (T) = h µ (T ) and D x B x. It remains to prove that h B top(t D x ) h µ (T). Since h B top(t D x ) = lim j + h B top(t,α j D x ) and h µ (T) = lim j + h µ (T,α j ), it is sufficient to show that for all j N. h B top(t,α j D x ) h µ (T,α j ) Fix j N. Without loss of generality, we suppose h µ (T,α j ) > 0. For any ǫ (0,h µ (T,α j )) and k N, it follows from the fact µ x = µ y for all y D x Γ x that D x (k,ǫ) =: {y D x : µ y (( n i=0 T i α j )(y)) e n(hµ(t,α j) ǫ) for all n k} = {y D x : µ x (( n i=0 T i α j )(y)) e n(hµ(t,α j) ǫ) for all n k}. Since D x, we have by (4.4) that k= D x(k,ǫ) = D x for any ǫ > 0. The fact µ x(d x ) = implies that there is an N N such that µ x(d x (N,ǫ)) > 0. Let n N with n N and {U i : i =,2, } be a countable cover of D x (N,ǫ) with n T,αj (U i ) n for any i N. For each U i there exists a B i n T,α j (U i ) l=0 T l α j such that U i B i. Hence if U i D x (N,ǫ), then B i D x (N,ǫ) U i D x (N,ǫ). Taking x i B i D x (N,ǫ), then Using the fact µ x (B i ) = µ x (( n T,αj (U i ) l=0 i N:U i D x(n,ǫ) = T l α j )(x i )) e n T,α j (U i )(h µ(t,α j ) ǫ). B i i N:U i D x(n,ǫ) = U i D x (N,ǫ),

14 4 C. FANG, W. HUANG, Y. YI, AND P. ZHANG we have e n T,α j (U i )(h µ(t,α j ) ǫ) Since {U i : i =,2, } is arbitrary, i N:U i D x(n,ǫ) = i N:U i D x(n,ǫ) = e n T,α j (U i )(h µ(t,α j ) ǫ) µ x (B i ) µ x (D x(n,ǫ)). m n (T,h µ (T,α j ) ǫ,α j D x (N,ǫ)) µ x(d x (N,ǫ) > 0 for all n N, which, when passing to the limit n +, yields m(t,h µ (T,α j ) ǫ,α j D x (N,ǫ)) µ x(d x (N,ǫ)) > 0. This implies that h B top (T,α j D x (N,ǫ)) h µ (T,α j ) ǫ and therefore h B top(t,α j D x ) h B top(t,α j D x (N,ǫ)) h µ (T,α j ) ǫ. Now, by taking ǫ ց 0, we have h B top(t,α j D x ) h µ (T,α j ), and the Claim is proved. To prove (i), we note that by Claim and the fact that µ(e) = there exists a Borel subset 3 of with µ( 3 ) = such that µ x (W s (x,t) W u (x,t) E) = µ x (E) = for all x 3. For each x 2 3, since µ x (W s (x,t) W u (x,t) E) =, we have by Claim 2 that h B top (T W s (x,t) W u (x,t) E) h µ (T) and h B top (T W s (x,t) W u (x,t) E) h µ (T). This proves (i) since µ( 2 3 ) =. To prove (ii), we note that µ( 2 ) =. By Claim 2 it is sufficient to show under the continuum hypothesis that for µ-a.e. x there exists a strong scrambled set S x W s (x,t) W u (x,t) E for T, T with µ x(s x ) =. Define a measure λ(µ) on 2 by λ(µ) = µ x µ x dµ(x). It is well-known (see e.g. [2, 5]) that µ x is non-atomic for µ-a.e. x and λ(µ) is a T T-invariant ergodic measure on. Let W = supp(λ(µ)). Since λ(µ) is an ergodic measure for T T, both (W,T T) and (W,(T T) ) are transitive. Since µ(e) =, µ x (E) = for µ-a.e. x. Hence λ(µ)(e E) = µ x µ x (E E)dµ(x) =. Let G + be the set of generic points of λ(µ) for T T and G be the set of generic points of λ(µ) for (T T). Then λ(µ)(g + G (E E)) = and Since G + G (E E) W trans (T T) W trans ((T T) ). = λ(µ)(g + G (E E)) = µ x µ x (G + G (E E))dµ(x) and µ x is non-atomic for µ-a.e. x, there exists a subset 4 with µ( 4 ) = such that µ x µ x (G + G (E E)) =, supp(µ x ) W s (x,t) W u (x,t), µ x (Γ x ) =, and µ x is non-atomic for x 4.

15 DIMENSION OF STABLE SETS AND SCRAMBLED SETS 5 For each x 4, let C x = W s (x,t) W u (x,t) E. Since µ x µ x (G + G (E E)) = and supp(µ x ) W s (x,t) W u (x,t), we have µ x (C x ) = and µ x µ x (G + G (C x C x )) =. By Lemma 4.3 there exists S x such that µ x(s x ) = and S x S x \ G + G (C x C x ). This implies that S x C x. Since µ x is non-atomic, S x must be uncountable. Next we show that for each x 4, S x is a strong scrambled set for T, T. Let (x,x 2 ) W trans (T T) W trans ((T T) ). On one hand, since {(z,z) : z supp(µ)} W, we have lim inf d(t n x,t n x 2 ) = 0 and lim inf d(t n x,t n x 2 ) = 0. On the other hand, since µ x is non-atomic for µ-a.e x, we have W. This implies that x x 2. Hence {x,x 2 } is a strong Li-Yorke pair for T, T. Since S x S x \ W trans (T T) W trans ((T T) ), S x is a strong scrambled set for T, T. Definition 4.6. As in [8], an extension π : (Z,R) (,T) between two TDSs is said to be a principal extension if h ν (R) = h πν (T) for every ν M(Z,R). Lemma 4.7. ([8]) Every invertible TDS (,T) with h top (T) < has a zero dimensional principal extension (Z,R) with R being invertible. Proof. See Proposition 7.8 in [8]. Remark 4.8. For an invertible TDS (, T) Lindenstrauss and Weiss [22] introduced the mean dimension mdim(,t) (an idea suggested by Gromov). It is well-known that for an invertible TDS (,T), if h top (T) < or the topological dimension of is finite, then mdim(,t) = 0 (see Definition 2.6 and Theorem 4.2 in [22]). In general, one can show that for an invertible TDS (,T), if mdim(,t) = 0 then (, T) has a zero dimensional principal extension (Z, R) with R being invertible. Indeed, let (Y,S) be an irrational rotation on the circle. Then ( Y,T S) admits a nonperiodic minimal factor (Y,S) and mdim( Y,T S) = 0. Hence ( Y,T S) has the socalled small boundary property [2, Theorem 6.2], which implies the existence of a basis of the topology consisting of sets whose boundaries have measure zero for every invariant measure. With these results it is easy to construct a refining sequence of small-boundary partitions for ( Y,T S), where the partitions have small boundaries if their boundaries have measure zero for all µ M( Y,T S). Then by a standard construction (see [8]), associated to this sequence there exists a zero dimensional principal extension (Z,R) of ( Y,T S) with R being invertible. Finally, noting that ( Y,T S) is a principal extension of (,T), we know that (Z, R) is also a zero dimensional principal extension of (, T) since the composition of two principal extensions is still a principal extension. The following theorem implies Theorem 2 ). Theorem 4.9. Let (,T) be an invertible TDS with h top (T) <. If µ M e (,T) with h µ (T) > 0 and E B with µ(e) =, then the following holds. () For µ-a.e x, h B top(t W s (x,t) W u (x,t) E) h µ (T) and h B top(t W s (x,t) W u (x,t) E) h µ (T). (2) Under the continuum hypothesis, for µ-a.e x there exists a set S x W s (x,t) W u (x,t) E such that

16 6 C. FANG, W. HUANG, Y. YI, AND P. ZHANG (a) S x is a strong scrambled set for T, T, (b) h B top (T S x) h µ (T) and h B top (T S x ) h µ (T). Proof. We only show (2) as the proof of () is similar to that of (2). By Lemma 4.7 there exists a principal extension π : (Z,R) (,T) where Z is zerodimensional and R is invertible. Take a ν M e (Z,R) such that πν = µ. Since π (E) B Z with ν(π (E)) =, Proposition 4.5 implies that there exists a Borel set Z 0 Z with ν(z 0 ) = such that for each z Z 0 there exists a strong scrambled set S z W s (z,r) W u (z,r) π (E) for both R and R, and h B top (R S z) h ν (R) and h B top (R S z ) h ν (R). Let 0 = π(z 0 ). Then µ( 0 ) =. For x 0, we take z Z 0 with π(z) = x and define S x = π(s z ). It is clear that S x W s (x,t) W u (x,t) E and S x is a strong scrambled set for T, T. Since h top (T) <, by the variational principle of condition entropy (see [9,, 6]), we have sup h(r,π (x)) = sup (h θ (R) h πθ (T)) = 0, and x θ M(Z,R) sup h(r,π (x)) = sup (h θ (R ) h πθ (T )) = 0. x θ M(Z,R ) Now, we have by Theorem 3.3 that h B top (T S x) = h B top (R S z) h ν (R) = h µ (T). Similarly, h B top (T S x ) h µ (T). This proves (2). The following theorem implies Theorem ). Theorem 4.0. Let (,T) be a TDS with h top (T) <. If µ M e (,T) with h µ (T) > 0 and E B with µ(e) =, then the following holds. () h B top(t W s (x,t) E) h µ (T) for µ-a.e x. (2) Under the continuum hypothesis, for µ-a.e x there exists a scrambled set S x W s (x,t) E for T satisfying h B top(t S x ) h µ (T). Proof. We only prove (2) since the proof of () is similar to that of (2). Without loss of generality, we assume that T is surjective, for otherwise we can replace with supp(µ). Let π : (, T) (,T) be the natural extension. Then T is a homeomorphism and h top ( T) = h top (T) <. Take ν M e (, T) with π ν = µ. By Theorem 4.9, under the continuum hypothesis there exists a Borel subset 0 of with µ( 0 ) = such that for x 0 there exists a strong scrambled set S x W s ( x, T) for T satisfying h B top ( T S x ) h ν ( T). Let 0 = π ( 0 ). Obviously µ( 0 ) =. For any x 0 take x 0 with x = π ( x) and let S x = π (S x ). Then S x W s (x,t) and by Lemma 3.4 h B top(t S x ) = h B top( T S x ) h ν ( T) = h µ (T). Now, since {ỹ, z} is a Li-Yorke pair for T iff {π (ỹ),π ( z)} is a Li-Yorke pair for T, S x is a scrambled set for T. This proves (2). Let (,T) be a TDS with a compatible metric d. Given ǫ > 0, the ǫ-stable set of x under T is the set of points whose forward orbit ǫ-shadows that of x: W s ǫ (x,t) = {y : d(t n x,t n y) ǫ for all n = 0,, }.

17 DIMENSION OF STABLE SETS AND SCRAMBLED SETS 7 Definition 4.. As in Bowen [6], a TDS (,T) is called h-expansive if there exists an ǫ > 0 such that sup h(t,wǫ s (x,t)) = 0, x while as in Misiurewicz [23], (,T) is called asymptotically h-expansive if lim sup h(t,wǫ s (x,t)) = 0. ǫ 0 x It was shown by Bowen [6] that all expansive systems, expansive homeomorphisms, endomorphisms of a compact Lie group, and Axiom A diffeomorphisms are h-expansive, by Misiurewicz [23] that every continuous endomorphism of a compact metric group is asymptotically h-expansive if its entropy is finite, and by Buzzi [0] that each C diffeomorphism on a compact manifold is asymptotically h-expansive. In [23], Misiurewicz showed that for an asymptotically h-expansive system (,T), the entropy map ν M(,T) h ν (T) R + is upper semi-continuous. Hence for an asymptotically h-expansive system (, T), there always exists a µ M e (,T) such that h µ (T) = h top (T) < (an asymptotically h-expansive system always admits finite topological entropy). Corollary 4.2. Let (, T) be an asymptotically h-expansive, invertible TDS. Then there exists x for which the following holds. () h B top(t W s (x,t) W u (x,t)) = h top (T) and h B top(t W s (x,t) W u (x,t)) = h top (T); (2) Under the continuum hypothesis, there exists a scrambled set S x W s (x,t) W u (x,t) satisfying h B top (T S x) = h top (T) and h B top (T S x ) = h top (T). Proof. We only show (2) since the proof of () is similar to that of (2). Since (,T) is an asymptotically h-expansive system, we always have a µ M e (,T) such that h µ (T) = h top (T) <. Under the continuum hypothesis, we have by Theorem 2 ) that for µ-a.e. x there exists a scrambled set S x W s (x,t) W u (x,t) satisfying h B top(t S x ) h µ (T) = h top (T) and h B top(t S x ) h µ (T) = h top (T). Since h B top(t E) h top (T) and h B top(t E) h top (T) for any E, the proof is complete. Corollary 4.3. Let (, T) be an asymptotically h-expansive TDS. Then there exists x for which the following holds. () h B top (T W s (x,t)) = h top (T); (2) Under the continuum hypothesis, there exists a scrambled set S x W s (x,t) satisfying h B top(t S x ) = h top (T). Proof. Using Theorem 4.0, the proof is completely similar to that of Corollary Hausdorff dimension of stable sets and scrambled sets Let (,d) be a metric space. We first recall the definition of Hausdorff dimension of a set. Fix t 0. For each δ > 0 and subset A, define + d (A) = inf{ diam(u i ) t }, H t,δ where the infimum is taken over all countable covers {U i : i =,2, } of A of diameter not exceeding δ. This definition induces an outer measure on, i.e., a function defined on all

18 8 C. FANG, W. HUANG, Y. YI, AND P. ZHANG subsets of taking values in [0,+ ] satisfying H t,δ d and H t,δ d ( A n ) n= n= ( ) = 0, Ht,δ d H t,δ d (A n) for any countable family {A n : n =,2, } of subsets of. Since H t,δ d (A) increases as δ decreases for any A, we can define Hd t (A) = lim δ 0 Ht,δ d (A) = suph t,δ d (A). δ>0 (A) Ht,δ d (B) if A B The case Hd t (A) = + is not excluded. Since all Ht,δ d ( ) are outer measures, Ht d ( ) is also an outer measure. It is well-known that H t d (A B) = Ht d (A) + Ht d (B) for each pair of positively separated sets A,B, i.e., d(a,b) = inf{d(x,y) : x A,y B} > 0. The metric outer measure Hd t is called the Hausdorff outer measure associated to t. Its restriction to the σ-algebra of Hd t -measurable sets, which includes all the Borel sets, is called the Hausdorff measure associated to t. Fix A. Since for every 0 < δ the function t H t,δ d (A) is non-increasing, so is the function t Hd t (A). Moreover, if 0 < s < t, then for every δ > 0 H s,δ d (A) δs t H t,δ d (A) which implies that if Hd t(a) > 0, then Hs d (A) = +. Thus there is a unique value H d(a) [0,+ ], which is called the Hausdorff dimension of A with respect to the metric d on, such that { Hd t +, if 0 t < (A) = Hd (A), 0, if H d (A) < t <. The Hausdorff dimension is a monotone function of sets, i.e., if A B then H d (A) H d (B). Moreover if {A n } n is a countable family of subsets of then H d ( A n ) = suph d (A n ). n n= Hence if E is countable then H d (E) = 0. In the following we investigate the interrelation of Hausdorff dimension and Bowen entropy of a set in some specific TDSs. Let (,T) be a TDS with metric d. We assume that T is Lipschitz continuous with Lipschitz constant L, i.e., d(tx,ty) Ld(x,y) for any x,y. It is easy to see that if h top (T) > 0, then L >. Indeed, by [2] we know that there exists an uncountable scrambled set S for T. Take x y S. Then {x,y} is a Li-York pair and hence lim sup d(t n x,t n y) > 0 and lim inf d(t n x,t n y) = 0. Thus there exist i,j N with i < j such that d(t i x,t i y) < d(t j x,t j y). Moreover, we have d(t i x,t i y) < L j i d(t i x,t i y) since d(t j x,t j y) L j i d(t i x,t i y). This implies that L >. The following result is just Theorem in [24].

19 DIMENSION OF STABLE SETS AND SCRAMBLED SETS 9 Lemma 5.. Let (,T) be a Lipschitz continuous TDS with Lipschitz constant L > associated to metric d. Then H d (Y ) hb top(t Y ) for any subset Y. Remark 5.2. Let (,T) be as in Lemma 5.. Then h top (T) H d (). Hence when H d () <, we always have h top (T) <. When H d () =, the following example shows that h top (T) = can happen. Example 5.3. Let = [0,] N and be endowed with the product topology. Then the compact space is metrizable, and a compatible metric on can be chosen as d(x i,y i ) d(x,y) = 2 i, for any x = (x 0,x, ),y = (y 0,y, ). With the shift map T : : i=0 T(x,x 2, ) = (x 2,x 3, ), (x,x 2, ), it is clear that (,T) is a Lipschitz continuous TDS with Lipschitz constant L = 2, and moreover H d () =. It is not hard to see that h top (T) = as well. Lemma 5.4. Let (,T) be a TDS with metric d. If there exist ǫ > 0 and L > such that d(tx,ty) Ld(x,y) whenever d(x,y) < ǫ, then for any subset Y. H d (Y ) hb top (T Y ) Proof. Let Y be given and U be a finite open cover of with diam(u) < ǫ 2 for any U U. It is sufficient to show that H d (Y ) hb top (T,U Y ). Fix k N. For any A with n T,U (A) k, it is obvious that diam(t i (A)) < ǫ for i = 0,,,n T,U (A). Since d(tx,ty) Ld(x,y) when d(x,y) < ǫ, we have (5.) diam(a) L n T,U(A)+ diam(t n T,U(A) A) L n T,U(A)+ ǫ L k+ ǫ. Moreover, (5.2) e sn T,U(A) C s,ǫ,l (diam(a)) s for any s 0, where C s,ǫ,l = (Lǫ) s. Let A = {A i } be any cover of Y satisfying n T,U(A i ) k. Then A is a L k+ ǫ-cover of Y by (5.). By (5.2), we have e sn T,U(A i ) C s,ǫ,l (diam(a i )) s s C s,ǫ,l H,L k+ ǫ d (Y ), s 0. Since A is arbitrary, we have s,l k+ ǫ m k (T,s, U Y ) C s,ǫ,l Hd (Y ), s 0. Taking limit k + yields that m(t,s, U Y ) C s,ǫ,l Hd (Y ), s 0. s

20 20 C. FANG, W. HUANG, Y. YI, AND P. ZHANG This implies that H d (Y ) hb top (T,U Y ). We are now ready to prove Theorem 2) and Theorem 2 2). Proof of Theorem 2 2). Let µ be a T-invariant ergodic measure with h µ (T) > 0. Under the continuum hypothesis, we have by Theorem 4.9 that for µ-a.e x there exists a scrambled set S x W s (x,t) W u (x,t) for both T and T such that h B top (T S x) h µ (T). Moreover, since d(tx,ty) Ld(x,y) for any x,y, we have by Lemma 5. that H d (S x ) h B top (T Sx) hµ(t) for µ-a.e. x. The proof of the remaining part of Theorem 2 2) uses a similar argument. Proof of Theorem 2). By using Theorem 4.0 and Lemma 5., the proof is completely similar to that of Theorem 2 2). Using Remark 5.2 and Theorem 2), we have the following result. Theorem 5.5. Let (,T) be a TDS with metric d such that H d () < and T be a Lipschitz continuous self-map with Lipschitz constant L >. If µ is a T-invariant ergodic measure with h µ (T) > 0, then the following holds. () H d (W s (x,t)) hµ(t) for µ-a.e x. (2) Under the continuum hypothesis, for µ-a.e x there exists a scrambled set S x W s (x,t) for T such that H d (S x ) hµ(t). As a direct consequence of the above theorem, we have the following results. Corollary 5.6. Let (,T) be a TDS with metric d such that h top (T) < and T be a Lipschitz continuous self-map with Lipschitz constant L >. If there exists µ M e (,T) such that h µ (T) = h top (T), then the following holds. () H d (W s (x,t)) htop(t) for µ-a.e. x. (2) Under the continuum hypothesis, for µ-a.e x there exists a scrambled set S x W s (x,t) for T satisfying H d (S x ) htop(t). Corollary 5.7. Let (, T) be an asymptotically h-expansive TDS with metric d and T be a Lipschitz continuous self-map with Lipschitz constant L >. Then there exists x for which the following holds. () H d (W s (x,t)) htop(t). (2) Under the continuum hypothesis, there exists a scrambled set S x W s (x,t) for T satisfying H d (S x ) htop(t). Proof. Since (,T) is an asymptotically h-expansive system, there exists a µ M e (,T) such that h µ (T) = h top (T) <. Hence the corollary follows from Corollary 5.6. Let A = {0,,,N } for some integer N 2 endowed with the discrete metric d, and Σ + (N) be the space of one-sided sequences in A endowed with the product topology. Then Σ + (N) is metrizable, and a compatible metric ρ on Σ(N) can be chosen as ρ(x,y) = i=0 d(x i,y i ) N i, x = (x 0,x, ), y = (y 0,y, ) Σ + (N).

21 DIMENSION OF STABLE SETS AND SCRAMBLED SETS 2 We consider the shift map σ : Σ + (N) Σ + (N): σ(x) i = x i+, i = 0,,. If is a σ-invariant non-empty closed subset of Σ + (N), then we say that (,σ) is a subshift of (Σ + (N),σ). Theorem 5.8. Let (,σ) be a subshift of (Σ + (N),σ) with a metric ρ as above. Then there exists µ M e (,σ) with h µ (σ) = h top (σ,) for which the following holds. () H ρ (W s (x,σ)) = htop(σ,) log N for µ-a.e. x. (2) Under the continuum hypothesis, for µ-a.e. x there exists a scrambled set S x W s (x,σ) for σ satisfying H ρ (S x ) = htop(σ,) log N. Proof. Since the entropy map θ M(,T) h θ (σ,) is upper semi-continuous, there exists µ M e (,T) such that h µ (σ) = h top (σ,). To finish the proof, we only need to show (2) since the proof of () is similar to that of (2). Since ρ(σx,σy) Nρ(x,y) for any x,y, it follows from Theorem 5.5 that for µ-a.e x there exists a scrambled set S x W s (x,σ) for σ satisfying H ρ (S x ) hµ(σ) log N = htop(σ) log N. Since ρ(σx,σy) Nρ(x,y) for any x,y with ρ(x,y) < N, Lemma 5.4 implies that H ρ (S x ) htop(σ Sx) log N htop(σ) log N for µ-a.e x. Hence H ρ(s x ) = htop(σ,) log N for µ-a.e x. Remark 5.9. ) In [2] Furstenberg proved that for a subshift (,σ) of (Σ + (N),σ), H ρ () = h top(σ,) log N. Hence for any E, we always have H ρ(e) htop(σ,) log N (see also Lemma 5.4). 2) Let (,σ) be a subshift of (Σ + (N),σ). For any x, it is clear that W s (x,σ) is a countable set. Hence h B top (σ W s (x,σ)) = 0 and H ρ (W s (x,σ)) = 0 (in fact, the former is true for any TDS (,T)). Thus taking closures of the stable sets in the statement of Theorems -2 is necessary. We end this section by posting the following questions: Question 5.0. Let (, T) be an invertible TDS and µ be a T-invariant ergodic measure on with h µ (T) > 0. Then whether the following statements hold for µ-a.e x : () h B top(t W u (x,t)) h µ (T) and h B top(t W s (x,t)) h µ (T)? (2) (W s (x,t) W u (x,t)) \ {x}? 6. C self-maps on Riemannian manifold Let M be a smooth Riemannian manifold. The Riemannian structure on M induces a natural norm x on each tangent space T x M which we simply denote by if there is no confusion. M turns out to be a metric space with the metric ρ: ρ(x,y) = inf{ b a γ(t) dt : γ : [a,b] M is a C map with γ(a) = x,γ(b) = y}, for any x,y M. For x M and r > 0, let B(x,r) = {y M : ρ(x,y) < r} denote the ball centered at x of radius r. Any C self-map g on M gives rise to the tangent map Dg on the tangent bundle TM, which is a linear operator Dg x : T x M T gx M, x M, with norm defined by Dg x = max Dg x(u). u T xm: u =