Hyperbolic Nonwandering Sets on Two-Dimensional Manifolds 1

Hyperbolic Nonwandering Sets on Two-Dimensional Manifolds 1 S. NEWHOUSE and J. PALIS Institute» de Matemâtica Pura e Aplicada Rio de Janeiro, Brasil We are concerned here with diffeomorphisms of a compact manifold M whose nonwandering sets are hyperbolic. Let/be such a diffeomorphism and Ω its nonwandering set. Consider the following questions: (a) Are the periodic orbits of / dense in Ω? (b) Can / be approximated by an ß-stable diffeomorphism? We show in this chapter that both questions have a positive answer when M is a closed two-dimensional manifold. The first question was suggested by Smale in [7]. Related to it is Anosov's closing lemma : The periodic orbits are dense in the nonwandering set of//?,/ being as above. Also, one should notice that if/ is an Anosov diffeomorphism, then the periodic points are dense in the nonwandering set. This follows, for instance, from the structural stability of / and Pugh's closing lemma. With respect to the second question, if / has the no-cycle property, then/is ß-stable [8], [3]. On the other hand, if there is a cycle, one can radically change the nonwandering set by small perturbations, so / is not ß-stable [5]. What we do here is to break all existing cycles without changing Ω to achieve an ß-stable diffeomorphism. + AMS (MOS) 1970 SUBJECT CLASSIFICATION: 34C35, 34C40. 293

294 S. NEWHOUSE AND J. PALIS We now set some notation, basic definitions, and the precise statements of our results. Let DifP(M) be the set of C r diffeomorphisms of M with the C r topology, r > 1. For / e DifP(Af), we denote by P = P(f) the set of periodic points and by Ω = Ω(/) the nonwandering set of/. We say that Ω = Ω(/) is hyperbolic if there exist a continuous splitting Τ Ω Μ = E* 0 E u > a riemannian norm on TM, and a constant 0 < λ < 1 such that Tf(v) < λ \v for v e E* and Tf~ l {v) < λ \v for v e E\ The diffeomorphism^ e Diff r (M) is called ß-stable if there exists a neighborhood N of g in DifP(M) such that for any g e N there is a homeomorphism h : Ω^) > Ω( ) satisfying the condition hg(x) = gh(x) for all X G Q(g). Assume M is a compact manifold without boundary and dim M = 2. THEOREM A For/ e DifP(M), if Ω(/ ) is hyperbolic, then P(f ) is dense in Ω(/). That is, / satisfies Smale's Axiom A. THEOREM B For f e DifP(M), if Ω(/) is hyperbolic, then / can be approximated in Diff r (M) by an ß-stable diffeomorphism g with û(^) = fl(/) The starting point for proving these results is the analogue of the spectral decomposition theorem [7] for limit sets as considered in [3]. Let / e DifP(M). For x e M y define ω(χ) = {y e M : there exists a sequence of integers n { oo such that lim/ w *(x) =y}. Dually, one defines a(x). Let ω (/) == U ω (^) xem anc^ a (f) = U a ( x )- The limit set L = L(f ) of / is the closure of ω u a. If L is hyperbolic, then P = L y and L can be written as the disjoint union L = L x u ul n, each Ζ^ being closed, invariant by /, and topologically transitive. The periodic points are dense in each L {, and L { has a local product structure, i.e., there is an ε > 0 such that W e *(Li) n W»(Li) c ^([3], [7]). The L { are called basic limit sets. Notice that M = \jw»(l i ) = \jw»(l i ), xem

HYPERBOLIC NONWANDERING SETS 295 and using [2] W\L % )= (J W*(x), W«{Li)= (J W a (x). xcli xel t W s and W u stand, respectively, for the stable and unstable manifolds of sets involved. By Wf oc (x) and W c (#), x e L, we mean W e *(x) and W u e (x), for some small e > 0 as defined in Section 2 of [2] (see also [1]). If dim W*(x) = 1, define W + *(x) and WJ(x) to be the components of From now on, we assume dim M = 2 and/ e Diff r (M) with Ω = hyperbolic. Ω(/) PROPOSITION 1 There exists a finite subset F* a P such that if x e L, dim W*{x) = 1 and x is not a limit point of both W + *(x) n and Wj(x) n, then A? e W u (p),p e F*. Proof Let ε > 0 be such that no arc V cz W^f oc (jc) with both boundary points in L and length less than ε contains x in its interior. We first show that x e W\p\ for some p e P. Suppose not. Since x e L y x e Li for some basic limit set L {. Let y e a(x) and ^ a sequence of integers such that n t oo and/~ n< (#). < y. Clearly y e Li and using the local product structure of Li we get an integer n > 0 and an arc (0, w) <=/~ w Wf 0C (tf) of length smaller than ε containing f~ n (x) in its interior, z and w in Li. Thus we reach a contradiction, since / n (#), / n (w) G L, and the arc (/ n (#)>/ n («0) <= W? oc (*) ^as length less than ε and contains x in its interior. A similar argument shows that the subset of P for which there are points in their unstable manifolds satisfying the hypotheses of the proposition is finite. The same fact is true interchanging stable and unstable manifold in the above proof. This yields a corresponding finite subset F u cz P. We denote by F the union of F* and F u. We now define basic regions and a partial ordering relation on them. This will play a key role in proving the main results in the paper. Let p e F n L { with period n, L h being a basic limit set. Clearly, p is a saddle point. Let U, f(u)>...,/ w-1 (C7) be a pairwise disjoint small open disks containing the orbit O(p) such that in U' = U*=o _1 f k (U). w foc(0(p)) and W$JO{p)) are well defined and U' nlj=0 for/ φ t, In U' we distinguish the open regions R l9..., R m bounded by Wf oc (0(p)) and W?oc(0(/>)) such that U' nf l (R k ) cz R k for 0 < / < n. The R k

296 S. NEWHOUSE AND J. PALIS are called basic regions. For instance, if (Tf n ) p has positive eigenvalues then there are four basic regions R ly R 2f R 3, i? 4 associated to 0(p)> each one has n connected components corresponding to the open quadrants defined in U' by Wf oc (0(p)) and Wf OQ (0(p)). If one eigenvalue of (Tf% is negative, then there are two basic regions, each having In connected components. Notice that Wf oc (0(p)) nr k = 0 and W^(0{p)) nr k = 0 for each k. For a basic limit set L jy which is not a periodic orbit in F, the basic region is defined simply as a neighborhood R of L jy where Wf oc (Lj) and W& c (Lj) are well defined and R n L r 0 for r φ]. Let R ly R 29..., R n be the basic regions considered above. In Ri we define or Rf = WUO(p)) Ri a = Wf oe (0{p)) nri-l, nr { -L, R* = WULi) - {W*(F n L t ) u Lt), *«= WULÙ - {W\F n L t ) u Li), according to whether i?^ is associated to O(p) c: F or to a basic limit set Z^ not contained in F. Notice that the correspondences R t Rf and R t 7?^ may be twoto-one for periodic orbits in F. We also globalize these "local components' * of stable and unstable manifolds by setting i 8 =U/-W)> o oo < u = U/W). 0 Given two basic regions R { and i?^, we say that R t almost visits Rj if there exist sequences x n #, x n e R { and x e Rf, and /w n oo, such that f mn (x n ) >y, f mn e (x n ) Rj> an( l y e ^/ Ι η ^is case, we also say that x almost visits y in Rj from R {. LEMMA 2 Let Ri and ^ be basic regions such that R { almost visits Rj. Then, given any open set V meeting Rf, we have Rj u c \Jn>of n (V)- Proof We construct a sequence of basic regions R io = Ri, R ix,..., R it = Rj, and a sequence of points x 0, x x,..., ^ such that

HYPERBOLIC NONWANDERING SETS 297 (a) R ijc almost visits R ijc+1 for 0 < k < /; (b) Either x k+1 e Rf k n h ik+l or x k+1 e W*(q) n Ë» ik+1,.where q e F Π L k and R i]c is associated to some O(p) <z F Γ) L k or to L k, 0 < k < I; (c) For any open set V k meeting i?? fc, Rf p c (Jn>o/ n (^*) for 0 < Λ Such a sequence, whose existence will be proved by induction, implies the lemma. Since Ri almost visits Rj, we have sequences x n x 0, # n G i?; and x 0 e /?,, and / w»(*n) y> with w w oo, /»«. (# n ) G Jfy, and y G Ä/. Take R io = /?$. Let U be a small neighborhood of Li, /^ being associated to Li or to O(p) a F n L {. From [2], there exist x x e Wf 0C {Li) L { and a sequence ρ η < m n such that ρ Λ oo, /'(Λ^) G /, for 0 < j < q n, and/*»(* n ) - *!. Either ^ G «*(, ) W U CF Π L { ) or ^ G W u (?) for some q e F n Li- Now we can find an index i x such that χ λ e R\ x, Ri almost visits R ix and R ix almost visits Rj, and given any neighborhood Vmeeting #Λ Ä c \J n^0f n (V). In the case ^ e W u (Li) W U (F n *), this is easily checked since W u (Li) c Un*o/ n (H and i? u c W u (Z,i) for the possible indices v such that x x e R v *. IfΛΐχ G W U (F n Li), then W^x^) c= Un>o/ n (^) Of course, W u (Xi) and W 8^) might meet transversely at x x or not. Analyzing the types of possible intersections of W^{x^) and W*{x x ) at x 1 and using the λ-lemma [4], one can choose R ix with x ± e ft^ as claimed. Once x k9 R ijc have been chosen, a similar construction yields a point # Α+1 and a region ^+1 with x k+1 e R% k+1, so that conditions (a)-(c) above hold for the sequence R io,..., Ri k+1 Finally, we claim that we must reach Rj with this process. In fact, the set of basic regions is finite, so the only possible obstruction to that would be the existence of cycles in the sequence, i.e., sequences R im, R% t,., R in as above with R im = R in. But if this happens, condition (c) implies that all the points x k e È% k,m < k < n, are nonwandering. Since Ω is hyperbolic, W u (x k ) and W a (x k ) meet transversely at x k. This implies that each x k is in the closure of the set of transversal homoclinic points of the periodic points. It now follows from [6] that x k e P = L> contradicting the construction of x k. Thus the lemma is proved. For basic regions R { and Rj, define R { > Rj if either Ri = Rj or R { almost visits Rj, As an immediate consequence of Lemma 2 we have THEOREM 3 regions. The relation > defines a partial ordering on the set of basic

298 S. NEWHOUSE AND J. PALIS We can now prove Theorem A. Proof of Theorem A Since P = L, it is enough to prove Ω = L. Suppose there exists x e Ω L. Then x e Rf n Rj u for basic regions R {, Rj, and Ri almost visits Rj. We now proceed as in Lemma 2 constructing a sequence of basic regions R io = R iy R ii9..., R it = Rj, and a sequence of points x 0 = x, x 1,..., x x such that (a) R ik almost visits R{ k+1, for 0 < k < I; (b) Either x k+1 e Rf k n + R» ik+1 or x k+1 e W*{q) n?, +1, where?ef Π Ljj. and R ijc is associated to some O(p) cz F n L k or to L^, 0 < k < /; (c) For any open set V k meeting ^, Ä? p c Un^o/ n (^) for 0 < k <p<l Also, since x e Ω and i2 is hyperbolic, (d) W u (x) and W\x) meet transversely at x. Thus taking any neighborhood V of x l9 R^ c Un>o/ n (^)> an^ using (d), Un>o/ n (^) nf^i, which implies that χ λ e Ω. Again since Ω is hyperbolic, W^fa) and W^8^) meet transversely at x ±. Repeating the argument, it follows that each x k is nonwandering and consequently W u (x k ) and W\x k ) meet transversely at x k, 0 < k < I. Again, from [6], x e P = L 9 which contradicts the assumption that x e Ω L. Therefore, Ω L and so Theorem A is proved. As a consequence of Theorem A, we have that each basic limit set L { in the spectral decomposition theorem corresponds, in fact, to a basic set Ω { as defined by Smale in [7]. We keep the notation Ω = L = L x u u L n. As in [3], define the relation > on the basic sets by L { > Lj if there is a sequence L io = L i9 L ii9..., L h = Lj such that W u (L ik ) n W a (L ijc+l ) Φ 0 for 0 < k < L A corresponding equivalence relation is defined on the {Li} by L { ~ Lj if L { > Lj and Lj >L {. Let {γ χ,..., y m } be the distinct equivalence classes of {L { } under ~. The {γ^ are partially ordered by γ { > yj if there exist L t e γ { and Lj e yj such that L { > L^. In what follows, the union of the basic sets in a class y { will also be denoted by y {. We now relate these notions with basic regions and stable and unstable manifolds of basic regions. A component of the class y is a global unstable manifold R u k of the basic region R ky where R k is associated to a periodic orbit in F n y. u The component R k is free if R u k c= \J Yj<Y W*(yj).

HYPERBOLIC NONWANDERING SETS 299 a LEMMA 4 Suppose that Ë k <= W u (y) and R k " n Ψ*(γ) φ 0 for some equivalence class γ. Then there are basic regions associated to γ, R ko = R k > R kl > > R k and R kn+1, R kn+i,..., R K+m such that (a) R kn >R kn+r îor \<r<m\ (b) # B+r is free for 1 < r < m ; (c) Ë{CZ US* W*(0(/V)) U %< Y W*{ Yj ), and (3ξ. - (^ u L t )) n W*(y) c UîM? B+r u 0(A), where Z,^ is the basic set to which R kn is associated, and 0(p r ) periodic orbit of F n y to which Rjt^ is associated. (d) R kn+r is connected, for 1 < r < m. is the Proof The lemma is proved by induction on the number of basic regions involved. Suppose i?^o = R k > R kl > > R k has been defined. If there are no R kj+1,., Rh +8 such that conditions (a)-(d) are satisfied, we can proceed to construct R kj+1 such that Rk 0 = Rk > Rki > "' > Rk, > Rk j+l Since the number of basic regions is finite and by Theorem 3 they are partially ordered by >, eventually we get a sequence of basic regions as desired. COROLLARY 5 In the previous lemma the basic region R kn is associated to a periodic orbit in F n y. From Lemma 4 and Corollary 5, it follows that if each class y i has only free components, then / has the no-cycle property and thus is.q-stable ([3, 8]). Theorem B will be proved through a finite number of small perturbations that keep the nonwandering set the same and increase the number of free components. We now restrict our attention to the periodic orbits 0(p 1 ) 9..., 0(p r ) that occur in Lemma 4. Let y be a ~-class consisting of more than one basic set. Let R ix and R i2 be basic regions associated to a periodic orbit 0(p) in F n L i9 Li e y, so that Sf x = Rf 2 is free. Suppose R is and R^ are the other basic regions associated to 0(p)> R iz adjacent to R it and R^ adjacent to Z? ia,i.e., R% i = R% 3 and βξ = ϋξ. Notice that possibly i x = i 2 and

300 S. NEWHOUSE AND J. PALIS i 3 = i 4. For ε > 0 small and δ < ε/2, let G = G' n W e a (0(p)), where G'=HV(L«)-/»V(L 4 ). i.e., G' is a proper fundamental domain for Li [2]. LEMMA 6 There exists a neighborhood N oi G such that for any j> G M ß and χ', x" e N n {Ä^ u /^3} if re' almost visits y from if is, then y can only visit x" in Z?.^. The same is true interchanging R^ and R is by 7? iz and i? i4. Proof If the lemma were false, choosing appropriate convergent subsequences, we could find x', x" e i?f 3 n iv and jy e M Ω such that #' almost visits y from i? ia and j> almost visits x" in i? ij}. Since y $ Ω = L> we can choose R jx and i? j2 such that R^ nl$f 2 3y, R jt almost visits R ia and R i3 almost visits R J2. Therefore, Rj almost visits R J2 by Theorem 3. Moreover, constructing a sequence as in Lemma 2, we see that V n (J n >of n (V) Φ 0 f r an Y neighborhood V of y. Thus y e Ω, contradicting the assumption. Lemma 6 is proved. Notice that if we perform a perturbation of/ with support in N that creates a new nonwandering point y e M, then there must exist a pair of points x' y x" as above. Proof of Theorem B The theorem is proved by making a sequence of approximations, each increasing the number of free components, without changing Ω. We can take on {γι} a linear ordering compatible with >, so that Yn > 7n-i ^ > Yi Suppose all components of Yj are free for n Yj < Yi-i y so each class γ^ is made of a unique basic set. Suppose R k y^ \ t N r 4 \ f N r i i pr R n + r i i Pf N r N r Figure 1

HYPERBOLIC NONWANDERING SETS 301 u is associated to the class y { and R k n W*(yi) φ 0. From Lemma 4, there are regions R kn, R kn+i,..., R kn+m such that R k > R ki for n < i <m-\-n, and (a)-(d) in the lemma are satisfied. Using Lemma 6, we can perform an arbitrarily small C r perturbation of / such that Ω remains the same, the free components of γ$ remain free for j < /, and tty becomes free. This perturbation has support in the fundamental neighborhoods N r, 1 < r < m, defined in Lemma 6 and is taken so small as to be disjoint from all free components of y jy for j < i. For each r, 1 < r < m, the perturbation is the time-one map of a vector field parallel to R kn+r with the same * 'orientation/ ' as described in Figure 1. In this way, we achieve a diffeomorphism g, C r near /, such that O(g) = Ω(/) and all the components of the basic sets for g are free. Therefore g has the no-cycle property and by [8] is ß-stable. References [1] M. Hirsch and C. Pugh, Stable manifolds for hyperbolic sets, Global Anal. Proc. Symp. Pure Math. 14, pp. 133-163. Amer. Math. Soc, Providence, Rhode Island, 1970. [2] M. Hirsch, J. Palis, C. Pugh, and M. Shub, Neighborhoods of hyperbolic sets, Inventions Math. 9 (1970), 212-234. [3] S. Newhouse, Hyperbolic limit sets, Trans. Amer. Math. Soc. 167 (1972), pp. 125-150. [4] J. Palis, On Morse-Smale dynamical systems, Topology 8 (1969), 385-405. [5] J. Palis, A note on instability, Global analysis, Proc. Symp. Pure Math. 1970, 14, pp. 221-222. Amer. Math. Soc, Providence, Rhode Island, 1970. [6] S. Smale, Diffeomorphisms with many periodic points, "Differential and Combinatorial Topology," Princeton Univ. Press, Princeton, New Jersey, 1965, pp. 63-80. [7] S. Smale, Differentiable dynamical systems, Bull. Amer. Math. Soc. 73 (1967), 747-817. [8] S. Smale, The ß-stability theorem, Global analysis, Proc. Symp. Pure Math. 14, Amer. Math. Soc, Providence, Rhode Island, 1970.