BOUNDARY EFFECT OF RICCI CURVATURE

Transcription

1 BOUNDARY EFFECT OF RICCI CURVATURE PENGZI MIAO AND XIAODONG WANG Abstract. On a compact Riemannian manifold with boundary, we study how Ricci curvature of the interior affects the geometry of the boundary. First we establish integral inequalities for functions defined solely on the boundary and apply them to obtain geometric inequalities involving the total mean curvature. Then we discuss related rigidity questions and prove Ricci curvature rigidity results for manifolds with boundary.. Introduction and statement of results In this paper, we consider the question how the Ricci curvature of a compact manifold with boundary affects the boundary geometry of the manifold. For the scalar curvature the same question is related to the quasi-local mass problem in general relativity. Indeed, much of the formulation of the results in this paper is motivated by that in [7, 20, 0]. We begin with integral inequalities that hold for functions solely defined on the boundary. For simplicity, all manifolds and functions in this paper are assumed to be smooth. Theorem.. Let Ω, g) be an n-dimensional, compact Riemannian manifold with nonempty boundary. Let K be a constant that is a lower bound of the Ricci curvature of g, i.e. Ric Kg. Let be the mean curvature of in Ω, g) with respect to the outward normal. Suppose > 0. Given any function η on, define η 2 η Aη) = dσ, Bη) = η dσ, [ ] η) 2 Cη) = II η, η) dσ, where, are the gradient, the Laplacian on respectively, II is the second fundamental form of and dσ is the volume form on. 200 Mathematics Subject Classification. Primary 53C20; Secondary 53C24. Research partially supported by Simons Foundation Collaboration Grant for Mathematicians #2805.

2 2 PENGZI MIAO AND XIAODONG WANG Then, for each nontrivial η, either ) 2 Bη).) Cη) Aη) Aη) or.2) 2 K Bη) Aη) Bη) Aη) ) 2 Cη) Aη). Remark.. If the term II η, η) were absent in Cη), then.) would always hold by ölder inequality. Remark.2. When Ω is the closure of a bounded domain in R 3, the functional Cη), up to a constant multiple of, is the 2nd variation 8π of the Wang-Yau quasi-local energy [2, 22]) at = Ω in R 3,, where R 3, is the 4-dimensional Minkowski spacetime. See [0, ] for details.) Remark.3. If has a component 0 on which II > 0, then.) always fails for an η which is a non-constant eigenfunction on 0 and zero elsewhere. In this case,.2) yields estimates on the first nonzero eigenvalue of 0. See Corollary 2. for details.) The conclusion of Theorem. is easily seen to be equivalent to a statement.3) II η, η)dσ η + tη)2 dσ for all constants t K. In Theorem 2. of Section 2, we prove a 2 more general version of.3) which allows 0. Interpreted this way, Theorem. and its generalization Theorem 2.) have natural applications to the total mean curvature of the boundary. We first state the case of nonnegative Ricci curvature. Theorem.2. Let Ω, g) be an n-dimensional, compact Riemannian manifold with nonnegative Ricci curvature, with connected boundary which has nonnegative mean curvature. Let X : R m be an isometric immersion of into some Euclidean space R m of dimension m n. Then.4) dσ 0 2 dσ, where 0 is the mean curvature vector of the immersion X, 0 is the length of 0, and = {x 0 x) 0}. Moreover, if equality in.4) holds, then

3 BOUNDARY EFFECT OF RICCI CURVATURE 3 a) = 0 identically on. b) Ω, g) is flat and X) lies in an n-dimensional plane in R m. c) Ω, g) is isometric to a domain in R n if X is an embedding. Remark.4. In light of the Nash imbedding theorem [2], the boundary always admits an isometric immersion into some Euclidean space. Therefore, Theorem.2 applies to any compact Riemannian manifold with nonnegative Ricci curvature, with mean convex boundary i.e. 0). One may compare Theorem.2 with the result in [7] in which a weaker curvature condition R 0 is assumed, where R is the scalar curvature, while a more stringent boundary condition is imposed. Remark.5. If Ω, g) has nonnegative Ricci curvature and nonempty mean convex boundary, it was shown in [7, 8] also cf. [5]) that Ω has at most two components, and Ω has two components only if Ω, g) is isometric to N I for a connected closed manifold N and an interval I. This is why we only consider connected boundary in Theorem.2. Remark.6. Theorem.2 generalizes [5, Proposition 2], which proves that b) and c) hold under a pointwise assumption 0. Indeed the proof in [5] can be easily adapted to prove our Theorem.2. Remark.7. If > 0 on, Theorem.2 implies dσ C where C > 0 is a constant depending only on the induced metric on and a positive lower bound of. Next, we give an analogous result for manifolds with positive Ricci curvature. Theorem.3. Let Ω, g) be an n-dimensional, compact Riemannian manifold with positive Ricci curvature, with connected boundary which has nonnegative mean curvature. Let k > 0 be a constant such that Ric n )kg. Suppose there exists an isometric immersion X : S m k, where Sm k is the sphere of dimension m n with constant sectional curvature k. Then.5) dσ < S n )2 k dσ, where S is the mean curvature vector of the immersion X. Like.4),.5) imposes constraints on the boundary mean curvature when the Ricci curvature of the interior has a positive lower bound. For instance, consider the standard hemisphere S n +, g S ) of dimension n.

4 4 PENGZI MIAO AND XIAODONG WANG Let Ω S n + be a smooth domain with connected boundary. It follows from Theorem.3 that there does not exist a metric g on Ω satisfying Ric n ), g T Ω = g S T Ω and S ) 2 + n 4 )2, where and S are the mean curvature of Ω in Ω, g) and Ω, g S ) respectively. This could be compared with the first step, i.e. [, Theorem 4], in the construction of the counterexample to Min-Oo s Conjecture, in which a metric on S n + is produced so that it satisfies R nn ), but the mean curvature of S n + is raised to be everywhere positive. One may also compare this with the Ricci curvature rigidity theorems in [6]. When a manifold has negative Ricci curvature somewhere, we have Theorem.4. Let Ω, g) be an n-dimensional, compact Riemannian manifold with boundary which has nonnegative mean curvature. Let k > 0 be a constant satisfying Ric n )kg. Suppose has a component 0 which admits an isometric immersion X = t, x,..., x n ) : 0 m k R m,, where R m, is the m+)-dimensional Minkowski spacetime with m n and { } m m k = y 0, y,..., y m ) R m, y0 2 + yi 2 = k, y 0 > 0. Then.6) dσ + II t, t)dσ 0 0 { < 2 4 n )2 k + [ t 2 ] } 2 n )kt dσ, 0 where is the mean curvature vector of the immersion X into m k, 2 [ 4 n )2 k + t ] 2 2 n )kt 0 on 0, and 0 is the set consisting of x 0 such that 2 x) [ 4 n )2 k + t ] 2 2 n )kt x) > 0. Remark.8. The term 0 II t, t)dσ in.6) can be dropped if either II 0 or X 0 ) m k {t = t 0} for some constant t 0. For instance, this is the case if 0 can be isometrically immersed in a sphere. i=

5 BOUNDARY EFFECT OF RICCI CURVATURE 5 The fact that.5) and.6) are strict inequalities is due to the characterization of equality case in Theorem 2.. This leads naturally to rigidity questions in the context of Theorem.3 and.4. We have the following two related results. Theorem.5. Let Ω, g) be an n-dimensional, compact Riemannian manifold with boundary. Suppose Ric n ) g there exists an isometric immersion X : S m, where S m is a standard sphere of dimension m n II v, v) II S v, v), for any v T. ere II is the second fundamental form of in Ω, g) and II S is the vector-valued, second fundamental form of the immersion X. Then Ω, g) is spherical, i.e. having constant sectional curvature. Moreover if is simply connected, then Ω, g) is isometric to a domain in S n +. Theorem.6. Let Ω, g) be an n-dimensional, compact Riemannian manifold with boundary. Suppose Ric n ) g there exists an isometric immersion X : m, where m is a hyperbolic space of dimension m n II v, v) II v, v), for any v T. ere II is the second fundamental form of in Ω, g) and II is the vector-valued, second fundamental form of the immersion X. Then Ω, g) is hyperbolic, i.e. having constant sectional curvature. Moreover if is simply connected, then Ω, g) isometric to a domin in n. The rest of this paper is organized as follows. In Section 2, we prove Theorem 2. which implies Theorem.. In Section 3, we consider applications of Theorem 2. to the total boundary mean curvature and prove Theorem.2.4. In Section 4, we discuss the related rigidity question and prove Theorem.5 and A geometric Poincaré type inequality The main result of this section is the following geometric Poincaré type inequality for functions defined on the boundary of a compact Riemannian manifold. Theorem 2.. Let Ω, g) be an n-dimensional, compact Riemannian manifold with nonempty boundary. Suppose Ric n )kg and 0,

6 6 PENGZI MIAO AND XIAODONG WANG where Ric is the Ricci curvature of g, k is some constant, and is the mean curvature of in Ω, g) with respect to the outward normal. Suppose is not identically zero. Then 2.) II η, η)dσ \{ η+tη=0} η + tη)2 dσ for any nontrivial function η on and any constant t n )k. 2 ere II, ) is the second fundamental form of, and denote the gradient and the Laplacian on respectively. Moreover, equality in 2.) holds only if either k > 0, t = 0 and η is a constant; or k = t = 0 and η is the boundary value of some function u on Ω satisfying 2 u = 0. ere 2 denotes the essian on Ω, g). Remark 2.. The case k = 0, > 0 and t = 0 in 2.) was first proved in [] and is related to the second variation of Wang-Yau quasi-local energy [2, 22] at a closed 2-surface in R 3 R 3,. Proof. The basic tool we use is Reilly s formula [4]) [ 2 u 2 u) 2 + Ric u, u) ] dv 2.2) Ω = [ II u, u) 2 u) u ν ) ] 2 u dσ, ν which follows from integrating the Bochner formula. ere, dv denote the Laplacian, the volume form on Ω, g) respectively; ν is the outward unit normal to, and u is any function defined on Ω. Given any nontrivial η on and any constant λ nk, let u be the unique solution to { u + λu = 0 on Ω 2.3) u = η at. The fact that 2.3) has a unique solution in the case k > 0 follows from another theorem of Reilly [4, Theorem 4]) which states that the first Dirichlet eigenvalue λ of satisfies λ nk and λ = nk if and only if Ω, g) is isometric to a hemisphere in which case II is identically zero. Plug this u in 2.2), using the fact 2 u 2 = n u)2 + 2 u n u)g 2, λ u 2 dv = u 2 dv u u Ω Ω ν dσ,

7 BOUNDARY EFFECT OF RICCI CURVATURE 7 and the assumption Ric n )kg, we have 2.4) n [ II η, η) 2 ) nk λ) u 2 dv + Ω Ω η + n 2n λη Given any constant ɛ > 0, 2.4) implies 2.5) + 0. [ [ n 2 u + n λug 2 dv ) u ν II η, η) + + ɛ η + n ) 2 2n λη + ɛ η + n ) + ɛ 2n λη + + ɛ u ] 2 dσ ν ) nk λ) u 2 dv + 2 u + Ω Ω n λug 2 dv ) ] 2 u dσ. ν ) ] 2 u dσ ν Define η,λ = { x η + n λη = 0}. By Lebesgue s monotone 2n convergence theorem, we have 2.6) = lim ɛ 0 \ η,λ \ η,λ η + n + ɛ η + n 2n λη ) 2 2n λη dσ ) 2 dσ. Therefore, it follows from 2.5) and 2.6) that 2.7) II η, η)dσ \ η,λ η + n ) 2 2n λη dσ, which proves 2.) by setting t = n λ. 2n Next, suppose 2.8) II η, η)dσ = \ η,λ η + n ) 2 2n λη dσ. In particular, this shows 2.9) η + n ) 2n λη L 2 \ η,λ )

8 8 PENGZI MIAO AND XIAODONG WANG and the set {x \ η,λ x) = 0} has dσ-measure zero. ence, by 2.4) and 2.8), we have ) nk λ) u 2 dv + 2 u + n Ω Ω n λug 2 dv [ η + n ) \ η,λ 2n λη + u ] 2 2.0) dσ ν ) 2 u dσ, ν which implies η,λ 2.) nk λ) u = 0, 2 u + λug = 0 on Ω n and 2.2) η + u ν + n λη = 0 at. 2n If λ < nk, 2.) shows u is identically a constant, therefore λ = 0, k > 0 and η is a constant on. If λ = nk, 2.) shows 2.3) 2 u + kug = 0 on Ω, which implies 2.4) η + u ν + n )kη = 0 at. Comparing 2.4) to 2.2) with λ = nk, we have k = λ = 0. This completes the proof. When > 0, 2.) is simplified to II η, η)dσ η + tη)2 dσ. In this case, Theorem 2. is equivalent to a statement that, given any nontrivial η on, the quadratic form satisfies Q η t) := Aη)t 2 + 2Bη)t + Cη) 2.5) Q η t) 0, t n )k, 2

9 where 2.6) 2.7) BOUNDARY EFFECT OF RICCI CURVATURE 9 Aη) = Cη) = η 2 η dσ, Bη) = η dσ, [ ] η) 2 II η, η) dσ. Clearly 2.5) is equivalent to asserting that, for each fixed η, either 2.8) Bη) 2 Aη)Cη) or 2.9) n )k Bη) 2 Aη) Bη) Aη) ) 2 Cη) Aη). This explains how Theorem. follows from Theorem 2.. Next, we apply Theorem 2. to eigenvalue estimates on the boundary. Corollary 2.. Let Ω, g),, k,, II be given as in Theorem 2.. Suppose has a component 0 which is convex, i.e. II > 0 on 0. Let κ > 0 be a constant such that II κγ, where γ is the induced metric on 0. Let λ be a positive eigenvalue of on 0, γ). If κ 2 + 2k > 0, then [ 2.20) λ / 4 n ) κ ] 2 [ κ 2 + 2k, 4 n ) κ + ] ) 2 κ 2 + 2k. In particular, if k 0, the first nonzero eigenvalue λ 0 ) of 0, γ) satisfies 2.2) λ 0 ) 4 n ) [κ + κ 2 + 2k] 2. Proof. By defining η = 0 everywhere on \ 0, Theorem 2. implies II η, η)dσ 0 0 η + tη)2 dσ, for any η defined on 0 and any t n )k. Let Aη), Bη) and 2 Cη) be given in 2.6) and 2.7) with replaced by 0. Suppose η is a nonzero eigenfunction, i.e. η + λη = 0. Then ) ) Bη) 2 η Aη)Cη) = 0 2 dσ II η, η)dσ > 0. 0 Therefore, 2.9) holds, which shows 2.22) n )k λ 2 0 II η, η)dσ 0 η 2 dσ ) 2.

10 0 PENGZI MIAO AND XIAODONG WANG On the other hand, 2.23) II η, η)dσ κ η 2 dσ = κλ η 2 dσ and η 2.24) 0 2 dσ η 2 dσ. n )κ 0 ence, 2.22) 2.24) imply 2.25) 2 n )k λ κ n )λ. When κ 2 + 2k > 0, it follows from 2.25) that λ / 2 n [ κ κ 2 + 2k which completes the proof. ], 2 n [ κ + κ 2 + 2k] ), Remark 2.2. Corollary 2. is motivated by results in [3, 23]. If k = 0, 2.2) reduces to λ 0 ) n )κ 2 which is the estimate in [23]. 3. Application to total mean curvature In this section, we recall the statement of Theorem.2.4 and give their proof. We begin with the case of nonnegative Ricci curvature. Theorem 3.. Let Ω, g) be an n-dimensional, compact Riemannian manifold with nonnegative Ricci curvature, with connected boundary which has nonnegative mean curvature. Let X : R m be an isometric immersion of into some Euclidean space R m of dimension m n. Then 3.) dσ 0 2 dσ, where 0 is the mean curvature vector of the immersion X and is the set { 0 x) 0}. Moreover, if equality in 3.) holds, then a) = 0 identically on. b) Ω, g) is flat and X) lies in an n-dimensional plane in R m. c) Ω, g) is isometric to a domain in R n if X is an embedding. Proof. Since X is an isometric immersion, one has 3.2) X = 0. At any x, let {v α α =,..., n } T x be an orthonormal frame that diagonalizes II, i.e. IIv α, v β ) = δ αβ κ α where {κ,..., κ n }

11 BOUNDARY EFFECT OF RICCI CURVATURE are the principal curvature of in Ω, g) at x. Let {e,..., e m } denote the standard basis in R m and x i be the i-th component of X. Then m m n II x i, x i ) = IIv α, v β ) e i, v α e i, v β 3.3) i= = i= α,β= n κ α =. α= Set k = 0 in Theorem 2. and choose η = x i, t = 0 in 2.), we have 3.4) II x i, x i )dσ x i) 2 i dσ where i is the characteristic function of the set i = \ { x i = 0}. Summing 3.4) over i, using 3.2), 3.3) and the fact i, we have 3.5) dσ 0 2 dσ, which proves 3.). Next suppose 3.6) dσ = 0 2 dσ. Then it follows from 3.4) that 3.7) II x i, x i )dσ = x i) 2 i dσ, i. By the rigidity part of Theorem 2., there exist functions u i, i m, such that u i = x i at and 3.8) 2 u i = 0 on Ω. Moreover, by 2.2) or 2.4), we have 3.9) 0 + dφν) = 0 at, where Φ : Ω R m is a harmonic) map defined by Φ = u,..., u m ), dφ = du,..., du m ) is the associated tangent map, and ν is the unit outward normal to in Ω, g). We claim 3.0) dφν)x) 0, x. To see this, first consider a point y by 3.2)). At y, 3.9) implies 3.) dφν)y) 0 and dφν)y) X).

12 2 PENGZI MIAO AND XIAODONG WANG ence, the rank of dφ at y is n by 3.) and the fact Φ = X. On the other hand, 3.8) shows du i is parallel on Ω, i. Therefore, the rank of dφ equals n everywhere on Ω. In particular, this proves 3.0). By 3.9) and 3.0), we now have 3.2) {x x) 0} =. Thus 3.6) becomes 3.3) dσ = dφν) 2 dσ by 3.9). As is a nonempty open set in, 3.2) and 3.3) imply 3.4) dφν) z) = and dφν)z) X), z. It follows from 3.4), 3.9) and 3.2) that = 0 identically on. The rest of the claims now follows from [5, Proposition 2]. For completeness, we include the proof. By 3.4) and the fact Φ = X, one knows g = m i= du i du i at. As a result, g = m i= du i du i on Ω as both tensors are parallel. Clearly this shows Ω, g) is flat and Φ is an isometric immersion. Next, let v, w be any tangent vectors to Ω. 3.8) implies 3.5) 0 = vwφ) v wφ) = dφv) dφw)) dφ v w), where and denote the connection on Ω, g) and R m respectively. By definition, 3.5) shows Φ : Ω R m is totally geodesic. Therefore, ΦΩ) hence X)) lies in an n-dimensional plane in R m. Without losing generality, one can assume ΦΩ) R n. If X : R m is indeed an embedding, then X) = W where W is the closure of a bounded domain in R n. Since Φ is an immersion and Φ = X, one can show ΦΩ) W and Φ : Ω \ W \ W by checking that ΦΩ) \ W is both open and closed in R n \ W ). On the other hand, Φ being a local isometry implies Φ : Ω W is a covering map. Therefore, Φ is a homeomorphism, and hence an isometry between Ω and W. Theorem 3.2. Let Ω, g) be an n-dimensional, compact Riemannian manifold with positive Ricci curvature, with connected boundary which has nonnegative mean curvature. Let k > 0 be a constant such that Ric n )kg. Suppose there exists an isometric immersion X : S m k, where Sm k is the sphere of dimension m n with constant sectional curvature k. Then 3.6) dσ < S n )2 k dσ,

13 BOUNDARY EFFECT OF RICCI CURVATURE 3 where S is the mean curvature vector of the immersion X into S m k. Proof. We identify S m k with the sphere of radius k centered at the origin in R m+, i.e. S m k = { y,..., y m+ ) R m+ m+ i= y2 i = k} and view X = x,..., x m+ ) : S m k R m+ as an isometric immersion of into R m+. Let 0 denote the mean curvature vector of X : R m+, then 3.7) 0 = S + k 0, X X, where, is the inner product in R m+. Apply the fact 3.8) X = 0 and X, X = k, we have 3.9) have 0 = m+ i= xi x i + x i 2) = 0, X + n ). In Theorem 2., choose η = x i and t = n )k > 0 in 2.), we ) where i II x i, x i )dσ < is the characteristic function of the set i = \ { x i + 2 n )kx i = 0 [ x i + 2 n )kx i] 2 i dσ }. Summing 3.20) over i and using 3.7) 3.9), we have dσ < [ n )k 0, X + 4 ] n )2 k 2 X 2 dσ [ = S 2 + ] 4 n )2 k dσ, where we have also used 3.3). This proves 3.6). Theorem 3.3. Let Ω, g) be an n-dimensional, compact Riemannian manifold with boundary which has nonnegative mean curvature. Let k > 0 be a constant satisfying Ric n )kg. Suppose has a component 0 which admits an isometric immersion X = t, x,..., x n ) : 0 m k R m,,

14 4 PENGZI MIAO AND XIAODONG WANG where R m, is the m+)-dimensional Minkowski spacetime with m n and { } m m k = y 0, y,..., y m ) R m, y0 2 + yi 2 = k, y 0 > 0. Then 3.2) dσ + II t, t)dσ 0 0 { < 2 4 n )2 k + [ t 2 ] } 2 n )kt dσ, 0 where is the mean curvature vector of the immersion X into m k, 3.22) 2 [ 4 n )2 k + t ] 2 2 n )kt 0 on 0, and 0 is the set consisting of all x 0 such that 2 x) [ 4 n )2 k + t ] 2 2 n )kt x) > 0. Proof. Let M be the mean curvature vector of X : 0 R m,, then i= 3.23) M = k M, X X, and 3.24) X = M, X, X = k, where, = dy0 2 + m i= dy2 i is the Lorentzian product on R m,. By 3.24), we have m 0 = t t + t 2 ) + xi x i + x i 2) 3.25) = M, X + n ). At any x 0, let {v α α =,..., n } be an orthonormal frame in T x 0 such that IIv α, v β ) = δ αβ κ α where {κ,..., κ n } are the principal curvature of 0 in Ω, g) at x. We have i= n 3.26) II t, t) = κ α y0, v α 2 α=

15 BOUNDARY EFFECT OF RICCI CURVATURE 5 and 3.27) m II x i, x i ) = i= = n m ) κ α yi, v α 2 α= i= n κ α + y0, v α 2). α= Therefore, 3.28) dσ + 0 II t, t)dσ = 0 m i= 0 II x i, x i )dσ. Now choose η = x i on 0, η = 0 on \ 0, and t = n )k < 0 2 in Theorem 2., we have 3.29) 0 II x i, x i )dσ < 0i [ x i 2 i] 2 n )kx dσ 0i where 0i is the characteristic function of the set 0i = 0 \ { x i } 2 n )kx i = 0. Direct calculation using 3.23) 3.25) shows 3.30) m i= [ x i ] 2 2 n )kx i = 2 4 n )2 k + [ t 2 n )kt ] 2, which also proves 3.22). Summing 3.29) over i =,..., m and using 3.28) 3.30) together with the fact 0i 0, we have dσ + II t, t)dσ 0 0 { < 2 4 n )2 k + [ t 2 ] } 2 n )kt dσ. 0 This completes the proof.

16 6 PENGZI MIAO AND XIAODONG WANG 4. Rigidity results Inequalities 3.6) and 3.2) are not sharp in the context of Theorem 3.2 and Theorem 3.3. In these cases, one wonders if there exist sharp integral inequalities involving and S or ) which include a rigidity statement in the case of equality. In what follows, by scaling the metric, we assume Ric n )g or Ric n )g. In the latter case, the scalar curvature R of g satisfies R nn ). By the results in [20, 8, 9], there exists a sharp integral inequality relating and if the manifold Ω is spin and the boundary embeds isometrically in the hyperbolic space n as a convex hypersurface. On the other hand, the counterexample to Min-Oo s conjecture in [] shows that even the pointwise condition = S is not sufficient to guarantee rigidity if one only assumes R nn ). This gives rise to the following rigidity question: Question 4.. Let Ω, g) be an n-dimensional, compact Riemannian manifold with boundary. Let D S n + be a bounded domain with smooth boundary D, where S n + is the standard n-dimensional hemisphere. Suppose Ric n ) g there exists an isometry X : D S X, where, S are the mean curvature of, D in Ω, g), S n + respectively. Is Ω, g) isometric to D in S n +? At this stage, we do not know the answer to Question 4.. owever, it was shown in [6] that Question 4. has an affirmative answer if the assumption S X is replaced by a stronger assumption on the second fundamental forms. Theorem 4. [6]). Let Ω, g) be an n-dimensional, compact Riemannian manifold with boundary. Let D S n + be a bounded domain with smooth boundary D, where S n + is the standard n-dimensional hemisphere. Suppose Ric n ) g there exists an isometry X : D II II S X, where II, II S are the second fundamental form of, D in Ω, g), S n + respectively. Then Ω, g) is isometric to D in S n +. In the rest of this section, we prove Theorem.5 and.6, which are analogues of Theorem 4. when the boundary is only isometrically immersed in a sphere or in a hyperbolic space of higher dimension.

17 BOUNDARY EFFECT OF RICCI CURVATURE 7 Theorem 4.2. Let Ω, g) be an n-dimensional, compact Riemannian manifold with boundary. Suppose Ric n ) g there exists an isometric immersion X : S m, where S m is a standard sphere of dimension m n II v, v) II S v, v), for any v T. ere II is the second fundamental form of in Ω, g) and II S is the vector-valued, second fundamental form of the immersion X. Then Ω, g) is spherical, i.e. having constant sectional curvature. We divide the proof of Theorem 4.2 into a few steps. First, we fix some notations. Let denote the covariant derivative on S m, which is identified with the unit sphere centered at the origin in R m+. Given any α = α,..., α m+ ) S m, let F = F α be the restriction of the linear function α, x = α x + + α m+ x m+ to S m. The gradient of F on S m, denoted by F, is 4.) F x) = α α, x x, x S m. On, define f = F X. For simplicity, given any p, we let F be the component of F Xp)) normal to X T p ). Given v, w T p, recall that II S v, w) = X v)x w) ) is the component of X v)x w) normal to X T p ). We let S denote the mean curvature vector of X, which is the trace of II S. Lemma 4.. Along, one has f 2 + f ) F 2 =, 4.3) f + n ) f S, F = 0, 4.4) X f) F, n + II S f, f), n = 0. ere n is any vector that is normal to X) in S m. Proof. 4.2) follows from the fact F 2 + F 2 =. To show 4.3) and 4.4), we note that F on S m satisfies 4.5) 2 F = F g S, where g S is the standard metric on S m. 4.5) readily implies 4.6) 2 fv, w) II Sv, w), F = f v, w, v, w T,

18 8 PENGZI MIAO AND XIAODONG WANG where 2 denotes the essian on. Taking trace of 4.6) gives 4.3). 4.5) also implies, 0 = 2 F X f), n) = X f) F, n + II S f, f), n, which proves 4.4). The condition IIv, v) II S v, v), v T, implies S 0. By Reilly s theorem [4, Theorem 4]), to prove Theorem 4.2, it suffices to assume λ > n, where λ is the first Dirichlet eigenvalue of Ω, g). Under this assumption, we let u be the unique solution to { u + nu = 0 on Ω, 4.7) u = f at. On Ω, g), define φ = u 2 + u 2. A basic fact about φ is that it is subharmonic, which follows from 4.8) 2 φ = 2 u + ug 2 + Ric u, u) n ) u 2 0. On, define χ = u, where ν is the unit outward normal to in ν Ω, g). Then 4.9) φ = f 2 + χ 2 + f 2 = + χ 2 F 2 by 4.2) in Lemma 4.. Lemma 4.2. Along, the normal derivative of φ is given by φ 2 ν = f, χ II f, f) χ2 Proof. Direct calculation gives 4.0) φ 2 ν = 2 u u, ν) + fχ = 2 u f, ν) + χ [ 2 u ν, ν) + f ] S, F χ. = f, χ II f, f) + χ [ 2 u ν, ν) + f ]. By 4.2) and 4.7), at we have nf = u = f + χ + 2 u ν, ν) = n ) f + S, F + χ + 2 u ν, ν),

19 BOUNDARY EFFECT OF RICCI CURVATURE 9 which gives 4.) 2 u ν, ν) + f = S, F χ. The lemma follows from 4.0) and 4.). Proof of Theorem 4.2. Given any q, we choose α = Xq) S m. Then F Xq)) = 0 by 4.). ence, 4.2) fq) = 0, F q) = 0 and φq) = + χ 2 q). Consider p such that φp) = max Ω φ. By 4.9) and 4.2), 4.3) χ 2 p) F 2 p). Since f, φ p) = 0, taking n = F in 4.4), at p we have χ f, χ = X f) F, F 4.4) = II S f, f), F. If χ p) 0, it follows from Lemma 4.2, 4.3) and 4.4) that 4.5) φ 2 ν p) = II S χ f, f), F II f, f) S, F χ χ 2 ) II S f, f) II f, f) + S χ 2 0. If χ p) = 0, then Lemma 4.2 gives 4.6) φ 2 ν p) = f, χ II f, f). Moreover, F p) = 0 by 4.3). Taking the second order derivative of φ along f at p, we have 4.7) 0 2 f fφ))p) ) = f χ f, χ X f) F, F = f, χ 2 X f) F 2.

20 20 PENGZI MIAO AND XIAODONG WANG We claim, at p, 4.8) X f) F ) = II S f, f). To see this, take any v T, 4.5) and 4.9) imply X f) F, X v) 4.9) = 2 F X f), X v)) X f)x f), X v) = fvf) 2 f f, v) = 2 v f 2 + f 2) = 2 v φ χ 2). Clearly, v φ χ 2 ) varnishes at p. ence, X f) F )p) is normal to X T p ). This, together with 4.4), implies 4.8). Now it follows from 4.6), 4.7) and 4.8) that φ 2 ν p) II S f, f) II f, f) 0. By the strong maximum principle precisely the opf boundary point lemma), we conclude that φ must be a constant. ence, 2 u = ug by 4.8). Moreover, by 4.9) and 4.2), 4.20) χ 2 F 2 = c for some constant c 0. We have the following two cases: If c > 0, then χ 2 > F 2 0. This together with 4.5) and the fact φ is a constant implies S = = 0. Since II 0, we have II = 0 and II S = 0. Thus, X : S m is totally geodesic. ence X) lies in an n )-dimensional standard sphere S n S m. Since X : S n is an isometric immersion, we have X) = S n ; moreover X : S n is one-to-one as S n is simply connected. Therefore, is isometric to S n and is totally geodesic in Ω, g). By [6, Theorem 2], we conclude that Ω, g) is isometric to a standard hemisphere S n +. If c = 0, then φ = on and hence on Ω). In this case, along, u 2 = f 2 + χ 2 = f 2 + F 2 = F 2 X. In particular, u q) = 0 by 4.2). We also note that u q) = fq) = by the definition of F. Finally, we are in a position to show Ω, g) has constant sectional curvature. It suffices to assume Ω, g) is not isometric to S n +. Given any x in the interior of Ω, let q x such that distx, ) = distx, q x ), where dist, ) denotes the distance functional on Ω, g). Consider the

21 BOUNDARY EFFECT OF RICCI CURVATURE 2 function f and u constructed in the above proof by taking q = q x. Since Ω, g) is not isometric to S n +, the constant c in 4.20) must be 0, hence u satisfies 4.2) 2 u = ug, uq x ) = 0, uq x ) =. Let γ : [0, L] Ω, g) be the geodesic satisfying γ0) = q x, γl) = x and L = distx, ). Let ξ = γ 0). Given any constant l 0, L), there exists an open neighborhood W of ξ in S n such that the exponential map exp qx, ) is a diffeomorphism from 0, l) W R + S n onto its image in Ω, g). Now it is a standard fact that 4.2) implies 4.22) exp qx ) g) = dr 2 + sin r) 2 g S n on 0, l) W, where g S n is the standard metric on S n cf. [6] for details). Therefore, g has constant sectional curvature at γt) for any t < L. By continuity, g has constant sectional curvature at x. This completes the proof that Ω is spherical. As an application, we have the following rigidity result which is a spherical analogue of [5, Theorem ]. Corollary 4.. Let Ω, g) be an n-dimensional, compact Riemannian manifold with boundary. Suppose Ric n )g g has constant sectional curvature at every point on. If is simply connected with nonnegative second fundamental form II, then Ω, g) is isometric to a domain in S n +. Proof. Let R,,, ), denote the curvature tensor, the connection on respectively. By the Gauss equation and the Codazzi equation, R v, v 2, v 3, v 4 ) = v, v 3 v 2, v 4 v, v 4 v 2, v 3 + II v, v 3 ) II v 2, v 4 ) II v, v 4 ) II v 2, v 3 ), 0 = v II ) v 2, v 3 ) v 2 II ) v, v 3 ) where v,... v 4 T. As is simply connected, the fundamental theorem of hypersurfaces cf. [9]) implies there exists an isometric immersion Φ : S n with II as its second fundamental form. Since II 0, by a result of Do Carmo and Warner in [4], Φ is an embedding and Φ) is a convex hypersurface in a hemisphere S n +. Now apply Theorem 4.2, we conclude that Ω, g) has constant sectional curvature everywhere. Let D be the region bounded by Φ) in S n +. We glue Ω and S n \ D along the boundary via the isometric embedding Φ to get a closed manifold M, g). The fact that has the same second fundamental form in Ω, g) and S n and that both Ω, g) and S n have constant

22 22 PENGZI MIAO AND XIAODONG WANG sectional curvature imply that g is C 2 indeed smooth) across in M. ence, M, g) is a spherical space form whose volume is greater than half of S n because it contains a hemisphere). Therefore, M, g) is isometric to S n and Ω, g) is isometric to a domain in S n +. Remark 4.. One can also apply Theorem 4. in the above proof. Theorem.5 now follows from Theorem 4.2 and Corollary 4.. When Ric n ), similarly we have Theorem 4.3. Let Ω, g) be an n-dimensional, compact Riemannian manifold with boundary. Suppose Ric n ) g there is an isometric immersion X : m, where m is a hyperbolic space of dimension m n II v, v) II v, v), for any v T. ere II is the second fundamental form of in Ω, g) and II is the vector-valued, second fundamental form of the immersion X. Then Ω, g) is hyperbolic, i.e. having constant sectional curvature. The proof is parallel to that of Theorem 4.2. Let, denote the dot product on R m, and be the connection on m. Identify m with {x R m, x, x =, x 0 > 0}. For any α X) m, consider F x) = F α x) = α, x on m. Its gradient is F x) = α + α, x x. Thus F x) 2 = + F 2. Given any p, let F X p) be the component of F Xp) orthogonal to X T p ). On, define f = F X. Let u be the smooth solution to { u = nu on Ω u = f at and define χ = u. Then φ := ν u 2 u 2 is subharmonic as seen from 2 φ = 2 u ug 2 + Ric u, u) + n ) u 2. Similar to 4.9), we have φ = f 2 + χ 2 f 2 = + χ 2 F 2. By analyzing the normal derivative φ in the same way as in the proof ν of Theorem 4.2, we conclude by the strong maximum principle that φ is constant. Therefore, 2 u = ug and χ 2 F 2 is a nonnegative constant c along. If c > 0, it implies II = 0, which contradicts the fact m does not contain a closed totally geodesic submanifold.

23 BOUNDARY EFFECT OF RICCI CURVATURE 23 Therefore χ 2 F 2 = 0 at, which shows φ = on Ω. By the same argument as that of Theorem 4.2, we conclude that Ω, g) has constant sectional curvature. Corollary 4.2. Let Ω, g) be an n-dimensional, compact Riemannian manifold with boundary. Suppose Ric n )g g has constant sectional curvature at every point on. If is simply connected with nonnegative second fundamental form II, then Ω, g) is isometric to a domain in n. The proof is similar to that of Corollary 4.. Since g has constant sectional curvature along and is simply connected, by the Gauss and Codazzi equations and the fundamental theorem of hypersurfaces cf. [9]), there exists an isometric immersion Φ : n with II as its second fundamental form. Since II 0, by the remark in Section 5 of Do Carmo and Warner [4], Φ is an embedding and Φ) is a convex hypersurface in n. Apply Theorem 4.3 to Ω, g) and the embedding Φ, we conclude that g has constant sectional curvature everywhere on Ω. Now let D be the region bounded by Φ) in n. We glue Ω and n \ D along the boundary via the isometric embedding Φ to get a complete manifold M, g). The fact has the same second fundamental form in Ω, g) and n and both Ω, g) and n have constant sectional curvature imply that g is smooth across in M. ence, M, g) is a complete, hyperbolic manifold which, outside a compact set, is isometric to n minus a ball. We conclude that M, g) is isometric to n and Ω, g) is isometric to a domain in n. This final claim can be seen, for instance, by the following: Proposition 4.. Let M, g) be a complete, n-dimensional Riemannian manifold with Ric n ) g. Suppose that there exists a compact set K M s.t. M \ K is isometric to n \ B where B is a geometric ball. Then M is isometric to n. The Euclidean version of the result is well known e.g. it appears as an exercise in [3] several times). The hyperbolic case can be proved by similar methods. For lack of an exact reference, we outline a proof using Busemann functions. The main idea comes from Cai-Galloway [2]. We use the upper space model n = {x R n : x n > 0}. Without loss of generality we take o = 0,..., 0, ). For a sequence ɛ k 0, let S k M be the hypersurface corresponding to x n = ɛ k and q k the point corresponding to 0,..., 0, /ɛ k ). Let p k be the point on S k closest to

24 24 PENGZI MIAO AND XIAODONG WANG q k and γ k : [ a k, b k ] M be a minimizing geodesic from p k to q k s.t. γ k 0) K it is easy to see that any minimizing geodesics from p k to q k must intersect K). Passing to a subsequence ɛ k 0 if necessary, we can assume that γ k 0) ō and γ k converges to a geodesic line γ : R M. We consider the following generalized Busemann function βx) = lim k dō, S k ) dx, S k ). Then we have Claim: β n in the support sense. The crucial fact is that S k has constant mean curvature = n. The argument is the same as in Cai-Galloway [2]. We also have the standard Busemann function b associated with the ray γ [0, ) defined by bx) = lim k s dx, γs)). It is known that b n in the support sense. The rest of the proof is similar to Cai- Galloway [2] or the proof of the Cheeger-Gromoll splitting theorem. We have b+β) 0. By the triangle inequality one can show b+β 0. On the other hand b + β = 0 along γ. Therefore by the strong maximum principle b + β = 0. Then β = b and it is a smooth function with beta =. By the Bochner formula one can show that 2 β = g dβ dβ. From this identity one can show that M is isometric to the warped product R S n, dt 2 + e 2t h), where S, h) is a flat Riemannian manifold. It is then clear that S, h) must be the standard R n. This finishes the proof of Proposition 4.. References [] Brendle, S., Marques, F. C. and Neves, A., Deformations of the hemisphere that increase scalar curvature, Invent. Math ), [2] Cai, M. and Galloway, G. Boundaries of zero scalar curvature in the AdS/CFT correspondence. Adv. Theor. Math. Phys ), no. 6, ). [3] Choi,. I. and Wang, A. N., A first eigenvalue estimate for minimal hypersurfaces, J. Diff. Geom ), [4] do Carmo, M. P. and Warner, F. W. Rigidity and convexity of hypersurfaces in spheres. J. Differential Geometry 4 970), [5] ang, F. and Wang, X., Vanishing sectional curvature on the boundary and a conjecture of Schroeder and Strake, Pacific J. Math ), no. 2, [6] ang,f. and Wang,X., Rigidity theorems for compact manifolds with boundary and positive Ricci curvature, J. Geom. Anal. 9, 2009), [7] Ichida, R., Riemannian manifolds with compact boundary, Yokohama Math. J ) no. 2, [8] Kasue, A., Ricci curvature, geodesics and some geometric properties of Riemannian manifolds with boundary, J. Math. Soc. Japan ), no., 7 3.

25 BOUNDARY EFFECT OF RICCI CURVATURE 25 [9] Kwong, K.-K., On the positivity of a quasi-local mass in general dimensions, Comm. Anal. Geom ) no. 4, [0] Miao, P. and Tam, L.-F., On second variation of Wang-Yau quasi-local energy, Ann. enri Poincaré 5 204), no. 7, [] Miao, P. and Tam, L.-F. and Xie, N.-Q., Critical points of Wang-Yau quasilocal energy, Ann. enri Poincaré 2 20), no. 5, [2] Nash, J., The imbedding problem for Riemannian manifolds, Ann. of Math. 2) ), [3] Petersen, P., Riemannian geometry. Second edition. Graduate Texts in Mathematics, 7. Springer, New York, [4] Reilly, R. C., Applications of the essian operator in a Riemannian manifold, Indiana Univ. Math. J ), [5] Sacksteder, R., On hypersurfaces with no negative sectional curvatures, Amer. J. Math ), [6] Schroeder, V. and Strake, M., Rigidity of convex domains in manifolds with nonnegative Ricci and sectional curvature, Comment. Math. elv. 64 2) 989), [7] Shi, Y.-G. and Tam, L.-F., Positive mass theorem and the boundary behaviors of compact manifolds with nonnegative scalar curvature, J. Differential Geom ), [8] Shi, Y.-G. and Tam, L.-F., Rigidity of compact manifolds and positivity of quasi- local mass, Class. Quantum Grav ), [9] Spivak, M., A comprehensive introduction to differential geometry. Vol. IV. Second edition. Publish or Perish, Inc., Wilmington, Del., 979. [20] Wang, M.-T. and Yau, S.-T., A generalization of Liu-Yaus quasi-local mass, Comm. Anal. Geom., ), no. 2, [2] Wang, M.-T. and Yau, S.-T., Quasilocal mass in general relativity, Phys. Rev. Lett ), 020. [22] Wang, M. -T. and Yau, S.-T., Isometric embeddings into the Minkowski space and new quasi-local mass, Comm. Math. Phys ), no. 3, [23] Xia, C., Rigidity of compact manifolds withboundary and nonnegative Ricci curvature, Proc. Amer. Math. Soc ) no. 6, Pengzi Miao) Department of Mathematics, University of Miami, Coral Gables, FL 3346, USA. address: pengzim@math.miami.edu Xiaodong Wang) Department of Mathematics, Michigan State University, East Lansing, MI 48864, USA. address: xwang@math.msu.edu