Calculus of Variations

Transcription

1 Calc. Var. DOI /s Calculus of Variations Extension of a theorem of Shi and Tam Michael Eichmair Pengzi Miao Xiaodong Wang Received: 13 November 2009 / Accepted: 31 January 2011 Springer-Verlag 2011 Abstract In this note, we prove the following generalization of a theorem of Shi and Tam (J Differ Geom 62:79 125, 2002): Let (, g) be an n-dimensional (n 3) compact Riemannian manifold, spin when n > 7, with non-negative scalar curvature and mean convex boundary. If every boundary component i has positive scalar curvature and embeds isometrically as a mean convex star-shaped hypersurface ˆ i R n, then Hdσ Ĥdˆσ i ˆ i where H is the mean curvature of i in (, g), Ĥ is the Euclidean mean curvature of ˆ i in R n, and where dσ and d ˆσ denote the respective volume forms. Moreover, equality holds for some boundary component i if, and only if, (, g) is isometric to a domain in R n. In the proof, we make use of a foliation of the exterior of the ˆ i s in R n by the H R -flow studied by Gerhardt (J Differ Geom 32: , 1990) and Urbas (Math Z205(3): ,1990). We also carefully establish the rigidity statement in low dimensions without the spin assumption that was used in Shi and Tam (J Differ Geom 62:79 125, 2002). Communicated by G. Huisken. M. Eichmair Department of Mathematics, MIT, 77 Massachusetts Avenue, Cambridge, MA 02139, USA eichmair@math.mit.edu P. Miao Department of Mathematics, University of Miami, Coral Gables, FL 33146, USA p.miao@math.miami.edu P. Miao School of Mathematical Sciences, Monash University, Melbourne VIC 3800, Australia Pengzi.Miao@sci.monash.edu.au X. Wang (B) Department of Mathematics, Michigan State University, East Lansing, MI 48824, USA xwang@math.msu.edu

2 M. Eichmair et al. Mathematics Subject Classification (2010) 53C21 53C24 58J35 1 Introduction and statement of results In the work of Shi and Tam [22, Theorem 4.1], the positive mass theorem was used in a novel way to yield beautiful results on the boundary behavior of compact Riemannian manifolds with non-negative scalar curvature: Theorem 1 (Shi Tam) Let (, g) be an n-dimensional compact Riemannian spin manifold with non-negative scalar curvature and mean convex boundary. If every component i of the boundary is isometric to a strictly convex hypersurface ˆ i R n, then Hdσ Ĥdˆσ (1) ˆ i i where H is the mean curvature of i in (, g), Ĥ is the Euclidean mean curvature of ˆ i in R n, and where dσ and d ˆσ denote the respective volume forms. Moreover, equality in (1) holds for some boundary component i if, and only if, (, g) is isometric to a domain in R n. Note that every compact strictly convex hypersurface ˆ as above bounds a compact region in R n. Throughout this article, the (scalar) mean curvature of the boundary of a set is computed with respect to the outward unit normal. With this convention, the boundary of a unit ball in R n has mean curvature (n 1). As a consequence of the Weyl embedding problem solved by Pogorelov [18] and Nirenberg [17] independently, in dimension n = 3 the assumption in Theorem 1 that every boundary component i embeds isometrically as a strictly convex hypersurface in R 3 is equivalent to the requirement that the boundary of (, g) has positive Gaussian curvature. Hence, the conclusion (1) in the theorem shows that positively curved mean convex boundaries in time-symmetric initial data sets satisfying the dominant energy condition have non-negative Brown York mass [5]. This result has been generalized to subsets of general initial data sets by Liu and Yau in [11,12], and to a hyperbolic setting by Wang and Yau in [25]. In dimensions n > 3, there are no analogous intrinsic conditions on the boundary of (, g) that guarantee that its components embed isometrically into R n. There are two major ingredients in Shi and Tam s proof of Theorem 1. For simplicity, let us assume that the boundary of has only one component. Let ι : := R n be its isometric embedding. Let ν : ι() S n 1 be the outer unit normal. Since ι() is assumed to be a strictly convex hypersurface in R n there is a smooth family of embeddings F : [0, ] R n where F t (σ ) = F(σ, t) = ι (σ ) + tν (ι(σ)). Note that F t () are the outer distance surfaces of ι(). If ˆ denotes the bounded domain enclosed by ι(), then {F t ()} t 0 foliates R n \ ˆ and the Euclidean metric on this set can be written as G = dt 2 + g t, where g t is the first fundamental form of the embedding F t : R n. Shi and Tam then find an asymptotically flat scalar flat metric G = u 2 dt 2 + g t

3 Extension of a theorem of Shi and Tam on R n \ ˆ such that the mean curvature of {0} in (R n \ ˆ, G) coincides with the mean curvature of in (, g). The function u : [0, ) (0, ) is obtained as the solution of a certain nonlinear heat equation. The second crucial ingredient of their proof is the observation that ( m(t) = H t 1 u 1 ) dσ t, (2) {t} is a non-increasing function of t 0whereH t is the mean curvature of F t () and dσ t is its volume form, and that the limit of m(t) as t is the ADM mass of G. The final step in their proof is to apply the positive mass theorem for spin manifolds to the asymptotically flat manifold obtained from gluing (, g) and (R n \ ˆ, G) along their boundaries. Given Theorem 1, a natural question to ask is whether the requirement that the embeddings of the boundary components be strictly convex is really necessary. In this article we present some variations on Shi Tam s method. We prove that their theorem continues to hold provided each boundary component can be embedded isometrically into R n in such a way that the unbounded component of the complement of the embedded surface is foliated by mean convex leaves of positive scalar curvature. Moreover, we note that the spin assumption can be dropped when 3 n 7. To be precise, we have the following Proposition 1 Theconclusionof Theorem 1 remains valid if the spin assumption is dropped in dimensions 3 n 7 and if the hypothesis that every boundary component can be embedded isometrically as a strictly convex hypersurface in R n is eased to the requirement that every boundary component is isometric to an embedded surface ˆ i R n that can be deformed to a strictly convex hypersurface via an expanding flow through embedded mean convex hypersurfaces of positive scalar curvature. That is, if there exists a smooth map F : i [0, 1] R n such that F( i, 0) = ˆ i, F( i, t) is a smooth embedded, mean convex hypersurface with positive scalar curvature for every t [0, 1], t F = ην where ν is the outward unit normal of F( i, t) and where η>0is a smooth positive function, and F( i, 1) R n is a strictly convex hypersurface. This proposition makes it possible to try some other foliations even when the embeddings are not convex. In particular, if each ˆ i is a star-shaped surface with positive scalar curvature and positive mean curvature, we can use the result of Gerhardt [6] and Urbas [24] to obtain a foliation of R n \ ˆ i. This leads to Theorem 2 The conclusionof Theorem1 remains valid if the assumption that every boundary component embeds as a strictly convex hypersurface in R n is relaxed to the requirement that the boundary of (, g) has positive scalar curvature and that each boundary component is isometric to a mean-convex, star-shaped hypersurface in R n. Moreover, the spin assumption can be dropped in dimensions 3 n 7. This article is organized as follows. In Sect. 2, we discuss the generalized quasi-spherical metric construction of Bartnik [2], Smith Weinstein [23], and Shi Tam [22] along any finitetime geometric flow in the generality that is appropriate for our needs (see also [16], where this construction is carried out along a foliation of inverse mean curvature flow), and we derive the crucial monotonicity of (2). In Sect. 3, we prove Proposition 1 with the spin assumption, and point out that the inequality in Proposition 1 remains true without the spin assumption. In Sect. 4, we examine carefully the equality case in Proposition 1 without the spin assumption. It appears that the rigidity case of the positive mass theorem [21] and[20]

4 M. Eichmair et al. on non-spin asymptotically flat manifolds with Lipschitz singularities as in [13,22] is not addressed in the literature, so we derive it carefully in Sect. 4. In Sect. 5,wereviewtheworks of Gerhardt [6] and Urbas [24], which together with Proposition 1, imply Theorem 2. 2 Generalized quasi-spherical metric construction and the monotonicity of the Brown York mass In this section we review the results in [2,22,23] on the construction of certain asymptotically flat, scalar flat metrics defined outside smooth compact sets ˆ R n such that both boundary metric and mean curvature are prescribed. Using geometric arguments, we re-derive the necessary evolution equations contained in these references, as well as the required a priori estimate. We discuss how this construction is used in our proof of Proposition 1.The monotonicity of the Brown York mass is derived in Proposition 3 and generalizes the result in [22, Lemma 4.2]. Let beaclosed(n 1)-dimensional manifold and let t 0 > 0. Let {g t } t [0,t0 ] be a smooth family of Riemannian metrics on. Given any smooth positive function f on [0, t 0 ], consider the Riemannian metric g f := f 2 dt 2 + g t. (3) Let H f, h f denote the mean curvature and second fundamental form of {t} with respect to the g f metric (where the mean curvature is the trace of the second fundamental form, and the signs are so that H f measures the rate of change of the area element in direction t ). If we let 1 denote the constant function that is identically one, then fh f = h 1 and fh f = H 1. As in Proposition 1 we make the following standing Assumption The scalar curvature R(g t ) =: 2K of g t and the mean curvature H 1 of the leaves {t} with respect to g 1 are everywhere positive. Proposition 2 (cf. [2,22,23]) Under the above assumption, given any positive function u 0 on {0}, there is a smooth positive function u on [0, t 0 ] such that the scalar curvature R(g u ) of g u is identically zero and u t=0 = u 0. Proof By the Jacobi equation and the Gauss equation, we have that d dt H u = d H 1 dt u = u u( h u 2 + Ric(g u )(ν u,ν u )) = u u 2 (R(g u) 2K + Hu 2 + h u 2 ) = u + Ku 1 2u (H h 1 2 ) (4) where 2 is taken with respect to g t, Ric(g u ) denotes the Ricci curvature of g u,ν u = u 1 t, and is taken with respect to g t. Setting R(g u ) = 0, we have that u + u2 u = u 3 K u (H1 2 H 1 H 1 2H + h H 1 ) (5) 1 where we abbreviated the t-derivative with a dash. The assumptions u 0, H 1 > 0 guarantee that (5) has a smooth positive solution u with initial condition u t=0 = u 0 on some small interval [0,δ),δ > 0. To show such a solution

5 Extension of a theorem of Shi and Tam u can be extended to the whole interval [0, t 0 ], it suffices to prove that u remains bounded from above and from below by some positive constants (depending only on t 0 ) by standard parabolic theory. To derive an upper bound for u, let C > max u 0 be a positive constant such that C 2 max [0,t 0 ] H1 2 + h H 1. (6) 2K Suppose u C somewhere on [0,δ).Since u(σ, 0) <C, there exists ( σ, t) (0,δ) such that u( σ, t) = C and u(σ, t) C for t t. At ( σ, t), by the assumptions H 1 > 0and K > 0, we have u + u2 u 0, u 3 K u (H1 2 H 1 H 1 2H + h H 1 )>0, 1 thus a contradiction to (5). Hence, u < C. To get a lower bound of u, define u = βe γ t where β<min u 0 is a positive constant and γ is another positive constant such that ( ) γ> max [0,t 0 ] β 2 K H 1 H h H 1 2H 1. (7) Let v = u u, then v satisfies v + u2 v < (u 3 u 3 ) K v (H1 2 H 1 H 1 2H + h H 1 ). (8) 1 Now suppose v 0 somewhere on [0,δ). Since v t=0 > 0, there exists ( σ, t) (0,δ)such that v( σ, t) = 0andv 0fort t. At ( σ, t), by the assumptions H 1 > 0 and K > 0, we have v + u2 v 0, (u 3 u 3 ) K v (H1 2 H 1 H 1 2H + h H 1 ) = 0, 1 thus a contradiction to (8). Hence v>0, i.e. u(σ, t) >βe γ t for all (σ, t) [0, t 0 ). The monotonicity formula in the following proposition generalizes [22, Lemma 4.2]. Proposition 3 Suppose u and η are two smooth positive functions on [0, t 0 ] such that R(g u ) = 0 and Ric(g η ) = 0. Then (H η H u ) dσ t {t} is monotone non-increasing in t. Here dσ t denotes the volume form of g t on {t}. Proof By (4), we have d (H η H u ) dσ t dt = {t} {t} (η 1 u 1 )H K (η u) 1 2 (η 1 u 1 )(H h 1 2 ) dσ t. (9)

6 M. Eichmair et al. By the Gauss equation and the assumption that Ric(g η ) = 0, we have 2K = H 2 η h η 2 = η 2 (H 2 1 h 1 2 ). (10) Therefore, it follows from (9)and(10)that d (H η H u ) dσ t = dt {t} where we also used the assumption that K > 0. {t} K (η u) 2 u 1 dσ t 0, (11) 3 Proof of Proposition 1 using the spin assumption for the equality case Let (, g) be as in Proposition 1. Let{ 1,..., p } be the boundary components of, andfixone i. The pull-back of the Euclidean metric by F to i [0, 1] has the form g ηi = ηi 2dt2 + g t where g t is a time dependent metric on i. Using Proposition 2 we can find a smooth positive function u i : i [0, 1] R so that the scalar curvature of the metric g ui := ui 2dt2 + g t vanishes and so that u i (σ, 0) = η(σ, 0) ι Ĥ H. Here, Ĥ is the Euclidean mean curvature of ˆ i = ι( i ) R n and H is the mean curvature of i in (, g). This means that the mean curvature of i i {0} equals H in the g ui metric. By Proposition 3, we have that ˆ i Ĥ H = i = ˆ i Ĥ i {0} i {0} H ui (H η H ui ) i {1} (H η H ui ) (12) where as before, H η is the mean curvature of i {1} in the g η metric (or, equivalently, the Euclidean mean curvature of F( i, 1)), and H ui the mean curvature of i {1} with respect to the metric g ui. For convenience, we omit writing the volume forms. Next, let N i be the exterior region of the strictly convex hypersurface F( i, 1) in R n. By the result of Shi and Tam [22] (cf. Theorem 1) there exists an asymptotically flat metric g i on N i such that g i has vanishing scalar curvature and such that along the hypersurface F( i, 1) the metric g i coincides with the Euclidean metric, and its mean curvature with respect to g i equals the mean curvature of i {1} with respect to g ui via the identification F(, 1). Furthermore, (H η H ui ) = Ĥ H ui c(n)m(g i ), (13) i {1} F( i,1) i {1} where c(n) is a positive constant depending only on n and m(g i ) is the ADM mass of g i [1]. Now we have 1 + 2p Riemannian manifolds (, g), ( i [0, 1], g ui ), (N i, g i ) with non-negative scalar curvature whose boundaries are identified via ι and F(, 1) respectively such that the inner and outer mean curvatures match along these identifications. Glue these manifolds together to obtain an asymptotically flat manifold (M, G) as in [22]. If is spin, then the positive mass theorem for spin manifolds with Lipschitz singularities in [22, Sect. 3] can be applied to (M, G) and shows that m(g i ) 0, which together with (12), (13) gives inequality (1). Moreover, if equality in (1) holds, then the rigidity statement of the positive

7 Extension of a theorem of Shi and Tam mass theorem for spin manifolds with Lipschitz singularities in [22, Sect.3]implies that (, g) is isometric to a subset of (R n, g). We note that if 3 n 7but is not assumed to be spin, then m(g i ) 0 follows from the positive mass theorem for manifolds with corners along hypersurfaces in [13]. Hence inequality (1) remains valid provided 3 n 7even when is not spin. 4 Equality case in Proposition 1 without the spin assumption In this section, we examine the equality case in Proposition 1 when is not assumed to be spin but the dimension n satisfies 3 n 7. We will use several ideas from [21] related to the change of mass under conformal changes of the metric. The following lemma is standard and well-known to experts in geometric measure theory: Lemma 1 Let (, g) be a compact Riemannian manifold of dimension 3 n 7 with boundary components { 1,..., p } where p 2. Assume that the boundary is mean convex with respect to the outward pointing unit normal. Then for every i {1,...p} there exists a subdomain i so that i has two components: i and a smooth embedded minimal surface. Put differently, every boundary component is homologous to a minimal surface in (, g). We divide the proof of the equality case in Proposition 1 into a sequence of lemmas. Lemma 2 Suppose 3 n 7. The inequality (1) in Proposition 1 remains true if (, g) has additional boundary components, whose union we denote by 0, provided 0 is a minimal surface. Proof We proceed as in the proof of inequality (1) in the case where 0 is empty by constructing an asymptotically flat manifold (M, G) which contains (, g) isometrically and has non-negative scalar curvature (distributionally across the boundary components i for i {1,...,p}). We double (M, G) across the minimal surface 0 and verify that the positive mass theorem applies in this situation, cf. [3]. For ease of notation, we continue to denote the doubled manifold by (M, G). By [13, Proposition 3.1], there exists a sequence of smooth metrics g δ such that g δ agrees with G outside a δ-neighborhood of the union of p i, p F( i, 1), their reflections, and 0 such that g δ G in C 0 as δ 0andsuch that the scalar curvature R(g δ ) is bounded below by a constant that is independent of δ>0. Let R(g δ ) denote the negative part of R(g δ ). As in [13, Sect. 4.1], we let u δ : M R be the (unique, positive) solution of gδ u δ + n 2 4(n 1) R(g δ) u δ = 0, n 2 which tends to 1 in all of the 2p asymptotically flat ends. Let G δ := u 4 δ g δ. By [13, Lemma 4.2], lim δ 0+ m(g δ, N i ) = m(g, N i ) for each end N i of M. Since G δ has non-negative scalar curvature, the positive mass theorem implies m(g, N i ) 0. Therefore, m(g) 0, and the lemma follows as in the proof of Proposition 1 in Sect. 3. Lemma 3 Suppose 3 n 7 and that the equality (1) in Proposition 1 holds for some boundary component i, i {1,...,p}. Then H i ι (Ĥ ˆ i ). Moreover, if p 2 and if 0 is the minimal surface provided by Lemma 1 such that 0 and i bound a subdomain i of, then R(g) 0 on i.

8 M. Eichmair et al. Proof Let N i be the asymptotically flat end corresponding to i that is constructed in Sect. 3. Then by assumption 0 m(n i, G) i (Ĥ H) = 0. By Proposition 3, we conclude that u i = η i, where u i and η i are the functions in the proof in Sect. 3. In particular, this implies H i ι (Ĥ ˆ i ). Next suppose p 2andlet 0, i be as in the statement of the lemma. Suppose R(g) is strictly positive somewhere in i. Let u : i R be the solution of the Dirichlet problem g u + 4(n 1) n 2 R(g)u = 0 such that u = 1on i. Note that u cannot be constant. The strong maximum principle gives that 0 < u 1, with equality only on the boundary, and further that ν(u) >0whereν is the outward pointing unit normal. It follows that in the conformal metric ḡ := u n 2 4 g the boundary components 0, i have strictly greater mean curvature than with respect to the g metric, so that in particular 0 becomes mean convex in (, ḡ). Now let N i be the asymptotically flat end constructed in Sect. 3 corresponding to the boundary component i in ( i, ḡ). Let (M, Ḡ) be the asymptotically flat manifold obtained by gluing (, ḡ) and N i along i. Let H be the mean curvature of i in ( i, ḡ). By the proof in Sect. 3 and the fact that H > H, we then have m(ḡ, N i ) (Ĥ H) <0. (14) i On the other hand, we can double the manifold (M, Ḡ) across its mean convex boundary 0 just as in the proof of Lemma 2 to conclude that m(ḡ, N i ) 0. This is a contradiction to (14). Therefore, R(g) must be identically zero in, as asserted. Our next lemma is a minor modification of [15, Corollary 2.1]: Lemma 4 Let (, g) be a smooth compact connected manifold with disconnected boundary = 0 such that is mean convex, 0 is a minimal surface, and such that the scalar curvature R(g) 0. Suppose that g is such that H is largest amongst all nearby (in C 2 ) metrics ḡ that are scalar flat, so that ḡ T = g T, and such that 0 and are respectively minimal and mean convex (with mean curvature H) in (, ḡ). Then Ric(g) 0. Proof Let h be a smooth symmetric (0, 2)-tensor compactly supported in the interior of, let g t = g + th for small t, and let u t be the unique positive solution of gt u t + n 2 4(n 1) R(g t)u t = 0 on u t = 1 on ν(u t ) = 0 on 0. (The difference with the result in [15] is the minimal boundary on which we are prescribing Neumann boundary data here.) Existence of such a u t follows from the proof of (3.4) in [21, Lemma 3.2]. That u t is differentiable with respect to t near t = 0 follows as in n 2 [21, pp ]. Consider the metric ḡ t := u 4 t g t. Then ḡ t competes with g for least H and hence 0 = d dt t=0 H t = 2(n 1) n 2 d dt t=0 ν(u t ).

9 Extension of a theorem of Shi and Tam Using the fact ν(u t ) = 0on 0, integrating by parts and using the variation formula for scalar curvature dt d t=0 R(g t ) = tr(h) + div(divh) (h, Ric) this implies that d 0 = dt t=0 gt u t = n 2 tr(h) + div(divh) (h, Ric) 4(n 1) = n 2 (h, Ric). 4(n 1) Since this is true for all directions h as above, this implies that Ric 0, as desired. Finally, we need the following lemma, which is a slight extension of [9, Proposition 1]. Lemma 5 Let (, g) be a smooth compact connected Riemannian manifold of dimension n 2 with smooth boundary = 0. We assume that 0 is either empty or else has non-negative mean curvature with respect to the outward pointing unit normal. We assume that is non-empty and connected, and that there exists an isometric embedding ι : R n such that ˆ := ι() is a closed embedded hypersurface of R n, and such that H ι Ĥ. Here, H and Ĥ denote the respective mean curvatures of and ˆ R n with respect to the outward normals. If (, g) has non-negative Ricci curvature, then 0 = and (, g) is isometric to the (compact) domain bounded by ˆ in R n. Proof We only discuss the parts of the proof which differ slightly from [9]. Consider the harmonic functions X i : R with boundary values ι (x i ) on, where (x 1,...,x n ) are coordinates on R n, and boundary value 0 on 0. As in [9], one computes using Reilly s Bôchner formula that n n 2 X i H 0 ν(x i ) 2 H { 1 2 (dx(ν) ˆν) + dx(ν) 2} { [1 ] 2 [ H dx(ν) ˆν + dx(ν) 2 (dx(ν) ˆν) 2]} 0 where the map X : R n is defined as X = (X 1,...,X n ), where dx is its tangent map, and where ˆν is the unit normal vector pointing out of ˆ R n. Since it is assumed that H 0 0 it follows as in [9]that 2 X i = 0 on, i = 1,...,n (15) and, since H Ĥ is greater than zero at least at one point, dx(ν) ˆν =1, dx(ν) 2 = (dx(ν) ˆν) 2 at some point p, (16)

10 M. Eichmair et al. which implies readily that g n i, j=1 δ ij dx i dx j X (δ ij ) on. Note that since n 2 and X i 0 = 0, this is only possible if 0 =. In that case, it follows that X : (, g) (R n,δ)is a local isometry. In particular, a collar neighbourhood of = is isometric to a collar neighbourhood of ι() = ˆ R n so that can be glued smoothly to R n. Since is flat, it follows that is a domain in R n. We are now in a position to prove the rigidity statement in Proposition 1 without the spin assumption: Suppose that i H = ˆ i Ĥ for some i {1,...,p}. If p 2, let i be adomainasinlemma1 such that i = 0 i where 0 is a minimal surface. We can use Lemma 2 to justify the use of Lemmas 3 and 4 to conclude that H = Ĥ on i, R(g) 0on i, and in fact Ric(g) 0on i. By Lemma 5 we conclude that 0 =. So there could only have been one boundary component 1 in the first place. Then Lemmas 4, 5 show that (, g) is isometric to a domain in R n. Remark 1 In the above approach, we followed the idea in [21, Sect. 3] to show that, if equality in (1) holds, then must have a single boundary component. On the other hand, the presence of a minimal surface in Lemma 1 also suggests that one can apply the higher dimensional Riemannian Penrose Inequality [4] to directly prove that the strict inequality in (1) must hold whenever has more than one components. This second approach can be made rigorous by the arguments in [14, Sect. 3.2]. 5 Results of Gerhardt and Urbas To finish the proof of Theorem 2, we recall the following result on expanding star-shaped surfaces in R n into spheres, which was obtained by Gerhardt [6] and Urbas [24]. Theorem 3 (Gerhardt, Urbas) Let ˆ be a smooth, closed, compact hypersurface in R n, given by a smooth embedding ι : n 1 R n, and suppose that ˆ is star-shaped with respect to a point P 0 R n. Let Ɣ R n 1 be an open, convex, symmetric cone with vertex at the origin, which contains the positive cone Ɣ + ={(λ 1,...,λ n 1 ) λ i > 0, i}. Let f C (Ɣ) C 0 ( Ɣ) be a symmetric function satisfying (1) f is homogeneous of degree one f (2) λ i > 0 and f is concave (3) f > 0 on Ɣ and f 0 on Ɣ. Suppose f (κ 1 (σ ),..., κ n 1 (σ )) > 0, where {κ i } n 1 are the principal curvatures of ˆ at σ ˆ. Then the initial value problem { F t = f (σ,t) 1 ν(σ,t) F(σ, 0) = ι(σ) (17) has a unique smooth solution F : n 1 [0, ) R n, where ν(σ,t) is the outer unit normal vector field to F(, t) at F(σ, t), and f (σ, t) = f (κ 1,...,κ n 1 ), where {κ i } are the principal curvatures of F(, t) at F(σ, t). Moreover, for each t, F(, t) is a star-shaped surface with respect to P 0, and e t F(, t) converges to a sphere centered at P 0 in the C topology as t.

11 Extension of a theorem of Shi and Tam Now suppose ˆ = ι() R n is a smooth closed embedded hypersurface, star-shaped and such that ( n 1 n 1 ) 2 n 1 H = κ i > 0, 2K = κ i κi 2 > 0, (18) i.e., the surface has positive mean and scalar curvatures. Let Ɣ R n 1 be the open, convex, connected symmetric cone ( Ɣ = (λ n 1 n 1 ) 2 n 1 1,...,λ n 1 ) λ i > 0and λ i λi 2 > 0. (19) On Ɣ, let f be the smooth, positive function defined by f (λ 1,...,λ n 1 ) = ( n 1 λ i ) 2 n 1 λ2 i n 1 λ i. (20) It is easily seen that f can be continuously extended to Ɣ such that f 0on Ɣ. We leave it as an entertaining exercise for the reader to verify that f satisfies the conditions in Theorem 3. (See the introduction of [24] for reference on the application of the theorem for symmetric functions of the principle curvatures.) It follows that there exists a smooth map F : [0, ) R n such that F(, 0) = ˆ, F t = ( n 1 κ i n 1 κ i ) 2 n 1 κ2 i ν = H R ν (21) where ν is the outer unit normal to the closed hypersurface F( ˆ,t), {κ i } are its principal curvatures, H its mean curvature, and R its scalar curvature. Moreover, e t F(, t) converges to a sphere in the C topology as t. In particular it follows that F(, t) has positive mean and scalar curvatures throughout the evolution, and that it is a strictly convex hypersurface for t sufficiently large. Remark 2 An obvious approach to proving Theorem 2 is to use this H R -flow of Gerhardt and Urbas to connect ˆ i to round spheres at infinity and to carry out the program of Shi and Tam along this foliation without using their distance surface foliation once ˆ i is deformed to a strictly convex set. While this approach might have some independent interest, it would require detailed knowledge of the asymptotics of the H R -flow to ensure that the resulting metric is indeed asymptotically flat, and that the Brown York mass of the leaves converges to its mass. Acknowledgements Michael Eichmair was partially supported by Australian Research Council Discovery Grant #DP and by the NSF grant DMS Pengzi Miao was partially supported by Australian Research Council Discovery Grant #DP Xiaodong Wang was partially supported by the NSF grant DMS References 1. Arnowitt, R., Deser, S., Misner, C.W.: Coordinate invariance and energy expressions in general relativity. Phys. Rev. 122(2), (1961) 2. Bartnik, R.: Quasi-spherical metrics and prescribed scalar curvature. J. Differ. Geom. 37(1), (1993)

12 M. Eichmair et al. 3. Bray, H.: Proof of the Riemannian Penrose inequality using the positive mass theorem. J. Differ. Geom. 59, (2001) 4. Bray, H., Lee, D.: On the Riemannian Penrose inequality in dimensions less than eight. Duke Math. J. 148(1), (2009) 5. Brown, J., York, J.: Quasilocal energy and conserved charges derived from the gravitational action. Phys. Rev. D (3) 47(4), (1993) 6. Gerhardt, C.: Flow of nonconvex hypersurfaces into spheres. J. Differ. Geom. 32, (1990) 7. Gilbarg, D., Trudinger, N.S.: Elliptic partial differential equations of second order, 2nd edn. Springer- Verlag, New York (1983) 8. Han, Q., Lin, F.: Elliptic partial differential equations. Courant Lecture Notes in Mathematics, vol. 1, (1997) MR (2001d:35035) 9. Hang, F., Wang, X.: A note on a conjecture of Schroeder and Strake. Pacific Math. J. 232, (2007) 10. Huisken, G., Ilmanen, T.: The invers mean curvature flow and the Riemannian Penrose Inequality. J. Differ. Geom. 59, (2001) 11. Liu, C.-C.M., Yau, S.-T.: Positivity of quasilocal mass. Phys. Rev. Lett. 90(23), (2003) 12. Liu, C.-C.M., Yau, S.-T.: Positivity of quasilocal mass II. J. Amer. Math. Soc. 19(1), (2006) 13. Miao, P.: Positive mass theorem on manifolds admitting corners along a hypersurface. Adv. Theor. Math. Phys. 6(6), (2003) 14. Miao, P.: On a localized Riemannian Penrose inequality. Commun. Math. Phys. 292(1), (2009) 15. Miao, P., Shi, Y.-G., Tam, L.-F.: On geometric problems related to Brown York and Liu-Yau quasilocal mass. Comm. Math. Phys. 298(2), (2010) 16. Neves, A.: Insufficient convergence of inverse mean curvature flow on asymptotically hyperbolic manifolds. J. Differ. Geom. 84(1), (2010) 17. Nirenberg, L.: The Weyl and Minkowski problems in differential geoemtry in the large. Comm. Pure Appl. Math. 6, (1953) 18. Pogorelov, A.: Deformation of convex surfaces (Russian). Gosudarstv. Izdat. Tehn.-Teor. Lit., Moscow- Leningrad (1951) 19. Reilly, R.: Applications of the Hessian operator in a Riemannian manifold. Indiana Univ. Math. J. 26(3), (1977) 20. Schoen, R.: Variational Theory for the Total Scalar Curvature Functional for Riemannian Metrics and Related Topics. Topics in the Calculus of Variations. Lecture Notes in Mathematics, vol. 1365, pp Springer-Verlag, Berline (1987) 21. Schoen, R., Yau, S.-T.: On the proof of the positive mass conjecture in general relativity. Comm. Math. Phys. 65, (1979) 22. Shi, Y.-G., Tam, L.-F.: Positive mass theorem and the boundary behaviors of compact manifolds with nonnegative scalar curvature. J. Differ. Geom. 62, (2002) 23. Smith, B., Weinstein, G.: Quasiconvex foliations and asymptotically flat metrics of non-negative scalar curvature. Comm. Anal. Geom. 12(3), (2004) 24. Urbas, J.: On the expansion of starshaped hypersurfaces by symmetric functions of their principal curvatures. Math. Z. 205(3), (1990) 25. Wang, M.-T., Yau, S.-T.: A generalization of Liu-Yau s quasi-local mass. Comm. Anal. Geom. 15(2), (2007) 26. Witten, E.: A new proof of the positive energy theorem. Comm. Math. Phys. 80, (1981)