HESSIAN MEASURES III. Centre for Mathematics and Its Applications Australian National University Canberra, ACT 0200 Australia

Transcription

1 HESSIAN MEASURES III Neil S. Trudinger Xu-Jia Wang Centre for Mathematics and Its Applications Australian National University Canberra, ACT 0200 Australia 1

2 HESSIAN MEASURES III Neil S. Trudinger Xu-Jia Wang Centre for Mathematics and Its Applications Australian National University Canberra, ACT 0200 Australia address: Running title: Hessian measures III. Keywords: Hessian measures, Hessian capacities, weak convergence, Dirichlet problem. AMS mathematics subject classification: 58C35, 28A33, 35J60, 31B15. 2

3 HESSIAN MEASURES III Neil S. Trudinger Xu-Jia Wang Centre for Mathematics and Its Applications Australian National University Canberra, ACT 0200 Australia Abstract. In this paper, we continue previous investigations into the theory of Hessian measures. We extend our weak continuity result to the case of mixed k-hessian measures associated with k-tuples of k-convex functions, on domains in Euclidean n-space, k = 1, 2,, n. Applications are given to capacity, quasicontinuity, and the Dirichlet problem, with inhomogeneous terms, continuous with respect to capacity or combinations of Dirac measures. 1. Introduction In our previous papers [7, 8] we introduced the notion of k-hessian measure, associated to certain upper semi-continuous functions through the k-hessian operators, F k, and proved the weak convergence of Hessian measures with respect to the pointwise convergence of functions. For k = 1,, n and u C 2 (), the k-hessian operator, F k, is defined by F k [u] = S k (λ(d 2 u)), (1.1) where λ = (λ 1,, λ n ) denotes the eigenvalues of the Hessian matrix of second derivatives, D 2 u, and S k is the k th elementary symmetric function in R n, given by S k (λ) = i 1 < <i k λ i1 λ ik. (1.2) That is, F k [u] = [D 2 u] k, the sum of the k k principal minors of D 2 u, which may also be called the k-trace of D 2 u. Associated with F k we introduce the notion of k-convexity, (or k-subharmonicity, alternatively). An upper semi-continuous function u : [, ) is called k-convex in if it is subharmonic with respect to the operator F k, that is, F k [q] 0 for all quadratic polynomials q for which the difference u q has a finite local maximum in. We will also call a k-convex function proper if it does not assume the value identically on Research supported by Australian Research Council 3

4 any component of and denote the class of proper k-convex functions in by Φ k (). A function u C 2 () is k-convex if and only if λ(d 2 u) lies in the convex cone Γ k = {λ R n Sj (λ) 0, j = 1,, k}. (1.3) A basic property of k-convex functions is that if u Φ k (), then u L 1 loc () and the mollification u h = u ρ h is k-convex in h, where h = {x dist(x, ) > h}; see [8]. In this paper, we introduce mixed Hessian measures corresponding to mixed Hessian operators, F k, determined by the polarized form S k of S k, and prove their weak continuity with respect to pointwise convergence (or equivalently local L 1 convergence) in Theorem 2.4. Our approach follows that in [8], with the crucial local gradient estimates being extended to the mixed case in Lemma 2.1. Using the multilinearity of mixed measures, we are able to define signed Hessian measures associated with differences of k-convex functions. In Section 3, we introduce a notion of k-hessian capacity and using our convergence results from Section 2, we prove the corresponding quasicontinuity of k-convex functions, Theorem 3.2, together with an improvement of earlier monotonicity results for continuous functions, Theorem 3.3. A further notion of capacity is also introduced, yielding a further monotonicity result, Theorem 3.4. In the final section, we consider comparison and uniqueness results for the Dirichlet problem, µ k [u] = ν in, (1.4) u = ϕ on, where ν is a non-negative Borel measure and ϕ a continuous function on. A comparison principle is established for measures ν which are continuous with respect to capacity, Theorem 4.1. Finally we prove a uniqueness result for Dirac measures ν, Theorem 4.5, or more general measures ν which are finite combinations of measures, continuous with respect to capacity, and Dirac measures, Theorem Mixed Hessian measures Let S k : (R n ) k R be the polarized form of the k-homogeneous polynomial S k. It is uniquely characterized by being linear in each argument, λ i R n, i = 1,, k, invariant under permutation of λ i, and satisfying S k (λ,, λ) = S k (λ) (2.1) for λ R n. Explicitly, we have the formula, S k (λ 1,, λ k ) = 1 λ 1 i k! 1 λ k i k. (2.2) i 1 =i k 4

5 A fundamental inequality of Garding [3], k S k (λ 1,, λ k [ ) Sk (λ s ) ] 1/k s=1 (2.3) for λ 1,, λ k Γ k, guarantees S k 0 on (Γ k ) k. For u 1,, u k C 2 (), we introduce the mixed k-hessian operator: F k [u 1,, u k ] = S k (λ(d 2 u 1 ),, λ(d 2 u k )). (2.4) From the above properties of S k, we see immediately that F k is linear in each u s C 2 (), s = 1,, k, invariant under permutations and F k [u,, u] = F k [u] (2.5) for any u C 2 (). Moreover, we have the explicit representation F k [u 1,, u k ] = 1 k! δ i 1,,i k j 1,,j k u 1 i 1 j 1 u k i k j k, (2.6) where u s ij = D iju s, i, j = 1,, n, s = 1,, k, and δ i 1,,i k j 1,,j k denotes the generalized Kronecker delta, which vanishes if (i 1,, i k ) (j 1,, j k ) and equals ±1 according to whether (i 1,, i k ) is an even or odd permutation of (j 1,, j k ). Let k [u1,, u k 1 ] = u k ij S k [λ(d 2 u 1 ),, λ(d 2 u k )] (2.7) be the coefficient of u k ij in (2.6). Then we have k [u,, u] = 1 k F ij [u], (2.8) k where It is easy to check that F ij k [u] = u ij S k [λ(d 2 u)]. (2.9) Hence we may write F k in the divergence form D i F ij k [u1,, u k 1 ] = 0, j = 1,, n. (2.10) F k [u 1,, u k ] = u k ij F ij k [u1,, u k 1 ] = ij D i { F k [u1,, u k 1 ]D j u k }. (2.11) 5

6 By (2.3) we have F k [u 1,, u k ] for u 1,, u k Φ k () C 2 (), while the matrix k s=1 for u 1,, u k 1 Φ k () C 2 (), with trace given by for k 2. F 1/k k [u s ] (2.12) [ F ij k [u1,, u k 1 ] ] 0 (2.13) F k ii [u 1,, u k 1 ] = n k + 1 F k 1 [u 1,, u k 1 ] (2.14) k The local estimates, Theorems 3.1, 4.1, 4.3 in our previous paper [8] are readily extended to mixed Hessian operators. Lemma 2.1. Let u 1,, u k Φ k () C 2 () satisfy u i 0 in, ui 1, i = 1,, k. Then we have the estimates Fk [u 1,, u k ] C, (2.15) u k p Fl [u 1,, u l ] C, (2.16) Du k q Fl [u 1,, u l ] C, (2.17) for any subdomain, l = 1,, k, and exponents p, q 0 satisfying p < n(k l) n 2k if 2k < n, q < n(k l) n k if k < n, where C is a constant depending on,, n, k, and l, p, q as appropriate. Proof. Setting v = u i, by linearity and Garding s inequality (2.3), we have F k [u 1,, u k ] F k [v,, v] = F k [v] and u k p Fl [u 1,, u l ] v p F l [v] and hence (2.15), (2.16) follow immediately from the corresponding inequalities, (3.1), (4.24) in [8]. To obtain (2.17), we write w = l i=1 u i, w k = w + u k, so that Du k q Fl [u 1,, u l ] ( Dw + Dw k ) q Fl [u 1,, u l ] } 2 { Dw q q Fl [w,, w] + Dw k q Fl [w k,, w k ] 6

7 and again (2.17) follows from the corresponding inequality (4.1) in [8]. Remark 2.1. Letting η 0, C 1 0(), we have by (2.11), (2.14), η F k [u 1,, u k ] = n k + 1 k k [u1,, u k 1 ]D i ηd j u k F k 1 [u 1,, u k 1 ] Dη Du k, (2.18) so that (2.15) follows from (2.17). By the proof of Theorem 4.3 in [8], (2.16) is also a consequence of (2.17). Accordingly Lemma 2.1 can be derived directly from Theorem 4.1 in [8], whose proof is purely local, thereby avoiding the global existence theory as used in our proof of Theorem 3.1 in [8]. With the help of (2.15)-(2.17) we can prove Lemma 2.2. Let B r = B r (0) B R (0) = B R be concentric balls. Let w 1,, w k 1, v 1, v 2 0, C 2 (B R ) Φ k (B R ). Suppose w s L 1 (B R ) 1, v i L 1 (B R ) 1, s = 1,, k 1, i = 1, 2. Then for any ε > 0, there exists δ > 0 depending only on r, R, n, k, and ε, such that if v 1 v 2 L 1 (B R ) < δ, B r Fk 1 [w 1,, w k 1 ] v 1 v 2 ε. (2.19) Proof. Let A ε = {x B (R+r)/2 v 1 (x) v 2 (x) > ε}. Then A ε 0 as δ 0. We have, in view of (2.15), (v 1 v 2 ) + Fk 1 [w 1,, w k 1 ] (v 1 v 2 ε) + Fk 1 + 2ε B r B r (v 1 v 2 ε) + Fk 1 + Cε B r Let ζ 0, C 2 0() be a cut-off function, with r < r < (R + r)/2 and ζ 1 on B r. Setting z = (v 1 v 2 ε) +, we then have, for k > 1, z F k 1 [w 1,, w k 1 ] ζz F k 1 [w 1,, w k 1 ] B r ij = ζz F k 1 [w1,, w k 2 ]D ij w k 1 = k 1 D i(ζz)d j w k 1 ( A ε A ε k 1 D iw k 1 D j w k 1 ) 1/2( 7 B r Fk 1 k 1 D i(ζz)d j (ζz)) 1/2.

8 By Hölder s inequality, A ε ( k 1 D iw k 1 D j w k 1 C F k 2 Dw k 1 q ) 2/q ( A ε F k 2 ) 1 2/q for 2 < q < 2n n k. By (2.17) we have F k 2 [w 1,, w k 2 ] Dw k 1 q C. By induction we suppose that (2.19) holds when k is replaced by k 1. It then follows that when δ is small enough, A ε with ε 1 0 as δ 0. Next we estimate = 2 2 = 2 k 1 [w1,, w k 2 ]D i (ζz)d j (ζz) F k 2 [w 1,, w k 2 ] < ε 1 k 1 D i[ζ(v 1 v 2 ε)]d j [ζ(v 1 v 2 ε)] k 1 D i[(v 1 v 2 )ζ]d j [(v 1 v 2 )ζ] + 2ε 2 k 1 D i[(v 1 v 2 )ζ]d j [(v 1 v 2 )ζ] + Cε 2 ζ 2 (v 1 v 2 ) k 1 D iζd j ζ ij F k 1 D ij(v 1 v 2 ) + 2 (v 1 v 2 ) 2 ij F k 1 D iζd j ζ + Cε 2. The first integral on the right hand side can be estimated as follows, ζ 2 (v 1 v 2 ) k 1 D ij(v 1 v 2 ) F ij v 1 + v 2 F k 1 [w 1,, w k 2, v 1 + v 2 ], which is bounded by (2.16). To control the second integral, we observe that (v 1 v 2 ) 2 ij F k 1 D iζd j ζ C C F k 2 [w 1,, w k 2 ] v 1 v 2 2 v 1 + v 2 2 Fk 2 [w 1,, w k 2 ] C, by virtue of (2.16). Combining the above estimates and interchanging v 1 and v 2 we obtain (2.19). 8

9 Lemma 2.3. Let {u 1 m},, {u k m} C 2 () Φ k () converge to u 1,, u k in L 1 loc (). Then F k [u 1 m,, u k m] converges weakly to a Borel measure µ in. Proof. We may assume without loss of generality that u s m 0 in, s = 1,, k. Let us fix concentric balls B r = B r (y) B R (y) = B R. Let 0 < r < ρ < R h 0 and fix a function η C 2 0(B ρ ). Then for l, m = 1, 2,, and w s = w s t = tv s l + (1 t)v s m, 0 t 1, we have, by integration by parts, η ( Fk [vl 1,, vl k ] F k [vm, 1, vm] k ) 1 = = 0 1 C 0 1 dt η B R dt 0 B R dt k s=1 k [w1 t,, w s 1 t, w s+1 t,, w k t ]D ij (v s l v s m) k (vl s vm) s s=1 B ρ k s=1 F ij k [w1 t,, w s 1 Therefore Lemma 2.3 follows from Lemma 2.2. t, wt s+1,, wt k ]D ij η F k 1 [w 1 t,, w s 1 t, w s+1 t,, w k t ] v s l v s m From Lemma 2.3 we may define the mixed k-hessian measure associated with u 1,, u k Φ k () by ηd µ k [u 1,, u k ] = lim η F k [u 1 h 0 h,, u k h] for any η C0 (), where u h is the mollification of u. Therefore we obtain Theorem 2.4. For any u 1,, u k Φ k (), there exists a Borel measure µ k [u 1,, u k ] in such that µ k [u 1,, u k ] = F k [u 1,, u k ] when u 1,, u k C 2 (), and if {u 1 m},, {u k m} are sequences in Φ k () converging locally in measure to u 1,, u k, respectively, then µ k [u 1 m,, u k m] converges to µ k [u 1,, u k ] weakly. From Lemma 2.2 we also have u 1 m µ k 1 [u 2 m,, u k m] u 1 µ k 1 [u 2,, u k ] weakly as measures, provided u s m, s = 1,, k, converges to u s in L 1 loc (). Remark 2.2. Since the mixed Hessian operator F k [u 1,, u k ] is invariant under permutation, so is the mixed Hessian measure µ k [u 1,, u k ]. Furthermore, the mixed Hessian measures are additive, that is µ k [u 1,, u k 1, v 1 + v 2 ] = 9 2 µ k [u 1,, u k 1, v i ] (2.20) i=1

10 for u 1,, u k 1, v 1, v 2 Φ k (). Therefore we can introduce, for any w L 1 loc () which can be decomposed as with w 1, w 2 Φ k (), a signed mixed Hessian measure w = w 1 w 2 (2.21) µ k [u 1,, u k 1, w] = µ k [u 1,, u k 1, w 1 ] µ k [u 1,, u k 1, w 2 ]. (2.22) From (2.20) we see that the signed mixed Hessian measure is independent of the decomposition (2.21). For any w 1,, w k L 1 () such that w s = w s 1 w s 2 with w s i Φk (), we can similarly define the mixed Hessian measure µ k [w 1,, w k ] by the expansion (2.22). In particular, the mixed Hessian measures can be extended to semi-k-convex functions, as defined in [7]. Remark 2.3. Not only can F k [u 1,, u k ] be extended as a Borel measure for u 1,, u k Φ k ij (), but also the coefficients F k [u1,, u k 1 ]. To see this we fix u k (x) = x i x j and define, for any u 1,, u k 1 Φ(), a signed measure Then for any smooth function u k, we have µ ij k [u1,, u k 1 ] = µ k [u 1,, u k ]. µ k [u 1,, u k ] = µ ij k [u1,, u k 1 ]u k ij. (2.23) The weak convergence result, Theorem 2.4, thus also holds for µ ij k [u1,, u k 1 ]. When the sequences {u 1 m},, {u k m} are bounded and monotone, the proof of Theorem 2.4 can be simplified by following the plurisubharmonic case [1,4]. We will use this approach to obtain a further convergence result, relevant to our treatment of capacity in the next section. First we note a couple of monotonicity results, corresponding to Lemmas 2.1 and 5.2 in [7]. Namely, if u 0, u 1,, u k 1, u k 1, u k 2 C 2 () Φ k (), and u k 1 u k 2 in, u k 1 = u k 2 on, then we have the inequalities µ k [u 1,, u k 1]() µ k [u 1,, u k 2](), (2.24) u 0 d µ k [u 1,, u k 1]() u 0 d µ k [u 1,, u k 2](). (2.25) To prove (2.24), we have by (2.11), { F k [u 1,, u k 1] F k [u 1,, u k 2]} = 10 k [u1,, u k 1 ]γ i D j (u k 1 u k 2) 0,

11 where γ denotes the unit outer normal to. Consequently, we may assume u 0 0 in (2.25), whence we have u 0 { F k [u 1,, u k 1] F k [u 1,, u k 2]} = 0 (u k 1 u k 2) F k [u 1,, u k 1, u 0 ] and hence (2.25) follows. u 0 F ij k [u1,, u k 1 ]γ i D j (u k 1 u k 2) Lemma 2.5. Let {u 0 m}, {u 1 m},, {u k m} be sequences in Φ k () converging decreasingly to functions u 0, u 1,, u k Φ k (). Suppose u 0 L (), µ k [u 1,, u k ]() <, and u j m, m = 1, 2,, coincide outside a compact subset K of for each j = 0, 1,, k. Then u 0 md µ k [u 1 m,, u k m] u 0 d µ[u 1,, u k ]. (2.26) Proof. We first prove lim sup m u 0 md µ k [u 1 m,, u k m] u 0 d µ k [u 1,, u k ] =: L. (2.27) Indeed, if (2.27) is not true, there exist subsequences of {u 0 m},, {u k m} such that u 0 md µ k [u 1 m,, u k m] L + ε. Since {u 0 m} are decreasing, we have, for l < m, u 0 l d µ k [u 1 m,, u k m] L + ε. By Theorem 2.4 we have µ k [u 1 m,, u k m] µ k [u 1,, u k ] as measures. upper semi-continuity of u 0 l, we have, by sending m, u 0 l d µ k [u 1,, u k ] L + ε. Hence by the We reach a contradiction as the above integral converges to L as l. To complete the proof of Lemma 2.5, by mollification and successive application of (2.25), we obtain for fixed m, u 0 md µ k [u 1 m,, u k m] u 0 md µ m [u 1,, u k ] u 0 d µ k [u 1,, u k ] 11

12 and hence (2.26) follows from (2.27). Remark 2.4. The condition that u i m coincide near may be replaced by being uniformly smooth near and coinciding on, i = 0, 1,, k. For functions u 1,, u k Φ k () and u 0 locally integrable with respect to µ k [u 1,, u k ], we may define a Borel measure L[u 0, u 1,, u k ] := u 0 d µ k [u 1,, u k ]. (2.28) From Lemma 2.5, we then obtain a further weak convergence result. Theorem 2.6. Let {u 0 m}, {u 1 m},, {u k m} be sequences in Φ k () converging decreasingly to functions u 0,, u k Φ(), respectively. Then, if u 0 L loc (), the sequence of measures L[u 0 m, u 1 m,, u k m] converges weakly to L[u 0, u 1,, u k ]. Proof. Let B = B R be a ball of radius R and centre y in and B r the concentric ball of radius r < R. For u Φ k () and h > 0, we let u h be the mollification of u and construct ũ h Φ k () satisfying ũ h = u h in B r together with µ k [ũ h ] = 0 in B R B r, (2.29) ũ h = u h on B r, ũ h = 0 on B R, where we may suppose ũ h 1 in B by subtracting a linear function. Sending h 0, we obtain ũ 0 := lim h 0 ũ h Φ k () C 0,1 (B R B r ). Let ψ = C( x y 2 R 2 ), where C > 0 is chosen so that ψ < ũ on B (R+r)/2. Let ω be the component of {ũ 0 < ψ} which contains B r, Define Applying Lemma 2.5, we then obtain { ũ0 in B R ω, ũ = ψ in ω ũ 0 md µ k [ũ 1 m,, ũ k m] ũ 0 d µ k [ũ 1,, ũ k ]. The result then follows by replacement of in (2.27) by compact K B, in the first part of the proof of Lemma

13 3. Hessian capacities The weak convergence result, Theorem 2.4, is a powerful tool in developing a potential theory for Hessian operators. In this section we introduce a notion of Hessian capacity and prove the quasicontinuity of k-convex functions. We follow the treatment of pluripotential theory in [1]. It is not hard to see that the results in [1] can be extended to Hessian equations since Hessian equations have a similar integral structure to complex Monge-Ampère equations and our weak convergence result is stronger than that for plurisubharmonic functions in [1]. We refer to [5] for further discussion in this direction. First we introduce a capacity. Let be a bounded domain in R n and E a Borel set. We define the k-hessian capacity by Cap k (E) = Cap k (E, ) = sup{µ k [u](e) u Φ k (), 0 u 1}. (3.1) The Hessian capacity satisfies the following properties: (i) for any E 1, E 2, Cap k (E 1 E 2 ) Cap k (E 1 ) + Cap k (E 2 ); (ii) if E 1 E 2 and 1 2, then Cap k (E 1, 1 ) Cap k (E 2, 2 ); and (iii) if E 1 E 2 are Borel subsets of, then Cap k ( m E m ) = lim m Cap k (E m ). Indeed, (i) and (ii) are obvious. To see (iii), for ε > 0 let u Φ k (), 0 u 1, be chosen such that Cap k (E) µ k [u](e)+ε, where E = m E m. Since lim m Cap k (E m ) lim m µ k [u](e m ), we have Cap k (E) lim m Cap k (E m ). The reverse inequality follows from (ii). Lemma 3.1. Let {u m } C 2 () Φ k () converge decreasingly to u Φ k (). Then for any ε, δ > 0, lim Cap ( k {x δ um (x) > u(x) + ε} ) = 0. (3.2) m Proof. There is no loss of generality in assuming ε = 1, u m 1 in, and replacing by a sub-ball, B = B R of radius R and centre y. First we suppose {u m } is uniformly bounded. Replacing u m and u by max{u m, ψ} and max{u, ψ}, where ψ = C( x y 2 R 2 ), with C chosen so that ψ < inf x u m (x) on δ, we can suppose u m and u coincide near, with u m = u = 0 on. Let O m = {u m > u + 1}. For v C 2 () Φ k () satisfying 0 v 1, we estimate F k [v] (u m u) F k [v] O m O m (u m u) F k [v] = (u m u) i v j F ij k [v] [ ] 1/2[ ] 1/2. (u m u) i (u m u) j F ij k [v] v i v j F ij k [v] {ψ< 1} 13 (3.3)

14 The last integral on the RHS is bounded. To estimate the first integral on the RHS we integrate by parts and obtain (u m u) i (u m u) j F ij k [v] = (u m u)dµ k [v,, v, u m u] (u m u)dµ k [v,, v, u m + u] (3.4) Repeating the argument we finally reach [ µ k [v] C (u m u)f k [u m + u] O m ] 1/2 n. (3.5) The last integral converges to zero by Lemma 2.5. Hence Lemma 3.1 holds for bounded functions. In the unbounded case we need only to prove that lim Cap k(e m,n ) 0 N uniformly in m, where E m,n = {u m < N}. To see this, we remark that for any closed set K, Cap k (K) µ k [w]() (3.6) for any function w Φ k () vanishing continuously on and satisfying w < 1 in K. Indeed, for any u Φ k () such that 0 > u 1, let ũ = max(u, w). Then K {w < 1} and we have µ k [u](k) = µ k [ũ](k) µ k [ũ](). Noting that ũ = w on and ũ w in, we have µ k [ũ]() µ k [w](). Hence (3.6) holds. By (3.6) and (iii) above, we have Cap k (E m,n ) µ k [N 1 u m ]() N k µ k [u m ]() from our modification of u m near. Lemma 3.1 is proved. CN k u m k L 1 () (3.7) Theorem 3.2. Let be a bounded domain and u Φ k (). Then for any ε > 0, there exists an open subset O with Cap k (O) ε such that u is continuous on O. Proof. We may suppose that is the unit ball and u is smooth near. Let {u m } be a sequence of smooth k-convex functions converging decreasingly to u. By Lemma 3.1, there exists m j large enough such that Cap k (O j, ) < 2 j, 14

15 where O j = {x u mj (x) > u 1/j}. Let G s = j>s O j. Then u m converges to u uniformly in \G s, and Cap k (G s, ) j>s Cap k (O j, ) 2 s. Hence Theorem 3.2 holds. We say that a measure µ is continuous with respect to capacity if for any ε > 0, there is δ > 0 such that for any open set E with Cap k (E, ) < δ, we have µ(e) < ε. If follows that if µ k [u] is continuous with respect to capacity, then for any ε > 0, there is an open set O with µ k [u](o) < ε such that the restriction of u on O is continuous. Obviously µ k [u] is continuous with respect to capacity if it is locally integrable. It is easy to see that µ k [u] is continuous with respect to capacity if and only if µ k [u]({u < t}) 0 as t +. It follows that µ k [u] is continuous with respect to capacity if u Φ k () is bounded. Moreover, if µ k [u] is continuous with respect to capacity, and if v u, v Φ k (), then µ k [v] is also continuous with respect to capacity. We point out that a nonnegative measure µ is continuous with respect to capacity if there exists ε > 0 such that µ(b r (x)) Cr n 2k+ε (3.8) whenever B r (x). Indeed, for any set E with zero capacity, the Hausdorff dimension of E is not larger than n 2k, i.e., H n 2k+ε 1 (E) = 0 for any ε 1 > 0; see [5]. Hence µ(e) = 0. Theorem 3.3. Suppose u, v Φ k (), u = v = ϕ continuously on. If µ k [u] and µ k [v] are continuous with respect to capacity, then µ k [u]({u < v}) µ k [v]({u < v}). (3.9) Proof. First we show that (3.9) holds when u, v are bounded functions. We may assume, by replacing u by u + 2δ and letting δ 0, that u v + 2δ continuously on, (namely, lim inf x y [u(x) v(x)] 2δ). Hence the set = {u < v + δ} is relatively compact in. Let u j = ρ rj u and v j = ρ rj v be mollifications of u, v such that u j u and v j v in as r j 0. By the smoothness of u j and v j we have µ k [u m ]({u m < v l }) µ k [v l ]({u m < v l }). (3.10) Indeed, in the smooth case we may take = {u < v}. For c > 0, let v c = max(v c, u). Then v c = u near. Hence µ k [u]() = µ k [v](). Sending c 0, we obtain (3.10). For any ε > 0, let O be an open set with Cap k (O) < ε such that the restrictions of u, v on O are continuous. By Tietze s extension theorem, there is a continuous function ṽ 15

16 such that ṽ = v on O. We may suppose without loss of generality that u + v 1, so that µ k [u](o) < ε and µ k [v](o) < ε. Then µ k [v]({u m < v}) µ k [v]({u m < ṽ}) + ε lim µ k [v l ]({u m < ṽ}) + ε l lim µ k [v l ]({u m < v l }) + 2ε l since {u m < ṽ} is open and ṽ < v l on O. By (3.10) we have then lim µ k[v l ]({u m < v l }) lim µ k [u m ]({u m < v l }) l l µ k [u m ]({u m v}) since {u m v} is closed. Next we have, when m is large enough, lim µ k[u m ]({u m v}) lim µ k[u m ]({u v}) m m lim m µ k[u m ]({u v} O) + ε µ k [u]({u v} O) + ε µ k [u]({u v}) + 2ε since {u v} O is closed. Note that µ k [v]({u m < v}) µ k [v]({u < v}) as m. We therefore obtain µ k [v]({u < v}) µ k [u]({u v}). Replace u by u + c for some constant c > 0 and notice that {u + c v} {u < v} and {u + c < v} {u < v}, as c 0, we obtain (3.9) for bounded functions. In the unbounded case, we observe that for any ε > 0, we may take O = {u < t} {v < t}, where t > 1 is chosen large such that µ k [u](o), µ k [v](o) < ε. The above proof is still applicable. We introduce another capacity for k-convex functions. Let be a bounded domain in R n and E a Borel set. Let cap k (E) = cap k (E, ) = sup{µ k 1 [u](e) u Φ k (), u 0, and u L 1 () 1}. (3.11) Then cap k (E) satisfies the same properties (i)-(iii) above. By Theorem 5.2 in [8] it is easy to verify that for any sequence {u m } Φ k () which converges to u Φ k () in L 1 loc (), and any cut-off function ζ C 0 (), lim m sup{ ζ(u m u)dµ k 1 [v]; v Φ k (), v L1 () 1} 0. (3.12) Similar to Theorem 3.2 one can prove that k-convex functions are quasicontinuous with respect to the capacity cap k (E, ). That is, for any given u Φ k () and ε > 0, there is an open set O with cap k (O, ) < ε such that u is continuous in O. By (2.16) we see that µ k 1 [u]({u < t}) 0 as t. Hence for any u Φ k (), µ k 1 [u] is continuous with respect to the capacity Cap k (E, ). Therefore the proof of Theorem 3.3 also yields the following variant. 16

17 Theorem 3.4. Suppose u, v Φ k (), u = v = ϕ continuously on. Then µ k 1 [u]({u < v}) µ k 1 [v]({u < v}). (3.13) 4. The Dirichlet problem We consider the Dirichlet problem µ k [u] = ν in, (4.1) u = ϕ on, where is a uniformly (k 1)-convex domain, ϕ is a continuous function, and ν is a finite nonnegative Borel measure. Let ν be decomposed to ν = ν 1 + ν 2, (4.2) such that ν 1 L 1 () and ν 2 is the singular part of ν, which is supported on a set K of Lebesgue measure zero. For simplicity we suppose dist(k, ) > 0. The existence of solutions has been proved in [8]. When k > n/2, a k-convex function is Hölder continuous, with Hölder exponent α = 2 n/k. Therefore a solution of (4.1) is automatically locally Hölder continuous. In this case, the uniqueness has been proved in [7] by a comparison principle. In the following we are concerned with the case k n/2. In [5], Labutin established pointwise estimates for k-convex functions. He proved that for any nonpositive u Φ k (B 2R ) and point x B R, R/2 [ µk [u](b(x, r)) C 1 0 r n 2k ] 1/k d r r u(x) C 2 R 0 [ µk [u](b(x, r)) r n 2k ] 1/k d r r C 3 sup B R u, (4.3) where C 1, C 2, C 3 are constants depending only on k and R. From the estimate (4.3) it follows that a k-convex function is locally Hölder continuous in if and only if the Hessian measure µ k [u] satisfies, for some ε > 0, µ k [u](b r ) Cr n 2k+ε (4.4) for all, B r, where C is a constant depending on. In this section we will prove two uniqueness results for the problem (4.1) in the case k n/2. The first one, which follows from Theorem 4.1 below, asserts that the solution is unique if the measure ν is continuous with respect to capacity. Hence the uniqueness holds when ν is integrable. The second uniqueness result is for the case when ν is a Dirac measure. Our proof also applies to certain quasilinear divergence structure operators, such as the p-laplacian operators, leading to uniqueness results for their Green s functions [9]. 17

18 Theorem 4.1. Suppose u, v Φ k () are such that the measures µ k [u] and µ k [v] are continuous with respect to capacity, and u = v = ϕ continuously on. If µ k [u] µ k [v], then u v in. Proof. For if not, we replace u by u δ = u + δψ 0, where ψ 0 solves By Theorem 3.3 we have F k [ψ 0 ] = 1 in, ψ 0 = 0 on. (4.5) µ k [u]({u δ > v}) < µ k [u δ ]({u δ > v}) µ k [v]({u δ > v}). We reach a contradiction. Hence Theorem 4.1 holds. Next we prove the uniqueness of fundamental solutions. We need a few lemmas. Lemma 4.2. Suppose u and v are two solutions to (4.1). If u v in, then u = v in. Proof. For c > 0 small, let u c = max(u c, v) such that u c = v near. Let {u m } and {v m } be two sequences in Φ k () C 2 () which converge to u c and v, respectively, and such that u m = v m on. For any η C 2 () satisfying η = 0 on, we have η(f k [u m ] F k [v m ]) = 1 k = 1 k dt dt ηf ij k [w t](u m v m ) ij (u m v m )F ij k [w t]η ij, where w t = (1 t)u m + tv m. Let η be a uniformly k-convex function. Then F ij k [w t]η ij c 0 F k 1 [w t ] for some c 0 > 0 depending only on n, k, and η. Since we obtain η(f k [u m ] F k [v m ]) F k 1 [w t ] (1 t) k F k 1 [u m ] + t k F k 1 [v m ], c 0 (u m v m )(F k 1 [u m ] + F k 1 [v m ]). k(k + 1) Sending m first and then sending c 0 we obtain, by Theorems 5.1 and 5.2 in [8], 0 (u v)d(µ k 1 [u] + µ k 1 [v]). 18

19 That is, µ k 1 [u] = µ k 1 [v] = 0 on {u > v}. (4.6) If the lemma is not true, we choose ψ 0 Φ k () such that µ k [ψ 0 ] = 1 in, ψ 0 = 0 on. (4.7) Replacing u by u δ = u + δψ 0, by Theorem 3.4 we have µ k 1 [u δ ]({u δ > v}) µ k 1 [v]({u δ > v}). (4.8) Since ψ 0 < 0, we have {u δ > v} {u > v}. Hence by (4.6) we see the RHS is equal to zero, but the LHS is greater than µ k 1 [δψ 0 ], which is positive. Hence the uniqueness is proved. Remark 4.1. From Lemma 4.2 we see that to prove the uniqueness it suffices to prove that if u, v are two solutions of (4.1), then max(u, v) is a subsolution of (4.1). So far, we have succeeded only in the case when µ k [u] and µ k [v] are Dirac measures. Lemma 4.3. If there is a subsolution u of (4.1), then there exists a solution w of (4.1) which satisfies w u in. Proof. By approximation we may suppose that ν = 0 in δ whence we can assume u = ϕ on and u C 0,1 loc ( δ), by replacing u by ũ = sup{ψ Φ k () ψ u in δ and ψ ϕ on }. For any σ (0, δ n ), we decompose the domain into the union of a finitely many disjoint smooth subdomains D j with diameters σ (i.e., = j D j ) such that ν( D j ) = 0. Let u h be the mollification of u. Since u is a subsolution of (4.1), we see that α j := lim h 0 ν(d j ) µ k [u h ](D j ) 1. Let ν σ,h = j α j µ k [u h ]χ Dj, (4.9) where χ is the characteristic function. Then ν σ,h µ k [u h ]. Let w σ,h be the solution of (4.1) with ν replaced by ν σ,h. Then w σ,h is Hölder continuous and w σ,h u h. Sending h 0 we have, by choosing subsequences if necessary, w σ,h w σ in L 1 (), w σ u, and µ k [w σ ] = ν σ in, where ν σ = lim h 0 ν σ,h in the weak sense. Note that ν σ ν weakly as measures. We see that the limit w = lim σ 0 w σ satisfies µ k [w] = ν and w u in, and w = ϕ on. 19

20 Lemma 4.4. Suppose u Φ k (). Suppose µ k [u] is supported on a closed set E with E = 0. Suppose lim x y E u(x) =. Then for any v Φ k (), if u v in, we have µ k [u] µ k [v]. Proof. There is no loss of generality in assuming is uniformly (k 1)-convex. By Lemma 4.3 we may suppose u = 0 on. We can also suppose, by replacing v by ṽ = sup{ψ Φ k () ψ v in δ, ψ 0 on }, δ > 0 small, that v = 0 on. To prove Lemma 4.4 it suffices to prove that for any closed set K, µ k [u](k) µ k [v](k). By Lemma 4.3, there exists w Φ k () vanishing on such that µ k [w] = µ k [u]χ K. Hence to prove Lemma 4.3 it suffices to prove that µ k [u](e) µ k [v](e). Let u N = max{u, N} and let η = η N = 1 N u N C 0 (). By approximation we have, ηd(µ k [u] µ k [v]) = 1 where w t = (1 t)u + tv. On the LHS we have that is independent of N. Since η N 0 in E and E is closed, 0 dt (u v)η ij dµ ij k [w t] 0, (4.10) η N dµ k [u] = µ k [u](e) (4.11) lim η N dµ k [v] = µ k [v](e). (4.12) N It follows that µ k [v](e) µ k [u](e). Hence Lemma 4.4 holds. Theorem 4.5. Suppose the right hand side of equation (4.1) is a Dirac measure. Then a solution u Φ k (), assuming the boundary value ϕ continuously, is unique. Proof. Let ν = δ x0, where x 0. Suppose u, v are two solutions of (4.1). We want to prove w = max(u, v) is subsolution of (4.1). First we claim the uniqueness of solutions to the problem µ k [G] = δ x0 in B r (x 0 ), (4.13) G = 0 on B r (x 0 ). Indeed, if u is a solution of (4.13), then u is locally Lipschitz in the punctured ball, B r (x 0 ) {x 0 }. Hence the method of moving planes is applicable and we conclude that a solution to (4.13) must be a radial function. The uniqueness of radial solutions to (4.13) is obvious. 20

21 Let B r (x 0 ). By Lemma 4.3 we see that the unique solution w of (4.13) satisfies w max(u, v). Hence by Lemma 4.4, max(u, v) is a subsolution to (4.1) with ν = δ x0. We point out that the same proof actually yields the uniqueness if ν is a nonnegative Borel measure supported on a countable set. Moreover, our proof also yields the uniqueness of fundamental solutions to the equation µ k [u] = δ 0 in R n. (4.14) such that u 0 as x, where k n/2. The solution is given by [ ] 1/k 1 log x, k = ( n n k)ω n 2, u(x) = [ ] 1/k 1 1 ( n k)ω n 2 n/k x 2 n/k, k < n 2, where ω n is the volume of the unit ball B 1 (0). (4.15) We conclude this paper by proving uniqueness when ν = ν 1 + ν 2, where ν 1 is continuous with respect to capacity and ν 2 is supported on a finite number of points, i.e, ν 2 = m i=1 α iδ xi, where α i > 0. Theorem 4.6. Let ν be as above. Then the uniqueness for (4.1) holds. Proof. We will only prove the case ν 2 = δ 0. The general case can be proven in the same way. Suppose u, v are two different solutions. By Lemma 4.3, and using approximation, there exist u r, v r Φ k () such that u r = u and v r = v in B r (0), and µ k [u r ] = µ k [v r ] = δ 0 in B r (0). Observe that, since ν 1 is continuous with respect to capacity, µ k [u r ]( B r ) = µ k [v r ]( B r ) = ν 1 (B r ) 0 as r 0. Therefore if u v, we may suppose that for some δ > 0, the set E = {v < u δ } and E r = {v r < u δ r} are not empty, where u δ = (1 + δ)u + δψ, u δ r = (1 + δ)u r + δψ, and ψ is a uniformly k-convex, smooth function vanishing on. Observe that, by Theorem 4.5, u r (x) lim x 0 G(x) = lim v r (x) x 0 G(x) = 1, where G is the solution of (4.13). Hence E does not contains the origin and so by Theorem 3.3, µ k [v r ](E r ) µ k [u δ r](e r ). That is, as r 0, µ k [v](e r ) µ k [u δ ](E r ) o(1) (1 + δ) k µ k [u](e r ) + δ k µ k [ψ](e r ) o(1) µ k [v](e r ) + δ k µ k [ψ](e r ) o(1) 21

22 where µ k [ψ](e r ) C > 0. We reach a contradiction if we fix a δ > 0 small and send r 0. This completes the proof. References [1] E. Bedford and B.A. Taylor, A new capacity for plurisubharmonic functions, Acta Math. 149(1982), [2] L. Caffarelli, L. Nirenberg, and J. Spruck, The Dirichlet problem for nonlinear second order elliptic equations, III, Functions of the eigenvalues of the Hessian, Acta Math. 155(1985), [3] L. Garding, An inequality for hyperbolic polynomials, J. Math. Mech. 8(1959), [4] M. Klimek, Pluripotential theory, Oxford University Press, New York [5] D. Labutin, Potential estimates for a class of fully nonlinear elliptic equations, preprint. [6] N.S. Trudinger, Weak solutions of Hessian equations, Comm. Partial Diff. Eqns 22(1997), [7] N.S. Trudinger and X.J. Wang, Hessian measures I, Topol. Methods Nonlinear Anal. 19(1997), [8] N.S. Trudinger and X.J. Wang, Hessian measures II, Ann. Math. 150(1999), to appear. [9] N.S. Trudinger and X.J. Wang, (in preparation). 22