ON THE BACKGROUNDS OF THE THEORY OF M-HESSIAN EQUATIONS. Nina Ivochkina and Nadezda Filimonenkova

Transcription

1 COMMUNICATIONS ON doi:0.3934/cpaa PURE AND APPLIED ANALYSIS Volume 2, Number 4, July 203 pp ON THE BACKGROUNDS OF THE THEORY OF M-HESSIAN EQUATIONS Nina Ivochkina and Nadezda Filimonenkova Saint-Petersburg State University of Architecture and Civil Engineering 2-nd Krasnoarmeiskaya St. 4, St. Petersburg, Russia Abstract. The paper presents some pieces from algebra, theory of function and differential geometry, which have emerged in frames of the modern theory of fully nonlinear second order partial differential equations and revealed their interdependence. It also contains a survey of recent results on solvability of the Dirichlet problem for m-hessian equations, which actually brought out this development.. Introduction. One of the principle goals in the theory of fully nonlinear second order partial differential equations is to describe functions F, domains Ω R n, n > 2, and boundary data, which provide a solvability of the Dirichlet problem F [u] = F (x, u, u x, u xx ) = f, u = ϕ. () The modern development of this theory has been started by the famous results of L. Evans [5] and N.V.Krylov [9] confirming local Hölder continuity of second derivatives of solutions to equations () under condition of the uniform ellipticity 0 < ν F u ij ξ i ξ j µ, ξ R n, ξ = (2) and concavity either convexity of F in u xx. From then on there are two trends in the research of fully nonlinear equations. One of them takes into consideration the totally elliptic (parabolic) equations, i.e., the requirement (2) has to be satisfied for all u C 2 (Ω) with some constants ν, µ. The second treats non totally elliptic equations with F [u] = F (u x, u xx ) invariant under orthogonal transformations. This development was generated by the Monge Ampère operator F [u] = detu xx considered on the set of convex functions, which up to 975 had been treated in frames of geometric approach (see for references [22]). The first attempt to fill up the gap between Monge Ampère and Poisson equations has been presented in [], [2]. The paper of L.Caffarelli, L.Nirenberg and J.Spruck [2] contains a general approach to the problem () with F [u] = F (u xx ), where in particular the necessary for the classic solvability conditions on Ω have been discovered. At present one of the actual questions is a solvability of the Dirichlet problem in a weak sense. In this course the papers [3], [4] deal with some classes of fully nonlinear equations under condition (2). There are two approaches to solvability of the Dirichlet problem for 2000 Mathematics Subject Classification. 35J65. Key words and phrases. m-hessian equations, m-curvatures. The work is supported by the Russian Foundation for Basic Research grant No , by the State special program Scientific and scientific-pedagogical personnel of innovative Russia No. 4.B and by the Saint-Petersburg Committee on science and higher education. 687

2 688 NINA IVOCHKINA AND NADEZDA FILIMONENKOVA m-hessian equations in a weak sense and both were started by N.Trudinger. A notion of approximate solution was introduced in [26] and some ideas from there were developed in [7], [8]. Investigation of weak solutions in viscosity sense is the subject of [28], [27]. In the paper [20] the extension of this concept was carried over larger classes of non uniformly fully nonlinear elliptic equations. In fact, investigation of m-hessian equations brought out some new problems in algebra, differential geometry and theory of functions, what was somewhat subdued in pursuit of existence theorems. In our opinion these problems are worthy of a better look in order to inspire further development of the theory of fully nonlinear equations and that is what about our paper. By the backgrounds of the theory of m- Hessian equations we mean some new developments in algebra, theory of functions and differential geometry connected with m-hessian and m-th curvature operators. In this course we deal in Section 2 with the half-space of n n symmetric matrices, {S : trs > 0} and establish, in particular, orthogonal invariant stratification in there. This stratification contains a new sub-stratification of positive definite matrices, which ends with {Id}. On the basis of stratification from Section 2 we introduce in Section 3 the cones K m (Ω) C 2 (Ω) of m-admissible functions, K m (Ω) = {u C 2 (Ω) : tr i u xx > 0, i =,...m, x Ω}. (3) It turns out that these cones are invariant under standard mollification and could be expanded, for instance, on C( Ω). Section 3 contains also a version of Sard lemma for m-admissible functions. In the paper [27] there were introduced the cones K m (Ω), what implied non strict inequalities in (3), and in particular, the invariance of K m (Ω) was established, Lemma 2.3. Unlike [27] our point here is to never involve into reasoning the characters from the theory of partial differential equations but in Section 6. In Section 4 we discuss a notion of p-convexity, p n, of hypersurfaces Γ R n, which is vital in the modern theory of m-hessian and m-curvature equations. This notion should appear in frames of differential geometry but did not. Moreover, there are discrepancies in common definitions of the principal curvatures of Γ. Our approach carries out the precise definition of a collection of all principal curvatures, i.e., we avoid the problem of their enumeration in order to keep to geometric invariant characters. In fact the description of this collection sequels the definition of p-convexity and assumes at least strict -convexity, what means that the points of Γ with zero mean curvature are out of our reach at present. In fact, we never use the notion of principal curvatures in our reasoning keeping instead to curvature matrices. It turns out that m-admissible functions generate (m-)-convex surfaces and vice versa and the following theorem reveals this interdependence. Theorem.. Let a hypersurface Γ Ω R n be a level set of m-admissible in Ω function, m n. Then Γ is strictly (m-)-convex. Assume now that Γ is a closed strictly (m-)-convex C 4 -hypersurface. Then there are m-admissible functions with level set Γ. The well known examples of the curvature equations are the mean curvature equation and the equation of prescribed Gauss curvature. The latter may be considered as Monge Ampere equation with righthand side depending on the gradient of solution, i.e., it is an equation with n-hessian operator in terminology of Section 3. Hence the natural set of solvability here is the cone of the convex functions or

3 ON THE BACKGROUNDS OF THE THEORY OF M-HESSIAN EQUATIONS 689 n-admissible functions, what is the same. In contrast to the case m = n there is no connection of m-curvature operators with m-hessian ones, m < n. In Section 5 we present the precise definition of m-convex (concave) functions and derive their main properties. It is shown, for instance, that the set of m-convex functions is not a cone if m < n, what is responsible for extra difficulties in research of the Dirichlet problem for curvature equations. We do not consider here its solvability, neither construct a priory estimates. Although some requirements known as necessary in this context spring up as the natural properties of m-convex functions for m < n. Section 6 contains a survey of the recent results concerning a solvability of the Dirichlet problem for m-hessian equations. It deals with the scale of solutions from classic to weak and presents minimal known requirements on data guaranteeing a solvability in the chosen class. One may find some intersections with the results obtained in [20]. 2. Algebraic equipment. We introduce the m-hessian function S m (S) as the sum of all m-th order principal minors of the matrix S naming it as the m-th-order trace of S and denote tr m S = S m (S), m =,..., n, S Sym(n), (4) where Sym(n) is the space of n n symmetric matrices with scalar product (S, S ) = trss. We also associate with Sym(n) the sets Sym (n, m) via functions S m by the following definition. Definition 2.. The matrix S (m) belongs to Sym (n, m) if there exists at least one S Sym(n) such that S (m) = (S ij m(s)), S ij m = S m(s) s ij, S = (s ij ). It is easy to see that Sym (n, n)=sym(n), Sym (n, ) = {Id}. The structure of Sym (n, m) for < m < n is an open problem. The following identity is a consequence of skew-symmetry of minors, [0] S m (S + ξ ξ) = S m (S) + (S (S; m)ξ, ξ), m n, ξ R n. (5) It is obvious that m-hessian functions are orthogonal-invariant in the following sense S m (BSB T ) = S m (S) if BB T = Id and we could reduce our consideration to m-th-order elementary symmetric functions in R n instead of m-hessian functions (4). However, such substitute only complicates if suits at all the application to partial differential equations. Let P n (t; S) = S n (S + tid) = det(s + tid), t R. The roots of the polynomial P n (t; S) are the eigenvalues of S and obviously are real whatever S Sym(n). It is also true for all P m (t; S) = S m (S + tid), < m < n, due to the identity (n m)p m (t; S) = dp m+ (t; S)/dt. We associate now with m-hessian functions (4) the cones K m Sym(n), by the following definition. Definition 2.2. A matrix S belongs to K m if all the roots of the polynomial P m (t; S) are negative. It turns out that K m admits the following equivalent to above descriptions. K m = {S : S i (S) > 0, i =,..., m}, (6) K m = {S : inf t 0 S m(s + tid) > 0}. (7)

4 690 NINA IVOCHKINA AND NADEZDA FILIMONENKOVA It follows from some results in [] that the relations (5), (7) suffice the inequality S m (S + ξ ξ) > S m (S), ξ R n, ξ > 0, S K m, (8) which may be also satisfied for some S / K m even in presence of S m (S) > 0, [4]. Hence, (8) does not define the cone K m. The definitions 2.2, (6), (7) were introduced in the paper [] where also was proved that the cones K m are convex in Sym(n) for m =, 2, n and remarked that it must be true for all m. In 985 the author of [] discovered the reference in [2] to the paper of L.Gårding [9], where the theory of a-hyperbolic m-homogenous polynomials was developed. It turned out that the polynomial P m (t; S) satisfies all the requirements of Gårding s theory. Here we extract from [9] the most essential for applications to fully nonlinear differential equations pieces. Let a R N be some fixed vector, x R N. Consider a polynomial Q m (x + ta) as the polynomial in t R. Definition 2.3. Q m (x + ta) is a-hyperbolic polynomial if its roots {t i } m are real. With every a-hyperbolic polynomial L.Gårding associates the cone C(a; Q m ) R N, C(a; Q m ) = {x R N : t i < 0, i =,..., m}. (9) and in particular proves the following. Theorem 2.4. The cone C(a; Q m ) is convex in R N. Moreover, if b C(a; Q m ) then Q m (x + tb) is b-hyperbolic and C(b; Q m ) = C(a; Q m ). There are many examples in [9] and one of them is the polynomial generated by the elementary symmetric function of m-th order. But the functions (4) with N = n(n + )/2, a = Id also suit the case, definitions 2.3 are identical for P m (t; S) and (9) reduces to (7) in our case. L.Gårding had developed his theory in order to derive some special inequalities for a-hyperbolic polynomials in the corresponding cones C(a). We present here one particular case of these. Theorem 2.5. Let S, S K m, S (m; S) be defined by definition 2.. Then (S (m; S), S ) ms m m m (S)S m (S ), m =,..., n. (0) The inequality (0) is precise but it is also precise in the following sense. Lemma 2.6. Let S Sym(n). Assume that (S, S) > 0 for all S K m. Then there is S K m such that S = S (m; S ). Moreover, S is positive definite. We introduce now the dual to K m cone K m by restriction of definition 2. to S K m. Namely, K m = {S = S (m, S), S K m }, m >, K = {Id}. () Due to lemma 2.6 the following description of K m is equivalent to (). K m = {S Sym(n) : (S, S) > 0, S K m }. Since K n is the cone of all positive definite matrices, K n = K n. We have no constructive description of K m for < m < n but the sequence of inclusions: K K 2... K n = K n K n... K. (2) Notice that all the cones in (2) are convex in Sym(n).

5 ON THE BACKGROUNDS OF THE THEORY OF M-HESSIAN EQUATIONS 69 It is sometimes reasonable to substitute S m, m n for the functions F m (S) = S m m (S), S K m, (3) which are positive -homogeneous. In this course the following normalization of Km arrives { km = s = } m S m m m (S)S (m; S). (4) It is obvious that k = {Id} and easy to derive that { kn = s = } n S det n S, S Kn. In terms (3), (4) the inequality (0) looks particulary impressive: (s, S) F m (S), s k m, S K m, (5) and means that F m is concave in K m. Hence, the level surfaces of function F m are convex in K m and the equation of C-level surface may be expressed in the Bellmann form: Consider the closure of the cone (7): inf(s, S) = C, S K m. km K m = {S : S + εid K m, ε > 0}. (6) It follows from (6), (6) that K m = K m {F m (S) = 0} and the inequality F m (S) δ > 0 keeps S strictly inside K m. The following proposition emphasizes the significance of such control. Lemma 2.7. Let S K m, S 0 = S/trS. Then F l (S 0 ) F ij l (S)ξ i ξ j (F l (S 0 )) l, F ij l = F l, l m, ξ =. (7) s ij Proof. The inequalities Ŝ l l Ŝ l l, S K m, Ŝ l = S l /Cn, l 2 l m, which are well known for the non-negative definite matrices, m = n, as Maclaurin inequalities, will carry out the right hand side of (7). In the paper [] they were extended to the arbitrary m n and the line F m (S) F m (S)... F (S), S K m is a rough version of these, which suffices the validity of (7). As to the left hand side of (7), it follows from monotonicity of quotients: S m S l S m,i S l,i, l < m n, i =,..., n, S K m, (8) where S p,i = S p / s ii, S 0 = by definition. The inequality (8) has been established in the paper [2] and we just apply it with l =.

6 692 NINA IVOCHKINA AND NADEZDA FILIMONENKOVA 3. On mollification of m-admissible functions. Let u C 2 (Ω), Ω be a bounded domain in R n, u xx be the Hessian matrix of u, F m [u] = F m (u xx ). On the basis of the matrix cone (6), (7) we single out a functional cone in C 2 (Ω) by the following definition. Definition 3.. A function u is m-admissible in Ω if u K m, where K m = {u C 2 (Ω) : u xx K m, x Ω}. (9) By (6) we also introduce K m. Notice that due to (5), (8) the m-hessian operator F m is non degenerate elliptic in K m, what we can not guarantee in K m. To prevent possible degeneration of ellipticity it suffices to require F m [u] δ with some δ > 0. Notice also that the above notions may be adapted for closed domains Ω in obvious way and we use relevant analogs below without additional explanation. To proceed with denote by Ω ε the equidistant to Ω domain with dist{ Ω; Ω ε } = ε and introduce a standard ε-regularization of u in Ω ε : u ε (x) = ε n B ε(x) ( x y u(y)ρ ε ) dy, x Ω ε, (20) where a mollifier ρ(z) is a non-negative function in C (R n ) vanishing outside the unit ball B (0) and satisfying R n ρ(z)dz =. Lemma 3.2. Let u be an m-admissible function in Ω. Then function (20) is m- admissible in Ω ε. Moreover, if F m [u] δ > 0 in Ω, then F m [u ε ] δ > 0 in Ω ε. Proof. Note firstly that inclusions (2) make valid inequality (0) for all l < m and we have under conditions of lemma the following version of (0), (5) for u, u K m : F ij l (u xx)u ij F l (u xx ), l m, x Ω. The latter means that all functions F l, l m are concave in K m. Hence, for an arbitrary collection {y k } N Ω the following line is true ( N ) N N F l α k u yy (y k ) α k F l (u yy (y k )), α k 0, α k =. (2) Keeping in mind the definition (20) we represent now u ε xx in the form u ε xx(x) = ( ) x y ε n u yy (y)ρ dy, x Ω ε. (22) ε B ε(x) Due to the properties of mollifier and (2), (22) it follows that ( ( ) ) ( ) x y x y F l u yy (y)ρ dy F l (u yy )ρ dy δε n, (23) B ε(x) ε ε l m, what validates Lemma 3.2. B ε(x) Our further goal is to present a weak analog of m-admissibility in order to envelop u C(Ω). Some steps in this direction may be found in frames of viscosity approach in the modern theory of fully nonlinear second-order partial differential elliptic equations. Here we just consider a closure of K m (Ω) in C(Ω). Definition 3.3. A function u C(Ω) is the m-subfunction if there exists a sequence {u k } K m (Ω) such that u(x) = lim k uk (x), x Ω. (24)

7 ON THE BACKGROUNDS OF THE THEORY OF M-HESSIAN EQUATIONS 693 If in addition F m [u k ] δ > 0 in Ω for all k N, then u is a strong m-subfunction. The following proposition is the straightforward corollary of Lemma 3.2 and (24). Theorem 3.4. Assume that u is an m-subfunction. Then its regularization (20) belongs to K m (Ω ε ) and if F m [u k ] δ > 0 in Ω for all k N, then F m [u ε ] δ > 0 in Ω ε. Theorem 3.4 contains a method to evaluate if u is an m-subfunction in Ω or not. In this course we introduce another definition of m-subfunction. Definition 3.5. The function u is an m-subfunction in Ω if u ε K m (Ω ε ) for all ε 0 and it is a strong m-subfunction if in addition F m [u ε ] δ with some δ > 0. The set of -subfunctions consists of subharmonic in Ω functions and the above definitions extend this well known notion to arbitrary m. But there are superharmonic functions and we introduce m-superfunctions in C 2 (Ω) on the basis of definition (7). Definition 3.6. A function u C 2 is an m-superfunction in Ω if inf tr m(u xx + tid) 0, x Ω. (25) t 0 For instance, u is an n-superfunction if its Hessian matrix has at least one non positive eigenvalue. Unlike m-admissibility the definition (25) does not stand mollification (20) for m >. There are no suitable analogs of the inequality (23). That is the reason one should keep to K m dealing with Hessian operators. Our next concern is the critical points of m-admissible in Ω functions, which are the points, where u x = 0. Lemma 3.7. Let a function u be m-admissible in Ω, {y} be the set of its critical points in Ω, Ω Ω. Then mes n m+ {y} = 0. (26) Proof. Due to the properties of m-admissible functions there is a finite covering of Ω : {B r (x i )} N, r = r( u C2 (Ω )), x i Ω, such that in every ball B r (x i ) at least one term of the sum tr m u xx is the determinant of positive definite m m matrix. Consider one of this balls, say B r (x ). Without loss of generality assume the matrix we speak about is generated by x,..., x m. We fix up some y {y} B r (x ). Then the function u(x,..., x m, y m+,..., y n ) is strictly convex in the cutset B r (x ) {x Ω, x m+ = y m+,..., x n = y n } and y its only critical point. The above argument does not depend on the choice of y, what validates (26). On the basis of lemma 3.7 we now extend Sard theorem to m-admissible functions. Theorem 3.8. Let u K m (Ω) C( Ω), y = {y Ω : u x (y) = 0}. Then mes {c R : u(x) = c, x Ω\y} = sup Ω u inf u. (27) Ω Indeed, let r > 0, B r (x 0 ) Ω. As it follows from (26), mes{b r (x 0 )\y} = mesb r and by continuity reasons mes {c R : u(x) = c, x B r (x 0 )\y} = sup Br(x 0) u inf Br(x 0) u. The latter suffices (27).

8 694 NINA IVOCHKINA AND NADEZDA FILIMONENKOVA 4. On the (m-)-convexity of the hypersurfaces. Let Γ R n be a C 2 - hypersurface and n + (M), n = n + be its unit normals. Consider M 0 Γ and fix up r > 0 thus that Γ r = Γ B r (M 0 ) admits local parametrization X = X(θ), θ = (θ,..., θ n ). Here X = (x,..., x n ) is the position vector of Γ r and without loss of generality we locate its origin at M 0. Then the metric tensor g = (g ij ) of Γ r is given by g ij = (X i, X j ), X i = X/ θ i. We also associate with Γ r the tensors η = (η i j ), τ, X θθ as g = η T η, det η > 0, τ = η, X θθ = (x ij,..., x n ij). (28) In order to substitute tensor approach (28) for geometric invariant one, i.e., independent on parametrization, we introduce the following characters X (i) = X k τ k i, X (ij) = X kl τ k i τ l j, i, j =,..., n. (29) It is obvious that (X (i), X (j) ) = δ ij, (X (i), n) = 0, i, j =,..., n and construction (29) completed with n presents a moving frame on Γ r. The variety of τ in (28) is responsible for rotations of moving frames in tangential to Γ r plane at the relevant points. We have two applicants for the position of geometric invariant curvature matrix K[Γ] [3]. Namely, K + [Γ](M) = (X (θθ), n + )(M), K [Γ] = K + [Γ]. (30) If a hypersurface is the boundary of the bounded domain Ω R n, then the classic approach in our terms appoints eigenvalues of matrices (30) generated by the interior normal n as the principal curvatures of Γ. We are interested in a local definition and in this course suggest the following. Definition 4.. A C 2 -hypersurface Γ R n is strictly (m )-convex if one of the matrices (30) belongs to K m for all M Γ. We say that the eigenvalues of the matrix, which enjoys this inclusion, are the principal curvatures of Γ and qualify this matrix as the curvature matrix K[Γ]. We also say that k l [Γ] = S l (K[Γ]), l m is the l-curvature of Γ. Concerning the notion of the principal curvatures the above definition covers only the points of Γ for which matrices (30) have non-zero traces and correlates with a common notion of the curvature of plane curve. Hence, we deal with the hypersurfaces of positive mean curvature. Notice that we does not fix enumeration of the principal curvatures. The latter depends on parametrization and can not be considered as geometric invariant. We formulate now some propositions to expose the interdependence of the notions of m-admissibility of functions and strict (m-)-convexity of hypersurfaces. Theorem 4.2. Let u K m (Ω), x 0 Ω and Γ r = {x Ω B r (x 0 ) : u(x) = u(x 0 ), u x (x 0 ) > 0}. Then there is r such that Γ r is a strictly (m-)-convex hypersurface. Proof. We fix up r by requirement u x (x) > 0, x Ω B r (x 0 ). Let x Γ r. We associate with x a cartesian basis {e i } n with the origin at x and direct e n according to the equality (e n, u x (x )) = u x (x ). It avails cartesian parametrization θ = x = (x,..., x n ) such that u x (0) = 0, u n (0) < 0. Then there is r r such that Γ B r (0) = {x Ω : x < r, x n = ω( x)}, where the function ω satisfy

9 ON THE BACKGROUNDS OF THE THEORY OF M-HESSIAN EQUATIONS 695 ω(0) = 0, ω x = 0 and the matrix ω x x (0) coincides with one of the matrices (30). Differentiating twice the identity u( x, ω( x)) = u(0) we arrive to the identity u x x (0) = u n (0)ω x x (0), (3) where u n (0) < 0 by the choice of e n. Since u is m-admissible in Ω the inequalities (7) hold for all l m, ξ =, x Ω. Let x = 0, ξ = (0,..., ) in (7) and denote S l,i = S l+ / s ii. Then at x = 0, l m the relations (7), (3) carry out the inequalities 0 < S l+(u xx ) S (u xx ) l + S l,n(u xx ) = l + ( u n) l S l (ω x x ) Sl+ (u xx ) S /(l+) l+ (u xx ) and (3), (32) suffice ω x x (0) K m. Hence, in accordance with definition 4. ω x x (0) is indeed the curvature matrix K[Γ r ](x ) and Γ r is strictly (m-)-convex at x. The above argument suits all x Γ r, what validates theorem 4.2. Moreover, 0 < ν u x l k l[γ r ] µ, l =,..., m (33) u x l with some 0 < ν < µ under control. The following global analog of the theorem 4.2 is a straightforward corollary of the theorems 3.8, 4.2. Corollary. Let u K m (Ω) C( Ω). Assume that for all c c R the level sets {x Ω : u = c} are C 2 hypersurfaces. Then mesc = sup Ω u inf Ω u. Moreover, all these hypersurfaces are strictly (m-)-convex. Actually theorem 3.8 could be proved via inequality (33) by pure geometric argument. Although we prefer an analytic approach of Section 3. Everybody knows that the graph of strictly convex (n-admissible) function is the strictly convex (n-convex) surface and nobody would say that the graph of C 2 - subharmonic (-admissible) function is of positive mean curvature (-convex). The situation on the whole is described in the following proposition. Lemma 4.3. Let u K m and Γ be the graph of u. Then Γ is a strictly (m-)-convex hypersurface. Proof. Consider some M 0 Γ. Without loss of generality assume that M 0 = (0,..., u(0)), e n u x (0). Then u x = 0, x = (x,..., x n ). The position vector of Γ is given by X = (x,..., x n, u(x)) and its metric tensor is g = (δ ij + u i u j ). We appoint in (28) τ = g and in capacity of curvature matrix in (30) we choose K[Γ] = (u (xx), n), n = ( u,..., u n, ). (34) + u 2 x We compute at M 0 : (32) u ( x x) (0) = u x x, u ( xn) (0) = u xn, u (nn) (0) = u nn + u 2 x + u 2. (35) x The formulaes (34), (35) bring out the presentation: ( + u 2 x) l 2 Sl (K[Γ])(M 0 ) = S l (u x x )(0) + + u 2 x(0) (S l(u xx )(0) S l (u x x )(0)), (36) l =,..., m. If m = n, then S n (u x x ) = 0 by definition and (36) just confirms that the graph of convex function is a convex hypersurface. If m =, the equality (36) does not contain any positive information because the sign of S (u x x ) is out

10 696 NINA IVOCHKINA AND NADEZDA FILIMONENKOVA of control for -admissible functions. If < l < n, then m-admissibility of function prescribes S l (u x x ) > 0, let alone S l (u xx ), for l m and hence Γ is strictly (m-)-convex in the sense of definition 4.. Corollary represents m-admissible functions in capacity of generators of (m-)- convex hypersurfaces. The opposite is also true in the following sense. Theorem 4.4. Let Ω R n be a bounded domain. Assume that Ω is strictly (m- )-convex, Ω C 4. Then there is a convex subcone K 0 m( Ω) K m ( Ω) such that the hypersurface Ω is a zero level set for all u K 0 m( Ω). Indeed, the fact that K 0 m( Ω) is not empty is a consequence of existence theorems surveyed in Section 5. Convexity of K 0 m( Ω) succeeds convexity of K m ( Ω). Remark. According to definition 3.3 the hypersurface from theorem 4.4 generates actually the whole cone of m-subfunctions in Ω with Ω in capacity of zero level set. Remark 2. Similar to the functional cone K m (Ω) the matrix cone K m Sym(n) generates also a cone of p-convex hypersurfaces in the following sense. Let X R n+ be a position vector of the m-convex hypersurface Γ. Denote by λγ the hypersurface with a position vector λx. Then and hence λγ is also m-convex hypersurface K[λΓ] = K[Γ], λ > 0. (37) λ 5. On the m-convexity of functions. Actually the term m-admissible function has appeared in the paper [25] in the context of curvature operators and meant that the set of functions with m-convex graphs are under consideration. It differs from the terminology of the previous section, when m < n. We see it reasonable to call such functions as m-convex in Ω. Although to adjust this notion with definition 4. and lemma 4.3 we have to be more precise. Definition 5.. A function u C 2 is m-convex in the domain Ω R n if n ( H [u] = δ ij u iu j + u 2 x + u 2 x ) u ij > 0 (38) and its graph Γ u R n+ is an m-convex hypersurface. If H [u] < 0 while Γ u is m-convex, then u is m-concave in Ω. The operator H [u] is well known as the mean curvature operator and the requirement (38) prescribes for the normal to Γ u to be directed as in (34). In this course H [u] = k [Γ u ]. It is rather remarkable that there is no analogs of (37) for the set of m-convex functions, i.e., this set is not a cone. To make our point precise we consider two examples. Notice firstly that if Γ u is the graph of C 2 -function, then the following presentation is true at x Ω ( u 2 k p [Γ] = x ( + u 2 x) p 2 + u 2 S p,n (u xx ) + x + u 2 S p (u xx ) x ), p n, (39)

11 ON THE BACKGROUNDS OF THE THEORY OF M-HESSIAN EQUATIONS 697 where e n is parallel to u x (x). Let ( ) u = n a(x ) 2 + (x i ) 2 + bx n, a > 0, m < n, 2 2 n < (a + )m < n, b 2 < n n (a + )m n m (a + )m n +. By direct computation we derive that (39) brings out the following at x = 0: k p [u ](0) > 0, p =,..., m, S m,n (u xx )(0) < 0. We see that the function u is strictly m-convex in the neighborhood of 0 but λu is not if λ >>. Although λu is strictly (m-)-convex for all λ > 0. Our second example is ( u 2 = n ) (x i ) 2 + (x n ) 2 + bx n, a > n m 2 m, b2 > a m. (40) n m The functions (40) are strictly m-convex and this time {λu 2, λ > } is the set of strictly m-convex functions. On the other hand a product λu 2 with sufficiently small λ > 0 is not m-convex since S m [u 2 ](0) < 0. We notice eventually that there are a(p), b(p) in the example (40) such that a strictly m-convex function u 2 is not p-admissible whatever p m < n be. However, there are common properties of strictly m-convex and strictly m- admissible functions. To expose them and some other we relate to an m-convex function u the set of its critical points {y Ω : u x (y) = 0}, the graph Γ u and introduce the level surfaces as Γ u0 (x) = {x Ω : u(x) = u 0 = u(x 0 ), u x (x 0 ) > 0}. To keep orientation prescribed by (34) we also associate with Γ u moving frames {e i } n with e n = u x / u x and a domain Ω r [ Γ u ] = {x = x + τe n ( x), x Γ u, 0 < τ < r}. Theorem 5.2. Let u be a strictly m-convex in Ω function. Then the equality (3.8) is true and Γ u0 (x) K m. Indeed, the identities (39) ensure strict m-admissibility of u in some neighborhood of {y}. Hence, lemma 3.7 is valid. The strict (m-)-convexity of Γ u0 at x 0 may be established similarly to the proof of theorem 4.2 on the basis of (39). Now we present some special properties of m-convex functions. Theorem 5.3. Let u be m-convex at x 0. Assume that u x (x 0 ) > 0. Then k p [ Γ u0 ](x 0 ) 0 for p m if and only if λu is m-convex at x 0 for all λ >>. Indeed, due to the assumption that x 0 is non critical point of u the identities (39) are equivalent to the relations ( ux p+2 k p [Γ u ] = ( + u 2 x) p 2 + u 2 k p [ Γ u0 ] + ) x + u 2 S p (u xx ), p n (4) x in some neighborhood of x 0. If k p [Γ λu ](M 0 ) > 0 for all λ >, p m, the following inequalities hold: u x p+2 k p [ Γ u0 ](x 0 ) + λ 2 S p(u xx )(x 0 ) > 0, p m, what ensures the desirable inequalities for Γ u0. On the other hand, these inequalities obviously suffice the m-convexity of functions λu at x 0 for sufficiently large λ. In the further proceeding we drop the lower index u for the sake of simplicity.

12 698 NINA IVOCHKINA AND NADEZDA FILIMONENKOVA Theorem 5.4. Let x 0 Γ be a non critical point of u. Assume there is r > 0 such that u x (x 0 ) u x (x), x Ω r [ Γ] B r (x 0 ). (42) Assume also that for some p < n the inequality holds. Then k p [Γ](M 0 ) < k p [ Γ](x 0 ) u 2 k 2 p [Γ](M 0 ) x, x Ω k 2 p [ Γ](x 0 ) k 2 r [ Γ] B r (x 0 ). (43) p [Γ](M 0 ) Proof. To begin with we rewrite at x 0 the inequality (4) in the form k p [Γ] = u x p ( + u 2 x) k p[ Γ] p/2 + ( + u 2 x) ((p ) u x p u p 2 + (nn) k p [ Γ] S p 2,ns u 2 (ns)), (44) where S 0 =, S l = 0, l < 0 by definition, S p 2,ns = 2 S p / u (ss) u (nn), {e s } n is the tangential to Γ u part of the relevant moving frame. On the other hand, due to our agreement u x = u n, u (ns) = 0 and hence u (nn) 0 at x 0 by (42). In this course the inequality (43) sequels the identity (44) considered at M 0 and the assumption (42). Remark 3. If m < n it may happen in the theorem 5.4 that u nn (x 0 ) = 0. Then k p [Γ](M 0 ) < k p [ Γ](x 0 ), p m < n, as it follows from (44). The proceeding of this section gives new points of view on some known developments in the theory of Dirichlet problem for curvature equations. To explicitly expose these points we consider in C 2 ( Ω) m-curvature operators H m [u] = tr m K(Γ u ). Although in the theory of nonlinear equations they prefer to work with Ĥm[u] = ( + u 2 x) m/2+ H m [u] and we will formulate our observations in terms of Ĥm. The following proposition is a straightforward consequence of the theorem 5.3. Corollary 2. Let u be strictly m-convex in Ω r, Ω be its level surface, λ. Assume that Ω K m {k m [ Ω] 0}. Then Ĥ m [λu] λ m Ĥ m [u], x Ω. (45) The inequality (45) is necessary for u to be a barrier function near the boundary and in this sense the requirement Ω K m {k m [ Ω] 0} is necessary. For m = it has been found in the paper [8] for the Dirichlet problem for the minimal surface equation with arbitrary boundary data. Although it does not suffice the classic solvability of the Dirichlet problem Ĥ m [u] = ( + u 2 x) m 2 + f, u Ω = 0 (46) in the class of strictly m-convex in Ω functions. The following proposition presents close to necessary sufficient conditions. Corollary 3. Let u C 2 ( Ω) be a strictly m-convex solution to the problem (46), m < n. Assume that k m [ Ω] > f, x Ω. (47)

13 ON THE BACKGROUNDS OF THE THEORY OF M-HESSIAN EQUATIONS 699 Then either u x attains its maximal value inside Ω, or u 2 x sup Ω f 2/m k 2/m m [ Ω] f 2/m, x Ω. (48) Indeed, the inequality (48) is a rough version of (43), where Γ = Ω. For m = the assumption (47) has been discovered by J.Serrin [23] as necessary and sufficient to admit arbitrary Dirichlet data. For < m < n it may found in [24], [3] as the basis to derive a priory bound for u x also for arbitrary Dirichlet data. It follows from the remark 3 that condition (47) is close to necessary to keep u x bounded at the boundary even for the constant Dirichlet data for all m < n. The case m = n needs different from above approach. 6. On solvability of the Dirichlet problem for m-hessian equations. As above let Ω be a bounded domain in R n, u C 2 (Ω), u xx be the Hessian matrix of u, F m [u] = F m (u xx ) = (tr m u xx ) m, m n. We will consider the Dirichlet problem for the m-hessian equation F m [u] = f, u Ω = ϕ. (49) The case m = corresponds to the Poisson equation, with m = n we have the Monge-Ampère equation, both are thoroughly studied. Here we deal with the full range of equations in between the former and the latter. We will pay attention to one specific aspect of the problems (49), which singles out them from the variety of fully nonlinear not totally elliptic differential equations. Namely, the structure of the m-hessian operator ensures the following version of Alexandrov maximum principle ([], an updated version is presented in [8]): Lemma 6.. Assume that u K m (Ω), z C 2 (Ω), Ω + = {x Ω : z xx (x) 0}. If Ω B r (0) then ij z inf z c(n)r Fm [u]z ij Ln (Ω Ω + ). If Ω {x R n : x i < r i, i =, 2,..., n} then z inf Ω z c(n)(r r 2...r n ) n F ij m [u]z ij L n (Ω + ). Since the equations (49) are not totally elliptic, the theory of their classic solvability is connected with the cone of m-admissible functions (9). The -homogeneity of the function F m allows to represent the equation (49) as Fm ij [u]u ij = f. We suggest the following version of inequality (7) for a solution u K m (Ω) to equation (49): ( ) ( ) m f(x) F ij F [u] m [u]ξ i ξ j c(n), ξ R n, ξ =. c(n) F [u] f(x) If a priory estimate of the value F [u] is available and f ν > 0 in Ω the quadratic form Fm ij [u]ξ i ξ j is uniformly positive definite in Ω. It means that the equation (49) is uniformly elliptic in case f ν > 0 in Ω and if there exists a priory estimate of a solution u in C 2 ( Ω) and it can degenerate when f vanishes. In our survey we would like to show the connection between the smoothness of f and the vanishing of f on the one hand and the quality of a solution to m-hessian equation on the other hand. We demonstrate the scale of solutions from classic to weak and present minimal known requirements on the data that guarantee the solvability in the chosen class.

14 700 NINA IVOCHKINA AND NADEZDA FILIMONENKOVA The problem of the classic solvability of (49) in the set of m-admissible functions was first considered in 985 in paper [2] by N.M.Ivochkina for a particular case (in convex domains and zero boundary condition) and in the fundamental paper of L.Caffarelli, L.Nirenberg, J.Spruck [2] where the goal was apparently to consider the most general class of Hessian equations, which made the authors to suppose that the data belong to C ( Ω). Certain optimization of this results has recently been obtained in the papers [5], [8]: the classic solvability of the problem (49) is proved in C 2+α ( Ω) with 0 < α. The following theorem presents the optimal correspondence between the smoothness of the data and a solution to (49) starting with C 4+α : Theorem 6.2. Assume that Ω C l+α is a strictly (m )-convex surface, ϕ C l+α ( Ω), f C l 2+α ( Ω), f ν > 0 in Ω, l 4, 0 < α <. Then there exists an m-admissible solution u C l+α ( Ω) of (49). A complete proof of this theorem via continuity method is implemented in the paper [8]. Suitable a priory estimates are derived in the papers [8], [5]. In the former a priory estimate of u C2 (Ω) is produced in the following way: the Alexandrov maximum principle from Lemma 6. reduces the estimation to Ω, where special barriers are constructed in terms of curvature matrix (30) of Ω to conclude with. This procedure depends on the minimal value of (m-)-curvature of Ω and requires Ω to be a strictly (m )-convex surface. The paper [5] delivers a priori estimate of the Hölder seminorm of u xx which is based on the methods of L.Evans, N.V.Krylov and M.V.Safonov constructed in the early 980s for obtaining an estimate for solutions of fully nonlinear uniformly elliptic second order differential equations. At present the problem of studying the smoothness properties of weak solutions has become important. In 997 N.Trudinger [26] introduced a notion of m- approximate solution to m-hessian equation (49) with f L p (Ω), f 0. In terms of m-subfunctions introduced in definitions 3.3, 3.5 this notion looks like the following Definition 6.3. Let Ω be a Lipschitz domain. A m-subfunction v is called m- approximate solution of (49) with f L p (Ω), p >, ϕ C( Ω) if its set of regularizations {u ε } satisfies Dirichlet problem (49) in the sense: F m [u ε ] f L p (Ω) 0, u ε ϕ C( Ω) 0 if ε 0. If f L n (Ω) we can apply the Aleksandrov maximum principle from Lemma 6. to derive the existence, uniqueness and continuity in Ω of m-approximate solution under assumption of classic solvability of associated with (49) regularized problems, i.e., on the basis of the theorem 6.2. Lemma 6.4. Let Ω be a Lipschitz domain. Assume that ϕ C( Ω), f L n (Ω), f 0. Then an m-approximate solution v of (49) is unique. If in addition Ω C 4+σ, 0 < σ < is strictly (m )-convex surface, then there exists an m-approximate solution v of (49) and v C( Ω). We should notice that the requirement Ω C 4+σ looks excessive for the existence of an m-approximate solution. But we assume that Ω is at least C 4+σ - hypersurface whatever weak solution is under consideration because mollification of strictly (m )-convex surfaces yet stays an open question. We plan in future to weaken this requirement. We can also construct solutions of class C 2+α ( Ω) with 0 < α and C 3+α ( Ω) with any 0 < α < as an m-approximate solution:

15 ON THE BACKGROUNDS OF THE THEORY OF M-HESSIAN EQUATIONS 70 Theorem 6.5. Assume that Ω C 4+σ, 0 < σ <, is a strictly (m )-convex surface, ϕ C 4 ( Ω), f ν > 0 in Ω, f is convex in Ω. If f x L ( Ω), then there exists an m-admissible solution u C 2+α ( Ω) to (49) with some 0 < α. If f C +α ( Ω), 0 < α <, then there exists an m-admissible solution u C 3+α ( Ω) to (49). We present this theorem in a simplified form which is due to the assumption of convexity f. In fact the convexity condition on f may be replaced with the boundedness of the value fxx Ln (Ω), where f xx(x) = sup (f ij (x)ξ i ξ j ). ξ R n, ξ = For convex functions this characteristics is equal to 0. The theorem 6.5 and its more generale case arise from the a priori estimate of a solution u in C 2+α ( Ω) from the paper [5]. In theorems 6.2 and 6.5 we assume that f does not vanish in Ω, i.e. both are true for equations (49) with the non-degenerate uniform ellipticity. The next theorem admits the degeneration inside Ω. We associate with Ω a boundary strip: Π d = {x Ω : dist(x, Ω) < d}. Theorem 6.6. Assume that Ω C 4+σ, 0 < σ <, is strictly (m )-convex surface, ϕ C 4 ( Ω). If f is convex, f x L ( Ω) and f ν > 0 in Π d. Let v be an m-approximate solution to (49). Then v C + ( Ω). Similar to the theorem 6.5 the convexity condition for f may be replaced by the boundness of the value fxx Ln (Ω) and condition f x L ( Ω) must be substituted f W nq (Π d) W n (Ω) with q > (n + )/2. The theorem 6.6 is the consequence of the a priori estimate for a solution u to (49) in C 2 ( Ω), which is constructed in the paper [8]. Finally we consider conditions admitting the complete vanishing of f in Ω. It is well known that the inequality f 0 does not guarantee continuity of the second derivatives of solutions to the problem (49) however smooth the data are. In this situation the first attempt to obtain C + ( Ω)-solution to (49) has been made in the paper [6]. The estimate (.0) from the theorem. from the paper [6] provides the existence of the following approximate solution: Theorem 6.7. Assume that Ω C 4+σ, 0 < σ <, is strictly (m )-convex surface, ϕ C 4+σ ( Ω), f C + ( Ω), f 0. Then there exists the unique subfunction v C + ( Ω) which is the weak solution to(49). A considerably different approach characterizes the papers [7], [6] [8], where the following results are proved for f from Lebesgue and Sobolev spaces: Theorem 6.8. Assume that Ω C 4+σ, 0 < σ <, is strictly (m )-convex surface, ϕ C 2 ( Ω), f 0. Let v be an m-approximate solution to (49). (i) If f L nq (Π d ) W n (Ω) and q > (n + )/2 then v Lip( Ω). (ii) If f ( L nq (Π) d ) L n (Ω) and q (n + )/2 then v C α ( Ω) with any 0 < α < n+ 2n 2 q. (iii) If f L n (Ω) then v C( Ω) C α (Ω) with any 0 < α <.

16 702 NINA IVOCHKINA AND NADEZDA FILIMONENKOVA The statement (i) is the immediate corollary of the a priory estimate of a solution to (49) in C ( Ω) from the papers [6], [8]. The idea of the proof of (iii) originates from N.Trudinger paper [26]. A priory estimate of the Hölder constant for solutions in closed domains under conditions (ii) were established recently in [7], [7]. Acknowledgments. We are grateful to the referees for the really helpful comments. REFERENCES [] A. D. Aleksandrov, Dirichlet problem for the equation Det z ij = ϕ, (Russian) Vestnik Leningrad. Univ. Ser. Mat. Meh. Astr., 3 (958), [2] L. Caffarelli, L. Nirenberg and J. Y. Spruck, The Dirichlet problem for nonlinear second-order elliptic equations. III. Functions of the eigenvalues of the Hessian, Acta Math., 55 (985), [3] L. Caffarelli and L. Silvestre, Smooth approximations to solutions of nonconvex fully nonlinear elliptic equations, AMS Transl., Series 2, 229 (200), Advances in the Math. Sci., textbf64 (200), Nonlinear Part. Diff. Eq. and Related Topics, [4] H. Dong, N. V. Krylov and X. Li, On fully nonlinear elliptic and parabolic equations in domains with VMO coefficients, Algebra i Analiz, 23 (20). [5] L. C. Evans, Classical solutions of fully nonlinear, convex, second-order elliptic equations, Comm. Pure Appl. Math., 35 (982), [6] N. V. Filimonenkova, Analysis of the behavior of a solution to m-hessian equations near the boundary of a domain, Problems in mathematical analysis, no. 45, J. Math. Sci. (N. Y.), 66 (200), [7] N. V. Filimonenkova, An estimate for the Hölder constant for weak solutions to m-hessian equations in a closed domain, Vestnik St. Petersburg Univ. Math., 43 (200), [8] N. V. Filimonenkova, A Quality Analysis of Weak Solutions to m-hessian Equations, Ph.D thesis, PDMI RAS, 200. [9] L. Garding, An inequality for hyperbolic polynomials, J. Math. Mech., 8 (959), [0] N. M. Ivochkina, Second order equations with d-elliptic operators, Trudy Mat. Inst. Steklov, 47 (980), 40 56, English transl. in Proc. Steclov Inst. Math., 2 (98). [] N. M. Ivochkina, A description of the stability cones generated by differential operators of Monge - Ampere type, Mat. Sb., 22 (983), , English transl. in Math. USSR Sb., 50 (985). [2] N. M. Ivochkina, Solution of the Dirichlet problem for some equations of Monge - Ampere type, Mat. Sb., 28 (985), , English transl. in Math. USSR Sb., 56 (987). [3] N. M. Ivochkina, Solution of the Dirichlet problem for the curvature equation order m, Algebra i Analiz, 2 (990), 92 27, English transl. in Leningrad Math. J., 2 (99). [4] N. M. Ivochkina, The Dirichlet principle in the theory of equations of Monge - Ampere type, Algebra i Analiz, 4 (993), English transl. in St. Petersburg Math. J., 4 (993). [5] N. M. Ivochkina, On the Hölder constant for the second order derivatives of admissible solutions to m-hessian equations, Problems in mathematical analysis, no. 50, J. Math. Sci. (N. Y.), 70 (200), [6] N. M. Ivochkina, N. S. Trudinger and X.-J. Wang, The Dirichlet problem for degenerate Hessian equations, Comm. Partial Differ. Equations, 29 (2004), [7] N. M. Ivochkina and N. V. Filimonenkova, Estimate of the Hölder constant for solutions to m-hessian equations, Problems in mathematical analysis, no. 40, J. Math. Sci. (N. Y.), 59 (2009), [8] H. Jenkins and J. Serrin, The Dirichlet problem for the minimal surface equation in higher dimensions, J. Reine Angew. Math., 229 (968), [9] N. V. Krylov, Boundedly inhomogeneous elliptic and parabolic equations in a domain, (Russian) Izv. Akad. Nauk SSSR Ser. Mat., 47 (983), 75 08, English transl. Math. USSR Izv., 22 (984), [20] D. Labutin, Potential theory for a class of fully nonlinear elliptic equations, Duke Math. J., (2002), 49. [2] M. Lin and N. S. Trudinger, On some inequalities for elementary symmetric functions, Bull. Austr. Math. Soc., 50 (994),

17 ON THE BACKGROUNDS OF THE THEORY OF M-HESSIAN EQUATIONS 703 [22] A. V. Pogorelov, The Mincowski Multidimensional Problem, Nauka, Moscow, 975, English transl. in New York, J.Wiley, 978. [23] J. Serrin, The problem of Dirichlet for quasilinear elliptic differential equations with many independent variables, Philos. Trans. Roy. Soc. London Ser. A, 264 (969), [24] N. S. Trudinger, The Dirichlet problem for the prescribed curvature equations, Arch. Rat. Mech. Anal., (990), [25] N. S. Trudinger, Maximum principles for curvature quotient equations, J. Math. Sci. Univ. Tokyo, (994), [26] N. S. Trudinger, Weak solutions of Hessian equations, Comm. Partial Differential Equation, 22 (997), [27] N. S. Trudinger and X.-J. Wang, Hessian measures II, Ann. of Math., 50 (999), [28] N. S. Trudinger and X.-J. Wang, Hessian measures I, Topol. Methods Nonlinear Anal., 0 (997), Received July 20; revised April address: Ninaiv@NI570.spb.edu address: nf33@yandex.ru