Singularities of affine fibrations in the regularity theory of Fourier integral operators

Transcription

1 Russian Math. Surveys, 55 (2000), Singularities of affine fibrations in the regularity theory of Fourier integral operators Michael Ruzhansky In the paper the regularity properties of Fourier integral operators are considered in different function spaces. The most interesting case are L p spaces, for which a survey of recent results is made. Thus, the sharp orders are known for operators, satisfying the so-called smooth factorization condition. Further in the paper this condition is analyzed in both real and complex settings. In the last case conditions for the continuity of Fourier integral operators are related to the singularities of affine fibrations in (subsets of) C n, defined by the kernels of Jacobi matrices of holomorphic mappings. Singularities of such fibrations are analyzed in this paper in the general case. In particular, it is shown that for small dimensions n or for small ranks of the Jacobi matrix, all singularities of an affine fibration are removable. Fourier integral operators lead to fibrations, given by the kernels of the Hessian of a phase function of the operator. Based on the analysis of singularities for operators, commuting with translations, in a number of case the factorization condition is shown to be satisfied, which leads to L p estimates for operators. In the other cases, the failure of the factorization condition is exhibited by a number of examples. Results are applied to derive L p estimates for solutions of the Cauchy problem for hyperbolic partial differential operators. Contents 1 Introduction Regularity of Fourier integral operators Fibrations with affine fibers Formulation for the parametric fibrations Several applications Estimates for Fourier integral operators Fourier integral operators Function spaces Estimates Parametric fibrations Affine fibrations Methods of complex analytic geometry Main results Further results

2 4 Affine fibrations of gradient type Localization Fibrations of gradient type Reconstruction of the phase function Existence of singular fibrations of gradient type Further estimates for Fourier integral operators Complexification Operators, commuting with translations Application to Fourier integral operators Cauchy problem for hyperbolic equations Estimates for the solutions Monge Ampère equation Sharpness of the estimates Essentially homogeneous case A representation formula for continuous operators of small negative orders 65 1 Introduction 1.1 Regularity of Fourier integral operators The regularity theory of solutions of partial differential equations has been attracting the attention of mathematicians for a long time. One of important examples is the wave equation, whose solution is well known and admits a representation as a sum of two elliptic Fourier integral operators with phase functions solving the eikonal equation of geometric optics. In general, solutions to the Cauchy problem for hyperbolic equations admit similar representations. This and other applications led to the development of the theory of singular integral operators, pseudo-differential operators and Fourier integral operators. In contrast with elliptic equations, singularities of solutions of hyperbolic equations propagate along the singularities of the Schwartz integral kernel of the solution operator. In this case the theory of Fourier integral operators replaces the theory of pseudo-differential operators, which was sufficient for elliptic equations. In this paper we will discuss the regularity theory of non-degenerate Fourier integral operators in L p and other function spaces. In this paper by the continuity in L p (or from L p to L p ) we will understand the continuity (of a linear operator) from L p comp to L p loc. Because Lp results may depend on the geometric structure of the wave front of the operator, we will discuss several related problems from the singularity theory of the wave fronts. Singularities arising in the L p theory are a particular case of the singularities of affine fibrations. For the latter we will derive estimates on the dimensions of the set of its essential singularities and will analyze its structure. Let T be a Fourier integral operator. This means that locally T is of the form T u(x) = e iφ(x,y,θ) a(x, y, θ)u(y)dθdy, (1.1) R m R N 2

3 where a S ν (R n R m R N ) is a symbol of order ν. This means that a is a smooth function for which the estimate β x,y α ξ a(x, y, θ) C(α, β, K)(1 + θ ) ν α (1.2) holds in every compact set K and all multi indices α and β. The phase function Φ is smooth, nondegenerate, and positively homogeneous of degree one in θ. We should note that we will give a more general definition of general symbols in Sρ μ and phase functions on arbitrary smooth manifolds in Section 2.1. The Schwartz integral kernels of operators of the form (1.1) are called the Lagrangian distributions (or Fourier integral distributions). The wave front of the Lagrangian distribution of an operator T defines a geometric invariant for the operator T. Indeed, the set W F (T ) = {(x, d x Φ(x, y, θ), y, d y Φ(x, y, θ)) : d θ Φ(x, y, θ) = 0} (1.3) in the cotangent bundle T (R n R m ) does not depend on the choice of a phase function Φ. If T (R n R m ) is equipped with its standard symplectic form, then the conic set in (1.3) becomes a Lagrangian submanifold of the cotangent bundle T (R n R m ). The converse statement is one of the main results of the global theory of Fourier integral operators. It can be formulated more conveniently in the manifold setting. Let X and Y be real smooth manifolds of dimensions n and m, respectively, and in this paper we will assume that n = m. Let σ X and σ Y denote the canonical symplectic forms on T X and T Y, respectively. Let C be a conic Lagrangian submanifold of the cotangent bundle T X\0 T Y \0 equipped with the symplectic form σ X σ Y. Then C defines a family of Fourier integral operators T with W F (T ) = C, locally of the form (1.1). The set C is called the canonical relation. If we fix an order ν of a symbol a in (1.1), the family of operators T is denoted by I μ (X, Y ; C), where μ = ν + (N n)/2. Let us give now several important examples of Fourier integral operators. If we take n = m = N, a = 1, and Φ(x, y, θ) = x y, θ, (1.4) then the right hand side of (1.1) is a composition of the Fourier transform and its inverse in R n, and T is the identity operator in this case. If a phase function Φ of an operator T is given by (1.4) and its symbol a is polynomial in θ, then T defines a partial differential operator with symbol a. If a phase function Φ of T is given by (1.4) and its symbol a is arbitrary, as in (1.2), then T defines a pseudo-differential operator with symbol a. The space of pseudo-differential operators of order μ is denoted by Ψ μ. The solution operator to the wave equation has the phase function of the form Φ(x, y, θ) = x y, θ + θ. In Section 5.2 we will give more examples of convolution operators and in Section 6 the solution operators to the Cauchy problem for hyperbolic equations will be regarded as Fourier integral operators as well. In this paper we will be interested in the continuity properties of the described operators in various function spaces. The best behavior is exhibited by pseudo-differential operators. Thus, pseudo-differential operators P Ψ 0 of zero order are continuous (as linear operators) from L p to L p for all 1 < p <. Moreover, a pseudo-differential operator P Ψ μ of order μ R can be extended to a continuous operator from the Sobolev space L p k to Lp k μ for all k R, k μ, and 1 < p <. Similar results hold in Lipschitz spaces, operators P Ψ μ are continuous from Lip (γ) to Lip (γ μ) for all γ > μ. However, the phase function Φ for pseudo-differential operators is of the 3

4 form (1.4) and its canonical relation C is equal to the conormal bundle to the diagonal in R n R n. For the Fourier integral operators the structure of the canonical relation C is much more complicated and their continuity properties depend on the geometric structure of the corresponding canonical relations. Let T I 0 (X, Y ; C) be a Fourier integral operator of zero order, associated to the canonical relation C. Let π X Y, π X, π Y be the canonical projections: T X\0 π X π C Y T Y \0. π X Y X Y (1.5) It turns out that the continuity properties of a Fourier integral operator T rely heavily on singularities of the projections π X Y, π X, π Y. Projections π X, π Y can be diffeomorphic only simultaneously and in this case for every λ 0 = (x 0, ξ 0, y 0, η 0 ) C there exists a symplectomorphism χ (a diffeomorphism preserving the symplectic structure) in a neighborhood of the point (y 0, η 0 ) T Y \0 such that in a neighborhood of λ 0, the canonical relation C has the form {(x, ξ, y, η) : (x, ξ) = χ(y, η)}. (1.6) In this case, C is locally equal to the graph of a canonical symplectic transformation and is called a local canonical graph or just a local graph. It is clear that C being a canonical graph implies that the projections π X, π Y are diffeomorphic from C to T X\0 and T Y \0, respectively. In particular, this implies n = m. The converse is also true. In fact, assume that, say, π Y : C T Y \0 is a local diffeomorphism. Then, the canonical relation is locally of the form (1.6) in (y, η) coordinates and the condition of C to be Lagrangian for σ X σ Y implies that σ X σ Y vanishes on C and σ Y = χ (σ X ). The latter means that χ is a symplectomorphism and C is a canonical graph. In this paper we are interested in applications to the hyperbolic partial differential equations, where canonical relations are local canonical graphs. Such operators arise as solution operators of hyperbolic Cauchy problems. In this case, the mapping π X C π 1 Y is equal to χ in (1.6) and defines a local diffeomorphism from T Y \0 to T X\0. It follows that dimensions of X and Y coincide. Operators T I 0 (X, Y ; C) with a local canonical graph C are continuous in L 2 ([14], [32], [33]). The proof is based on the fact that the canonical relation of T T is the conormal bundle of the diagonal in X X, and, therefore, it is a pseudo-differential operator of order 0 and hence bounded on L 2. In general, for 1 < p <, p 2, Fourier integral operators of order zero need not be bounded on L p. From the point of view of the L p continuity of Fourier integral operators, pseudo-- differential operators and operators arising as solution operators to the wave equation are two opposite cases. The phase function of the latter has the form x y, ξ + ξ in R n, and Littman ([37]) has shown that the corresponding operators T I μ (X, Y ; C) are not bounded in L p when μ > (n 1) 1/p 1/2. The L p properties of solutions to hyperbolic Cauchy problems for the equations of the wave type have been studied in many papers ([59], [43], [40], [15]). Lipschitz and L p estimates for the wave equation on compact manifolds were derived in [19] and some results for hyperbolic equations are in [63]. General results on L p continuity were obtained in [56]. Let us describe them in more detail. Operators T I μ (X, Y ; C) are bounded from L p comp to L p loc 4

5 if μ (n 1) 1/p 1/2 and this order is sharp if T is elliptic and dπ X Y C has full rank, equal to 2n 1, anywhere. These conditions hold for operators arising as solutions to strictly hyperbolic Cauchy problems in some cases. There is the same loss of order by (n 1) 1/p 1/2 in Sobolev and in Lipschitz spaces (in Lipschitz spaces p = ). If we denote α p = (n 1) 1/p 1/2, then operators T I μ (X, Y ; C) are continuous from L p α to L p α α p μ and from Lip (α) to Lip (α α μ). The proof of the L p boundedness is based on the complex interpolation method. Having the L 2 boundedness of zero order operators, the problem reduces to showing that operators of order (n 1)/2 are locally bounded from the Hardy space H 1 to L 1. Hardy spaces will be discussed in Section 2.2 in more detail. In general, the projection π X Y satisfies inequalities n rank dπ X Y C 2n 1, (1.7) because C is conic. The boundary cases are pseudo-differential operators with no loss of smoothness and solutions to strictly hyperbolic partial differential equations with the loss of (n 1)/2 derivatives. An important ingredient influencing the best order for L p continuity is the rank in (1.7). Here it is essential that p 2 because the L 2 results do not depend on the rank in (1.7). Now, we will formulate the important result due to [56] for the operators in Iρ μ (X, Y ; C), which assures the improved L p regularity under the following condition. The canonical relation C will be said to satisfy the smooth factorization condition, if there exists k, 0 k n 1, such that π X Y can be locally factored by fiber preserving homogeneous maps on C of constant rank n + k. More precisely, this means that for every λ 0 = (x 0, ξ 0, y 0, η 0 ) C there is a conic neighborhood U λ0 of λ 0 in C, and a smooth map π λ0 : C U λ0 C, homogeneous of degree 0, such that rank dπ λ0 n + k, π X Y C Uλ0 = π X Y π λ0. (1.8) Under the smooth factorization condition, the operators in I μ (X, Y ; C) are bounded from L p comp to L p loc provided 1 < p < and μ k 1/p 1/2. We will discuss the sharpness of these orders in Section 7 and will give general results for the class Iρ μ in Section 2.3. The smooth factorization condition is not necessary for operators T I μ with μ k 1/p 1/2 to be continuous in L p. Several examples show that the continuity is possible when the factorization condition fails ([55]). The relaxation of the smooth factorization condition is an open problem. The best result would be to show that operators T I μ (X, Y ; C) are bounded from L p comp to L p loc provided μ k 1/p 1/2, 1 < p <, and rank dπ X Y C n + k. Note that the proof of this result for k = n 1 is based on the continuity result from H 1 to L 1 for operators of the order (n 1)/2. One can assume that these operators are also weakly continuous in L 1. However, this problem is still open. The factorization condition is interesting in its own right and it allows fascinating generalizations, which we will discuss in Sections 3 and 4. The smooth factorization condition is trivially satisfied in two cases: pseudo-- differential operators with k = 0 and the maximal rank case with k = n 1, where π λ0 is the projection along the conical direction. It turns out, that under some natural conditions on the canonical relation C, corresponding to the most important cases, it is extremely difficult to exhibit the failure of the smooth factorization condition. 5

6 Now, we will describe the geometric meaning of this condition. On a smooth submanifold Σ of the set Σ = π X Y (C) the canonical relation can be found back as the conormal bundle N Σ of Σ. The set Σ is the singular support of the Schwartz integral kernel of the operator T. The conormal bundle N Σ of Σ X Y is defined by N Σ = {(x, ξ, y, η) T (X Y ) : (x, y) Σ, ξ(δx) + η(δy) = 0, (δx, δy) T (x,y) Σ }. (1.9) Thus, the diagram N Σ C π X Y (1.10) Σ defines a smooth local fibration over Σ with affine fibers. In these terms the factorization condition becomes equivalent to the condition that the smooth fibration of N Σ allows a smooth extension to C. This smooth extension is defined by the levels of the mapping π λ0. In Sections 3 and 4 we will analyze this property with different degrees of smoothness of fibrations. However, the theory becomes much more subtle if we assume that the described fibration over Σ is analytic. The analyticity assumption is quite natural and is almost always satisfied. In fact, if a critical point of a phase function has finite order of degeneracy, then the phase function is actually even a polynomial in a suitably chosen coordinate system in a neighborhood of the critical point. On the other hand, there are very few infinitely degenerate critical points, because the coefficients of the Taylor expansion of such phase function have to satisfy infinitely many independent algebraic equations. Another reason is that the study of Fourier integral operators is often reduced to the asymptotics of the oscillatory integrals corresponding to the associated Lagrangian distributions. However, the asymptotic behavior (as λ ) of the oscillatory integrals e λφ ψ depends essentially on the first terms of Taylor expansion, namely the power of the highest order term of the asymptotic expansion corresponds to the first non-vanishing term of the Taylor expansion ([20], [2]). In particular, the analyticity assumption is satisfied for the propagation operators of hyperbolic partial differential equations with analytic coefficients. The properties of the oscillatory integrals with analytic phase functions were studied in, for example, [2]. The corresponding geometric constructions can be found in [33], [4]. The singularities of wave fronts are analyzed in [1]. The smooth factorization condition is called the holomorphic factorization condition if all mappings in (1.8) are analytic (and hence holomorphic after the continuation to a complex domain). We will show, that under the analyticity assumption the holomorphic factorization condition holds in the most important cases. Some results of this type can be found in [50] and [51]. There are alternative presentations of Lagrangian distributions which make use of complex-valued phase functions. Fourier integral operators associated to such functions have been studied in [38] and global representations of Lagrangian distributions were obtained in [36]. A survey of the main theory is in [4]. The converse sharpness results for the wave-type equations can be found in [40], [43], [56] for the case when rank dπ X Y C = 2n 1 somewhere. For essentially homogeneous symbols (Sρ μ with ρ = 1) we will generalize the sharpness results to arbitrary ranks. We 6

7 will show that the order k 1/p 1/2 is sharp for all elliptic Fourier integral operators provided rank dπ X Y C n + k. As a consequence it follows that elliptic operators of small negative orders which are continuous in L p or from L p to L q can be obtained as a composition of pseudo-differential operators with Fourier integral operators induced by a smooth coordinate change. Some results of this type appeared in [52] and we will describe them in Section 7. There, we will also mention the case of an arbitrary ρ where the sharpness of the orders for general elliptic operators under a rank restriction condition for the projection π X Y C is not settled in general. In Section 5 we will provide examples of the failure of the factorization condition in general and for the canonical relations corresponding to the translation invariant operators in R n. Now we will briefly describe the smooth factorization condition in terms of the phase function of a Fourier integral operator. From now on, we will replace the frequency variable θ by ξ in the cases when the dimensions of X, Y and the frequency space Ξ coincide. By the equivalence-of-phase-function theorem (Theorem 2.3 below), we can assume that a phase function Φ of an operator T I μ (X, Y ; C) is of the form Φ(x, y, ξ) = x, ξ φ(y, ξ). Therefore, the corresponding wave front is given by Λ Φ = {( ξ φ(y, ξ), ξ, y, y φ(y, ξ))} and C = Λ Φ = {(x, ξ, y, η) : (x, ξ, y, η) Λ Φ}. The local graph condition is equivalent to det φ yξ(y, ξ) 0 (1.11) on the support of the symbol of the operator T. The mapping γ(y, ξ) = Y Ξ (y, ξ) ( ξ φ(y, ξ), ξ, y, y φ(y, ξ)) T X T Y defines a diffeomorphism from Y Ξ to C. The level sets of the mapping π X Y : C X Y correspond to the kernels of the linear mapping dπ X Y C, or to the kernels of the mapping dπ X Y dγ. It is straightforward that ker dπ X Y dγ(y, ξ) = (0, ker 2 φ (y, ξ)). ξ2 Therefore, fibration (1.11) reduces to the fibration defined by the kernels ker φ ξξ (y, ξ), or by the level sets of the mapping (y, ξ) ξ φ(y, ξ) on the set where the rank of φ ξξ is maximal. A simple example of the failure of the factorization condition is already possible in R 3. The fibers of the function φ(y, ξ) = y, ξ + 1 ξ 3 (y 1 ξ 1 + y 2 ξ 2 ) 2, (1.12) (i.e. the level sets of φ(y, ξ) with respect to ξ) are straight lines with the slope equal to y 2 /y 1. It is clear that the corresponding fibration is not continuous at zero. In the case when a phase function is real analytic we will concentrate on its complex extension and on the holomorphic mapping (y, ξ) ξ φ(y, ξ). Let us discuss the corresponding singularity problem in terms of affine fibrations in more detail. 7

8 1.2 Fibrations with affine fibers First, we formulate the problem in the invariant case corresponding to the Fourier integral operators commuting with translations. In this case we have φ(y, ξ) = y, ξ H(ξ). Functions H : V R on an open subset V of R n have the following property. The maximal rank k of the Hessian D 2 H(ξ) is strictly less than n and points ξ where rank D 2 H(ξ) = k is maximal form an open set U. One of the interesting properties of the gradient Γ : ξ H(ξ) is that for every point ξ U the level set Γ 1 (Γ(ξ)) locally in a neighborhood of ξ coincides with the affine space ξ + ker D 2 H(ξ). If H is real analytic, then the holomorphic extension Γ of the gradient H has the same property in some open neighborhood of the open set V in C n. This property motivates the study of holomorphic mappings Γ with properties (A1) (A3) formulated below. Let Γ be a holomorphic mapping from a connected open subset Ω of C m C n to C p. Let DΓ(ξ) C p n denote the Jacobian of Γ. The rows of DΓ have the form D ξ Γ i (ξ) = ( ξ Γ i ) T for 1 i p. Let k be such that (A1) rank DΓ(ξ) k for all ξ Ω. (A2) ξ Ω with rank DΓ(ξ) = k. Because we will be interested in the level sets of the mapping Γ, we may assume k n 1. The set Ω can be stratified in the union of disjoint sets Ω (i) consisting of points ξ Ω with rank DΓ(ξ) = i, i = 0,..., k. The set Ω\Ω (k) where the rank of DΓ(ξ) < k is an analytic subset of Ω. The mapping κ : ξ ker DΓ(ξ) (1.13) is holomorphic from the open dense in Ω set Ω (k) to the Grassmannian G n k (C n ). Let Ω sing denote the set of essential singularities of the mapping κ, i.e. the set of points ξ Ω\Ω (k) such that κ does not allow a holomorphic extension over ξ. The mapping κ is regular (holomorphic) in ξ Ω (k) and such points are called regular. The set Ω\Ω (k) consists of essential singularities (points of Ω sing ) and removable singularities in Ω\(Ω (k) Ω sing ). An additional assumption on the mapping Γ will be that the affine spaces ξ + κ(ξ) define a fibration in Ω (k). Let us explain this in more detail. Let ξ Ω (k). Then, by the implicit function theorem, the level sets Γ 1 (Γ(ξ)) are smooth analytic manifolds of dimension n k. The tangent plane to the level set Γ 1 (Γ(ξ)) in the point ξ is given by ξ + κ(ξ). The condition that κ defines a fibration means that each level set coincides with its tangent space at ξ, for all ξ Ω (k). (A3) For every ξ Ω (k) there exists a neighborhood U of ξ such that the affine space ξ + κ(ξ) coincides with the level set Γ 1 (Γ(ξ)) in U. Thus, Γ is constant on ξ + κ(ξ) which is equivalent to saying that κ(ξ + ζ) = κ(ξ) for all ξ Ω (k) and ζ κ(ξ) with ξ + ζ Ω (k) (cf. Proposition 3.18 below). Because we are interested in local properties of the mapping κ, we will always assume that Ω is convex. In this case the set (ξ + κ(ξ)) Ω is connected and Γ is constant on (ξ + κ(ξ)) Ω. It also means that κ(ξ) = κ(η) for all η (ξ + κ(ξ)) Ω (k). Therefore, condition (A3) implies its global form: (A3 ) For every ξ Ω (k), the affine space (ξ + κ(ξ)) coincides with the level set Γ 1 (Γ(ξ)) in the whole of Ω. 8

9 The mapping κ is holomorphically extendible to the open set Ω\Ω sing containing Ω (k). This extension will be also denoted by κ. Thus, for every ξ Ω\Ω sing, Γ is also constant on (ξ + κ(ξ)) Ω. And, if ξ, η Ω\Ω sing, then (ξ + κ(ξ)) (η + κ(η)) (Ω\Ω sing ) is empty, or, if not, κ(ξ) = κ(η). This enables us to introduce an equivalence relation in Ω\Ω sing. Let us say that the points ξ and η are equivalent if η ξ + κ(ξ). The element κ(ξ) = κ(η) defines their common equivalence class. The factor space (Ω\Ω sing )/ is a smooth analytic manifold of dimension k and the projection ξ (ξ +κ(ξ)) (Ω\Ω sing ) is an analytic submersion. In this sense spaces (ξ + κ(ξ)) (Ω\Ω sing ) with ξ Ω\Ω sing define a smooth fibration in Ω\Ω sing. The mapping Γ factorizes through this fibration (see the beginning of section 3.3). In view of this, the mapping κ itself will be sometimes called the fibration. The simplest singular fibration can be defined for Ω = C n by taking for one dimensional fibers open straight rays emanating from the origin in radial directions. It is clear that this fibration is regular (analytic) in all points except the origin. The set of its essential singularities is Ω sing = {0}. However, we will show in Section 3 that in this case there exists no holomorphic mapping Γ for which the described rays are given by the kernels of the Jacobian DΓ (even locally). Note that conditions (A1) (A3) can be formulated in greater generality. In the sequel, we will show that the mapping κ is meromorphic and this allows to put the problem in a general framework of meromorphic mappings. For more generality, one can work with morphisms κ from an analytic set X to a (compact) analytic set Y, such that κ is holomorphic on the complement in X to an analytic set of strictly lower dimension. However, in this paper we are primarily interested in the properties of the mapping κ for a given fixed mapping Γ. This is the reason for us to avoid the general category language and to restrict to the local properties of κ. 1.3 Formulation for the parametric fibrations Let us briefly consider a more general problem allowing y dependence. Let Γ be a holomorphic mapping from an open connected set Ω C m C n to C p. Let k < n be the maximal rank of the Jacobian D ξ Γ(y, ξ) in Ω, (R1 R2) max (y,ξ) Ω rank D ξ Γ(y, ξ) = k. Let Ω (k) be the set of points (y, ξ) where the maximal rank k is attained. It is open and dense in Ω. The mapping κ : (y, ξ) ker D ξ Γ(y, ξ) is holomorphic from Ω (k) to the Grassmannian G n k (C n ) of (n k)-dimensional linear subspaces of C n. Let Ω sing denote the set of the essential singularities of κ, i.e. the set of points (y, ξ) Ω such that the mapping κ does not allow a holomorphic extension to any neighborhood of (y, ξ) in Ω. It is clear that the sets Ω (k) and Ω sing are disjoint. The following condition guarantee the linearity of the level sets of Γ with respect to ξ: (R3) For every (y, ξ) Ω (k) the affine space (y, ξ) + (0, κ(y, ξ)) locally coincides with the level set Γ 1 (Γ(y, ξ)) through the point (y, ξ). Similar to conditions (A1) (A3), (R3) means that κ defines a local holomorphic fibration in Ω\Ω sing. Example (1.12) with Γ = φ shows that in general fibrations can 9

10 have essential singularities. If we fix a value of y or if we take Γ independent of y, then conditions (R1) (R3) are equivalent to conditions (A1) (A3) above. It turns out that simple examples as in (1.12) are impossible in the problem (A1) (A3) and the analysis becomes more interesting. For example, one of the necessary conditions for an analytic set Ω sing to be the set of the essential singularities of a mapping κ associated to a holomorphic mapping Γ is the following dimension estimate: max{k 1, n k + 1} dim ξ Ω sing n 2. In particular, Ω sing can not contain isolated points. There is a number of interesting problems related to fibrations arising in this way. Thus, for a given fibration κ 0 in an open dense set Ω 0 in Ω, we would like to determine whether there exists a holomorphic mapping Γ satisfying conditions (A1) (A3), such that the fibration defined by κ coincides with the fibration defined by κ 0 in Ω 0 Ω (k). The construction of a phase function φ for a given Γ = ξ φ leads to further complications. In Section 4 we will derive necessary and sufficient conditions in terms of a system of partial differential equations with coefficients corresponding to a given fibrations. However, the regularity (analyticity) of the fibration does not immediately imply that solutions of the constructed system of differential equations are sufficiently regular. Moreover, this system depends on the choice of a local coordinate system in the Grassmannian. It would be interesting to obtain an invariant description of the results of section 4 as well as their generalization to spaces of higher dimensions. The results of section 3 (for example, Theorems ) describe possible dimensions of the set Ω sing under conditions (A1) (A3). They also give some understanding of its structure. It would be interesting to investigate its further properties, especially in the case of gradient fibrations. For example, in Section 4.4, we will give examples of fibrations of gradient type for which Ω sing is not empty. However, in all our examples, the set Ω sing is affine and its dimension is equal to n 2. It is not clear whether the condition dim Ω sing = n 2 is necessary for the fibrations of gradient type. In view of the similarity of two problems described above we will use the same notations in their analysis. In order to eliminate any confusion, we will always consider problem (A1) (A3) unless we explicitly state otherwise. 1.4 Several applications One of the main applications of the theory of Fourier integral operators is the theory of hyperbolic partial differential equations. Let P (t, x, t, x ) = m t + m j=1 P j (t, x, x ) m j t be a strictly hyperbolic operator of order m in a set of points (t, x) R 1+n. The hyperbolicity means that the principal symbol p(t, x, τ, ξ) of the operator P (p is polynomial in τ or degree m and is often denoted by σ P ) has m distinct real roots τ j in τ. In this case, the Cauchy problem { P u(t, x) = 0, t 0, j (1.14) t u t=0 = f j (x), 0 j m 1. 10

11 is well posed, and for small t its solution can be written as a sum of Fourier integral operators T jl t : m m 1 u(t, ) = T jl t f l, j=1 l=0 where each operator T jl t depends smoothly on t, is of order l and depends on the root τ j (t, x, ξ). This construction will be described in more detail in section 6. Note that operators T jl t satisfy the local graph condition and it follows from some of the results below (for example, Theorem 2.9 and [56]) that for small t and 1 < p < for the solution u of the Cauchy problem holds u(t, ) L p loc provided that the Cauchy data f l are in Sobolev spaces f l (L p α p l ) comp, where α p = (n 1) 1/p 1/2. Thus, there is a loss of smoothness by α p derivatives in L p spaces. It follows from the stationary phase method that the loss of α p derivatives is sharp in a number of cases. For example, it is sharp for equations of the wave type with variable coefficients (when m = 2). It is also sharp if one imposes an additional condition that for almost all t at least one of the roots τ j (t, x, ξ) is elliptic in ξ, i.e. τ j (t, x, ξ) 0 for ξ 0. However, in a number of cases the order α p can be improved. Let k denote the minimal integer for which the ranks of standard projections from the canonical relations of operators T jl t to R n R n do not exceed n + k for all j and l. These projections are discussed in more detail in Section 1.1. In the present case, projection π X Y in (1.7) satisfies the inequality rank dπ X Y C jl n + k, where X and Y are (open subsets of) R n and C jl t are the t canonical relations of operators T jl t. It terms of phase functions it means that if Φ j is the solution of the eikonal equation ( / t)φ j (t, x, ξ) = τ j (t, x, x Φ j ) with initial condition Φ j (0, x, ξ) = x, ξ, then rank ξξ 2 Φ j(t, x, ξ) k for all j, x, ξ. In this case one can show that the order α p = k 1/p 1/2 would be sharp, i.e. the condition that for some α holds u(t, ) L p loc for all f l (L p α l ) comp implies α α p = k 1/p 1/2. The converse is known in a number of special cases only. For operators with constant coefficients (in particular) the converse result will be proved in section 6 for k 2. The complete picture of the L p properties of such operators becomes clear in the 5- dimensional space. We will describe it in section 6. For 3 k n 2 the sharp L p are not proved in general. In the case k = 0 there is no loss of smoothness for the solutions (in this case the regularity properties of solutions are the same as the regularity properties of solutions to the elliptic Cauchy problems, α p = 0 and, therefore, k = 0), the principal symbol of P has a special form. Its roots τ j are linear in ξ (cf. Theorem 6.4). Another important applications of Fourier integral operators are convolution operators and Radon transforms. Let X and Y be smooth manifolds and let S be a smooth submanifold of X Y. For x X, let S x be the set of points y Y such that (x, y) S. Let σ be a smooth measure on S. It naturally induces the measure σ x on S x. The Radon transform corresponding to S and σ is now defined by Rf(x) = fdσ x. S x It defines an integral operator R : C 0 (Y ) D (X). The Schwartz integral kernel of the operator R is a Dirac measure supported in S and equal to σ there. Because we are interested in local properties, we can assume that the set S is compact. Let 11

12 dim S = dim X + dim Y d. Locally S is given by d equations h 1 (x, y) =... = h d (x, y) = 0, where functions h j are smooth and their gradients are linearly independent on S. In this way, the operator R can be regarded as a Fourier integral operator with the phase function d θ j h j (x, y). j=1 The canonical relation of R is then equal to the conormal bundle N S. The local graph condition means that dim X = dim Y and that projections π X, π Y from N S to T X and T Y are locally diffeomorphic. In this case the L p estimates of the subsequent sections can be applied. The smooth factorization condition is satisfied for N S because S is a smooth manifold. Therefore, the L p estimates follow from the general theory of Fourier integral operators. The results depend on the order of the operator R as a Fourier integral operator. Let us assume for simplicity that Rf = f σ, where the measure σ is supported on some (smooth) submanifold Σ of R n. The connection between R and Fourier integral operators will be described in detail in Section 5.2. Now, we restrict ourselves to some remarks only. The order of R as a Fourier integral operator is related to the order of decrease of the Fourier transform σ at infinity. This order depends on the curvature of Σ. Let Σ be a hypersurface (dim Σ = n 1) with nonvanishing Gaussian curvature at all points and let dσ = ψdμ, where dμ is induced on S by the Lebesgue measure and ψ C0 (R n ). One can show that the Fourier transform of the measure σ satisfies the estimate σ(ξ) C ξ (n 1)/2. It follows that R is a Fourier integral operator of order (n 1)/2. There are generalizations of this estimate to (n d) dimensional manifolds Σ of finite type. Roughly speaking, Σ has type m if the order of intersection of Σ with affine hyperplanes does not exceed m. In this case holds σ(ξ) C ξ 1/m. If the manifold Σ is analytic, then, in turn, this estimate implies that Σ has type m. Indeed, otherwise σ would be constant in directions orthogonal to the tangent space to Σ at the points of infinite order of tangency. The detailed discussion as well as other forms of the Radon transform can be found, for example, in [60], [27], [44], [45], [47]. For the failure of the smooth factorization condition it is necessary (but not sufficient, see Remark 2.23) that S is not smooth. In this case there are additional problems in determining the order of R as a Fourier integral operator since the estimates of σ at infinity are more complicated. The local graph condition holds in a number of important cases. For example, Radon transforms along hypersurfaces in R n satisfy this condition. It also holds for the convolution operators with measures supported on hypersurfaces in R n with nonvanishing Gaussian curvature. As an example, one has solution operators to the Cauchy problem for the wave equation (see above), where for a fixed t > 0, the manifold S x consists of points y R n with x y = t. Let us remark, finally, that in some applications the canonical relation fails to be a local graph. For instance, the local graph condition fails for the Radon transforms with d > n/2 (cf., for example, [44]). For submanifolds of codimension larger than 1 in a general position, the canonical relation C is a Whitney fold ([61], [46], [45]). Projections π X and π Y are also singular in applications to the diffraction theory ([64], [65]), in the theory of X-ray transforms ([26]), and deformation theory ([28], [26]). Singular Radon 12

13 transforms and Fourier integral operators with densities with Calderón Zygmund type of singularities led to the microlocal analysis of the boundedness for the degenerate type of Fourier integral operators (see [29], [27], [45]) and references therein). In many degenerate cases one looses smoothness of functions even in the L 2 case, with the loss dependent on the order of degeneracy, which can be expressed in terms of the stratification of Lagrangian ([46], [47]). A similar loss occurs in L p norms for the Radon transforms with folding canonical relation and for other types of singularities. However, such degenerate cases fall beyond the scope of the present paper. The bibliography of the paper was selected with the purpose of clarifying some of the aspects of the regularity theory of Fourier integral operators and related geometric problems and it does not pretend to be complete. Many results of this paper were obtained by the author during his work on the doctoral dissertation at Utrecht University. I would like to use the opportunity to thank professor Hans Duistermaat for numerous discussions and his remarks. 2 Estimates for Fourier integral operators 2.1 Fourier integral operators In the sequel, the local properties will be of primary concern. However, global constructions often help to give a better insight in problems at hand. The intrinsic global characterization of Fourier integral operators was systematically developed in [32], [20], [33]. Excellent expositions of the theory can be found in [4], [5]. Many important to us notions are described in [9]. The global theory is based on constructions of the symplectic geometry. Let us recall main definitions. Let M be a smooth real manifold. A form ω is called symplectic on M if it is a 2-form on M such that dω = 0 and such that for each m M the bilinear form ω m is antisymmetric and nondegenerate on T m M. The pairs (T m M, ω m ) and (M, ω) are called the symplectic vector space and the symplectic manifold, respectively. Let X be a smooth real manifold of dimension n. The canonical symplectic form σ on the cotangent bundle T X of X can be introduced as follows. Let π : T X (x, ξ) x X be the canonical projection. Then for (x, ξ) T X the mappings Dπ (x,ξ) : T (x,ξ) (T X) T x X and ξ : T x X R are linear. Their composition α (x,ξ) = ξ Dπ (x,ξ) (2.1) is a 1-form on T X. Its derivative σ = dα is called the canonical 2-form on T X and it follows that σ is symplectic. This form corresponds to the form n j=1 dp j dq j in mechanics, with possible change of sign. It can be shown that any symplectic form takes the latter form in symplectic coordinates. The same objects can be introduced on complex analytic manifolds. A submanifold Λ of T X is called Lagrangian if (T (x,ξ) Λ) σ = T (x,ξ) Λ, where (T (x,ξ) Λ) σ = {p T (x,ξ) (T X) : σ(p, q) = 0 q T (x,ξ) Λ}. In particular, this implies dim Λ = n. A submanifold Λ of T X\0 = {(x, ξ) T X : ξ 0} is called conic if (x, ξ) Λ implies (x, τξ) Λ for all τ > 0. Let Σ X be a smooth submanifold of X of dimension k. Its conormal bundle in T X is defined by N Σ = {(x, ξ) T X : x Σ, ξ(δx) = 0, δx T x Σ}. 13

14 The proofs of the subsequent statements can be found in [4], [5], [33]. We mainly follow [22]. Proposition 2.1 (1) Let Λ T X\0 be a closed submanifold of dimension n. Then Λ is a conic Lagrangian manifold if and only if the form α in (2.1) vanishes on Λ. (2) Let Σ X be a submanifold of dimension k. Then its conormal bundle N Σ is a conic Lagrangian manifold. (3) Let Λ T X\0 be a conic Lagrangian manifold and let Dπ (x,ξ) : T (x,ξ) Λ T x X have constant rank equal to k for all (x, ξ) Λ. Then each (x, ξ) Λ has a conic neighborhood Γ such that (a) Σ = π(λ Γ) is a smooth submanifold of X of dimension k, (b) Λ Γ is an open subset of N Σ. In the sequel we will mainly deal with conic Lagrangian manifolds and we will need their local representations. For this purpose, we consider a local trivialization X (R n \0) of T X\0, where we can assume X to be an open set of dimension n. However, in the sequel we will also need a slight generalization of it, so that we allow the dimensions of X and the fibers differ. Thus, let Γ be a cone in X (R N \0). A smooth function φ : X (R N \0) R is called a phase function, if it is homogeneous of degree one in θ and has no critical points as a function of (x, θ): φ(x, τθ) = τφ(x, θ) for τ > 0 and d (x,θ) φ(x, θ) 0 for all (x, θ) X (R N \0). A phase function is called φ(x,θ) nondegenerate in Γ if (x, θ) Γ, d θ φ(x, θ) = 0 imply that d (x,θ) θ j are linearly independent for j = 1,..., N. Proposition 2.2 (1) Let Γ be a cone in X (R N \0) and let φ be a nondegenerate phase function in Γ. Then there exists an open cone Γ Γ such that the set C φ = {(x, θ) Γ : d θ φ(x, θ) = 0} is a smooth conic submanifold of X (R N \0) of dimension n. The mapping T φ : C φ (x, θ) (x, d x φ(x, θ)) T X\0 is an immersion, commuting with the multiplication with positive real numbers in the fibers. Let us denote Λ φ = T φ (C φ ). (2) Let Λ be a submanifold of T X\0 of dimension n. Then Λ is a conic Lagrangian manifold if and only if every (x, ξ) Λ has a conic neighborhood Γ such that Λ Γ = Λ φ for some nondegenerate phase function φ. Naturally, the cone condition for Λ corresponds to the homogeneity of φ. Let Λ be a closed conic Lagrangian submanifold of T X\0. A distribution u is called the Lagrangian distribution of order m associated to Λ, u I m (X, Λ), if n j=1 P j u H m n/4 loc (X), (2.2) 14

15 whenever P j Ψ 1 (X) are properly supported pseudo-differential operators whose principal symbols p j (x, ξ) vanish on Λ and H m n/4 loc is the localization of the usual Besov space. First, a distribution u S (R n ) belongs to the Besov space H σ (R n ), if û L 2 loc (Rn ) and u H σ(r n ) = ( 1/2 û(ξ) dξ) 2 + sup ξ 1 j 0 ( 2 j ξ 2 j+1 2 σj û(ξ) 2 dξ H loc ) 1/2 <. For a smooth manifold X of dimension n the space σ (X) is defined to be the space of all u D (X) such that (ψu) κ 1 is in H σ (R n ) whenever Ω X is a coordinate patch with coordinates κ and ψ C0 (Ω). More details can be found in [33, 25.1] and [57, 6.1]. Let now X, Y be open in R n and let Φ be a nondegenerate phase function. Let a Sρ m (X Y R N ) be a symbol of type ρ and order m, which means that 0 ρ 1, a C (X Y R N ), and for every compact subset K of X Y and any multi-indices α, β holds x,y β ξ α a(x, θ) C(α, β, K)(1 + θ ) m ρ α +(1 ρ) β for all (x, y) K and θ R N \0. Operators of the form T u(x) = e iφ(x,y,θ) a(x, y, θ)u(y)dθdy. (2.3) Y R N are called Fourier integral operators. Expression (2.3) can be understood in the classical sense if m + N < 0 and u C0 (X), when the integral is absolutely convergent. The integral kernel of (2.3) is equal to K(x, y) = e iφ(x,y,θ) a(x, y, θ)dθ. (2.4) R N According to Proposition 2.2, the set Λ φ = {(x, y, d x φ(x, y, θ), d y φ(x, y, θ)) : d θ φ(x, y, θ) = 0} is a conic Lagrangian submanifold of T (X Y )\0 of dimension 2n. It follows that for ρ = 1, the kernel (2.4) is a Lagrangian distribution in X Y associated to Λ Φ of order μ = m n/2 + N/2, K I μ (X Y, Λ Φ ). Conversely, any Lagrangian distribution K I μ (X Y, Λ) can be microlocally written in the form (2.4) modulo C, with ρ = 1 (cf. [57, teorema 6.1.4]). The kernel (2.4) is also called the Fourier integral distribution. In general, Iρ μ (X, Λ) will denote the space of Fourier integral distributions consisting of Fourier integral distributions u(x) = e iφ(x,θ) a(x, θ)dθ, (2.5) R N with Λ locally equal to Λ φ as in Proposition 2.2, a S m ρ (X R N ), ρ > 1/2, and μ = m n/4 + N/2. The behavior of the integral (2.3) can be independent of some of the variables θ and the order of T is taken to be μ = m + (N n)/2, which is the order of its Lagrangian distribution (in this case we have X Y instead of X). The following theorem describes 15

16 the family of phase functions corresponding to the same Fourier integral distribution (2.4) (Theorem in [22]). We formulate it for the general case of I μ ρ (X, Λ) as in (2.5), rather than for a particular case I μ ρ (X Y, Λ). Theorem 2.3 Suppose φ(x, θ) and φ(x, θ) are nondegenerate phase functions at (x 0, θ 0 ) X (R N \0) and at (x 0, θ 0 ) X (RÑ \0), respectively. Let Γ and Γ be open conic neighborhoods of (x 0, θ 0 ) and (x 0, θ 0 ) such that T φ : C φ Γ φ and T φ : C φ Γ φ are injective, respectively. If Λ φ = Λ φ, then any Fourier integral distribution, defined by the phase function φ and an amplitude a S m ρ (X R N ), ρ > 1/2, with ess supp a contained in a sufficiently small conic neighborhood of (x 0, θ 0 ), is equal to a Fourier integral distribution defined by the phase function φ and an amplitude ã S m+ 1 2 (N Ñ) ρ (X RÑ ). In particular, the phase function Φ of the operator T in (2.3) can be always written in the form Φ(x, y, ξ) = x, ξ ψ(y, ξ), with some function ψ and ξ R n. This fact will be often used in the sequel. The function ψ (as well as Φ) is also called the generating function for Λ Φ. Thus, the notion of Fourier integral operator becomes independent of a particular choice of a phase function associated to the Lagrangian manifold Λ. The set C = Λ = {((x, ξ), (y, η)) T X T Y : (x, y, ξ, η) Λ} is a conic Lagrangian manifold in T X\0 T Y \0 with respect to the symplectic structure σ X σ Y and it is called a homogeneous canonical relation from T Y to T X. The space of integral operators with distributional kernels in Iρ μ (X Y, Λ) will be denoted by Iρ μ (X, Y ; C) and it is the space of Fourier integral operators associated to the canonical relation C = Λ. 2.2 Function spaces In this section we will briefly discuss function spaces, which will be used throughout this paper. Let X be a smooth manifold with a measure λ. For 1 p < by L p (X) we will denote the usual space (of the equivalence classes) of functions f on X with finite norm f p = ( X f p dλ) 1/p. For 0 < p < 1 this expression fails to be a norm and the substitute for L p (X) in the analysis of singular integrals are Hardy spaces H p (X). The general theory of complex and real variable versions of Hardy spaces can be found in [59], [62], [25], [60], where one can also find proofs of subsequent statements of this section. Since our interest are the local properties, we restrict to the real Euclidean case of H p (R n ). Let S be the Schwartz space of smooth rapidly decreasing functions, equipped with a countable family of seminorms φ α,β = sup x R n x α β x φ(x). Let Φ S and for t > 0 define Φ t (x) = t n Φ(x/t). Then for a distribution f the convolutions f Φ t are smooth and one defines the maximal operator M Φ f(x) = sup (f Φ t )(x). t>0 16

17 Let F be a finite collection of seminorms on S and one defines S F = {Φ S : Φ α,β 1 for all α,β F}. The maximal operator associated to the family S F, determining the approximation of the identity, is now defined by M F f(x) = sup Φ S F M Φ f(x). For the definition of the Hardy space H p (R n ), we take the space of functions, satisfying one of the following equivalent conditions. Proposition 2.4 Let f be a distribution and let 0 < p. conditions are equivalent: Then the following (1) There is a function Φ S with Φdx 0 so that M Φ f L p (R n ). (2) There is a collection F so that M F f L p (R n ). The expression f H p = M Φ f L p can be taken to be the norm of H p (R n ). Note, that it is equivalent to M F L p and it is actually a norm only if p 1. For 0 < p < 1, the topology of H p can be defined by the metric d(f, g) = f g p H p. In the case 1 < p, a simple argument shows that H p (R n ) coincide with the Lebesgue spaces L p (R n ). However, already for p = 1 one only has H 1 (R n ) L 1 (R n ). On the other hand, if f L 1 comp(r n ) satisfies the moment condition fdx = 0 (which is in fact necessary for f to belong to H 1 (R n )), then f L q (R n ) for any q > 1 implies f H 1 (R n ). One often makes use of an atomic decomposition of Hardy spaces, similar to the classical Calderón-Zygmund decomposition. For 0 < p 1, an H p atom is a function a such that (1) a is supported in a ball B, (2) a B 1/p almost everywhere, (3) x α a(x)dx = 0 for all α with α n(p 1 1). An H p atom belongs to H p (R n ) with uniform bound and to L p (R n ) with a(x) p dx 1, which follows from (1) and (2). Proposition 2.5 Let 0 < p 1. (1) Let a k be a collection of H p atoms and let λ k C satisfy k λ k p <. Then the series f = λ k a k (2.6) k converges distributionally, its sum f belongs to H p (R n ), and ( ) 1/p f H p c λ k p. k 17

18 (2) Let f H p (R n ). Then f can be written as a sum of H p atoms as in (2.6), which converges in H p norm. Moreover, ( ) 1/p λ k p c f H p. k Let X be an open subset of R n. For 0 < γ < 1 the Lipschitz (Hölder) space Lip (X, γ) consists of functions f for which there exists a constant A such that f(x) A almost everywhere and sup f(x y) f(x) A y γ x holds for all y with x y X. Minimal A satisfying these two inequalities can be taken to be the norm of f (cf. [58], [66], [16]). The Hardy space H 1 plays an important role in the complex interpolation method. Proposition 2.6 Let T z be a family of linear operators on R n, parameterized by complex z with 0 Re (z) 1. Suppose that for all simple (step-) functions f, g, vanishing outside a set of finite measure, the map z (T z f)gdx R n is bounded and analytic in the open strip 0 < Re (z) < 1 and is continuous in its closure. Suppose that T z f L 1 C 0 f H 1 for Re (z) = 0 and T z f q1 C 1 f p1 for Re (z) = 1. Then also T t f qt C 1 t 0 C t 1 f pt (2.7) with p t and q t defined by 1/p t = (1 t) + t/p 1 and 1/q t = (1 t) + t/q 1. The proof is based on the duality between H 1 and BMO ([25]). Proposition 2.7 (Hardy-Littlwood-Sobolev) For every 0 < γ < n, 1 < p < q < and 1/q = 1/p (n γ)/n, there exists a constant A pq such that f ( y γ ) L q A pq f L p. There is a similar result in Hardy spaces ([60, III.5.21]). Proposition 2.8 The operator I α f = f ( y γ ) allows an analytic extension on the set n(p 1 1) Re γ < n, when x α f(x)dx = 0 hold for α n(p 1 1) and p 1. For every 0 < p < q < and 1/q = 1/p (n γ)/n, there exists a constant A pq such that f ( y γ ) H q A pq f H p. The same result holds for q =, p 1 and 0 < p = q <, Re γ = n. The development of the complex Hardy space theory can be traced in [68] for C and in [23], [58], [62] for C n. The real theory in terms of maximal operators and Calderon- Zygmund decomposition can be found in [25]. Applications of Hardy spaces to several problems of the theory of singular integral operators appeared already in [30]. The general theory and applications can be found in [58], [62] and [60]. 18