February 12, :52 WSPC/Book Trim Size for 9in x 6in book. Publishers page

Transcription

1 Publishers page

2 Publishers page

3 Publishers page

4 Publishers page

5 To the memory of my parents

6 vi

7 Preface This book presents analytic theory of random fields estimation optimal by the criterion of minimum of the variance of the error of the estimate. This theory is a generalization of the classical Wiener theory. Wiener s theory has been developed for optimal estimation of stationary random processes, that is, random functions of one variable. Random fields are random functions of several variables. Wiener s theory was based on the analytical solution of the basic integral equation of estimation theory. This equation for estimation of stationary random processes was Wiener-Hopf-type of equation, originally on a positive semiaxis. About 25 years later the theory of such equations has been developed for the case of finite intervals. The assumption of stationarity of the processes was vital for the theory. Analytical formulas for optimal estimates (filters) have been obtained under the assumption that the spectral density of the stationary process is a positive rational function. We generalize Wiener s theory in several directions. First, estimation theory of random fields and not only random processes is developed. Secondly, the stationarity assumption is dropped. Thirdly, the assumption about rational spectral density is generalized in this book: we consider kernels of positive rational functions of arbitrary elliptic selfadjoint operators on the whole space. The domain of observation of the signal does not enter into the definition of the kernel. These kernels are correlation functions of random fields and therefore the class of such kernels defines the class of random fields for which analytical estimation theory is developed. In the appendix we consider even more general class of kernels, namely kernels R(x, y), which solve the equation QR = P δ(x y). Here P and Q are elliptic operators, and δ(x y) is the delta-function. We study singular perturbation problem for the basic integral equation of estimation theory Rh = f. The solution to this equation, which is of interest vii

8 viii Random Fields Estimation Theory in estimation theory, is a distribution, in general. The perturbed equation, ɛh ɛ +Rh ɛ = f has the unique solution in L 2 (). The singular perturbation problem consists of the study of the asymptotics of h ɛ as ɛ 0. This theory is not only of mathematical interest, but also a basis for the numerical solution of the basic integral equation in distributions. We discuss the relation between estimation theory and quantum-mechanical non-relativistic scattering theory. Applications of the estimation theory are also discussed. The presentation in this book is based partly on the author s earlier monographs [Ramm (1990)] and [Ramm (1996)], but also contains recent results [Ramm (2002)], [Ramm (2003)],[Kozhevnikov and Ramm (2005)], and [Ramm and Shifrin (2005)]. The book is intended for researchers in probability and statistics, analysis, numerical analysis, signal estimation and image processing, theoretically inclined electrical engineers, geophysicists, and graduate students in these areas. Parts of the book can be used in graduate courses in probabilty and statistics. The analytical tools that the author uses are not usual for statistics and probability. These tools include spectral theory of elliptic operators, pseudodifferential operators, and operator theory. The presentation in this book is essentially self-contained. Auxiliary material which we use is collected in Chapter 8.

9 Contents Preface vii 1. Introduction 1 2. Formulation of Basic Results Statement of the problem Formulation of the results (multidimensional case) Basic results Generalizations Formulation of the results (one-dimensional case) Basic results for the scalar equation Vector equations Examples of kernels of class R and solutions to the basic equation Formula for the error of the optimal estimate Numerical Solution of the Basic Integral Equation in istributions Basic ideas Theoretical approaches Multidimensional equation Numerical solution based on the approximation of the kernel Asymptotic behavior of the optimal filter as the white noise component goes to zero A general approach ix

10 x Random Fields Estimation Theory 4. Proofs Proof of Theorem Proof of Theorem Proof of Theorems 2.4 and Another approach Singular Perturbation Theory for a Class of Fredholm Integral Equations Arising in Random Fields Estimation Theory Introduction Auxiliary results Asymptotics in the case n = Examples of asymptotical solutions: case n = Asymptotics in the case n > Examples of asymptotical solutions: case n > Estimation and Scattering Theory The direct scattering problem The direct scattering problem Properties of the scattering solution Properties of the scattering amplitude Analyticity in k of the scattering solution High-frequency behavior of the scattering solutions Fundamental relation between u + and u Formula for det S(k) and the Levinson Theorem Completeness properties of the scattering solutions Inverse scattering problems Inverse scattering problems Uniqueness theorem for the inverse scattering problem Necessary conditions for a function to be a scatterng amplitude A Marchenko equation (M equation) Characterization of the scattering data in the 3 inverse scattering problem The Born inversion Estimation theory and inverse scattering in R Applications 159

11 Contents xi 7.1 What is the optimal size of the domain on which the data are to be collected? iscrimination of random fields against noisy background Quasioptimal estimates of derivatives of random functions Introduction Estimates of the derivatives erivatives of random functions Finding critical points erivatives of random fields Stable summation of orthogonal series and integrals with randomly perturbed coefficients Introduction Stable summation of series Method of multipliers Resolution ability of linear systems Introduction Resolution ability of linear systems Optimization of resolution ability A general definition of resolution ability Ill-posed problems and estimation theory Introduction Stable solution of ill-posed problems Equations with random noise A remark on nonlinear (polynomial) estimates Auxiliary Results Sobolev spaces and distributions A general imbedding theorem Sobolev spaces with negative indices Eigenfunction expansions for elliptic selfadjoint operators Resoluion of the identity and integral representation of selfadjoint operators ifferentiation of operator measures Carleman operators Elements of the spectral theory of elliptic operators in L 2 (R r ) Asymptotics of the spectrum of linear operators Compact operators Basic definitions

12 xii Random Fields Estimation Theory Minimax principles and estimates of eigenvalues and singular values Perturbations preserving asymptotics of the spectrum of compact operators Statement of the problem A characterization of the class of linear compact operators Asymptotic equivalence of s-values of two operators Estimate of the remainder Unbounded operators Asymptotics of eigenvalues Asymptotics of eigenvalues (continuation) Asymptotics of s-values Asymptotics of the spectrum for quadratic forms Proof of Theorem Trace class and Hilbert-Schmidt operators Trace class operators Hilbert-Schmidt operators eterminants of operators Elements of probability theory The probability space and basic definitions Hilbert space theory Estimation in Hilbert space L 2 (Ω, U, P ) Homogeneous and isotropic random fields Estimation of parameters iscrimination between hypotheses Generalized random fields Kalman filters Appendix A Analytical Solution of the Basic Integral Equation for a Class of One-imensional Problems 325 A.1 Introduction A.2 Proofs Appendix B Integral Operators Basic in Random Fields Estimation Theory 337

13 Contents xiii B.1 Introduction B.2 Reduction of the basic integral equation to a boundary-value problem B.3 Isomorphism property B.4 Auxiliary material Bibliographical Notes 359 Bibliography 363 Symbols 371 Index 373

14 Chapter 1 Introduction This work deals with just one topic: analytic theory of random fields estimation within the framework of covariance theory. No assumptions about distribution laws are made: the fields are not necessarily Gaussian or Markovian. The only information used is the covariance functions. Specifically, we assume that the random field is of the form U(x) = s(x) + n(x), x R r, (1.1) where s(x) is the useful signal and n(x) is noise. Without loss of generality assume that s(x) = n(x) = 0, (1.2) where the bar denotes the mean value. If these mean values are not zeros then one either assumes that they are known and considers the fields s(x) s(x) and n(x) n(x) with zero mean values, or one estimates the mean values and then subtracts them from the corresponding fields. We also assume that the covariance functions U (x)u(y) := R(x, y), U (x)s(y) := f(x, y) (1.3) are known. The star stands for complex conjugate. This information is necessary for any development within the framework of covariance theory. We will show that, under some assumptions about functions (1.3), one can develop an analytic theory of random fields estimation. If the functions (1.3) are not known then one has to estimate them from statistical data or from some theory. In many applications the exact analytical expression for the covariance functions is not very important, but rather some general features of R or f are of practical interest. These features include, for example, the correlation radius. 1

15 2 Random Fields Estimation Theory The estimation problem of interest is the following one. The signal u(x) of the form (1.1) is observed in a domain R r with the boundary Γ. Assuming (1.2) and (1.3) one needs to linearly estimate As(x 0 ) where A is a given operator and x 0 R r is a given point. The linear estimate is to be best possible by the criterion of minimum of variance, i.e. the estimate is by the least squares method. The most general form of a linear estimate of u observed in the domain is LU := h(x, y)u(y)dy (1.4) where h(x, y) is a distribution. Therefore the optimal linear estimate solves the variational problem ɛ := (LU As) 2 = min (1.5) where Lu and As are computed at the point x 0. A necessary condition on h(x, y) for (1.5) (with A = I) to hold is (see equations (2.11) and (8.423)) Rh := R(x, y)h(z, y)dy = f(x, z), x, z := Γ. (1.6) The basic topic of this work is the study of a class of equations (1.6) for which the analytical properties of the solution h can be obtained, a numerical procedure for computing h can be given, and properties of the operator R in (1.6) can be studied. Since z enters (1.6) as a parameter, one can study the basic equation of estimation theory Rh := R(x, y)h(y)dy = f(x), x. (1.7) A typical one-dimensional example of the equation (1.7) in estimation theory is 1 1 exp( x y )h(y)dy = f(x), 1 x 1. (1.8) Its solution of minimal order of singularity is h(x) = ( f +f)/2+δ(x+1)[ f ( 1)+f( 1)]/2+δ(x 1)[f (1)+f(1)]/2. (1.9) One can see that the solution is a distribution with singular support at the boundary of the domain. By sing supp h we mean the set having no open neigborhood to which the restriction of h can be identified with a locally integrable function.

16 Introduction 3 In the case of equation (1.8) this domain = ( 1, 1). Even if f C () the solution to equations (1.7), (1.8) are, in general, not in L 2 (). The problem is: in what functional space should one look for the solution? Is the solution unique? oes the solution to (1.7) provide the solution to the estimation problem (1.5)? oes the solution depend continuously on the data, e.g. on f and on R(x, y)? How does one compute the solution analytically and numerically? What are the properties of the solution, for example, what is the order of singularity of the solution? What is the singular support of the solution? What are the properties of the operator R as an operator in L 2 ()? These questions are answered in Chapters 2-4. The answers are given for the class of random fields whose covariance functions R(x, y) are kernels of positive rational functions of selfadjoint elliptic operators in L 2 (R r ). The class R of such kernels consists of kernels R(x, y) = Λ P (λ)q 1 (λ)φ(x, y, λ)dρ(λ) (1.10) where Λ, dρ, Φ(x, y, λ) are respectively the spectrum, spectral measure and spectral kernel of an elliptic selfadjoint operator L in L 2 (R r ) of order s, and P (λ) and Q(λ) are positive polynomials of degrees p and q respectively. The notions of spectral measure and spectral kernel are discussed in Section 8.2. If p > q then the operator in L 2 () with kernel (1.10) is an elliptic integrodifferential operator R; if p = q then R = ci + K, where c = const > 0, I is the identity operator, and K is a compact selfadjoint operator in L 2 (); if p < q, which is the most interesting case, then R is a compact selfadjoint operator in L 2 (). In this case the noise n(x) is called colored. If φ(λ) is a measurable function then the kernel of the operator φ(l) is defined by the formula φ(l)(x, y) = Γ φ(λ)φ(x, y, λ)dρ(λ). (1.11) The domain of definition of the operator φ(l) consists of all functions f L 2 (R r ) such that Γ φ(λ) 2 d(e λ f, f) < (1.12) where E λ is the resolution of the identity for L. It is a projection operator

17 4 Random Fields Estimation Theory with the kernel E λ (x, y) = λ In particular, since E + = I, one has δ(x y) = Φ(x, y, λ)dρ(λ). (1.13) Φ(x, y, λ)dρ(λ). (1.14) In (1.13) and (1.14) the integration is actually taken over (, λ) Λ and (, ) Λ respectively, since dρ = 0 outside Λ. The kernel in (1.8) corresponds to the simple case r = 1, L = i d, Λ = (, ), dρ = dλ, dx Φ(x, y, λ) = (2π) 1 exp{iλ(x y)}, P (λ) = 1, Q(λ) = (λ 2 + 1)/2, e x = (2π) ( ) 1 1 λ exp(iλx)dλ, and formula (1.9) is a very particular case of the general formulas given in Chapter 2. Let R(x, y) R, α := 1 2 s(q p), Hl () be the Sobolev spaces and Ḣ l () be its dual with respect to H 0 () = L 2 (). Then the answers to the questions, formulated above, are as follows. The solution to equation (1.7) solves the estimation problem if and only if h Ḣ α (). The operator R : Ḣ α () H α () is an isomorphism. The singular support of the solution h Ḣ α () of equation (1.7) is Γ =. The analytic formula for h is of the form h = Q(L)G, where G is a solution to some interface elliptic boundary value problem and the differentiation is taken in the sense of distributions. Exact description of this analytic formula is given in Chapter 2. The spectral properties of the operator R : L 2 () L 2 () with the kernel R(x, y) R are given also in Chapter 2. These properties include asymptotics as n of the eigenvalues λ n of R, dependence λ n on, and asymptotics of λ 1 () as R r, that is growing uniformly in directions. Numerical methods for solving equation (1.7) in the space Ḣ α () of distributions are given in Chapter 3. These methods depend heavily on the analytical results given in Chapter 2. The necessary background material on Sobolev spaces and spectral theory is given in Chapter 8 so that the reader does not have to consult the literature in order to understand the contents of this work. No attempts were made by the author to present all aspects of the theory of random fields. There are several books [Adler (1981)], [Yadrenko (1983)], [Vanmarcke (1983)], [Rosanov (1982)] and [Preston (1967)], and many papers on various aspects of the theory of random fields. They have

18 Introduction 5 practically no intersection with this work which can be viewed as an extension of Wiener s filtering theory. The statement of the problem is the same as in Wiener s theory, but we study random functions of several variables, that is random fields, while Wiener (and many researchers after him) studied filtering and extrapolation of stationary random processes, that is random functions of one variable. Wiener s basic assumptions were: 1) the random process u(t) = s(t) + n(t) is stationary, 2) it is observed on the interval (, T ), 3) it has a rational spectral density (this assumption can be relaxed, but for effective solution of the estimation problems it is quite useful). The first assumption means that R(t, τ) = R(t τ), where R is the covariance function (1.3). The second one means that = (, T ). The third one means R(λ) = P (λ)q 1 (λ), where P (λ) and Q(λ) are polynomials, R(λ) 0 for < λ <, R(λ) := R(t) exp( iλt)dt. The analytical theory used by Wiener is the theory of Wiener-Hopf equations. Later the Wiener theory was extended to the case = [T 1, T ] of a finite interval of observation, while assumptions 1) and 3) remained valid. A review of this theory with many references is [Kailath (1974)]. Although the literature on filtering and estimation theory is large (dozens of books and hundreds of papers are mentioned in [Kailath (1974)]), the analytic theory presented in this work and developed in the works of the author cited in the references has not been available in book form in its present form, although a good part of it appeared in [?, Ch. 1]. Most of the previously known analytical results on Wiener-Hopf equations with rational R(λ) are immediate and simple consequences of our general theory. Engineers can use the theory presented here in many applications. These include signal and image processing in TV, underwater acoustics, geophysics, optics, etc. In particular, the following question of long standing is answered by the theory given here. Suppose a random field (1.1) is observed in a ball B and one wants to estimate s(x 0 ), where x 0 is the center of B. What is the optimal size of the radius of B? If the radius is too small then the estimate is not accurate. If it is too large then the estimate is not better than the one obtained from the observations in a ball of smaller radius, so that the efforts are wasted. This problem is of practical importance in many applications. We will briefly discuss some other applications of the estimation theory, for example, discrimination of hypotheses, resolution ability of linear

19 6 Random Fields Estimation Theory systems, estimation of derivatives of random functions, etc. However, the emphasis is on the theory, and the author hopes that other scientists will pursue further possible applications. Numerical solution of the basic integral equation of estimation theory was widely discussed in the literature [Kailath (1974)] in the case of random processes (r = 1), mostly stationary that is when R(x, y) = R(x y), and mostly in the case when the noise is white, so that the integral equation for the optimal filter is (I + R)h := h(t) + T 0 R s (t τ)h(τ)dτ = f(t), 0 t T, (1.15) where R s is the covariance function of the useful signal s(t). Note that the integral operator R in (1.10) is selfadjoint and nonnegative in L 2 [0, T ]. Therefore (I + T ) 1 exists and is bounded, and the numerical solution of (1.15) is not difficult. Many methods are available for solving onedimensional second order Fredholm integral equation (1.15) with positivedefinite operator I + R. Iterative methods, projection methods, colloqation and many other methods are available for solving (1.15), convergence of these methods has been proved and effective error estimates of the numerical methods are known [Kantorovich and Akilov (1980)]. Much effort was spent on effective numerical inversion of Toeplitr matrices which one obtains if one discretizes (1.15) using equidistant colloqation points [Kailath (1974)]. However, if the noise is colored, the basic equation becomes T 0 R(t τ)h(τ)dτ = f(t), 0 t T. (1.16) This is a Fredholm equation of the first kind. A typical example is equation (1.8). As we have seen in (1.9), the solution to (1.8) is a distribution, in general. The theory for the numerical treatment of such equation was given by the author [Ramm (1985)] and is presented in Chapter 3 of this book. In particular, the following question of singular perturbation theory is of interest. Suppose that equation ɛh ɛ + Rh ɛ = f, ɛ > 0, (1.17) is given. This equation corresponds to the case when the intensity of the white-noise component of the noise is ɛ. What is the behavior of h ɛ when ɛ +0? We will answer this question in Chapter 5. This book is intended for a broad audience: for mathematicians, engineers interested in signal and image processing, geophysicists, etc. There-

20 Introduction 7 fore the author separated formulation of the results, their discussion and examples from proofs. In order to understand the proofs, one should be familiar with some facts and ideas of functional analysis. Since the author wants to give a relatively self-contained presentation, the necessary facts from functional analysis are presented in Chapter 8. The book presents the theory developed by the author. Many aspects of estimation theory are not discussed in this book. The book has practically no intersection with works of other authors on random fields estimation theory.

21 8 Random Fields Estimation Theory

22 Chapter 2 Formulation of Basic Results 2.1 Statement of the problem Let R r be a bounded domain with a sufficiently smooth boundary Γ. The requirement that is bounded could be omitted, it is imposed for simplicity. The reader will see that if is not bounded then the general line of the arguments remains the same. The additional difficulties, which appear in the case when is unbounded, are of technical nature: one needs to establish existence and uniqueness of the solution to a certain transmission problem with transmission conditions on Γ. Also the requirement of smoothness of Γ is of technical nature: the needed smoothness should guarantee existence and uniqueness of the solution to the above transmission problem. Let L be an elliptic selfadjoint in H = L 2 (R r ) operator of order s. Let Λ, Φ(x, y, λ), dρ(λ) be the spectrum, spectral kernel and spectral measure of L, respectively. A function F (L) is defined as an operator on H with the kernel F (L)(x, y) = Λ F (λ)φ(x, y, λ)dρ(λ) (2.1) and domf (L) = {f : f H, F (λ) 2 d(e λ f, f) < }, where (E λ f, f) = λ { } Φ(x, y, µ)f(y)f(y)dxdy dρ(µ), :=. (2.2) R r efinition 2.1 Let R denote the class of kernels of positive rational functions of L, where L runs through the set of all selfadjoint elliptic operators 9

23 10 Random Fields Estimation Theory in H = L 2 (R r ). In other words, R(x, y) R if and only if R(x, y) = P (λ)q 1 (λ)φ(x, y, λ)dρ(λ), (2.3) Λ where P (λ) > 0 and Q(λ) > 0, λ Λ, and Λ, Φ, dρ correspond to an elliptic selfadjoint operator L in H = L 2 (R r ). Let p = deg P (λ), q = degq(λ), s = ordl, (2.4) where deg P (λ) stands for the degree of the polynomial P (λ), and ordl stands for the order of the differential operator L. An operator given by the differential expression Lu := a j (x) j u, (2.5) j s where j = (j 1, j 2...j r ) is a multiindex, j u = x j1 1 x j x jr r u j = j 1 + j j r, j m 0, are integers. The expression (2.5) is called elliptic if, for any real vector t R r, the equation a j (x)t j = 0 j =s implies that t = 0. The expression L + u := ( 1) j j (a j (x)u) (2.6) j s is called the formal adjoint with respect to L. The star in (2.6) stands for complex conjugate. One says that L is formally selfadjoint if L = L +. If L is formally selfadjoint then L is symmetric on C0 (R r ), that is (Lφ, ψ) = (φ, Lψ) φ, ψ C0 (R r ), where (φ, ψ) is the inner product in H = L 2 (R r ). Sufficient conditions on a j (x) can be given for a formally selfadjoint differential expression to define a selfadjoint operator in H in the following way. efine a symmetric operator L 0 with the domain C0 (R r ) by the formula L 0 u = Lu for u C0 (R r ). Under suitable conditions on a j (x) one can prove that L 0 is essentially selfadjoint, that is, its closure is selfadjoint (See Chapter 8). In particular this is the case if a j = a j = const. In what follows we assume that R(x, y) R. Some generalizations will be considered later. The kernel R(x, y) is the covariance function (1.3) of the random field U(x) = s(x)+n(x) observed in a bounded domain R r.

24 Formulation of Basic Results 11 A linear estimation problem can be formulated as follows: find a linear estimate Û := LU := h(x, y)u(y)dy (2.7) such that ɛ := Û As 2 = min. (2.8) The kernel h(x, y) in (2.7) is a distribution, so that, by L. Schwartz s theorem about kernels, estimate (2.7) is the most general linear estimate. The operator A in (2.8) is assumed to be known. It is an arbitrary operator, not necessarily a linear one. In the case when AU = U, that is A = I, where I is the identity operator, the estimation problem (2.8) is called the filtering problem. From (2.8) and (2.7) one obtains ɛ = h(x, y)u(y)dy h (x, z)u (z)dz Here = 2Re h(x, z)u(z)dz(as) (x) + As(x) 2 h(x, y)h (x, z)r(z, y)dzdy 2Re h (x, z)f(z, x)dz + As(x) 2 = min. (2.9) f(y, x) := U (y)(as(x)) = f (x, y), (2.10) the bar stands for the mean value and the star stands for complex conjugate. By the standard procedure one finds that a necessary condition for the minimum in (2.9) is: R(z, y)h(x, y)dy = f(z, x), x, z := Γ. (2.11) In order to derive (2.11) from (2.9) one takes h + αη in place of h in (2.9). Here α is a small number and η C0 (). The condition ɛ(h) ɛ(h + αη) implies ɛ α α=0 = 0. This implies (2.11). Since h is a distribution, the left-hand side of (2.11) makes sense only if the kernel R(z, y) belongs to the space of test functions on which the distribution h is defined. We

25 12 Random Fields Estimation Theory will discuss this point later in detail. In (2.11) the variable x enters as a parameter. Therefore, the equation Rh := R(x, y)h(y)dy = f(x), x, (2.12) is basic for estimation theory. We have supressed the dependence on x in (2.11) and have written x in place of z in (2.12). From this derivation it is clear that the operator A does not influence the theory in an essential way: if one changes A then f is changed but the kernel of the basic equation (2.12) remains the same. If Au = u x j, one has the problem of estimating the derivative of u. If Au = u(x + x 0 ), where x 0 is a given point such that x + x 0, then one has the extrapolation problem. Analytically these problems reduce to solving equation (2.12). If no assumptions are made about R(x, y) except that R(x, y) is a covariance function, then one cannot develop an analytical theory for equation (2.12). Such a theory will be developed below under the basic assumption R(x, y) R. Let us show that the class R of kernels, that is the class of random fields that we have introduced, is a natural one. To see this, recall that in the one-dimensional case, studied analytically in the literature, the covariance functions are of the form R(x, y) = R(x y), x, y R 1, R(λ) := R(x) exp( iλx)dx = P (λ)q 1 (λ), where P (λ) and Q(λ) are positive polynomials [Kai]. This case is a very particular case of the kernels in the class R. Indeed, take r = 1, L = i d, Λ = (, ), dρ(λ) = dλ, dx Φ(x, y, λ) = (2π) 1 exp{iλ(x y)}. Then formula (2.3) gives the above class of convolution covariance functions with rational Fourier transforms. If p = q, where p and q are defined in (2.4), then the basic equation (2.12) can be written as Rh := σ 2 h(x) + R 1 (x, y)h(y)dy = f(x), x, σ 2 > 0, (2.13)

26 Formulation of Basic Results 13 where P (λ)q 1 (λ) = σ 2 + P 1 (λ)q 1 (λ), p 1 := deg P 1 < q, (2.14) and σ 2 > 0 is interpreted as the variance of the white noise component of the observed signal U(x). If p < q, then the noise in U(x) is colored, it does not contain a white noise component. Mathematically equation (2.13) is very simple. The operator R in (2.13) is of Fredholm type, selfadjoint and positive definite in H, R σ 2 I, where I is the identity operator, and A B means (Au, u) (Bu, u) u H. Therefore, if p = q then equation (2.12) reduces to (2.13) and has a unique solution in H. This solution can be computed numerically without difficulties. There are many numerical methods which are applicable to equation (2.13). In particular (see Section 2.3.2), an iterative process can be constructed for solving (2.13) which converges as a geometrical series; a projection method can be constructed for solving (2.13) which converges and is stable computationally; one can solve (2.13) by collocation methods. However the important and practically interesting question is the following one: what happens with the solution h σ to (2.13) as σ 0? This is a singular perturbation theory question: for σ > 0 the unique solution to equation (2.13) belongs to L 2 (), while for σ = 0 the unique solution to (2.13) of minimal order of singularity is a distribution. What is the asymptotics of h σ as σ 0? As we will show, the answer to this question is based on analytical results concerning the solution to (2.13). The basic questions we would like to answer are: 1) In what space of functions or distributions should one look for the solution to (2.12)? 2) When does a solution to (2.12) solve the estimation problem (2.8)? Note that (2.12) is only a necessary condition for h(x, y) to solve (2.8). We will show that there is a solution to (2.12) which solves (2.8) and this solution to (2.12) is unique. The fact that estimation problem (2.8) has a unique solution follows from a Hilbert space interpretation of problem (2.8) as the problem of finding the distance from the element (As)(x) to the subspace spanned by the values of the random field u(y), y. Since there exists and is unique an element in a subspace at which the distance is attained, problem (2.8) has a solution and the solution is unique. It was mentioned in the Introduction (see (1.9)) that equation (2.12) may have no solutions in L 1 () but rather its solution is a distribution. There can be

27 14 Random Fields Estimation Theory several solutions to (2.12) in spaces of distributions, but only one of them solves the estimation problem (2.8). This solution is characterized as the solution to (2.12) of minimal order of singularity. 3) What is the order of singularity and singular support of the solution to (2.12) which solves (2.8)? 4) Is this solution stable under small perturbations of the data, that is under small perturbations of f(x) and R(x, y)? What is the appropriate notion of smallness in this case? What are the stability estimates for h? 5) How does one compute the solution analytically? 6) How does one compute the solution numerically? 7) What are the properties of the operator R : L 2 () L 2 () in (2.12)? In particular, what is the asymptotics of its eigenvalues λ j () as j +? What is the asymptotics of λ 1 () as R r, that is, as expands uniformly in directions? These questions are of interest in applications. Note that if is finite then the operator R is selfadjoint positive compact operator in L 2 (), its spectrum is discrete, λ 1 > λ 2 > 0, the first eigenvalue is nondegenerate by Krein-Rutman s theorem. However, if = R r then the spectrum of R may be continuous, e.g., this is the case when R(x, y) = R(x y), R(λ) = P (λ)q 1 (λ). Therefore it is of interest to find λ 1 := limλ 1 () as R r. The quantity λ 1 is used in some statistical problems. 8) What is the asymptotics of the solution to (2.13) as σ 0? 2.2 Formulation of the results (multidimensional case) Basic results We assume throughout that R(x, y) R, f(x) is smooth, more precisely, that f H α, H α = H α ()is the Sobolev space, α := (q p)s/2, where s, q, p are the same as in (2.4), and the coefficients a j (x) of L (see (2.5)) are sufficiently smooth, say a s (x) C(R r ), a j (x) L r/(s α ) loc if s j < r/2, a j L 2 loc if s j > r/2, a j L 2+ɛ loc, ɛ > 0, if s j = r/2 (see [Hörmander ( ), Ch 17]). If q p, then the problem of finding the is simple: such a solution does exist, is unique, and belongs to H m+2 α if f H m. This follows from

28 Formulation of Basic Results 15 the usual theory of elliptic boundary value problems [Berezanskij (1968)], since the operator P (L)Q 1 (L) is an elliptic integral-differential operator of order 2 α if q p. The solution satisfies the elliptic estimate: h H m+2 α c f H m, q p, where c depends on L but does not depend on f. If q > p, then the problem of finding the mos solution of (2.12) is more interesting and difficult because the order of singularity of h, is in general, positive, ordh = α. The basic result we obtain is: The mapping R : Ḣ α H α is a linear isomorphism between the spaces Ḣ α and H α. The of h is = Γ, provided that f is smooth. If h 1 is a solution to equation (2.12) and ordh 1 > α then ɛ =, where ɛ is defined in (2.8). Therefore if h 1 solves (2.12) and ordh 1 > α then h 1 does not solve the estimation problem (2.8). The unique solution to (2.8) is the unique mos solution to (2.12). We give analytical formulas for the mos solution to (2.12). This solution is stable towards small perturbations of the data. We also give a stable numerical procedure for computing this solution. In this section we formulate the basic results. Theorem 2.1 If R(x, y) R, then the operator R in (2.12) is an isomorphism between the spaces Ḣ α and H α. The solution to (2.12) of minimal order of singularity, ordh α, can be calculated by the formula: where h(x) = Q(L)G, (2.15) G(x) = { g(x) + v(x) in u(x) in Ω := R r \, (2.16) g(x) H s(p+q)/2 is an arbitrary fixed solution to the equation P (L)g = f in, (2.17) and the functions u(x) and v(x) are the unique solution to the following (2.18)-(2.20): Q(L)u = 0 in Ω, u( ) = 0, (2.18) P (L)v = 0 in, (2.19)

29 16 Random Fields Estimation Theory j N u = s(p + q) j N (v + g) on Γ, 0 j 1. (2.20) 2 By u( ) = 0 we mean limu(x) = 0 as x. Corollary 2.1 If f H 2β, β α, then sing supph = Γ. (2.21) Corollary 2.2 If P (λ) = 1, then the transmission problem (2.18)-(2.20) reduces to the irichlet problem in Ω: Q(L)u = 0 in Ω, u( ) = 0, (2.22) j N u = j N f on Γ, 0 j sq 2 1, (2.23) and (2.15) takes the form h = Q(L)F, F = { f in u in Ω. (2.24) Corollary 2.1 follows immediately from formulas (2.15) and (2.16) since g(x)+v(x) and u(x) are smooth inside and Ω respectively. Corollary 2.2 follows immediately from Theorem 2.1: if P (λ) = 1 then g = f, v = 0, and p = 0. Let ω(λ) 0, ω(λ) C(R 1 ), ω( ) = 0, R(x, y) = ω := max ω(λ), (2.25) λ Λ Λ ω(λ)φ(x, y, λ)dρ(λ), (2.26) λ j = λ j () be the eigenvalues of the operator R : L 2 () L 2 () be the eigenvalues of the operator R : L 2 () L 2 () with kernel (2.26), arranged so that λ 1 λ 2 λ 3 > 0. (2.27) Theorem 2.2 If then λ j λ j, where λ j = λ j( ). If R(x, y) dy := A <, (2.28) sup x R r

30 Formulation of Basic Results 17 then where and ω is defined in (2.25). λ 1 = ω, (2.29) lim λ 1() := λ 1, (2.30) R r Theorem 2.3 If ω(λ) = λ a (1 + o(1)) as λ, and a > 0, then the asymptotics of the eigenvalues of the operator R with kernel (2.26) is given by the formula: where c = γ as/r and with λ j cj as/r as j, c = const > 0, (2.31) η(x) := meas{t : t R r, γ := (2π) r α = β =s/2 η(x)dx, (2.32) a αβ (x)t α+β 1}. (2.33) Here the form a αβ (x) generates the principal part of the selfadjoint elliptic operator L: Lu = α (a αβ (x)) β u + L 1, ordl 1 < s. α = β =s/2 Corollary 2.3 If ω(λ) = P (λ)q 1 (λ) then a = q p, where q = deg Q, p = deg P, and λ n cn (q p)s/r, where λ n are the eigenvalues of the operator in equation (2.12). This Corollary follows immediately from Theorem 2.3. Theorems answer questions 1)-5) and 7) in section 2.1. Answers to questions 6) and 8) will be given in Chapter 3. Proof of Theorem 2.3 is given in Section Generalizations First, let us consider a generalization of the class R of kernels for the case when there are several commuting differential operators. Let L 1,...L m be a system of commuting selfadjoint differential operators in L 2 (R r ). There

31 18 Random Fields Estimation Theory exists a dµ(ξ) and a spectral kernel Φ(x, y, ξ), ξ = (ξ 1,..., ξ m ) such that a function F (L 1,...L m ) is given by the formula F (L 1,...L m ) = M F (ξ)φ(ξ)dµ(ξ), (2.34) where Φ(ξ) is the operator with kernel Φ(x, y, ξ). The domain of definition of the operator F (L 1,...L m ) is the set of all functions u L 2 (R r ) for which M F (ξ) 2 (Φ(ξ)u, u)dµ <, M is the support of the spectral measure dµ, and the parentheses denote the inner product in L 2 (R r ). For example, let m = r, L j = i x j. Then ξ = (ξ 1,...ξ r ), dµ = dξ 1... dξ r, φ(x, y, ξ) = (2π) r exp{iξ (x y)}, where dot denotes the inner product in R r. If F (ξ) = P (ξ)q 1 (ξ), where P (ξ) and Q(ξ) are positive polynomials and the operators P (L) := P (L 1,...L r ) and Q(L) := Q(L 1,...L r ) are elliptic of orders m and n respectively, m < n, then the theorems, analogous to Theorems , hold with sp = m and sq = n. Theorem 2.3 has also an analogue in which as = n m in formula (2.31). Another generalization of the class R of kernels is the following one. Let Q(x, ) and P (x, ) be elliptic differential operators and QR = P δ(x y) in R r. (2.35) Note that the kernels R R satisfy equation (2.35) with Q = Q(L), P = P (L). Let ordq = n, ordp = m, n > m. Assume that the transmission problem (2.18)-(2.20), with Q(x, ) and P (x, ) in place of Q(L) and P (L) respectively, and ps = m, qs = n, has a unique solution in H (n+m)/2. Then Theorem 2.1 holds with α = (n m)/2. The transmission problem (2.18)-(2.20) with Q(x, ) and P (x, ) in place of Q(L) and P (L) is uniquely solvable provided that, for example, Q(x, ) and P (x, ) are elliptic positive definite operators. For more details see Chapter 4 and Appendices A and B. 2.3 Formulation of the results (one-dimensional case) In this section we formulate the results in the one-dimensional case, i.e., r = 1. Although the corresponding estimation problem is the problem for random processes (and not random fields) but since the method and the results are the same as in the multidimensional case, and because of

32 Formulation of Basic Results 19 the interest of the results in applications, we formulate the results in onedimensional case separately Basic results for the scalar equation Let r = 1, = (t T, t), R(x, y) R. The basic equation (2.12) takes the form t t T Assume that f H α, α = s(q p)/2. R(x, y)h(y)dy = f(x), t T x t. (2.36) Theorem 2.4 The solution to equation (2.36) in Ḣ α exists, is unique, and can be found by the formula h = Q(L)G, (2.37) where sq/2 j=1 b j ψ j (x), x t T G(x) = g(x), t T t t (2.38) sq/2 j=1 b+ j ψ+ j (x), x t. Here b ± j are constants, the functions ψ j ± (x), 1 j sq/2, form a fundamental system of solutions to the equation Q(L)ψ = 0, ψ j ( ) = 0, ψ+ j (+ ) = 0. (2.39) The function g(x) is defined by the formula g(x) = g 0 (x) + sp j=1 where g 0 (x) is an arbitrary fixed solution to the equation c j φ j (x), (2.40) P (L)g = f, t T x t, (2.41) the functions φ j, 1 j sp, form a fundamental system of solutions to the equation P (L)φ = 0, (2.42)

33 20 Random Fields Estimation Theory and c j, 1 j sp, are constants. The constants b ± j, 1 j sq/2, and c j, 1 j sp, are uniquely determined from the linear system: sq/2 sp k b j ψ j = k g 0 + c j φ j, (2.43) x=t T x=t T j=1 k sq/2 j=1 b + j ψ+ j j=1 = k g 0 + x=t sp j=1 c j φ j, (2.44) x=t where = d/dx, 0 k 1 2 s(p + q) 1. The map R 1 : f h, where h is given by formula (2), is an isomorphism of the space H α onto the space Ḣ α. Remark 2.1 This theorem is a complete analogue of Theorem 2.1. The role of L is played now by an ordinary differential selfadjoint operator L in L 2 (R 1 ). An ordinary differential operator is elliptic if and only if the coefficient in front of its senior (that is, the highest order) derivative does not vanish: s Lu = a j (x) j u, a s (x) 0, (2.45) j=0 One can assume that a s (x) > 0, x R 1, and the condition of uniform ellipticity is assumed, that is 0 < c 1 a s (x) c 2, (2.46) where c 1 and c 2 are positive constants which do not depend on x. Corollary 2.4 If f H α, then sing supph =, where consists of two points t and t T. Corollary 2.4 is a complete analogue of Corollary 2.1. Corollary 2.5 Let Q(λ) = a + (λ)a (λ), where a ± (λ) are polynomials of degree q/2, the zeros of the polynomial a + (λ) lie in the upper half plane Imλ > 0, while the zeros of a (λ) lie in the lower half-plane Imλ < 0. Since Q(λ) > 0 for < λ <, the zeros of a + (λ) are complex conjugate of the corresponding zeros of a (λ). Assume that P (λ) = 1. Then formula (2.37) can be written as h(x) = a + (L)[θ(x t + T )a (L)f(x)] a (L)[θ(x t)a + (L)f(x)], (2.47)

34 Formulation of Basic Results 21 { 1 if x 0 where θ(x) =, and the differentiation in (2.47) is understood 0 if x < 0 in the sense of distributions. This Corollary is an analogue of Corollary 2.2. Remark 2.2 Formula (2.47) is convenient for practical calculations. Let us give a simple example of its application. Let L = i, r = 1, P (λ) = 1, Q(λ) = (λ 2 +1)/2, R(x, y) = exp( x y ), Φ(x, y, λ) = (2π) 1 exp{iλ(x y)}, dρ(λ) = dλ, t = 1, t T = 1. Equation (2.36) becomes 1 1 exp( x y )h(y)dy = f(x), 1 x 1, (2.48) a + (λ) = λ i 2, a (λ) = λ+i 2. Formula (2.47) yields: h(x) = 1 ( i i)[θ(x + 1)( i + i)f(x)] 2 1 ( i + i)[θ(x 1)( i i)f(x)] 2 = 1 2 ( + 1)[θ(x + 1)( 1)f] + 1 ( 1)[θ(x 1)( + 1)f] 2 = 1 2 θ(x + 1)( 2 1)f 1 δ(x + 1)( 1)f θ(x 1)( 2 1)f + 1 δ(x 1)( + 1)f 2 = f + f 2 + f (1) + f(1) δ(x 1) 2 + f ( 1) + f( 1) δ(x + 1). (2.49) 2 Here we have used the well known formula θ (x a) = δ(x a) where δ(x) is the delta-function. Formula (2.49) is the same as formula (1.9). The term ( f + f)/2 in (2.49) vanishes outside the interval [ 1, 1] by definition. Remark 2.3 If t = + and t T = 0, so that equation (2.36) takes the form of Wiener-Hopf equation of the first kind then formula (2.47) reduces to 0 R(x, y)h(y)dy = f(x), x 0, (2.50) h(x) = a + (L)[θ(x)a (L)f(x)]. (2.51)

35 22 Random Fields Estimation Theory If L = i formula (2.51) can be obtained by the well-known factorization method. Example 2.1 Consider the equation By formula (2.51) one obtains 0 exp( x y )h(y)dy = f(x), x 0. (2.52) h(x) = f + f 2 + f (0) + f(0) δ(x), (2.53) 2 if one uses calculations similar to the given in formula (2.49) Vector equations In both cases r = 1 and r > 1 it is of interest to consider estimation problems for vector random processes and vector random fields. For vector random processes the basic equation is (2.36) with the positive kernel R(x, y) in the sense that (Rh, h) > 0 for h 0, given by formula (2.3) in which R(λ) = P (λ)q 1 (λ) is a matrix: R(λ) = ( R ij (λ)), R ij (λ) := P ij (λ)q 1 ij (λ), 1 i, j d, (2.54) where P ij (λ) and Q ij (λ) are relatively prime positive polynomials for each fixed pair of indices (ij), 1 i, j d, d is the number of components of the random processes U, s and n. Let Q(λ) be the polynomial of minimal degree, deg Q(λ) = q, for which any Q ij (λ), 1 i, j d, is a divisor, A ij (λ) := R ij (λ)q(λ). enote by E the unit d d matrix and by A(L) the matrix differential operator with entries A ij (L). Assume that det A ij (λ) > 0, λ R 1, (2.55) where m := s max 1 i,j d deg A ij (λ), s = ordl, det B m (x) 0, x R 1, (2.56) m A(L) := B j (x) j, j=0 = d dx. (2.57)

36 Formulation of Basic Results 23 Let S(x, y) denote the matrix kernel { 1 i = j S(x, y) := δ ij Q 1 (λ)φ(x, y, λ)dρ(λ), δ ij = 0 i j Λ (2.58) of the diagonal operator Q 1 (L)E. The operator Q(L)E is a diagonal matrix differential operator of order n = sq. Let us write the basic equation t t T R(x, y)h(y)dy = f(x), t T x t, (2.59) where R(x, y) is the d d matrix with the spectral density (2.54), h and f are vector functions with d components, f H α, α = (n m)/2, H α denotes the space of vector functions (f 1,...f d ) such that f H α:= ( d j=1 f j 2 H α ) 1/2. Remark 2.4 In the vector estimation problem h and f are d d matrices, but for simplicity and without loss of generality we discuss the case when h is a vector. Matrix equation (2.59) is equivalent to d vector equations. Equation (2.59) one can write as v := Q 1 (L)Eh = A(L)v = f, (2.60) S(x, y)h(y)dy. (2.61) Let Φ j, 1 j m, be a fundamental system of matrix solutions to the equation A(L)φ = 0, (2.62) and Ψ ± j, 1 j n/2 be the fundamental system of matrix solutions to the equation such that Q(L)EΨ = 0, (2.63) Ψ + j (+ ) = 0, Ψ j ( ) = 0. (2.64) The choice of the fundamental system of matrix solutions to (2.63) with properties (2.64) is possible if L is an elliptic ordinary differential operator,

37 24 Random Fields Estimation Theory that is a s (x) 0, x R 1 (see Remark 2.1 and [N] p. 118). Let us write equations (2.60) and (2.61) as S(x, y)h(y)dy = g 0 (x) + m Φ j (x)c j, (2.65) where g 0 (x) is an arbitrary fixed solution to the equation (2.60), and c j, 1 j m, are arbitrary linearly independant constant vectors. Theorem 2.5 If R R with R given by (2.54), the assumptions (2.46), (2.55), (2.56) hold, and f H α, α = (n m)/2, then the matrix equation (2.59) has a solution in Ḣ α, this solution is unique and can be found by the formula j=1 h = Q(L)EG, (2.66) where the vector function G is given by n/2 j=1 Ψ j b j, x t T G(x) = g 0 (x) + m j=1 Φ jc j, t T x t, (2.67) n/2 j=1 Ψ+ j b+ j, x t. Here the functions Ψ ± j, Φ j and g 0 (x) were defined above, and the constant vectors b ± j, 1 j n/2 and c j, 1 j m, can be uniquely determined from the linear system n/2 m k Ψ j (x)b j = k g 0(x) + Φ j (x)c j (2.68) x=t T x=t T k j=1 n/2 m Ψ + j (x)b+ j = k g 0(x) + Φ j (x)c j, (2.69) x=t x=t j=1 where 0 k (n + m)/2 1. The map R 1 : f h, given by formulas (2.66)-(2.69) is an isomorphism between the spaces H α and Ḣ α, α = (n m)/2. Remark 2.5 The conditions (2.68), (2.69) guarantee that the function G(x), defined by formula (2.67), is maximally smooth so that the order of singularity of G and, therefore, of h (see formula (2.66) ) is minimal. j=1 j=1

38 Formulation of Basic Results Examples of kernels of class R and solutions to the basic equation 1. If r = 1, L = i, = d/dx, Φ(x, y, λ) = (2π) 1 exp{iλ(x y)}, dρ = dλ, then R(x, y) R if where R(x, y) = (2π) 1 R(λ) exp{iλ(x y)}dλ, (2.70) R(λ) = P (λ)q 1 (λ) (2.71) and P (λ), Q(λ) are positive polynomials. 2. If r > 1, L = (L 1,...L r ), L r = i r, r = / x r, Φ(x, y, λ) = (2π) r exp{iλ (x y)}, λ = (λ 1,...λ r ), dρ(λ) = dλ = dλ 1...dλ r, then R(x, y) = (2π) r R r R(λ) exp{iλ (x y)}dλ, (2.72) where R(λ) is given by (2.71) and P (λ) = P (λ 1,...λ r ) > 0, Q(λ) = Q(λ 1...λ r ) > 0 (2.73) are polynomials. For the operators P (L) and Q(L) to be elliptic of orders p and q respectively, one has to assume that 0 < c 1 P (λ) λ p c 2, 0 < c 3 Q(λ) λ q c 4, λ R r (2.74) where λ = (λ λ 2 r) 1/2 and c j, 1 j 4, are positive constants. 3. If r = 1, L = d2 dx, (L) = {u : u H 2 (0, ), u (0) = 0}, (L) = 2 domain of L, then where R(x, y) = 1 [A( x + y ) + A( x y )], x, y 0 (2.75) 2 A(x) = π 1 P (λ)q 1 (λ) cos( λx)λ 1/2 dλ (2.76) 0 and P (λ) > 0, Q(λ) > 0 are polynomials. Indeed, one has for L { π 1 cos( λx) cos( λy)λ 1/2 dλ, λ 0 Φ(x, y, λ)dρ(λ) = 0, λ < 0,

39 26 Random Fields Estimation Theory 0 x, y <. Since cos(kx) cos(ky) = 1 [cos(kx ky) + cos(kx + ky)], 2 k = λ one obtains (2.75) and (2.76). If one put λ = k in (2.76) one gets A(x) = 2 π 0 P (k 2 )Q 1 (k 2 ) cos(kx)dk, (2.77) which is a cosine transform of a positive rational function of k. The eigenfunctions of L, normalized in L 2 (0, ), are ( ) 2 1/2 π cos(kx) and dρ = dk in the variable k. If L = d2 dx is determined in L 2 (0, ) by the boundary 2 condition u(0) = 0, then R(x, y) = 1 [A( x y ) A(x + y)], x, y 0, (2.78) 2 where A(x) is given by (2.77), the eigenfunctions of L with the irichlet 2 boundary condition u(0) = 0 are π sin(kx), dρ = dk in the variable k, and Φ(x, y, k)dρ(k) = 2 π sin(kx) sin(ky)dk one can compare this with the formula Φ(x, y, k)dρ(k) = 2 π cos(kx) cos(ky)dk, which holds for L determined by the Neumann boundary condition u (0) = If L = d2 dx + (ν )x 2, ν 0, x 0, then Φ(x, y, λ)dρ(λ) = { xλjν (xλ) yλj ν (yλ)dλ, if λ 0 0, if λ < 0, (2.79) so that R(x, y) = xy 0 P (λ)q 1 (λ)j ν (λx)j ν (λy)λdλ, (2.80) where P (λ) and Q(λ) are positive polynomials on the semiaxis λ Let R(x, y) = exp( a x y )(4π x y ) 1, x, y R 3, a = const > 0. Note that ( + a 2 )R = δ(x y) in R 3. The kernel R(x, y) R. One has L = (L 1, L 2, L 3 ), L j = i j, P (λ) = 1, Q(λ) = λ 2 + a 2, λ 2 = λ λ λ 2 3, Φdρ = (2π) 3 exp{iλ (x y)}dλ, R(x, y) = (2π) 3 exp{iλ (x y)} λ 2 + a 2 dλ. (2.81) R 3 6. Let R(x, y) = R(xy). Put x = exp(ξ), y = exp( η). Then R(xy) = R(exp(ξ y)) := R 1 (ξ y). If R 1 R with L = i, then one can solve