Hausdorff moment problem: Reconstruction of probability density functions

Statistics and Probability Letters 78 (8) 869 877 www.elsevier.com/locate/stapro Hausdorff moment problem: Reconstruction of probability density functions Robert M. Mnatsakanov Department of Statistics, West Virginia University, Morgantown, WV 656, USA Health Effects Laboratory Division, National Institute for Occupational Safety and Health, Morgantown, WV 655, USA Received 9 January 7; received in revised form 7 September 7; accepted 9 January 8 Available online February 8 Abstract The problem of recovering a moment-determinate probability density function (pdf) from its moments is studied. The proposed construction provides a method for recovery of different pdfs via simple transformations of the moment sequences. Uniform and L -rates of convergence of moment-recovered pdfs are obtained. Finally, some applications and eamples are briefly discussed. c 8 Elsevier B.V. All rights reserved.. Introduction The problem of non-parametric estimation of indirectly observed probability density functions frequently arises in survival analysis, industry, and biological sciences. In many indirect models, e.g., those based on convolutions, mitures, multiplicative censoring, and biased sampling, the moments of an unobserved distribution of actual interest can be easily estimated from the transformed moments of an observed distribution. Reconstruction of the domain of a function, or equivalently, the characteristic function of this domain, given its moments, represents another interesting area where the moment problem arises (see, Goldenshluger and Spokoiny (4)). In all such models one can use the procedure which recovers a function via assigned moments. Recovering a density function from its moments can be considered as a special case of the classical moment problem (Akhiezer, 965). There are several formulations of the moment problem. First, let us consider the Hamburger moment problem: for a given sequence ν ={ = µ,µ,...} of real numbers, define a distribution F or its density function f on the real line (, ), such that µ j represents the jth moment of F, i.e., µ j = t j df(t) := µ j,f j =,,.... We say that the moment problem has a unique solution if for two distributions F and G, defined on (, ), the equations µ j,f = µ j,g, j =,,..., imply that F = G. In the latter case we say that the distribution F is This research was supported by the National Institute for Occupational Safety and Health, contract no. -6-M-6936. Corresponding address: Department of Statistics, West Virginia University, P.O. Bo 633, Morgantown, WV 656, USA. Fa: + 34 93 7. E-mail address: rmnatsak@stat.wvu.edu. 67-75/$ - see front matter c 8 Elsevier B.V. All rights reserved. doi:.6/j.spl.8..54

87 R.M. Mnatsakanov / Statistics and Probability Letters 78 (8) 869 877 moment-determinate (M-determinate), otherwise, F is M-indeterminate. If the support of F, supp{f} = [, ), the problem is known as the Stieltjes moment problem. In this paper we consider the Hausdorff moment problem, where supp{f} =[, T ], < T <. There are many articles investigating the conditions (for eample, Carleman s and Krein s conditions) under which the distributions are M-determinate or M-indeterminate, see Akhiezer (965), Lin (99, 997), and Stoyanov () among others. Several inversion formulas were obtained by inverting the moment-generating function and Laplace transform (Shohat and Tamarkin, 943; Feller, 97; Chauveau et al., 994; Tagliani and Velasquez, 3). But there are very few approaches recovering the density functions via their moments. This can be eplained by unstable behavior of the approimants when the higher-order moments are involved. Let us briefly discuss the Maimum Entropy (ME) method (Kevasan and Kapur, 99). In this approach it is proved that the approimant of f has the form of f M () = ep{ M j= λ j j }, and the maimum value of the entropy of f under the constraints t j f M (t) dt = µ j,f, j =,,... M, is equal to the minimum value of the potential function Γ (λ,..., λ M ): { [ { } ] M min Γ (λ,..., λ M ) = ln ep λ j j d + λ,...,λ M min λ,...,λ M j= } M λ j µ j,f. It is known that the evaluation of coefficients λ j, j =,... M, becomes unstable as the number of assigned moments M increases. To reduce the instability in ME approach the fractional moments were recommended to use instead of ordinary ones (see Novi Inverardi et al. (3) and Frontini and Tagliani (997) among others). Note also that the uniform, the L, and the entropy convergence of f M to f, as M, have been derived (see, for instance, Borwein and Lewis (99, 993), Novi Inverardi et al. (3) and the references therein), although the rates of convergence were not established. The aim of this paper is to construct a stable approimant f,ν of f via its moment sequence ν, to investigate the asymptotic properties of f,ν, and to derive the uniform and L -rates of approimation for f,ν, as. The paper is organized as follows. In Section the construction of moment-recovered function f,ν is given. In Section 3 we study the asymptotic behavior of functions f,ν corresponding to different choices of the moment sequence ν and its transformations. In Theorem, we derive the uniform and L -rates of convergence for f,ν. The question of recovering the convolutions f f and f f via transformations of corresponding moment sequences ν ={µ j,f, j N} and ν ={µ j,f, j N} is addressed in Theorem. In Section 4 some applications and eamples are discussed. In particular, L -consistency of moment-density estimator fˆ derived from (5) is proved.. Notations and preliminaries Suppose that the cumulative distribution function (cdf) F is absolutely continuous with respect to the Lebesgue measure and has a finite support, say, supp{f} =[, ]. Denote the pdf and the moments of F by f and µ j,f = t j df(t) = (KF)( j), j N ={,,...}, µ,f =, () respectively. Assume that the moment sequence ν = (µ,f,µ,f,...) determines F uniquely, i.e., F is M- determinate. Our construction in (5) is based on the inverse of the operator K from (). It is given by (K [] ν)() = k= j=k ( j )( ) j ( ) j k µ j,f,, N, () k and is such that K KF w F, as (see, Mnatsakanov and Ruymgaart (3)). Here w denotes the weak convergence of cdfs, i.e., convergence at each continuity point of the limiting cdf F. In what follows we assume, j=

R.M. Mnatsakanov / Statistics and Probability Letters 78 (8) 869 877 87 for simplicity, that the parameter is a positive integer. The following notations will be useful as well: denote the supremum norm of a function φ :[, ] R by φ = sup φ(), and the L -norm by φ L = φ() d, respectively. For any moment sequence ν ={ν j, j N}, let F,ν = K ν be the moment-approimant function of F: F,ν w F, as. When F is a smooth function, we can derive the uniform and L -rate of approimations in (3) (see, Mnatsakanov (8)): Theorem. Let ν ={µ j,f, j N}. If f is bounded on [, ] and, then F,ν F { f + } ( ) + f + o, F,ν F L { + } ( ) + f + o. To approimate pdf f let us consider the ratio: F,ν (), as,. Here F,ν () = F,ν () F,ν ( ) and = /. Scaling the left-hand side of (4) by ( + )/ and denoting it by f,ν (), we obtain the very simple form: [] ( ) m µ m+[],f, [, ]. (5) Γ ([]+) m! ( [] m)! m= If f is defined on [, T], < T <, and not on [, ], the previous display takes the form Γ ([/T ]+) T [/T ]+ [/T ] m= ( /T ) m µ m+[/t ],F, [, T ], (6) m! ( [/T ] m)! (cf. Mnatsakanov (8), formula ()). In Section 3 the uniform convergence of f,ν to f ν, as, is denoted by f,ν u f ν, and the modulus of continuity of f as ( f, δ) = sup t s δ f (t) f (s), < δ <. 3. Some properties of moment-recovered pdfs In this Section we study some asymptotic properties of the approimants f,ν in (5) and derive the uniform and L -rates of convergence for f,ν, as. Theorem. Let ν ={µ j,f, j N}. (i) If pdf f is continuous on [, ], then f,ν u f and for some < δ <, f,ν f ( f, δ) + (ii) If f is continuous on [, ], then f,ν () f () = + f δ ( + ) ; [ f ()( ) + f () ( ) ] + o (3) (4) (7) ( ), as ; (8)

87 R.M. Mnatsakanov / Statistics and Probability Letters 78 (8) 869 877 and f,ν f L f ()( ) + ( ) + f () ( ) d + o, as ; (9) (iii) If f is bounded on [, ], then f,ν f + Proof. (i): Let us denote by β(u, a, b) = { f + f + + f }. () Γ (a + b) Γ (a)γ (b) ua ( u) b, < u <, () a pdf of Beta (a, b) distribution with the shape parameters a and b. Assume here that a =[]+ and b = []+, respectively. Changing the order of summation and integration on the right-hand side of (5), we obtain = = [] Γ ([]+) ( ) m µ m+[],f m! ( [] m)! Γ ([]+) m= [] t [] m= ( t) m m! ( [] m)! f (t) dt β(t, []+, []+) f (t)dt. () Now one can derive f,ν u f from the last line in () (see, Bouezmarni and Rolin (3), or Feller (97), Ch. VII)), since {β(, []+, []+), N} forms a δ-sequence at, as, its mean η = ([]+)/(+) and variance σ are such that = ([]+) ( []+) ( + ) ( + 3) +, (3) η = + + () and σ = + ( ) + (), (4) with k () ( + ), k =, (see, Johnson et al. (994) and Chen (999)). To derive the inequality in (i), let us split the integration on the right-hand side of () into two parts. We obtain ( ) f (t) f () β(t, []+, []+) dt ( f, δ) + R, sup + t δ t >δ where < δ <. Furthermore, since the beta distribution β(, []+, []+) concentrates its masses around the point, we can estimate its tail using (3) and Tchebishev s inequality: R f sup β(t, []+, []+)dt f t >δ δ ( + ). Combining the last two inequalities we complete the proof of (i). To prove (ii), we will use the Taylor epansion ( ) of f at. Application of the continuity property of f and (4) leads to (8) (cf. Chen (999)). In (8) the term o is uniform in. Therefore, (9) follows from (8). To prove (iii) let us note that f (t) f () = f ()(t ) + t dτ τ f (y) dy, (5)

R.M. Mnatsakanov / Statistics and Probability Letters 78 (8) 869 877 873 and η +. (6) Thus the combination of (3), (5) and (6) yields (), since f,ν f sup f () (t )β(t, a, b) du + sup [ t β(t, a, b)dt dτ τ f (y) dy] f sup η + f sup + f + ( + ) f + (t ) β(t, a, b) dt ( + ) f. For any two M-determinate pdfs f and f with moment sequences ν ={µ j,f, j N} and ν ={µ j,f, j N}, respectively, let us use the following notations for the convolutions f f () = f (/τ) f (τ) dτ, τ, f f () = f ( τ) f (τ) dτ,. The following notations will be useful as well: define ν ν ={µ j,f µ j,f, j N} and ν ν = { ν j, j N}, with ν j = j m= ( ) j µ m,f µ j m,f. m One can easily verify that the moment sequences of f f and f f are ν ν and ν ν, respectively. Also, the reverse statements are true, see (i) and (ii) in Theorem. Let us consider the following conditions: (7) f k (t) t dt <, for k =,. (8) Theorem. Assume that the densities f, f k, k =,, are continuous on [, ]. (i) If ν = ν ν and the conditions (8) are satisfied, then (7) holds with f ν = f f ; (ii) If ν = ν ν, then (7) holds with f ν = f f ; (iii) If ν ={µ j, j N} with µ j = [φ()] j df(), for some continuous and increasing function φ : [, ] [, ], then (7) holds with f ν () = [φ (φ ())]. (iv) If ν = { µ j, j N} with µ j = β µ j,f + β µ j,f, β,β >, then f,ν u β f + β f. (9) f (φ ()) Proof. (i): Assume that ν = ν ν. By substitution of the product µ m+[],f µ m+[],f instead of µ m+[],f on the right-hand side of (5) we obtain

874 R.M. Mnatsakanov / Statistics and Probability Letters 78 (8) 869 877 [] Γ ([]+) m= ( ) m µ m+[],f µ m+[],f m! ( [] m)! [] = (t u) [] ( t u) m f (u) f (t) du dt Γ ([]+) m! ( [] m)! m= [ ] ( τ ) = β(τ, []+, []+) f dτ f (t) t t dt f f () () uniformly on [, ], as. In the last line of () the Lebesgue dominant convergence theorem is used: by taking into account (4) and the conditions of Theorem, we have for each fied t φ (, t) := ( τ ) ( ) β(τ, []+, []+) f dτ f t t uniformly on [, ], as, (see, Feller (97), Ch. VII); on the other hand, the sequence of functions {φ (, ), N} is uniformly bounded on [, ]. Now, in a similar way, as we did in (), we can derive (ii) and (iii). Let us prove only (ii). Replacing the moments µ m,f and µ j m,f in (7) by corresponding integrals taken with respect to df and df we obtain ν j = (t + u) j f (u) f (t) du dt. Since f f is defined on [, ], the substitution of ν j into (6), where T =, yields Γ ([ ] ) + [ = ( t + u ( [ β τ, ] [ +, ) [ ] [ ] m= ] + ) ( ) t+u m m! ( [ ] m)! ] f ( τ t) dτ f (u) f (t) du dt f (t) dt u f f (), as. The proof of statement (iv) follows from Theorem (i) and from the linearity of f,ν with respect to ν. Corollary. (i) If f is continuous and for some a > and b, ν ={µ aj+b,f, j N}, then (7) holds with f ν () = a µ b,f f ( ) a b+ a ; (ii) If f is continuous and ν ={a j µ j,f, j N} for some positive a, then (7) holds with f ν () = f (/a)/a, [, a]. (iii) If f is continuous, f,ν () = f,ν(φ()) φ (), and ν ={µ j, j N} with µ j defined according to (9), then f,ν u f ; (iv) If f is continuous, f,ν () = f,ν( a ) a a, and ν ={µ aj,f, j N} for some a >, then f,ν u f. Proof of Corollary. The statement (i) with b = is a special case of Theorem (iii), where φ() = t a. When b, the proof of (i) is reduced to the case with b = by replacing pdf f with f b () = b f ()/µ b,f. The case (ii) can be proved using (6), with T = a, and with the argument similar to the one used in the proof of Theorem (i): in last line of () we have an etra factor /a, and /a instead of. The transformation φ() combined with the statement of Theorem (iii) yields Corollary (iii), while (iv) is a special case of (iii), where φ() = a. Remark. The construction f,ν from Corollary (iv) can be very helpful when the pdf f has only a finite number of moments up to, say, m <. In this case one can use the fractional moments µ j = µ aj,f, j =,,...,, with a = m /, and reconstruct f by means of f,ν where ν ={µ j, j =,,... } (cf., for eample, Novi Inverardi et al. (3)). ()

4. Some applications and eamples R.M. Mnatsakanov / Statistics and Probability Letters 78 (8) 869 877 875 The constructions () and (5) can be useful in the estimation problem of a cdf F or its density function f in indirect models (cf. Mnatsakanov and Ruymgaart (3, 5)). Apart from being an alternative to the traditional estimation technique in a direct model, this approach is applicable to situations where we do not have the observations from a target distribution. For eample, assume that we have n copies Y,..., Y n from a cdf G which is related in some specific way to the target distribution F with supp{f} =[, ]. Suppose that we can estimate the moments of F from the given data. Denote these estimated moments by ˆν = { ˆµ j,f, j N}. Estimation of f may now be carried out by replacing the moments µ j,f by their estimators ˆµ j,f in (5). This yields a moment-density estimator (mde) of f : f,ˆν () = Γ ([]+) [] m= ( ) m ˆµ m+[],f, [, ], () m! ( [] m)! which can be helpful when only estimated moments of the target distribution F are available. These types of situations occur in many indirect models like length-biased, multiplicative censoring, and miture models where the moments of an unobserved distribution F can be consistently estimated from the moments of an observed distribution G. Let us briefly discuss the direct case (when G = F) and the length-biased model. Direct model. Suppose that the observations X,..., X n from the target cdf F are available. Consider estimator () with the empirical moments: ˆµ j,f = t j d ˆF n (t), j N, ˆF n (t) = n {X i t}. n i= This estimator looks etremely unstable, because the higher-order empirical moments are unstable. But, somewhat surprisingly, substitution of the empirical moments ˆµ j,f into () justifies that the estimator f,ˆν is actually stable: ˆ f () = f,ˆν () = n n β(x i, []+, []+), [, ]. (3) i= Here β(, a, b) is the beta pdf defined in (). The estimator fˆ does not represent a traditional kernel density estimator of f : although the sequence of beta kernels in (3) represents a δ-sequence, the shape of each kernel varies according to the position of a point, where f is estimated. Some asymptotic properties of fˆ such as the Integrated Mean Squared Error, uniform convergence in probability, and convergence in L -norm were studied in Chen (999) and Bouezmarni and Rolin (3). Note that fˆ is free from the edge effects as well. By checking the necessary and sufficient condition for L -consistency of general density estimators (see, Mnatsakanov and Khmaladze (98)), we derive very easily the L -consistency of mde fˆ : Theorem 3. If f is continuous on [, ], then E ˆ f f L, as n and, n. (4) Proof. From () and (3) we conclude E ˆ f = f,ν, while according to Theorem (i), we have E ˆ f f L f,ν f, as. (5) Now, to prove (4), i.e., E fˆ E fˆ L, it is sufficient to show that { F{A,n (δ)} =F : β (τ, []+, []+) f (τ) dτ nδ }, for any δ> and, n (see, Theorem in Mnatsakanov and Khmaladze (98)). But β(τ, []+, []+) c ( ), < <, (6)

876 R.M. Mnatsakanov / Statistics and Probability Letters 78 (8) 869 877 Fig.. Appro. of f by f,ν. is valid for some positive constant c (see, for eample, Chen ()). Hence, application of (5) and (6) yields F{A,n (δ)} f () d β (τ, []+, []+) f (τ) dτ nδ A,n (δ) c f () f,ν f d + c f () d, (7) nδ ( ) nδ ( ) as /n,, n. Remark. The condition /n in (4) is equivalent to the condition nh, h, when = h. The latter condition represents one of the necessary and sufficient conditions for L -consistency in the traditional kernel density estimation (see, for eample, Devroye (984)). Length-biased model. In this case we observe a random variable Y with a cdf y G(y) = W df (), y [, ], where W = df(). It is immediate that the moments µ j,f of F are related to the moments of G according to µ j,f = W µ j,g. The corresponding mde f,ˆν in (), where ˆν ={Ŵ ˆµ j,g, j }, will have the form with fˆ,ˆν () = Ŵ n n i= Y i β(y i, []+, []+), [, ], (8) ( ) Ŵ = t dĝ n (t) and Ĝ n (t) = n n {Y i t}. i= Applying a similar argument used in the proof of Theorem 3, one can derive the L -consistency of mde ˆ f,ˆν as well. We investigate this model in detail in a later paper. Eample. Assume that we want to recover the pdf g of a distribution G whose moments are µ j,g = 3/( j + 3), j N. Since µ j,g = µ aj,f, with the moment sequence µ j,f = /( j + ), j N, corresponding to the uniform pdf f on [,], we conclude from (), where a = /3 and b =, that g() = 3,. The approimation of g() by f,ν () is very good already for = 3 and = k/, k =,,...,.

R.M. Mnatsakanov / Statistics and Probability Letters 78 (8) 869 877 877 Eample. Let us recover the distribution F via moments ν = {µ j,f = 9/( j + 3), j N}. In this case µ j,f = µ j,g µ j,g, j N, so that the pdf f of F has the form (see, Theorem (i)): ( ) f () = g g(τ) τ τ dτ = 9 log,. We conducted the computations of the moment-recovered pdf f,ν () at = k/, k =,,...,, with = 3. See Fig., where we plotted the pdf f (the solid curve), and its moment-recovered pdf f,ν (the dashed curve), respectively. We can see that the approimation is good. Acknowledgments The author is thankful to Estate Khmaladze, Vladimir Hnizdo, and Cecil Burchfiel for helpful discussions and to Artak Hakobyan for conducting calculations in Eamples and. The author is also grateful to the anonymous referee for many helpful comments and suggestions. The findings and conclusions in this paper are those of the authors and do not necessarily represent the views of the National Institute for Occupational Safety and Health. References Akhiezer, N.I., 965. The Classical Moment Problem and Some Related Questions in Analysis. Oliver and Boyd, Edinburgh. Borwein, J.M., Lewis, A.S., 99. Convergence of best entropy estimates. SIAM J. Optim., 9 5. Borwein, J.M., Lewis, A.S., 993. A survey of convergence results for maimum entropy methods. In: Djafari, M., Demoments, G. (Eds.), Maimum Entropy and Bayesian Methods. Kluwer Academic Publishers, pp. 39 48. Bouezmarni, T., Rolin, J.-M., 3. Consistency of the beta kernel density function estimator. Canad. J. Statist. 3, 89 98. Chauveau, D.E., van Rooij, A.C.M., Ruymgaart, F.H., 994. Regularized inversion of noisy Laplace transforms. Adv. Appl. Math. 5, 86. Chen, S.X., 999. Beta kernel estimators for density functions. Comput. Statist. Data Anal. 3, 3 45. Chen, S.X.,. Beta kernel for regression curves. Statist. Sinica, 73 9. Devroye, L., 984. Nonparametric Density Estimation: The L View. John Wiley and Sons, New York. Feller, W., 97. An Introduction to Probability Theory and its Applications, vol. II. Wiley, New York. Frontini, M, Tagliani, A., 997. Entropy-convergence in Stieltjes and Hamburger moment problem. Appl. Math. Comput. 88, 39 5. Goldenshluger, A., Spokoiny, V., 4. On the shape-from-moments problem and recovering edges from noisy Radon data. Probab. Theory Related Fields 8, 3 4. Johnson, N.L., Kotz, S., Balakrishnan, N., 994. Continuous Univariate Distributions. Wiley, New York. Kevasan, H.K., Kapur, J.N., 99. Entropy Optimization Principles with Applications. Academic Press, New York. Lin, G.D., 99. Characterizations of distributions via moments. Sankhya 54 (Series A), 8 3. Lin, G.D., 997. On the moment problem. Statist. Probab. Lett. 35, 85 9. Mnatsakanov, R.M., 8. Hausdorff moment problem: reconstruction of distributions. J. Statist. Probab. Lett. doi:.6/j.spl.8... Mnatsakanov, R.M., Khmaladze, E.V., 98. On L -convergence of statistical kernel estimators of distribution densities. Soviet Math. Dokl. 3, 633 636. Mnatsakanov, R., Ruymgaart, F.H., 3. Some properties of moment-empirical cdf s with application to some inverse estimation problems. Math. Methods Statist., 478 495. Mnatsakanov, R., Ruymgaart, F.H., 5. Some results for moment-empirical cumulative distribution functions. J. Nonparametr. Stat. 7, 733 744. Novi Inverardi, P.L., Petri, A., Pontuale, G., Tagliani, A., 3. Hausdorff moment problem via fractional moments. Appl. Math. Comput. 44, 6 74. Shohat, J.A., Tamarkin, J.D., 943. The Problem of Moments. Amer. Math. Soc., New York. Stoyanov, J.,. Krein condition in probabilistic moment problems. Bernoulli 6, 939 949. Tagliani, A., Velasquez, Y., 3. Numerical inversion of the Laplace transform via fractional moments. Appl. Math. Comput. 43, 99 7.