Deconvolution problems in density estimation

Transcription

1 Deconvolution problems in density estimation Dissertation zur Erlangung des Doktorgrades Dr. rer. nat. der Fakultät für Matematik und Wirtscaftswissenscaften der Universität Ulm vorgelegt von Cristian Wagner aus Öringen 009

2

3 Amtierender Dekan: Erstgutacter: Zweitgutacter: Prof. Dr. Werner Kratz Prof. Dr. Ulric Stadtmüller Prof. Dr. Volker Scmidt Tag der Promotion: 8. Juni 009

4

5 Contents Introduction and Summary Motivation Concepts in density estimation Focus of tis work Deconvolution of densities Results about approximative deconvolution estimators Results about contaminated-data-only models Aggregated data models and corresponding results Conclusions iii iii iii v vi vii viii x xi Density Estimation Metods. Direct density estimation Quality measures Asymptotic properties of te direct density estimation Density deconvolution Classical consistency results Supersmoot target densities Approximative deconvolution metods Aggregated data models Unknown Error Density 9. TAYLEX and SIMEX estimators Justification of te appearing bias reduction Consistency results Simulations Proofs Modified variance Model and estimator Consistency results Simulations Proof of Teorem Additional error Model and estimator Consistency results Proof of Teorem i

6 ii Contents 3 Aggregated Data Models Estimators and assumptions Consistency and minimax rates Properties of te unweigted estimator Aggregated data models in density deconvolution Proofs Proofs of Lemmas 3.. and Proofs of Teorems 3.. and Proof of Teorem Proof of Teorem Proof of Teorem A Appendix 4 A. Spaces of continuous functions A. Integration teory A.3 Fourier transforms A.4 Caracteristic functions A.5 Sobolev spaces A.6 Inequalities B Auxiliary Results 49 List of Figures 55 List of Symbols 57 Bibliograpy 6 German Summary 65

7 Introduction and Summary Motivation In many circumstances repeated measurements of a quantity are observed and one would like to gain as muc information as possible from tese observations. Examples are te log return per day of a company sare, or te policy olders lifespans for life insurances. Oter quantities of interest could be participants blood pressures in a clinical study or ouseolds consumptions of electricity per year. Tere are a variety of oter situations were repeated observations are possible for instance in biology, geology, oter fields of science or social science. In statistics tese repeated measurements are often modeled as realisations of random objects, so-called random variables. Under tis assumption te first caracteristics of te data one migt consider are te mean or te variance. Yet, bot values contain only partial information about te distribution of a random variable, wereas complete information is given by te cumulative distribution function, wit its empirical counterpart, te so-called empirical distribution function. However, wen plotting te empirical distribution function, one only receives a step function and it is ard to obtain more detailed information from tis grap. In case of an absolutely continuous random variable, te density function of te quantity of interest can be estimated to circumvent tis downside. Using an appropriate estimate of te density function allows for information about modes, symmetry, and frequent values of te random variable to be gatered. Even more information about modes or te cange in frequency of te random variable s values sould be attainable troug an estimate of te density s derivatives. Bot te estimation of a density and its derivatives will be addressed in tis work. However, since bot problems lead to comparable considerations, te focus is on density estimation in te introduction. Concepts in density estimation Tere are two basically different approaces for estimating a density. Te first is te parametric density estimation approac, were one assumes tat te observations come from a parametric family of densities tat as to be specified in advance. Ten, te task is to estimate te parameters tat fit te data best. Te second approac is te nonparametric density estimation, were one does not impose a certain functional form on te density. Instead, one tries to find an estimate using only minor assumptions suc as some smootness of te density. Te parametric estimation procedure as advantages but also large drawbacks. Foremost, it is difficult to specify an appropriate parametric family since tis information migt often be not directly accessible from te situation, were te data was observed. However, a misspecification of te family of densities will lead to an estimate tat does not capture important structures of te density; a fact tat contradicts te objective of obtaining as muc information from te data as possible. Additionally, even if one insists on using a parametric approac, a nonparametric estimate will give a good starting point to find an appropriate model. For tis reason, nonparametric density estimation will be studied in more detail ere. iii

8 iv Introduction and Summary Tere are various density estimation procedures in nonparametric density estimation like ortogonal series metods and istograms wit fixed or random partition, among oters. Histograms wit fixed partition were for example studied in Révész [97], for an introduction to te oter mentioned metods see for instance Prakasa Rao [983]. However, te estimators most commonly used in nonparametric density estimation are te so-called kernel density estimators introduced in Rosenblatt [956] and studied in more detail in Parzen [96]. An estimator of tis type as already been studied in a less general setting in Akaike [954]. Due to its importance te explicit formula will be given ere. For n independent random variables X,...,X n tat are identically distributed as X, te kernel density estimator of te density f X at te point ξ is given by ˆf X ξ = n n ξ Xj K, j= were is a positive real number, te so-called bandwidt, and Ky is a so-called kernel function. For instance, te kernel could be a standard normal density. Te kernel density estimator is not very sensitive to te coice of te kernel, in contrast to te coice of te bandwidt. Te bandwidt s importance is justified by te observation tat te kernel density estimator is in general not an unbiased estimator of f X, wic means tat its expected value is not te value f X ξ itself. However, it is common for estimators in nonparametric estimation procedures to be biased. Because of tis bias, tere are two opposing objectives to find an appropriate bandwidt. On te one and, it would be good to coose small suc tat only observations very close to ξ ave an impact on te density estimate at te point ξ. Tis approac will usually give an estimate wit a small bias, wereas it as a large variance since only very few observations determine te value of te estimator. On te oter and, it would terefore be good to utilize a large in order to use many observations, resulting in a small variance but a large bias. Tis so-called bias variance tradeoff is an intrinsic difficulty of nonparametric estimation procedures. Since tis tradeoff can be controlled by te coice of te bandwidt, it is very important to find appropriate ones. In finite samples, tere are various procedures to find reasonable bandwidts. A popular approac in tis context is least squares cross validation developed independently in Rudemo [98] and Bowman [984]. Anoter commonly used tecnique is te plug-in bandwidts selection, in wic an approximative formula for te so-called mean integrated squared error MISE, defined in..5 below, is derived and subsequently minimized in. Furter metods are, among oters, smooted cross-validation, see Müller [985] and Staniswalis [989], or empirical-bias bandwidts selection EBBS, see Ruppert [997]. In te context of tis work, owever, asymptotically optimal bandwidts coices are relevant. Tis means tat a sequence of bandwidts n wit good properties as to be found. Tus, it is te aim to find sequences n suc tat te distance of te resulting estimate to te true underlying density f tends to zero wit te fastest possible rate. Terefore, some concept of distance between an estimator ˆf n and te true density f is required. One possible approac is to define te error only at a single point, but one estimates a function defined on te wole real line, ence it is preferable to measure te distance on te wole real line as well. Te two mostly studied measures of deviation on te real line are te MISE, wic gives te expected value of te squared L -distance between te estimator ˆf n and te target density f, and te mean integrated absolute error MIAE, defined in..7 below, wic utilizes te L -distance. Altoug using te L -norm seems to give te correct distance, considering tat te distance between densities as to be evaluated, te MISE is more commonly used. Tis popularity is justified by te facts tat te MISE allows a direct decomposition in a bias and variance part, see..6 below, and its easier manageable computational properties.

9 Introduction and Summary v Results about te MIAE as a measure of quality in density estimation are for instance given in Devroye and Györfi [985], Devroye [987], and Eggermont and LaRiccia [00]. In order to find te fastest possible rates of convergence for te distances introduced above, it is necessary to restrict te considerations to subsets of all densities, so-called density classes, suc tat te difficulty of te density estimation is comparable for te wole subset. Te optimal rates of convergence of te MISE for different density classes were first studied in Watson and Leadbetter [963]. In Davis [977], it is proved tat te usual kernel density estimator reaces tese rates wen using a special kernel, te so-called sinc-kernel defined in..8 below. It is not clear in advance, owever, tat coosing appropriate bandwidts n te kernel estimator can also reac optimal rates of convergence for more general kernels. Noneteless, e.g. for te MIAE in Devroye [987], see Teorem.. below, a bandwidts coice can be found suc tat te kernel estimator reaces te fastest attainable rates. Te coice of te optimal bandwidts and te optimal convergence rates usually depend on n and parameters influenced by te kernel K and te underlying density f tat are in general unknown in advance. Hence, it could be argued tat for te finite sample case te asymptotically optimal bandwidts coices are not elpful. However, to analyse te convergence properties of an estimator for growing sample size n and to compare te derived rates to te best attainable ones in te corresponding situation are important questions in teir own rigt. Tis importance is justified by te fact tat for growing sample size tese rates indicate ow large te error improvement is tat one can ope for. Te parameter tat essentially determines te optimal convergence rates is te smootness of te density f. In sort, te smooter f is, te faster is te attainable rate of convergence. Yet, for common smootness classes te best attainable rates of te MISE, tat can be reaced uniformly over te wole class, are slower tan te usual parametric rate /n, see Teorem..4 below. It is noneteless possible to reac te rate /n ere too, as proved in Watson and Leadbetter [963]. In tis reference it is sown tat for densities f wit caracteristic function wit bounded support te rate /n is attainable. Moreover, it is proved tere tat tis rate is te best rate any density estimator can reac for arbitrary densities f. It is important to note tat to reac te rate /n in parametric density estimation it is mandatory tat te density follows a parametric model and one specified te correct one beforeand. Focus of tis work Altoug density estimation is well studied, tere are many settings were classical approaces are not directly applicable. From te introductory examples it can already be seen tat in many practical applications direct access to te data of interest migt not be possible. In suc situations, te target density f as to be restored from te data. Wit te purpose of explaining tis problem furter, te example of te blood pressure from above is considered in more detail. Tere, te observations migt additionally depend on te time of day te blood pressure was measured, te person tat performed te measurement, or some plain measurement error. Yet, of interest is only te participant s true blood pressure. Suc models were te data is not observable directly but only contaminated wit some unobservable additive effects are so-called errors-in-variables models. In tis situation a so-called deconvolution problem for te densities as to be solved, wic will be explained a little later. Anoter possible situation, were reconstruction of te density is crucial, is exibited in te model of electricity consumption. Here one migt not only be interested in te consumption per ouseold but also in te consumption per individual. In tis setting, owever, a large amount of te data is not a direct observation of an individual person s consumption but te consumption of a larger ouseold. Hence, tese observations are te sum of te consumptions

10 vi Introduction and Summary of more tan one individual. Here, te data obtained from te larger ouseolds cannot be used directly, wereas from a statistical point of view it would be desirable to include tis data in an estimation procedure. Tis type of models are called aggregated data models, it also will be introduced in more detail in te next sections. Te interest in tis work is on introducing estimators for te different studied settings tat fit realistic datasets better tan te classical approaces. For all estimators teir respective convergence properties will be analysed and, in particular, optimal rates will be derived if possible. Terefore, all proved rates will be compared to te - under additional assumptions - optimal rates tat are known and in one situation a minimax rate of convergence will be proved. Since te interest is on asymptotic properties, data dependent coices of te parameters will not be addressed ere. Deconvolution of densities As explained above, for many realistic datasets an errors-in-variables model is useful. Furter examples and justifications for suc models can for instance be found in Carroll et al. [995]. In tese models, te observable quantity is usually modeled as a random variable W, wic can be written as te sum of te random variable X of interest and te error variable ε. Consequently, te observable random variable is given by W = X + ε, were X and ε are assumed to be independent. Hence, one can only observe a sample from te convoluted distribution and assuming X and ε to be absolutely continuous from te convoluted density f W = f X f ε. Tus, being interested in te density f X and requiring f ε to be known, a deconvolution problem as to be solved. Usually tese problems are easier to solve on te Fourier domain. To find an estimator for f X, one uses te fact tat a convolution becomes a multiplication on te Fourier domain, i.e. ϕ W t = ϕ X tϕ ε t, were ϕ W t, ϕ ε t denote te caracteristic function of te random variables W and ε respectively. Hence, if ϕ ε t is nonzero on te wole real line, it is possible to evaluate ϕ X from ϕ X t = ϕ W t/ϕ ε t. A commonly used idea is to find an estimator of ϕ W t, called ˆφ W t. Afterwards, tis estimator is divided by ϕ ε t and Fourier inversion is applied to define an estimator for f X. In general, te quotient ˆφ W t/ϕ ε t is not integrable, so some regularization tecnique for te inverse Fourier transform is needed. Te amount of necessary regularization depends eavily upon te beaviour of ϕ ε t as t approaces infinity, te so-called tail beaviour. In order to distinguis different tail beaviours, two different types of density classes are usually studied. First, for so-called supersmoot random variables ε te caracteristic function ϕ ε is supposed to ave exponential decay, see..6 below for an exact definition. Tis exponential decay implies tat f ε is infinitely often differentiable, see Teorem A.5.3 below. Te second density class studied consists of so-called ordinary smoot random variables ε, were te caracteristic function ϕ ε is supposed to ave polynomial decay, see..7 below for an exact definition. Here te decay implies te existence of finitely many derivatives of f ε, see again Teorem A.5.3. For te best attainable convergence rates it is very important weter te error density is ordinary smoot or supersmoot. In case of an ordinary smoot density f X te optimal rates are algebraic if te known error density is also ordinary smoot, see Teorem..6 below for a precise statement, wereas te best attainable rates in case of a known supersmoot error density f ε are only logaritmic, see also Teorem..6. Tese logaritmic rates are rater unpleasant since many popular densities are supersmoot, like te normal density for instance. Te first proofs of lower bounds for te rates were given in Fan [99a]. More precisely, it is sown in tis reference tat te convergence rates mentioned before are optimal for te estimation of a density and its derivatives at a point ξ. Te same result for L p -norms over bounded intervals is sown in Fan [993]. Yet, in case te density f X is supersmoot faster rates are attainable. Tere in

11 Introduction and Summary vii some situations even rates close to /n are possible, te exact rates are given in Subsection.. below. Tese faster rates were to te autors knowledge first mentioned in Pensky and Vidakovic [999]. Tere are various different approaces tat reac te optimal rates for ordinary smoot f X assuming tat te distribution of ε is completely known. Te most common metod is te deconvoluting kernel approac, presented in.. below, wic was originally introduced in Stefanski and Carroll [990]. A modification of tis estimator is already studied in Devroye [989]. In Fan [99a,b, 993], not only lower bounds are given, but it is also sown tat te deconvoluting kernel estimator reaces te optimal rates for bot supersmoot and ordinary smoot f ε pointwise and over te wole real line in L -norm. For anoter approac tat reaces tese optimal rates, see for example Pensky and Vidakovic [999], were band-limited wavelets are applied. Furtermore, in Bissantz et al. [007] a density estimator tat applies regularization metods is given, tere convergence rates are derived in two special cases. A furter metod is introduced in Hesse and Meister [004], were an iterative procedure is used to estimate te density f X. Common to all mentioned deconvolution metods is te fact tat one needs complete knowledge about te noise density f ε or its corresponding caracteristic function. Tis is often unrealistic since it contradicts te model assumption tat te error variable ε is unobservable. Yet, it is sown in Meister [004] tat it is crucial to correctly specify te error density wen using a generalized form of te deconvoluting kernel estimator. In tis reference examples are given sowing tat under te assumption of a misspecification of te error density, te MISE even migt tend to infinity for growing sample size n, contrary to te usual situation. For tis kind of misspecification, it is even enoug to correctly specify a normal density but wit a variance tat is too large. Considering te usage of a correctly specified caracteristic function ϕ ε e.g. in te deconvoluting kernel estimator, tere are two possible solutions typically studied in te literature to omit tis prerequisite. One solution is te use of partial information about te distribution of ε to define an approximative deconvolution metod. Anoter alternative is to presume tat observations of te error quantity tat contaminates te measurements are available. One aim of tis work is to furter develop te teory for unknown error variables ε and to introduce new deconvolution models, were neiter observations of ε nor a priori knowledge about ε are necessary. Results about approximative deconvolution estimators Here estimators will be studied tat do not use or try to estimate complete knowledge about te distribution of ε, but utilize only partial information about te distribution of ε. Obviously for tis kind of estimator one cannot ope for consistency in a general setting. Yet, one would ope to improve upon te naive estimator, defined in.. below, wic completely ignores te measurement errors. Moreover, one would expect to furter improve te estimates if one adds more and more information about ε as te sample size n increases. Te first metod studied is te Taylor series expansion corrected estimator TAYLEX introduced in Carroll and Hall [004]. Tis estimator uses only low order moments of ε to correct te kernel density estimator for te effect of te errors-in-variables. Te second procedure is te simulation-extrapolation metod SIMEX introduced in Cook and Stefanski [994] and studied in Carroll et al. [995] and Stefanski and Cook [995]. Tis estimation procedure uses only te second moment of ε to perform a correction for te bias introduced by te errors-in-variables model. Te idea of te SIMEX approac can be utilized in a variety of settings, see Carroll et al. [995]. Yet, in Subsection..3 below its definition togeter wit te TAYLEX procedure will only be given in

12 viii Introduction and Summary te case of density deconvolution. In simulation studies, bot te TAYLEX and te SIMEX metod mostly sow a quite noticeable error reduction wen compared to te naive estimator, see Subsection..3 below. Yet, tere are only few consistency results for tese approximative deconvolution metods. Furtermore, te analysis available for tese procedures is done assuming tat µ, te variance of ε, tends to zero; an assumption tat is not easily justified in te errors-in-variables model. Terefore, te question arises if it is feasible to derive consistency results or even rates of convergence for fixed nonzero µ if te sample size n grows. It is sown in Section. below tat tis question as a positive answer requiring tat te level of information about te distribution of ε grows wit n. To analyse te consistency properties of te TAYLEX and te SIMEX metod in Subsection.. below tree lemmas are given tat explain te bias reduction generated by te SIMEX and te TAYLEX estimators compared to te naive estimator. Tese lemmas ave an apparent interpretation tat sows ow tese estimators correct te error introduced by te errorsin-variables model. In addition, tey are used in te subsection tereafter to prove rates of convergence. More precisely, in Teorem.. below it is sown tat te TAYLEX tecnique can be extended to a consistent estimator of te density f X and its derivatives, provided ε follows a normal distribution and all parameters are cosen to satisfy te constraints given in tis teorem. Particularly, te rates of convergence are derived bot for ordinary smoot and for supersmoot f X. In te situation were te density f X is ordinary smoot, optimal rates are proved now improving sligtly upon te results given in Wagner and Stadtmüller [008]. For supersmoot f X te rates found are in all cases close to te optimal ones for known ϕ ε if optimality results are available in te corresponding situation. A more toroug discussion of te rates is given at te end of Subsection.. below. Te fact tat te assumption of a normally distributed ε can be generalized and te TAYLEX metod still reaces te optimal rates of convergence is sown in Teorem.. below. For tis teorem, f X is supposed to be ordinary smoot and it is assumed tat ϕ ε t can be written as te product of te caracteristic function of a normal density and of a rational function satisfying some constraints. Caracteristic functions of tis type appear for instance if te corresponding density is te convolution of a normal density and so-called double exponential densities, see Table. below for a definition. Consistency results wit rates for te SIMEX metod are given in Teorem..3 below. Tere it is sown tat te SIMEX metod also reaces te optimal rates of convergence for te estimation of a density f X and its derivatives. Tis optimality is proved for ordinary smoot f X and normally distributed ε, provided an appropriate coice of parameters is utilized. Finally, te analysis of te TAYLEX and te SIMEX metod is concluded wit a small simulation study, given in Subsection..3, tat compares tese tecniques to te naive estimator and te deconvoluting kernel estimator. Tis simulation study empasizes te possible advantages of tese estimators compared to te naive estimator. Te point of coosing te normal density or closely related densities as error densities in te teorems given in Section. is not tat tese densities are completely determined by a finite number of moments, but tat te reciprocals of teir caracteristic functions ave a Taylor expansion on te wole real line. Results about contaminated-data-only models As explained above, it is anoter common idea to assume tat direct observations of te error quantity are possible. Tis is done in order to avoid te use of te caracteristic function of te unknown ε. For example in Neumann [997], te effect of not specifying te error distribution a priori but estimating its density f ε is studied. Tere, te usual kernel estimator is modified

13 Introduction and Summary ix to take account of tis situation. Observations of te error quantity ε to construct estimators applicable to density deconvolution are for instance also used in Diggle and Hall [993] and Efromovic [997]. Altoug it is an often used assumption, in most situations observations of te contaminated quantity and te error quantity are not simultaneously available. Consequently, it is of interest to find consistent estimators of te density f X using only contaminated observations. Clearly, tis is not possible in general. Neverteless, if some structure is imposed on te types of observations, it will be sown ere tat a definition of a consistent estimator is feasible. Furtermore, it is even possible to derive rates of convergence. More precisely, in Sections. and.3 below models are studied tat do not use any direct observations of te error quantity or low order moments of ε, but utilize only observations tat can be modeled using two contaminated random variables tat ave a sligtly different structure. Te focus in tese two sections is on introducing corresponding estimators and analysing teir convergence properties. In a first contaminated-data-only model, studied in Section., it is assumed tat two different types of observations are available tat can be modeled as independent realisations of two different types of random variables. One part of te sample is assumed to be identically distributed as W = X + ε, te usual sum of a random variable X of interest and an unobservable error variable ε, were X and ε are required to be independent. Te second part of te sample is assumed to be identically distributed as W σ, were W σ is supposed to ave a lower error level tan W leading to te quantity W σ = X + σε wit some σ satisfying 0 < σ <. Furtermore, X and ε are assumed to be independent satisfying X, ε d = X, ε, i.e. tey are equidistributed. Moreover, bot X and ε are supposed to be absolutely continuous. Suc a model is applicable wenever it is feasible to perform measurements wit a smaller error level but tis is expensive or time consuming. For tese reasons, as little as possible of te improved measurements sould be used. An estimator tat can utilize bot te realisations of W and W σ to define a consistent estimator for te estimation of f X and its derivatives is introduced in.. below. Since te introduced estimator implicitly employs a division by an estimate of ϕ X t, lower bounds for ϕ X t are needed to prove consistency results wit rates. Tis assumption is not commonly used in density deconvolution. Yet, under tis prerequisite te consistency of te estimator can be proved in many situations. Tis can be seen from Teorem.. below, were te corresponding rates of convergence are given. In te mentioned teorem rates of te MISE are derived for all combinations of supersmoot or ordinary smoot f X and supersmoot or ordinary smoot f ε. At te end of Subsection.. below te obtained rates of convergence are compared to te optimal rates for known distribution of ε. From tis comparison wit te - under additional assumptions - optimal rates it can be seen tat in some situations only te square root of te usual rates is reaced. Tus, tere is a price to pay for not requiring a known caracteristic function ϕ ε. It is te aim of ongoing researc to sow tat te alteration of te rates is an intrinsic difficulty of te setting and not a limitation of te introduced estimator. To assess te finite sample properties of te studied estimator, a small simulation study is included in Subsection..3 below. Te second contaminated-data-only model is studied in Section.3 below. Tere again no direct observations of te error quantity are utilized but only measurements of two contaminated quantities. One part of te sample is supposed to be identically distributed as W and te oter part as Z. In tis setup W is required to be distributed as te usual sum W = X +ε. For Z it is assumed tat tis random variable is contaminated not only wit one error variable ε but wit two independent replicates of ε ence it is modeled as Z = X + ε + ε. Here it is assumed tat X d = X, ε d = ε d = ε, and tat bot X and ε are absolutely continuous. Practical settings were suc a model is applicable are for instance situations were a signal travels different distances until it is measured, like in applications in areas of telecommunication or seismology. Tere for

14 x Introduction and Summary larger distances an additional error is added to te error of a sorter distance. In tis setting, it is again feasible to define in.3. below a consistent estimator of f X and its derivatives utilizing only realisations of W and Z. Here as well an implicit division by an estimate of ϕ X t is used and terefore lower bounds for te caracteristic function ϕ X t are inevitable. Under tis assumption, rates of consistency of te introduced estimator are studied in Teorem.3. below tat depend on te different smootness classes f X and f ε are an element of. Finally, te derived rates of consistency are compared to te optimal convergence rates for a known distribution of ε at te end of Subsection.3. below. Once again, te optimal rates cannot be reaced in all situations. Here as well, it is still an open question if tis is a limitation of te studied estimator or an immanent feature of te studied setting. Aggregated data models Oter situations were no direct access to te data associated wit te density f X is possible are te so-called aggregated data models, already mentioned above. Tere, at least part of te data is not a direct observation of te quantity of interest but is only an observation of te sum of more tan one target quantity. Again, tis quantity of interest is modeled as an absolutely continuous random variable X. Moreover, te aggregated data is supposed to be te sum of independent random variables wit te same distribution function as X. Te aggregated data models migt appear in situations were some quantity is measured only over longer periods of time, for instance a week, altoug independent canges could occur eac day. A model of tis type is for example given in Linton and Wang [00], were among oter topics density estimation from grouped and contaminated data is studied. In Meister [007], optimal convergence rates for density estimation from grouped data are derived, for te setting were te size of te different data groups, te so-called group size, is te same. Te application of data only available on an aggregated level is also studied in oter practical settings, see for instance van Dijk and Paap [008] and Ciu and Bois [007]. Results about aggregated data models In te setting of aggregated data models, te question arises if it is feasible to generalize te results given in Meister [007] to situations were different group sizes are allowed. Here it is also of concern weter, correctly standardized, te introduced estimator converges in distribution to a normal distribution. For an answer to tese questions a weigted estimate of te caracteristic function depending on positive weigts w j,n is used to introduce in 3..6 below an estimator of f X. Tis estimator is called unweigted, if all weigts w j,n are equal to one, and weigted oterwise. For te analysis of tese estimators an ordinary smoot target density is required trougout. Wen studying te weigted estimator, bot te weigts w j,n and te group sizes are allowed to vary wit te observation number j and te sample size n. Ten, two different settings are distinguised. In bot settings, te density f X is assumed to be symmetric. Moreover, in one situation it is required tat ϕ X t is nonnegative, wereas in a second situation negative values of ϕ X t are allowed if all group sizes are odd. Ten, Lemmas 3.. and 3.. below give for bot settings coices of te weigts tat are used later on to derive convergence rates of te MISE. Tese lemmas give an upper bound for te MISE, as well. For te two situations distinguised, te rates of convergence of te MISE and optimal bandwidts coices are given in Teorems 3.. and 3.. below respectively. In addition, te corresponding lower bounds are proved in Teorem 3..3 below. More precisely, it is sown tere tat te weigted estimator is consistent wenever consistent estimation is teoretically feasible, and tat in tis aggregated

15 Introduction and Summary xi data model te rates of convergence reaced are te optimal ones for any estimator of f X and its derivatives. See Meister et al. [009] for parts of te results. In te case of an unweigted estimator of f X, defined in 3.3. below, only settings were te group size of observation j does not vary wit n are considered. It is sown in Teorem 3.3. below tat, correctly standardized, tis unweigted estimator converges in distribution to a standard normal distribution for an appropriate coice of te bandwidts. Tis generalizes a result given in Teorem.3. below to te more general setting of different group sizes. Yet, it does not include errors in te aggregated data as done tere. In te last part of tis work, bot te errors-in-variables model and te aggregated data model are combined in one setting. Here again, te caracteristic function ϕ ε is not assumed to be known. Explicitly, a model is introduced in Section 3.4 below, were contaminated measurements are possible, and it is assumed tat observations of te error quantity ε and te sum of two independent error quantities are possible, as well. An exact definition of te available data is given in Ten, in below a deconvolution estimator of f X is suggested tat employs te ideas applied to te aggregated data setting. For ordinary smoot f X and f ε it is sown in Teorem 3.4. below tat te introduced estimator reaces te rates of convergence of te MISE tat are optimal for a known distribution of ε. Tis is te case if eiter enoug direct observations of ε are available or te smootness of f X compared to te smootness of f ε is large enoug. Conclusions Te main focus of tis work lies on generalizations of te usual nonparametric density estimation problem to cases were one does not ave access to te data associated wit f X directly. In te first part errors-in-variables models are studied, were te data is disturbed by additive error effects. Te purpose ere is to get results, wen te strong assumption tat one knows te distribution of te errors is not suitable. Terefore, on te one and rates of consistency for two approximative deconvolution procedures are proved, on te oter and new models are introduced, were only contaminated data, but wit different contamination, is used to derive consistent estimators. In tese new models, in some situations a deterioration of te convergence rates appears, compared to te rates for a completely known error distribution. Yet, it is to be expected tat te rates from classical models are not attainable ere. Since upper bounds for te rates of te estimators are proved ere, it is still an open problem of interest to prove lower bounds to sow tat te observed deterioration is inevitable. However, it is ard to define appropriate caracteristic functions to prove tis fact since tere are only very few applicable results tat sow tat a given function is a caracteristic function. Moreover, adaptive coices of te parameters sould be considered for te estimators tat use neiter te smootness of f X nor of f ε explicitly. In a second part it is assumed tat some of te observations are measurements were te quantity of interest is accumulated. Tis situation is described by te aggregated data models. Tere new estimators for density estimation are introduced tat are able to use data wit different group sizes. It is sown ere tat tese estimators reac te optimal rates of convergence attainable in te studied setting. Tus, it would be interesting to apply te ideas of tese new estimators to te estimation of a regression function, too. More precisely, additional researc is necessary to define a regression estimator and analyse its consistency properties. Bot in errors-in-variables models and in aggregated data models it was possible to introduce estimation procedures tat fit realistic datasets better tan te classical approaces.

16

17 Capter Density Estimation Metods As mentioned in te introduction in many practical settings for example coming from economics, biology or medicine we observe repeated measurements of some quantity of interest. In statistics te resulting observations are modeled using a random variable X. More precisely, we assume tat our data x,...,x n is a realisation of a sequence of independent random variables X,...,X n tat are identically distributed as X. Returning to te practical settings, tere one often tries to gater as muc information from te measurements as possible. For te statistical model tis is possible if one estimates te distribution function of te random variable X, or for absolutely continuous X te corresponding density f X. Terefore, we explain density estimation in more detail in Section.. However, since directly observing te quantity of interest is impossible in many circumstances, two common statistical models are introduced, were reconstruction of te density f X is necessary. On te one and tis is te errors-in-variables model in Section., on te oter and tis is te aggregated data model in Section.3. Moreover, a main focus of our work is on deriving consistency results for estimation procedures. Terefore, in eac of te sections rates of convergence for te corresponding estimation procedures are given.. Direct density estimation Our basic aim is to obtain information about te distribution of X from te sample vector X = X,...,X n T. One possibility to reac tis objective is te empirical distribution function ˆF n ξ defined by ˆF n ξ := n n {Xj ξ}, j= were ξ R. Tis estimator as desirable properties. For example it is unbiased, wat can be seen from te fact tat E[ ˆF n ξ] = F X ξ since n ˆF n ξ follows a binomial distribution wit parameters n and F X ξ. Anoter property is tat n / ˆF n F X = O P as n, and ence we are able to uniformly estimate a function wit te rate known from parametric estimation procedures. Yet, te empirical distribution function as te disadvantage tat, wen plotting it, one does not obtain muc information from te grap. In te case of an absolutely continuous random variable X, one can circumvent tis downside by finding an estimator of te density function f X. From tis estimate one sould be able to obtain more information about te symmetry of te density or its modes. In Akaike [954], te first approac to find a density estimator at a point ξ is studied. Te analysed approac tere is to differentiate te empirical distribution

18 . Direct density estimation function to receive ˆf n ξ = ˆF n ξ + ˆF n ξ = n n j= ξ, ξ+]x j = n n j= [, ξ Xj, were = n > 0 is a so-called bandwidt. Te estimator above is based on Lebesgue s differentiation teorem, see Teorem A..4 below, and is called moving window estimator. It determines te percentage of te observations tat is in a small window about te point ξ and divides tis number by te lengt of te window. Te result is ten te estimate of te density f X at te point ξ. Despite its simplicity te estimator ˆf n as some drawbacks. Easy to observe is te fact tat it is not smoot, but one migt prefer smooter more eye-pleasing estimates. Tus, an apparent generalization of tis estimator, suggested in Rosenblatt [956] and studied in more detail in Parzen [96], is to substitute te function [, y/ by some smoot function. Te used function K is called kernel and is supposed to integrate to one over te real line. Tis procedure leads to te so-called kernel density estimator or Parzen-Rosenblatt estimator. For a good introduction to tis topic see for example Wand and Jones [995]. Applying te kernel K, te resulting estimator can be written as ˆf X ξ := n n ξ Xj K... j= A typical kernel in tis setting is te so-called Epanecnikov kernel Ky = 3 y /4 { y } first studied in Epanecnikov [969]. Here muc weigt is put on observations close to ξ and less weigt is put on observations furter away from ξ. Te advantage of te kernels used in.. is tat tey mostly do not cut off te weigt abruptly, wen leaving te window ξ, ξ + ], as te moving window estimator does, but often ave a smooter transition to zero resulting in a smooter estimate. For example te Epanecnikov kernel is at least continuous at te boundary of its support. Anoter advantage of te kernel estimator, sared also by te moving window estimator, is tat for a nonnegative function K integrating to one te kernel estimator is a density. Yet, it turns out tat te property of non-negativeness is rater restrictive. A furter interesting problem is te estimation of derivatives f l X of te density f X. Tis problem was to our knowledge first studied in Battacarya [967] and Scuster [969]. In bot cases te idea used ere is already applied. Assuming tat te kernel K as l N continuous derivatives, we can differentiate te density estimator l-times to obtain ˆf basic l ξ := n l+ n j= ξ K l Xj,.. were K l denotes te lt derivative of K. Ten.. sould be an estimator of f l X ξ, tat tis intuitive idea really works will be sown later on in Teorem.. below. Tere are various oter metods for density estimation like ortogonal series metods, istograms or istosplines. For furter reference see for example Prakasa Rao [983]. Spline metods are for instance explained in Waba [990]. In order to introduce one metod in more detail ere, we need te Fourier transform from Section A.3. Explicitly, we denote for functions g tat are eiter integrable or square integrable Fg as te Fourier transform of g. Tis definition can be used to define te so-called Fourier inversion metod. For tis approac let l N 0 and suppose tat te kernel K satisfies K l L and t l FK t L. Ten, we can use A.3.

19 . Direct density estimation 3 and Teorem A.3.6 to rewrite our kernel estimators.. and.. as ˆf l basic ξ = n = π n j= l+kl ξ Xj = π n exp it ξ it l FK t n n j= l FK l t exp it ξ X j dt n expit X j dt...3 In te last integral te quantity n j= expit X j/n is te so-called empirical caracteristic function and it is an estimator of ϕ X t, te caracteristic function of f X. Denoting tis estimator by ˆφ X t, te equation..3 can be written as ˆf basic l ξ = π j= exp it ξ it l FK t ˆφ X t dt...4 Te formula above can be taken as a regularized Fourier inversion. We will make use of te Fourier inversion metod for defining estimators of f X in more complicated density estimation models trougout tis work... Quality measures Since one can find different estimators for te density function or its derivatives, it is important to find metods to compare te quality of te estimators. A first idea would be to try to find unbiased estimators of te density f. However, tat tis attempt will not be successful is already sown in Rosenblatt [956]. Explicitly, assuming te density to be continuous and ˆfξ; x,...,x n 0 to be measurable in ξ; x,...,x n, it follows under tese mild prerequisites tat ˆfξ; X,...,X n is not an unbiased estimator of fξ. Hence, we need different metods for appraising te performance of an estimator. Following Prakasa Rao [983], we will mention different quality measures commonly studied in density estimation. For tis purpose, let g denote te density f X itself or one of its derivatives. Suppose tat ĝ n is an estimator of g based on te sample variables X,...,X n tat are independent and identically distributed as X. Ten E fx [ ĝn ξ gξ ] is te mean squared error MSE of ĝ n at te point ξ wit respect to te density f X. Furtermore, [ MISEĝ n := E fx ĝn y gy ] dy = E fx [ ĝn y gy ] dy..5 is te mean integrated squared error MISE of ĝ n wit respect to te density f X. Te standard decomposition of te MISE is given by gy MISEĝ n = EfX [ĝ ny] dy + E fx [ĝn y E fx [ĝ n y] ] dy...6 Te first of tese terms is called te integrated squared bias ISBIAS and te second term is called integrated variance IVAR. It is also possible to use te L norm ere to derive te mean integrated absolute error MIAE defined as [ ] MIAEĝ n := E fx ĝ n y gy dy = E fx [ ĝ n y gy ] dy...7

20 4. Direct density estimation Because we are interested in rates of convergence we sall now introduce quality measures applicable to sequences of estimators. Suppose tat S is a family of univariate densities. A sequence ĝ n of estimators is integratedly consistent in quadratic mean for g if te MISE tends to zero as n, for every f X S, tis means [ lim E f n X ĝn y gy ] dy = 0. Tere are also oter measures of convergence for sequences of estimators ĝ n like for example uniform weak consistency and uniform strong consistency. We will, owever, mostly study estimators tat are integratedly consistent in quadratic mean... Asymptotic properties of te direct density estimation Te asymptotic results we are studying are all based on smootness restrictions for te densities f X. It is a consequence of te next teorem tat tese restrictions are necessary. Tis result can for instance be found in Corollary.0. in Prakasa Rao [983]. Teorem... Define G to be te set of all densities wit te properties tat f is continuously differentiable on R and sup x R fx M for a fixed constant M. Let ˆf n 0 be any estimator of f0. Ten, inf ˆf n [ sup E f ˆfn 0 f0 ] > 0. f G Te teorem above sows tat, if we are coosing our class of densities to large, tere are no estimators tat are consistent uniformly over te wole density class even for te estimation at only one point. We will take account of tis fact by using smootness assumptions for f X trougout tis work. We sall first introduce te concept of a kernel K more precisely. For some n N we call a function K : R R a kernel in Kn if it satisfies te following conditions { y k, for k = 0 Ky dy = 0, for 0 < k < n, Kn y n Ky dy <. Elements of Kn, even wit bounded support say [, ], can be constructed explicitly using a polynomial approac. Ten one can evaluate te moment conditions to find a system of linear equations for te coefficients tat can be solved afterwards. If a kernel K satisfies Kn, for any n N, it is called a superkernel. For constructions of superkernels see for example capter 7. in Devroye [987]. A kernel tat is not exactly a superkernel but as te property tat y k Kydy = 0 for all k N, is te so-called sinc-kernel K s y = π siny y, wit FK s t = [, ] t...8 Considering te caracteristic function of te sinc-kernel, we see tat tis kernel is an element of te Sobolev space W s R from A.5. for all s N. Terefore, it follows from Lemma V.. in Werner [997] tat a similar result to te one given in Teorem A.3.6 olds ere, too. More precisely, for t R we ave FK k s t = it k FK s t for all k N...9

21 . Direct density estimation 5 Wen we are measuring te quality of our estimators in te L -sense, we need te notion of a parameter m associated kernel L for a kernel K tat satisfies Kn. Tis notion was introduced in Bretagnolle and Huber [978], a definition can also be found on page 04 in Devroye and Györfi [985]. Here we will denote by C m,λ,m R te subset of C m,λ,m R consisting of densities wit support [0, ], were C m,λ,m R is from Definition A... Ten te following upper bound for te MIAE can for example be found in Teorem 7.5 in Devroye [987]. In tis result E fx denotes te expectation wit respect to f X. Teorem... Consider for l = 0 te estimator ˆf basic 0 defined in... Suppose tat m N 0, 0 λ <, and M > 0 are given and coose a symmetric kernel K satisfying K ydy < and Kn, wit n > m. Define wit te parameter m associated kernel L θ λ := y λ Ly dy < and put Ten, = fx sup E fx y f X C m,λ,m R K ydy / /m+λ+/ 4 m + λm θ λ n /. 0 y dy ˆf basic + o C n m+λ/ m+λ+ θ λ M K ydy m+λ / m+λ+, were C = / + m + λ m + λ / m+λ+. Also from Devroye [987] we ave a lower bound of te same order given by inf ˆf n sup E fx f X C m,λ,m R f X y ˆf n y dy cm, λ + o M / + m+λ n m+λ, were cm, λ depends on m and λ only and ˆf n is any measurable function based upon te sample variables X,...,X n. Terefore, te kernel estimator reaces te so-called minimax rate, wic is te fastest rate of convergence any estimator can reac uniformly over te wole density class. Tus, one can only improve upon ˆf basic 0 by finding an estimator tat leads to a smaller constant tan te one given in Teorem... Hence, altoug ˆf basic 0 defined in.. generalizes a very simple idea it reaces te best attainable rates in tis situation. For density estimation often Hölder classes are used to define te smootness of te density f X, see for instance te teorem above. However, in more recent works, see Butucea [004] among oters, tis is often replaced by te assumption tat te density belongs to a subset of W s R a so-called Sobolev class given by { WMR s := f L } f is a density and + t s Ff t dt < π M,..0 were s, M > 0. Ten te Sobolev imbedding teorem, see Teorem A.5.3 below, implies tat elements of tis Sobolev class ave derivatives up to order l, were l is te largest integer satisfying l < s /. Additionally, we know tat te lt derivative satisfies a Hölder condition of order λ = s / l if tis is not equal to one, for te case λ = see Remark A.5.. Hence, te