Estimation of boundary and discontinuity points in deconvolution problems

Transcription

1 Estimation of boundary and discontinuity points in deconvolution problems A. Delaigle 1, and I. Gijbels 2, 1 Department of Matematics, University of California, San Diego, CA USA 2 Universitair Centrum voor Statistiek, Katolieke Universiteit Leuven, Belgium. Abstract We consider estimation of te boundary of te support of a density function wen only a contaminated sample from te density is available. Tis estimation problem is a necessary step wen estimating a density wit unknown support, different from te wole real line, since ten modifications of te usual kernel type estimators are needed for consistent estimation of te density at te endpoints of its support. Boundary estimation is also of interest on its own, since it is te basic problem in, for example, frontier estimation in efficiency analysis in econometrics. Te metod proposed in tis paper can also be used for estimating locations of discontinuity points of a density in te same deconvolution context. We establis te limiting distribution of te proposed estimator as well as approximate expressions for its mean squared error, for various types of error densities, and deduce rates of convergence of te estimator. Te finite sample performance of te procedure is investigated via simulation. Key words and prases: asymptotic distribution, boundary points, deconvolution problem, density estimation, diagnostic function, discontinuity points, endpoints, rates of convergence. Supported in part by a Belgian American and Educational Foundation Post-Doctoral Fellowsip. Tis researc was supported by Projet d Actions de Recerce Concertées No. 98/ from te Belgian government. Financial support from te IAP researc network nr P5/24 of te Belgian Government Belgian Science Policy is also gratefully acknowledged. 1

2 1 Introduction In tis paper, we consider kernel estimation of boundary points and discontinuity points of a density from a contaminated sample of tat density, i.e. from a sample tat contains measurement errors. Te contamination problem, often referred to as a deconvolution problem, as applications in many different fields suc as cemistry or public ealt. In tis context a so-called deconvolution kernel estimator of te density as been proposed in te literature. See, for example, Carroll and Hall 1988 and Stefanski and Carroll Tis deconvolution kernel density estimator, owever, is not consistent at a discontinuity point or at a finite left/rigt endpoint of te density to estimate, and as to be modified by taking tese points into account. See for example ang and Karunamuni 2000 for te modifications to apply in te case of boundary points. It is necessary to provide good estimators of tese boundary points or, more generally, discontinuity points wen tey are unknown. Boundary estimation also arises wen investigating efficiencies of firms like banks, or public services. Tese investigations involve estimation of quantities suc as te maximum level of output tat can be produced for a given level of input, wic is often referred to as an economic frontier estimation problem, but can be seen as a problem of estimation of te boundary of a density. Many different metods ave been proposed to estimate a frontier in te case were te observations do not contain any measurement error, but tese metods do generally not provide consistent estimators of te more realistic stocastic frontiers, i.e. of frontiers or boundary points to be estimated from data tat are contaminated by noise. Boundary estimation from contaminated data as been studied by Kneip and Simar 1996, Neumann 1997b or Hall and Simar 2002, for example. Tese papers, owever, eiter focus on very specific contexts, or propose metods tat are difficult to implement in practice. Our goal is to provide a metod tat works in more general contexts, and to provide a way to implement te metod in practice. Te idea is to estimate te boundary point by te maximiser of a certain diagnostic function. Tis is related to procedures used in te error-free case to estimate discontinuity points of a density or a regression function. Similarly as for density estimation in te deconvolution context, te beaviour of te 2

3 proposed estimator depends strongly on te type of error tat contaminates te data. See for example Fan 1991c, wo considers two classes of error densities: te ordinary smoot and te supersmoot error densities. We prove consistency of te proposed boundary estimation metod for bot classes of error densities. Tis paper is organized as follows. In Section 2 we present te problem of boundary point estimation and introduce te estimation procedure. In Section 3 we establis te asymptotic distribution for te estimator and deduce approximate expressions for its bias and variance. In Section 4, te finite sample performance of te procedure is illustrated on simulated examples. Te proofs of te results are given in Section 5. 2 Te estimation metod Suppose we are interested in a density f X, but we observe an i.i.d. sample Y 1,..., Y n from te density f Y, were Y i = X i + i, i = 1, 2,..., n, and were for all i, i is a r.v. independent of X i, of known continuous density f, representing te error in te data and X i is a r.v. of density f X. Te case were f is totally or partially unknown may also be considered, if furter observations, suc as for example a sample from f itself, are available. See for example Barry and Diggle 1995, Neumann 1997a and Li and Vuong In te case were te density f X is continuous, a so-called deconvolving kernel density estimator of f X as been proposed. Consider a kernel function K and a smooting parameter = n > 0, depending on n, called te bandwidt. Te deconvolving kernel estimator of f X is ten defined by f X x; = 1 n n j=1 x Yj K, 2.1 were K u = 2π 1 e itu ϕ K t/ϕ t/ dt, wit ϕ L te Fourier transform resp. caracteristic function of a function resp. random variable L. See Carroll and Hall 1988 and Stefanski and Carroll 1990 for an introduction to tis estimator. Trougout tis paper, we assume tat f is real and symmetric and tat, for all t IR, ϕ t 0. In order to guarantee tat te integral in 2.1 exists, we coose te kernel K to be a real, continuous 3

4 and symmetric function, suc tat ϕ K as a compact support [ B K, B K ], wit 0 < B K <. Note tat, under our assumptions, K is real and symmetric and ϕ K <. In tis paper, we consider te case were te uncontaminated density f X as one or two finite boundary points and f X is not continuous in tese points, wic is of particular interest wen estimating an economic frontier or as a first step to kernel estimation of f X. Wen te data of interest are observed directly i.e. witout error, a simple and consistent approac for estimating te boundary points is to estimate te left endpoint of te support by te smallest observation, and te rigt endpoint by te largest observation. In te case of measurement error owever, tese simple estimators miny 1,..., Y n and maxy 1,..., Y n will not be consistent estimators of te boundary points of f X but rater of tose of f Y. Hence, we need a more elaborated procedure. Te metod we propose uses te fact tat a boundary point is a particular discontinuity point of te density. Te idea is ten to use metods tat exist in te error-free case to detect a discontinuity point, and adapt tem to te case of boundary point estimation wit contaminated data. We focus on kernel metods. In te error-free case, several suc metods to detect a discontinuity point ave been proposed. Tey are all based on te following basic idea: estimate a discontinuity point by te maximizer of an appropriate diagnostic function. Cu and Ceng 1996 coose as diagnostic function te difference of two kernel density estimators. Couallier 1999, 2000 uses te derivative of a kernel density estimator. See Müller 1992, Wu and Cu 1993, Gijbels, Hall and Kneip 1999, Goderniaux 2001 and Gijbels and Goderniaux 2004, among oters, for similar metods in te regression context. We propose a diagnostic function based on derivative estimation. For a density f X wit a single boundary point τ, we define te estimator of τ by τ = argmax x Ĵx, 2.2 were te diagnostic function Ĵx is proportional to te derivative of te deconvolution n kernel density estimator of f X : Ĵx = 1 K x Yi n. Unlike fx, te kernel estimate f X is i=1 a smoot function, even in τ. In te current context, it is to be expected tat tis estimate will be continuous but wit large derivatives wen approacing te endpoints large positive 4

5 derivative for a left endpoint and large negative derivative for a rigt endpoint. In te next section, we prove tat te metod is consistent, and provide asymptotic distribution of te estimator. 3 Asymptotic distribution of te estimator Consider a density f X wit a single boundary or discontinuity point τ. In tis section we sow tat te estimator for τ introduced in te previous section is a consistent estimator, and establis its asymptotic law. Te basic ideas of proof use tecniques and conditions somewat similar to tose used for proving te consistency of a discontinuity point estimator in te error free case. See for example Müller 1992, Cu and Ceng 1996 and Couallier In particular, we assume tat τ lies in a compact interval [A, B], and tus our estimator is defined as τ = argmax Ĵx. We partition te interval [A, B] in x [A,B] n1+q intervals of equal size, and define E n as te set of endpoints of te partition. More precisely, let E n = {z 0,..., z n 1+q}, were z 0 = A < z 1 <... < z n 1+q = B, and z j+1 z j = B A/n 1+q for j = 0,..., n 1+q 1. Note tat, unlike te error free case, were te kernel K is usually a positive, bounded and compactly supported function, te pseudo kernel K of te error case is supported on te wole real line, not positive everywere and is asymptotically unbounded. Hence, despite some similarities in te main ideas, te error case is muc more difficult to deal wit. In particular, te asymptotic properties of our estimator depend strongly on te error distribution, since te latter dictates te beaviour of K. As in Fan 1991c we consider two types of error distributions, te ordinary smoot distributions and te supersmoot distributions. Definition 1. Te distribution of a random variable is said to be i ordinary smoot of order β if its caracteristic function ϕ t satisfies: d 0 t β ϕ t d 1 t β as t, for some positive constants d 0, d 1 and β. ii supersmoot of order β if its caracteristic function ϕ t satisfies: d 0 t β 0 exp t β /γ ϕ t d 1 t β 1 exp t β /γ as t, for some positive constants d 0, d 1, β, γ and 5

6 constants β 0 and β 1 ; We will see tat for supersmoot error densities e.g. normal and Caucy densities te rate of convergence of te estimator is logaritmic. Tis rate is muc faster algebraic for ordinary smoot error densities e.g. gamma and Laplace densities. Te same distinction sows up wen considering deconvolving kernel density estimation see for example Fan 1991c. In wat follows, we treat te ordinary smoot and te supersmoot error cases separately. We first define some useful notations. For any set D IR and positive integer m, let C m D denote te set of functions m times continuously differentiable on D and define D m D = { f Cm D : sup 0 j m sup x D f j x < }. For a square integrable function f, let also Rf denote f 2 x dx. Finally, let d denote te size of te discontinuity of f X in τ, i.e. d = f X τ + f X τ, were, for any function g and point a IR, we use te notation ga + = lim x > a gx and ga = lim x < a gx. Ten, we define te function r X by r X = f X d I [τ,+ [. Clearly, r X is continous on IR, and, in particular in τ, since we ave r X τ = r X τ + = f X τ. 3.1 Ordinary smoot error case In te case were te error is ordinary smoot of order β, we introduce te following assumptions. Condition A: A1 K C 3 IR is a symmetric, kt order kernel k 2 even, suc tat K = K0 > max x 0 Kx, K 0 < 0, and uk r u du < for r = 0, 1, 2, 3; A2 r X is Lipscitz continuous wit Lipscitz constant L; A3 f Y is differentiable on IR \ {τ} suc tat sup x IR\{τ} f l Y x < for l = 0, 1; A4 K C 4 IR is suc tat K u du = O β and u K u 2 du = O 2β, and, for r = 0, 1,..., 4, K r = O β and RK r 2β ; A5 0 as n, suc tat, for some 0 < δ < 1/2 and p IN 0 = IN \ {0}, n 2+2δ+β 1+q n=1 n1+q p 4δp p 2βp <, wit q IN 0 as at page 4. 6

7 Altoug some of tese conditions look rater involved, tey are quite common in density deconvolution problems. A discussion on tese and similar conditions is provided in Delaigle Tere, it is sown tat te conditions can be expressed in a rater simple form, but to te extent of less generality of te functions f and K. In particular, Condition A1 is satisfied by most kernels commonly used in deconvolution problems. See also Fan 1991a,b and Delaigle and Gijbels 2002,2004a,b. Under te conditions stated, we prove te asymptotic normality of te estimator. We approximate te first two moments of te estimator by te first two moments of te asymptotic distribution, and deduce an approximation of te Mean Squared Error MSE of te estimator τ. We discuss rates of convergence of te estimator to its target point τ. Te proofs of te results are deferred to Section 5. Te asymptotic distribution of te estimator is described in te next teorem. Tis distribution depends on te number of derivatives l of te function r X. We need to distinguis te case l = 0 from te case l 1. Teorem 3.1. Suppose te error is ordinary smoot of order β. Under Conditions A1 A5, if r X C l IR D 3 IR \ {τ} wit l 0, and u 3 K u du <. Let k 2 = 0 if l = 0 and 1 oterwise. Ten, for = on 1 2β+2k 2 +5, we ave n [ τ τ R K k2+2 D τ dk 0 R K ] L N 0; B τ, 3.1 d 2 {K 0} 2 were D τ = 1k 2 +1 k 2 +1! [rk 2+1 X τ k 2+1 r k 2+1 X τ ] 0 uk 2+1 K u du and B τ = [f Y τ + + f Y τ ]/2. An approximation of te mean squared error AMSE of te estimator τ of τ can be found by using te moments of te asymptotic distribution. 7

8 Corollary 3.1. Under te conditions of Teorem 3.1, we ave, AMSE[ τ] = 2k 2+4 Dτ 2 d 2 {K 0} + R 2 K Bτ nd 2 {K 0} Wen r X C l IR, wit l > 1, we see tat we obtain te same asymptotic expression watever te value of l. If D τ = 0, one as to go one or several steps furter in te Taylor expansions used in te proofs, until finding a non-zero leading term. Te above results sow tat, te larger te discontinuity, te easier te estimation, wic is easy to understand intuitively, as a large discontinuity is more likely to produce large derivatives of f X, and tus easily detectable maxima for te diagnostic function Ĵ. From Teorem 3.1, we deduce tat R K τ τ = O P + O k n Under te conditions of te teorem, we know tat R K is of order 1 2β by condition A4. To obtain rates of convergence of te estimator, we need to investigate bot terms at te rigt-and side of expression 3.3 and make te distinction between te case were te exponent 1 2β is 0 and te case were 1 2β < 0. Similarly, tis distinction is coming up wen looking at AMSE in 3.2. If 0 β 1/2 and ence 1 2β 0, minimization of 3.3 wit respect to leads to coosing as small as possible. Under te conditions of te teorem, we can take n 1 2β+1 +η, wit η > 0 wic provides a rate of convergence sligtly slower tan n 1 2β+1, more precisely, τ τ = O P n 1 2β+1 +ɛ wit ɛ > 0. For β > 1/2, or equivalently 1 2β < 0, we see tat te optimal bandwidt is te balancing bandwidt i.e. te bandwidt wic makes te two terms of 3.3 of te same order. If D τ 0, tis bandwidt satisfies n 1 2β+2k From 3.3, we conclude ten tat τ τ = O P n k β+2k

9 3.2 Supersmoot error case If te error is supersmoot of order β wit β 0 and β 1 as in Definition 1, we assume, for simplicity, tat te support of ϕ K is [ 1, 1], i.e. B K = 1, and we introduce some conditions. Condition B: B1 B3: te same as Conditions A1 A3, wit te extra condition t 2r 2β 0 ϕ K t 2 dt < in B1; B4 K C 4 IR is suc tat, for r = 0, 1,... 4, K r = O β 0 exp β /γ, RK r = O 2β 0 exp2 β /γ, and K u du = O β 3 exp β /γ, wit β 3 a real constant; B5 f Y τ > 0 and for n large enoug, 1 {K y} 2 f Y τ y dy c 5 f Y τ c 6 exp 2 β γ 4βb n β γ 3.4 were b = β/4+10, c 5 is a positive constant and c 6 is a constant depending on β 0. As in te ordinary smoot error case, tese conditions are rater common for tis difficult class of error densities. Fan 1991a proves tat B5 is satisfied under mild additional assumptions on K and f. See Delaigle 2003 for similar results for Condition B4. If te error is supersmoot of order β and if Condition B is satisfied, we obtain te same asymptotic law of te estimator as in te ordinary smoot error case. From tere, we deduce te same approximate expression for te MSE of τ. However te rates of convergence of te estimator τ to its target point τ are different. Te asymptotic distribution of te estimator τ is provided in te following teorem. Note tat in te supersmoot error case te study of te variance of te random quantity Ĵ τ is more tedious tan in te ordinary smoot error case. Te proofs of te results can be found in Section 5. 9

10 Teorem 3.2. Suppose te error is supersmoot of order β > 0 and = d 2/γ 1/β ln n 1/β wit d > 1. Under Conditions B1 B5, if r X C l IR D 3 IR \ {τ} wit l 0, and u 3 K u du <. Let k 2 = 0 if l = 0 and 1 oterwise. Ten τ τ E[Ĵ τ] VarĴ τ dk 0 VarĴ τ L N 0; 1, 3.5 d 2 {K 0} 2 wit E[Ĵ τ] = k 2+1 D τ + O k 2+2, and were VarĴ τ = n 1 { K u}2 f Y τ u du + On 1. Furter, if u K u 2 du = ORK, we can write [ τ τ n R E[Ĵ τ] K dk 0 R K ] L N 0; B τ. d 2 {K 0} 2 Te following corollary establises an approximate mean squared error of te estimator τ. Corollary 3.2. Under te conditions of Teorem 3.2, we ave AMSE[ τ] = 2k 2+4 Dτ 2 d 2 {K 0} + 2 VarĴ τ d 2 {K 0} 2 If we furter assume tat u K u 2 du = ORK, we ave AMSE[ τ] = 2k 2+4 Dτ 2 d 2 {K 0} + R 2 K Bτ nd 2 {K 0}. 2 Under te conditions of te teorem, we ave RK = O 2β 0 exp2 β /γ. Wit te condition imposed on te bandwidt, we obtain, from 3.3, tat τ τ = O P ln n k 2 +2/β, wic is a muc slower rate of convergence tan in te ordinary smoot error case. 4 Simulations In tis section, we illustrate te finite sample performance of te procedure on a few examples. As noted in Section 2, te estimator f X is expected to ave large positive derivative for a left endpoint and large negative derivative for a rigt endpoint. Hence, in practice, our estimator can be calculated as τ = argmax x Ĵx, respectively τ = argmin x Ĵx, for a 10

11 Jx Jx Jx x x x Jx Jx Jx x x x Figure 4.1: Typical sape of Ĵx for a sample of size n = 100 from Density #3 contaminated wit a Laplace error, for increasing bandwidts from te left to te rigt and from te top to te bottom. left respectively rigt boundary point τ. In te case of two boundary points τ 1 and τ 2, we define, using similar ideas, te estimators τ 1 = argmax x Ĵx and τ 2 = argmin x Ĵx. Note tat te above metod may be applied for te estimation of a discontinuity point as well, but, in tat case, one as no information on te sign of te discontinuity and as to stick to te original definition 2.2. Typical sape of te diagnostic function Ĵ is illustrated in Figure 4.1 for increasing bandwidts ere for a sample of size n = 100 from Density #3 below contaminated by a Laplace error, wit te actual endpoints ere 3 and 3 indicated by vertical lines. We see tat te diagnostic function is indeed maximized at points close to 3 and minimized at points close to 3. We select te bandwidt by estimating te asymptotic MISE optimal bandwidt for estimating te derivative of a density developed in te case of continuous and differentiable densities, i.e., for a second order kernel K, we estimate te bandwidt tat minimizes AMISE { f X x; } = 2πn 5 1 t 4 ϕ K t 2 ϕ t/ 2 dt µ2 K,2 θ 4, were µ K,2 denotes te second moment of te kernel K, and θ 4 = Rf 4 X is unknown. We use a plug-in estima- 11

12 density x x density density density x x Figure 4.2: Densities wit a bounded support. Density #1 top left panel, density #2 top rigt panel, density #3 bottom left panel and density #4 bottom rigt panel. tion metod similar to te plug-in bandwidt selector of Delaigle and Gijbels 2002, 2004b. More precisely, we coose = argmin AMISE, wit AMISE = 1 2πn 5 t 4 ϕ Kt 2 4 dt + ϕ t/ 2 4 µ2 K,2 θ 4, 4.7 were θ 4 is te two-stage plug-in estimator of θ 4 proposed by Delaigle and Gijbels See Delaigle and Gijbels 2002 for detailed implementation of te procedure. We use te second order kernel K corresponding to ϕ K t = 1 t [ 1,1] t, commonly used in deconvolution problems. We consider four densities wit a compact support, illustrated in Figure 4.2: 1. Density #1 a uniform density: f X x = 1/3 1 [0,3] x; 2. Density #2 a concave density: f X x = 3/175 x 2 + 6x [0,5] x; 3. Density #3 a sinus type density: f X x = sin 2 x/2 + 2/15 sin 3 1 [ 3,3] x; 4. Density #4 a multimodal density: f X x = sin x + 1.1/28.5 cos 25 1 [0,25] x. For eac of te above densities #1 4, we ave generated 100 samples of size n = 100 and 250, contaminated by a Laplace error wit a noise to signal ratio Var / Var X = 10%. 12

13 Figure 4.3: Scatterplots of 100 replicated estimators for samples of size n = 100 left panels, or 250 rigt panels, from density #1 top panels, density #2 second line panels, density #3 tird line panels or density #4 bottom panels contaminated by a Laplace error wit a noise to signal ratio Var / Var X = 10%. Estimates of te rigt respectively left endpoint are indicated by te caracter respectively +. Figure 4.3 sows scatterplots of te 100 replicated estimators of te left and rigt endpoints, indicated by respectively te caracters + and. Te true endpoints are indicated by orizontal lines. From tat figure, we see tat te metod performs quite well, even for te more difficult Densities #2 and #4, and te results improve as te sample size increases. As 13

14 one could ave expected, te left endpoint of Density #2 is more difficult to estimate tan te rigt endpoint, because it as a smaller jump size, but yet, we see tat te bias decreases as te sample size increases. See Delaigle and Gijbels 2004c for more detailed results on tis type of problem and oter more callenging difficulties. 5 Proofs of te results In tis section we prove te results of Sections 3.1 and 3.2. For u < 0 and r X C l+1 IR\{τ}, te lt order Taylor expansion of r X around a boundary point τ may be written as r X τ +u = r X τ + ur X τ ul l! rl X τ + R l τ, were R l τ = l + 1! rl+1 X τ + θu, wit 0 < θ < 1. We obtain a similar expansion for u > 0, wit τ + instead of τ. ul Auxilary results for te ordinary smoot case Te following sequence of lemmas lead to te proof of Teorem 3.1 and Corollary 3.1. Trougout, K is a kt order symmetric kernel wit k 2. Te following conditions will be useful. Condition C: C m 1 K C mir is suc tat lim x K m 1 x = 0; C m 2 K C m IR; C m 3 u K m u 2 du = O 2β ; C m 4 K m u du = O β ; C m 5 K m = O β ; C m 6 RKm 2β ; C m 7 uk m u du <. Te next lemma is a generalization of a result of Stefanski and Carroll See Delaigle and Gijbels 2002 for a proof. Lemma 5.1. Let r 0. If K C r IR, we ave E [ K r x Y ] [ ] = E K r x X. Lemma 5.2. Assume Conditions C 2 1 and C2 2, and r X C l IR D 3 IR \ {τ} wit l 0. Let k 2 = 0 if l = 0 and 1 oterwise. Ten, if K 0 = 0 and u 3 K u du <, [ 1 E n n i=1 K τ Yi ] = k2+1 D τ + O k

15 Proof. From Lemma 5.1 and te condition K 0 = 0, we can write [ 1 E n n i=1 K τ Yi ] 0 + = K ur X τ u du + K ur X τ u du. 0 A Taylor expansion of r X of order 2 around τ resp. τ + for u > 0 resp. u < 0, combined wit te fact tat u j K u du = 0 for j = 0, 1, provides te result. Lemma 5.3. Let r 0. Under A2, C r+1 1, C r+1 2 and C r+1 7, we ave, for all x, uniformly in x. E [ r Ĵ r x ] = dk r x τ + O, Proof. Follows from Lemma 5.1 and Lipscitz continuity of r X. Lemma 5.4. Let r 0. Under Conditions A2, A3, C r+1 1, C r+1 2, C r 3, Cr+1 7 and if K r is symmetric, we ave [ Var 1 K r τ Y ] = f Y τ + + f Y τ 2 {K r u}2 du + O 2β. Proof. From Lemma 5.3, a first order Taylor expansion of f Y around τ + or τ and te symmetry of K r, we ave were R 2 [ Var 1 K r τ Y ] sup f Y x x IR\{τ} = 1 {K r = f Y τ + + f Y τ 2 u K r u 2 du = O 2β. u}2 f Y τ u du + O1 {K r u}2 du + R 2 + O1, Te next lemma generalizes a result of Fan 1991a to te case were te density f X is not continuous. 15

16 Lemma 5.5. Let r 0. Under Conditions A2, A3, C r+1 1, C r+1 2, C r 3, Cr 4, Cr 5, C r 6, Cr+1 7, and if K r 1 n n i=1 Proof. Denoting 1 K r is symmetric and n as n, we ave K r τ Yi Var [ E [ 1 n n i=1 1 n K r n τ Yi ] ] L K r i=1 τ Yi N0; τ Yi by Yn,i, i = 1,..., n, were, for all i j, Y n,i Y n,j, we use te central limit teorem for triangular arrays of random variables. See for example Serfling 1980, page 32. We need to verify te following Lyapounov condition: for some η > 0, n E Y n,1 EY n,1 2+η lim = n n VarY n,1 2+η/2 From Minkowski s inequality, we ave E Y n,1 EY n,1 2+η { E Y n,1 2+η} 1 2+η +E Y n,1 2+η. Under te conditions of te lemma, E Y n,1 sup x IR\{τ} f Y x K r u du = O β, and 1+η E Y n,1 2+η K r 1+η sup x IR\{τ} f Y x K r u du = O β2+η. We conclude tat 1+η E Y n,1 EY n,1 2+η = O β2+η. From te proof of Lemma 5.4, we ave VarY n,1 2β 1, and te Lyapounov condition is satisfied. Lemma 5.6. Let r 0. Under A2, C r+1 1, C r+1 2, C r+1 5, C r+1 6, C r+1 7, if sup x IR\{τ} f Y x < and if n as n, we ave, for all p IN 0 and for n large enoug, { 2 E[ r Ĵ r x r E Ĵ r x] 2p 2n p p 2βp π Proof. Let T j denote K r+1 x Yj. We ave E[ r Ĵ r x r E Ĵ r x] 2p = 1 n 2p E = 1 n 2p n i= n 2p E [ T i ET 1 ] 2p + 1 n 2p i 1 i 2... i p i p n i 1,i 2,...,i 2p =1 n i=1 j i i 2p t 2r+2+2β ϕ K t dt j=i 1 [ Tj ET 1 ] 2p 2 l i =2 } p. sup f Y x 2/d 0 2 x IR\{τ} E [ T i ET 1 ] li E [ T j ET 1 ] 2p l i j=i 1 E [ T j ET 1 ]2, 5.4 since, from E [ T i ET i ] = 0, we only ave to consider te terms of te sum were at most p different indices appear and eac index i k appears l ik 2 times. Let C j l denote te binomial 16

17 coefficient. We ave, for all l 2, E [ T i ET 1 ]l = l [ 1 j C j l E j=0 K r+1 since, by Lemma 5.3, we ave E [ K r+1 x Yi ] j [ similar to te proof of Lemma 5.5, we ave E [ K r+1 E K r+1 x Y1 ] l j = O 1 lβ, x Y1 ] = O, and, for all j 2, using arguments E[ r Ĵ r x r E Ĵ r x] 2p =O n p p 2βp, x Yi ] j = O 1 jβ. Finally we get since n as n. From te above calculations, we also see tat we can write p 1 E[ r Ĵ r x r E Ĵ r x] 2p l=0 n l { = n 2p = 1 { [ E n p 2p 1 n p 2p { 2 K r+1 sup x IR\{τ} { n p p 2βp 2c β K E [ T 1 ET 1 ] 2 } p 1 + o1 x Y1 ] 2 } p 1 + o1 f Y x RK r+1 } p 1 + o1 sup x IR\{τ} f Y x 2/d 0 2 } p 1 + o1, were c β K = π 1 t 2r+2+2β ϕ K t 2 dt, and details for obtaining te last inequality can be found in Delaigle and Gijbels Lemma 5.7. Let r 0. Assume Conditions A2, A5, C r+1 1, C r+2 2, C r+1 5, C r+2 5, C r+1 6 and C r+1 7. Furter assume tat K r+1 <, and sup x IR\{τ} f Y x <. Ten, if n as n, we ave for all ɛ > 0 n=1 wit δ > 0 as in Condition A5. P 2δ sup x [A,B] r Ĵ r x r E Ĵ r x > ɛ <, 5.5 Proof. Let E n be as defined on page 5. For eac x in [A, B] tere exists at least one point z in E n suc tat x z B An 1+q. Denote tat point by zx. For all ω Ω, te sample space, we ave sup r Ĵ r x r E Ĵ r x S 1,n + S 2,n + S 3,n, 5.6 x [A,B] 17

18 were S 1,n sup x [A,B] r Ĵ r x r Ĵ r zx, S 2,n sup x [A,B] r Ĵ r zx r E Ĵ r zx and S 3,n sup x [A,B] r E Ĵ r zx r E Ĵ r x, and were to simplify te notations, we do not indicate specifically te dependence of te random variables on ω. We treat tese tree terms separately. For te first term, note tat for all ω Ω and for all x [A, B] we ave by te mean-value teorem r Ĵ r x r Ĵ r zx r Ĵ r+1 ξ x zx 2 K r+2 x zx were ξ lies between x and zx and K r+2 c 1 β, wit c 1 a positive constant independent of ω and n. We conclude tat 2δ S 1,n = 2δ sup r Ĵ r x r Ĵ r zx c 1 B A 2 2δ β n 1 q. x [A,B] For andling te second term in 5.6, note tat we ave, for all ω Ω, sup r Ĵ r zx r E Ĵ r zx sup r Ĵ r z r E Ĵ r z. x [A,B] z E n Hence for all ɛ > 0 and l 1, we can write l n=1 P 2δ S 2,n > ɛ l n=1 P 2δ sup r Ĵ r z r E Ĵ r z > ɛ z E n l n=1 z E n [ ] P 2δ r Ĵ r z r E Ĵ r z > ɛ. Now by Cebycev s Inequality and Lemma 5.6, we ave tat, for any z IR, and n large enoug say n M P 2δ r Ĵ r z r E Ĵ r z > ɛ E[r Ĵ r z r E Ĵ r z] 2p c ɛ 2p / 4δp 2 n p p 2βp 4δp, were c 2 is independent of n. Taking te limit as l, we deduce tat n=1 P 2δ S 2,n > ɛ M 1 + c 2 n=m n 1+q p 4δp p 2βp + c 2 by Condition A5. For te tird term in 5.6, we ave n=m n p 4δp p 2βp <, r E Ĵ r zx r E Ĵ r x I + II, 18

19 wit I = and II = K r+1 u[r X zx u r X x u] du B A n 1 q L K r+1 u du K r+1 u[di [τ,+ [ zx u di [τ,+ [ x u] du B A n 1 q d K r+1 1, were we used Lipscitz continuity of r X and te bound zx x B A n 1 q. Finally we obtain 2δ S 3,n = 2δ sup r E Ĵ r zx r E Ĵ r x c 3 n 1 q 1 2δ, x [A,B] wit c 3 a positive constant independent of n and of ω. Let ɛ be any positive real number. Since we ave sown tat, for all ω Ω, 2δ S 1,n + 2δ S 3,n c 1 B A 2 2δ β n 1 q + c 3 n 1 q 1 2δ, wic, under Condition A5, tends to zero as n, we ave, for n large enoug say n M, 2δ S 1,n + 2δ S 3,n ɛ/2. Tus we can write P 2δ S 1,n + 2δ S 2,n + 2δ S 3,n > ɛ M 1 + P 2δ S 2,n > ɛ/2 <. n=1 n=m Lemma 5.8. Suppose tat τ = τ +O 1+η a.s., wit η > 0. Assume Conditions A2, A5, C 3 1, C4 2, C3 5, C4 5, C3 6, C3 7, and suppose tat K3 < and sup x IR\{τ} f Y x <. Ten, if n as n, we ave for any ξ between τ and τ. Proof. We ave 2 Ĵ ξ a.s dk 0, Ĵ ξ dk 0 2 Ĵ ξ 2 E Ĵ τ + 2 E Ĵ τ dk 0 T 1,n + T 2,n + T 3,n, wit T 1,n = sup x [A,B] 2 Ĵ x 2 E Ĵ x, T 2,n = sup x [τ τ,τ τ] 2 E Ĵ x 2 E Ĵ τ, and T 3,n = E 2 Ĵ τ dk 0, and were, by Lemmas 5.7 and 5.3, T 1,n a.s. 0 and T 3,n 0. Now, by Lemma 5.3 and a Taylor expansion of K around 0, we ave, for all x [A, B], 2 E Ĵ x 2 E Ĵ τ = d x τ K3 θ + O, wit θ between 0 and x τ/ and were te remainder term O is uniform in x. In particular, since [τ τ, τ τ] [A, B] we can write T 2,n 1 d K 3 τ τ + O = O η + O, were te last equality olds almost surely. Tis proves tat T 2,n a.s 0. 19

20 Proposition 5.2 below sows tat under te conditions of te teorem, te condition τ τ = O 1+η a.s. of Lemma 5.8 is satisfied, wit η corresponding to δ appearing in Condition A5. Its proof requires Proposition 5.1 below. Te proof of Proposition 5.1 follows from standard arguments and is ommitted ere. See Delaigle and Gijbels 2003 for more details. See also Couallier Let I n = {x [A, B] : x τ > 1+δ }, wit δ > 0. Proposition 5.1. Under Conditions A1 to A5, we ave n=1 Proposition 5.2. Under Conditions A1 to A5, we ave P sup Ĵx Ĵτ <. 5.8 x I n τ τ = O 1+δ a.s., 5.9 wit δ > 0 as in Condition A5. Proof. By definition of τ, we ave P τ I n P supx In Ĵx Ĵτ, and n=1 by te Borel-Cantelli lemma, if we define A = tat ω A : lim sup n Proposition 5.1. τ n τ 1+δ n P sup Ĵx Ĵτ < = P A = 1, x I n { w : n=1 m=n } { τ m τ / 1+δ m 1}. We ave 1, and tus τ τ = O 1+δ almost surely. We conclude wit 5.2 Proofs of te main results for te ordinary smoot case Proof of Teorem 3.1. By applying a Taylor expansion of Ĵ around τ, we can write 0 = Ĵ τ = Ĵ τ + τ τĵ ξ, were ξ lies between τ and τ. Tus we ave τ τ = Ĵ τ/ĵ ξ, 5.10 were by Lemma 5.8, Ĵ ξ is almost surely different from zero as n, since K 0 < 0. Under te conditions of te teorem, we ave τ τ = O 1+δ a.s. see Proposition 5.2. Hence te conditions of Lemma 5.8 are all satisfied. 20

21 Below, we searc for te asymptotic law of Ĵ τ/ĵ ξ and deduce from tere te asymptotic law of τ τ. From Lemma 5.5, for r = 2, we know tat Ĵ τ E[Ĵ τ] VarĴ τ L N0; 1 were, by Lemma 5.4, Var [ Ĵ τ ] = Bτ R K n +On 1 2β, wit B τ = f Y τ + +f Y τ, and 2 were RK 2β see Condition A4. From Lemma 5.2, we know tat E[Ĵ τ] = k 2+1 D τ + O k 2+2. Hence we ave Ĵ τ k2+1 D τ L B τ R a n + b n N0; 1, K n Bτ n RK Bτ n RK were a n = 1 and b +On 1 2β n = RK 2β, and we deduce tat O k 2 +2 Bτ n RK +On 1 2β 0 n, since Ĵ τ k2+1 D τ B τ R K n and te conclusion follows from 5.10 and Lemma 5.8. L N0; 1 Proof of Corollary 3.1. From Teorem 3.1, an approximate expression for te expectation [ ] τ τ AE may be derived from AE k 2 +2 D τ = o /n. Using Condition A4, we obtain AE [ τ ] = τ k 2 +2 D τ dk 0 variance AVar is given by R K n [ AVar τ τ R k2+2 D τ K dk 0 dk 0 R K + on 1/2 1 2β/2. An approximate expression for te R K ] = B τ 1 + o1, d 2 {K 0} 2 and tus AVar[ τ] = RK Bτ nd 2 {K 0} o1. Te conclusion follows immediately. 5.3 Auxilary results for te supersmoot case Te following sequence of lemmas lead to te proof of Teorem 3.2 and Corollary 3.2. Again, K is a kt order symmetric kernel wit k 2. Te proof of Lemma 5.12 is straigtforward, ence it is ommitted. Te following conditions will be useful. 21

22 Condition D: D m 1 K m u du = O β 3 exp β /γ, wit β 3 a real constant; D m 2 K m = O β 0 exp β /γ ; D m 3 RKm = O 2β 0 exp2 β /γ. Te next lemma generalizes a result of Fan 1991a to te case were te density f X is not continuous. Lemma 5.9. Let r 0. Suppose tat = d 2/γ 1/β ln n 1/β wit d > 0. Assume Conditions A2, C r+1 1, C r+1 2, C r+1 7, D r 1, Dr 2. If expression 3.4 is satisfied and sup x IR\{τ} f Y x <, ten statement 5.2 in Lemma 5.5 olds. Proof. We need to verify te Lyapounov condition 5.3 for Y n,i = 1 K r τ Yi, i = 1,..., n. Let β 4 = minβ 0, β 3. We ave E Y n,1 sup x IR\{τ} f Y x K r u du = O β 4 exp β /γ and 1+η E Y n,1 2+η O 2+ηβ 4 K r 1+η exp2 + η β /γ. We conclude tat sup x IR\{τ} f Y x K r u du = 1+η E Y n,1 EY n,1 2+η =O β42+η exp2 + η β /γ For te denominator, note tat, from expression 3.4, we ave, as in te proof of Lemma 5.4 using Lemma 5.3 { } 2fY { } 2 VarY n,1 = 1 K r u τ u du dk r O c 5 f Y τ c 6 exp 2 γ 4βb { } 2 β γ dk r O β c 5 2 f Y τ c 6 exp 2 γ β 4βb γ β, for n large enoug, were b = β/2r+10 and c 5 is a positive constant independent of n and c 6 is a constant independent of n. We ave c lim n n β42+η exp2 + η β /γ η/2 [ c 6+1 exp 2 4βb ] 1+η E Y n,1 EY n,1 2+η 2+η/2 β 42+η exp2 + η β /γ γ β γ { β n η/2 β 5 exp 22 + ηβb β /γ }, =c lim n 1+η E Y n,1 EY n,1 2+η β 42+η exp2 + η β /γ were c = [ ] 2 2+η/2 c 5 f Y τ and β5 = 2 + η β 4 2+η c 2 6 η 1. Under te assumptions of te lemma, we ave exp 22 + ηβb β /γ = n 2+ηβd βb, wit b = β/2r+10 tending to zero 22

23 as n, and tus exp 22 + ηβb β /γ n η/2 β 5 n η/2+2+ηβd β b ln n β5/β, wic tends to 0 as n even if β 5 < 0, since b 0 as n and η > 0. Hence, by 5.11, te Lyapounov condition 5.3 is satisfied. Lemma Let r 0. Assume Conditions A2, C r+1 1, C r+1 2, C r+1 7, D r+1 2 and D r+1 3. Suppose tat sup x IR\{τ} f Y x < and t 2r+2 2β 0 ϕ K t 2 dt <. Ten, if n as n, we ave, for all p IN 0 and for n large enoug, { E[ r Ĵ r x 2 E Ĵ r x] 2p 2n p p+2β0p exp2p β /γ c β0 K were c β0 K = t 2r+2 2β 0 ϕ K t 2 dt. sup x IR\{τ} f Y x 8 πd 2 0,0 Proof. Similar to te proof of Lemma 5.6. In tis case, owever, for all j 2, we find E [ K r+1 ] j = O jβ0+1 expj β /γ and ten get E[ r Ĵ r x r E Ĵ r x] 2p = x Yi O n p p+2β 0p exp2p β /γ, since n as n. From te above calculations, we also ave } p, E[ r Ĵ r x r E Ĵ r x] 2p 1 { p 2 sup f n p 2p Y x RK r+1 } 1 + o1 x IR\{τ} { n p p+2β0p exp2p β /γ c β0 K sup f Y x x IR\{τ} 8 πd 2 0,0 } p 1 + o1, were details for obtaining te last inequality can be found in Delaigle and Gijbels Lemma Let r 0 and = d 2/γ 1/β ln n 1/β wit d > 1. Assume Conditions A2, C r+1 1, C r+2 2, C r+1 7, D r+1 2, D r+2 2 and D r+1 3. Suppose tat K r+1 <, sup x IR\{τ} f Y x < and t 2r+2 2β 0 ϕ K t 2 dt <. Ten statement 5.5 in Lemma 5.7 olds for any δ > 0. Proof. Similarly as in te proof of Lemma 5.7, we need to bound te sum in 5.6. For te first term, we ave, for all ω Ω sup x [A,B] r Ĵ r x r Ĵ r zx 2 K r+2 x zx cn 1 q β0 2 exp β /γ, 23

24 wit c > 0 a constant independent of ω and n, and tus 2δ S 1,n c 1 n d β /2 1 q ln n 2+2δ β 0 β, wit c 1 > 0 a constant independent of ω and n. For te second term in 5.6, by Cebycev s Inequality and Lemma 5.10, we ave tat, for any z IR, and n large enoug say n M P 2δ r Ĵ r z r E Ĵ r z > ɛ c 2 ln n p+4δp 2β 0 p β n pd β p, were c 2 is independent of ω and n. We deduce tat P 2δ S 2,n > ɛ <M + c 2 ln n p+4δp 2β 0 p β n pd β p+1+q + c 2 ln n p+4δp 2β 0 p β n pd β p n=1 <, n=m as soon as pd β p+1+q < 1, wic is equivalent to requiring tat p > 2+q 1 d β, since d > 1. For te tird term in 5.6, we get 2δ S 3,n c 3 n 1 q 1 2δ, wit c 3 a positive constant. Since c 1 n d β/2 1 q ln n 2+2δ β 0 β te proof of Lemma 5.7. n=m + c 3 n 1 q 1 2δ tends to zero as n, we conclude as in Lemma Suppose tat τ = τ + O 1+η a.s., wit η > 0, and = d 2/γ 1/β ln n 1/β wit d > 1. Assume Conditions A2, C 3 1, C4 2, C3 7, D3 2, D4 2, D3 3. Suppose tat K 3 <, sup x IR\{τ} f Y x < and t 6 2β 0 ϕ K t 2 dt <. Ten statement 5.7 in Lemma 5.8 olds. Te proofs of Propositions 5.4 and 5.3 are similar to te ordinary smoot error case. Proposition 5.3. Suppose tat = d 2/γ 1/β ln n 1/β wit d > 1, and 0 < δ < 1/2. Ten under Conditions B1 to B4, statement 5.8 of Proposition 5.1 olds. Proposition 5.4. Suppose tat = d 2/γ 1/β ln n 1/β wit d > 1, and 0 < δ < 1/2. Ten under Conditions B1 to B4, statement 5.9 of Proposition 5.2 olds. 5.4 Proofs of te main results for te supersmoot case Proof of Teorem 3.2. As in te proof of Teorem 3.1, we ave τ τ = Ĵ τ/ĵ ξ, were ξ lies between τ and τ. From Lemma 5.9 for r = 2 and Lemma 5.12, Ĵ τ E[Ĵ τ] L 1 2Ĵ N 0; ξ VarĴ d τ 2 {K 0} 2 24

25 wic implies τ τ E[Ĵ τ] VarĴ τ dk 0 VarĴ τ L N 0; 1, 5.12 d 2 {K 0} 2 were E[Ĵ τ] = k2+1 D τ + O k2+2. We may furter develop te variance term if we note tat, as in te proof of Lemma 5.4, we ave VarĴ τ = Bτ R K n + On 1 RK. Proof of Corollary 3.2. From 5.12, an approximate expression for te variance AVar is given by AVar [ τ ] = 2 VarĴ τ. An approximate expression for te expectation AE d 2 {K 0} 2 may be derived as follows [ τ τ AE Hence, we obtain AE[ τ] = τ k 2 +2 D τ dk 0 squared error may be written as AMSE[ τ] = E[Ĵ τ] ] = o VarĴ dk 0 τ. + o VarĴ τ. Finally an approximate mean 2k 2+4 Dτ 2 d 2 {K 0} + 2 VarĴ τ. 2 d 2 {K 0} 2 If we furter assume tat u K u 2 du = ORK, we ave VarĴ τ = Bτ n R K + On 1 RK and te conclusion follows. References Barry, J. and Diggle, P Coosing te smooting parameter in a Fourier approac to nonparametric deconvolution of a density function. Journal of Nonparametric Statistics, 4, Carroll, R.J. and Hall, P Optimal rates of convergence for deconvolving a density. Journal of te American Statistical Association, 83, Cu, C.K. and Ceng, P.E Estimation of jumps points and jump values of a density function. Statistica Sinica, 6, Couallier, V Estimation non paramétrique d une discontinuité dans une densité. Comptes Rendus de l Académie des Sciences, I, t.329,

26 Couallier, V Inférence statistique pour des estimateurs de discontinuités dans un cadre non paramétrique. PD. tesis, Laboratoire de Statistique et Probabilités, Université Paul Sabatier, Toulouse III, France. Delaigle, A Kernel estimation in deconvolution problems. PD dissertation. Institut de Statistique, Université catolique de Louvain, Belgium. Delaigle, A. and Gijbels, I Estimation of integrated squared density derivatives from a contaminated sample. Journal of te Royal Statistical Society, Series B, 64, Delaigle, A. and Gijbels, I Boundary estimation and estimation of discontinuity points in deconvolution problems, Institut de Statistique, Université catolique de Louvain, Discussion Paper # ttp:// Delaigle, A. and Gijbels, I. 2004a. Bootstrap bandwidt selection in kernel density estimation from a contaminated sample. Te Annals of te Institute of Statistical Matematics, 56, Delaigle, A. and Gijbels, I. 2004b. Practical bandwidt selection in deconvolution kernel density estimation. Computational Statistics and Data Analysis, 45, Delaigle, A. and Gijbels, I. 2004c. Practical implementation of deconvolution kernel estimation of boundaries and discontinuity points, Institut de Statistique, Université catolique de Louvain, Discussion Paper # Fan, J. 1991a. Asymptotic normality for deconvolution kernel density estimators. Sankya A, 53, Fan, J. 1991b. Global beaviour of deconvolution kernel estimates. Statistica Sinica, 1, Fan, J. 1991c. On te optimal rates of convergence for nonparametric deconvolution problems. Te Annals of Statistics, 19, Gijbels, I. and Goderniaux, A.-C Bandwidt selection for cange point estimation in nonparametric regression. Tecnometrics, 46, Gijbels, I., Hall, P. and Kneip, A On te estimation of jump points in smoot curves. Annals of te Institute of Statistical Matematics, 51,

27 Goderniaux, A-C Automatic detection of cange-points in nonparametric regression. PD. tesis, Institut de Statistique, Université catolique de Louvain, Belgium. Hall, P. and Simar, L Estimating a cangepoint, boundary or frontier in te presence of observation error. Journal of te American Statistical Association, 97, Kneip, A. and Simar, L A general framework for frontier estimation wit panel data. Journal of Productivity Analysis, 7, Li, T. and Vuong, Q Nonparametric estimation of te measurement error model using multiple indicators. Journal of Multivariate Analysis, 65, Müller, H-G Cange-points in noparametric regression analysis. Te Annals of Statistics, 20, Neumann, M.H. 1997a. On te effect of estimating te error density in nonparametric deconvolution. Journal of Nonparametric Statistics, 7, Neumann, M.H. 1997b. Optimal cange-point estimation in inverse problems. Scandinavian Journal of Statistics, 24, Serfling, R.J Approximation Teorems of Matematical Statistics. Wiley, New York. Stefanski, L. and Carroll, R.J Deconvoluting kernel density estimators. Statistics, 2, Wu, J.S. and Cu, C.K Kernel-type estimators of jump points and jump values of a regression function. Te Annals of Statistics, 21, ang, S. and Karunamuni, R Boundary bias correction for nonparametric deconvolution. Annals of te Institute of Statistical Matematics, 52,