ISSN Asymptotic Confidence Bands for Density and Regression Functions in the Gaussian Case

Transcription

1 Journal Afrika Statistika Journal Afrika Statistika Vol 5, N,, page ISSN Asymptotic Confidence Bs for Density egression Functions in the Gaussian Case Nahima Nemouchi Zaher Mohdeb Department of Mathematics, Mentouri University, Constantine, Algeria eceived 7 July ; Accepted November Copyright c, Journal Afrika Statistika All rights reserved Abstract In this paper, we obtain asymptotic confidence bs for both the density regression functions in the framework of nonparametric estimation Beforeh, the asymptotic behaviors in probability of the kernel estimator of the density the Nadaraya-Watson estimator of the regression function are described while local global optimal smoothing parameters are investigated A simulation study is conducted, showing the good performance of the confidence bs obtained for small sample ésumé Dans cet article, nous obtenons les bes de confiance asymptotiques pour la densité et la fonction de régression dans le cadre de l estimation non paramétrique L étude a porté également sur le comportement asymptotique en probabilité de l estimateur à noyau de la densité et de l estimateur de Nadaraya-Watson de la fonction de régression, ainsi que le paramètre de lissage optimal, local et global Une étude par simulation est menée pour montrer la bonne performance des bes de confiance obtenues pour une petite taille d échantillon ey words: Confidence bounds; Density estimation; ernel estimation; Nonparametric estimation; egression estimation AMS Mathematics Subject Classification : 6F35, 6G, 6F5 Introduction Let X, Y, X, Y,, be independent identically distributed rom replicates of the rom vector X, Y Let us assume that the marginal distributions of X Y are normal with unknown means µ X, µ Y stard deviations >, σ Y > respectively, with an unknown linear correlation coefficient ρ In the sequel, the following notations assumptions will be adopted I = a, b J = a, b denote two fixed intervals of such that < a < a < b < b < denotes the Lebesgue measure of I is a real kernel weight function satisfying the folllowing conditions: For every x, x = x; tdt = ; 3 t = t tdt < ; 4 = tdt < ; 5 u = for u / α, α, where α, For every u, set log θ, u = logθ u{ tdt}, where A B = maxa, B θ > is a specified constant Nahima Nemouchi: nemouchi nahima@yahoofr Zaher Mohdeb: zmohdeb@gmailcom

2 N Nemouchi Z Mohdeb, Journal Afrika Statistika, Vol 5, N,, page Asymptotic Confidence Bs for Density egression Functions in the Gaussian Case 8 The regression function the conditional variance of Y given X = x are defined for x J, by rx = EY X = x = y f X,Y x, y dx = µ Y + ϱ x µ X f X x σ Y v x = V ary X = x = y EY X = x f X,Y x, y dy = ρ σy, f X x where f X,Y x, y is the joint density of X, Y f X x is the marginal density of the rom variable X Let n be a sequence of real numbers such that as n Following osenblatt 5, arzen 3, Nadaraya Watson, the kernel estimators of the functions f X x, rx v x are defined by respectively vnx; = r n x; = Introduce now the following centering factors f X,n x; = n Êr n x; = which are needed hereafter in our study We have x Xi, x X Y i i if x Xi n Y i if x Xi, Y i r n x; x Xi if x X i Y i Y j if n n j= Ef X,n x; = E X x E Y x Xi = x Xi, x Xi =, x X X x if E, x X EY if E =, E X x h { } lim Ef X,n x; f X x = According to Deheuvels Mason 3, the conditions n are necessary sufficient to ensure, that for each x I, we have Êr n x; Er n x; = O, n From the seminal works by osenblatt 5, arzen 3, Nadaraya Watson, nonparametric function estimation has been widely investigated Theoretical aspects properties have been described numerical implementations have been performed For an overview on the question, we refer to rakasa ao 4, Devroye Györfi 5, W Jones, Bosq Lecoutre, Silverman 8, Nadaraya, Scott 7 the references therein

3 N Nemouchi Z Mohdeb, Journal Afrika Statistika, Vol 5, N,, page Asymptotic Confidence Bs for Density egression Functions in the Gaussian Case 8 The behavior of nonparametric estimators depends strongly upon the smoothing parameter Several procedures have been proposed describing how to choose this parameter We quote the cross-validation the plug-in methods related essentially to the mean square error criterion Asymptotic properties of estimators need also conditions upon the smoothing parameter The minimal conditions to set are n It is well known that these conditions are necessary sufficient for the pointwise weak consistency of the kernel estimator of f X the Nadaraya-Watson estimator of the regression function r Moreover, these conditions allow to state a rate of uniform convergence of Ê r n x; towards rx, ie we have {Ê } r n x; rx, Deheuvels Mason 3 obtained the rate of uniform convergence in probability for the kernel density estimator Their result is stated in the following terms If n, as n, log n then nh n f X,n x; E f X,n x; sup as n, log f X x where I is a compact interval in Deheuvels Mason 3 have also obtained the optimal rate of convergence in probability for the estimate r n x; of the regression function r These results allow them to construct asymptotic confidence bs for the functions f X r Consider now the problem related to the optimal choices x of the smoothing parameter with respect to the mean square error the integrated mean square error criteria respectively Our aim is to construct the estimates ĥ n x ĥn of the optimal parameter for these criteria in terms of the empirical estimators µ X, µ Y,, σ Y ρ, such that ĥ n x ĥn x Confidence bounds for f X x rx in terms of f X,n x; ĥnx r n x; ĥnx respectively are then deduced The interest for our particular case is that we use the bwidth explicitly in these asymptotic confidence bs The remainder of the paper is organized as follows In section, we present the results A Monte Carlo study is conducted in Section 3 Section 4 is devoted to the proofs esults Mean square error of f X,n x; In this section, we give the optimal choices of the smoothing parameter in the expression of the kernel estimators of the functions f X x rx with respect to the mean square error the integrated mean square error Having replaced the unknown parameters µ X by their empirical estimators µ X respectively, we also give the asymptotic behaviors of the estimates of the functions f X r According to W Jones, the mean square error of f X,n x; is given by Ef X,n x; f X x = Varf X,n x; + Ef X,n x; f X x This quantity is minimized with respect to, by H n, x = n /5 fx x f /5 X x t = n /5 π exp x µx t x µx /5

4 N Nemouchi Z Mohdeb, Journal Afrika Statistika, Vol 5, N,, page Asymptotic Confidence Bs for Density egression Functions in the Gaussian Case 8 As µ X are unknown parameters, we replace them by their empirical estimators µ X = X i n respectively Therefore, we obtain as an estimator of H n, x Set now, an estimator of Θ x σ X = n Ĥ n, x = n /5 Θ x = Θ n, x = X i µ X, π exp x µ X t x µ X σx π3 / exp 3 x µ X t x µx Following Deheuvels Mason 3, one can see that π3 / exp 3 x µ X t x µ X /5 /5 /5 3 ĥ n, x = Ĥn,x/,, x = H n, x/ Θ n, x satisfy the following statements B For any ε >, B For any ε >, Θ For any ε >, inf,x εn /5 inf sup sup ĥ n, x Integrated mean square error of f X,n x; ĥ n, x sup,x ĥ n, x sup, x + εn /5, ε, as n Θ n,x Θ x > ε, as n W Jones stated that the integrated mean square error relative to f X,n x; is given by Ef X,n x; f X x dx = n t + f Xxt dx + oh n The minimum of this quantity, with respect to, is, = n /5 t f X x dx /5 8 π = n /5 /5 3 t

5 N Nemouchi Z Mohdeb, Journal Afrika Statistika, Vol 5, N,, page Asymptotic Confidence Bs for Density egression Functions in the Gaussian Case 83 As the parameter is unknown, we replace it by the empirical estimator, to obtain Set an estimator of the following quantity Θ n, x = Θ x = 8 π ĥ n, = n /5 /5 3t 4 { exp x µx } /, 5 π σx { exp x µx } / π σx As in Deheuvels Mason 3, one can see that ĥn, satisfies the statement, εn /5 < ĥn, <, + εn /5, as n, ε > Moreover, we have sup Θ n,x Θ x > ε, as n, ε > 3 Mean square error of r n x; In a similar way, according to W Jones, the value of,3 that minimizes the mean square error of r n x; given by Er n x; rx = Varr n x; + Er n x; rx is H n,3 x = n /5 vx f X x r x + r x f X x f X x t /5 π ρ σ = n /5 X σ exp x µ X /5 X 4t ρ x As, µ X ρ are unknown parameters, we replace them by their empirical estimators, ie, µ X Therefore, H n,3 x is estimated by ρ = n Σn X i µ X Y i µ Y σ Y π ρ σ Ĥ n,3 x = n /5 X σ exp x µ X /5 X 4 t ρ x 6 Set now as the estimator of the quantity Θ n,3 x = ρ ρx exp x µ X 4π σ 3/ X σ5 Y t /5, 7 Θ 3 x = Similarly, as above, one may show that ρ ρx exp x µ X 4π σ 3/ X σ5 Y t /5,3 x = H n,3 x/, ĥ n,3 x = Ĥn,3x/ Θ n,3 x satisfy the statements B, B, Θ The asymptotic behaviors of the estimates of the functions f X r are described in the following theorems

6 N Nemouchi Z Mohdeb, Journal Afrika Statistika, Vol 5, N,, page Asymptotic Confidence Bs for Density egression Functions in the Gaussian Case 84 Theorem Let Ĥn,x, Θ n, x be the estimates given in the statements 3 respectively Then, we have log θ, n 4/5 n /5 / sup ±Θ n, x f X,n x, Ĥn,x f X x, emark Note from Theorem that, for any < ε <, as n, we have for any x I, lim f X x f X,n x, Ĥn,x + ε n, x, f X,n x, Ĥn,x + + ε n, x = where lim f X x f X,n x, Ĥn,x ε n, x, f X,n x, Ĥn,x + ε n, x =, n, x = log θ, Θ n, x n /5 n 4/5 / Therefore, the interval f X,n x, Ĥn,x n, x, f X,n x, Ĥn,x + n, x sts as an asymptotic confidence domain for f X x Theorem Let ĥn, Θ n, x be the estimates given in the statements 4 5 respectively Then, we have log θ, nĥn, n /5 / sup ±Θ n, x f X,n x, ĥn, f X x, Theorem 3 Let Ĥn,3x, Θ n,3 x be the estimates given in the statements 6 7 respectively Then, we have log θ, n 4/5 n /5 / sup ±Θ n,3 x r n x, Ĥn,3x rx emark Note from Theorem 3 that, for any < ε <, we have for any x I, lim rx r n x; Ĥn,3x + ε n, x, r n x; Ĥn,3x + + ε n, x = where lim rx r n x; Ĥn,3x ε n, x, r n x; Ĥn,3x + ε n, x =, n,3 x = log θ, Θ n,3 x n /5 n 4/5 / Therefore, we obtain for any x I, the following asymptotic confidence bounds for the regression function r n x; Ĥn,3x n, x, r n x; Ĥn,3x + n, x

7 N Nemouchi Z Mohdeb, Journal Afrika Statistika, Vol 5, N,, page Asymptotic Confidence Bs for Density egression Functions in the Gaussian Case 85 3 Monte Carlo Study In order to investigate the small sample properties of the confidence bs for the density f X x regression function rx, some simulation studies are conducted We conduct a Monte Carlo study for sample sizes n =, n = 5, n = n = 8 We consider the model Y = X + Z, with X Z normally distributed N, N, respectively We choose = n /5, x = /,/ Figure Figure show the estimators of the density function f X the regression function respectively, with the confidence bs Examination of Figure Figure reveal that, when the sample size n increases, the confidence bs become smaller with the density, the regression their estimators are always in bounds It appears that the confidence bs have a good performance even for small sample size n= n=5 4 n= 4 n= n= Estimator of f Lower bound n=8 Upper bound f n= Estimator of r Lower bound n=8 4 Upper bound r Figure Confidence bs of the density function Figure Confidence bs of the regression function 4 roofs of results 4 roof of Theorem Taking = n /5, it follows that the conditions B, B Θ are satisfied, Theorem of Deheuvels Mason implies

8 N Nemouchi Z Mohdeb, Journal Afrika Statistika, Vol 5, N,, page Asymptotic Confidence Bs for Density egression Functions in the Gaussian Case 86 n log θ, { sup as n, where / sup ±Θ n, x Ef X,n x, ĥn,x f X,n x, ĥn,x Θ / xf X x }, C x C x = { π exp x µ X t x µ X } /5 8 Taking into account the expressions Θ x C x given by 8 respectively, we obtain n log θ, Now, with this choice of the parameter, we have see Nadaraya 989 Let n =, we have / n log θ, sup ±Θ n, x Ef X,n x, ĥn,x f X,n x, ĥn,x, 9 / sup ±Θ n, x Ef X,n x, f X x, n σx According to Section of Deheuvels Mason 3 works of Einmahl Mason 8, the locally adapted bwith defined by fulfills, via 9 log θ, n 4/5 n /5 Ĥ n, x = n C x = ĥ n, x / sup ±Θ n, x f X,n x, Ĥn,x f X x 4 roof of Theorem This proof is based on Corollary of Deheuvels Mason 3, then / nĥn, sup ±Θ log θ, n, x f X,n x, ĥn, f X x ĥ n, ie nĥn, log θ, ĥ n, / { } sup ±Θ xf X x tdt sup Θ n, x f X,n x, ĥn, f X x,

9 N Nemouchi Z Mohdeb, Journal Afrika Statistika, Vol 5, N,, page Asymptotic Confidence Bs for Density egression Functions in the Gaussian Case roof of Theorem 3 According to the Theorem to Section of Deheuvels Mason 3 using the works of Einmahl Mason 8, the proof of this theorem is the same as the proof of Theorem Acknowledgment The authors would like to thank the Editors the anonymous reviewers for their comments suggestions that improved the quality of the paper eferences Akaike, H, 954 An approximation of the density function Ann Inst Statist Math, 6, 7-3 Bosq, D Lecoutre, J, 987 Théorie de l Estimation Fonctionnelle Economica, aris 3 Deheuvels, Mason, D M 4 General asymptotic confidence bs based on kernel-type function estimators Stat Inference Stoch rocess, 73, Devroye, L 978 The uniform convergence of the Nadaraya-Watson regression function Estimate Can J Statist, 6, Devroye, L Györfi, L, 985 Nonparametric Density Estimation: The view Wiley, New York 6 Devroye, L Lugosi, G, Combinatorial methods in density estimation Springer Series in Statistics Springer- Verlag, New York, pp xii+8 ISBN: Einmahl, U Mason, DM, An empirical process approach to the uniform consistency of kernel-type function estimators J Theoret robab, 3, Einmahl, U Mason, DM, 5 Uniform in bwidth consistency of kernel-type function estimators Ann Statist, 333, Härdle, W, Jansen, Serfling,, 988 Strong uniform consistency rates for estimators of conditional functionals Ann Statist, 64, Izenman, AJ, 99 ecents developments in nonparametric density estimation J Amer Statist Assoc, 8643, 5-4 Nadaraya, EA, 976 On the nonparametric estimator of Bayesian risk in the classification problem roc AN Georg SS, 8, 77-8 in ussian Nadaraya, ÈA, 989 Nonparametric estimation of probability densities regression curves Translated from the ussian by Samuel otz Mathematics its Applications Soviet Series, luwer Academic ublishers Group, Dordrecht pp x+3 ISBN: arzen, E, 96 On estimation of a probability density function mode Ann Math Statist, 33, rakasa, ao, BLS, 983 Nonparametric Functional Estimation Academic ress, New York 5 osenblatt, M, 956 emarks on some nonparametric estimates of a density function Ann Math Statist, 7, oussas, G, 99 Nonparametric Functional Estimation elated Topics NATO ASI series 355 luwer, Dordrecht 7 Scott, DW, 99 Multivariate Density Estimation-Theory, ractice Visualization Wiley, New York ISBN: Silverman, BW, 986 Density Estimation for Statistics Data Analysis Chapman & Hall, London, pp x+75 ISBN: Stone, CJ, 984 An asymptotically optimal window selection rule for kernel estimates Ann Statist, Volume 4, W, M Jones, MC, 995 ernel Smoothing Chapman Hall Chapman & Hall/CC Monographs on Statistics & Applied robability ISBN: 4557 Watson, GS, 964 Smooth egression Analysis Sankhya A, 6, Weissman, I, 978 Estimation of parameters large quantiles based on the k largest observations J Amer Statist Assoc 73364, Woodroofe, M, 97 On choosing a Delta-Sequence Ann Math Statist, 4,