On Asymptotic Normality of the Local Polynomial Regression Estimator with Stochastic Bandwidths 1. Carlos Martins-Filho.

Transcription

1 On Asymptotic Normality of the Local Polynomial Regression Estimator with Stochastic Bandwidths 1 Carlos Martins-Filho Department of Economics IFPRI University of Colorado 2033 K Street NW Boulder, CO , USA & Washington, DC , USA carlos.martins@colorado.edu c.martins-filho@cgiar.org Voice: Voice: and Paulo Saraiva Department of Economics University of Colorado Boulder, CO , USA paulo.saraiva@colorado.edu Voice: February, 2010 Abstract. Nonparametric density and regression estimators commonly depend on a bandwidth. The asymptotic properties of these estimators have been widely studied when bandwidths are nonstochastic. In practice, however, in order to improve finite sample performance of these estimators, bandwidths are selected by data driven methods, such as cross-validation or plug-in procedures. As a result nonparametric estimators are usually constructed using stochastic bandwidths. In this paper we establish the asymptotic equivalence in probability of local polynomial regression estimators under stochastic and nonstochastic bandwidths. Our result extends previous work by Boente and Fraiman 1995 and Ziegler Keywords and Phrases. local polynomial estimation; asymptotic normality; mixing processes; stochastic bandwidth. AMS Subject classification. 62G05, 62G08, 62G20 1 We thank Juan Carlos Escanciano, Yanqin Fan and Jeff Racine for helpful comments. We are particularly grateful to Yanqin Fan for bringing to our attention the work of Ziegler 2004.

2 1 Introduction Currently there exist several papers that establish the asymptotic properties of kernel based nonparametric estimators. For the case of density estimation, Parzen 1962, Robinson 1983 and Bosq 1998 establish the asymptotic normality of Rosenblatt s density estimator under independent and identically distributed IID and stationary strong mixing data generating processes. For the case of regression, Fan 1992, Masry and Fan 1997 and Martins-Filho and Yao 2009 establish asymptotic normality of local polynomial estimators under IID, stationary and nonstationary strong mixing processes. All of these asymptotic approximations are obtained for a sequence of nonstochastic bandwidths 0 < 0 as the sample size n. In practice, to improve estimators finite sample performance, bandwidths are normally selected using data-driven methods see, e.g., Ruppert et. al., 1995 and Xia and Li, As such, bandwidths are in practical use generally stochastic. Therefore, it is desirable to obtain the aforementioned asymptotic results when is data dependent and consequently stochastic. There have been previous efforts in establishing asymptotic properties of nonparametric estimators constructed with stochastic bandwidths. Consider, for example, the local polynomial regression estimator proposed by Fan Dony et. al prove that such estimator, when constructed with a stochastic bandwidth, is uniformly consistent. More precisely, suppose {Y t, X t } n t=1 is a sequence of random vectors in R 2 with regression function mx = EY t X t = x for all t. The local polynomial regression estimator of order p is defined by m LP x; ˆb n0 x; where ˆb n0 x;,..., ˆb np x; = argmin b 0,...,b p n Y t t=1 2 p b j X t x j Xt x K and K : R R is a kernel function. If the sequence {Y t, X t } n t=1 is IID, then it follows from Dony et. al. j= that lim sup n sup [a n,b n] nhn sup x G m LP x; mx log hn log log n = O a.s. 1 where a n, b n is a nonstochastic sequence such that 0 a n < b n 0 as n, G is a compact set in R and log log log n = max{ log, log log n}. If there exists a stochastic bandwidth ĥn such that 1

3 ĥ n 1 = o p 1 and we define a n = r and b n = s with 0 < r < 1 < s. Then it follows that sup m LP x; ĥn mx = o p 1. x G When p = 1 and the sequence {Y t, X t } n t=1 is IID, if ĥn is obtained by a cross validation procedure, Li and Racine 2004 show that nĥn m LP x; ĥn mx ĥ2 n 2 m2 x Kuu 2 du d N 0, σ2 x f X x K 2 udu where X is a random variable that has the same distribution of X t, f X is the density function of X and σ 2 x = V ary t X t = x. Xia and Li 2002 establish that, if ĥn is obtained through cross validation, ĥ n 1 = o p 1 for strong mixing and strictly stationary sequences {Y t, X t } n t=1. When p = 0, the case of a Nadaraya-Watson regression estimator m NW x;, and the sequence {Y t, X t } n t=1 is IID Ziegler 2004 shows that nĥn d m NW x; ĥn E m NW x; ĥn X N 0, σ2 x f X x K 2 udu given that ĥn 1 = o p 1. For the case where {Y t, X t } n t=1 is a strictly stationary strong mixing random process, Boente and Fraiman 1995 show that if ĥn 1 = o p 1, then nĥn m NW x; ĥn E m NW x; ĥn X d N 0, σ2 x f X x K 2 udu where X = X 1,..., X n. In this paper we expand the result of Boente and Fraiman 1995 by obtaining that local polynomial estimators for the regression and derivatives of orders j = 1,..., p constructed with a stochastic bandwidth ĥ n are asymptotically normal. We do this for processes that are strong mixing and stationary. Our proofs build and expand on the those of Boente and Fraiman 1995 and Masry and Fan Preliminary Results and Assumptions Define the vector b n x; h = ˆb n0 x; h,..., ˆb np x; h and the diagonal matrix H n = diag{h j n} p j=0. Given that Masry and Fan 1997 have established the asymptotic normality of n H n b n x; EH n b n x; X, 2

4 it suffices for our purpose to show that nhn H n b n x; EH n b n x; X nĥn Ĥn b n x; ĥn EĤnb n x; ĥn X = o p 1, 1 where ĥn is a bandwidth that satisfies ĥn 1 = o p 1 and Ĥn = diag{ĥj n} p j=0. Lemma 2.1 simplifies condition 1 further. It allows us to use a nonstochastic normalization in order to obtain the asymptotic properties of the local polynomial estimator constructed with stochastic bandwidths. Throughout the paper, for an arbitrary stochastic vector W n, all orders in probability are taken element-wise. Lemma 2.1 Define n h = Hb n x; h EHb n x; h X. Suppose that n n n ĥn = o p 1 and d n n W a suitably defined random variable. nĥn n ĥn = o p 1 provided that ĥn 1 = o p 1. Then it follows that n n Our subsequent results depend on the following assumptions. A1. 1. The process {Y t, X t } n t=1 is strictly stationary. 2. for some δ > 2 and a > 1 2 δ : l a αl 1 2 δ <. l=1 3. σ 2 x V ary t X t = x is a continuous and differentiable function at x. 4. The pth-order derivative of the regressions, m p x exists at x. A2. 1. The bandwidth 0 < 0 and n as n. 2. There exists a stochastic bandwidth ĥn such that ĥn 1 = o p 1 holds. A3. 1. The kernel function K : R R is a bounded density function with support suppk = [ 1, 1]. 2. u 2pδ+2 Ku 0 as u for δ > The first derivative of the kernel function, K 1, exists almost everywhere with K 1 uniformly bounded whenever it exists. A4. The density f X x for X t is differentiable and satisfies a Lipschitz condition of order 1, i.e., f X x f X x C x x, x, x R. A5. 1. The joint density of X t, X t+s, f s u, v, is such that f s u, v C for all s 1 and u, v [x, x + ]. 2. f s u, v f X uf X v C for all s 1. 3

5 A6. EY Y 2 l X 1 = u, X l = v <, l 1 and E Y t δ X t = u <, t, for all u, v [x, x + ] and some δ > 2. A7. There exists a sequence of natural numbers satisfying s n as n such that s n = o n and h αs n = o nn. A8. The conditional distribution of Y given X = u, f Y X=u y is continuous at the point u = x. Let s n,l x; = g n,l x; = g n,lx; = 1 n 1 n 1 n n l Xt x Xt x K, 2 t=1 n l Xt x Xt x K Y t and 3 t=1 n l Xt x Xt x K Y t mx t for l = 1,..., 2p. 4 t=1 Then b n x; = H 1 n Sn 1 x; G n x; where S n x; = {s n,i+j 2 x; } p+1,p+1 i,j=1 and G n x; = {g n,l x; } p l=0. Masry and Fan 1997 show that under assumption A1 through A8 nhn H n b n x; ES n x; 1 G n x; X d N 0, σ 2 x f X x S 1 1 SS, 5 where S = {µ i+j 2 } p+1,p+1 i,j=1, S = {ν i+j 2 } p+1,p+1 i,j=1 with µ l = ψ l Kψdψ and ν l = ψ l K 2 ψdψ. Equation 5 gives us n n d W in Lemma 2.1. In particular, W N0, f 1 X xσ2 xs 1 SS 1. Consequently it suffices to show that nhn n n n ĥn = o p 1. 6 As will be seen in Theorem 3.1, the key to establish 6 resides in obtaining asymptotic uniform stochastic equicontinuity of n n x; τ with respect to τ. To this end we establish the following auxiliary lemmas. Lemma 2.2 Let Z n x; l, τ = d dτ s n,lx; τ, for some τ finite and l = 0,..., 2p. If A1.1, A2.1, A3.1, A3.3 and A4 hold, then sup τ [r,s] Z n x; l, τ = O p 1 where r, s > 0 and r < s. Lemma 2.3 Let B n x; l, τ = n d dτ g n,l x; τ, for l = 0,..., 2p. If A1 through A6 hold, then s r B2 nx, l, τdτ = O p 1 where r, s > 0 and r < s. 4

6 3 Main Results The following theorem and corollary establish nĥn-normality of the local polynomial estimator constructed with stochastic bandwidths. As in Masry and Fan 1997, we are able to obtain asymptotic normality for the regression estimator as well as for the estimators of the regression derivatives. Theorem 3.1 Suppose A1 through A8 hold, then it follows that nĥn n ĥn d N 0, σ2 x f X x S 1 1 SS. With the following corollary we also obtain the asymptotic bias for local polynomial estimators with stochastic bandwidths. Corollary 3.1 Let m j denote the jth-order derivative of m. Suppose A1 through A8 hold, then nĥn Ĥ n b n x; ĥn bx ĥp+1 n m p+1 x + p + 1! ĥp+1 d n o p 1 N 0, σ2 x f X x S 1 1 SS where Ĥn = diag{ĥj n} p j=0 and bx = mx, m 1 x,..., 1 p! mp x. 4 Monte Carlo study In this section we investigate some of the finite sample properties of the local linear regression and derivative estimators constructed with a bandwidth selected by cross validation for data generating processes DGP exhibiting dependence. In our simulations two regression functions are consider, m 1 x = sinx and m 2 x = 3x x with first derivatives given respectively by m 1 1 x = cosx and m1 2 x = 9x We generate {ɛ t } n t=1 by ɛ t = ρɛ t 1 + σu t, where {U t } t 1 is a sequence of IID standard normal random variable and ρ, σ 2 = 0, 0.052, 0.2, and 0.9, This implies that {ɛ t } n t=1 is a dependent sequence with identical normal distribution with mean zero and and variance. For m 1 we draw IID regressors {X t } n t=1 from a uniform distribution that takes value on [0, 2π]. For m 2 we draw IID regressors {X t } n t=1 from a beta distribution with parameters α = 2 and β = 2 given by { x α 1 1 x R β 1 1 if x [0, 1] f X x; α, β = 0 uα 1 1 u β 1 du 0 otherwise. 5

7 The regressands are constructed using Y t = m i X t + ɛ t, where i = 1, 2. Two sample sizes are considered n = 200, 600 and 1, 000 repetitions are performed. We evaluate the regression and regression derivative estimators at x = 0.5π, π, 1.5π and x = 0.25, 0.5, 0.75 for m 1 and m 2 respectively. These estimators are constructed both with a nonstochastic optimal regression bandwidth, h AMISE, and with a cross validated bandwidth, h CV, which is clearly data dependent. The cross validated bandwidth is given by h CV n = argmin t=1 m LP,tX t ; h Y t 2, where m LP,t x; h h is the local linear regression estimator constructed with the exclusion of observation t. The nonstochastic bandwidth is given by h AMISE = 1 λ 1/5, nλ 2 where λ1 = V arɛ t K 2 udu 1f X x 0dx and λ 2 = u 2 Kudu m 2 x 2 f X xdx see, e.g., Ruppert et. al., 1995 and Xia and Li, The results of our simulations are summarized in Tables 1-2 and Figures 1-2. Tables 1 and 2 provide the bias ratio and mean squared error MSE ratio of estimators constructed with h CV and h AMISE for m 1 and m 2 respectively. These ratios are constructed with estimators using h CV in the numerator and h AMISE in the denominator. Figure 1 shows the estimated density of the difference between the estimated regression constructed with h CV and h AMISE, for m 1 π and m 2 0.5, wit = 200 and ρ = 0, 0.9. Similarly, figure 2 shows the estimated density of the difference between the estimated regression first derivative constructed with h CV and h AMISE, for m 1 1 π and m1 2 0, 5, wit = 200 and ρ = 0, 0.9. As expected from the asymptotic results the bias and MSE ratios are in general close to 1, especially for the regression estimator. Ratios that are farther form 1 are more common in the estimation of the regression derivative. This is consistent with the asymptotic results since the rate of convergence of the regression estimator is n, whereas regression first derivative estimators have rate of convergence nh 3 n. Hence, for fixed sample sizes we expect regression estimators to outperform those associated with derivatives. Note that most bias and MSE ratios given in tables 1 and 2 are positive values larger than 1. Since we constructed both bias and MSE ratios with estimators constructed with h CV in the numerator and estimators constructed with h AMISE in the denominator, the results indicate that bias and MSE are larger for estimators constructed with h CV. This too was expected, since h AMISE is the true optimal bandwidth for the regression estimator. Positive bias ratios indicate that the direction of the bias is the same for estimators 6

8 constructed with h AMISE and h CV. We note that in general the estimators for the function m 1 outperformed those for function m 2. We observe that m 2 takes value on [0.75, 1.75] and ɛ t on R. Thus, although the variance of ɛ t was was chosen to be small, 0.052, estimating the bandwidth was made difficult due to the fact that ɛ t had a large impact on Y t in terms of its relative magnitude. The regression function m 1 also took values on a bounded interval, however this interval had a larger range. In fact the standard deviation of h CV for n = 200 and ρ = 0.5 was and for the DGP s associated with m 1 and m 2 respectively. The kernel density estimates shown in figures 1 and 2 were calculated using the Gaussian kernel and bandwidths were selected using the rule-of-thumb procedure of Silverman We observe that the change from IID ρ = 0 to dependent DGP ρ 0 did not yield significantly different results in terms of estimator performance. In fact, our results seem to indicate that for ρ = 0.9 the estimators had slightly better general performance than for the case where ρ = 0. As expected from our asymptotic results, figures 1 and 2 showed that the difference between derivative estimates using h CV and h AMISE were more disperse around zero than those associated with regression estimates, especially for the DGP for m 2. Even though the DGP for m 1 provided better results, the estimators of m 2 and m 1 2, as seen on figures 1 b and 2 b, performed well, in the sense that such estimators produced estimated densities with small dispersion around zero. 5 Final Remarks We have established the asymptotic properties of the local polynomial regression estimator constructed with stochastic bandwidths. Our results validate the use of the normal distribution in the implementation of hypotheses tests and interval estimation when bandwidths are data dependent. Most assumptions that we have imposed, were also explored by Masry and Fan The assumptions we place on ĥn coincides with the properties of the bandwidths proposed by Ruppert et. al and Xia and Li 2002 under IID and strong mixing respectively. 7

9 Appendix 1: Proofs Proof of Lemma 2.1: Since n n n ĥn = o p 1, we have that nhn n nĥn n ĥn = o p 1 + n n ĥn nĥn n ĥn. nhn n d W and n n n n ĥn = o p 1 imply that n ĥn = O p nhn 1/2. Consequently, since nhn n ĥn nĥn n ĥn = 1 ĥ n O p 1 = o p 1 1 ĥn = o p 1. Proof of Lemma 2.2: For any ɛ > 0, we must find M ɛ < such that P sup Z n x; l, τ > M ɛ ɛ. 7 τ [r,s] By Markov s inequality, we have that P sup τ [r,s] Z n x; l, τ > 1 ɛ E sup Z n x; l, τ τ [r,s] ɛ. 8 Thus it suffices to show that E sup τ [r,s] Z n x; l, τ = O1. Let K l x = Kxx l 1 + l + K 1 xx l+1 and write Z n x; l, τ = 1 n n τ 2 t=1 Xt x K l τ. 9 By strict stationarity, we write E sup Z n x; l, τ τ [r,s] 1 r 2 E sup τ [r,s] K Xt x l τ 10 Now, note that X E sup τ [r,s] Kl t x τ = supτ [r,s] τ K l φ f X x + τφ f X x + f X x dφ h 2 nc K l φφ dφ sup τ [r,s] τ 2 + f X x K l φ dφ sup τ [r,s] τ h 2 ns 2 C K l φφ dφ sup τ [r,s] + f X xs K l φ dφ h 2 ns 2 C l Kφ φl+1 + K 1 φ φ l+2 dφ + f X xs l Kφ φl + K 1 φ φ l+1 dφ h 2 ns 2 C + s f X x l Kφ + K1 φ dφ 11 8

10 Hence, E sup Z n x; lτ τ [r,s] 1 r 2 C + f X xc 12 = O1 13 as 0 as n Proof of Lemma 2.3: Using Markov s inequality it suffices to establish that s E Bnx; 2 l, τdτ = r s r E B 2 nx; l, τ dτ = O1. 14 Note that, Bnx; 2 l, τ = 1 n τ 4 n t=1 K 2 l Xt x τ ɛ 2 t + 2 n t=1 i t Xt x Xi x K l ɛ t Kl τ τ ɛ i 15 where ɛ t = Y t mx t. Thus, by the law of iterated expectations and strict stationarity, we obtain E Bnx; 2 l, τ 1 = E K 2 X t x τ 4 l τ σ 2 X t n τ 4 t=2 1 t n E X Kl 1 x X h Kl t x nτ τ ɛ 1 ɛ t 1 E K 2 X t x τ 4 l τ σ 2 X t n τ 4 t=2 1 t n E X Kl 1 x X h Kl t x nτ τ ɛ 1 ɛ t Notice that E 1 τ 3 K 2 l X t x τ σ 2 X t where wx = f X xσ 2 x. Let ξ t = K l = 1 K 2 τ 3 l φσ2 x + τφf X x + τφdφ = { } τ 3 K2 l φ σ 2 xf X x + dwx dx τφ dφ σ 2 xf X xτ 3 K2 l φdφ + τ 2 O = O1, and without loss of generality take s 1. 1 Then, X t x τ Eξ 1 ξ t ɛ 1 ɛ t = E E ɛ 1 ɛ t X 1, X t ξ 1 ξ t E sup X1,X t [x s,x+s] E ɛ 1 ɛ t X 1, X t ξ 1 ξ t E sup X1,X t [x s,x+s] E Y 1 + B Y t + B X 1, X t ξ 1 ξ t { } E sup X1,X t [x s,x+s] E Y1 + B 2 X 1, X t E Yt + B 2 1 X 1, X 2 t ξ 1 ξ t CE ξ 1 ξ t = C u x v x Kj τh Kj n τh ft n u, vdudv C u x v x Kj τh Kj n τ dudv h 2 nτ 2 2 C Kl φ dφ 1 Let s = max{1, s} and note that this proof follows with s = s. 18 9

11 where B = sup X [x shn,x+s] mx. Let {d n } n 1 be a sequence of positive integers, such that d n as n. Then we can write n E ξ 1 ξ t ɛ 1 ɛ t = t=2 d n+1 t=2 E ξ 1 ξ t ɛ 1 ɛ t + n t=d n+2 E ξ 1 ξ t ɛ 1 ɛ t 19 and note that dn+1 t=2 E ξ 1 ξ t ɛ 1 ɛ t d n+1 t=2 τ 2 h 2 nc Kl φ dφ = d n h 2 nτ 2 C. Then using the fact that Eξ t ɛ t = 0 and Davydov s Inequality we obtain, 2 20 E ξ 1 ξ t+1 ɛ 1 ɛ t+1 8[αt] 1 2/δ E ξ 1 ɛ 1 δ 2/δ. 21 Note also that which leads to, E ξ 1 ɛ 1 δ = E Y 1 mx 1 K X 1 x δ l τ E sup X1 [x s,x+s] E{ Y 1 + B δ X 1 } K l δ CE Kl X 1 x τ = C u x δ Kl h fx nτ udu = Cτ Kl v δ f X v + τ xdv Cτ n t=d n+2 Eξ 1ɛ 1 ξ t+1 ɛ t+1 n t=d n+2 8αt1 2/δ E ξ 1 ɛ 1 δ 2/δ h 2/δ n h 2/δ n = Ch 2/δ n τ 2/δ C n τ 2/δ C n t=d n+2 d 1+2/δ n d 1+2/δ n = Ch 2/δ n = Cτ 2/δ o X 1 x δ τ t=d n+2 αt1 2/δ t a αt 1 2/δ dn 1 2/δ τ 2/δ n t=d n+2 ta αt 1 2/δ τ 2/δ o given that d n is chosen as the integer part of h 1 n and a > 1 2 δ. Consequently, EB 2 nx; l, τ = τ 3 O1 + τ 2 O + τ 2 O1 + τ 4+2/δ o1. 24 Proof of Theorem 3.1: From Mansry and Fan 1997 and Lemma 2.1, it suffices to show that nhn n n n ˆτ n = o p 1 10

12 where ˆτ n = ĥn. It suffices to show that all elements of the vector n n τ are stochastically equicontinuous on τ. For any ɛ > 0, given that ˆτ n = O p 1 there exists r, s 0, with r < s such that P ˆτ n / [r, s] ɛ/3, n. For δ > 0, let w n i, δ = sup {τ1,τ 2 [r,s] [r,s]: τ 1 τ 2 <δ} d n i, τ 1, τ 2 where d n x; i, τ 1, τ 2 = e i n τ 2 e i n τ 1, e i is a row vector with i-th component equal to 1, and 0 elsewhere. Then, for η > 0 P d n i, 1, ˆτ n η P 1 ˆτ n 1 δd n i, 1, ˆτ n η + P ˆτ n 1 > δ where 1A is the indicator function for the set A. By assumption, there exists N ɛ,1 such that P ˆτ n 1 > δ ɛ 3, n N ε,1. Also, P 1 τ n 1 δd n i, 1, ˆτ n η P 1ˆτ n [1 δ, 1 + δ] [r, s]d n i, 1, ˆτ n η + P ˆτ n / [r, s] P w n i, δ η + P ˆτ n / [r, s] where, as mentioned before P ˆτ n / [r, s] ɛ 3. Furthermore, if n e i n τ is asymptotically stochastically uniformly equicontinuous with respect to τ on [r, s], then there exists N ɛ,2 such that 25 P w n i, δ η ɛ 3 whenever n N ɛ,2. Setting N ɛ = max{n ɛ,1, N ɛ,2 } we obtain that with stochastic equicontinuity we have nhn n n n ˆτ n = o p 1. Now, since τ 1, τ 2, r and s are nonstochastic, then w n i, δ = n sup {τ 1,τ 2 [r,s] [r,s]: τ 1 τ 2 <δ} where G nx; = g n,0x;,..., g n,px;. Thus, if e i S n x; τ 1 1 G nx; τ 1 e i S n x; τ 2 1 G nx; τ 2 and the desired result is obtained. sup s n,l x; τ 1 s n,l x; τ 2 = o p 1 26 {τ 1,τ 2 [r,s] [r,s]: τ 1 τ 2 <δ} sup n gn,lx; τ 1 n gn,lx; τ 2 = o p 1 27 {τ 1,τ 2 [r,s] [r,s]: τ 1 τ 2 <δ} 11

13 By the Mean Value Theorem of Jennrich 1969 sup s n,l x; τ 1 s n,l x; τ 2 sup {τ 1,τ 2 [r,s] 2 : τ 1 τ 2 <δ} τ [r,s] ds nl x; τ dτ δ a.s. Lemma 2.2 and Theorem in Davidson 1994 imply that equation 26 holds. Furthermore, by the Mean Value Theorem of Jennrich 1969 and by Cauchy-Schwarz Inequality, we have that n gn,l x; τ 1 n gn,l x; τ 2 = τ 1 nhn dg nl x;τhn τ 2 dτ dτ τ 1 dg nhn nl x;τhn τ 2 dτ dτ τ 1 τ 2 1dτ 1/2 τ1 τ 2 = τ 1 τ 2 1/2 τ1 τ 2 τ 1 τ 2 1/2 s r nhn dg nl x;τhn dτ nhn dg 2 1/2 nl x;τhn dτ dτ nhn dg 2 1/2 nl x;τhn dτ dτ. 2 dτ 1/2 28 Once again, Theorem in Davidson 1994 and Lemma 2.3 imply that equation 27 holds. Proof of Corollary 3.1: that P ˆτ n / [r, s] ɛ/3, For any ɛ > 0, given that ˆτ n = O p 1 there exists r, s 0, with r < s such n.. From Theorem 3.1, it suffices to show that nhn H nτ b n x; τ bx τ p+1 m p+1 x p + 1! + τ p+1 o p 1 is stochastic equicontinuous with respect to τ on [r, s] with H nτ = diag{τ j } p j=0. Masry and Fan 1997, showed that H nτ b n x; τ bx τ p+1 m p+1 x p + 1! + τ p+1 o p 1 = S 1 n x; τ G nx; τ thus, from theorem 3.1, the result follows. 12

14 Appendix 2: Tables and Graphs Table 1: Bias and MSE ratios for m 1 x and m 1 1 x using h CV and h AMISE m 1 x n x = 0.5π x = π x = 1.5π ρ = Bias MSE Bias MSE ρ = Bias MSE Bias MSE ρ = Bias MSE Bias MSE m 1 1 x n x = 0.5π x = π x = 1.5π ρ = Bias MSE Bias MSE ρ = Bias MSE Bias MSE ρ = Bias MSE Bias MSE

15 Table 2: Bias and MSE ratios for m 2 x and m 1 2 x using h CV and h AMISE m 2 x n x = 0.25 x = 0.5 x = 0.75 ρ = Bias MSE Bias MSE ρ = Bias MSE Bias MSE ρ = Bias MSE Bias MSE m 1 2 x n x = 0.25 x = 0.5 x = 0.75 ρ = Bias MSE Bias MSE ρ = Bias MSE Bias MSE ρ = Bias MSE Bias MSE

16 Figure 1: Estimated density of regression a Estimated density of m 1 0.5π using h CV b Estimated density of m using h CV 15

17 Figure 2: Estimated density of regression derivative a Estimated density of m π using h CV b Estimated density of m using h CV 16

18 References [1] Boente, G. and Fraiman, R., Asymptotic distribution of data-driven smoothers in density and regression estimation under dependence. The Canadian Journal of Statistics, 23, [2] Bosq, D., Nonparametric statistics for stochastic processes: estimation and prediction. Springer- Verlag, New York. [3] Davidson, J., Stochastic limit theory. Oxford University Press, Oxford. [4] Dony, J., Einmahl, U. and Mason, D., Uniform in bandwidth consistency of local polynomial regression function estimators. Austrian Journal of Statistics, 35, [5] Doukhan, P., Mixing. Springer-Verlag, New York. [6] Fan, J., Design-adaptive nonparametric regression. Journal of the American Statistical Association, 87, [7] Jennrich, R. I., Asymptotic properties of nonlinear least squares estimators. The Annals of Mathematical Statistics, 40, [8] Li, Q. and Racine, J., Cross-validated local linear nonparametric regression. Statistica Sinica, 14, [9] Martins-Filho, C. and Yao, F., Nonparametric regression estimation with general parametric error covariance. Journal of Multivariate Analysis, 100, [10] Masry, E. and Fan, J., 1997 Local polynomial estimation of regression function for mixing processes. The Scandinavian Journal of Statistics, 24, [11] Parzen, E., On estimation of a probability density function and mode. Annals of Mathematical Statistics, 33, [12] Robinson, P., Nonparametric estimators for time series. Journal of Time Series Analysis, 4, [13] Silverman, B.W., Density estimation for statistics and data analysis. Chapman & Hall, London. [14] Ruppert, D., Sheather, S. and Wand, M., An effective bandwidth selector for local least squares regression. Journal of the American Statistical Association, 90, [15] Xia, Y. and Li, W. K., 2002 Asymptotic behavior of bandwidth selected by the cross-validation method for local polynomial fitting. Journal of Multivariate Analysis, 83, [16] Ziegler, K., Adaptive kernel estimation of the mode in nonparametric random design regression model. Probability and Mathematical Statistics, 24,