Asymptotic normality of Powell s kernel estimator

Transcription

1 Asymptotic normality of Powell s kernel estimator Kengo Kato This version: July 3, 2009 Abstract In this paper, we establish asymptotic normality of Powell s kernel estimator for the asymptotic covariance matrix of the quantile regression estimator for both i.i.d. and weakly dependent data. As an application, we derive the optimal bandwidth that minimizes the approximate mean squared error of the kernel estimator. Key words: asymptotic normality; bandwidth selection; density estimation; quantile regression. AMS subject classifications: 62G07, 62J05. Introduction This paper establishes asymptotic normality of Powell s 99 kernel estimator for the asymptotic covariance matrix of the quantile regression estimator. Let us first introduce a quantile regression model. Let Y i, X i i =, 2,... n be i.i.d. observations from Y, X where Y is a response variable and X is a d-dimensional covariate vector. The τ-th τ 0, conditional linear quantile regression model is defined as Q Y τ X = X β 0 τ, where Q Y τ X = inf{y : PY y X τ} is the τ-th conditional quantile function of Y given X. Koenker and Bassett 978 propose the estimator ˆβ KB τ for β 0 τ which minimizes the objective function ρ τ Y i X iβ, 2 where ρ τ u = {τ Iu 0}u is called the check function. It is well known that, under suitable regularity conditions, ˆβKB τ satisfies consistency and asymptotic normality; see Department of Mathematics, Graduate School of Science, Hiroshima University, -3- Kagamiyama, Higashi-Hiroshima, Hiroshima , Japan. kkato@hiroshima-u.ac.jp The first version appeared on May, The former Section 4 is removed.

2 Chapter 4 of Koenker Letting fy x denote the conditional density of Y given X = x, the asymptotic covariance matrix of n ˆβ KB τ β 0 τ is given by J τστj τ, where Jτ = E[fX β 0 τ XXX ] and Στ = τ τe[xx ]. The estimation of the matrix Στ is straightforward. However, since the matrix Jτ involves the conditional density, its estimation is not a trivial task. Section 3.4 of Koenker 2005 introduces two approaches to the estimation of the matrix Jτ. The first one, suggested by Hendricks and Koenker 992, is a natural extension of the scalar sparsity estimation Siddiqui, 960. On the other hand, Powell 99 proposes the kernel estimator Ĵ P τ = nh n Y i X i K ˆβτ X i X h i, n where ˆβτ is a n-consistent estimator of β 0 τ, h n > 0 is a bandwidth and K is the uniform kernel { if u, 2 Ku = 3 0 otherwise. In usual, we take ˆβτ = ˆβ KB τ. He shows that ĴPτ is consistent under some regularity conditions. Especially, he imposes the condition on the bandwidth h n that h n 0 and nh 2 n. The recent study by Angrist et al shows that ĴPτ is uniformly consistent over a closed interval of τ even when the model is misspecified. However, to the author s knowledge, there is no literature that rigorously studies the asymptotic distribution of ĴPτ. This paper establishes asymptotic normality of ĴPτ under the conditions that the conditional density is twice continuously differentiable and that the bandwidth h n is such that h n 0 and n /2 h n / logn. The condition on the bandwidth is close to the one required for proving consistency of ĴPτ. As an application, we evaluate the approximate mean squared error AMSE of ĴPτ and derive the optimal h n that minimizes the AMSE, which is another contribution of this paper. Since the kernel estimator contains the estimated parameter in the sum, the direct calculation of the mean squared error MSE is infeasible. So the evaluation of the MSE is a complicated task. This paper is the first result that derives the optimal bandwidth for ĴPτ under a certain criterion. In addition, we extend the results to weakly dependent data. We now review the literature related to this paper. Koul 992 discusses the uniform convergence of the kernel estimator of the error density in a linear model based on the weak convergence results of the residual empirical processes. Chai et al. 99, Chai and Li 993 and Li 995 show several important asymptotic results for the kernel estimation of the error density in a linear model with fixed design when using the least squares method and the least absolute deviation method to estimate the coefficients. Especially, the latter two papers show asymptotic normality of the histogram estimator namely, the estimator using the uniform kernel of the error density. Unfortunately, the proof of Lemma 4 in 2

3 Chai and Li 993, which is a key to their asymptotic normality results, is incorrect. See the remark after the proof of Lemma below. Except for the correctness of the proof, the differences of the present paper from them are as follows: i Chai and Li treat the estimation of the scalar unconditional error density and the present paper treats the estimation of the matrix that involves the conditional density. This difference affects the bandwidth selection. See Section 3. ii Chai and Li impose the stringent condition that the covariate vectors are bounded over all observations. Actually, the boundedness of the covariate vectors is essential to their proofs. The present paper removes this condition. iii Chai and Li only treat independent data, while the present paper treats both i.i.d. and weakly dependent data. The estimation of the innovation density in parametric time series models is studied by Robinson 987, Liebscher 999, Müller et al and Schick and Wefelmeyer Among them, Liebscher 999 establishes asymptotic normality of the residualbased kernel estimator of the innovation density of a nonlinear autoregressive model. He assumes that the kernel function is Lipschitz continuous see equation 3.5 of his paper, which is essential to his proof, while the uniform kernel treated in the present paper is not. The estimation of the error density in nonparametric regression causes much attention in recent years. Several authors who address this issue include Ahmad 992, Cheng 2002, 2004, 2005, Efromovich 2005, 2007a,b and Liang and Niu Cheng 2005 and Liang and Niu 2009 show asymptotic normality of their kernel estimators; both of them use the uniform kernel when deriving the asymptotic distributions. The rest of the paper is organized as follows. In Section 2, we prove asymptotic normality of Powell s kernel estimator ĴPτ for i.i.d. data. In Section 3, we use the asymptotic distribution to evaluate the AMSE and derive the optimal h that minimizes the AMSE. In Section 4, we establish asymptotic normality of ĴPτ under a weak dependence condition. In Section 5, we leave some concluding remarks. We introduce some notations used in the present paper. Let IA denote the indicator of an event A. The symbols p and d denote convergence in probability and convergence in distribution, respectively. We use the stochastic orders o p and O p in the usual sense. For a real number a, [a] denotes the greatest integer not exceeding a. For a d d matrix A = [a a d ], veca = a,..., a d. 2 Asymptotic normality of Powell s kernel estimator In this section, we study the first order asymptotic property of ĴPτ for i.i.d. data. Throughout this paper, we fix τ and suppress the dependence on τ for notational convenience. For example, we simply write β 0 for β 0 τ. Then, the model may be written as Y = X β 0 + U, Q U τ X = 0, 4 3

4 where Q U τ X = inf{u : PU u X τ}. It should be noted that the distribution of U generally depends on τ and X. For example, let us consider a linear location scale model Y = X θ 0 + X γ 0 ϵ, 5 where X γ 0 > 0 and ϵ is independent of X. In this model, U corresponds to X γ 0 {ϵ F τ}, where F is the distribution function of ϵ. Typically, the model 5 allows for the heteroscedasticity of U. We now return to the general model 4. Letting f 0 u x denote the conditional density of U given X = x, the matrix J is expressed as E[f 0 0 XXX ]. In order to justify our asymptotic theory, we impose the following regularity conditions: A {U i, X i, i =, 2,... } is an i.i.d. sequence whose marginal distribution is same as U, X. A2 The conditional density f 0 u x of U given X = x is twice continuously differentiable with respect to u for each x. Furthermore, there exist measurable functions G j x j = 0,, 2 such that f j 0 u x G j x j = 0,, 2 for every realization u, x of U, X, E[ X 2 + X 4 + X 5 G 0 X] <, E[ X 2 + X 3 + X 4 G X] < and E[ X 2 G 2 X] <, A3 As n, h n 0 and n /2 h n / logn. We state some remarks on the conditions. We substantially assume the existence of the fifth order moment of X, which is slightly stronger than the one assumed in proving consistency of ĴP. For example, Angrist et al assume the fourth order moment of X to prove uniform consistency of ĴP. The first part of condition A2 is standard in the conditional density estimation literature for example, see Fan and Yao, 2005, Chapter 5. Unlike the fully nonparametric conditional density estimation, the effect of localization on the X-space does not work in the present situation. Thus, the latter part of A2 is needed to ensure the dominated convergence. Condition A3 allows for bandwidth rules such as the rule used in R implementation of the kernel estimation in quantreg package Koenker, 2009, the Bofinger 975 and the Hall and Sheather 988 rules, although the latter two bandwidth rules are originally for the scalar sparsity estimation. Powell 99 and other authors show consistency of ĴP under the condition that h n 0 and nh 2 n. For any fixed matrix A R d d, define T n β = nh n = nh n Yi X Z i K iβ h n Ui X Z i K iβ β 0, 6 where Z i = trax i X i. We first show asymptotic normality of T n ˆβ. Then, we use the Cramér-Wold device to derive the asymptotic distribution of ĴP. The proof of asymptotic h n 4

5 normality of T n ˆβ consists of series of lemmas. Lemma uses the empirical process technique to establish the uniform convergence in probability. See, for example, Chapter 2 of van der Vaart and Wellner 996 for related materials. Lemma. Suppose that conditions A-A3 hold. Then, for any fixed l > 0, we have uniformly in nβ β 0 l. T n β E[T n β] = T n β 0 E[T n β 0 ] + o p nh n /2 Proof. We have to show T n β 0 + n /2 t E[T n β 0 + n /2 t] = T n β 0 E[T n β 0 ] + o p nh n /2 uniformly in t l. Observe that h{t n β 0 + n /2 t T n β 0 } = Ui n Z i {K /2 X it K n = 2 { n n + n n h n Ui h n Z i Ih n < U i h n + n /2 X it Z i I h n U i < h n + n /2 X it Z i I h n + n /2 X it U i < h n } Z i Ih n + n /2 X it < U i h n } =: 2 {W nt W 2n t + W 3n t W 4n t}. 7 It suffices to show that n /2 h /2 n {W jn t E[W jn t]} p 0 uniformly in t l for j =, 2, 3, 4. We only prove the j = case since the proofs for the other cases are completely analogous. Fix any ϵ > 0. Define Ui t = Z i Ih n < U i h n + n /2 X it. Let σ,..., σ n be independent and uniformly distributed over {, } and independent of U, X,..., U n, X n. Using the symmetrization technique van der Vaart and Wellner, 996, Lemma 2.3.7, we have η n P sup W n t E[W n t] > n /2 h /2 2P sup σ i Ui t > n /2 h /2, 4 where η n = 4/ϵ 2 h n sup E[{U t} 2 ]. Let F 0 u x denote the conditional distri- 5

6 bution function of U given X = x. Then, we have sup E[{Ui t} 2 ] E[ Z 2 Ih n < U h n + n /2 l X ] = E[ Z 2 {F 0 h n + n /2 l X X F 0 h n X}] ln /2 E[ Z 2 G 0 X X ], where we have used F 0 h n +n /2 l X X F 0 h n X ln /2 G 0 X X. Since nh 2 n, η n = o as n and consequently η n /2 for large n. Thus, for large n, P W n t E[W n t] > n /2 h /2 4P sup σ i Ui t > n /2 h /2. 4 sup Let D n = {U i, X i, i =,..., n}. Given D n, at most finite elements are contained in the functional set {σ n n n σ iui t : t l}, where σ n = σ,..., σ n, since every element of the functional set is of the form σ n n i {subset of {,...,n}} σ iz i. Let k n denote the cardinality of this set. Then, there exist k n points t j {t : t l}, j =,..., k n such that P sup σ i Ui t > n /2 h /2 4 D n k n j= P σ i Ui t j > n /2 h /2 4 D n. It is noted that k n and t j j =,..., k n depend on D n. Observe that for any t l, Z i Ih n < U i h n + n /2 l X i σ i U i t Z i Ih n < U i h n + n /2 l X i. 8 By Hoeffding s inequality van der Vaart and Wellner, 996, Lemma 2.2.7, P σ i Ui t > n /2 h /2 4 D n 2 exp ϵ2 h n, 32v n sup where v n = n n Z i 2 Ih < U i h n + n /2 l X i. Hence, P sup σ i Ui t > n /2 h /2 4 D n 2k n exp ϵ2 h n. 32v n We now bound k n. It is not difficult to see that k n is bounded by the cardinality of the set { A {U, X,..., U n, X n } : A A }, where A = { {u, x : u > h, u h + x t} : h > 0, t R d}. Application of Lemma in van der Vaart and Wellner 996 shows that the Vapnik-Červonenkis VC index V A of A is finite, namely 0 < V A < ; see van der Vaart and Wellner 996, pp. 35 for the definition of the VC index. Then, Sauer s 6

7 lemma van der Vaart and Wellner, 996, Corollary implies that k n is bounded by cn VA for some constant c not depending on D n. Therefore, we have P sup σ i Ui t n > n /2 h /2 4 D n 2cn VA exp ϵ2 h n. 9 32v n Define A n = { v n > } ϵ 2 h n. 32V A logn Using 9 and the obvious inequality, we have P sup σ i U i t > n /2 h /2 4 [ ] PA n + 2cn VA E exp ϵ2 h n IA c 32v n n PA n + 2cn. To show that PA n 0, it suffices to show that lognh n v n p 0. By Markov s inequality, for any δ > 0, P v n > h nδ δ lognh n E[ Z 2 Ih n < U h n + n /2 l X ] logn lδ n /2 lognh n E[ Z 2 G 0 X X ] 0, where we have used n /2 h n / logn. Therefore, we complete the proof. Remark. The proof of Lemma 4 in Chai and Li 993 states that the cardinality of the functional set {σ n n n σ iia n < e i < a n + h i : 0 < h i b n } is bounded by n +, where {e i } is arbitrarily fixed, a n is the bandwidth such that a n 0 and b n = Cn /2. However, this statement is incorrect. For example, if a n < e i < a n + b n for i =,..., n, the cardinality of the functional set is 2 n. Remark 2. It is not possible to directly apply Theorem II 37 in Pollard 984 to obtain the uniform convergence result of Lemma since Z i is not bounded random variable. Instead of relying on Lemma II 33 in Pollard 984, we use the explicit bound 8 when using Hoeffding s inequality in the proof of Lemma. Lemma 2. Suppose that conditions A-A3 hold. Then, for any fixed l > 0, we have uniformly in nβ β 0 l. E[T n β] = E[T n β 0 ] + On /2 7

8 Proof. We have to show E[T n β 0 + n /2 t] = E[T n β 0 ] + On /2 uniformly in t l. Using the relation E[T n β] E[Zf 0 0 X] [ ] = E Z Ku h n X β β 0 {f 0 uh X f 0 0 X}du [ ] = he Z uku h n X β β 0 g hn u Xdu, where g hn u x = uh n {f 0 uh n x f 0 0 x} for u 0 and g hn 0 x = 0, the absolute value of the difference E[T n β 0 + n /2 t] E[T n β 0 ] is evaluated as u{ku n /2 h n X t Ku}g hn u Xdu] he [Z he [ Z G X ] u{ku n /2 h n X t Ku} du, 0 where we have used g hn u x sup u f 0 u x G x. Using the identity I u v I u = {I < u + v I u < + v}iv > 0 + {I + v u < I + v < u }Iv < 0, we have u{i u v I u } du { +v +v } = udu + u du Iv > 0 { } + u du + u du Iv < 0 +v 2 + v v. +v Since nh 2 n, n /2 h n by for large n. Therefore, the right hand side of 0 is bounded ln /2 E [ Z G X + l X X ] for any t l. This yields the desired result. Lemma 3. Under conditions A-A3, we have nh n /2 {T n β 0 E[T n β 0 ]} d N0, E[Z 2 f 0 0 X]/2. Proof. This result can be proved by checking the conditions of the Lindeberg-Feller central limit theorem. Since the argument is standard, we omit the detail. 8

9 Suppose that ˆβ is n-consistent for β 0, namely ˆβ = β 0 +O p n /2. Then, by Lemmas and 2, nh n /2 {T n ˆβ E[T n β 0 ]} = nh n /2 {T n ˆβ E[T n β] β= ˆβ} + nh n /2 {E[T n β] β= ˆβ E[T n β 0 ]} = nh n /2 {T n β 0 E[T n β 0 ]} + o p. Using the Taylor expansion, we see that E[T n β 0 ] = E[Zf 0 0 X] + h2 n 2 E[Zf 0 0 X] + oh 2 6 n. Therefore, by Lemma 3, we get the following theorem: Theorem. Suppose that conditions A-A3 hold and ˆβ is n-consistent for β 0. Then, { } nh n /2 T n ˆβ E[Zf 0 0 X] h2 n 2 E[Zf 0 0 X] + oh 2 d 6 n N0, E[Z 2 f 0 0 X]/2. We now describe the asymptotic distribution of the matrix estimator ĴP. Let S = XX. Since tras = veca vecs, the asymptotic covariance matrix of T n ˆβ is written as 2 veca E[f 0 0 X vecs vecs ] veca. Therefore, the Cramér-Wold device leads to the next theorem: Theorem 2. Suppose that conditions A-A3 hold and ˆβ is n-consistent for β 0. Then, { } nh n /2 Ĵ P J h2 n 2 E[f 0 0 XXX ] + oh 2 6 n is asymptotically normally distributed with zero mean matrix. The asymptotic covariance of the j, k-th and the l, m-th elements is given by where j, k, l, m =,..., d. 2 E[f 00 XX j X k X l X m ], We end this section with a remark. While we put the conditional quantile restriction on U, the proof of Theorem 2 does not use the restriction. Therefore, Theorem 2 is valid for any ˆβ such that ˆβ = β 0 + O p n /2 for some β 0. For example, when the model is misspecified, ˆβ KB is n-consistent for β 0 that uniquely solves E[{τ IY X β 0 }X] = 0, where the existence and the uniqueness of such β 0 is assumed. See Angrist et al for a proof of this result. Thus, Theorem 2 is valid for ˆβ = ˆβ KB even when the model is misspecified. 9

10 3 Application: bandwidth selection Since ĴP contains the estimated parameter in the sum, the direct calculation of the bias and the variance of ĴP is infeasible. However, Theorem 2 enables us to approximate the mean squared error MSE of ĴP. From Theorem 2, we can see that the MSE is approximated as MSEh n := E[tr{ĴP J 2 }] d h4 2 n E[f XX j X k ] nh n j,k= =: AMSEh n. The optimal h n that minimizes AMSEh n is given by h opt d E[f 0 0 XXj 2 Xk] 2 j,k= 4.5 d n = n /5 j,k= E[f 00 XXj 2 Xk 2] d j,k= E[f XX j X k ] 2 /5, where we assume that the denominator is not zero. It should be noted that h opt n depends on τ, namely h opt n = h opt n τ, since the distribution of U generally depends on τ. We further note that h opt n depends on the distribution of X, which is the difference from the scalar unconditional density estimation. In the simple case where f 0 u x is independent of x, namely f 0 u x = f 0 u, h opt depends on the unconditional error density and the second and the fourth order moments of X. It is well known that convergence in distribution does not necessarily imply moment convergence. In order to make the argument rigorous, we introduce the truncated MSE MSE T h n := E[min[tr{n 4/5 ĴP J 2 }, T ]] and take the limit n and T. In a different context, Andrews 99 uses the same device to evaluate covariance matrix estimators that contain estimated parameters. Then, the optimality of h opt n is stated as follows. Proposition. Suppose that conditions A-A3 hold and ˆβ is n-consistent for β 0. Then, lim lim {MSE T h n MSE T h opt n } 0, T n where the inequality is strict unless h n = h opt n + on /5. Proof. The proposition follows from the fact that for a bounded sequence of random variables, convergence in distribution implies moment convergence of any order. As well as the usual density estimation, h opt n involves unknown quantities and is not directly usable. In the density estimation literature, there are several methods, namely rule of thumb, cross validation and plug-in methods, to cope with this difficulty. For a 0

11 comprehensive treatment on practical aspects of density estimation, see Sheather 2004 and references therein. For example, the optimal bandwidth h opt n for a Gaussian location model Y = X θ 0 + ϵ, ϵ X N0,, 2 is given by { 4.5 d n = n /5 j,k= E[X2 j Xk 2] } /5 ατ d j,k= E[X, jx k ] 2 h opt where ατ = { Φ τ} 2 ϕφ τ, Φ and ϕ are the distribution function and the density function of the standard normal distribution. Thus, a rule of thumb bandwidth for the Gaussian location model is given by ĥ rot n { 4.5 d = n /5 j,k= n n X2 ijxik 2 ατ d j,k= n n X ijx ik 2 4 Extension to weakly dependent data } /5. So far this paper has considered i.i.d. data. We now make note of sufficient conditions for asymptotic normality of Powell s kernel estimator for weakly dependent data. Let {U i, X i, i =, 2,... } be a strictly stationary sequence whose marginal distribution is same as U, X. Under a sufficient weak dependence condition and additional regularity conditions, it can be shown that n ˆβKB β 0 d N0, J ΩJ, where Ω is the asymptotic covariance matrix of n /2 n {τ IU i 0}X i. Of course, Ω reduces to Σ when {U i, X i } is i.i.d. See, for example, Phillips 99, pp.459. In this case, the estimation of Ω is not straightforward. It should be noted that Theorem in Andrews 99 does not apply to the estimation of Ω since the smoothness of the moment function is violated in the present situation. However, we concentrate on the estimation of J in this paper and will discuss the estimation of Ω in another place. Here we state some regularity conditions to ensure asymptotic normality of ĴP. B {U i, X i, i =, 2,... } is a strict stationary sequence whose marginal distribution is same as U, X. B2 The sequence {U i, X i, i =, 2,... } is β-mixing; that is [ ] where F j i βj := sup E i sup PA F i PA A Fi+j 0, as j, is the σ-field generated by {U k, X k, k = i,..., j} j i. In addition, j λ {βj} 2/δ <, 3 j= for some δ > 2 and λ > 2/δ.

12 B3 E[ X max{6,2δ} ] <, where δ is given in condition B2. B4 The conditional density f 0 u x of U given X = x is twice continuously differentiable with respect to u for each x. Furthermore, there exist a constant A 0 > 0 and measurable functions G j x j =, 2 such that f 0 u x A 0, f j 0 u x G j x j =, 2 for every realization u, x of U, X, E[ X 2 + X 3 + X 4 G X] < and E[ X 2 G 2 X] <, where f j 0 u x = j f 0 u x/ u j for j =, 2. B5 Let f 0 u, u +j x, x +j ; j denote the conditional density of U, U +j given X, X +j = x, x +j j. Then, there exists a constant A > 0 independent of j such that f 0 u, u +j x, x +j ; j A for every realization u, u +j, x, x +j of U, U +j, X, X +j. B6 As n, h n 0 and n /2 h n / logn. In addition, there exists a sequence of positive integers s n satisfying s n and s n = onh n /2 as n such that n/h n /2 βs n 0 as n. 4 The β-mixing condition is required for establishing the uniform convergence result corresponding to Lemma because our approach uses the blocking technique as in Yu 994 and Arcones and Yu 994. The blocking technique enables us to employ the symmetrization technique and an exponential inequality available in the i.i.d. case. In order to validate the blocking technique, we use Lemma 4. in Yu 994, which requires the β-mixing condition. A set of conditions such as 3, E[ X 2δ ] <, the boundedness of the conditional densities included in conditions B4-B5 and the latter part of condition B6 is often assumed in density estimation and nonparametric regression. See Condition of Theorem 6.3 in Fan and Yao 2005 we note that Theorem 6.3 of Fan and Yao 2005 assumes the corresponding α-mixing condition, which is weaker than the current β-mixing condition. These conditions are sufficient for asymptotic normality of nh n /2 {T n β 0 E[T n β 0 ]}, where T n β is given by 6. A sufficient condition on the mixing coefficient βj to satisfy the conditions 3 and 4 is provided in Fan and Yao 2005, pp.387. Below we follow the notations used in Section 2. The next lemma is essential to our purpose. Lemma 4. Under conditions B-B6, the conclusion of Lemma is valid in the present situation. Proof. Working with the same notations as in the proof of Lemma, we show that n /2 h /2 n {W n t E[W n t]} p 0, uniformly in t l. Before proceeding to the proof, we introduce a sequence of independent blocks as in Yu 994 and Arcones and Yu 994. Divide the n-sequence {,..., n} into blocks of length a n = [n 2/δ/ 2/δ+λ ] one after the other: H k = {i : 2k a n + i 2k a n }, T k = {i : 2k a n + i 2ka n }, 2

13 for k =,..., µ n, where µ n = [n/2a n ]. Let {Ũi, X i, i µ n k= H k} be a set of random vectors such that blocks {Ũi, X i, i H k } k =,..., µ n are independent and have the same distribution as {U i, X i, i H }. Replacing U i, X i with Ũi, X i, define Z i and Ũi t as Z i and Ui t, respectively, for i µ n k= H k. Fix any ϵ > 0. Observe that P sup W n t E[W n t] > n /2 h /2 2a n µ n P sup {Ui t E[U i t]} > n /2 h /2 2 + P sup {Ui t E[Ui t]} > n /2 h / i=2a nµ n+ A simple calculation shows that the second term of the right hand side of 5 converges to zero; use the fact that sup Ui t Z i Ih n < U i h n + n /2 l X i. On the other hand, using the same argument as in Lemma 4.2 of Yu 994, we may bound the first term of the right hand side of 5 by µ n 2P sup {Ṽkt E[Ṽkt]} > n /2 h /2 + 2µ n βa n, 4 where k= Ṽ k t = i H k Ũ i t. Because of the condition 3, µ n βa n = o. Therefore, it suffices to show that µ n P sup {Ṽkt E[Ṽkt]} > n /2 h / k= Let σ,..., σ µn be independent and uniformly distributed over {, } and independent of {Ũi, X i, i H k } k =,..., µ n. Since {t Ṽkt, k =,..., µ n } is a sequence of i.i.d. stochastic processes, the symmetrization technique van der Vaart and Wellner, 996, Lemma yields that µ n ξ n P sup {Ṽkt E[Ṽkt]} > n /2 h /2 4 k= µ n 2P sup σ k Ṽ k t > n /2 h /2, 6 where ξ n = 6µ n /ϵ 2 nh n sup E[{Ṽt} 2 ]. We show that sup E[{Ṽt} 2 ] = Oa n n /2. By stationarity, a n E[{Ṽt} 2 ] = a n E[U t} 2 ] + 2a n j/a n E[U tu+jt]. 3 j= k=

14 Observe that sup E[{U t} 2 ] = On /2. By conditioning on X, X +j, we have E[U tu+jt] [ E Z Z +j hn+n /2 l X hn+n /2 l X +j h n const. n E[ Z Z +j X X +j ] = On, h n f 0 u, u +j X, X +j ; jdu du +j uniformly in t l and j. This yields that a n sup E[U tu +jt] = Oa nn = on /2. j= Thus, we have shown that sup E[{Ṽt} 2 ] = Oa n n /2, which implies that ξ n = On /2 h n = o and consequently ξ n /2 for large n. Therefore, for large n, µ n P sup {Ṽkt E[Ṽkt]} > n /2 h /2 4 k= µ n 4P sup σ k Ṽ k t > n /2 h /2. 6 The rest of the proof is similar to the latter part of the proof of Lemma. Arguing as in the proof of Lemma, it is shown that the cardinality of the functional set {σ µ n n µ n k= σ kṽkt : t l}, where σ µ n = σ,..., σ µn, is bounded by some polynomial of n uniformly over every realization of Dn := {Ũi, X i, i µ n k= H k}. In addition, Hoeffding s inequality implies that sup P µ n k= σ k Ṽ k t > n /2 h /2 6 D n k= 2 exp h nϵ 2, 52w n where w n = n µ n k= { i H k Z i Ih n < Ũi h n + n /2 l X i } 2. Thus, it suffices to show that lognh n w n p 0. 6 From the evaluation of E[{Ṽt} 2 ] above, it is shown that { } 2 E Z i Ih n < Ũi h n + n /2 l X i = Oa n n /2, i H which leads to E[w n ] = Oµ n a n n 3/2 = On /2. Since n /2 h n / logn, 6 follows from Markov s inequality. Therefore, we complete the proof. 4 ]

15 The proof of Lemma 2 does not use the independence assumption and hence the conclusion of Lemma 2 applies to the present situation. Thus, for any n-consistent estimator ˆβ, we have nh n /2 {T n ˆβ E[T n β 0 ]} = nh n /2 {T n β 0 E[T n β 0 ]} + o p. In addition, mimicking the proof of Theorem 6.3 in Fan and Yao 2005, it can be shown that under conditions B-B6, nh n /2 {T n β 0 E[T n β 0 ]} d N0, E[Z 2 f0 X]/2. Therefore, arguing as in Section 2, we get the following theorem: Theorem 3. Suppose that conditions B-B6 hold and ˆβ is n-consistent for β 0. Then, { } nh n /2 Ĵ P J h2 n 2 E[f 0 0 XXX ] + oh 2 6 n is asymptotically normally distributed with zero mean matrix. The asymptotic covariance of the j, k-th and the l, m-th elements is given by where j, k, l, m =,..., d. 2 E[f 00 XX j X k X l X m ], The result of Theorem 4 is same as that of Theorem 2 which deals with the i.i.d. case. Hence, the optimal bandwidth that minimizes the AMSE under the current weak dependence condition is the same as that for the i.i.d. case. 5 Concluding remarks In this paper, we have shown asymptotic normality of Powell s kernel estimator for the asymptotic covariance matrix of the quantile regression estimator for both i.i.d and weakly dependent data. The asymptotic distribution of the kernel estimator enables us to calculate the approximate mean squared error. It should be noted that since the kernel estimator contains the estimated parameter in the sum, the direct calculation of the mean squared error is infeasible. We have derived the optimal bandwidth that minimizes the AMSE. In this paper, we have treated the uniform kernel, which is suited to apply the VC theory to obtain the uniform convergence results see the proof of Lemma ; see also Angrist et al., 2006, Appendix. As Powell 984 states, however, more elaborating kernel functions could be devised. Although the kernel selection is less important than the bandwidth selection in density estimation cf. Fan and Yao, 2005, Section 5.2, the extension of the present paper s results to a general kernel remains in the future research. Acknowledgments The author would like to thank Professor Naoto Kunitomo and Professor Tatsuya Kubokawa for their encouragement and suggestions. 5

16 References Ahmad, I.A Residual density estimation in nonparametric regression. Statistics and Probability Letters Andrews, D.W.K. 99. Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica Angrist, J., Chernozhukov, V. and Fernandéz-Val, I Quantile regression under misspecification, with an application to the U.S. wage structure. Econometrica Arcones, M.A. and Yu, B Central limit theorems for empirical and U-processes of stationary mixing sequences. Journal Theoretical Probability Bofinger, E Estimation of a density function using order statistics. Australian Journal of Statistics 7-7. Chai, G.X. and Li, Z.Y Asymptotic theory for estimation of error distributions of linear model. Science in China Series A Chai, G.X., Li, Z.Y. and Tian, H. 99. Consistent nonparametric estimation of error distributions in linear model. Acta Mathematicae Applicatae Sinica Cheng, F Consistency of error density and distribution function estimators in nonparametric regression. Statistics and Probability Letters Cheng, F Weak and strong uniform consistency of a kernel error density estimator in nonparametric regression. Journal of Statistical Planning and Inference Cheng, F Asymptotic distributions of error density and distribution function estimators in nonparametric regression. Journal of Statistical Planning and Inference Efromovich, S Estimation of the density of regression errors. Annals of Statistics Efromovich, S. 2007a. Adaptive estimation of error density in nonparametric regression in small sample. Journal of Statistical Planning and Inference Efromovich, S. 2007b. Optimal nonparametric estimation of the density of regression errors with finite support. Annals of the Institute of Statistical Mathematics Fan, J. and Yao, Q Nonlinear Time Series: Nonparametric and Parametric Methods. New York: Springer-Verlag. Hall, P. and Sheather, S On the distribution of a Studentized quantile. Journal of the Royal Statistical Society Series B

17 Hendricks, W. and Koenker, R Hierarchical spline models for conditional quantiles and the demand for electricity. Journal of the American Statistical Association Koenker, R Quantile Regression. Oxford: Oxford University Press. Koenker, R Quantile regression in R: a vignette. Available from Koenker, R. and Bassett, G Regression quantiles. Econometrica Koul, H.L Weighted Empiricals and Linear Models. Lecture Notes-Monograph Series, Vol. 2. Hayward, CA: Institute of Mathematical Statistics. Li, Z.Y A study of nonparametric estimation of error distribution in linear model based on L -norm. Hiroshima Mathematical Journal Liang, H.-Y. and Niu, S.-L Asymptotic properties of error density estimator in regression model under α-mixing assumptions. Communications in Statistics: Theory and Methods Liebscher, E Estimating the density of the residuals in autoregressive models. Statistical Inference for Stochastic Processes Müller, U.U., Schick, A. and Wefelmeyer, W Weighted residual-based density estimators for nonlinear autoregressive models. Statistica Sinica Phillips, P.C.B. 99. A shortcut to LAD estimator asymptotics. Econometric Theory Pollard, D Convergence of Stochastic Processes. New York: Springer-Verlag. Powell, J. L Least absolute deviations estimation for the censored regression model. Journal of Econometrics Powell, J. L. 99. Estimation of monotonic regression models under quantile restrictions. In W. Barnett, J. Powell and G. Tauchen Ed. Nonparametric and Semiparametric Models in Econometrics. Cambridge: Cambridge University Press. Robinson, P.M Time series residuals with application to probability density estimation. Journal of Time Series Analysis Sheather, S.J Density estimation. Statistical Science Schick, A. and Wefelmeyer, W Uniformly root-n consistent density estimators for weakly dependent invertible linear processes. Annals of Statistics Siddiqui, M Distribution of quantiles from a bivariate population. Journal of Research of the National Bureau Standards

18 van der Vaart, A.W. and Wellner, J.A Weak convergence and empirical processes: With applications to statistics. New York: Springer-Verlag. Yu, B Rates of convergence for empirical processes of stationary mixing sequences. Annals of Probability