NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS WITH HETEROSCEDASTICITY

Transcription

1 Statistica Sinica , doi: NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS WITH HETEROSCEDASTICITY Jie Guo, Maozai Tian and Kai Zhu 2 Renmin University of China and 2 Chinese Academy of Sciences Abstract: Varying-coefficient models with heteroscedasticity are considered in this paper. Based on local composite quantile regression, we propose a new estimation method to estimate the coefficient functions and heteroscedasticity simultaneously. Moreover, we can get the estimated conditional quantile curves of the error part. The conditional biases, variances, and asymptotic normalities of these estimators are studied explicitly. A simple and quick plug-in bandwidth selector is employed to select the optimal bandwidth. The estimators of the coefficient functions perform efficiently and robustly regardless of the error distributions. When the error ε follows a non-normal distribution, the proposed estimators of the coefficient functions are much more efficient than local polynomial weighted least squares estimators and almost as efficient for normal random errors. The estimator of heteroscedasticity also outperforms other classical estimators in the literature. A goodness-of-fit test based on a bootstrap procedure is proposed to test whether the coefficient functions are actually varying. Both simulations and data analysis are used to illustrate the proposed method. Key words and phrases: Goodness-of-fit test, heteroscedasticity, local composite quantile regression, plug-in bandwidth selector, varying-coefficient models.. Introduction Varying-coefficient models, see, Hastie and Tibshirani 993, arise naturally when one wishes to examine how regression coefficients change over certain factors such as time. Their main appeal is that the modeling bias can significantly be reduced and the curse of dimensionality can be avoided. Heteroscedasticity, which occurs when the variance of the error varies across observations, exists in many cases, and varying-coefficient models with heteroscedasticity have been widely considered in the literature. Fan and Zhang 999, 2000, 2008, among others, considered the model Y i = X T i βt i + σt i ε i,. where {T i, X i, Y i : i =,..., n} is an i.i.d. random sample from T, X, Y, T i R is called smoothing variable, X i = X i, X i2,..., X ip T R p and Y i R,

2 076 JIE GUO, MAOZAI TIAN AND KAI ZHU β = β,..., β p T is an R p -valued unknown smooth function, σ is an unknown positive function of the smoothing variable T, the ε i are i.i.d. and independent of T i, X i, and Varε =. There are many estimates of β in model.. Hastie and Tibshirani 993 considered L 2 estimation and penalized least squares estimation. Fan and Zhang 999 proposed a two-step local polynomial least squares estimation procedure. Chiang, Rice, and Wu 200 proposed a componentwise smoothing spline method. Huang, Wu, and Zhou 2002 proposed a polynomial splines estimation method. All these methods are based on least squares, which suggests that error is homoscedastic and normal with finite variance. The works aside from Hastie and Tibshirani 993 took the variance of the error to be an unknown function of the smoothing variable, but none considered the estimation of heteroscedasticity. Kim 2007 and Wang, Zhu, and Zhou 2009 considered varying-coefficient models in quantile regression using spline polynomials to approximate β. Some papers consider the estimation of heteroscedasticity in., most of which are based on regression residuals. For example, Wu, Chiang, and Hoover 998 and Fan and Zhang 2000 used the weighted residual sum of squares from the local polynomial least squares fit to estimate σ. Zhao 200 proposed a k-nn method based on residuals. The estimator in Tian and Chan 200 was also based on residuals. To our best knowledge, there is no literature considering the simultaneous estimation of β and σ in.. The estimation accuracies of β and σ are relevant to each other, so it is meaningful to develop an estimation approach that can estimate β and σ simultaneously. Quantile regression, proposed by Koenker and Bassett 978, is a statistical technique designed to estimate and conduct inference about conditional quantile functions, a more complete statistical model than mean regression. Based on quantile regression, Zou and Yuan 2008 considered the linear model Y = p j= X jβ j + ε, and proposed composite quantile regression CQR estimates of the coefficients. Let ρ τk s = sτ k Is < 0, k =, 2,..., q, be q check loss functions at q quantile positions τ k = k/q +. CQR estimates β by solving ˆb,..., ˆb q, ˆβ CQR = argmin b,...,b q,β { } ρ τk Y i b k Xi T β, where b k is the 00τ k % quantile of ε. CQR can be more efficient and sometimes arbitrarily more efficient than least squares for non-normal random errors, and almost as efficient for normal random errors. Based on CQR, Kai, Li, and Zou 200 proposed local composite quantile regression LCQR smoothing that outperforms local polynomial regression for k=

3 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 077 various non-normal random errors and is efficient for normal random errors. The theoretical properties and efficiency of LCQR motivate us to apply it to the estimation of model.. Here we propose a new estimation method called local Composite Quantile Regression with Averaged Quantile-Ratio estimation CQR-AQR that can estimate β and σ in model. simultaneously. We also get estimated conditional quantile curves of σtε in. so as to give a thorough description of the error part. The local linear and quadratic CQR are employed to estimate β and its derivative, respectively. To estimate σ, we propose an estimator called the Averaged Quantile-Ratio estimator AQR, see 2.2 or 2.3. More generally, local m-polynomial CQR-AQR is studied. A plug-in method, which can be implemented easily, is employed to select the optimal bandwidth. In a simulation study, we found the proposed CQR-AQR estimation to be very efficient and robust. By comparison with local polynomial weighted least squares estimation of β in model., we find that local CQR estimation is more efficient and robust for non-normal distributed ε, especially for the Cauchy distribution, while it is almost as efficient for normal distributed ε. Thus the estimation of β requires no specification of error distributions. By comparison with the triangular k-nn weight estimator Zhao 200 of σ, the proposed AQR estimator is also more efficient and accurate. One important inference question in model. is whether the coefficient functions are actually varying. A goodness-of-fit test procedure based on a resampling bootstrap is proposed. Simulation suggests that the test procedure is indeed powerful. The rest of the paper is organized as follows. Section 2 and Section 3 study the local linear and quadratic CQR-AQR estimation of model., respectively. We discuss the plug-in bandwidth selector in Section 4. Section 5 presents a hypothesis testing procedure. Section 6 and Section 7 present the results of simulations and empirical study. In Section 8, we consider a more general estimation method for model., local m-polynomial CQR-AQR. Section 9 concludes the paper with discussion. Proofs are presented in the Appendix. 2. Local Linear CQR-AQR Estimation 2.. Estimation In model., if β is second-order differentiable, it can be approximated locally as βt i βt + β tt i t for T i in a neighborhood of any given grid point t. Similarly, σt i can be approximated locally by a constant σt. Let a τk t denote the 00τ k % quantile of σtε, for k =,..., q, where τ, τ 2,..., τ q satisfy 0 < τ < τ 2 < < τ q <. Typically, we use equally spaced quantile

4 078 JIE GUO, MAOZAI TIAN AND KAI ZHU positions: τ k = k/q + for k =,..., q. The estimators ˆβt, ˆβ t and â τk t, for k =,..., q, are obtained by minimizing ρ τk Yi a τk t Xi T βt + β tt i t Ti t K 2. h k= with respect to βt, β t, and a τk t for k =,..., q, where K is a kernel function and h is a bandwidth. For any grid point t, â τk t is the estimator of the 00τ k % quantile of σtε, for k =,..., q. Therefore, we can get q different estimated conditional quantile curves of σtε at quantile positions τ k, for k =,..., q. The q different estimated conditional quantile curves reflect the properties of the error part σtε. We denote the 00τ k % quantile of ε by c τk, for k =,..., q. For brevity, we assume that the density function of ε is non-vanishing everywhere. Therefore c τk is uniquely defined for any 0 < τ k <. Obviously, σt can be easily estimated by â τk t/c τk, which is called quantile-ratio estimator, for any k =,..., q, as long as c τk 0. However, based on the idea of CQR, we combine the strength of the quantile-ratio estimators â τk t/c τk for k =,..., q and propose a new estimator of σt for any grid point t as follows. a If c τk 0 for any k =,..., q, then ˆσt = q b If c τj = 0 for j {,..., q}, then k= ˆσt = q â τk t c τk. 2.2 k= k j â τk t c τk. 2.3 From 2.2, ˆσt is the average of q different Quantile-Ratio estimators. Thus, we call the estimator of σt presented in 2.2 or 2.3 the AQR, and the new estimation method the CQR-AQR. By minimizing 2. and employing the AQR estimator, we get simultaneous estimates of βt and σt. Note that when we apply 2.2 or 2.3, the values of c τk for k =,..., q, have to be known. If the distribution of ε is known, it is easy to get ˆσt using 2.2 or 2.3. If it is unknown, we assume that ε is normal, and take c τk = Φ τ k Asymptotic properties of CQR estimator In this subsection, we establish the asymptotic properties of ˆβt. Let f T denote the marginal density function of the covariate T and f denote

5 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 079 the density function of ε. The observed covariates vector is written as D = T,..., T n, X,..., X n,..., X p,..., X pn T, and we impose the following regularity conditions.. βt is m + times continuously differentiable in a neighborhood of t. 2. σt is positive and continuous. 3. f T and f are positive continuous functions. 4. The kernel function K is symmetric with a compact support. 5. EXX T T = t is positive-definite and continuous in a neighborhood of any given t. 6. The bandwidth h 0, as n and. We use the notation µ j = u j Kudu, ν j = u j K 2 udu, j = 0,, 2,..., q q k= k R q = = τ kk q k= fc τ k 2, R 2 q = τ kk q 2 c k= k τk c τk fc τk fc τk, = R 3 q = q k= τ kk q 2 c τk c q τk k= fc τ k fc τk, k= k = where τ kk minτ k, τ k τ k τ k. Let c = /q q k= /c τ k, Ξ = EX T T = t, Σ X = VarX T = t, Ψ = EXX T T = t, and Ω = EX T T = t VarX T T = t EX T = t. Theorem. Under Conditions 6, bias{ ˆβt D} = 2 β tµ 2 h 2 + o p h 2, cov{ ˆβt D} = ν 0Σ X σ2 t f T t R q + o p. Furthermore, { ˆβt βt } 2 β tµ 2 h 2 d N 0, ν 0Σ f T t where d is convergence in distribution. X σ2 t R q,

6 080 JIE GUO, MAOZAI TIAN AND KAI ZHU By a simple calculation, we have MSE ˆβt = 4 β T tψβ tµ 2 2h 4 + ν 0σ 2 ttr{ψσ f T t X } R q, and global optimal bandwidth for ˆβt, obtained by minimizing the mean integrated squared error, is ν 0 R qtr{ψσ X h opt = } /5 σ 2 tdt β T tψβ n tf T tdt /5 n / µ Asymptotic properties of AQR estimator In this subsection, we state the asymptotic properties of ˆσt at 2.2; the statistical properties of 2.3 are similar. Theorem 2. Under Conditions 6, bias{ˆσt D} = 2 Ξβ tµ 2 h 2 c Ω + o p h 2 Var{ˆσt D} = ν 0 σ 2 t f T t R 2q R 3 qω + o p Furthermore, { ˆσt σt } 2 Ξβ tµ 2 h 2 d c Ω N 0, ν 0σ 2 t f T t R 2q R 3 qω. 3. Local Quadratic CQR-AQR Estimation 3.. Estimation In many situations we are interested in estimating the derivatives of the coefficient functions; local quadratic regression is often preferred, although we can get estimators by employing local linear regression. The estimation of β t can be improved minimizing k= ρ τk Y i a τk t Xi T βt+β tt i t+ 2 β tt i t 2 K with respect to a τk t, for k =,..., q, βt, β t and β t Asymptotic properties We state the properties of the improved estimate ˆβ t. Ti t h 3.

7 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 08 Theorem 3. Under Conditions 6, Furthermore, { 3 bias{ ˆβ t D} = 6 β th 2 µ 4 µ 2 + o p h 2, cov{ ˆβ t D} = ν 2 Ψ σ 2 t 3 µ 2 2 f T t ˆβ t β t 6 β th 2 µ } 4 µ 2 d N R q + o p 3. 0, ν 2Ψ σ 2 t µ 2 2 f R q T t. 4. Bandwidth Selection Bandwidth selection is an important issue in local smoothing problems and there are existing techniques, such as plug-in bandwidth selector Ruppert, Sheather, and Wand 995, and cross-validation Wu, Chiang, and Hoover 998. In practice, leave-one-out cross-validation is quite computationally expensive, although it is natural and data-driven. We employ a plug-in method to select the optimal bandwidth. To deal with 2.4, let Γ = β T tψβ tf T tdt, Γ 2 = σ 2 tdt. The estimator ˆβ t can be obtained by local cubic CQR fitting with an appropriate pilot bandwidth h, so a natural estimator of Γ is n grid ˆΓ = n grid ˆβ T t i Ψ ˆβ t i, 4. where {t i : i =,..., n grid } are grid points in the support of T. The estimators â τk t, for k =,..., q, can be obtained as byproducts when we use local cubic CQR fitting with a pilot bandwidth h to estimate ˆβ t. Then employing 2.2 or 2.3, we can obtain the estimator ˆσ 2 t. The natural estimator of Γ 2 is n grid ˆΓ 2 = n grid ˆσ 2 t i. 4.2 By replacing Γ and Γ 2 in 2.4, respectively, with 4. and 4.2, we have the selected optimal bandwidth for estimating βt. In the calculation of ˆσ 2 t and R q at 2.4, we take ε to be normal if its true distribution is unknown.

8 082 JIE GUO, MAOZAI TIAN AND KAI ZHU 5. Hypothesis Testing In model., it is often of practical interest to test whether coefficient functions are actually varying. We consider the testing problem H 0 : β p t = β p H : β p t β p, 5. where β p is an unknown constant. Cai, Fan, and Yao 2000 and Huang, Wu, and Zhou 2002 developed goodness-of-fit tests of 5. based on the comparison of the residual sum of squares under the null and alternative models. In this paper, we propose a goodness-of-fit test based on the comparison of the residual sums of quantiles RSQ from local linear CQR fits under both the null hypothesis and the alternative. RSQ is an analog of residual sum of squares, see below. Under the null hypothesis, model. can be written as p Y i = Xijβ T j T i + β p X ip + σt i ε i. 5.2 j= Following Fan and Zhang 2000, we propose a method to estimate β p in 5.2 under the null hypothesis. First, we ignore the fact that β p is a constant, and treat it as an unknown function β p t. Based on local linear CQR estimation, we obtain an estimator ˆβ p t. Each of { ˆβ p T i } is an estimator of the unknown parameter β p under the null hypothesis, and we average them to obtain the estimator ˆβ p = ˆβ p T i. n The RSQ under H 0 is RSQ 0 = The RSQ under H is k= RSQ = j= p ρ τk Y i Xij T ˆβ j T i ˆβ p X ip â τk T i. k= ρ τk Y i The goodness-of-fit test statistic is p j= Xij T ˆβ j T i â τk T i. Q n = RSQ 0 RSQ RSQ = RSQ 0 RSQ, 5.3 and we reject H 0 for large value of Q n. Let p ˆη i = Y i Xij T ˆβ j T i j=

9 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 083 and define p Yi = Xij T ˆβ j T i + ˆβ p X ip + ˆη i. j= We use a bootstrap procedure to evaluate the null distribution of Q n and the p-values of the test. Step. Resample n subjects with replacement from {Y i, X i, T i : i =,..., n} and repeat the sampling procedure J times. Step 2. From each bootstrap sample, calculate the test statistic Q n and the empirical distribution of Q n based on the J independent bootstrap samples. Step 3. Reject the null hypothesis H 0 at level α if the observed test statistic Q n is greater than the upper-α point of the empirical distribution of Q n. The p-value of the test is the relative frequency of the event {Q n Q n } in the J replications of the bootstrap sampling. For simplicity, we use the same bandwidth in calculating Q n as for Q n. 6. Simulation Study In this section, we illustrate the performance of the proposed local CQR and AQR estimators through simulation studies. The proposed estimators were found using the majorization-minimization MM algorithm of Hunter and Lange Examples The model, as in Fan and Zhang 2000 is Y = β T X + β 2 T X 2 + σt ε, 6. where X and X 2 are standard normal with correlation coefficient 2 /2, T follows a uniform distribution on [0, ]. Further, T, ε and X, X 2 are independent. The coefficient functions were β t = cos2πt, β 2 t = 4t t. To illustrate the robustness and efficiency of the proposed estimators, we considered five distributions of ε N0,, t-distribution with 3 degrees of freedom, Lognormal0,, mixed normal distribution 0.9N0, + 0.N0, 0 2 and Cauchy0, and three kinds of heteroscedasticity similar to the examples in Tian and Chan 200: Example. Homoscedasticity σt = 0.2var{EY U, X, X 2 } /2.

10 084 JIE GUO, MAOZAI TIAN AND KAI ZHU Example 2. Small jumps heteroscedasticity 0., 0 t 0.3, σt = 0.2, 0.3 < t 0.6, 0., 0.6 < t. Example 3. High frequency heteroscedasticity 50t 2 2, 0 t 0.3, σt = 3t, 0.3 < t 0.6, 8t, 0.6 < t. To assess the performance of local linear CQR estimator, we compared it with local linear least squares and local linear weighted least squares estimators solved, respectively, by minimizing { Y i 2 j= { σ 2 Y i T i X T ij 2 j= βj t + β jtt i t } 2 Ti t K, 6.2 h X T ij βj t + β jtt i t } 2 Ti t K. 6.3 h In 6.3, we used the true value of σt i as if the values of heteroscedasticity were known. We computed the ratio of averaged square errors RASEĝt j = ASEĝLS t j ASEĝ CQR t j, RASE2ĝt j = ASEĝW LS t j ASEĝ CQR t j, where ASEĝt j = n n grid j= ĝt j gt j 2, with g either β or β, and {t j, j =,..., n grid } the grid points at which the functions {ĝ } were evaluated. ĝ LS and ĝ W LS are the minimizers of 6.2 and 6.3, respectively. ĝ CQR is the local linear CQR estimator. To assess the performance of AQR estimator, we compared it with the triangular k-nn weight estimator see, Stone 997, Zhao 200 ˆσ i = w ij ˆε j, i =,..., n, j=

11 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 085 where w i,..., w in is the set of Stone s triangular k-nn weights corresponding to the ith observation, and ˆε j, for j =,..., n, are the residuals of local linear least squares fitting. To study how the performance of the proposed estimators varies with the number of quantiles q, we considered q = 5, 9, 9, respectively. The results are summarized in the following tables. CQR 5, CQR 9 and CQR 9 correspond to the local linear CQR estimates with q = 5, 9, 9, respectively. In the simulation, we set n grid = 200 and grid points evenly distributed over the interval for which β and β were estimated. We conducted 200 simulations for each case with sample size n = 200. The kernel used is Epanechnikov kernel Ku = 3 4 u2 I u. The bandwidths were selected using the plug-in method of Section Simulation results and findings From Tables, 2 and 3, we can see that, for all types of heteroscedasticity with ε normal distributed, the values of RASE and RASE2 are slightly less than ; it is clear that local linear weighted least squares is the best estimation method. For all non-normal errors, the values of RASE and RASE2 are greater than, indicating the CQR estimator s huge gain in efficiency, especially for the Cauchy distribution. When error is homoscedastic, as in Table, CQR 9 and CQR 9 perform better than CQR 5, but when error is heteroscedastic, as in Tables 2 and 3, CQR 9 performs better than CQR 5 and CQR 9. When error is heteroscedastic, combining the strength of relatively more quantile regressions at different quantile positions describe the data structures more thoroughly. However, the CQR estimator is not very sensitive to the number of quantiles, q, and a moderate q, such as q = 9, is enough. Figure is the plot of estimated coefficient functions when ε is Cauchy0, and q = 9 in Example. We can see that the local CQR estimator performs much better than local LS estimator; The least squares method fails when the variance of ε is infinite, while CQR method does not. Figure 2 is the plot of estimated conditional quantile curves of σtε at at quantile positions τ = 0., 0.3, 0.5, 0.7, 0.9 of Examples, 2, and 3, with ε Lognormal0, and q = 9. From the left panel of Figure 2, we can see that the five estimated conditional quantile curves are parallel and horizontal, indicating that the model is homoscedastic, which is consistent with σt in Example being constant. From the right panel of Figure 2, we can see that the five estimated quantile curves are not parallel or horizontal. Moreover, with the increase of t, the divergence of the quantile curves becomes more pronounced, indicating that the model is heteroscedastic and the heteroscedasticity increases sharply with

12 086 JIE GUO, MAOZAI TIAN AND KAI ZHU Table. Comparisons between local linear least squares LS and the local CQR based on the RASE criterion for Example Homoscedasticity. ˆβ t ˆβ t ˆβ2 t ˆβ 2 t ε method RASE RASE RASE RASE CQR N0, CQR CQR CQR t3 CQR CQR CQR lognormal CQR CQR CQR mixnormal CQR CQR CQR Cauchy CQR CQR Table 2. Comparisons between local linear LS, local linear weighted LS, and the local CQR based on the RASE and RASE2 criteria for Example 2 Small jumps Heteroscedasticity. ˆβ t ˆβ t ˆβ2 t ˆβ 2 t ε method RASE RASE2 RASE RASE2 RASE RASE2 RASE RASE2 CQR N0, CQR CQR CQR t3 CQR CQR CQR lognormal CQR CQR CQR mixnormal CQR CQR CQR Cauchy CQR CQR the increase of t if t is bigger than 0.6, also consistent with the variation of σt in Example 3.

13 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 087 Table 3. Comparisons between local linear LS, local weighted LS, and the local CQR based on the RASE and RASE2 criteria for Example 3 High frequency Heteroscedasticity. ˆβ t ˆβ t ˆβ2 t ˆβ 2 t ε method RASE RASE2 RASE RASE2 RASE RASE2 RASE RASE2 CQR N0, CQR CQR CQR t3 CQR CQR CQR lognormal CQR CQR CQR mixnormal CQR CQR CQR Cauchy CQR CQR Figure 3 is the plot of estimated σt, with ε Lognormal0, and q = 9. From Figure 3, we can see that the AQR estimator performs much better than the triangular k-nn weight estimator. In the simulation study, we can conclude that CQR-AQR estimation is an efficient and robust method to simultaneously estimate the coefficient functions and the heteroscedasticity Hypothesis testing To demonstrate the power of the goodness-of-fit test in Section 5, we considered the null hypothesis that β 2 t in model 6. is constant versus the alternative that it is varying. We considered the case that σt in 6. is the same as that of Example and ε is normal. The power was evaluated under a sequence of the alternative models indexed by λ: β 2 t; λ = c + λ{β 2 t c} 0 λ, where c = 0 β 2tdt. For each λ in {0, 0., 0.2,...,.0}, we applied the goodnessof-fit test with 00 replications of sample size n = 200. For each replication, we repeated the bootstrap resampling 00 times. The significance level α was Figure 4 shows the simulated power against λ, indicating the hypothesis testing procedure worked appropriately.

14 088 JIE GUO, MAOZAI TIAN AND KAI ZHU Figure. Plots of ˆβ t and ˆβ 2 t in Example, where ε is Cauchy. Dotdashed curves: true functions; Long-dashed curves: local linear CQR estimated functions; Dot curves: local linear LS estimated functions. Figure 2. Plots of five estimated conditional quantile curves of σtε of Examples, 2, and 3 from the left panel to the right, with ε Lognormal0,. 7. Empirical Application We illustrate the methodology of this paper via an application to an air pollution data set. The data were a subsample of 500 observations from a study conducted by the Norwegian Public Roads Administration, investigating how the air pollution at a road relates to the traffic volume and wind speed. The data was obtained from StatLib. It is of interest to study the association between the levels of pollutants and the traffic volume and wind speed, and to examine the extent to which the association varies over time. We consider the relation among the hourly values of the logarithm of the concentration of NO 2 Y, measured at Alnabru in Oslo, Norway, between October 200 and August 2003, and the logarithm of the number of cars per hour X, wind speed X 2 meters per

15 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 089 Figure 3. Plots of ˆσt in Examples, 2, and 3 from the left panel to the right, with ε Lognormal0,. Dot-dashed curves: true functions; Longdashed curves: AQR estimated functions; Dot curves: triangular k-nn weight estimated functions. Figure 4. Plot of power against λ for the goodness-of-fit test. second, and the hour of day T. All covariates except T are standardized. We considered the following varying-coefficient model with heteroscedasticity Y = β T X + β 2 T X 2 + σt ε. 7. to fit the given data. The proposed CQR-AQR estimation was employed to estimate β t, β 2 t, and σt. In our applications, we chose q = 9. The Epanechnikov kernel was employed and the plug-in selected bandwidth was The estimated coefficient functions are depicted in Figure 5. The figure shows that there is a strong time effect on the coefficient functions β and β 2. Moreover, the association between the number of cars per hour and the concentration of NO 2 is always positive, and the association between the wind speed and the concentration of NO 2 is always negative. When employing the AQR estimator to obtain ˆσt, we took ε to be normal; ˆσt is presented in Figure 6 left panel. We can see that the values of ˆσt

16 090 JIE GUO, MAOZAI TIAN AND KAI ZHU Figure 5. The estimated coefficient functions ˆβ t and ˆβ 2 t of model 7.. Figure 6. The plot of ˆσt in model 7. left panel; The plot of fitted y to corresponding residuals of model 7. right panel. varies with time. Moreover, the plot of the fitted values of Y that is, Ŷ = ˆβ T X + ˆβ 2 T X 2 to the residuals that is, Y Ŷ, presented in Figure 6 right panel, also demonstrates that there is heteroscedasticity in model 7.. A natural question is whether the coefficient functions are really time varying. We used the hypothesis testing procedure in Section 5 to answer this question. The number of bootstrap replicates was J = 500. The observed statistics and their p-values are summarized in Table 4. From Table 4, at the 0.05 significance level, there is sufficient evidence to reject H 0 and H 02.

17 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 09 Table 4. Test statistics and p-values for testing whether coefficient functions are actually time-varying. Null hypothesis Values of test statistic p-value H 0 : β = β H 02 : β 2 = β Local m-polynomial CQR-AQR Estimation As a generalization of local linear and quadratic CQR-AQR estimation, we consider local m-polynomial CQR-AQR and establish the asymptotic theory for local m-polynomial CQR-AQR estimators in model.. For each given t, we can get â τk t, ˆb j for k =,..., q, j = 0,..., m, by minimizing k= ρ τk Y i a τk t m Xi T b j T i t j Ti t K, 8. h j=0 where b j = β j t/j!. Let u k = a τk t σtc τk, v j = h j j!b j β j t/j!, and θ = u, u 2,..., u q ; v 0, v,..., v m T. Define S S S = 2, S 2 S 22 where S is a q q diagonal matrix with diagonal elements fc τk, k =, 2,..., q, S 2 is a q m + matrix with k, j-element fc τk EX T T = tµ j, k =, 2,..., q and j = 0,,..., m, S 2 = S2 T, and S 22 is a m+ m+ matrix with j, j -element EXX T T = tµ q j+j k= fc τ k, j, j = 0,,..., m. Partition S into submatrices as follows: S S S = 2 S = S 2 S 2 S 22 S 2 S 22 where here and hereafter denotes the top left-hand q q submatrix and 22 denotes the bottom right-hand m + m + submatrix. Define Σ Σ Σ = 2, Σ 2 Σ 22 where Σ is a q q matrix with k, k -element ν 0 τ kk, k, k =, 2,..., q, Σ 2 is a q m+ matrix with k, j-element EX T T = tν q j k = τ kk, k =, 2,..., q, j = 0,,..., m, Σ 2 = Σ T 2, Σ 22 is a m + m + matrix with j, j -element EXX T T = tν q j+j k,k = τ kk, j, j = 0,,..., m. Let d i,k = c τk {σt i σt} + r i,m, r i,m = Xi T βt i m j=0 βj tt i t j /j!, s i = T i t/h and KT i t/h = K i, and take ηi,k to be Iε i c τk

18 092 JIE GUO, MAOZAI TIAN AND KAI ZHU d i,k /σt i τ k. Let Wn = w, w 2,..., w q ; w 20, w 2,..., w 2m T, where wk = n K iηi,k, w 2j = q n k= XT i K is j i η i,k. Theorem 4. Under Conditions 6, for k =,..., q, j = 0,..., m, ˆθ + σt f T t S EWn D d N 0, σ2 t f T t S ΣS Discussion In this section, we discuss some directions to further extend this work. a The proposed CQR-AQR method is not very sensitive to the number of quantiles q but, practically speaking, we can choose q by some tuning methods such as K-fold cross-validation. To circumvent the problem of selecting q, we can consider, instead of 2., 0 ρ γ Yi a γ t Xi T βt + β tt i t Ti t K ωγdγ, 9. h where the weight function ωγ is a density function over 0,. In practice, we need to discretize the integral in order to numerically compute the estimators. Thus, a discrete distribution density ωγ is used to construct the weights. The proposed method uses a discrete uniform distribution on {/q +,..., q/q + }. b The model. can be extended to Y i = X T i βt i + σx i, T i ε i, where σx i, T i is an unknown form positive function. We can estimate βt and σx, t by minimizing ρ τk Yi a τk x, t Xi T k= βt + β tt i t K i,h, 9.2 where x = x,..., x p, a τk x, t = σx, tc τk, K i,h = KT i t/h 0, X i x /h,..., X ip x p /h p is a multivariate kernel function, and H = h 0, h,..., h p T is a vector of bandwidths. Denote the minimizers of 9.2 by ˆβt and â τk x, t, for k =,..., q. If the AQR estimator is employed, we can easily get ˆσx, t = /q q k= âτ k x, t/c τk, if c τk 0 for any k =,..., q; or ˆσx, t = /q q k= k j â τk x, t/c τk, if c τj = 0 for certain j {,..., q}. Note that estimating σx, t nonparametrically encounters the curse of dimensionality and how to estimate βt is an open problem.

19 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 093 Finally, we would like to point out that the proposed method is efficiently implemented by using the MM algorithm. For the simulation example in the case that q = 9 and the sample size is n = 5, 000, the proposed method fit at a given location t was computed within s on an Intel 2.8-GHz machine. Acknowledgement The authors gratefully acknowledge the helpful comments of an associate editor and the referees. The research is partially supported by the Major Project of Humanities Social Science Foundation of Ministry of Education No. 08JJD90247, Key Project of Chinese Ministry of Education No. 0820, National Natural Science Foundation of China No , Beijing Natural Science Foundation No. 0202, the Fundamental Research Funds for the Central Universities, and the Research Funds of Renmin University of China No. 0XNL08. Appendix Proof of Theorem 4. Let i,k = u k + m j=0 XT i v js j i. Because Y i a τk t Xi T m j=0 b jt i t j = σt i ε i c τk + d i,k i,k, ˆθ = û, û 2,..., û q ; ˆv 0, ˆv,..., ˆv m T is the minimizer of L n = k= Applying the identity we write L n as L n = [ ] ρ τk σt i ε i c τk + d i,k i,k ρ τk σt i ε i c τk + d i,k K i. ρ τ r s ρ τ r = sir 0 τ + k= + k= s 0 [Ir z Ir 0]dz, u k + m j=0 XT i v js j i Iε i c τk d i,k σt i τ k K i i,k K i 0 IσT i ε i c τk + d i,k z IσT i ε i c τk + d i,k 0 dz

20 094 JIE GUO, MAOZAI TIAN AND KAI ZHU = { k= + k= K i η i,k }u k + i,k K i 0 m { q j=0 k= IσT i ε i c τk + d i,k 0 dz = Wn T θ + B n,k θ, k= where B n,k θ = n d i,k 0 dz. Take K i i,k 0 X T i K is j i η i,k }v j IσT i ε i c τk + d i,k z c τk + d i,k z IσT i ε i c τk + d i,k z IσT i ε i c τk + Sn, S S n = n,2 S n,2 S n,22 where S n, is a q q diagonal matrix with diagonal elements /fc τk n K i /σt i, k =, 2,..., q, S n,2 is a q m + matrix with k, j-elements /fc τk n XT i K is j i /σt i, j = 0,,..., m, S n,2 = Sn,2 T, S n,22 is a m + m + matrix with j, j -elements / q k= fc τ k n X ixi T K is j+j i /σt i, j, j = 0,,..., m. [ i,k E[B n,k θ D] = E K i 0, {Iε i c τk + z d i,k σt i Iε i c τk d ] i,k σt i }dz D i,k = K i {F c τk d i,k 0 σt i + z σt i F c τ k d i,k σt i }dz i,k = K i { z 0 σt i fc τ k d i,k σt i + oz}dz = K i 2 fc τk d i,k /σt i i,k + o p 2σT i = K i 2 fc τk i,k 2σT i + o p. As well, [ i,k Var[B n,k θ D] = Var K i {IσT i ε i c τk + d i,k z 0

21 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 095 ] IσT i ε i c τk + d i,k 0}dz D i,k E [K i {IσT i ε i c τk + d i,k z 0 2 ] IσT i ε i c τk + d i,k 0}dz D i,k Ki 2 i,k {F c τk d i,k σt i + i,k σt i 0 F c τk d i,k σt i }dz dz 2 o Ki 2 2 i,k. = o p. Hence q k= B n,kθ p q n k= K i 2 i,k fc τ k /2σT i, and we can write the limit, q n k= K i 2 i,k fc τ k /2σT i, into the matrix 2 θt S n θ. That is, B n,k θ p 2 θt S n θ. k= As in Parzen 962, we have p K i ft t, X i K i s j i X i Xi T K i s j i 0 p f T tex T = tµ j, p f T texx T T = tµ j, where p stands for convergence in probability. Thus, This leads to S n p f T t σt S. L n θ p f T t 2 σt θt Sθ + Wn T θ. Since L n is a convex function, following Knight 998 we have ˆθ p σt f T t S W n.

22 096 JIE GUO, MAOZAI TIAN AND KAI ZHU Let η i,k = Iε i c τk τ k, and W n = w,..., w q, w 20,..., w 2m T with w k = K i η i,k and w 2j = k= Xi T K i s j i η i,k. By the Cramer-Wald Theorem and the Central Limit Theorem, we have W n D E[W n D] covwn D d N0, I m+q+ m+q+. Note that covη i,k, η i,k = τ kk and covη i,k, η j,k = 0, if i j. As in Parzen 962, we have Ki 2 Ki 2 s j i X i Ki 2 s j+j i X i Xi T Therefore, covw n D p f T tσ and Moreover, Varwk w k D = Var = p f T tν 0, p f T tν j EX T = t, p f T tν j+j EXX T T = t. W n D d N0, f T tσ. Ki 2 Var K i ηi,k η i,k D Iε i c τk d i,k σt i Iε i c τk D Ki 2 E [Iε i c τk d i,k σt i Iε i c τk ] D Ki 2 [F c τk + d i,k σt i F c τ k ] = o p, Varw2j w 2j D = Var k= Xi T K i s j i η i,k η i,k D

23 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 097 = q2 q2 q Xi T X i Ki 2 s 2j i Var ηi,k η i,k D Xi T X i Ki 2 s 2j i Iε i c τk ] D Xi T X i Ki 2 s 2j i = o p. max k k= E [Iε i c τk d i,k σt i max [F c τ k + d i,k k σt i F c τ k ] Thus, we have W n D p W n D. By Slutsky s theorem, conditioning on D, we have Wn D E[W n D] d N0, f T tσ, and so ˆθ + σt f T t S EWn D d N 0, σ2 t f T t S ΣS. This completes the proof of Theorem 4. Proof of Theorem. The asymptotic normality follows Theorem 4 with m =. We calculate the conditional bias and variance of ˆβt in this section. When m =, S = diag{fc τ,..., fc τq }, S 2 = {fc τ EX T = t,..., fc τq EX T = t T, 0 q }, S 22 = diag{exx T T = t fc τk, EXX T T = tµ 2 fc τk }. Note that k= { VarX T = t S 22 = S 22 S 2 S S 2 = diag q k= fc τ k S 2 = S 22 S 2 S = Ew k D = Ew 2j D = K i F c τk d i,k σt i F c τ k, X T i K is j i k= k= { EX T = tvarx T = t q k= fc τ k F c τk d i,k σt i F c τ k, },, EXXT T = t µ q 2 k= fc τ k } T q, 0 q,

24 098 JIE GUO, MAOZAI TIAN AND KAI ZHU Write e =, 0 T. By Theorem 4, bias{ ˆβt D} = σt f T t et {S 2 E[W n D] + S 22 E[W 2n D]} = σt VarX T = t f T t q k= fc τ k K i k= It is easy to check that and therefore bias ˆβt D X i EX T = t F c τk d i,k σt i F c τ k. F c τk d i,k σt i F c τ k = r i, σt i fc τ k { + o p }, = σt VarX T = t f T t = σt 2 f T t X i EX T = tk i r i, σt i { + o p} β tt i t 2 K i { + o p } = σt i 2 β tµ 2 h 2 + o p h 2. Furthermore, the conditional covariance of ˆβt is cov ˆβt D = σ2 t f T t et S ΣS 22 e = σ2 t ν q q 0 k= k = τ kk f T t q k= fc τ k 2 VarX T = t + o p = ν 0VarX T = t σ 2 t R q + o p f T t. This completes the proof of Theorem. Proof of Theorem 2. Here we only calculate the conditional bias and variance of ˆσt as presented in 2.2. The proof of the asymptotic normality of 2.3 is similar. Let e k be a q vector, whose kth element is and all other elements are 0. According to Theorem 4, bias{â τk t D} = σt f T t et k { S EW n D + S 2 EW 2n D }

25 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 099 = σt [ f T t fc τk EXT T =tvarx T T =t q k= fc τ k K i F c τk d i,k σt i F c τ k X i K i k= F c τk d ] i,k σt i F c τ k = σt r i, K i EX T T = t VarX T = t X i { + o p } f T t σt i = σt Xi T K β tt i t 2 i f T t 2σT i EX T T = t VarX T = t X i { + o p } = 2 EXT T = tβ tµ 2 h 2 Ω + o p h 2. Because ˆσt = q q k= âτ k t/c τk, we can get bias{ˆσt D} = q k= c τk bias{â τk t D} = 2 EXT T = tβ tµ 2 h 2 q c τk k= = 2 EXT T = tβ tµ 2 h 2 c Ω + o p h 2. The conditional variance of ˆσt is Varˆσt D = q 2 k= k = = q 2 σ 2 t f T t = q 2 ν 0 σ 2 t f T t k= k = +o p = covâ τk t, â τk t c τk c τk S ΣS c k= k τk c kk τk = [ τ kk q 2 c τk c τk fc τk fc τk k= k = c τk c τk ν 0 σ 2 t f T t R 2q R 3 qω + o p Ω + o p h 2 q k= τ kk q k= fc τ k EXT T =t VarX T T =t EX T =t ].

26 00 JIE GUO, MAOZAI TIAN AND KAI ZHU This completes the proof of Theorem 2. Proof of Theorem 3. The asymptotic normality follows from Theorem 4 with m = 2, and the proof of Theorem 3 is very similar to that of Theorem. Let e 2 = 0,, 0 T. We calculate the conditional bias and variance of ˆβ t bias ˆβ σt t D = h f T t et 2 {S 2 EWn D + S 22 EW2n D} = σt 2 f T t EXX T T = tµ 2 = σt 2 f T t EXX T T = tµ 2 = 6 β th 2 µ 4 µ 2 + o p h 2. X i K i s i r i,2 σt i { + o p} Xi T X i K i s β ts 3 i h3 i {+o p } 6σT i Furthermore, we can easily get the conditional covariance of ˆβ t, cov ˆβ t D = σ2 t 3 f T t 0 et 2 S ΣS 22 e 2 = σ2 t ν 2 EXX T T = t 3 µ 2 2 f T t = σ2 t 3 ν 2 EXX T T = t µ 2 2 f T t 0 This completes the proof of Theorem 3. References q k = τ kk q k= fc τ k 2 + o p. q k= R q + o p 3 3 Cai, Z., Fan, J. and Yao, Q Functional-coefficient regression models for nonlinear time series. J. Amer. Statist. Assoc. 95, Chiang, C. T., Rice, J. A. and Wu, C. O Smoothing spline estimation for varying coefficient models with repeatedly measured dependent variables. J. Amer. Statist. Assoc. 96, Fan, J. and Zhang, W Statistical estimation in varying coefficient models. Ann. Statist. 27, Fan, J. and Zhang, W Simultaneous Confidence Bands and Hypothesis Testing in Varying-coefficient Models. Scand. J. Statist. 27, Fan, J. and Zhang, W Statistical methods with varying coefficient models. Statistics and Its Interface, Hastie, T. J. and Tibshirani, R. J Varying-coefficient Models. J. Roy. Statist. Soc. Ser. B 55, Hunter, D. R. and Lange, K Quantile regression via an MM algorithm. J. Comput. Graph. Statist. 9,

27 NEW EFFICIENT AND ROBUST ESTIMATION IN VARYING-COEFFICIENT MODELS 0 Huang, J. Z., Wu, C. O. and Zhou, L Varying-coefficient models and basis function approximations for the analysis of repeated measurements. Biometrika 89, -28. Koenker, R. and Bassett, G Regression quantiles. Econometrica 46, Knight, K Limiting distribution for regression estimators under general conditions. Ann. Statist. 26, Kim, M. O Quantile regression with varying coefficients. Ann. Statist. 35, Kai, B., Li, R. and Zou, H Local composite quantile regression smoothing: an efficient and safe alternative to local polynomial regression. J. Roy. Statist. Soc. Ser. B 72, Parzen, E On estimation of a probability density function and model. Ann. Math. Statist. 33, Ruppert, D., Sheather, S. J. and Wand, M. P.995. An effective bandwidth selector for local least least squares regression. J. Amer. Statist. Assoc. 90, Stone, C. J Consistent nonparametric regression. Ann. Statist. 5, Tian, M. Z. and Chan, N. H Saddle point approximation and volatility estimation of value-at-risk. Statist. Sinica 20. Wu, C. O., Chiang, C. T. and Hoover, D. R Asymptotic confidence regions for kernel smoothing of a varying-coefficient model with longitudinal data. J. Amer. Statist. Assoc. 93, Wang, H. J., Zhu, Z. and Zhou, J Quantile regression in partially linear varyingcoefficient models. Ann. Statist. 37, Zhao, Q Asymptotically efficient median regression in the presence of heteroscedasticity of unknown form. Econom. Theory 7, Zou, H. and Yuan, M Composite quantile regression and the oracle model selection theory. Ann. Statist. 36, Center for Applied Statistics, School of Statistics, Renmin University of China, Beijing, 00872, China. guojieruc2009@ruc.edu.cn Center for Applied Statistics, School of Statistics, Renmin University of China, Beijing, 00872, China. mztian@ruc.edu.cn National Astronomical Observatories, Chinese Academy of Sciences, Beijing China. zhukai@bao.ac.cn Received October 200; accepted August 20