Local linear multivariate. regression with variable. bandwidth in the presence of. heteroscedasticity

Transcription

1 Model ISSN X Department of Econometrics and Business Statistics Local linear multivariate regression with variable bandwidth in the presence of heteroscedasticity AzhongYe, RobJHyndman and ZinaiLi May 2006 Working Paper 8/06

2 Local linear multivariate regression with variable bandwidth in the presence of heteroscedasticity Azhong Ye College of Management Fuzhou University, Fuzhou, China. Rob J Hyndman Department of Econometrics and Business Statistics, Monash University, VIC 3800 Australia. Rob.Hyndman@buseco.monash.edu Zinai Li School of Economics and Management, Tsinghua University, Beijing, , China. lizinai@mail.tsinghua.edu.cn 18 May2006 JEL classification: C12,C15,C52

3 Local linear multivariate regression with variable bandwidth in the presence of heteroscedasticity Abstract: We present a local linear estimator with variable bandwidth for multivariate nonparametric regression. We prove its consistency and asymptotic normality in the interior of the observed data and obtain its rates of convergence. This result is used to obtain practical direct plug-in bandwidth selectors for heteroscedastic regression in one and two dimensions. We show that the local linear estimator with variable bandwidth has better goodness-of-fit properties than the local linear estimator with constant bandwidth, in the presence of heteroscedasticity. Keywords: heteroscedasticity; kernel smoothing; local linear regression; plug-in bandwidth, variable bandwidth. 1

4 1 Introduction We are interested in the problem of heteroscedasticity in nonparametric regression, especially when applied to economic data. Heteroscedasticity is very common in economics, and heteroscedasticity in linear regression is covered in almost every econometrics textbook. Applications of nonparametric regression in economics are growing, and Yatchew(1998) argued that it will become an indispensable tool for every economist because it typically assumes little about the shape of the regression function. Consequently, we believe that heteroscedasticity in nonparametric regression is an important problem that has received limited attention to date. We seek to develop new estimators that have better goodness of fit than the common estimators in nonparametric econometric models. In particular, we are interested in using heteroscedasticity to improve nonparametric regression estimation. There have been a few papers on related topics. Testing for heteroscedasticity in nonparametric regression has been discussed by Eubank and Thomas (1993) and Dette and Munk (1998). Ruppert and Wand (1994) discussed multivariate locally weighted least squares regression when the variances of the disturbances are not constant. Ruppert et al. (1997) presented the local polynomial estimator of the conditional variance function in a heteroscedastic, nonparametric regression model using linear smoothing of squared residuals. Sharp-optimal and adaptive estimators for heteroscedastic nonparametric regression using the classical trigonometric Fourier basis are given by Efromovich and Pinsker (1996). Our approach is to exploit the heteroscedasticity by using variable bandwidths in local linear regression. Müller and Stadtmüller (1987) discussed variable bandwidth kernel estimators of regression curves. Fan and Gijbels(1992, 1995, 1996) discussed the local linear estimator with variable bandwidth for nonparametric regression models with a single covariate. In this paper, we extend these papers by presenting a local linear estimator with variable bandwidth for nonparametric multiple regression models. We demonstrate that the local linear estimator has optimal conditional mean squared error when its variable bandwidth is a function of the density of the explanatory variables and conditional variances. Numerical simulation shows that the local linear estimator with this variable 2

5 bandwidth has better goodness of fit than the local linear estimator with constant bandwidth for the heteroscedastic models. 2 Local linear regression with a variable bandwidth Suppose we have a univariate response variable Y and a d-dimensional set of covariates X, andweobservetherandomvectors(x 1, Y 1 ),...,(X n, Y n )whichareindependentandidentically distributed. Itisassumedthateachvariablein X hasbeenscaledsotheyhavesimilarmeasures of spread. Our aimistoestimatetheregression function m(x)=e(y X= x). Wecanregardthe dataas being generated from the model Y= m(x)+u, where E(u X)=0,Var(u X= x)=σ 2 (x) and the marginaldensity of X is denoted by f(x). We assume the second-order derivatives of m(x) are continuous, f(x) is bounded above 0 and σ 2 (x) is continuous and bounded. Let K bead-variatekernelwhichissymmetric,nonnegative,compactlysupported, K(u) du= 1 and uu T K(u) du=µ 2 (K)I whereµ 2 (K) 0 and I is the d d identity matrix. In addition, l all odd-order moments of K vanish, that is, u ul d K(u) du=0 for all nonnegative integers d l 1,..., l d such that their sum is odd. Let K h (u)= K(u/h). Then the local linear estimator of m(x) with variable bandwidth is ˆm n (x,h n,α)=e T 1 (X T xw x,α X x ) 1 X T xw x,α Y, (1) where h n = cn 1/(d+4), c is a constant that depends only on K,α(x) is the variable bandwidth function, e T 1 =(1,0,...,0), X x=(x x,1,...,x x,n ) T, X x,i =(1,(X i x)) T and W x,α = diag Khn α(x 1 )(X 1 x),..., K hn α(x n )(X n x). We assumeα(x) is continuously differentiable. We now state our main result. A proof is given in the Appendix. 3

6 Theorem 1 Let x be a fixed point in the interior of{x f(x)>0}, H m (x)= let s(h m (x)) be the sum of the elements of H m (x). Then 2 m(x) and x i x j d d 1 E[ˆm n (x,h n,α) X 1,...,X n ] m(x)=0.5h 2 nµ 2 (K)α 2 (x)s(h m (x))+ o p (h 2 n); 2 Var[ˆm n (x,h n,α) X 1,...,X n ]=n 1 h d n R(K)α d (x)σ 2 (x)f 1 (x)+ o p (n 1 h d n ), where R(K)= K 2 (u)du; and 3 n 2/(d+4) [ˆm n (x,h n,α) m(x)] d N 0.5c 2 µ 2 (K)α 2 (x)s(h m (x)), c d R(K)α d (x)σ 2 (x)f 1 (x). Whenα(x)=1, results 1 and 2 coincide with Theorem 2.1 of Ruppert and Wand (1994). By result 2 and the law of large numbers, we find thatˆm n (x,h n,α) is consistent. From result 3 we know that the rate of convergence of ˆm n (x,h n,α) in interior points is O(n 2/(d+4) ) which, according to Stone(1980, 1982), is the optimal rate of convergence for nonparametric estimation of a smooth function m(x). 3 Using heteroscedasticity to improve local linear regression Although Fan and Gijbels (1992) and Ruppert and Wand (1994) discuss the local linear estimator of m(x), nobody has previously developed an improved estimator using the information of heteroscedasticity. We now show how this can be achieved. UsingTheorem 1,wecangiveanexpressionfortheconditionalmeansquarederrorofthelocal linear estimator with variable bandwidth. MSE=E [ˆmn (x,h n,α) m(x)] 2 X 1,...,X n = 1 4 [h nα(x)] 4 µ 2 2 (K)s2 (H m (x))+ R(K)σ2 (x) n[h n α(x)] d f(x) + o p(h 2 n)+ o p (n 1 h d n ). Minimizing MSE, ignoring the higher order terms, we obtain the optimal variable bandwidth α opt (x)= c 1 dr(k)σ 2 (x) µ 2 2 (K)f(x)s2 (H m (x)) 1/(d+4). 4

7 Note that the constant c will cancel out in the bandwidth expression h n α opt (x). Therefore, without loss of generality, we can set c= dr(k)µ 2 2 (K) 1/(d+4) which simplifies the above expression to σ 2 1/(d+4) (x) α opt (x)= f(x)s 2. (H m (x)) Toapplythisnew estimator,weneed toreplace f(x), H m (x)andσ 2 (x)withestimators. There are several potential ways to do this, depending on the dimension d. Some proposals for d= 1 and d= 2, are outlined below. 3.1 Univariate regression When d= 1, we first use the direct plug-in methodology of Sheather and Jones (1991) to select the bandwidth of a kernel density estimate for f(x). Second, we estimateσ 2 (x)=e(u 2 X= x) using local linear regression with the model û 2 i=σ 2 (X i )+ v i where û i = Y i ˆm n (X i,ĥn,1), v i are iid with zero mean and ĥn is chosen by the direct plug-in methodology of Ruppert et al. (1995). Third, we estimate m(x) by fitting the quartic m(x)=α 1 +α 2 x+α 3 x 2 +α 4 x 3 +α 5 x 4, usingordinaryleastsquaresregressionandsoobtaintheestimateˆ m(x)=2ˆα 3 +6ˆα 4 x+12ˆα 5 x 2. Then, our direct plug-in bandwidth for univariate regression (d = 1) is ĥ(x)= ˆσ 2 (x) 2n πˆf(x)ˆ m 2 (x) 1/ Bivariate regression When d= 2, we use a bivariate kernel density estimator (Scott, 1992) of f(x), with the direct plug-in methodology of Wand and Jones (1995) for the bandwidth. To estimateσ 2 (x), we first calculate û i = Y i Ŷ i, where Ŷ i = ˆm n (X i,ĥn,1), ĥn = min(ĥ1,ĥ2), and ĥ1 and ĥ2 are chosen by the direct plug-in methodology of Ruppert et al. (1995) for Y = m 1 (X 1 )+u 1 and Y= m 2 (X 2 )+u 2 respectively. Then we estimateσ 2 (x 1 ) using local linear regression with the model û 2 i=σ 2 (X 1i )+v i,where v i areiidwithzeromean. Again,thedirectplug-inmethodology of Ruppert et al. (1995) is used for bandwidth selection. 5

8 To estimate the second derivative of m(x 1, x 2 ), we fit the model m(x 1, x 2 )=α 1 +α 2 x 1 +α 3 x 2 +α 4 x 2 1 +α 5x 1 x 2 +α 6 x 2 2 +α 7x 3 1 +α 8x 2 1 x 2+α 9 x 1 x 2 2 +α 10 x 3 2 +α 11x 4 1 +α 12 x 3 1 x 2+α 13 x 2 1 x2 2 +α 14x 1 x 3 2 +α 15x 4 2, using ordinary least squares, and so obtain estimates of α. Hence, estimators for the second derivatives of m(x 1, x 2 ) are obtained using 2 m(x) x 2 = 2α 4 + 6α 7 x 1 + 2α 8 x α 11 x α 12x 1 x 2 + 2α 13 x 2 2, 1 2 m(x) =α 5 + 2α 8 x 1 + 2α 9 x 2 + 3α 12 x 2 1 x 1 x + 4α 13x 1 x 2 + 3α 14 x and 2 m(x) x 2 = 2α 6 + 2α 9 x 1 + 6α 10 x 2 + 2α 13 x α 14x 1 x α 15 x Then, our direct plug-in bandwidth for bivariate regression (d = 2) is ĥ(x)= ˆσ 2 (x) nπˆf(x)s 2 (Ĥ m (x)) 1/6. 4 Numerical studies with univariate regression This section examines the performance of the proposed variable bandwidth selection method via several data sets of univariate regression, generated from known functions. For comparison, we also compare the performance of the constant bandwidth method, based on the direct plug-in methodology described by Ruppert et al. (1995). As the true regression function is known in each case, the performances of the bandwidth methods are measured and compared using the root mean squared error, 1/2 RMSE= n 1 [ˆm n (X i,h n,α) m(x i )] 2. Wesimulatedatafromthefollowingfivemodels,eachwith Y= m(x)+σ(x)uwhere u N(0,1) 6

9 and the covariate X has a Uniform( 2, 2) distribution. Model A: m A (x)= x 2 + x σ 2 A (x)=32x Model B: m B (x)=(1+ x) sin(1.5x) σ 2 B (x)=3.2x Model C: m C (x)= x+ 2 exp( 2x 2 ) σ 2 C (x)=16(x2 0.01)I (x 2 >0.01) Model D: m D (x)=sin(2x)+2 exp( 2x 2 ) σ 2 D (x)=16(x2 0.01)I (x 2 >0.01) Model E: m E (x)=exp( (x+ 1) 2 )+2 exp( 2x 2 ) σ 2 E (x)=32(x2 0.01)I (x 2 >0.01) We draw 1000 random samples of size 200 from each model. Table 1 presents a summary of the results and shows that the variable bandwidth method has smaller RMSE than the constant bandwidth method in each case. For each model, the variable bandwidth method has smaller RMSE than the constant bandwidth method, and is better for more than 50% of samples. We plot the true regression functions (the solid line) and four typical estimated curves in Figure 1. These correspond to the 10th, 30th, 70th and 90th percentiles. For each percentile, the variable bandwidth method (dotted line) is closer to the true regression function than the constant bandwidth method (dashed line). Therefore, we conclude that for heteroscedastic models, the local linear estimator with variable bandwidth has better goodness-of-fit than the local linear estimator with constant bandwidth. 7

10 5 Numerical studies with bivariate regression We now examine the performance of the proposed variable bandwidth selection method via several data sets of bivariate regression, generated from known functions. For comparison, we also compare the performance of a constant bandwidth method given by ĥ= ˆσ 2 π n s2 (Ĥ m (X i )) 1/6 where ˆσ 2 = n 1 û 2 i, and u i and Ĥ m are the same as for the variable bandwidth selector. We simulate data from four models, each with Y= m(x 1, X 2 )+σ(x 1 )u where u N(0,1) and the covariates X 1 and X 2 are independent and have a Uniform( 2,2) distribution. Model F: m A (x 1, x 2 )= x 1 x 2 σ 2 A (x 1, x 2 )=(x )I (x1 2 >0.04) Model G: m B (x 1, x 2 )= x 1 exp( 2x 2 2 ) σ 2 B (x 1, x 2 )=2.5(x )I (x 2 1 >0.04) Model H: m C (x 1, x 2 )= x sin(1.5x 2 ) σ 2 C (x 1, x 2 )=(x )I (x 2 1 >0.04) Model I: m D (x 1, x 2 )=sin(x 1 + x 2 )+2 exp( 2x 2 2 ) σ 2 D (x 1, x 2 )=3(x )I (x1 2 >0.04) We draw 200 random samples of size 400 from each model. Table 2 presents a summary of the results and shows that the variable bandwidth method has smaller RMSE than the constant bandwidth method. For each model, the variable bandwidth method has lower RMSE than the constant bandwidth method and is better for more than 50% of samples. 8

11 We plot the true regression functions with one fixed variable (solid line) and four typical estimated curves in Figures 2 5. These correspond to the 10th, 30th, 70th and 90th percentiles. For each percentile, the variable bandwidth method(dotted line) is closer to the true regression function than the constant bandwidth method (dashed line). Therefore, we conclude that for heteroscedastic models, the local linear estimator with variable bandwidth has better goodnessof-fit than the local linear estimator with constant bandwidth. 6 Summary We have presented a local linear nonparametric estimator with variable bandwidth for multivariate regression models. We have shown that the estimator is consistent and asymptotically normal in the interior of the sample space. We have also shown that its convergence rate is optimal for nonparametric regression (Stone, 1980, 1982). By minimizing the conditional mean squared error of the estimator, we have derived the optimal variable bandwidth as a function of the density of the explanatory variables and the conditional variance. Wehavealsoprovidedaplug-inalgorithmfor computingtheestimatorwhen d= 1or d = 2. Numerical simulation shows that our local linear estimator with variable bandwidth has better goodness-of-fit than the local linear estimator with constant bandwidth for heteroscedastic models. Acknowledgements We thank Xibin Zhang for his helpful comments. 9

12 References Dette, H. and A. Munk (1998) Testing heteroscedasticity in nonparametric regression, Journal of the Royal Statistical Society, Series B, 60(4), Efromovich, S. and M. Pinsker (1996) Sharp-optimal and adaptive estimation for heteroscedastic nonparametric regression, Statistica Sinica, 6, Eubank, R. L. and W. Thomas (1993) Detecting heteroscedasticity in nonparametric regression, Journal of the Royal Statistical Society, Series B, 55(1), Fan, J. and I. Gijbels (1992) Variable bandwidth and local linear regression smoothers, The Annals of Statistics, 20(4), Fan, J. and I. Gijbels(1995) Data-driven bandwidth selection in local polynomial fitting: variable bandwidth and spatial adaptation, Journal of the Royal Statistical Society, Series B, 57(2), Fan, J. and I. Gijbels(1996) Local polynomial modelling and its applications, Chapman and Hall. Müller, H.-G. and U. Stadtmüller (1987) Variable bandwidth kernel estimators of regression curves, The Annals of Statistics, 15(1), Ruppert, D., S. J. Sheather and M. P. Wand(1995) An effective bandwidth selector for local least squares regression (Corr: 96V91 p1380), Journal of the American Statistical Association, 90, Ruppert, D. and M. P. Wand (1994) Multivariate locally weighted least squares regression, The Annals of Statistics, 22(3), Ruppert, D., M. P. Wand, U. Holst and O. Hössjer (1997) Local polynomial variance-function estimation, Technometrics, 39(3), Scott, D. W. (1992) Multivariate Density Estimation: Theory, Practice, and Visualization, John Wiley & Sons. Sheather, S. J. and M. C. Jones (1991) A reliable data-based bandwidth selection method for kernel density estimation, J.R. Statist. Soc. B, 53(3),

13 Stone, C. J. (1980) Optimal convergence rates for nonparametric estimators, The Annals of Statistics, 8, Stone, C. J. (1982) Optimal global rates of convergence for nonparametric regression, The Annals of Statistics, 10, Wand, M. P. and M. C. Jones (1995) Kernel Smoothing, Chapman and Hall, London. White, H. (1984) Asymptotic theory for econometricians, Academic Press. Yatchew, A. (1998) Nonparametric regression techniques in economics, Journal of Economic Literature, 34,

14 Appendix: Proof of Theorem 1 Before we state a lemma that will be used in the proof, note that n 1 XxW T x,α X x = n n 1 K h n α(x i )(X i x) n K h n α(x i )(X i x)(x i x) n K h n α(x i )(X i x)(x i x) T n K h n α(x i )(X i x)(x i x)(x i x) T and e T 1 (X xw T x,α X x ) 1 XxW T x,α X x m(x) =e T 1 m(x) = m(x), D m (x) D m (x) where D m (x)= m(x) x 1 T,..., m(x). Therefore, x d ˆm n (x,h n,α) m(x)=e T 1 (X T xw x,α X x ) 1 X T xw x,α [0.5Q m (x)+u], where Q m (x) = Q m,1 (x),...,q m,n (x) T, Qm,i (x) = (X i x) T H m (z i (x,x i ))(X i x), H m (x)= 2 m(x), z x i x i (x,x i ) x X i x and U=(u 1,...,u n ) j d d T. Wecandeducethat {z i (x,x i )} n are independent because{x i} n are independent. Now we state a lemma using the notation of (1) and Theorem 1. Lemma 1 Let G(α, f,x)= f(x) udα(x)uu T T D K (u)du+µ 2 (K) d f(x)d α (x)+α 1 (x)d f (x), supp(k) B(x,α)= µ 2 (K) 1 f(x) 2 α(x)g(α, f,x) T and 1 be a generic matrix having each entry equal to 1, the dimensions of which will be clear from the context. Then n 1 n 1 K hn α(x i )(X i x)= f(x)+ o p (1) (2) K hn α(x i )(X i x)(x i x)=h 2 nα 3 (x)g(α, f,x)+o p (h 2 n1) (3) 12

15 n 1 K hn α(x i )(X i x)(x i x)(x i x) T =µ 2 (K)h 2 nα 2 (x)f(x)i+ o p (h 2 n1) (4) (XxW T x,α X x ) 1 f(x) 1 + o p (1) B(x,α)+o p (1) = (5) B T (x,α)+ o p (1) µ 2 (K) 1 h 2 n α 2 (x)f(x) 1 I+ o p (h 2 n 1) n 1 XxW T x,α Q m (x)=h 2 f(x)µ 2 (K)α 2 (x)s(h m (x)) n + o p (h 2 n1) (6) 0 and (nh d n) 1/2 (n 1 XxW T d x,α u) N 0, R(K)α d (x)σ 2 (x)f(x) (7) 0 We only prove results (3), (6) and (7) as the other results can be proved similarly. Proof of (3) It is easy to show that n 1 K hn α(x i )(X i x)(x i x)=e Khn α(x 1 )(X 1 x)(x 1 x) + O p ( n 1 Ψ), (8) where Ψ= Var(K hn α(x 1 )(X 1 x)(x 11 x 1 )),...,Var(K hn α(x 1 )(X 1 x)(x d1 x d )) T. Because x is a fixed point in the interior of supp(f)={x f(x) 0}, we have supp(k) {z :(x+h n α(x)z) supp(f)}, provided the bandwidth h n is small enough. Due to the continuity of f, K andα, we have E Khn α(x 1 )(X 1 x)(x 1 x) = supp(f) h d n α d (x)k(h 1 n (α(x 1 )) 1 (X 1 x))(x 1 x)f(x 1 )dx 1 13

16 = (α(x+h n Q)) d K(Q(α(x+h n Q)) 1 )f(x+h n Q)h n QdQ Ω n = h 2 nα(x) 3 G(α, f,x)+o(h 2 n1), (9) whereω n ={Q : x+h n Q supp(f)}. It is easy to see Var Khn α(x 1 )(X 1 x)(x 1 x) = E Khnα(X1)(X 1 x)(x 1 x) K hnα(x1)(x 1 x)(x 1 x) T E Khn α(x 1 )(X 1 x)(x 1 x) E Khn α(x 1 )(X 1 x)(x 1 x) T. (10) Again by the continuity of f, K andα, we have E Khnα(X 1 )(X 1 x)(x 1 x) K hnα(x1 )(X 1 x)(x 1 x) T = E Khn α(x 1 )(X 1 x) 2 (X 1 x)(x 1 x) T = supp(f) = h d+2 n = h d+2 n h d n (α(x)) d K(h 1 n (α(x 1 )) 1 (X 1 x)) 2 (X1 x)(x 1 x) T f(x 1 )dx 1 (α(x+h n Q)) d K(Q(α(x+h n Q)) 1 ) 2 f(x+h n Q)QQ T dq Ω n (α(x)) d K(Q(α(x)) 1 ) 2 f(x)qq T dq+o(h d+2 n 1)=O(h d+2 n 1). (11) Ω n Therefore we have O p ( n 1 Ψ)= o p (h 2 n1). (12) Then (3) follows from (8) (12). Proof of (6) It is straightforward to show that n 1 X T xw x,α Q m (x) 14

17 n 1 n 1 n = K h n α(x i )(X i x)(x i x) T H m (z i (x,x i ))(X i x) n 1, n K h n α(x i )(X i x)(x i x) T H m (z i (x,x i ))(X i x)(x i x) K hn α(x i )(X i x)(x i x) T H m (z i (x,x i ))(X i x) and n 1 = h 2 n f(x)µ 2 (K)(α(x)) 2 s(h m (x))+ o p (h 2 n1), K hn α(x i )(X i x)(x i x) T H m (z i (x,x i ))(X i x)(x i x)=o p (h 3 n1). Therefore (6) holds. Proof of (7) It is obvious that E n 1 XxW T x,α u =E n 1 XxW T x,α u x = 0, and n 1 XxW T n 1 n x,α u= K h n α(x i )(X i x)u i n 1. n K h n α(x i )(X i x)(x i x)u i By the continuity of f, K,σ 2 andα, we have Var n 1 K hn α(x i )(X i x)u i =n 1 Var Khn α(x 1 )(X 1 x)u 1 = n 1 h d n (α(x)) d K(h 1 n (α(x 1 )) 1 (X 1 x)) 2 σ 2 (X 1 )f(x 1 )dx 1 supp(f) = n 1 h d n = n 1 h d n and Var n 1 ((α(x+h n Q)) d K(Q(α(x+h n Q)) 1 )) 2 σ 2 (x+h n Q)f(x+h n Q)dQ Ω n Ω n ((α(x)) d K(Q(α(x)) 1 )) 2 σ 2 (x)f(x)dq+ o(n 1 h d n ) = n 1 h d n R(K)(α(x)) d σ 2 (x)f(x)+ o(n 1 h d n ), Then (7) holds. K hn α(x i )(X i x)(x i x)u i = o(n 1 h d+2 n 1). 15

18 Proof of Theorem 1 By (5) and (6), we have E[ˆm n (x,h n,α) X 1,...,X n ] m(x)=0.5e T 1 (X T xw x,α X x ) 1 X T xw x,α Q m (x) = 0.5h 2 nµ 2 (K)(α(x)) 2 s(h m (x))+ o p (h 2 n). Therefore Theorem 1(1) holds. Let V =diag σ 2 (X 1 ),...,σ 2 (X n ). Then Var[ˆm n (x,h n,α) X 1,...,X n ]=e T 1 (X T xw x,α X x ) 1 X T xw x,α V W x,α X x (X T xw x,α X x ) 1 e 1, (13) and n 1 XxW T a 11 (x,h n,α) (a 21 (x,h n,α)) T x,α V W x,α X x =, a 21 (x,h n,α) a 22 (x,h n,α) where a 11 (x,h n,α)= n 1 (K hn α(x i )(X i x)) 2 σ 2 (X i ) a 21 (x,h n,α)= n 1 (K hn α(x i )(X i x)) 2 (X i x)σ 2 (X i ) and a 22 (x,h n,α)= n 1 (K hn α(x i )(X i x)) 2 (X i x)(x i x) T σ 2 (X i ). It is easy to prove that n 1 n 1 (K hn α(x i )(X i x)) 2 σ 2 (X i )=h d n R(K)(α(x)) d σ 2 (x)f(x)o p (h d n ), (K hn α(x i )(X i x)) 2 (X i x) T σ 2 (X i )=O p (h d+1 n 1) and n 1 (K hn α(x i )(X i x)) 2 (X i x)(x i x) T σ 2 (X i )=O p (h d+2 n 1). (14) By (13) (14) and (5), we have Var[ˆm n (x,h n,α) X 1,...,X n ]=n 1 h d n R(K)(α(x)) d σ 2 (x)f(x) 1 + o p (n 1 h d n ). 16

19 Therefore Theorem 1(2) holds. By (6) and (7) and the central limit theorem, we have n 2/(d+4) n 1 XxW T x,α [0.5Q m (x)+u] d N(0.5c2 µ 2 (K)(α(x)) 2 f(x)s(h m (x)), c d R(K)(α(x)) d σ 2 (x)f(x)). 0 Applying White (1984, Proposition 2.26) and (5), we can easily deduce Theorem 1(3). 17

20 Tables Table 1: The percentage of 1000 samples in which the variable bandwidth method is better than the constant bandwidth method, and the RMSE of the two methods. Model Percentage Root mean squared error better Constant bandwidth Variable bandwidth Model A Model B Model C Model D Model E Table 2: The percentage of 200 samples in which the variable bandwidth method is better than the constant bandwidth method, and the RMSE of the two methods. Model Percentage Root mean squared error better Constant bandwidth Variable bandwidth Model F Model G Model H Model I

21 replacements Figures Figure 1: Results for the simulated univariate regression data of models A E. The true regression functions (the solid line) and four typical estimated curves are presented. These correspond to the 10th, the 30th, the 70th, the 90th percentile. The dashed line is for the constant bandwidth method and the dotted line is for the variable bandwidth method. Model A Model B Model C x x x m(x) m(x) m(x) m(x) m(x) Model D Model E x x 19

22 Figure 2: Results for the simulated bivariate data of model F. The true regression functions (the solid line) and four typical estimated curves are presented. These correspond to the 10th, the 30th, the 70th, the 90th percentile. The dashed line is for the constant bandwidth method and the dotted line is for the variable bandwidth method. Model F: x 2 = 1 Model F: x 1 = 1 m(x 1,0) m(x 1,1) m(x 1, 1) x 1 Model F: x 2 = 0 x 1 Model F: x 2 = 1 m( 2, x 2 ) m(0, x 2 ) m(1, x 2 ) x 2 Model F: x 1 = 0 x 2 Model F: x 1 = 1 x 1 x 2 20

23 Figure 3: Results for the simulated bivariate data of model G. The true regression functions (the solid line) and four typical estimated curves are presented. These correspond to the 10th, the 30th, the 70th, the 90th percentile. The dashed line is for the constant bandwidth method and the dotted line is for the variable bandwidth method. Model G: x 2 = 1 Model G: x 1 = 1 m(x 1,0) m(x 1,1) m(x 1, 1) x 1 Model G: x 2 = 0 x 1 Model G: x 2 = 1 m( 2, x 2 ) m(0, x 2 ) m(1, x 2 ) x 2 Model G: x 1 = 0 x 2 Model G: x 1 = 1 x 1 x 2 21

24 Figure 4: Results for the simulated bivariate data of model H. The true regression functions (the solid line) and four typical estimated curves are presented. These correspond to the 10th, the 30th, the 70th, the 90th percentile. The dashed line is for the constant bandwidth method and the dotted line is for the variable bandwidth method. Model H: x 2 = 1 Model H: x 1 = 1 m(x 1,0) m(x 1,1) m(x 1, 1) x 1 Model H: x 2 = 0 x 1 Model H: x 2 = 1 m( 2, x 2 ) m(0, x 2 ) m(1, x 2 ) x 2 Model H: x 1 = 0 x 2 Model H: x 1 = 1 x 1 x 2 22

25 Figure 5: Results for the simulated bivariate data of model I. The true regression functions (the solid line) and four typical estimated curves are presented. These correspond to the 10th, the 30th, the 70th, the 90th percentile. The dashed line is for the constant bandwidth method and the dotted line is for the variable bandwidth method. Model I: x 2 = 1 Model I: x 1 = 1 m(x 1,0) m(x 1,1) m(x 1, 1) x 1 Model I: x 2 = 0 x 1 Model I: x 2 = 1 m( 2, x 2 ) m(0, x 2 ) m(1, x 2 ) x 2 Model I: x 1 = 0 x 2 Model I: x 1 = 1 x 1 x 2 23