Day 4A Nonparametrics

Transcription

1 Day 4A Nonparametrics A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Economics Norwegian School of Economics Advanced Microeconometrics Aug 28 - Sep 2, Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 -Advanced Sep 2, 2017 Microeconometri 1 / 32

2 1. ntroduction 1. ntroduction Nonparametric methods place few restrictions on the data generating process density estimation - use kernel density estimate regression curve estimation - use kernel-weighted local constant or local linear regression F but curse of dimensionality as # regressors increases Semiparametric regression places some structure e.g. E [yjx] = g(x 0 β) where g() is unspeci ed reduces nonparametric component to one dimension. Bootstrap most often used to get standard errors more re ned bootstraps can give better nite sample inference. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 -Advanced Sep 2, 2017 Microeconometri 2 / 32

3 1. ntroduction Summary 1 ntroduction 2 Nonparametric (kernel) density estimation 3 Nonparametric (kernel) regression 4 npregress command (Stata 15) 5 Semiparametric regression 6 Stata commands A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 -Advanced Sep 2, 2017 Microeconometri 3 / 32

4 2. Nonparametric (kernel) density estimation Summary 2. Nonparametric (kernel) density estimation Parametric density estimate assume a density and use estimated parameters of this density e.g. normal density estimate: assume y i N [µ, σ 2 ] and use N [ȳ, s 2 ]. Nonparametric density estimate: a histogram break data into bins and use relative frequency within each bin Problem: a histogram is a step function, even if data are continuous Smooth nonparametric density estimate: kernel density estimate. smooths a histogram in two ways: F F use overlapping bins so evaluate at many more points use bins of greater width with most weight at the middle of the bin. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 -Advanced Sep 2, 2017 Microeconometri 4 / 32

5 2. Nonparametric (kernel) density estimation Histogram Histogram estimate A histogram is a nonparametric estimate of the density of y break data into bins of width 2h form rectangles of area the relative frequency = freq/n the height is freq/2nh (check: area = (freq/2nh) 2h = freq/n). Use freq = N i=1 1(x 0 h < x i < x 0 + h) where indicator function 1(A) equals 1 if event A happens and equals 0 otherwise The histogram estimate of f (x 0 ), the density of x evaluated at x 0, is bf HST (x 0 ) = 1 2Nh N i=1 1(x 0 h < x i < x 0 + h) = 1 Nh N 1 i=1 2 1 x i x 0 h < 1. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 -Advanced Sep 2, 2017 Microeconometri 5 / 32

6 2. Nonparametric (kernel) density estimation Histogram example Data example: histogram of lnwage for N = 175 observations Varies with the bin width (or equivalently the number of bins) default is p Nfor N 861 and 10 ln(n)/ ln(10) for N > 861 here specify 30 bins, each of width 2h ' 0.20 so h ' 0.10 histogram lnhwage, bin(30) scale(1.1) Density natural log of hwage A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 -Advanced Sep 2, 2017 Microeconometri 6 / 32

7 2. Nonparametric (kernel) density estimation Kernel density estimate Kernel density estimate Recall bf HST (x 0 ) = 1 Nh N i= x i x 0 h < 1 Replace 1 (A) by a kernel function Kernel density estimate of f (x 0 ), the density of x evaluated at x 0, is bf (x 0 ) = 1 Nh N i=1 K x i x 0 h K () is called a kernel function h is called the bandwidth or window width or smoothing parameter h Example is Epanechnikov kernel K (z) = 0.75(1 z 2 ) 1(jzj < 1) in Stata epan2 kernel more weight on data at center, less weight at end More generally kernel function must satisfy conditions including Continuous, K (z) = K ( z), R K (z)dz = 1, R K (z)dz = 1, tails go to zero. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 -Advanced Sep 2, 2017 Microeconometri 7 / 32

8 2. Nonparametric (kernel) density estimation Kernel density example Data example: kernel of lnwage for 175 observations Stata s epanechnikov kernel K (z) = 0.75(1 z 2 )/ p 5 1(jzj < p 5) default h = 0.9m/N 0.2 where m = min(st.dev.(x), interquartilerange x /1.349) yields h = h = 0.07 (oversmooths), 0.21 (default) or 0.63 (undersmooths) e.g. kdensity lnhwage, bw(0.21) kdensity lnhwage x Default Twice default Half default. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 -Advanced Sep 2, 2017 Microeconometri 8 / 32

9 2. Nonparametric (kernel) density estimation mplementation mplementation Key is choice of bandwidth The default can oversmooth: may need to decrease bw() For kernel choice for given bandwidth get similar results across kernels if K (z) > 0 for jzj < 1 and K (z) = 0 for jzj 1. this is most kernels aside from epanichnikov and gaussian. Other smooth estimators exist most notably k-nearest neighbors but usually no reason to use anything but kernel. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 -Advanced Sep 2, 2017 Microeconometri 9 / 32

10 3. Kernel regression Local average estimator 3. Kernel regression: Local average estimator We want to estimate at various values x 0 the conditional mean function m(x 0 ) = E [yjx = x 0 ] The functional form m() is not speci ed. A local average estimator is The weights w(x i, x 0, h) bm(x 0 ) = N i=1 w(x i, x 0, h)y i, sum over i to one decrease as the distance between x i and x 0 increases place more weight on observations with x i close to x 0 as bandwidth h decreases most common: kernel weights, Lowess and k-nearest neighbors (average the yi 0 s for the k x 0 i s closest to x 0). Evaluate bm(x 0 ) at a variety of points x 0 gives a regression curve. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 10 / 32

11 3. Kernel regression Kernel (local constant) regression Kernel (local constant) regression Let w(x i, x 0, h) = K x i x 0 h / N j=1 K xj x 0 h. Kernel regression with 95% con dence bands, default kernel (Epanechnikov) and default bandwidth lpoly lnhwage educatn, ci msize(medsmall) Local polynomial smooth natural log of hwage years of completed schooling % C natural log of hwage lpoly smooth k ernel = epanec hnik ov, degree = 0, bandw idth = 1.53, pw idth = 2.3. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 11 / 32

12 3. Kernel regression Di erent bandwidths Local linear regression A sample mean of y = OLS of y on an intercept. A weighted sample mean of y = weighted OLS of y on an intercept. So the kernel (local constant) estimator bm(x 0 ) = bα 0 that minimizes N i=1 w(x i, x 0, h)(y i α 0 ) 2. The local linear estimator generalizes to bm(x 0 ) = bα 0 that minimizes N i=1 w(x i, x 0, h)fy i α o β 0 (x i x 0 )g 2. furthermore bβ 0 = bm 0 (x 0 ), an estimate of E [yjx]/ xj x0. Advantage - better estimates at endpoints of the data. n Stata lpoly lnhwage educatn,degree(1). And can extend to higher order polynomials. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 12 / 32

13 3. Kernel regression Lowess Lowess (locally weighted scatterplot smoothing) is a variation of local linear with variable bandwidth, tricubic kernel and downweighting of outliers. Kernel, local linear and lowess with default bandwidths graph twoway lpoly y x jj lpoly y x, deg(1) jj lowess y x kernel erroneously underestimates m(x) at the endpoint x = Kernel Local linear lowess A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 13 / 32

14 3. Kernel regression mplementation mplementation Di erent methods work di erently Local linear and local polynomial handle endpoints better than kernel. bm(x 0 ) is asymptotically normal this gives con dence bands that allow for heteroskedasticity Bandwidth choice is crucial optimal bandwidth trades o bias (minimized with small bandwidth) and variance (minimized with large bandwidth) theory just says optimal bandwidth for kernel regression is O(N 0.2 ) plug-in or default bandwidth estimates are often not the best so also try e.g. half and two times the default. cross validation minimizes the empirical mean square error i (y i bm i (x i )) 2, where bm i (x i ) is the leave-one-out" estimate of bm(x i ) formed with y i excluded F empirical estimate of MSE[ bm(x i )] = Variance + Bias 2. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 14 / 32

15 4. npregress command 4. npregress command Stata 15 has new npregress command. Does local constant and local linear regression. Determines bandwidth by cross-validation whereas lpoly uses plug-in value Evaluates at each x i value whereas lpoly default is to evaluate at 50 equally spaced values. For local linear computes partial e ects. Can use margins and marginsplot for plots and average partial e ects. Can have more than one regressor. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 15 / 32

16 4. npregress command npregress with defaults LOOCV separate for bandwidth for bm(x 0 ) and bm 0 (x 0 ). * npregress command local linear. npregress kernel lnhwage educatn Computing mean function Minimizing cross validation function: teration 0: Cross validation criterion = teration 1: Cross validation criterion = teration 2: Cross validation criterion = teration 3: Cross validation criterion = teration 4: Cross validation criterion = teration 5: Cross validation criterion = teration 6: Cross validation criterion = teration 7: Cross validation criterion = teration 8: Cross validation criterion = Computing optimal derivative bandwidth teration 0: Cross validation criterion = teration 1: Cross validation criterion = teration 2: Cross validation criterion = teration 3: Cross validation criterion = teration 4: Cross validation criterion = A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 16 / 32

17 4. npregress command npregress reports averages bα = 1 N N i=1 [ α(x i ) and bβ = 1 N N i=1 [ β(x i ) Bandwidth Mean Effect Mean educatn Local linear regression Number of obs = 177 Kernel : epanechnikov E(Kernel obs) = 177 Bandwidth: cross validation R squared = lnhwage Estimate Mean lnhwage Effect educatn Note: Effect estimates are averages of derivatives. Note: You may compute standard errors using vce(bootstrap) or reps(). Versus OLS bα = and bβ = Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 17 / 32

18 4. npregress command Get bootstrap standard errors. * npregress with bootstrap standard errors. npregress kernel lnhwage educatn, vce(bootstrap, seed(10101) reps(50)) (running npregress on estimation sample) Bootstrap replications (50) Bandwidth Mean Effect Mean educatn Local linear regression Number of obs = 177 Kernel : epanechnikov E(Kernel obs) = 177 Bandwidth: cross validation R squared = Observed Bootstrap Percentile lnhwage Estimate Std. Err. z P> z [95% Conf. nterval] Mean lnhwage Effect educatn Note: Effect estimates are averages of derivatives. Versus OLS se(bα) = and se(bβ) = A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 18 / 32

19 4. npregress command Predict at selected values of education. margins, at(educatn = (10(1)16)) vce(bootstrap, seed(10101) reps(50)) (running margins on estimation sample) Bootstrap replications (50) Adjusted predictions Number of obs = 177 Replications = 50 Expression : mean function, predict() 1._at : educatn = 10 2._at : educatn = 11 3._at : educatn = 12 4._at : educatn = 13 5._at : educatn = 14 6._at : educatn = 15 7._at : educatn = 16 Observed Bootstrap Percentile Margin Std. Err. z P> z [95% Conf. nterval] _at A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 19 / 32

20 4. npregress command marginsplot, legend(o ) scale(1.1) /// addplot(scatter lnhwage educatn if lnhwage<50000, msize(tiny)) Adjusted Predictions with 95% Cs Mean Function years of completed schooling 1992 A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 20 / 32

21 4. npregress command Now consider partial e ects at selected values of education * Partial e ects of changing hours margins, at(educatn = (10(1)16)) contrast(atcontrast(ar)) /// vce(bootstrap, seed(10101) reps(50)) Output includes Observed Bootstrap Contrast Std. Err. Percentile [95% Conf. nterval] _at (2 vs 1) (3 vs 2) (4 vs 3) (5 vs 4) (6 vs 5) (7 vs 6) A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 21 / 32

22 4. npregress command marginsplot, legend(o ) Contrasts of Mean Function Contrasts of Adjusted Predictions with 95% Cs years of completed schooling Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 22 / 32

23 5. Semiparametric estimation 5. Semiparametric estimation Nonparametric regression is problematic when more than one regressor in theory can do multivariate kernel regression in practice the local averages are over sparse cells called the curse of dimensionality Semiparametric methods place some structure on the problem parametric component for part of the model nonparametric component that is often one dimensional deally p N( β b β)! d N [0, V] despite the nonparametric component. Three leading examples partial linear single-index generalized additive model. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 23 / 32

24 5. Semiparametric estimation OLS estimates OLS estimates Consider log hourly wage regressed on years of education and annual hours worked. regress lnhwage educatn hours, vce(robust) Linear regression Number of obs = 177 F(2, 174) = Prob > F = R squared = Root MSE = Robust lnhwage Coef. Std. Err. t P> t [95% Conf. nterval] educatn hours _cons A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 24 / 32

25 5. Semiparametric estimation Partial linear model Partial linear model Model: E[y i jx i, z i ] = x 0 i β + λ(z i ) where λ() not speci ed. Robinson di erencing estimator kernel regress y on z and get residual y by kernel regress x on z and get residual x bx OLS regress y by on x bx. * Partial linear model Robinson differencing estimator. semipar lnhwage educatn, nonpar(hours) robust ci title("partial linear") Number of obs = 176 R squared = Adj R squared = Root MSE = lnhwage Coef. Std. Err. t P> t [95% Conf. nterval] educatn A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 25 / 32

26 5. Semiparametric estimation Partial linear model Plot of λ(z) against z where z is annual hours worked. Partial linear A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 26 / 32

27 5. Semiparametric estimation Single-index model Single-index model Model: E[y i jx i ] = g(xi 0 β) where g() not speci ed chimura semiparametric least squares β b and bg minimize N i=1 w(x i )fy i bg(x 0 i β)g 2 where w(x i )is a trimming function that drops outlying x values. Can only estimate β up to scale in this model Still useful as ratio of coe cients equals ratio of marginal e ects in a single-index models From next slide one more year of education has same e ect on log hourly wage as working 1,048 more hours versus OLS / = 785. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 27 / 32

28 5. Semiparametric estimation Single-index model. * Single index model chimura semiparametric least squares. sls lnhwage hours educatn, trim(1,99) initial: SSq(b) = alternative: SSq(b) = rescale: SSq(b) = SLS 0: SSq(b) = SLS 1: SSq(b) = SLS 2: SSq(b) = SLS 3: SSq(b) = SLS 4: SSq(b) = pilot bandwidth SLS 0: SSq(b) = (not concave) SLS 1: SSq(b) = SLS 2: SSq(b) = SLS 3: SSq(b) = SLS 4: SSq(b) = Number of obs = 177 root MSE = lnhwage Coef. Std. Err. z P> z [95% Conf. nterval] ndex educatn hours 1 (offset). Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 28 / 32

29 5. Semiparametric estimation Generalized additive model Generalized additive model Model: E [y i jx i ] = g 1 (x 1i ) + + g K (x Ki ) where g j () are unspeci ed. Estimate by back tting and here by smoothing spline for each g j (). * Generalized additive model. gam lnhwage educatn hours, df(3) 177 records merged. Generalized Additive Model with family gauss, link ident. Model df = No. of obs = 177 Deviance = Dispersion = lnhwage df Lin. Coef. Std. Err. z Gain P>Gain educatn hours _cons Total gain (nonlinearity chisquare) = (4.003 df), P = A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 29 / 32

30 5. Semiparametric estimation Generalized additive model Plot each g j () function looks like education linear or quadratic; hours linear Component & partial residuals for lnhwage GAM 3 df smooth for educatn, adjusted for covariates years of completed schooling 1992 Component & partial residuals for lnhwage GAM 3 df smooth for hours, adjusted for covariates annual work hours 1992 A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 30 / 32

31 6. Stata Commands 6. Stata commands Command kernel does kernel density estimate. Command lpoly does several nonparametric regressions kernel is default local linear is option degree(1) local polynomial of degree p is option degree(p) Command lowess does Lowess. Stata 15 command npregress does local constant and local linear for one or more regressors with bandwidth chosen by leave-on-out cross validation. For semiparametric use add-ons semipar, sls, gam gam requires MS Windows. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 31 / 32

32 6. Stata Commands 6. References A. Colin Cameron and Pravin K. Trivedi (2005), Microeconometrics: Methods and Applications (MMA), chapter 9, Cambridge Univ. Press. A. Colin Cameron and Pravin K. Trivedi (2009), Microeconometrics using Stata (MUS), chapter 2.6, Stata Press. A. Colin Cameron Univ. of Calif. - Davis... for Center of Labor Nonparametrics Economics Norwegian School of Economics Aug 28 - Sep Advanced 2, 2017 Microeconometri 32 / 32