8. Nonstandard standard error issues 8.1. The bias of robust standard errors

Transcription

1 8.1. The bias of robust standard errors Bias Robust standard errors are now easily obtained using e.g. Stata option robust Robust standard errors are preferable to normal standard errors when residuals are heteroskedastic, as is almost always the case if we try to approximate the CEF with a linear model The problem is that robust standard errors are only asymptotically valid (i.e. theoretically we need an infinitely large data set) In small samples, robust standard errors are biased. This bias can be worse than when using old-fashioned OLS standard errors Solutions that correct this bias are offered in Stata using options vce(hc2) and vce(hc3) In large samples, correcting standard errors for heteroskedasticity usually does not make a big difference E.g. coefficients that are significant before correction are often also significant after correction Jürges (Bergische Universität Wuppertal) Empirical Methods WS 2011/ / 268

2 A much bigger difference can arise when we correct for the clustered observations Whereas our textbook assumption often is that observations are independently drawn (coming from a simple random sample), many data sets contain clustered observations that are not drawn independently One prominent example are students that are clustered in classes that are in turn clustered in schools This affects standard errors because with a given number of individual observations (students), the number of independent observational units (classes or schools) shrinks considerably Jürges (Bergische Universität Wuppertal) Empirical Methods WS 2011/ / 268

3 Consider the following simple regression of student achievement (test scores) Y on class size x Y ig = β 0 + β 1 x g + e ig here class size only varies at the class level Observations within classes are not independent because they share the same teacher and a common learning environment. This all goes into the error term Error terms e ig are therefore correlated within class E[e ig e jg ]=ρ e σ 2 e > 0 where ρ e is the intraclass correlation and σe 2 is the residual variance This type of error structure can be modelled in a random effects framework e ig = ν g + η ig where ν g is the group level residual and η ig is an individual-specific residual that uncorrelated within groups Jürges (Bergische Universität Wuppertal) Empirical Methods WS 2011/ / 268

4 Clustered standard errors can be very different from conventional standard errors. This is exemplified in the simplified Moulton formula Let V c (β 1 ) the conventional variance (the square of the standard error) and V(β 1 ) the corrected variance, then V(β 1 ) V c (β 1 ) = 1 +(n 1)ρ e when group size n is the same in all groups and regressor x is non-stochastic The square root of this expression shows how much conventional standard errors must be inflated to get to the corrected standard errors Put differenty, it shows how much we overestimate precision when using conventional standard errors Jürges (Bergische Universität Wuppertal) Empirical Methods WS 2011/ / 268

5 Moulton factor N rho=0.1 rho=0.5 rho=0.9 rho=1 Jürges (Bergische Universität Wuppertal) Empirical Methods WS 2011/ / 268

6 The above formula is a special case of the general Moulton formula (variable n and stochastic x) ( ) V(β 1 ) V c (β 1 ) = 1 + V(ng ) + n 1 ρ x ρ e n where n is average group size and ρ x is the intraclass correlation of x The inflation factor thus also depends on ρ x. If explanatory variables are uncorrelated within groups (ρ x = 0), clustered standard errors are equal to conventional standard errors The impact of the intraclass correlation of error terms is biggest with fixed x within groups (i.e. ρ x = 1) Note: the special case shown on the previous slides is characterized by V(n g )=0, n = n,ρ x = 1 Jürges (Bergische Universität Wuppertal) Empirical Methods WS 2011/ / 268

7 There are several solutions to the standard error problem Parametric approach apply moulton formula to conventional standard errors (command moulton) Clustered standard errors account for heteroskedasticity and clustering (option cluster). Unreliable with few clusters Use group averages: attention individual level covariates require special treatment and the if we are interested in the effects of individual level covariates, the procedure changes the estimand Block Bootstrap: resample classes (not individuals) with replacement Random effects modeling (command xtreg): attention also changes the estimand Jürges (Bergische Universität Wuppertal) Empirical Methods WS 2011/ / 268

8 Example: Teacher Sex and Student Achievement score = PIRLS reading test score; boy = student sex; t_male = teacher sex; aged2-aged5 = teacher age groups. regress score boy t_male aged2-aged5,robust noheader Robust score Coef. Std. Err. t P> t [95% Conf. Interval] boy t_male aged aged aged aged _cons Jürges (Bergische Universität Wuppertal) Empirical Methods WS 2011/ / 268

9 Example: Teacher Sex and Student Achievement. moulton score boy t_male aged2-aged5,cluster(idclass) OLS Regression: standard errors Number of obs = 6869 adjusted for cluster effects using Moulton R-squared = Adj R-squared = Number of clusters (idclass) = 357 Root MSE = score Coef. Std. Err. t P> t [95% Conf. Interval] boy t_male aged aged aged aged _cons Intraclass correlation in boy = Intraclass correlation in t_male = Intraclass correlation in aged2 = Intraclass correlation in aged3 = Intraclass correlation in aged4 = Intraclass Jürges (Bergischecorrelation Universität Wuppertal) in aged5 = Empirical Methods WS 2011/ / 268

10 Example: Teacher Sex and Student Achievement. regress score boy t_male aged2-aged5,cluster(idclass) Linear regression Number of obs = 6869 F( 6, 356) = 9.46 Prob > F = R-squared = Root MSE = (Std. Err. adjusted for 357 clusters in idclass) Robust score Coef. Std. Err. t P> t [95% Conf. Interval] boy t_male aged aged aged aged _cons Jürges (Bergische Universität Wuppertal) Empirical Methods WS 2011/ / 268

11 Solutions in case of too few clusters Get more clusters not always possible, e.g. Germany only has 16 states Bias correction works well with the regular cluster problem but breaks down with serial correlation Base conference intervals and hypotheses testing on on t-distribution with G K degrees of freedom Group average estimation Block bootstrap Parametric correction adjusted Moulton formula Jürges (Bergische Universität Wuppertal) Empirical Methods WS 2011/ / 268