Graduate Econometrics Lecture 4: Heteroskedasticity

Graduate Econometrics Lecture 4: Heteroskedasticity Department of Economics University of Gothenburg November 30, 2014 1/43

and Autocorrelation Consequences for OLS Estimator Begin from the linear model y i = x i β + ε i (1) y = Xβ + ε (2) Gauss-Markov assumptions [A1]-[A4] summarized: E{ε X} = E{ε} = 0 (3) V{ε X} = V{ε} = σ 2 I (4) 2/43

and Autocorrelation Consequences for OLS Estimator cont. We saw that OLS is BLUE when these assumptions are fulfilled Both Heteroskedasticity and Autocorrelation = Eq.(4) no longer holds Heteroskedasticity = different error terms do not have identical variances = the diagonal elements of the covariance matrix are not the same Different groups in the sample (e.g. males and females) have different variances The variation of unexplained household savings increases with income (just as savings itself) Autocorrelation arises in cases where the data has time dimension The covariance matrix is nondiagonal such that different error terms are correlated. In this chapter we focus on heteroskedasticity 3/43

and Autocorrelation Consequences for OLS Estimator cont. Assume the error covariance can be written as follows: V{ε X} = σ 2 Ψ (5) Where Ψ is a positive definite matrix Ψ = I = we have Non spherical error terms Under (5) instead of (4), OLS will still be unbiased (but not efficient) = standard errors would be incorrect and the t-test and F-test will no longer be valid, inferences will be misleading We saw that: b = (X X) 1 X y = β + (X X) 1 X ε Conditional upon X, the covariance matrix of b thus depends upon the conditional variance matrix of ε Given (5), this implies V{b X} = V{(X X) 1 X ε X} = (X X) 1 X V{ε X}X(X X) 1 4/43

and Autocorrelation Consequences for OLS Estimator cont. = σ 2 (X X) 1 X ΨX(X X) 1 (6) Two solutions. 1 Derive an alternative (efficient) estimator 2 Adjust standard errors of the OLS model to allow heteroskedasticity and autocorrelation 3 Reconsider specification of the model 5/43

and Autocorrelation Alternative Estimator Assume Ψ is completely known and can be written as for some square, nonsingular matrix P Using (7), it is possible to write Ψ 1 = P P (7) Ψ = (P P) 1 = P 1 (P ) 1 PΨP = PP 1 (P ) 1 P = I Premultiply ε by P: E{Pε X} = PE{ε X} = 0 V{Pε X} = PV{ε X}P = σ 2 PΨP = σ 2 I We can see that Pε satisfies the Gauss-Markov conditions 6/43

and Autocorrelation Alternative Estimator Transforming the entire model by this P matrix gives Py = PXβ + Pε or y = X β + ε (8) ε now satisfies the Gauss-Markov conditions The resulting OLS model on this transformed model known as the Generalized Least Squares (GLS estimator), which would be BLUE is given by ˆβ = (X X ) 1 X y = (X Ψ 1 X) 1 X Ψ 1 y (9) But note that, to estimate the GLS estimator, we need to know Ψ In practice, we don t know Ψ and we need to estimate it first Using an estimated version of Ψ, will give rise to what is called the Feasible Generalized Least Squares (FGLS) or Estimated Generalized Least Squares EGLS estimator) 7/43

and Autocorrelation Alternative Estimator The variance covariance matrix for ˆβ V{ ˆβ} = σ 2 (X X ) 1 = σ 2 (X Ψ 1 X) 1 (10) Where σ 2 in this case is estimated by: ˆσ 2 = 1 N K (y X ˆβ) (y X ˆβ) = 1 N K (y X ˆβ) Ψ 1 (y X ˆβ) It can be proved that this estimator is BLUE and it has a smaller variance than the OLS estimator b (11) 8/43

Alternative Estimator Heteroskedasticity: A Formal Definition Heteroskedasticity: the case where V{ε X} is diagonal, but not equal to σ 2 I Consider a case where y i denotes expenditure on food and x i consists of a constant and disposable income dpi An Engel curve for food is expected to be upward sloping = higher income corresponds to higher expenditure on food It would be likely that the variation in food expenditures among high-income households is much larger that the variation among low-income households The variance of ε i increases with income One can model the heteroskedasticity in this case as V{ε i dpi i } = σi 2 = σ 2 exp{α 2 dpi i } = exp{α 1 + α 2 dpi i } (12) For some α 2 and α 1 = logσ 2 9/43

Alternative Estimator cont. Assume V{ε i X} = V{ε i x i } = σ 2 h 2 i (13) Where all h 2 i are known and positive Combining this (and assuming no autocorrelation), one can formulate a new assumption V{ε X} = σ 2 Diag{h 2 i } = σ2 Ψ [A9] Where Diag{h 2 i } is a diagonal matrix with elements h2 1,..., h2 N The new assumption (A9) clearly replaces assumptions or Gauss-Markov conditions (A3) and (A4) You can see that the variance of ε depends upon the explanatory variables = E{ε X} = 0 [A10] 10/43

Alternative Estimator cont. [A10] = ε is conditionally mean zero independent of X We want to get a BLUE for β in the model with the new conditions [A9] and [A10] y i = x i β + ε i, i = 1,..., N (14) The structure of Ψ = an appropriate transformation matrix P is a diagonal matrix with elements h 1 1,..., h 1 N Typical elements in the transformed data vector P y are thus yi = y i /h i and xi = x i /h i Running OLS on: y i = x i β + ε i (15) y i h i = ( x i h i ) β + ε i h i (16) 11/43

Alternative Estimator cont. ε i is now homoskedastic. The resulting least square estimator is given by ˆβ = ( N h 2 i i=1 x i x i ) 1 N i=1 h 2 i x i y i (17) This is a special case of the estimator given in eq. (9) is referred to as the weighted least squares estimator It is a least squares estimator in which each observation is weighted by (a factor proportional to) the inverse of the error variance 12/43

Alternative Estimator cont. Under [A9] [A10] this estimator is BLUE The use of weights implies observations with a higher variance get a smaller weight in estimation. The greatest weights are given to those observations that provide the most accurate information about the model parameters, and the smallest weights to those that provide relatively little information about β V{ ˆβ} = σ 2 ( ˆσ 2 = 1 N K N i=1 N i=1 h 2 i x i x i ) 1 (18) h 2 i (y i x i ˆβ) 2 (19) 13/43

Alternative Estimator When the Variances are Unknown If the h i s in Eq. (13) are unknown, they should be replaced by their consistent estimator ˆβ = ( N i=1 2 ĥ i xi x i N ) 1 2 ĥ i xi y i (20) i=1 The corresponding estimator is the FGLS because it is based on the estimated values of h 2 i Under some weak regularity conditions, the FGLS and GLS estimators are asymptotically equivalent But in small sample, the FGLS outperforms OLS although it usually does The covariance matrix is given by V{ ˆβ } = ˆσ 2 ( N ĥ i 2 xi x i ) 1 (21) 14/43 Graduate i=1 Econometrics Lecture 4: Heteroskedasticity

Alternative Estimator When the Variances are Unknown cont. where ˆσ 2 is the standard estimator for the error variance estimated from ˆσ 2 = 1 N K N i=1 ĥ i 2 (yi x i ˆβ ) 2 (22) 15/43

Alternative Estimator Heteroskedasticity Consistent SE for OLS Reconsider the model with heteroskedastic errors With E{ε i X} = 0 and V{ε i X} = σ 2 i. y i = x i β + ε i (23) From eq. (6), the appropriate covariance matrix is given by V{b X} = ( N i=1 N x i x i ) 1 ( σi 2 x ix N i )( x i x i ) 1 (24) i=1 i=1 To estimate the covariance matrix, we need to estimate the σi 2 for which we need additional assumptions 16/43

Alternative Estimator Heteroskedasticity Consistent SE for OLS cont. According to White (1980), only a consistent estimator of the K K matrix is required 1 N σ2 i x ix i (25) Under very general conditions, it can be shown that S 1 N N i=1 e 2 i x ix i (26) Where e i is the OLS residual, is a consistent estimator for. 17/43

Alternative Estimator Heteroskedasticity Consistent SE for OLS cont. Therefore, ˆV{b} = ( N i=1 N x i x i ) 1 ( e 2 i x ix N i )( x i x i ) 1 (27) i=1 i=1 Can be used as an estimate of the true variance of the OLS estimator. Appropriate inferences based upon b could be made without actually specifying the type of heteroskedasticity Standard errors computed this was are called heteroskedasticity-consistent or White standard errors 18/43

Alternative Estimator Heteroskedasticity Consistent SE for OLS cont. They are very common in applied work to use such robust standard errors Can be estimated by using the robust option in Stata Good to report both the usual and the heteroskedasticity-consistent standard errors to show how sensitive the results are to the type of standard error used Robust standard errors are likely to be larger than homoskedastic ones 19/43

Multiplicative Heteroskedasticity A common form of heteroskedasticity The error variance is related to a number of exogenous variables, gathered in J dimensional vector z i (excluding the intercept) V{ε i x i } = σi 2 = σ 2 exp{α 1 z i1 +... + α J z ij } = σ 2 exp{z i α} (28) where z i = a vector of observed variables that is a function of x i The error variance is therefore related to one or more exogenous variables To be able to compute the EGLS estimator, one needs consistent estimator for the unknown parameters in h 2 i = exp{z i α} i.e., for α which can be based up on the OLS residuals 20/43

Multiplicative Heteroskedasticity Cont. To see how, first note that log σi 2 = logσ 2 + z i α The OLS residuals e i = y i x i b have something to tell about σi 2 = loge 2 i = logσ 2 + z i α + υ i (29) Where υ i = log(e 2 i /z i) is an error term that is (asymptotically) homoskedastic and uncorrelated with z i However, υ i does not have zero expectation (even asymptotically) but this will affect estimation of the constant logσ 2 which is irrelevant Consequently, the EGLS estimator for β can be obtained along simple steps (we will se example later) 21/43

Formal Tests In order to claim that the standard errors generated from OLS are misleading due to heteroskedasticity, we need to perform a standard test A number of tests are available If these tests do not reject H 0 := heteroskedasticity, there is no problem in using OLS If we however reject H 0 :=, we should find a solution Use EGLS Use Heteroskedasticity-consistent standard errors for the OLS estimator Revise the specification of our model 22/43

Formal Tests Multiplicative Heteroskedasticity The alternative hypothesis for this test is eq. (28), i.e., σi 2 = exp{z i α} (30) where z i is a J dimensional vector. The null hypothesis of homoskedasticity corresponds to α = 0, so the problem under test is H 0 = α = 0 versus H 1 : α = 0 (31) H 0 can be tested using the results of the least square regression stated in eq.(29) using the standard F test for the hypothesis that all the parameters except the intercept are equal to zero Because υ i in (29) does not satisfy the Gauss-Markov conditions exactly, the F distribution (with J and N-J-1 d.f) holds by approximation 23/43

Formal Tests The Breusch-Pagan (BP) Test Provided by Breusch and Pagan (1980) σi 2 = σ 2 h(z i α) (32) where h is unknown, continuously differentiable function (that does not depend on i), such that h(.) > 0 and h(0) = 1 As a special case (if h(t) = exp{t}) one obtains eq.(30) A test for H 0 : α = 0 versus H 1 : α = 0 can be derived independently of the function h 24/43

Formal Tests The Breusch-Pagan (BP) Test cont. The simplest version of the BP test can be computed as the number of observations multiplied by the R 2 of an auxiliary regression (a regression of e 2 i, i.e., the OLS residuals) on z i and a constant The resulting test statistic given by ξ = NR 2 is asymptotically χ 2 distributed with J d.f The BP test is a Lagrange multiplier (LM) test for heteroskedasticity LM tests do not require the model to be estimated under alternative and that they are often simply computed from R 2 of some auxiliary regression (we will see them in the ML section) 25/43

Formal Tests The White Test We saw that the correct covariance matrix of the least squares estimator is given by ( N 1 ˆV{b} = s 2 x i x i) (33) i=1 If there is no heteroskedasticity, (33) will give a consistent estimator of V{b}, while if there is, it will not Obtain NR 2 in the regression of e 2 i on a constant and all (unique) first moments, second moments, and cross-products of the original regressors This test statistic is asymptotically distributed as χ 2 with P d.f, where P is the number of regressors in the auxiliary regression, excluding the intercept. 26/43

Formal Tests The White Test cont. The White test is a generalization of the BP test, which also involves an auxiliary regression of squared residuals but excludes any higher-order terms It can therefore detect more general forms of heteroskedasticity that the BP test Limitations: it may identify heteroskedasticity but it may instead simply identify some other specification error (e.g., incorrect functional form) If N is small, it may not have the power to detect heteroskedasticity 27/43

Formal Tests Which Test? Depends on how explicit one would want to be The more explicit we are, the more powerful the test will be = more likely it is that the test will correctly reject H 0 However, if the true heteroskedasticity is of different form, the chosen test may not indicate the presence of heteroskedasticity at all The White test has limited power against a large number of alternatives, while a specific test, like the one for multiplicative heteroskedasticity, has more power but only against a limited number of alternatives In some cases, a visual inspection of the residuals (e.g., a plot of OLS residuals against one or more exogenous variables, or the fitted values) or economic theory can help in choosing the appropriate alternative 28/43

Illustration Labour Demand We will consider a simple model of labor demand using cross-sectional data collected in 1996 from 569 Belgian firms List of variables labour : total employment (number of workers) capital : total fixed assets (in million euro) wage : total wage costs divided by number of workers (in 1000 euro) output : value added (in million euro) 29/43

Illustration Labour Demand Consider a simple production function Q = f (K, L) (34) where Q = output and K and L denote the capital and labour inputs respectively Total production costs = rk + wl, r = the costs of capital, w = the costs of labour (the wage rate) Minimizing total costs subject to the production function gives the demand functions for capital and labour 30/43

Illustration Labour Demand cont. The labour demand function is given by: for some function g L = g(q, r, w) (35) One approximates r by capital stock K since the former does not have variation in the cross-section 31/43

Illustration Labour Demand OLS. reg labor wage output capital Source SS df MS Number of obs = 569 F( 3, 565) = 2716.02 Model 198943126 3 66314375.3 Prob > F = 0.0000 Residual 13795026.5 565 24415.9761 R-squared = 0.9352 Adj R-squared = 0.9348 Total 212738152 568 374539 Root MSE = 156.26 labor Coef. Std. Err. t P> t [95% Conf. Interval] wage -6.741904.5014054-13.45 0.000-7.72675-5.757057 output 15.40047.3556333 43.30 0.000 14.70194 16.09899 capital -4.590491.2689693-17.07 0.000-5.118793-4.062189 _cons 287.7186 19.64175 14.65 0.000 249.1388 326.2984 32/43

Illustration Labour Demand BP Test- Aux. Reg.. reg e2 wage output capital Source SS df MS Number of obs = 569 F( 3, 565) = 262.05 Model 6.9733e+12 3 2.3244e+12 Prob > F = 0.0000 Residual 5.0117e+12 565 8.8702e+09 R-squared = 0.5818 Adj R-squared = 0.5796 Total 1.1985e+13 568 2.1100e+10 Root MSE = 94182 e2 Coef. Std. Err. t P> t [95% Conf. Interval] wage 228.8569 302.2172 0.76 0.449-364.7495 822.4632 output 5362.207 214.3544 25.02 0.000 4941.179 5783.236 capital -3543.509 162.1186-21.86 0.000-3861.938-3225.081 _cons -22719.51 11838.87-1.92 0.055-45973.09 534.0697 33/43

Illustration Labour Demand BP Test- Aux. Reg. High R 2 and t values = high possibility of heteroskedasticity Test statistic = N R 2 = 331.00 Sound rejection of H 0 = homoskedasticity as the asymptotic distribution under H 0 is Chi-squared with 3 d.f Common to find heteroskedasticity in situations like this The size of the observed units differs substantially The sample for e.g., includes firms with one employee and firms with 1000 employees Larger firms have larger absolute values of all variables in the model, including the error term Consider a log-linear (cobb-douglass) model like: Q = AK α L β 34/43

Illustration Labour Demand - a Log-Linear Model Source SS df MS Number of obs = 569 F( 3, 565) = 1011.02 Model 656.747035 3 218.915678 Prob > F = 0.0000 Residual 122.338812 565.21652887 R-squared = 0.8430 Adj R-squared = 0.8421 Total 779.085847 568 1.37163001 Root MSE =.46533 lnlabor Coef. Std. Err. t P> t [95% Conf. Interval] lnwage -.9277643.0714046-12.99 0.000-1.068015 -.7875133 lnoutput.9900474.0264103 37.49 0.000.938173 1.041922 lncapital -.0036975.0187697-0.20 0.844 -.0405644.0331695 _cons 6.17729.2462105 25.09 0.000 5.69369 6.660889 35/43

Illustration Labour Demand - a Log-Linear Model, BP Test We run the same auxiliary regression BP Test statistic = N R 2 = 569 0.0136 = 7.74 = rejection of H 0! at 5% (but at the margin) The data is still heteroskedastic! Let s consider the While Test next To compute the test statistic, we run an auxiliary regression of squared OLS residuals upon all original regressors, their squares and all their interactions 36/43

Illustration Labour Demand - a Log-Linear Model, White Test. reg le2 lnwage lnoutput lncapital lnwage2 lnoutput2 lncapital2 lnwlno lnwlnc lnclno Source SS df MS Number of obs = 569 F( 9, 559) = 7.12 Model 46.3960091 9 5.15511212 Prob > F = 0.0000 Residual 404.532761 559.723672202 R-squared = 0.1029 Adj R-squared = 0.0884 Total 450.92877 568.793888679 Root MSE =.85069 le2 Coef. Std. Err. t P> t [95% Conf. Interval] lnwage -1.299009 1.752744-0.74 0.459-4.741779 2.143761 lnoutput -.9037237.5598545-1.61 0.107-2.003399.1959521 lncapital 1.142052.3758217 3.04 0.002.4038572 1.880248 lnwage2.192742.2589536 0.74 0.457 -.3158991.701383 lnoutput2.1381977.0356469 3.88 0.000.0681795.208216 lncapital2.0895375.0139874 6.40 0.000.0620631.1170118 lnwlno.1380379.1625628 0.85 0.396 -.1812707.4573464 lnwlnc -.2517788.1049671-2.40 0.017 -.4579569 -.0456006 lnclno -.1916049.0368665-5.20 0.000 -.2640187 -.119191 _cons 2.544614 3.002783 0.85 0.397-3.353503 8.44273 37/43

Illustration Labour Demand - a Log-Linear Model, White Test The White test statistic = N R 2 = 569 0.1029 = 58.55 Reject H 0 because the statistic is highly significant for a Chi-squared variable with 9 d.f The aux. regression shows that output and capital are statistically significant = 38/43

Illustration Labour Demand - a Log-Linear Model with White Standard Errors. reg lnlabor lnwage lnoutput lncapital, robust Linear regression Number of obs = 569 F( 3, 565) = 544.73 Prob > F = 0.0000 R-squared = 0.8430 Root MSE =.46533 Robust lnlabor Coef. Std. Err. t P> t [95% Conf. Interval] lnwage -.9277643.0866604-10.71 0.000-1.09798 -.7575484 lnoutput.9900474.0467902 21.16 0.000.8981434 1.081951 lncapital -.0036975.037877-0.10 0.922 -.0780944.0706995 _cons 6.17729.2938869 21.02 0.000 5.600045 6.754534 39/43

Illustration Labour Demand - a Log-Linear, testing for Multiplicative heteroskedasticity Set z i = x i The variance of the error term depends upon log(wage), log(output) and log(capital) Compute the log of the squared OLS residuals and then run a regression of log e 2 i upon z i and a constant This gives the auxiliary regression results on the next page 40/43

Illustration Labour Demand testing for Multiplicative heteroskedasticity. reg lme2 lnwage lnoutput lncapital Source SS df MS Number of obs = 569 F( 3, 565) = 4.73 Model 71.1997167 3 23.7332389 Prob > F = 0.0029 Residual 2836.08166 565 5.01961356 R-squared = 0.0245 Adj R-squared = 0.0193 Total 2907.28138 568 5.11845313 Root MSE = 2.2404 lme2 Coef. Std. Err. t P> t [95% Conf. Interval] lnwage -.0610507.3437983-0.18 0.859 -.7363295.6142281 lnoutput.2669502.1271601 2.10 0.036.017186.5167144 lncapital -.3306879.0903722-3.66 0.000 -.5081943 -.1531814 _cons -3.253832 1.185452-2.74 0.006-5.582262 -.9254013 41/43

Illustration Labour Demand testing for Multiplicative heteroskedasticity We reject H 0 := homoskedasticity because the F-value for the auxiliary regression 4.73 > 2.6 (i.e., the critical value from the F-table at Was this specification for the form of heteroskedasticity is too restrictive? Check by running a version of the previous auxiliary regression where the three-squared terms are also included F-test from this reg = 1.85(p = 0.137) = H 0 is not rejected! 42/43

Illustration Labour Demand Weighted Least Square Results. wls0 lnlabor lnwage lnoutput lncapital, wvar(lnwage lnoutput lncapital) type(abse) noconst WLS regression - type: proportional to abs(e) (sum of wgt is 6.6712e+03) Source SS df MS Number of obs = 569 F( 3, 565) = 1222.58 Model 751.361952 3 250.453984 Prob > F = 0.0000 Residual 115.744451 565.204857436 R-squared = 0.8665 Adj R-squared = 0.8658 Total 867.106403 568 1.52659578 Root MSE =.45261 lnlabor Coef. Std. Err. t P> t [95% Conf. Interval] lnwage -.8872274.0696778-12.73 0.000-1.024087 -.7503682 lnoutput 1.044697.0266488 39.20 0.000.9923539 1.09704 lncapital -.0561104.0206173-2.72 0.007 -.0966063 -.0156145 _cons 5.987955.2370313 25.26 0.000 5.522385 6.453525 43/43