Graduate Econometrics Lecture 4: Heteroskedasticity

Transcription

1 Graduate Econometrics Lecture 4: Heteroskedasticity Department of Economics University of Gothenburg November 30, /43

2 and Autocorrelation Consequences for OLS Estimator Begin from the linear model y i = x i β + ε i (1) y = Xβ + ε (2) Gauss-Markov assumptions [A1]-[A4] summarized: E{ε X} = E{ε} = 0 (3) V{ε X} = V{ε} = σ 2 I (4) 2/43

3 and Autocorrelation Consequences for OLS Estimator cont. We saw that OLS is BLUE when these assumptions are fulfilled Both Heteroskedasticity and Autocorrelation = Eq.(4) no longer holds Heteroskedasticity = different error terms do not have identical variances = the diagonal elements of the covariance matrix are not the same Different groups in the sample (e.g. males and females) have different variances The variation of unexplained household savings increases with income (just as savings itself) Autocorrelation arises in cases where the data has time dimension The covariance matrix is nondiagonal such that different error terms are correlated. In this chapter we focus on heteroskedasticity 3/43

4 and Autocorrelation Consequences for OLS Estimator cont. Assume the error covariance can be written as follows: V{ε X} = σ 2 Ψ (5) Where Ψ is a positive definite matrix Ψ = I = we have Non spherical error terms Under (5) instead of (4), OLS will still be unbiased (but not efficient) = standard errors would be incorrect and the t-test and F-test will no longer be valid, inferences will be misleading We saw that: b = (X X) 1 X y = β + (X X) 1 X ε Conditional upon X, the covariance matrix of b thus depends upon the conditional variance matrix of ε Given (5), this implies V{b X} = V{(X X) 1 X ε X} = (X X) 1 X V{ε X}X(X X) 1 4/43

5 and Autocorrelation Consequences for OLS Estimator cont. = σ 2 (X X) 1 X ΨX(X X) 1 (6) Two solutions. 1 Derive an alternative (efficient) estimator 2 Adjust standard errors of the OLS model to allow heteroskedasticity and autocorrelation 3 Reconsider specification of the model 5/43

6 and Autocorrelation Alternative Estimator Assume Ψ is completely known and can be written as for some square, nonsingular matrix P Using (7), it is possible to write Ψ 1 = P P (7) Ψ = (P P) 1 = P 1 (P ) 1 PΨP = PP 1 (P ) 1 P = I Premultiply ε by P: E{Pε X} = PE{ε X} = 0 V{Pε X} = PV{ε X}P = σ 2 PΨP = σ 2 I We can see that Pε satisfies the Gauss-Markov conditions 6/43

7 and Autocorrelation Alternative Estimator Transforming the entire model by this P matrix gives Py = PXβ + Pε or y = X β + ε (8) ε now satisfies the Gauss-Markov conditions The resulting OLS model on this transformed model known as the Generalized Least Squares (GLS estimator), which would be BLUE is given by ˆβ = (X X ) 1 X y = (X Ψ 1 X) 1 X Ψ 1 y (9) But note that, to estimate the GLS estimator, we need to know Ψ In practice, we don t know Ψ and we need to estimate it first Using an estimated version of Ψ, will give rise to what is called the Feasible Generalized Least Squares (FGLS) or Estimated Generalized Least Squares EGLS estimator) 7/43

8 and Autocorrelation Alternative Estimator The variance covariance matrix for ˆβ V{ ˆβ} = σ 2 (X X ) 1 = σ 2 (X Ψ 1 X) 1 (10) Where σ 2 in this case is estimated by: ˆσ 2 = 1 N K (y X ˆβ) (y X ˆβ) = 1 N K (y X ˆβ) Ψ 1 (y X ˆβ) It can be proved that this estimator is BLUE and it has a smaller variance than the OLS estimator b (11) 8/43

9 Alternative Estimator Heteroskedasticity: A Formal Definition Heteroskedasticity: the case where V{ε X} is diagonal, but not equal to σ 2 I Consider a case where y i denotes expenditure on food and x i consists of a constant and disposable income dpi An Engel curve for food is expected to be upward sloping = higher income corresponds to higher expenditure on food It would be likely that the variation in food expenditures among high-income households is much larger that the variation among low-income households The variance of ε i increases with income One can model the heteroskedasticity in this case as V{ε i dpi i } = σi 2 = σ 2 exp{α 2 dpi i } = exp{α 1 + α 2 dpi i } (12) For some α 2 and α 1 = logσ 2 9/43

10 Alternative Estimator cont. Assume V{ε i X} = V{ε i x i } = σ 2 h 2 i (13) Where all h 2 i are known and positive Combining this (and assuming no autocorrelation), one can formulate a new assumption V{ε X} = σ 2 Diag{h 2 i } = σ2 Ψ [A9] Where Diag{h 2 i } is a diagonal matrix with elements h2 1,..., h2 N The new assumption (A9) clearly replaces assumptions or Gauss-Markov conditions (A3) and (A4) You can see that the variance of ε depends upon the explanatory variables = E{ε X} = 0 [A10] 10/43

11 Alternative Estimator cont. [A10] = ε is conditionally mean zero independent of X We want to get a BLUE for β in the model with the new conditions [A9] and [A10] y i = x i β + ε i, i = 1,..., N (14) The structure of Ψ = an appropriate transformation matrix P is a diagonal matrix with elements h 1 1,..., h 1 N Typical elements in the transformed data vector P y are thus yi = y i /h i and xi = x i /h i Running OLS on: y i = x i β + ε i (15) y i h i = ( x i h i ) β + ε i h i (16) 11/43

12 Alternative Estimator cont. ε i is now homoskedastic. The resulting least square estimator is given by ˆβ = ( N h 2 i i=1 x i x i ) 1 N i=1 h 2 i x i y i (17) This is a special case of the estimator given in eq. (9) is referred to as the weighted least squares estimator It is a least squares estimator in which each observation is weighted by (a factor proportional to) the inverse of the error variance 12/43

13 Alternative Estimator cont. Under [A9] [A10] this estimator is BLUE The use of weights implies observations with a higher variance get a smaller weight in estimation. The greatest weights are given to those observations that provide the most accurate information about the model parameters, and the smallest weights to those that provide relatively little information about β V{ ˆβ} = σ 2 ( ˆσ 2 = 1 N K N i=1 N i=1 h 2 i x i x i ) 1 (18) h 2 i (y i x i ˆβ) 2 (19) 13/43

14 Alternative Estimator When the Variances are Unknown If the h i s in Eq. (13) are unknown, they should be replaced by their consistent estimator ˆβ = ( N i=1 2 ĥ i xi x i N ) 1 2 ĥ i xi y i (20) i=1 The corresponding estimator is the FGLS because it is based on the estimated values of h 2 i Under some weak regularity conditions, the FGLS and GLS estimators are asymptotically equivalent But in small sample, the FGLS outperforms OLS although it usually does The covariance matrix is given by V{ ˆβ } = ˆσ 2 ( N ĥ i 2 xi x i ) 1 (21) 14/43 Graduate i=1 Econometrics Lecture 4: Heteroskedasticity

15 Alternative Estimator When the Variances are Unknown cont. where ˆσ 2 is the standard estimator for the error variance estimated from ˆσ 2 = 1 N K N i=1 ĥ i 2 (yi x i ˆβ ) 2 (22) 15/43

16 Alternative Estimator Heteroskedasticity Consistent SE for OLS Reconsider the model with heteroskedastic errors With E{ε i X} = 0 and V{ε i X} = σ 2 i. y i = x i β + ε i (23) From eq. (6), the appropriate covariance matrix is given by V{b X} = ( N i=1 N x i x i ) 1 ( σi 2 x ix N i )( x i x i ) 1 (24) i=1 i=1 To estimate the covariance matrix, we need to estimate the σi 2 for which we need additional assumptions 16/43

17 Alternative Estimator Heteroskedasticity Consistent SE for OLS cont. According to White (1980), only a consistent estimator of the K K matrix is required 1 N σ2 i x ix i (25) Under very general conditions, it can be shown that S 1 N N i=1 e 2 i x ix i (26) Where e i is the OLS residual, is a consistent estimator for. 17/43

18 Alternative Estimator Heteroskedasticity Consistent SE for OLS cont. Therefore, ˆV{b} = ( N i=1 N x i x i ) 1 ( e 2 i x ix N i )( x i x i ) 1 (27) i=1 i=1 Can be used as an estimate of the true variance of the OLS estimator. Appropriate inferences based upon b could be made without actually specifying the type of heteroskedasticity Standard errors computed this was are called heteroskedasticity-consistent or White standard errors 18/43

19 Alternative Estimator Heteroskedasticity Consistent SE for OLS cont. They are very common in applied work to use such robust standard errors Can be estimated by using the robust option in Stata Good to report both the usual and the heteroskedasticity-consistent standard errors to show how sensitive the results are to the type of standard error used Robust standard errors are likely to be larger than homoskedastic ones 19/43

20 Multiplicative Heteroskedasticity A common form of heteroskedasticity The error variance is related to a number of exogenous variables, gathered in J dimensional vector z i (excluding the intercept) V{ε i x i } = σi 2 = σ 2 exp{α 1 z i α J z ij } = σ 2 exp{z i α} (28) where z i = a vector of observed variables that is a function of x i The error variance is therefore related to one or more exogenous variables To be able to compute the EGLS estimator, one needs consistent estimator for the unknown parameters in h 2 i = exp{z i α} i.e., for α which can be based up on the OLS residuals 20/43

21 Multiplicative Heteroskedasticity Cont. To see how, first note that log σi 2 = logσ 2 + z i α The OLS residuals e i = y i x i b have something to tell about σi 2 = loge 2 i = logσ 2 + z i α + υ i (29) Where υ i = log(e 2 i /z i) is an error term that is (asymptotically) homoskedastic and uncorrelated with z i However, υ i does not have zero expectation (even asymptotically) but this will affect estimation of the constant logσ 2 which is irrelevant Consequently, the EGLS estimator for β can be obtained along simple steps (we will se example later) 21/43

22 Formal Tests In order to claim that the standard errors generated from OLS are misleading due to heteroskedasticity, we need to perform a standard test A number of tests are available If these tests do not reject H 0 := heteroskedasticity, there is no problem in using OLS If we however reject H 0 :=, we should find a solution Use EGLS Use Heteroskedasticity-consistent standard errors for the OLS estimator Revise the specification of our model 22/43

23 Formal Tests Multiplicative Heteroskedasticity The alternative hypothesis for this test is eq. (28), i.e., σi 2 = exp{z i α} (30) where z i is a J dimensional vector. The null hypothesis of homoskedasticity corresponds to α = 0, so the problem under test is H 0 = α = 0 versus H 1 : α = 0 (31) H 0 can be tested using the results of the least square regression stated in eq.(29) using the standard F test for the hypothesis that all the parameters except the intercept are equal to zero Because υ i in (29) does not satisfy the Gauss-Markov conditions exactly, the F distribution (with J and N-J-1 d.f) holds by approximation 23/43

24 Formal Tests The Breusch-Pagan (BP) Test Provided by Breusch and Pagan (1980) σi 2 = σ 2 h(z i α) (32) where h is unknown, continuously differentiable function (that does not depend on i), such that h(.) > 0 and h(0) = 1 As a special case (if h(t) = exp{t}) one obtains eq.(30) A test for H 0 : α = 0 versus H 1 : α = 0 can be derived independently of the function h 24/43

25 Formal Tests The Breusch-Pagan (BP) Test cont. The simplest version of the BP test can be computed as the number of observations multiplied by the R 2 of an auxiliary regression (a regression of e 2 i, i.e., the OLS residuals) on z i and a constant The resulting test statistic given by ξ = NR 2 is asymptotically χ 2 distributed with J d.f The BP test is a Lagrange multiplier (LM) test for heteroskedasticity LM tests do not require the model to be estimated under alternative and that they are often simply computed from R 2 of some auxiliary regression (we will see them in the ML section) 25/43

26 Formal Tests The White Test We saw that the correct covariance matrix of the least squares estimator is given by ( N 1 ˆV{b} = s 2 x i x i) (33) i=1 If there is no heteroskedasticity, (33) will give a consistent estimator of V{b}, while if there is, it will not Obtain NR 2 in the regression of e 2 i on a constant and all (unique) first moments, second moments, and cross-products of the original regressors This test statistic is asymptotically distributed as χ 2 with P d.f, where P is the number of regressors in the auxiliary regression, excluding the intercept. 26/43

27 Formal Tests The White Test cont. The White test is a generalization of the BP test, which also involves an auxiliary regression of squared residuals but excludes any higher-order terms It can therefore detect more general forms of heteroskedasticity that the BP test Limitations: it may identify heteroskedasticity but it may instead simply identify some other specification error (e.g., incorrect functional form) If N is small, it may not have the power to detect heteroskedasticity 27/43

28 Formal Tests Which Test? Depends on how explicit one would want to be The more explicit we are, the more powerful the test will be = more likely it is that the test will correctly reject H 0 However, if the true heteroskedasticity is of different form, the chosen test may not indicate the presence of heteroskedasticity at all The White test has limited power against a large number of alternatives, while a specific test, like the one for multiplicative heteroskedasticity, has more power but only against a limited number of alternatives In some cases, a visual inspection of the residuals (e.g., a plot of OLS residuals against one or more exogenous variables, or the fitted values) or economic theory can help in choosing the appropriate alternative 28/43

29 Illustration Labour Demand We will consider a simple model of labor demand using cross-sectional data collected in 1996 from 569 Belgian firms List of variables labour : total employment (number of workers) capital : total fixed assets (in million euro) wage : total wage costs divided by number of workers (in 1000 euro) output : value added (in million euro) 29/43

30 Illustration Labour Demand Consider a simple production function Q = f (K, L) (34) where Q = output and K and L denote the capital and labour inputs respectively Total production costs = rk + wl, r = the costs of capital, w = the costs of labour (the wage rate) Minimizing total costs subject to the production function gives the demand functions for capital and labour 30/43

31 Illustration Labour Demand cont. The labour demand function is given by: for some function g L = g(q, r, w) (35) One approximates r by capital stock K since the former does not have variation in the cross-section 31/43

32 Illustration Labour Demand OLS. reg labor wage output capital Source SS df MS Number of obs = 569 F( 3, 565) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = labor Coef. Std. Err. t P> t [95% Conf. Interval] wage output capital _cons /43

33 Illustration Labour Demand BP Test- Aux. Reg.. reg e2 wage output capital Source SS df MS Number of obs = 569 F( 3, 565) = Model e e+12 Prob > F = Residual e e+09 R-squared = Adj R-squared = Total e e+10 Root MSE = e2 Coef. Std. Err. t P> t [95% Conf. Interval] wage output capital _cons /43

34 Illustration Labour Demand BP Test- Aux. Reg. High R 2 and t values = high possibility of heteroskedasticity Test statistic = N R 2 = Sound rejection of H 0 = homoskedasticity as the asymptotic distribution under H 0 is Chi-squared with 3 d.f Common to find heteroskedasticity in situations like this The size of the observed units differs substantially The sample for e.g., includes firms with one employee and firms with 1000 employees Larger firms have larger absolute values of all variables in the model, including the error term Consider a log-linear (cobb-douglass) model like: Q = AK α L β 34/43

35 Illustration Labour Demand - a Log-Linear Model Source SS df MS Number of obs = 569 F( 3, 565) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = lnlabor Coef. Std. Err. t P> t [95% Conf. Interval] lnwage lnoutput lncapital _cons /43

36 Illustration Labour Demand - a Log-Linear Model, BP Test We run the same auxiliary regression BP Test statistic = N R 2 = = 7.74 = rejection of H 0! at 5% (but at the margin) The data is still heteroskedastic! Let s consider the While Test next To compute the test statistic, we run an auxiliary regression of squared OLS residuals upon all original regressors, their squares and all their interactions 36/43

37 Illustration Labour Demand - a Log-Linear Model, White Test. reg le2 lnwage lnoutput lncapital lnwage2 lnoutput2 lncapital2 lnwlno lnwlnc lnclno Source SS df MS Number of obs = 569 F( 9, 559) = 7.12 Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = le2 Coef. Std. Err. t P> t [95% Conf. Interval] lnwage lnoutput lncapital lnwage lnoutput lncapital lnwlno lnwlnc lnclno _cons /43

38 Illustration Labour Demand - a Log-Linear Model, White Test The White test statistic = N R 2 = = Reject H 0 because the statistic is highly significant for a Chi-squared variable with 9 d.f The aux. regression shows that output and capital are statistically significant = 38/43

39 Illustration Labour Demand - a Log-Linear Model with White Standard Errors. reg lnlabor lnwage lnoutput lncapital, robust Linear regression Number of obs = 569 F( 3, 565) = Prob > F = R-squared = Root MSE = Robust lnlabor Coef. Std. Err. t P> t [95% Conf. Interval] lnwage lnoutput lncapital _cons /43

40 Illustration Labour Demand - a Log-Linear, testing for Multiplicative heteroskedasticity Set z i = x i The variance of the error term depends upon log(wage), log(output) and log(capital) Compute the log of the squared OLS residuals and then run a regression of log e 2 i upon z i and a constant This gives the auxiliary regression results on the next page 40/43

41 Illustration Labour Demand testing for Multiplicative heteroskedasticity. reg lme2 lnwage lnoutput lncapital Source SS df MS Number of obs = 569 F( 3, 565) = 4.73 Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = lme2 Coef. Std. Err. t P> t [95% Conf. Interval] lnwage lnoutput lncapital _cons /43

42 Illustration Labour Demand testing for Multiplicative heteroskedasticity We reject H 0 := homoskedasticity because the F-value for the auxiliary regression 4.73 > 2.6 (i.e., the critical value from the F-table at Was this specification for the form of heteroskedasticity is too restrictive? Check by running a version of the previous auxiliary regression where the three-squared terms are also included F-test from this reg = 1.85(p = 0.137) = H 0 is not rejected! 42/43

43 Illustration Labour Demand Weighted Least Square Results. wls0 lnlabor lnwage lnoutput lncapital, wvar(lnwage lnoutput lncapital) type(abse) noconst WLS regression - type: proportional to abs(e) (sum of wgt is e+03) Source SS df MS Number of obs = 569 F( 3, 565) = Model Prob > F = Residual R-squared = Adj R-squared = Total Root MSE = lnlabor Coef. Std. Err. t P> t [95% Conf. Interval] lnwage lnoutput lncapital _cons /43