Heteroskedasticity and Autocorrelation

Transcription

1 Lesson 7 Heteroskedasticity and Autocorrelation Pilar González and Susan Orbe Dpt. Applied Economics III (Econometrics and Statistics) Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 1 / 39

2 Learning objectives To understand the concepts of heteroskedasticiy and serial correlation. To identify the consequences of the presence of heteroskedasticiy and/or serial correlation on the properties of the OLS estimator. To identify the consequences of the presence of heteroskedasticity and/or serial correlation on the inference based on the OLS estimator. To detect the presence of heteroskedasticity and/or serial correlation. To carry out inference robust to the presence of heteroskedasticity and/or serial correlation based on the OLS estimator. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 2 / 39

3 Contents 1 Multiple Regression Model assumptions. 2 Heteroskedasticity. Concept. Consequences. Detection. 3 Autocorrelation. Concept. Consequences. Detection. 4 Inference using the OLS estimator. 5 Task: T7. 6 Exercises: E7.1, E7.2 and E7.3. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 3 / 39

4 Contents 1 Multiple Regression Model assumptions. 2 Heteroskedasticity. Concept. Consequences. Detection. 3 Autocorrelation. Concept. Consequences. Detection. 4 Inference using the OLS estimator. 5 Task: T7. 6 Exercises: E7.1, E7.2 and E7.3. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 4 / 39

5 Multiple Regression Model assumptions. Assumptions. A.1. The model in the population can be written as: Y t = β 1 + β 2X 2t + β 3X 3t β k X kt + u t t = 1,.., T A.2. No perfect collinearity A.3. Zero conditional mean: E(u t X 2, X 3,..., X k ) = 0 t = 1, 2,..., T A.4. Homoskedasticity (constant variance): Var(u t X 2, X 3,..., X k ) = σu 2 t = 1, 2,..., T. A.5. No autocorrelation: Cov(u t, u s X 2, X 3,..., X k ) = 0 t s. A.6. Normality: The errors u t are independent of X and they are identically normally distributed. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 5 / 39

6 Multiple Regression Model assumptions. Matrix form. A.1. The model in the population can be written as: Y = Xβ + u A.2. No perfect collinearity A.3. Zero conditional mean: E(u X) = 0 A.4. + A.5. Homoskedasticity (constant variance) + No autocorrelation: σu σu V (u X) = E(uu X) = 0 0 σu = σu 2 I T σu 2 A.6. Normality. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 6 / 39

7 Multiple Regression Model assumptions. Under these assumptions, conditional on X: A. The OLS estimator ˆβ, is linear unbiased efficient in the Gauss-Markov sense. B. The estimator of the variance of the error term, ˆσ 2 u, is unbiased. C. ˆβ N(β, V ( ˆβ)) where V ( ˆβ) = σ 2 u (X X) 1 D. Test statistics: t = ˆβ j βj 0 (SSRR SSRUR)/q H0 t(t k) F = ˆσ ˆβj SSR UR/(T k) H0 F(q, T k) Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 7 / 39

8 Contents 1 Multiple Regression Model assumptions. 2 Heteroskedasticity. Concept. Consequences. Detection. 3 Autocorrelation. Concept. Consequences. Detection. 4 Inference using the OLS estimator. 5 Task: T7. 6 Exercises: E7.1, E7.2 and E7.3. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 8 / 39

9 Heteroskedasticity. Concept. Homoskedasticity = The variance of the disturbances u i is constant i, conditional on X. Heteroskedasticity = The variance of the disturbances u i is not constant i, conditional on X, because it depends on one or several variables. The presence of heteroskedasticity is quite frequent when working with cross-section data. Let s assume a regression model that determines consumption as a function of income. The variance of the error term might be expected to increase as income increases. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 9 / 39

10 Heteroskedasticity. Homoskedastic error term F(Y X) Y F(Y X=X 0 ) F(Y X=X 1 ) F(Y X=X 2 ) F(Y X=X 3 ) F(Y X=X 4 ) X Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 10 / 39

11 Heteroskedasticity. Heteroskedastic error term F(Y X) Y F(Y X=X X 0 ) F(Y X=X 1 ) F(Y X=X 2 ) F(Y X=X 3 ) F(Y X=X 4 ) X Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 11 / 39

12 Heteroskedasticity. Concept. Heteroskedasticity V (u i X) = σ 2 i i = 1, 2,..., N The covariance matrix of the error term conditional on X is: σ σ V (u X) = E(uu X) = 0 0 σ σn 2 This covariance matrix will be denoted by V (u). = Σ Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 12 / 39

13 Heteroskedasticity. Properties of the OLS estimator. Conditional on X, the OLS estimator is: Linear: ˆβ = (X X) 1 X Y = β + (X X) 1 X u Unbiased: E( ˆβ X) = E[(β + (X X) 1 X u) X] = β Not Efficient. Given that the error term in heteroskedastic, assumption A4 is not satisfied and it is not possible to apply the Gauss-Markov theorem. ˆβ OLS has NOT the smallest variance within the class of linear and unbiased estimators. It can be proved that there is another estimator with a smaller variance when the structure of heteroskedasticity is known: Generalized Least Squares Estimator (GLS): uses the information about the covariance matrix of u to estimate the coefficients β. The derivation of this estimator is out of the scope of this book. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 13 / 39

14 Heteroskedasticity. Inference using the OLS estimator, conditional on X. When the assumptions of the GLRM are satisfied: ˆβ N(β, σ 2 (X X) 1 ). When the error term is heteroskedastic, the covariance matrix of the error term is V (u) = Σ, then what is the distribution of the OLS estimator??? ˆβ N[β, (X X) 1 X ΣX(X X) 1 ] Proof: V ( ˆβ X) = E[( ˆβ E( ˆβ))( ˆβ E( ˆβ)) X] = E[( ˆβ β)( ˆβ β) X] = = E[((X X) 1 X u)((x X) 1 X u) X] = E[(X X) 1 X uu X(X X) 1 X] = = (X X) 1 X ΣX(X X) 1 Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 14 / 39

15 Heteroskedasticity. Inference using the OLS estimator conditional on X. Hypothesis testing based on the t and F statistics is NOT adequate and it can be misleading. The problem is that the usual estimator of V ( ˆβ) is not appropriate because it is BIASED. ˆV ( ˆβ) = ˆσ 2 (X X) 1 The t and F statistics do not follow the usual Student-t and F-Snedecor distributions. t = ˆβ j βj 0 H 0 (SSR R SSR UR )/q??? F = ˆσ ˆβj SSR UR /(N k) H 0??? What are the distributions of the t and F statistics??? Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 15 / 39

16 Heteroskedasticity. Detection. A. Graphic analysis. Estimate the model by OLS and compute the residuals. Plot the OLS residuals against the regressors that may be the cause of the heteroskedasticity. The plots below show the behaviour of the error term under the homoskedasticity assumption and the behaviour of a heteroskedastic error term. u i u i 0 0 Homoskedasticity X i Heteroskedasticity X i Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 16 / 39

17 Heteroskedasticity. Detection. B. Heteroskedasticity tests. Goldfeld-Quandt test. Breusch-Pagan test. White test. Consider the linear regression model: Y i = β 1 + β 2 X 2i + β 3 X 3i β k X ki + u i i = 1, 2,..., N. The set-up of the tests to detect heteroskedasticity is always the same: H 0 : Homoskedasticity: V (u i ) = σi 2 = σ2 i H a : Heteroskedasticity: V (u i ) = σi 2 See Example 7.1 for applications. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 17 / 39

18 Heteroskedasticity. Goldfeld-Quandt test. This simple test is used when it is believed that only one variable Z, usually one of the regressors, is the cause of the heteroskedasticity. That is, it assumes that the heteroskedasticity takes the form σi 2 = E(u i) 2 = h(z) where h is an increasing (decreasing) function of Z. Procedure: 1. Identify the variable Z (usually one of the regressors) that is related with σ 2 i. 2. Sort the observations by Z (increasing/decreasing). 3. Split the sample in three parts: the first N 1 observations, the last N 2 observations and delete the central observations. Usually, one third of the total number observations is deleted. 4. Estimate by OLS two regressions: one using the first N 1 observations and another one using the last N 2 observations. SSR2/(N2 k2) 5. Test statistic: GQ = SSR 1/(N 1 k 1) H 0 F((N 2 k 2), (N 1 k 1)) where SSR 1 and SSR 2 are the sum of squared residuals of the first and second regressions, respectively. 6. Reject the null hypothesis of homoskedasticity at the α % significance level if: GQ > F α((n 2 k 2), (N 1 k 1)) Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 18 / 39

19 Heteroskedasticity. Breusch-Pagan test. This test is based in the assumption that the heteroskedasticity takes the form σ 2 i = E(u i) 2 = h(z i α), z i = [1 z1i,..., zpi] is a vector of known variables, α = [α 0 α 1,..., α p] is a vector of unknown coefficients and h( ) is any function that only takes positive values. The null hypothesis of homoskedasticity is: H 0 : α 1 = α 3 =... = α p = 0 Procedure: 1. Estimate the regression model by OLS, compute the residuals û and estimate the variance of the error term as follows: σ 2 = û 2 /N. 2. Regress û 2 i / σ2 on z i by OLS and compute the explained sum of squares (SSE). 3. Test statistic: LM = SSE 2 H 0,a χ 2 (p) 4. Reject the null hypothesis of homoskedasticity at the α % significance level if: LM = SSE/2 > χ 2 α (p) Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 19 / 39

20 Heteroskedasticity. White test. This is a very flexible test because it is not necessary to make any assumption about the structure of homoskedasticity. In this sense, it is said to be a robust test. Procedure: 1. Estimate the regression model by OLS and compute the residuals û. 2. Undertake an auxiliary regression analysis. Regress the squared residuals from the original regression model onto a set of regressors that contain the original regressors, the cross-products of the regressors and the squared regressors. Compute the coefficient of determination of this auxiliary regression. 3. Test statistic: LM = NR 2 H 0,a χ 2 (p) where R 2 is the coefficient of determination of the auxiliary regression, N is the sample size and p is the number of estimated coefficients in the auxiliary regression minus Reject the null hypothesis of homoskedasticity at the α % significance level if: NR 2 > χ 2 α (p). Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 20 / 39

21 Contents 1 Multiple Regression Model assumptions. 2 Heteroskedasticity. Concept. Consequences. Detection. 3 Autocorrelation. Concept. Consequences. Detection. 4 Inference using the OLS estimator. 5 Task: T7. 6 Exercises: E7.1, E7.2 and E7.3. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 21 / 39

22 Autocorrelation. Concept. Autocorrelation : there are linear relationships among the error terms for different observations: cov(u t u s X) = E(u t u s X) 0 t, s t s The presence of autocorrelation in the error term is quite frequent when working with time series data. The covariance matrix of the error term is: V (u X) = E(uu X) = This covariance will be denoted by V (u). σ 2 σ 12 σ σ 1T σ 21 σ 2 σ σ 2T σ 31 σ 32 σ 2... σ 3T σ T 1 σ T 2 σ T 3... σ 2 Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 22 / 39 = Σ

23 Autocorrelation. Consequences of autocorrelation. A. Properties of the OLS estimators, conditional on X: The OLS estimator is linear, unbiased, but it is Not Efficient, in the sense of minimum variance because the Gauss-Markov theorem cannot be applied under autocorrelation. It can be proved that there is another estimator with a smaller variance when the structure of the autocorrelation is known: Generalized Least Squares Estimator (GLS). B. Inference using the OLS estimator. When assumption A.5 is not satisfied and the error term is autocorrelated, the true covariance matrix of the OLS estimator conditional on X is: V ( ˆβ) = (X X) 1 X ΣX(X X) 1 As a consequence, the true distribution of the t and F statistics used for testing linear restrictions is unknown. Inference based on the usual Student-t and F-Snedecor distributions is not adequate and may lead to wrong conclusions. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 23 / 39

24 Autocorrelation. Detection. A. Graphic analysis. Procedure: estimate by OLS the regression model and compute the residuals û. Plot the OLS residuals against time to analyse whether there is any pattern in the evolution of the error term that may suggest the presence of autocorrelation. The plot below shows the temporal evolution of the error terms when there is: Positive autocorrelation: clusters of positive and negative error terms. Negative autocorrelation: the error terms alternate sign, negative, positive, negative,... u t u t 0 t 0 t Positive autocorrelation Negative autocorrelation Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 24 / 39

25 Autocorrelation. Detection. B. Autocorrelation tests. 1. Durbin-Watson test (only valid for determinist regressors). 2. Breusch-Godfrey test. Consider the linear regression model: Y t = β 1 + β 2 X 2t + β 3 X 3t β k X kt + u t t = 1, 2,..., T. The set-up of the autocorrelation tests is: H 0 : NO autocorrelation (cov(u tu s) = E(u tu s) = 0, t s) H a : Autocorrelation (cov(u tu s) = E(u tu s) 0, t s) It is ALWAYS necessary to specify the autocorrelation structure under the alternative hypothesis. See Example 7.2 for applications. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 25 / 39

26 Autocorrelation. Durbin-Watson test. H a : First order autocorrelation in the error term, that is, the error term follows an autorregresive process of order 1: AR(1): u t = ρ u t 1 + v t v t NID(0, σ 2 v) ρ < 1 The presence of autocorrelation depends on the value of ρ. ρ = 0 : No autocorrelation ρ > 0 : Positive autocorrelation ρ < 0 : Negative autocorrelation Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 26 / 39

27 Autocorrelation. Durbin-Watson test. Positive autocorrelation. Procedure: { H0 : ρ = 0 (NO autocorrelation) 1. H a : u t = ρ u t 1 + v t ρ > 0 (Positive autocorrelation) T (û t û t 1) 2 2. Test statistic: DW = t=2 T H 0??? û 2 t t=1 where û t are the OLS residuals of the regression model. T û tû t 1 It can be proved that: DW 2 (1 ˆρ) where: ˆρ = t=2 T û 2 t 1 Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 27 / 39 t=2

28 Autocorrelation. Durbin-Watson test. Positive autocorrelation. { ˆρ = 0 DW 2 Therefore: ˆρ (0, 1] DW [0, 2) 3. Decision rule: Durbin and Watson did not derive the exact distribution of the test statistic under the null because this distribution depends on the sample. But they estimated the critical values depending on the sample size and the number of regressors of the regression model for a given level of significance. They were able to estimate an upper limit (d U ) and a lower limit (d L). If DW < d L, the null hypothesis is rejected: there is evidence in the sample of first order positive autocorrelation. If DW > d U, the null hypothesis is not rejected: there is no evidence in the sample of first order positive autocorrelation. If d L < DW < d U, the test is inconclusive. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 28 / 39

29 Autocorrelation. Durbin-Watson test. Positive autocorrelation. d L d U 0 2 Reject H d Evidence of positive autocorrelation Uncertainty region DO NOT Reject H d NO autocorrelation Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 29 / 39

30 Autocorrelation. Durbin-Watson test. Negative autocorrelation. Procedure: 1. { H0 : ρ = 0 (NO autocorrelation) H a : u t = ρ u t 1 + v t ρ < 0 (Negative autocorrelation) 2. Test statistic DW = T (û t û t 1) 2 t=2 T H 0??? t=1 û 2 t It can be proved: DW 2 (1 ˆρ) { ˆρ = 0 DW 2 Therefore: ˆρ (0, 1] DW (2, 4] Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 30 / 39

31 Autocorrelation. Durbin-Watson test. Negative autocorrelation. Durbin-Watson estimated the critical values depending on the sample size and the number of regressors of the model for a given significance level. They estimated only an upper limit (d U ) and a lower limit (d L ). If DW > 4 d L, the null hypothesis is rejected: there is evidence in the sample of first order negative autocorrelation. If 2 < DW < 4 d U, the null hypothesis is not rejected: there is no evidence in the sample of first order negative autocorrelation. If 4 d U < DW < 4 d L, the test is inconclusive. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 31 / 39

32 Autocorrelation. Durbin-Watson test. Negative autocorrelation. 4 - d U 4 - d L 2 DO NOT Reject H d NO autocorrelation Uncertainty region Reject H d Evidence of negative autocorrelation 4 Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 32 / 39

33 Autocorrelation. Durbin-Watson test. d L d U 4 - d U 4 - d L 0 Z ject H d Evidence of positive autocorrelation Uncertainty region DO NOT Reject H d NO autocorrelation 2 4 Uncertainty region Z ject H d Evidence of negative autocorrelation Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 33 / 39

34 Autocorrelation. Breusch-Godfrey test. This test may be used to test the presence of first order autocorrelation but it is designed to test the presence of autocorrelation of higher orders. Assume that the error term follows an autorregresive model of order q: { H0 : ρ 1 = ρ 2 =... = ρ q = 0 (NO autocorrelation) H a : Autocorrelation of order q Procedure: 1. Estimate the regression model by OLS and compute the residuals û. 2. Estimate an auxiliary regression of û t on X 2t, X 3t,..., X kt, û t 1, û t 2,..., û t q 3. Test the overall significance of û t 1, û t 2,..., û t q using a statistic based on the Lagrange multiplier: LM = T R 2 H 0,a χ 2 (q) where R 2 is the coefficient of determination of the auxiliary regression. Reject the null hypothesis at the α % significance level if: LM > χ 2 α (q). Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 34 / 39

35 Contents 1 Multiple Regression Model assumptions. 2 Heteroskedasticity. Concept. Consequences. Detection. 3 Autocorrelation. Concept. Consequences. Detection. 4 Inference using the OLS estimator. 5 Task: T7. 6 Exercises: E7.1, E7.2 and E7.3. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 35 / 39

36 Heteroskedasticity and Autocorrelation. Inference using the OLS estimator conditional on X. Is there any procedure to make inference using the OLS estimator? ˆβ N[β, (X X) 1 X ΣX(X X) 1 ] Idea : To estimate the true covariance matrix of the OLS estimator. V ( ˆβ) = (X X) 1 X ΣX(X X) 1 Not an easy task because usually little is known about the structure of matrix Σ. Some econometricians { White (heteroskedasticity) Newey-West (autocorrelation) have derived estimators for the covariance matrix V ( ˆβ) with good properties in large samples. } Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 36 / 39

37 Heteroskedasticity and Autocorrelation. Inference using the OLS estimator conditional on X. Denote by V R ( ˆβ) the estimator of the true covariance matrix of the OLS estimator of the coefficients β. This estimator is usually called: estimator robust to heteroskedasticity and/ or autocorrelation. A. Hypothesis on a single coefficient. It can be proved that the asymptotic distribution of the t-statistic under the null is: t = ˆβ j β 0 j ˆσ Rˆβj H 0,a N(0, 1) where ˆσ Rˆβj is the standard error of the OLS estimator robust to heteroskedasticity and/ or autocorrelation. The null hypothesis is rejected at the α % significance level if: t > N α/2 (0, 1). B. Hypothesis on multiple restrictions. It is necessary to multiply the usual F statistic by the number of restrictions to obtain a q F statistic that is going to follow asymptotically a χ 2 (q) distribution. The null hypothesis is rejected at the α % significance level if: q F > χ 2 α (q). Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 37 / 39

38 Contents 1 Multiple Regression Model assumptions. 2 Heteroskedasticity. Concept. Consequences. Detection. 3 Autocorrelation. Concept. Consequences. Detection. 4 Inference using the OLS estimator. 5 Task: T7. 6 Exercises: E7.1, E7.2 and E7.3. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 38 / 39

39 Contents 1 Multiple Regression Model assumptions. 2 Heteroskedasticity. Concept. Consequences. Detection. 3 Autocorrelation. Concept. Consequences. Detection. 4 Inference using the OLS estimator. 5 Task: T7. 6 Exercises: E7.1, E7.2 and E7.3. Pilar González and Susan Orbe OCW 2014 Lesson 7. Heteroskedasticity and Autocorrelation 39 / 39