Violation of OLS assumption - Heteroscedasticity

Transcription

1 Violation of OLS assumption - Heteroscedasticity What, why, so what and what to do? Lars Forsberg Uppsala Uppsala University, Department of Statistics October 22, 2014 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

2 Econometrics - Objectives and exam Violations of assumptions - Heteroscedasticity: What - Explain what is meant by heteroscedasticity Causes - Account for possible causes of heteroscedastic errors Consequences - Know what heteroscedasticity does to the OLS estimators (expectation and variance) Detection - Account for an informal way to detect heteroscedasticity (graphing) Detection - Account for two tests for heteroscedasticity (Explain the rationale behind them how they are done, and being able to perform them when given all the relevant numbers Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

3 Econometrics - Objectives and exam Violations of assumptions - Heteroscedasticity: Remedy - Know what to do in the presence of heteroscedasticity Explain the idea behind the Weighted Least Squares Perform a WLS estimation using scalar algebra (transformation of the regression equation and then OLS on the transformed equation) Interpret the parameters of the original equation after doing WLS Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

4 Heteroscedasticity - Questions Outline How to spell it? What - is Heteroscedasticity? Causes - How does Heteroscedasticity come about? Consequences - Is it a problem? In what way? When/in what situations? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

5 Violation of OLS assumption - Heteroscedasticity - Questions Outline (cont.) Detection - How do we know if there is a Heteroscedasticity problem? (Informal methods, graphs) Detection - How to test for Heteroscedasticity? Remedial measures - What can we do about it? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

6 OLS - Assumptions - Violations - Heteroscedasticity - How to spell it Heteroscedasticity Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

7 Heteroscedasticity - The word? The assumption here is Homo-Scedasticity Constant- Equal- Same- Variance Variation Spread Homo- Scedasticity That is, constant variance of the Error Term (for the di erent values of the regressor(s)) Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

8 Heteroscedasticity - Animal? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

9 Heteroscedasticity - Animal? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

10 Heteroscedasticity - What? To keep it simple, we have the single linear regression, for a arbitrary observation Y i = β 1 + β 2 X 2,i + u i The assumption (for OLS) is Homoscedasticity Var (u i ) = σ 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

11 Heteroscedasticity - What? Recall de ntion of variance Var (u i ) = E [u i E (u i )] 2 recall assumption E (u i ) = 0 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

12 Heteroscedasticity - What? We can write Var (u i ) = E [u i 0] 2 Var (u i ) = E ui 2 So the assumption can be written E u 2 i = σ 2 Note no i on the sigma-two. Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

13 Heteroscedasticity - What? If not homoscedasticity, then we have heteroscedasticity E u 2 i = σ 2 i i.e. it depends on i where the i refers to the di erent X i Remember the income - consumption example. Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

14 Heteroscedasticity - Why? Why do we have heteroscedasticity? When there are natural constraints on the variables, Consumption - Income Learning, e.g. number of errors vs time practicing Improved data quality Over time Among the X s Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

15 Heteroscedasticity - Why? Why do we have heteroscedasticity? (cont) Outliers Misspeci cation of model Skewness of variables Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

16 Heteroscedasticity - OLS Consequences Should we worry? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

17 Heteroscedasticity - OLS Consequences Should we worry? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

18 Heteroscedasticity - OLS Consequences - Expectation OLS: In the presence of Heteroscedastiticy, the OLS-estimator is still UNBIASED That is, it is on average correct The sampling distribution of the estimator bβ j is centered around the true value β j Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

19 Heteroscedasticity - Consequences - Variance Even though the OLS-estimator is unbiased it no longer have minimum variance It is NOT BLUE Not Best, in the sense it has minimun variance among, Linear Unbiased Estimators (It is only LUE - Note - Not standard notation...) Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

20 Heteroscedasticity - Consequences - Variance - Y-bar To see what happens to the variance in the presence of heterosceasticity, let us study the simplest case possible, let where Y i = β + u i u i N 0, σ 2 That is, a regression on only a constant. Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

21 Heteroscedasticity - Consequences - Variance - Y-bar We know that the OLS estimator of β, that is bβ, will be Ȳ, and that it is unbiased, i.e. E (Ȳ ) = β Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

22 Heteroscedasticity - Consequences - Variance - Y-bar Let s study the variance V (Ȳ ) = E [Ȳ E (Ȳ )] 2 V (Ȳ ) = E (Ȳ β) 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

23 Heteroscedasticity - Consequences - Variance - Y-bar Use de nition of the sample mean V (Ȳ ) = E " n 1 n Y i i=1 β # 2 because we have V (Ȳ ) = E Y i = β + u i " # 2 n 1 n (β + u i ) β i=1 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

24 Heteroscedasticity - Consequences - Variance - Y-bar (Again) V (Ȳ ) = E " 1 n! n (β + u i ) i=1 Split up sum and since β is a constant n i=1 β = nβ, we have V (Ȳ ) = E " 1 n nβ + n i=1 u i! β β # 2 # 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

25 Heteroscedasticity - Consequences - Variance - Y-bar Multiply in 1 n to get V (Ȳ ) = E V (Ȳ ) = E nβ n + n i=1 u i n β + n i=1 u i n 2 β 2 β Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

26 Heteroscedasticity - Consequences - Variance - Y-bar (again) V (Ȳ ) = E V (Ȳ ) = E β + n i=1 u i n n i=1 u 2 i n 2 β Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

27 Heteroscedasticity - Consequences - Variance - Y-bar Take the constant 1 n outside the square, and thus, square it, we have V (Ȳ ) = " 1 2 n # 2 E u i n i=1 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

28 Heteroscedasticity - Consequences - Variance - Y-bar Studying again, we have the situation " n # 2 u i i=1 (a + b + c) 2 = a 2 + b 2 + c 2 + 2ab + 2ac + 2bc but now with, not three, but n terms... " n # 2 u i = [u 1 + u u n ] 2 i=1 = u u u 2 n + 2u 1 u 2 + 2u 1 u 3... Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

29 Heteroscedasticity - Consequences - Variance - Y-bar Again V (Ȳ ) = V (Ȳ ) = 1 n 2 E " n " 1 2 n # 2 E u i n i=1 i=1 u 2 i + 2 n i 1 i=2 j=1 u i u j # Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

30 Heteroscedasticity - Consequences - Variance - Y-bar It can be instructive to study 2 n i 1 i=2 j=1 u i u j = 2u 2 u 1 + 2u 3 u 1 + 2u 3 u 2 + 2u 4 u 1 + 2u 4 u 2 + 2u 4 u 3 + 2u 5 u Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

31 Heteroscedasticity - Consequences - Variance - Y-bar Split up the expectation, (again) V (Ȳ ) = 1 n 2 E " n i=1 u 2 i + 2 n i 1 i=2 j=1 u i u j # V (Ȳ ) = 1 n 2 " n i=1 E u 2 i + 2 n i 1 i=2 j=1 E (u i u j ) # Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

32 Heteroscedasticity - Consequences - Variance - Y-bar Again study We assume that that is, we have...? n i 1 i=2 j=1 E (u i u j ) E (u i u j ) = 0, i 6= j Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

33 Heteroscedasticity - Consequences - Variance - Y-bar So (again) V (Ȳ ) = 1 n 2 " n i=1 V (Ȳ ) = 1 n 2 " n i=1 E E u 2 i u 2 i + 2 n + 2 n i 1 i=2 j=1 i 1 i=2 j=1 E (u i u j ) 0 # # Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

34 Heteroscedasticity - Consequences - Variance - Y-bar That is V (Ȳ ) = 1 n 2 " n i=1 V (Ȳ ) = 1 n 2 " n i=1 E E ui 2 ui # # Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

35 Heteroscedasticity - Consequences - Variance - Y-bar Since now, (under heteroscedasticity) E u 2 i = σ 2 i Again V (Ȳ ) = 1 n 2 " n i=1 V (Ȳ ) = 1 n n 2 σ 2 i i=1 E u 2 i # Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

36 Heteroscedasticity - Consequences - Variance - Y-bar So in the model Y i = β + u i the variance of the sample mean (the OLS-estimator) is given by V (Ȳ ) = V b β = n i=1 σ 2 i n 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

37 Heteroscedasticity - Consequences - Variance - Y-bar Note that if σ 2 i would be the same for all i that is, we would have (homoscedasticity) E = σ 2 Then u 2 i V (Ȳ ) = 1 n 2 " n i=1 V (Ȳ ) = 1 n 2 " n i=1 E σ 2 # u 2 i # Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

38 Heteroscedasticity - Consequences - Variance - Y-bar and we know that, for all constants n c = n c i=1 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

39 Heteroscedasticity - Consequences - Variance - Y-bar So, we have V (Ȳ ) = 1 n 2 " n i=1 σ 2 # = 1 n 2 n σ 2 = nσ2 n 2 V (Ȳ ) = σ2 n which is the "usual" variance of the sample mean. Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

40 Heteroscedasticity - Consequences - Variance - Slope est. Now, back to the (simple linear) regression model Y i = β 1 + β 2 X i + u i For the OLS estimator of β 2 : 2 V ˆβ 2 = E ˆβ 2 E ˆβ 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

41 Heteroscedasticity - Consequences - Variance - Slope est. Recall ˆβ 2 = β 2 + n j=1 (X i X ) u i (X X ) 2 and E ˆβ 2 = β2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

42 Heteroscedasticity - Consequences - Variance - Slope est. So V ˆβ 2 h 2 i = E ˆβ 2 E ˆβ 2 V ˆβ = E 4@β 2 + n j=1 (X i X ) u i (X X ) 2 β A 5 2 V ˆβ 2 = E 4@ n j=1 (X i X ) u i (X X ) 2 A 5 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

43 Heteroscedasticity - Consequences - Variance - Slope est. V ˆβ 2 = E 4 n j=1 (X i X ) u i 7 (X X ) V E h( n i=1 (X i X ) u i ) 2i ˆβ 2 = (X X ) 2 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

44 Heteroscedasticity - Consequences - Variance - Slope est. Numerator = E Numerator = E 2 4 n i=1 " n n i=1 j=1! 3 2 (X i X ) u i 5 (X i X ) (X j X ) u i u j # Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

45 Heteroscedasticity - Consequences - Variance - Slope est. Study numerator (Num.) 2! 3 2 Num. = E 4 n (X i X ) u i 5 i=1 Num. = E Num. = E " n " n i=1 j=1 (X i X ) (X j X ) u i u j # n (X i X ) 2 ui 2 i=1! + n i 1 i=2 j=1 2 (X i X ) (X j X ) u i u j!# Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

46 Heteroscedasticity - Consequences - Variance - Slope est. Split up expecation Num. = E n (X i X ) 2 ui 2 i=1! + E 2 n i 1 i=2 j=1 (X i X ) (X j X ) u i u j! Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

47 Heteroscedasticity - Consequences - Variance - Slope est. The Xi 0 s are constants, so we can move them out of the expectation, (again)!! n Num. = E (X i X ) 2 ui 2 n i 1 + E 2 (X i X ) (X j X ) u i u j i=1 i=2 j=1 Num. = n (X i X ) 2 E u 2 n i + 2 i=1 i=2 i 1 j=1 (X i X ) (X j X ) E (u i u j ) Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

48 Heteroscedasticity - Consequences - Variance - Slope est. Recall that assuming?, so E (u i u j ) = 0, i 6= j Num. = n (X i X ) 2 E u 2 n i + 2 i=1 i=2 i 1 j=1 (X i X ) (X j X ) E (u i u j ) Num. = n (X i X ) 2 E u 2 n i + 2 i=1 i=2 i 1 j=1 (X i X ) (X j X ) 0 Num. = n (X i X ) 2 E u 2 i + 0 i=1 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

49 Heteroscedasticity - Consequences - Variance - Slope est. and that now, (under heteroscedasticity) E u 2 i = σ 2 i so Num. = Num. = n (X i X ) 2 E u 2 i i=1 n (X i X ) 2 σ 2 i i=1 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

50 Heteroscedasticity - Consequences - Variance - Slope est. That was the numerator of the variance expcession. Recall V ˆβ 2 = E n i=1 n j=1 (X i X ) (X j X ) u i u j (X X ) 2 2 we just showed that E " n n i=1 j=1 (X i X ) (X j X ) u i u j # = n (X i X ) 2 σ 2 i i=1 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

51 Heteroscedasticity - Consequences - Variance - Slope est. Putting it together we get V ˆβ 2 = n i=1 (X i X ) 2 σ 2 i (X X ) 2 2 (Why cannot we just move the σ 2 i outside the summation?) Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

52 Heteroscedasticity - Consequences - Variance - Slope est. So the (correct) variance of the OLS-estimator for β 2 in the presence of heteroscedastiticy is given by V ˆβ 2 = n i=1 (X i X ) 2 σ 2 i (X X ) 2 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

53 Heteroscedasticity - Consequences - Variance - Slope est. Note that if σ 2 i = σ 2 for all i that is, if we would have homoscedasticity, we have V ˆβ 2 = (X i X ) 2 σ 2 i (X X ) 2 2 V ˆβ 2 = (X i X ) 2 σ 2 (X X ) 2 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

54 Heteroscedasticity - Consequences - Variance - Slope est. Recall here a a 2 = 1 a a = (X i X ) 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

55 Heteroscedasticity - Consequences - Variance - Slope est. so σ 2 (X i X ) 2 V ˆβ 2 = (X X ) 2 2 V ˆβ 2 = σ 2 (X X ) 2 V ˆβ 2 = σ 2 (X X ) 2 i.e. the "usual" variance of the estimator. Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

56 Heteroscedasticity - Consequences - Variance - Slope est. Compare the two variance expresstions Homo- Scedasticity Hetero- Scedasticity (A) (B) V ˆβ 2 = σ 2 V ˆβ (X X ) 2 2 = n i=1(x i X ) 2 σ 2 i ( (X X ) 2 ) 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

57 Heteroscedasticity - Consequences - Variance - Slope est. Now, unlike in the case of multicollinearity, where multicollinearity always in ates the variance, here there is no general result. So if we use V ˆβ 2 = σ 2 (X X ) 2 when we should use V ˆβ 2 = n i=1 (X i X ) 2 σ 2 i (X X ) 2 2 Do we under or over-estimate true variance? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

58 Heteroscedasticity - Consequences - Variance - Slope est. That is, is there a way of knowing n i=1 (X i X ) 2 σ 2 i (X X ) 2 > 2 σ 2 (X X ) 2 or n i=1 (X i X ) 2 σ 2 i (X X ) 2 < 2 σ 2 (X X ) 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

59 Heteroscedasticity - Consequences - Variance - Slope est. If we mistakenly ignore heteroscedasticity. What happens? For the following reasoning, introduce the (temporary) notation V TRUE ˆβ 2 = n i=1 (X i X ) 2 σ 2 i (X X ) 2 2 V FALSE ˆβ 2 = σ 2 (X X ) 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

60 Heteroscedasticity - Consequences - Variance - Slope est. So, in testing the signi cance of the individual parameters H 0 : β j = 0 H 1 : β j 6= 0 Using z obs = bβ j 0 σ b β j where σ b β j = s σ 2 (X X ) 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

61 Heteroscedasticity - Consequences - Variance - Slope est. Note that here we are dealing with population quantities (pretending to know σ 2, thus we use bβ j 0 z obs = since we assume that the error term is normal, and the estimator is a linear estimator, thus also normal. σ b β j Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

62 Heteroscedasticity - Consequences - Variance - Slope est. Of course in practice, we do not know σ 2 and need to estimate it by bσ 2 = Σbu2 i n k where, of course bu i = Y i b β 1 + bβ 2 X i,2 and k is the number of estimated coe cients in the regressrion equation, in this case 2. Then we use t obs = bβ j 0 bσ b β j t n k Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

63 Heteroscedasticity - Consequences - Variance - Slope est. If we UNDER-estimate the true variance, that is V Used but FALSE ˆβ 2 < VTRUE ˆβ 2 σ b β j bβ j σ b β j to SMALL + to BIG + Reject H 0 : β j = 0 to Often + Signi cance to Often Think model is "better" that it actually is Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

64 Heteroscedasticity - Consequences - Variance - Slope est. On the other hand, If we OVER-estimate the true variance, that is V Used but FALSE ˆβ 2 > VTRUE ˆβ 2 σ b β j bβ j σ b β j to BIG + to SMALL + Never Reject H 0 : β j = 0 + Never Signi cance Think model is "worse" that it actually is Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

65 Heteroscedasticity - Consequences - Variance - Slope est. In practice, unfortuently, there is no way of knowing if we under or over estimate the variance when ignoring heteroscedasticity. One would need to be careful in the analysis, and/or use Whites heteroscedastic robust standard errors. Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

66 Heteroscedasticity - Detection - Graphs Quick and dirty preliminary analysis: Just plot Y vs X Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

67 Heteroscedasticity - Detection - Tests Some tests of Heteroscedasticity, outline: Parks test Gleijser test Goldfeldt-Quandt test (GQ-test) Breusch-Pagan-Godfrey test (BPG-test) White s test Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

68 Heteroscedasticity - Detection - Tests - Parks test Parks test: σ 2 i = σ 2 X β 2,i ev ln bu i 2 = β1 + β 2 ln (X 2,i ) + v i Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

69 Heteroscedasticity - Detection - Tests - Gleiser test Gleiser test: jbu i j = β 1 + β 2 X 2,i + v i or other functions of X 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

70 Heteroscedasticity - Detection - Tests - BPG test Breusch-Pagan-Godfrey (BPG) test: BPG - Test: Idea Regress Test statistic bu 2 i ˆσ 2 = β 1 + β 2 X 2,i + β 3 X 3,i + e i ExplSS 2 χ 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

71 Heteroscedasticity - Detection - Tests - White s tests Run regression of f (u) on g (X ) White s test: Using square regresors bu 2 i = β 1 + β 2 X 2 + β 3 X β 4 X 3 + β 5 X e What problem could we encounter running this regression? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

72 Heteroscedasticity - Detection - Tests - White s tests To see if the residual variance increases with the increasing values of the regressors, we do a F test of H 0 : β j = 0, j > 1 H 1 : At least one β j 6= 0 j = 2,..., k Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

73 Heteroscedasticity - Detection - Tests - White s tests We can include cross-terms in the regression bu 2 i = β 1 + β 2 X 2 + β 3 X β 4 X 3 + β 5 X β 6 X 2 X 3 + e Again perform a F test of H 0 : β j = 0, j > 1 H 1 : At least one β j 6= 0 j = 2,..., k Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

74 Heteroscedasticity - Detection - Tests - White s tests Note: With many regressors What regressors to include in the auxiliry regression What functional form? What regressors to square? What, if any, crossproducts to include? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

75 Heteroscedasticity - Detection - Tests - Goldfeld-Quandt Golfeld-Quandt (GQ) test: σ 2 i = σ 2 X 2 i Y i = β 1 + β 2 X i + u i 1 Divide sample into two parts with respect to X 2 Estimate the model on each part 3 Test using RSS 1 /df 1 RSS 2 /df 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

76 Heteroscedasticity - Detection - Tests - Properties - Size Sample size 10 and 50: White, White + cross terms, BPG, F and χ 2 : Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

77 Heteroscedasticity - Detection - Tests - Properties - Power Power comparison Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

78 Heteroscedasticity - Remedy OK, we have heterosceasticity, what can we do? White s consistent estimator WLS (Weighted Least Squares) Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

79 Heteroscedasticity - Remedy - Whites HAC estimator Idea: Use bu 2 i as a proxy for σ 2 i \ V ˆβ 2 = n i=1 (X i X ) 2 bu 2 i (X X ) 2 2 use this estimate of the variance instead of the usual. Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

80 Heteroscedasticity - Remedy - WLS Instead of OLS, we should use the Weighted Least Squares (WLS)) Without matrix algebra notation, hard to show exactly what is going on. How can we "do" WLS without matrix algebra? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

81 Heteroscedasticity - Remedy - WLS Assume a functional form for E u 2 i = σ 2 i We usually assume that the increasing variance of u i is proportional to some function of (some) X. Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

82 Heteroscedasticity - Remedy - WLS Commonly used (assumed, mathematically convenient) functional forms: σ 2 i = σ 2 X 2 i σ 2 i = σ 2 jx i j σ 2 i = σ 2p X i Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

83 Heteroscedasticity - Remedy - WLS In general, if we have the variable u i that has the variance V (u i ) = cσ 2 What transform can we do on u i, that is, how can we standardize u i to get the standardized variable, say u i, to have variance σ 2? Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

84 Heteroscedasticity - Remedy - WLS Hint recall V (ay ) = a 2 V (Y ) Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

85 Heteroscedasticity - Remedy - WLS We try then u i = u i p c V (ui ui ) = V p c = 1 p c 2 V (u i ) = 1 c V (u i ) V (u i ) = V (u i ) c Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

86 Heteroscedasticity - Remedy - WLS Recall so (again) V (u i ) = cσ 2 V (u i ) = V (u i ) c = cσ2 c V (u i ) = σ 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

87 Heteroscedasticity - Remedy - WLS Back to the regression. How to implement, we start with Y i = β 1 + β 2 X i + u i Assuming σ 2 i = σ 2 jx i j Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

88 Heteroscedasticity - Remedy - WLS Do the transformation, that is divide by p jx i j to get Y p i jxi j = β 1 1 p jxi j + β X i 2 p jxi j + u i p jxi j Y i = β 1 X i + β 2 X i + u i Note: No constant in the "new", transformed regression line. Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

89 Heteroscedasticity - Remedy - WLS (again) Y i = β 1 X i study the "transformed" error term + β 2 X i u i = u i p jxi j + u i Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

90 Heteroscedasticity - Remedy - WLS Expectation E (u i ) = E u i p jxi j! = = 1 p jxi j E (u i ) 1 p jxi j 0 = 0 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

91 Heteroscedasticity - Remedy - WLS Study the variance of u i Var (u i ) = E (u i E (u i )) 2 = E (u i 0) 2 V (u i ) = E (u i ) 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

92 Heteroscedasticity - Remedy - WLS Focus on E (u i )2 Replace u i = u i p jxi j and we get E (u i ) 2 = E u i p jxi j! 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

93 Heteroscedasticity - Remedy - WLS Recall for any constant a E (a u i ) 2 = E ha 2 (u i ) 2i = a 2 E (u i ) 2 here a = 1 jx i j Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

94 Heteroscedasticity - Remedy - WLS (again) E (u i ) 2 = E u i p jxi j! 2 2 E (ui ) 2 = E 4! p (u i ) 2 5 jxi j Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

95 Heteroscedasticity - Remedy - WLS E (u i ) 2 = 1 pjxi 2 E ui j 2 = 1 jx i j E u2 i E (ui ) 2 = E u2 i jx i j Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

96 Heteroscedasticity - Remedy - WLS We have heteroscedasticity, so E u 2 i = σ 2 i That is (again) E (ui ) 2 = E u2 i jx i j E (u i ) 2 = σ2 i jx i j Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

97 Heteroscedasticity - Remedy - WLS Recall that we assumed that σ 2 i had the functional form σ 2 i = σ 2 jx i j We have, (again) E (u i ) 2 = σ2 i jx i j E (u i ) 2 = σ2 jx i j jx i j E (u i ) 2 = σ 2 Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

98 Heteroscedasticity - Remedy - WLS In the transformed regression, Y i = β 1 X i + β 2 X i + u i we have constant variance, i.e. homoscedasticity V (u i ) = σ 2 So, here, WLS is equivalent to using OLS on transformed data! Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

99 Heteroscedasticity - Remedy - WLS We started with but we estimate Y i Y i = β 1 + β 2 X i + u i = β 1 X i + β 2 X i + u i How do we interpret β 1 and β 2? Note that, in e ect, when tranforming the PRF, we transform the data - not the parameters, so in terms of the original regression, we keep the interpretation of the parameters. Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

100 Heteroscedasticity - Remedy - WLS What if we choose (assume) another functional form of the heterscedasticity, again, for the single linear regression Y i = β 1 + β 2 X i + u i Assuming given all X i > 0 σ 2 i = σ 2 X 2 i Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

101 Heteroscedasticity - Remedy - WLS Do q the transformation, we divide the terms of the PFR by that is divide by Xi 2 = X i to get Y i X i = β 1 1 X i + β 2 X i X i + u i X i Y i = β 1 X i + β 2 + u i Yi = β 2 + β 1 Xi + u i Note: The coe cients has changed "roles". It is straightforward to see that now Var (u i ) = σ2, i.e. constant. We have transformed away the heteroscedasticity Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

102 Heteroscedasticity - Remedy - WLS The transformed regression again Yi = β 2 + β 1 Xi + u i Once estimated, we interpret bβ 2 as the slope in the orginal regression Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103

103 Heteroscedasticity - Remedy - WLS For the WLS - some words of caution: What if we assume the wrong functional form? If we assume the wrong functional form, we might induce even "more" heteroscedasticity than we had to begin with, thus, causing more damage than good. So, if uncertain of what kind of functional form we have on the Lars Forsberg (Uppsala University) Hetero-sce-dasti-city October 22, / 103