Chapter 8 Heteroskedasticity

Transcription

1 Chapter 8 Walter R. Paczkowski Rutgers University Page 1

2 Chapter Contents 8.1 The Nature of 8. Detecting 8.3 -Consistent Standard Errors 8.4 Generalized Least Squares: Known Form of Variance 8.5 Generalized Least Squares: Unknown Form of Variance 8.6 in the Linear Model Page

3 8.1 The Nature of Page 3

4 8.1 The Nature of Consider our basic linear function: Eq. 8.1 E( y) 1 x To recognize that not all observations with the same x will have the same y, and in line with our general specification of the regression model, we let e i be the difference between the ith observation y i and mean for all observations with the same x i. Eq. 8. e y E( y ) y x i i i i 1 i Page 4

5 8.1 The Nature of Our model is then: Eq. 8.3 y x e i 1 i i Page 5

6 8.1 The Nature of The probability of getting large positive or negative values for e is higher for large values of x than it is for low values A random variable, in this case e, has a higher probability of taking on large values if its variance is high. We can capture this effect by having var(e) depend directly on x. Or: var(e) increases as x increases Page 6

7 8.1 The Nature of When the variances for all observations are not the same, we have heteroskedasticity The random variable y and the random error e are heteroskedastic Conversely, if all observations come from probability density functions with the same variance, homoskedasticity exists, and y and e are homoskedastic Page 7

8 8.1 The Nature of FIGURE 8.1 Heteroskedastic errors Page 8

9 8.1 The Nature of When there is heteroskedasticity, one of the least squares assumptions is violated: E e e e e ( i ) 0 var( i ) cov( i, j ) 0 Replace this with: Eq. 8.4 var( y ) var( e ) h( x ) i i i where h(x i ) is a function of x i that increases as x i increases Page 9

10 8.1 The Nature of Example from food data: yˆ x We can rewrite this as: eˆ y x i i i Page 10

11 8.1 The Nature of FIGURE 8. Least squares estimated food expenditure function and observed data points Page 11

12 8.1 The Nature of is often encountered when using cross-sectional data The term cross-sectional data refers to having data on a number of economic units such as firms or households, at a given point in time Cross-sectional data invariably involve observations on economic units of varying sizes Page 1

13 8.1 The Nature of This means that for the linear regression model, as the size of the economic unit becomes larger, there is more uncertainty associated with the outcomes y This greater uncertainty is modeled by specifying an error variance that is larger, the larger the size of the economic unit Page 13

14 8.1 The Nature of is not a property that is necessarily restricted to cross-sectional data With time-series data, where we have data over time on one economic unit, such as a firm, a household, or even a whole economy, it is possible that the error variance will change Page 14

15 8.1 The Nature of Consequences for the Least Squares Estimators There are two implications of heteroskedasticity: 1. The least squares estimator is still a linear and unbiased estimator, but it is no longer best There is another estimator with a smaller variance. The standard errors usually computed for the least squares estimator are incorrect Confidence intervals and hypothesis tests that use these standard errors may be misleading Page 15

16 8.1 The Nature of Consequences for the Least Squares Estimators Eq. 8.5 What happens to the standard errors? Consider the model: yi 1xi ei var( ei ) The variance of the least squares estimator for β as: Eq. 8.6 var( b ) N i1 ( x x) i Page 16

17 8.1 The Nature of Consequences for the Least Squares Estimators Eq. 8.7 Now let the variances differ: Consider the model: y 1 x e var( e ) i i i i i The variance of the least squares estimator for β is: Eq. 8.8 var( b ) N ( ) N xi x i i1 wi i N i1 ( xi x) i1 Page 17

18 8.1 The Nature of Consequences for the Least Squares Estimators If we proceed to use the least squares estimator and its usual standard errors when: var ei i we will be using an estimate of Eq. 8.6 to compute the standard error of b when we should be using an estimate of Eq. 8.8 The least squares estimator, that it is no longer best in the sense that it is the minimum variance linear unbiased estimator Page 18

19 8. Detecting Page 19

20 8. Detecting There are two methods we can use to detect heteroskedasticity 1. An informal way using residual charts. A formal way using statistical tests Page 0

21 8. Detecting 8..1 Residual Plots If the errors are homoskedastic, there should be no patterns of any sort in the residuals If the errors are heteroskedastic, they may tend to exhibit greater variation in some systematic way This method of investigating heteroskedasticity can be followed for any simple regression In a regression with more than one explanatory variable we can plot the least squares residuals against each explanatory variable, or against, y, to see if they vary in a ˆi systematic way Page 1

22 8. Detecting FIGURE 8.3 Least squares food expenditure residuals plotted against income 8..1 Residual Plots Page

23 8. Detecting 8.. LaGrange Multiplier Tests Eq. 8.9 Let s develop a test based on a variance function Consider the general multiple regression model: E y β β x β x 1 i i K ik A general form for the variance function related to Eq. 8.9 is: Eq var yi i E ei h 1 zi S zis This is a general form because we have not been specific about the function h Page 3

24 8. Detecting 8.. LaGrange Multiplier Tests Two possible functions for Exponential function: h are: Eq h z z exp z z 1 i S is 1 i S is Linear function: Eq. 8.1 h z z z z 1 i S is 1 i S is In this latter case one must be careful to ensure h 0 Page 4

25 8. Detecting 8.. LaGrange Multiplier Tests Notice that when then: 3 S 0 h z z h 1 i S is 1 But h 1 is a constant Page 5

26 8. Detecting 8.. LaGrange Multiplier Tests So, when: 3 S 0 heteroskedasticity is not present Page 6

27 8. Detecting 8.. LaGrange Multiplier Tests The null and alternative hypotheses are: Eq H H : : not all the in H are zero 1 i 0 S Page 7

28 8. Detecting 8.. LaGrange Multiplier Tests For the test statistic, use Eq and 8.1 to get: var yi i E ei 1 zi S zis Eq Letting we get v e E e i i i Eq e E e v z z v i i i 1 i S is i Page 8

29 8. Detecting 8.. LaGrange Multiplier Tests This is like the general regression model studied earlier: y E y e β β x β x e Eq i i i i K ik i Substituting the least squares residuals e for ˆi we get: e i Eq e z z v ˆi 1 i S is i Page 9

30 8. Detecting 8.. LaGrange Multiplier Tests Eq Since the R from Eq measures the proportion of variation in eˆi explained by the z s, it is a natural candidate for a test statistic. It can be shown that when H 0 is true, the sample size multiplied by R has a chi-square (χ ) distribution with S - 1 degrees of freedom: NR S 1 Page 30

31 8. Detecting 8.. LaGrange Multiplier Tests Important features of this test: It is a large sample test You will often see the test referred to as a Lagrange multiplier test or a Breusch-Pagan test for heteroskedasticity The value of the statistic computed from the linear function is valid for testing an alternative hypothesis of heteroskedasticity where the variance function can be of any form given by Eq Page 31

32 8. Detecting 8..a The White Test The previous test presupposes that we have knowledge of the variables appearing in the variance function if the alternative hypothesis of heteroskedasticity is true We may wish to test for heteroskedasticity without precise knowledge of the relevant variables Hal White suggested defining the z s as equal to the x s, the squares of the x s, and possibly their cross-products Page 3

33 8. Detecting 8..a The White Test Suppose: β1 β β3 3 E y x x The White test without cross-product terms (interactions) specifies: z x z x z x z x Including interactions adds one further variable z x x 5 3 Page 33

34 8. Detecting 8..a The White Test The White test is performed as an F-test or using: NR Page 34

35 8. Detecting 8..b Testing the Food Expenditure Example We test H 0 : α = 0 against H 1 : α 0 in the variance function σ i = h(α 1 + α x i ) First estimate e x v by least squares Save the R which is: SSE R SST Calculate: ˆi 1 i i N R Page 35

36 8. Detecting 8..b Testing the Food Expenditure Example Since there is only one parameter in the null hypothesis, the χ-test has one degree of freedom. The 5% critical value is 3.84 Because 7.38 is greater than 3.84, we reject H 0 and conclude that the variance depends on income Page 36

37 8. Detecting 8..b Testing the Food Expenditure Example For the White version, estimate: e x x v ˆi 1 i 3 i i Test H 0 : α = α 3 = 0 against H 1 : α 0 or α 3 0 Calculate: N R p value 0.03 The 5% critical value is χ (0.95, ) = 5.99 Again, we conclude that heteroskedasticity exists Page 37

38 8. Detecting 8..3 The Goldfeld- Quandt Test Estimate our wage equation as: Eq WAGE EDUC 0.133EXPER 1.54METRO se Page 38

39 8. Detecting 8..3 The Goldfeld- Quandt Test Eq. 8.0a Eq. 8.0b The Goldfeld-Quandt test is designed to test for this form of heteroskedasticity, where the sample can be partitioned into two groups and we suspect the variance could be different in the two groups Write the equations for the two groups as: WAGE β β EDUC β EXPER β METRO e i 1,,, N Mi M1 M Mi M 3 Mi M 4 Mi Mi M WAGE β β EDUC β EXPER β METRO e i 1,,, N Ri R1 R Ri R3 Ri R4 Ri Ri R Test the null hypothesis: M R Page 39

40 8. Detecting 8..3 The Goldfeld- Quandt Test Eq. 8.1 Eq. 8. The test statistic is: F ˆ M M F NM KM, NR KR ˆR R Suppose we want to test: H : against H : 0 M R 1 M R When H 0 is true, we have: Eq. 8.3 F ˆ ˆ R M Page 40

41 8. Detecting 8..3 The Goldfeld- Quandt Test The 5% significance level are F Lc = F (0.05, 805, 189) = 0.81 and F uc = F (0.975, 805, 189) = 1.6 We reject H 0 if F < F Lc or F > F uc Page 41

42 8. Detecting 8..3 The Goldfeld- Quandt Test Using least squares to estimate (8.0a) and (8.0b) separately yields variance estimates: ˆ M R We next compute: ˆ ˆM F ˆ R Since.09 > F UC = 1.6, we reject H 0 and conclude that the wage variances for the rural and metropolitan regions are not equal Page 4

43 8. Detecting 8..3a The Food Expenditure Example With the observations ordered according to income x i, and the sample split into two equal groups of 0 observations each, yields: ˆ Calculate: ˆ ˆ F 3.61 ˆ Page 43

44 8. Detecting 8..3a The Food Expenditure Example Believing that the variances could increase, but not decrease with income, we use a one-tail test with 5% critical value F(0.95, 18, 18) =. Since 3.61 >., a null hypothesis of homoskedasticity is rejected in favor of the alternative that the variance increases with income Page 44

45 8.3 -Consistent Standard Errors Page 45

46 8.3 - Consistent Standard Errors Recall that there are two problems with using the least squares estimator in the presence of heteroskedasticity: 1. The least squares estimator, although still being unbiased, is no longer best. The usual least squares standard errors are incorrect, which invalidates interval estimates and hypothesis tests There is a way of correcting the standard errors so that our interval estimates and hypothesis tests are valid Page 46

47 8.3 - Consistent Standard Errors Under heteroskedasticity: Eq. 8.4 var b N xi x i i1 N i1 x i x Page 47

48 8.3 - Consistent Standard Errors A consistent estimator for this variance has been developed and is known as: White s heteroskedasticity-consistent standard errors, or robust standard errors, or Robust standard errors The term robust is used because they are valid in large samples for both heteroskedastic and homoskedastic errors Page 48

49 8.3 - Consistent Standard Errors For K =, the White variance estimator is: Eq. 8.5 var b N x ˆ i x ei N i1 N N xi x i1 Page 49

50 8.3 - Consistent Standard Errors For the food expenditure example: yˆ x White se incorrect se Page 50

51 8.3 - Consistent Standard Errors The two corresponding 95% confidence intervals for β are: c c White: b t se b ,13.87 Incorrect: b t se b ,14.45 Page 51

52 8.3 - Consistent Standard Errors White s estimator for the standard errors helps avoid computing incorrect interval estimates or incorrect values for test statistics in the presence of heteroskedasticity It does not address the other implication of heteroskedasticity: the least squares estimator is no longer best Failing to address this issue may not be that serious With a large sample size, the variance of the least squares estimator may still be sufficiently small to get precise estimates To find an alternative estimator with a lower variance it is necessary to specify a suitable variance function Using least squares with robust standard errors avoids the need to specify a suitable variance function Page 5

53 8.4 Generalized Least Squares: Known Form of Variance Page 53

54 8.4 Generalized Least Squares: Known Form of Variance Variance Proportional to x Eq. 8.6 Recall the food expenditure example with heteroskedasticity: y β β x e i 1 i i 0, var, cov, 0 E e e e e i j i i i i j To develop an estimator that is better than the least squares estimator we need to make a further assumption about how the variances σ i change with each observation Page 54

55 8.4 Generalized Least Squares: Known Form of Variance Variance Proportional to x Eq. 8.7 An estimator known as the generalized least squares estimator, depends on the unknown σ i To make the generalized least squares estimator operational, some structure is imposed on σ i One possibility: e var i i xi Page 55

56 8.4 Generalized Least Squares: Known Form of Variance 8.4.1a Transforming the Model We change or transform the model into one with homoskedastic errors: Eq. 8.8 y 1 x e i β1 β i i x x x x i i i i Page 56

57 8.4 Generalized Least Squares: Known Form of Variance 8.4.1a Transforming the Model Define the following transformed variables: Eq. 8.9 * yi * 1 * xi * ei yi, xi 1, xi = xi, ei = x x x x i i i i Our model is now: Eq y β x β x e * * * * i 1 i1 i i Page 57

58 8.4 Generalized Least Squares: Known Form of Variance 8.4.1a Transforming the Model The new transformed error term is homoskedastic: Eq e i 1 1 var e var var e x * i i i x x i i xi The transformed error term will retain the properties of zero mean and zero correlation between different observations Page 58

59 8.4 Generalized Least Squares: Known Form of Variance 8.4.1a Transforming the Model To obtain the best linear unbiased estimator for a model with heteroskedasticity of the type specified in Eq. 8.7: 1. Calculate the transformed variables given in Eq Use least squares to estimate the transformed model given in Eq The estimator obtained in this way is called a generalized least squares estimator Page 59

60 8.4 Generalized Least Squares: Known Form of Variance 8.4.1b Weighted Least Squares One way of viewing the generalized least squares estimator is as a weighted least squares estimator Minimizing the sum of squared transformed errors: N N N * i 1 ei xi ei i1 i1 xi i1 The errors are weighted by e 1 x i Page 60

61 8.4 Generalized Least Squares: Known Form of Variance 8.4.1c Food Expenditure Estimates Eq. 8.3 Applying the generalized (weighted) least squares procedure to our food expenditure problem: yˆ x i se i A 95% confidence interval for β is given by: t c ˆ βˆ se β ,13.6 Page 61

62 8.4 Generalized Least Squares: Known Form of Variance Grouped Data Another form of heteroskedasticity is where the sample can be divided into two or more groups with each group having a different error variance Page 6

63 8.4 Generalized Least Squares: Known Form of Variance 8.4.1b Weighted Least Squares Most software has a weighted least squares or generalized least squares option Page 63

64 8.4 Generalized Least Squares: Known Form of Variance 8.4.1b Weighted Least Squares For our wage equation, we could have: Eq. 8.33a Eq. 8.33b WAGE 1 EDUC 3EXPER e i 1,,, N Mi M Mi Mi Mi M WAGE 1 EDUC 3EXPER e i 1,,, N Ri R Ri Ri Ri R with ˆ ˆ M R Page 64

65 8.4 Generalized Least Squares: Known Form of Variance 8.4.1b Weighted Least Squares The separate least squares estimates based on separate error variances are: b 9.05 b 1.8 b M1 M M 3 b b b R1 R R3 But we have two estimates for β and two for β 3 Page 65

66 8.4 Generalized Least Squares: Known Form of Variance 8.4.1b Weighted Least Squares Eq. 8.34a Obtaining generalized least squares estimates by dividing each observation by the standard deviation of the corresponding error term: σ M and σ R : WAGE Mi 1 EDUC Mi EXPER Mi e Mi M1 3 M M M M M i 1,,, N M Eq. 8.34b WAGE Ri 1 EDUC Ri EXPER Ri e Ri R 1 3 R R R R R i 1,,, N R Page 66

67 8.4 Generalized Least Squares: Known Form of Variance 8.4.1b Weighted Least Squares But σ M and σ R are unknown Transforming the observations with their estimates This yields a feasible generalized least squares estimator that has good properties in large samples. Page 67

68 8.4 Generalized Least Squares: Known Form of Variance 8.4.1b Weighted Least Squares Also, the intercepts are different Handle the different intercepts by including METRO Page 68

69 8.4 Generalized Least Squares: Known Form of Variance 8.4.1b Weighted Least Squares The method for obtaining feasible generalized least squares estimates: 1. Obtain estimated ˆ M and ˆ R by applying least squares separately to the metropolitan and rural observations.. Let ˆ M when METROi 1 ˆ i ˆ R when METROi 0 3. Apply least squares to the transformed model WAGE 1 i EDUC i EXPER i METRO i e i Eq R 1 3 ˆ ˆ ˆ ˆ ˆ ˆ i i i i i i Page 69

70 8.4 Generalized Least Squares: Known Form of Variance 8.4.1b Weighted Least Squares Our estimated equation is: Eq WAGE EDUC 0.13EXPER 1.539METRO (se) (1.0) (0.069) (0.015) (0.346) Page 70

71 8.5 Generalized Least Squares: Unknown Form of Variance Page 71

72 8.5 Generalized Least Squares: Unknown Form of Variance Consider a more general specification of the error variance: Eq var( e) i i x i where γ is an unknown parameter Page 7

73 8.5 Generalized Least Squares: Unknown Form of Variance To handle this, take logs ln( i) ln( ) ln( xi) and then exponentiate: Eq exp ln( ) ln( x ) i exp( z ) 1 i i Page 73

74 8.5 Generalized Least Squares: Unknown Form of Variance We an extend this function to: Eq exp( z z ) i 1 i S is Page 74

75 8.5 Generalized Least Squares: Unknown Form of Variance Now write Eq as: Eq ln i 1 zi To estimate α 1 and α we recall our basic model: β1 β y E y e x e i i i i i Page 75

76 8.5 Generalized Least Squares: Unknown Form of Variance Apply the least squares strategy to Eq using eˆi : Eq ˆ ln( ei ) ln( i ) vi 1 zi vi Page 76

77 8.5 Generalized Least Squares: Unknown Form of Variance For the food expenditure data, we have: ln( i ) z i Page 77

78 8.5 Generalized Least Squares: Unknown Form of Variance In line with the more general specification in Eq. 8.39, we can obtain variance estimates from: ˆ ˆ z ˆ i exp( 1 1 i) and then divide both sides of the equation by ˆ i Page 78

79 8.5 Generalized Least Squares: Unknown Form of Variance This works because dividing Eq. 8.6 by ˆ i yields: y 1 i x i e i 1 i i i i The error term is homoskedastic: Eq. 8.4 e 1 1 i var var( ) 1 ei i i i i Page 79

80 8.5 Generalized Least Squares: Unknown Form of Variance To obtain a generalized least squares estimator for β 1 and β, define the transformed variables: Eq y i 1 x i yi xi 1 xi ˆ ˆ ˆ i i i and apply least squares to: Eq y x x e i 1 i1 i i Page 80

81 8.5 Generalized Least Squares: Unknown Form of Variance To summarize for the general case, suppose our model is: Eq y x x e i 1 i k ik i where: Eq var( ei ) i exp( 1 zi szis ) Page 81

82 8.5 Generalized Least Squares: Unknown Form of Variance The steps for obtaining a generalized least squares estimator are: 1. Estimate Eq by least squares and compute the squares of the least squares residuals eˆi. Estimate 1,,, Sby applying least squares to the equation lne i 1 z i S z is v i 3. Compute variance estimates ˆ exp( ˆ ˆ z ˆ z ) 4. Compute the transformed observations defined by Eq. 8.43, including,, xi 3 xik if K 5. Apply least squares to Eq. 8.44, or to an extended version of (8.44), if K Page 8 ˆ i 1 i S is

83 8.5 Generalized Least Squares: Unknown Form of Variance Eq Following these steps for our food expenditure problem: yˆ x i (se) (9.71) (0.97) The estimates for β 1 and β have not changed much There has been a considerable drop in the standard errors that under the previous specification were ˆ 1 ˆ se β 3.79 and se β 1.39 Page 83

84 8.5 Generalized Least Squares: Unknown Form of Variance Using Robust Standard Errors Robust standard errors can be used not only to guard against the possible presence of heteroskedasticity when using least squares, they can be used to guard against the possible misspecification of a variance function when using generalized least squares Page 84

85 8.6 in the Linear Probability Model Page 85

86 8.6 in the Linear Probability Model We previously examined the linear probability model: E y p β1 βx β K x K and, including the error term: y E y e β β x β x e Eq i i i i K ik i Page 86

87 8.6 in the Linear Probability Model We know this model suffers from heteroskedasticity: Eq yi ei pi pi β β x β x 1 β β x β x var var 1 1 i K ik 1 i K ik Page 87

88 8.6 in the Linear Probability Model We can estimate p i with the least squares predictions: pˆ b b x b x Eq i i K ik Eq giving: e pˆ pˆ var 1 i i i Page 88

89 8.6 in the Linear Probability Model Generalized least squares estimates can be obtained by applying least squares to the transformed equation: y 1 x x e i i ik i β β β 1 K pˆ pˆ pˆ pˆ pˆ pˆ pˆ pˆ pˆ pˆ i i i i i i i i i i Page 89

90 8.6 in the Linear Probability Model Table 8.1 Linear Probability Model Estimates The Marketing Example Revisited Page 90

91 8.6 in the Linear Probability Model The Marketing Example Revisited A suitable test for heteroskedasticity is the White test: N R p value This leads us to reject a null hypothesis of homoskedasticity at a 1% level of significance. Page 91

92 Key Words Page 9

93 Keywords Breusch Pagan test generalized least squares Goldfeld Quandt test grouped data -consistent standard errors homoskedasticity Lagrange multiplier test linear probability model mean function residual plot transformed model variance function weighted least squares White test Page 93

94 Appendices Page 94

95 8A Properties of the Least Squares Estimator Assume our model is: y x e i 1 i i E e e e e i j ( i) 0 var( i) i cov( i, j) 0 ( ) Eq. 8A.1 The least squares estimator for β is: where b w we i i i x i x x i x Page 95

96 8A Properties of the Least Squares Estimator Since E(e i ) = 0, we get: β so β is unbiased i i β wie ei E b E E w e Page 96

97 8A Properties of the Least Squares Estimator But the least squares standard errors are incorrect under heteroskedasticity: var b var wiei Eq. 8A. w var e w w cov e, e w i i i j i j i j i i ( xi x) ( x x) i i Page 97

98 8A Properties of the Least Squares Estimator Eq. 8A.3 If the variances are all the same, then Eq. 8A. simplifies so that: var( b ) x i x Page 98

99 8B LaGrange Multiplier Tests for Recall that: Eq. 8B.1 e z z v ˆi 1 i S is i and assume that our objective is to test H 0 : α = α 3 = = α S = 0 against the alternative that at least one α s, for s =,..., S, is nonzero Page 99

100 8B LaGrange Multiplier Tests for The F-statistic is: Eq. 8B. F ( SST SSE) / ( S 1) SSE / ( N S) where i N N ˆ ˆ and ˆ i i1 i1 SST e e SSE v Page 100

101 8B LaGrange Multiplier Tests for Eq. 8B.3 Eq. 8B.4 Eq. 8B.5 We can modify Eq. 8B. Note that: SST SSE ( S1) F SSE / ( N S ) ( S 1) Next, note that: var( ei ) var( vi) Substituting, we get: SST SSE var( e ) SSE N S i Page 101

102 8B LaGrange Multiplier Tests for Eq. 8B.6 If it is assumed that e i is normally distributed, then: and: Further: var e 4 i e SST SSE ˆ 4 e 1 var ei var( e ) var( e ) 4 e e 4 i i e Page 10

103 8B LaGrange Multiplier Tests for Using Eq. 8B.6, we reject a null hypothesis of homoskedasticity when the χ -value is greater than a critical value from the χ (S 1) distribution. Page 103

104 8B LaGrange Multiplier Tests for Eq. 8B.7 Also, if we set: We can get: 1 var( e ) ( e e ) N ˆ ˆ i i N i1 SST SSE SST / N SST N Eq. 8B.8 N 1 SSE SST N R Page 104

105 8B LaGrange Multiplier Tests for At a 5% significance level, a null hypothesis of homoskedasticity is rejected when NR exceeds the critical value χ (0.95, S 1) Page 105