Introduction to Econometrics. Heteroskedasticity

Transcription

1 Introduction to Econometrics Introduction Heteroskedasticity When the variance of the errors changes across segments of the population, where the segments are determined by different values for the explanatory variables, we have heteroskedasticity. In other words the variance of the error may vary across observations. Homoskedasticity is when the unobservable error conditional on the explanatory variables is constant. Therefore, as you probably have suspected, heteroskedasticity is the opposite to homoskedasticity. If we were to examine the saving relationship based on various factors, heteroskedasticity would be present if the variance of the unobserved factors affecting savings increases with income. Therefore the variation in ε i is increasing with income. Another example would be where the independent variable is the expenditure on food and the independent variable is disposable income. Generally as income increases the variation in food expenditure increases. Therefore variation in food expenditure is higher for high incomes and lower for lower incomes. In other words the variation in ε i is increasing with income. Gauss-Markov (GM) Homoskedasticity Assumption Heteroskedasticity Var(ε X) = σ = Var(ε i x 1, x,, x k ) = σ Var(ε X) = σ h(x), where Var(ε i x 1, x,, x k ) = σ h i = σ i and σ i σ k With heteroskedasticity each observation is written as Var(ε i X) = σ h i which is often just denoted as Var(ε i X) = σ i. In the above general expression h(x) is a function of the explanatory variable which determines the heteroskedasticity. This function can take any positive form. h(x) > 0 because all variances must be positive for all possible values of the dependent variable y. Later in this document we will see that correcting for hetroskedasticity differs based on whether we know h(x). We do not know the population parameter σ and therefore it has to be estimated from the sample data. Homoskedasticity and Heteroskedasticity Homoskedasticity: Var(ε X) = Var(ε) = σ I = σ (Where I is the identity matrix, and X = x 1, x,, x k ) 1 0 σ 0 Var(ε X) = Var(ε) = σ I =σ = σ σ Heteroskedasticity: Var(ε X) = σ h(x) = σ Diag(h i ) = σ ψ (where ψ is a positive definite matrix Diag(h i ) not equal to I and X = x 1, x,, x k ). h 1 0 Var(ε X) = σ ψ = σ 0 h k 0 h k = h 1 σ 0 0 h k σ 0 h k σ

2 We use the assumption of homoskedasticity in OLS, however this assumption has no role in showing whether OLS is bias or inconsistent. In fact, heteroskedasticity does not cause bias or inconsistency in the OLS estimation of β j. However, remember that omitting an important variable would result in heteroskedasticity. In addition R is also unaffected by the presence of heteroskedasticity. This is because the R is essentially 1 R ε R y where R ε is the population error variance and R y is the population variance of y. The R is not affected by the heteroskedasticity because they are both population measures and unconditional. Example 1 Heteroskedasticity (Student Teacher Ratios) From a sample of schools we can look at the impact of class size on student performance. The diagram below shows that, as expected, the larger the class size the lower the students test scores. The diagram below shows that the errors at different student teacher ratios have equal variance which means that the errors are homoskedasticity. Var(ε X) = Var(ε) = σ I =σ σ 0 = 0 σ 0 σ Under heteroskedasticity the student teach ratio increases the variance also increases as shown by the increasing spread of the distribution.

3 Var(ε X) = σ ψ = h 1 σ 0 0 h k σ 0 h k σ Example Hetroskedasticity (Savings and Income) The diagram below shows the n observations from a sample. The dependent variable y represents the saving rate of an individual and x is the income of the individual. The line running through the plotted sample of observations is the regression line from OLS. Savings Hetroskedasticity: Savings and Income 0 Income The distance between the observations and the regression line is on average greater as we move to the right of the diagram, where income is higher. This means that the variance of ε i is increasing with income. Higher income results in higher expected savings rates, but also have a higher variance. Basically the variation in savings rates is higher among high income people than the variation in savings rates among low income people. This is evident by the difference in the length of red lines which shows the variation at two different levels of income. The Estimators of the Variance Var(β ) If homoskedasticity is violated the variance of the errors is no longer constant. Fortunately this does not cause the OLS estimator β j to be biased or inconsistent. However the variance of the estimator B is invalid because it is based on the wrong expression. As a result the variance of the estimator will be biased under heteroskedasticity. This means that although the OLS estimator β j is unbiased and consistent it is no longer the best because the variance is invalid. Please note that it is the variance of the estimator Var(β j), not the estimator β j, that is biased under heteroskedasticity. Therefore in the presence of

4 heteroskedasticity t stats do not have t ddistributions. In other words, t stats and confidence intervals will be incorrect because they are based on the wrong variance. The Variance of the OLS Estimator β β = (X X) X y = β + (X X) X ε Var β X = V((X X) X ε X) = (X X) X V(ε X)X(X X) = σ (X X) X ψx(x X) This will reduce to the σ (X X) if ψ is the identity matrix which implies homoskedasticity. However if the matrix ψ is not the identity matrix then we will have heteroskedasticity and the variance and standard errors will be invalid for t stats and hypothesis testing. Due to estimating the wrong variance we can no longer say that the OLS estimator is BLUE (Best Linear Unbiased Estimator). Although the estimator is still unbiased it is no longer the best. In the class of estimators OLS is no longer asymptotically efficient and its possible to find more efficient estimators if we know the form of the heteroskedasticity. Also this problem cannot be solved by increasing the sample size, however in larger sample sizes it may not be as important to obtain an efficient estimator. Assuming that the model is not misspecified there are essentially two methods to approaches used to derive an alternative estimator to solve the problem of heteroskedasticity. One first method involves deriving an alternative estimator that is best linear unbiased and is known as generalized least squares (GLS). This method is used when we known the form of heteroskedasticity by knowing the matrix ψ. Another method to derive a best linear unbiased estimator when the matrix ψ is unknown, is the feasible generalised least squares estimator (FGLS). Both are forms of weighted least squares. It must be stressed that the transformed model is only used for to determine the GLS estimators and it does not provide information for interpretation. The parameters are to be interpreted in through the original untransformed model. 1. When heteroskedasticity is known: Generalised Least Squares Var(ε X) = σ h(x) When we say the heteroskedasticity is known this means that we know the form of the function h(x). Knowing the best linear unbiased estimator under the GM assumptions allows us to transform the model under heteroskedasticity so that it satisfies the GM conditions again. We can start the transformation by writing, ψ = P P where ψ is a positive definite matrix Diag(h i ) where P is a square non-singular matrix (it is not necessarly unique). It is important to understand that the matrix is non-singular because it guarantees that no information is lost in the transformation. As ψ is positive there exists as P that satisfies the equation. This matrix transforms the errors so we only have the variance on the diagonals of the variance covariance matrix.

5 ψ = (P P) = P (P ) PψP = PP (P ) P = I (remember that if ψ = I then there is homoskedasticity because Var(ε X) = σ ψ =σ I, in fact the estimator will be reduced to the OLS estimator if ψ = I ) ow as previously mentioned we would like to transform our errors and we do this by pre multiplying the vector of errors by a the matrix P. This can be written as follows. E(P ε X) = PE(ε X) = 0 V(P ε X) = PVE(ε X)P = σ PψP = σ I So when we pre multiply the error vector with the transformation matrix P we satisfy the GM conditions. We also transform the entire model by multiplying everything in the model by the matrix P. Where the matrix ψ = Diag(h i ) which means that it is a square matrix with h 1 h k on the diagonal. Pre Multiplying by the Transformation Matrix h 1 σ 0 Var(ε X) = σ ψ = 0 h k σ 0 h k σ h 1 0 h 1 σ 0 h 1 0 V(P ε X) = σ PψP = 0 h k 0 h k σ 0 h k = σ I 0 h k 0 h k σ 0 h k Please note that we have to transform the whole model. We know that the error term vector ε satisfies the GM conditions and therefore transform the whole model. Py = PXB + Pε which can write as, y = X β + ε ow the OLS estimator β is also transformed and becomes the GLS estimator which we denote as β GLS. To generate the GLS estimator β GLS we run the OLS regression on the transformed model. y i = x i β + ε i or y i = x i β + ε i h i h i h i β GLS = h i x i x i h i x i We derive the GLS estimator directly from minimising the residual sum of squares after transforming each element in the sum by h i. The GLS Estimator β GLS = (X X ) X y = (X ψx) X ψ y y i

6 When transforming the model the choice of P is not important for the estimator. Only ψ is needed to transform the matrix. Remember that this is just a general model and is used when ψ is completely known. Please note that the OLS covariance matrix is larger than the GLS covariance matrix because the GLS estimator has a smaller variance than the OLS estimator. Also the GLS estimator β GLS is BLUE and OLS is not. Var β GLS = σ (X X ) = σ (X ψ X) The weights imply that observations with a higher variance get a smaller weight in estimation which means that the more accurate the observation the greater weight it is given. Example 3 The GLS Estimator Suppose that y denotes the price of a house and x represents the constant term and an explanatory variable such as number of bedrooms. Generally on average, we expect a house with more rooms to have a higher price. Therefore we would expect this relationship to be upward sloping with first increasing then decreasing slope as the change in price are increasing then decreasing as the number of rooms increases. We shall assume that the relationship is more or less linear and we can use OLS. In addition we expect the variation in house price among more expensive houses to be greater than the variation in low price housing. This means that we suspect that ε i increases with bedrooms. Therefore, on average, a higher number of bedrooms results in a higher expected price and a higher variance. Remember that we are solving the hetroskedasticity so we can have the best linear unbiased estimators of the βs in the model. The model is as usual, y= Xβ + ε which we represent as houseprice i = β 0 + β 1 bdrms i + ε i The variance of the error is Var(ε i brms i ) = σ exp(γ 1 brms i ) = σ exp (γ 1 brms i ) Var(ε i X) = V(ε i x i ) = σ h i We assume that the h i are positive and known. Var(ε X) = σ Diag(h i ) = σ ψ ψ represents a matrix with h i on the diagonals. This allows us to easily see what the correct transformation matrix P should be. P = Diag(h i ) P is a diagonal matrix with h i on the diagonals. The reason that the transformation matrix has h i on the diagonals and not h i is because ψ = P P. ψ = (P P) = P (P ) PψP = PP (P ) P = I Therefore when we apply this to the variance we have the following V(P ε X) = PVE(ε X)P = σ PψP = σ I

7 ow we transform the whole model To generate the GLS estimator β GLS we run the OLS regression on the transformed model. y i = x i β + ε i For our example this would be written as or y i = x i β + ε i h i h i houseprice i = β 0 + β i1 bdrms i + ε i. When ψ is unknown: Feasible Generalised Least Squares In the previous section we assumed that the heteroskedasticity was known up to a multiplicative form. This means that we know the matrix ψ and can use GLS. However most of the time we do not know the form of ψ and therefore it must be estimated first. This means that where we had known h i now we estimate them so we have h i on the diagonals of the matrix ψ. (Please note that we have used h i to denote the weight given to the variance of the error but it is often denoted as just h i ). β FGLS = (X X ) X y = (X ψx) X ψ y Var β FGLS = σ (X X ) = σ (X ψ X) Basically FGLS uses data to estimate the parameters and then to use these estimates to construct weights. There are steps used to estimate the FGLS estimator. As mentioned before the FGLS is, like GLS, a weighted least squares estimator. If we could use h i instead of h i our estimator would be unbiased and BLUE if we assume the hetroskedasticity is modelled correctly, however because we have to estimate h i the FGLS estimator is no longer unbiased and therefore not BLUE. However the FGLS estimator is consistent and asymptotically more efficient that OLS. Because we have to estimate the weights using the same data we no longer have an estimator that is unbiased. Therefore the FGLS is not BLUE. However the FGLS estimator is more efficient than OLS asymptotically. FGLS is a good option where it seems as though there may be hetroskedasticity that is inflating the standard errors. h i = exp (x i γ) Var(ε x) = σ exp(γ 0 + γ 1 x γ k x k ) = σ exp (x i γ) The variables x i can be based on other functions. For instance we could replace the x i variables above with z i 's which may represent a function or subset of the x i variables. However, For simplicity we will just work with the model above and will not make any assumptions about the functional form of x i. Also an exponential function is used because although linear models are fine when testing for hetroskedasticity but are difficult to work with when correcting hetroskedasticity. Linear models do not guarantee the predicted values will be positive and our estimated variances must be positive to undertake weighted least squares. Therefore the use of exponential form is widely used in practice. To be able to perform FGLS consistent estimators for the unknown parameters γ in h i = exp(z i γ) we need consistent estimators. Basically we want to estimate the γ and we do h i

8 this by using the OLS residuals. We make a slight transformation to the above model to get a linear form that can be estimated by OLS. ε = σ exp(γ 0 + γ 1 x γ k x k ) w Assuming that w is independent of x we have the following equation log(ε ) = α 0 + γ 1 x γ k x k + v where v has a zero mean and is independent of x. In addition the intercept has changed but this is not important. We run the above regression to get the estimated γ log(ε ) = α 0 + γ 1 x γ k x k + v ow we have the fitted values of γ which is what we need to get the weights h i h i = exp (x i γ ) We can now use the weighted least squares with h i in place of h i. To transform all the observations we use the weights h i y i = x i β + ε i = Py = PXB + Pε = y h i h i h = X β + ε i Remember that ψp = PP (P ) P = I, where the matrix ψ = Diag(h i ) which means that it is a square matrix with h 1 h k on the diagonal and that, V(P ε X) = PVE(ε X)P = σ PψP = σ I We divide by h i because the matrix P has h i on the diagonal, that is P = Diag(h i ). Pre Multiplying by the Transformation Matrix (Multiplicative Heteroskedasticity) h 1 σ 0 Var(ε X) = σ ψ = 0 h k σ 0 h k σ h 1 0 h 1 σ 0 h 1 0 V(P ε X) = σ PψP = 0 h k 0 h k 0 h k σ = σ 0 h 0 h k σ k I 0 h k We then run the OLS on this transformed model to derive the FGLS estimator β FGLS. Remember that the β FGLS are estimators of the parameters in the usual population model. Also the FGLS will have a consistent estimator for the covariance matrix. Var β FGLS = σ x ix i h i

9 The Procedure for FGLS to Correct for Heteroskedasticity 1. The first step is to estimate the model with OLS using the regression y~x 1, x,, x k and save the residuals ε. Compute log (ε ) (to do this square the residuals then take the natural log) 3. ow run the regression log (ε )~x 1, x,, x k and get the fitted values from this regression and call them γ 4. ow compute h i = exp (x i γ ) 5. Estimate the equation y = X β + ε by using the weights h i y i h i = x i h i β + ε i h i Example 4 The FGLS Estimator Suppose we are studying the demand for cigarettes and have the following OLS regression cigs = β 0 + β 1 educ + β logprice + β 3 loginc + β 4 age + β 5 age + ε cıgs = educ 0.966logprice loginc age 0.007age (0.0) (0.5) (0.0) (0.33) (0.04) R = = 885 Based on this information we may perform a test to detect whether the variance of the errors is heteroskedastic. Assume that we find evidence of heteroskedasticity but do not know the form. As a result we must estimate the weight then apply weighted least squares. 1. We get the OLS residuals from the model above ε. Compute log (ε ) (to do this square the residuals then take the natural log) 3. ow run the regression log(ε ) ~educ, logprice, loginc, age, age and get the fitted values from this regression and call them γ log(ε ) = α 0 + γ 1 educ + γ logprice + γ 3 loginc + γ 4 age + γ 5 age + v 4. ow compute h i = exp (x i γ ) (exponentiate the fitted values to get the weights) 5. Then estimate the equation y = X β + ε by using the weights h i y i h i = x i h i β + ε i h i cıgs = educ 0.966logprice loginc age 0.007age (0.0) (0.5) (0.0) (0.33) (0.04) R = = 885

10 The Breusch Pagan Test for Heteroskdedasticity The Breusch - Pagan Test is simply a Lagrange multiplier test for heteroskedasticity. The test involves estimating the original OLS model and obtain the residuals for each observation. Then we run the regression of the residuals on upon the explanatory variables and get the R. σ i = σ (x i γ ) We assume that the error is constant and there is homoskedasticity and require evidence to disprove this. Therefore the null hypothesis is as follows. H 0 : Var(ε x) = σ Because we assume the error has a zero mean conditional expectation Var(ε x) = E(ε x) and therefore the null hypothesis is equivalent to Run a regression and save the residuals. H 0 : E(ε x) = E(ε ) = σ y = β 0 + β 1 x β k x k + ε Run a regression of the estimated residuals on the explanatory variables. However we don't even know the value for the errors so we have to estimate them and we denote this as e (these are the saved residuals or fitted value of ε. e = γ 0 + γ 1 x γ k x k + u ow we get the R from this regression and multiply it by the number of observations to perform the LM test, LM = R. Where R has asymptotically a chi square distribution χ with k degrees of freedom. The Breusch-Pagan Test for Hetroskedasticity H 0 : Var(ε x) = σ 1. Estimate the original OLS model and obtain the OLS residuals e for each observation. Run the auxiliary regression of e = γ 0 + γ 1 x γ k x k + u and get R 3. Compute either the F- test and compute the p value using (F k,n k distribution) or the LM test, LM = R using χ k Example 6 The Bruesch Pagan Test Suppose we are studying what determines the sales price of a house. We use factors such as lot size, square feet and the number of bedrooms. Prıce = lotsize sqrft + 1.5bedrooms (33.) (0.005) (0.1) (7.33) R = = 85

11 Unfortunately the above regression tells us nothing about whether the errors in the population are hetroskedastic. ow we get the squared OLS residuals and regress them on the explanatory variables. ow we can perform the LM Test e = γ 0 + γ 1 lotsize + γ sqrft + γ 3 sqrft R = 0.1 = 85 k = 3 H 0 : Var(ε x) = σ LM = R = 85(0.1) = We compare the test statistic to the p-value from a χ distribution with 3 degrees of freedom which we denote as χ 3. The critical value at the 1% level for a chi square distribution and 3 degrees of freedom is Therefore we reject the null hypothesis and have evidence of hetroskedasticity being present in the model. Often it is useful transform the model into log form Prıce = lotsize sqrft + 1.5bedrooms (33.) (0.005) (0.1) (7.33) R = = 85 e = γ 0 + γ 1 lotsize + γ sqrft + γ 3 sqrft R = 0.0 = 85 k = 3 LM = R = 85(0.0) = 1.7 We fail to reject the null hypothesis as > 1.7 The White Test for Hetroskedasticity If we estimate the model with hetroskedasticity we know that the estimator is unbiased and consistent. The OLS standard errors and test statistics are valid if the GM assumptions hold. We are able to replace the homoskedasticity assumption, Var(ε i x 1, x,, x k ) = σ, with a weaker assumption that the squared error ε is uncorrelated with all the independent variables, the square of the independent variables and the cross product of all the independent variables. To perform the White Test for Hetroskedasticity we go back to the fact that we do not know the errors in the population model and have to estimate them. We use the OLS residual ε and then regress this on all the independent variables,, the square of the independent variables and the cross product of all the independent variables. This means that if we have 3 independent variables and a constant we will now have 9 regressors and a constant. This test is explicitly intended to test for forms of hetroskedasticity that invalidate OLS and the standard errors and t statistics. y = β 0 + β 1 x 1 + β x + β 3 x 3 + ε e = γ 0 + γ 1 x 1 + γ x + γ 3 x 3 + γ 4 x 1 + γ 5 x + γ 6 x + γ 7 x 1 x + γ 8 x 1 x + γ 9 x 1 x + u

12 The White Test is the LM test, or F- test which tests whether all of the γ i in the equation are zero except for the constant term. This means that a LM test for the case above will have 9 restrictions. The white test is a general form of the Breusch Pagan Test, and can often find more general forms of heteroskedasticity. Therefore the test captures more genearl forms of heteroskedasticity. However due to its wide scope the test can often identity specification errors instead of heteroskedasticity. The White Test for Hetroskedasticity H 0 : γ 1 = 0, γ = 0,, γ 9 = 0 H A : at least one explanatory variabe is not equal to zero 1. Estimate the original OLS model and obtain the OLS residuals e for each observation. Run the auxiliary regression of e = γ 0 + γ 1 x 1 + γ x + γ 3 x 3 + γ 4 x 1 + γ 5 x + γ 6 x + γ 7 x 1 x + γ 8 x 1 x + γ 9 x 1 x + u and get the R from this regression 3. Compute either the F- test and compute the p value using (F 9,n0 distribution) or the LM test, LM = R using χ 9 Example 7 The White Test for Hetroskedasticity Let us examine the relationship between we are studying what determines the sales price of a house. We use factors such as lot size, square feet and the number of bedrooms. Prıce = lotsize sqrft + 1.5bedrooms (33.) (0.005) (0.1) (7.33) R = = 85 Unfortunately the above regression tells us nothing about whether the errors in the population are heteroskedastic. ow we get the squared OLS residuals and regress them on the explanatory variables. e = lotsize + 0.3sqrft bedrooms lotsize 0.08sqrft (.34) (0.065) (0.14) (5.11) (0.1) (0.0) 0.013lotsize lotsizesqrft lotsizebedrooms + 0.1lotsizebedrooms (0.300) (0.1) (0.0) (0.045) R = 0.1 = 85 k = 9 ow we can perform the LM Test H 0 : Var(ε x) = σ LM = R = 85(0.1) = We compare the test statistic to the p-value from a χ distribution with 3 degrees of freedom which we denote as χ 3. The critical value at the 1% level for a chi square distribution and 3 degrees of freedom is Therefore we reject the null hypothesis and have evidence of hetroskedasticity being present in the model.

13 A ote on Testing: Which Test to Use? The more explicit we are about the form of heteroskedasiticy the more powerful the test will be. Therefore we are more likely that the test correctly rejects the null hypothesis, if we specify the form correctly. The White Test is more general but has limited power against a large number of alternatives. It is recommended that the OLS residuals are plotted against one or more of the explanatory variables to help choose the appropriate alternative. Heteroskedasticity-consistent standard errors It is possible to make inferences based upon β without actually specifying the form of hetroskedasticity. All we do is just replace the usual OLS covariance matrix with a covariance matrix that uses the estimated variance of the error. β = (X X) X y = β + (X X) X ε Var β X = V((X X) X ε X) = (X X) X V(ε X)X(X X) = σ (X X) X ψx(x X) This will reduce to the σ (X X) if ψ is the identity matrix which implies homoskedasticity. This can be written in the following form below, Var β X = x i x i σ i x i x i x i x i To derive an estimate of this covariance matrix we would need to estimate all the σ i s. This is very difficult without additional assumptions. Instead we just use a consistent estimator of the OLS residual. This means that we want to just estimate the middle term in the above expression. Σ 1 σ i x i x i It can be shown that under certain assumptions it can be shown that the above equation is estimated by the following equation where ε i is the OLS residual. S 1 ε i x i x i S is a consistent estimator of Σ and the probability limit of the difference between these two converges to zero or in other words becomes a null matrix. ow we just replace the population variance with an estimate of the true variance of the OLS estimate. Remember that we do not know the true variance from the OLS estimator and therefore we have to estimate the variance of the OLS estimator. This is an estimate of an estimator. This is why in the following equation we have put a hat on variance Var to show that we are estimating the variance of the OLS estimator.

14 Var β X = x i x i ε i x i x i x i x i Fortunately most software packages do this calculation for us automatically if we specify White Standard Errors or Heteroskedasticity - Robust Standard Errors (Robust Standard Errors). Usually the regression output will report both the OLS standard errors and the Robust Standard Errors. If you have a sound understanding of the form of the heteroskedasticity then FGLS may provide a more efficient estimator.