L7: Multicollinearity

Transcription

1 L7: Multicollinearity Feng Li School of Statistics and Mathematics Central University of Finance and Economics

2 Introduction ï Example Whats wrong with it? Assume we have this data Y X X We want to make a simple regression model Y i = β 1 + β X i + β 3 X 3i + u i By applying the formula in Chapter 7, ř ř yi x i x ˆβ = 3i ř ř y i x 3i xi x 3i ř ř x i x 3i ( ř x i x 3i ) = = 0 0 Something went wrong? Feng Li (Statistics, CUFE) Econometrics / 1

3 Multicollinearity Perfect multicollinearity: the covariates are exactly linear combined λ 1 X 1 + λ X λ k X k = 0 e.g. X 3 = X. Less perfect multicollinearity (common in practice): λ 1 X 1 + λ X λ k X k + v i = 0 where v i is some random values. E.g. X Feng Li (Statistics, CUFE) EconometricsX 3 / 1

4 Estimation problems if multicollinearity in the covariates A perfect multicollinearity: coefficients are indeterminate and infinite large of standard errors for the coefficients. A less perfect multicollinearity: coefficients are determinate but could not be estimated preciously and very large of standard errors for the coefficients. Feng Li (Statistics, CUFE) Econometrics 4 / 1

5 Sources of multicollinearity If we want to estimate how much electricity used in a family (Y) and we observe some variables might be used X 1 : How big is the house X : How many people in this house X3 : How many rooms in the house Discussion with these models Y i = α 0 + α 1 X 1 + α X + u i Yi = α 0 + α 1 X 1 + α X 3 + u i Feng Li (Statistics, CUFE) Econometrics 5 / 1

6 Practical consequences of high multicollinearity The OLS estimate is still BLUE (?) but with big variance. The OLS estimate can be very sensitive to a small change of data. Much bigger confidence interval ñ easy to accept the null hypothesis. R very high but not significant t statistic. Feng Li (Statistics, CUFE) Econometrics 6 / 1

7 How to detect high multicollinearity (1) R very high but not significant t statistic. Use VIF in the two-variable model Assume r ij is the coefficient of correlation between X i and X j. If X i and X j have collinearity problem, then r ij Ñ 1. Define variance-inflating factor (VIF) as 1 VIF = 1 r ij If there is no collinearity between Xi and X j, VIF = 1. If there is high collinearity between Xi and X j, VIF is usually bigger than 10 and tends to. Recall the variance of estimate ˆβ in our first example var(ˆβ ) = σ ř x i (1 r 3 ) = ř σ x VIF. i The inverse of VIF is called tolerance (TOL) why does the last equality hold? TOL = 1 VIF j = 1 r ij Feng Li (Statistics, CUFE) Econometrics 7 / 1

8 Use VIF in the k-variable model The variance of a coefficient in the model var(ˆβ j ) = σ ř x j (1 R j ) = σ ř x j VIF j. where R j is the R for the auxiliary regression X j with the remaining k 1 regressors X j = β 1 X 1 + β X β j 1 X j 1 + β j+1 X j β k X k You can calculate VIF in two ways VIF j = 1 1 R j VIF j = var(ˆβ j ) ř x j ˆσ The TOL j = 1/VIF j Rule of thumb: if VIF j ą 10, indicating R j ą 0.9 ñ highly collinear of X j. Feng Li (Statistics, CUFE) Econometrics 8 / 1

9 how to detect high multicollinearity () high pair-wise correlations among regressors. auxiliary regression. the scatter plot. eigenvalues and conditional numbers of X 1 X the basic idea: X 1 X is invertible if there is not strong collinearity (all eigenvalues of X 1 X are positive and in a reasonable range). leading to the conditional number k k = Max eigenvalue Minimal eigenvalue and the conditional index? k Rule of thumb: if 100 ă k ă 1000, moderate to strong collinearity; if k ě 1000, severe collinearity; if 0 ă k ă 100, good Feng Li (Statistics, CUFE) Econometrics 9 / 1

10 How to remedy multicollinearity problem? Drop a variable, usually firstly drop the most nonsignificant variable. Transform the variable. Do nothing if your purpose is prediction(see next slide). Feng Li (Statistics, CUFE) Econometrics 10 / 1

11 Is multicollinearity always bad? The higher R the better prediction. So multicollinearity is not really a problem, if your purpose is prediction only. Feng Li (Statistics, CUFE) Econometrics 11 / 1

12 Take home questions 10.10, 10.1, 10.19, 10.1 Feng Li (Statistics, CUFE) Econometrics 1 / 1