Multicollinearity. Filippo Ferroni 1. Course in Econometrics and Data Analysis Ieseg September 22, Banque de France.

Transcription

1 Filippo Ferroni 1 1 Business Condition and Macroeconomic Forecasting Directorate, Banque de France Course in Econometrics and Data Analysis Ieseg September 22, 2011

2 We have multicollinearity when two or more regressors are linear combinations of other regressors (perfect) or highly correlated (imperfect).

3 We have multicollinearity when two or more regressors are linear combinations of other regressors (perfect) or highly correlated (imperfect). The usual interpretation of the regression coefficient (as the average impact of one variable ceteris parisbus) does not apply any more.

4 We have multicollinearity when two or more regressors are linear combinations of other regressors (perfect) or highly correlated (imperfect). The usual interpretation of the regression coefficient (as the average impact of one variable ceteris parisbus) does not apply any more. Perfect violates the the assumption for the Gauss-Markov theorem to hold. Almost impossible in practice.

5 We have multicollinearity when two or more regressors are linear combinations of other regressors (perfect) or highly correlated (imperfect). The usual interpretation of the regression coefficient (as the average impact of one variable ceteris parisbus) does not apply any more. Perfect violates the the assumption for the Gauss-Markov theorem to hold. Almost impossible in practice. Imperfect multicollinearity causes:

6 We have multicollinearity when two or more regressors are linear combinations of other regressors (perfect) or highly correlated (imperfect). The usual interpretation of the regression coefficient (as the average impact of one variable ceteris parisbus) does not apply any more. Perfect violates the the assumption for the Gauss-Markov theorem to hold. Almost impossible in practice. Imperfect multicollinearity causes: 1 Estimates are typically unbiased, however not very precise (large standard errors around the OLS estimates, s k )

7 We have multicollinearity when two or more regressors are linear combinations of other regressors (perfect) or highly correlated (imperfect). The usual interpretation of the regression coefficient (as the average impact of one variable ceteris parisbus) does not apply any more. Perfect violates the the assumption for the Gauss-Markov theorem to hold. Almost impossible in practice. Imperfect multicollinearity causes: 1 Estimates are typically unbiased, however not very precise (large standard errors around the OLS estimates, s k ) 2 Thus, we tend to underestimate the t-statistic. Recall t s = b k β H0 s k = b k s k

8 We have multicollinearity when two or more regressors are linear combinations of other regressors (perfect) or highly correlated (imperfect). The usual interpretation of the regression coefficient (as the average impact of one variable ceteris parisbus) does not apply any more. Perfect violates the the assumption for the Gauss-Markov theorem to hold. Almost impossible in practice. Imperfect multicollinearity causes: 1 Estimates are typically unbiased, however not very precise (large standard errors around the OLS estimates, s k ) 2 Thus, we tend to underestimate the t-statistic. Recall t s = b k β H0 s k = b k s k 3 As a consequence, it is more likely to accept the null hypothesis (that the parameter is zero).

9 We have multicollinearity when two or more regressors are linear combinations of other regressors (perfect) or highly correlated (imperfect). The usual interpretation of the regression coefficient (as the average impact of one variable ceteris parisbus) does not apply any more. Perfect violates the the assumption for the Gauss-Markov theorem to hold. Almost impossible in practice. Imperfect multicollinearity causes: 1 Estimates are typically unbiased, however not very precise (large standard errors around the OLS estimates, s k ) 2 Thus, we tend to underestimate the t-statistic. Recall t s = b k β H0 s k = b k s k 3 As a consequence, it is more likely to accept the null hypothesis (that the parameter is zero). 4 A danger of such data redundancy is overfitting.

10 Detecting Compute the pairwise sample correlation among regressors. If it is large, then we have issues.

11 Detecting Compute the pairwise sample correlation among regressors. If it is large, then we have issues. High Variance Factors (VIF) Suppose you want ot detect MC in the following equation y = α + β 1x β k x k + ɛ

12 Detecting Compute the pairwise sample correlation among regressors. If it is large, then we have issues. High Variance Factors (VIF) Suppose you want ot detect MC in the following equation 1 Run The following regression y = α + β 1x β k x k + ɛ x 1 = γ + δ 2 x δ k x k + ε

13 Detecting Compute the pairwise sample correlation among regressors. If it is large, then we have issues. High Variance Factors (VIF) Suppose you want ot detect MC in the following equation 1 Run The following regression 2 Compute y = α + β 1x β k x k + ɛ x 1 = γ + δ 2 x δ k x k + ε VIF(b j ) = 1 1 R 2 1

14 Detecting Compute the pairwise sample correlation among regressors. If it is large, then we have issues. High Variance Factors (VIF) Suppose you want ot detect MC in the following equation 1 Run The following regression 2 Compute y = α + β 1x β k x k + ɛ x 1 = γ + δ 2 x δ k x k + ε VIF(b j ) = 1 1 R Repeat step 1 and 2 for all the regressors, if VIF(b 1 ) > 5 we have multicollinearity.

15 Remedies Do nothing. If in your original specification you get significant estimate, then never mind about.

16 Remedies Do nothing. If in your original specification you get significant estimate, then never mind about. Drop a redundant variable (the one that has the largest VIR or the largest pairwise correlation).

17 Remedies Do nothing. If in your original specification you get significant estimate, then never mind about. Drop a redundant variable (the one that has the largest VIR or the largest pairwise correlation). Add more data if possible

18 Remedies Do nothing. If in your original specification you get significant estimate, then never mind about. Drop a redundant variable (the one that has the largest VIR or the largest pairwise correlation). Add more data if possible The best regression models are those where the regressors correlate highly with the dependent (outcome) variable but correlate at most only minimally with each other.