Draft of an article prepared for the Encyclopedia of Social Science Research Methods, Sage Publications. Copyright by John Fox 2002

Transcription

1 Draft of an article prepared for the Encyclopedia of Social Science Research Methods, Sage Publications. Copyright by John Fox 00 Please do not quote without permission Variance Inflation Factors. Variance inflation factors (VIFs) measure the impact of collinearity among the predictors in a regression analysis on the precision of estimation. The variances in question are the sampling variances of the regression coefficients. The term variance inflation factor was possibly coined by Donald Marquardt (1970). Consider the linear regression model, y = α + β x + β x + L + β x + ε i 1 1i i k ki i where y i is the value of the response variable for the ith of n observations; x, 1i, xi, K xki are the values of the predictors; i ε is the error, usually assumed to be distributed independently of the errors for the other observations, with zero mean, and constant variance σ ; α is the regression constant; and (in a model in which the predictors are quantitative) the βs are slope coefficients. The sampling variance of the least-squares estimate b of β is V ( b ) = σ ( n 1) s 1 1 R (1) 1

2 where s = ( x i x) /( n 1) is the variance of the th predictor, and squared multiple correlation from the regression of R is the x on the other xs. The second factor on the right-hand side of equation (1), 1/(1 ), called the variance inflation factor for b, or VIF, expresses the degree to which collinearity among the predictors degrades the precision of b, relative to similarly dispersed uncorrelated predictors. The square-root of the variance inflation factor expresses the impact of collinearity on the size of the confidence interval for R β. It is worth mentioning the other, and more common, sources of imprecision in estimation revealed by equation (1): large error variance, small sample size, n; and predictors with small dispersion, σ ; s. As well, it should be pointed out that collinearity must be very strong before the precision of the regression estimates is substantially degraded. Figure 1 plots the square root of the variance inflation factor against the multiple correlation R ; even when R is as large as.8, for example, the square-root of the VIF is still less than. Multiple correlations among predictors as large as.8 are uncommon in social-science data. Variance-inflation factors are applicable to one-coefficient terms in linear models, such as quantitative predictors with linear effects. Multiple-coefficient terms such as a set of dummy regressors representing a categorical predictor require a more general approach, because correlations among regressors in a

3 related set are affected by inessential changes to the model (such as a change in baseline category for a set of dummy regressors). John Fox and Georges Monette (199) suggest a generalization of variance inflation factors to cover these situations, comparing the relative sizes of the oint-confidence region for a subset of coefficients for correlated and uncorrelated predictors. The oint confidence region for two regression coefficients is bounded by an ellipse; for three coefficients, it is ellipsoidal, and for more than three coefficients, it is hyper-ellipsoidal. Figure (a) shows two data ellipses : The solid ellipse is for uncorrelated predictors x 1 and x ; the axes of the ellipse are parallel to the x 1 and x axes. The broken ellipse is for predictors that are highly correlated, r 1 =.95; the tilt of the ellipse reflects the positive correlation between the predictors. Both ellipses are scaled so that their shadows on the x 1 and x axes are the standard deviations of the predictors; note that both ellipses are centered at the means of the predictors. For bivariate-normal predictors, the data ellipse is also a contour of constant probability-density, but regardless of the oint distribution of the xs, the data ellipse is a graphic representation of the dispersion and correlation of the predictors. The ellipses in Figure (b) give 95-percent confidence regions for the regression parameters β 1 and β assuming identical sample regression coefficients b 1 and b ; these confidence ellipses are rescaled 90-degree rotations of the corresponding data ellipses. Notice that the size of the confidence ellipse 3

4 for the highly correlated predictors is much larger than the ellipse for the uncorrelated predictors, and that the former ellipse includes values of 0 for both β 1 and β individually (though not simultaneously). Now consider dividing the regressors in the model into two subsets: one subset x 1 whose coefficients are ointly of interest, and the complementary subset x with the remaining regressors. The generalized variance inflation factor (or GVIF) compares the squared size of the oint confidence region for coefficients associated with x 1 with the squared size of the oint confidence region for similarly dispersed regressors that are uncorrelated with x : GVIF 1 = det R11 det R det R where R 11 is the correlation matrix among the regressors in x 1 ; R is the correlation matrix among the regressors in x ; R is the correlation matrix for all of the xs; and det is the matrix determinant. When there is only one regressor in x 1, the GVIF reduces to the usual VIF. JOHN FOX 4

5 References Marquardt, D. W. (1970) Generalized inverses, ridge regression, biased linear estimation, and nonlinear estimation. Technometrics, 1, Fox, J. and Monette, G. (199) Generalized collinearity diagnostics. Journal of the American Statistical Association, 87,

6 Figure Captions Figure 1. The square-root of the variance inflation factor VIF as a function of the multiple correlation R from the regression of x on the other predictors. Figure. (a) Data ellipses for uncorrelated (solid) and highly correlated (broken) predictors; (b) corresponding 95-percent confidence ellipses. 6

7 VIF R

8 (a) (b) x (x1, x ) β (b 1, b ) β 1 x 1 (0,0)