Multicollinearity Exercise

Transcription

1 Multicollinearity Exercise Use the attached SAS output to answer the questions. [OPTIONAL: Copy the SAS program below into the SAS editor window and run it.] You do not need to submit any output, so there is no need to print anything... Identify which variables are key participants in the most serious near linear dependency in the data. Hint: Look at the Variance Decomposition Proportions associated with the smallest eigenvalue of X X. 2. Which variable has the wrong sign for its coefficient in this regression? Explain why its sign is wrong. 3. What is the smallest value of the ridge constant (k) that fixes the sign of the coefficient you named in #2? 4. What is the smallest value of the ridge constant (k) that reduces all VIF s so that they are below the guideline of 0? 5. What is the smallest value of k that seems (in your opinion) to stabilize the coefficients? 6. If one principal component is removed, give the estimated coefficients for X, X2, X3, X4. Does this fix the one with the wrong sign? ******************************************************************* ************ LAW SCHOOL ADMISSION DATA ****************** **************** PARTLY FROM PAGE 599 OF SMITH *************** *******************************************************************; **** DATA FOR 20 STUDENTS ****** Y IS THE LAW SCHOOL GPA X IS THE UNDERGRADUATE SCHOOL GPA X2 IS THE LMAT PERCENTILE X3 IS A RATING OF THE UNDERGRADUATE SCHOOL QUALITY X4 IS THE GRE SCORE; DATA LAW; INPUT Y X X2 X3 X4 NO $; CARDS; a b c d e f g h i j k ; TITLE 'LAW SCHOOL ADMISSIONS DATA'; PROC CORR; VAR Y X X2 X3 X4; PROC REG; MODEL Y=X X2 X3 X4 / COLLIN VIF; PROC REG RIDGE = 0 TO.0 BY.00 OUTEST=B; MODEL Y=X X2 X3 X4 ; PROC PRINT; PROC PLOT; PLOT (X X2 X3 X4) * _RIDGE_ / VREF=0 VPOS=25 HPOS=45; PROC REG DATA = LAW RIDGE= 0 TO.0 BY.00 OUTEST=C OUTVIF; MODEL Y=X X2 X3 X4; PROC PRINT; PROC REG DATA = LAW PCOMIT= 2 3 OUTEST=C; MODEL Y=X X2 X3 X4; PROC PRINT; run;

2 The CORR Procedure 5 Variables: Y X X2 X3 X4 Simple Statistics Variable N Mean Std Dev Sum Minimum Maximum Y X X X X Pearson Correlation Coefficients, N = 20 Prob > r under H0: Rho=0 Y X X2 X3 X4 Y <.000 <.000 <.000 X X < <.000 X < X < <

3 The REG Procedure Model: MODEL Dependent Variable: Y Number of Observations Read 20 Number of Observations Used 20 Analysis of Variance Source DF Sum of Squares Mean Square F Value Pr > F Model <.000 Error Corrected Total Root MSE RSquare Dependent Mean Adj RSq Coeff Var Parameter Estimates Variable DF Parameter Estimate Standard Error t Value Pr > t Variance Inflation Intercept X X X < X Collinearity Diagnostics Number Eigenvalue Condition Index Proportion of Variation Intercept X X2 X3 X E E E E E8

4 Number Eigenvalue Condition Index Collinearity Diagnostics Proportion of Variation Intercept X X2 X3 X E E

5 The REG Procedure Model: MODEL Dependent Variable: Y

6

7 The REG Procedure Model: MODEL Dependent Variable: Y

8 MODEL TYPE DEPVAR RIDGE RMSE Intercept X X2 X3 X4 Y MODEL PARM S Y MODEL RIDGE Y MODEL RIDGE Y MODEL RIDGE Y MODEL RIDGE Y MODEL RIDGE Y MODEL RIDGE Y MODEL RIDGE Y MODEL RIDGE Y MODEL RIDGE Y MODEL RIDGE Y MODEL RIDGE Y

9 Plot of X*_RIDGE_. Legend: A = obs, B = 2 obs, etc. X 0.5 ˆ A 0.0 ˆ 0.05 ˆ A A A A A A A A A A 0.00 ˆƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Šƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒ Ridge regression control value NOTE: obs had missing values.

10 etc. Plot of X2*_RIDGE_. Legend: A = obs, B = 2 obs, X2 6 ˆ A 4 ˆ 2 ˆ A A A A A A A A A A 0 ˆƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Šƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒ Ridge regression control value NOTE: obs had missing values.

11 etc. Plot of X3*_RIDGE_. Legend: A = obs, B = 2 obs, X3 0.5 ˆ A A A A A A A A A A A 0.0 ˆ 0.05 ˆ 0.00 ˆƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ Šƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒ Ridge regression control value NOTE: obs had missing values.

12 Plot of X4*_RIDGE_. Legend: A = obs, B = 2 obs, etc. X ˆ A A A A A A A A A 0 ˆƒƒƒƒƒAƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒƒ ˆ ˆ ˆ A 0.00 ˆ Šƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒƒƒƒƒˆƒƒƒ Ridge regression control value NOTE: obs had missing values.

13 MODEL TYPE DE P LAW SCHOOL ADMISSIONS DATA RIDGE RMSE Intcpt X X2 X3 X4 MODEL PARMS Y MODEL RIDGEVIF Y MODEL RIDGE Y MODEL RIDGEVIF Y MODEL RIDGE Y MODEL RIDGEVIF Y MODEL RIDGE Y MODEL RIDGEVIF Y MODEL RIDGE Y MODEL RIDGEVIF Y MODEL RIDGE Y MODEL RIDGEVIF Y MODEL RIDGE Y MODEL RIDGEVIF Y MODEL RIDGE Y MODEL RIDGEVIF Y MODEL RIDGE Y MODEL RIDGEVIF Y MODEL RIDGE Y MODEL RIDGEVIF Y MODEL RIDGE Y MODEL RIDGEVIF Y MODEL RIDGE Y

14 Obs MODEL TYPE D E P PCOMIT RMSE Intcpt X X2 X3 X4 MODEL PARMS Y MODEL IPC Y MODEL IPC Y MODEL IPC Y