Chapter 11 : State SAT scores for 1982 Data Listing

Transcription

1 EXST3201 Chapter 12a Geaghan Fall 2005: Page 1 Chapter 12 : Variable selection An example: State SAT scores In 1982 there was concern for scores of the Scholastic Aptitude Test (SAT) scores that varied greatly between states. Researchers decided to study the scores to determine what factors caused the differences between the states. 1 ********************************************************; 2 *** State average SAT scores ***; 3 ********************************************************; 4 5 dm'log;clear;output;clear'; 6 options nodate nocenter nonumber ps=512 ls=99 nolabel; 7 ODS HTML style=minimal rs=none 7! body='c:\geaghan\current\exst3201\fall2005\sas\statesat01.html' ; NOTE: Writing HTML Body file: C:\Geaghan\Current\EXST3201\Fall2005\SAS\StateSAT01.html 8 9 Title1 ''; 10 filename input1 'C:\Geaghan\Current\EXST3201\Datasets\ASCII\case1201.csv'; data StateSAT; length state $ 15; 13 infile input1 missover DSD dlm="," firstobs=2; 14 input STATE $ SAT TAKERS INCOME YEARS PUBLIC EXPEND RANK; 15 label SAT = 'Average SAT score (verbal+quantitative)' 16 Takers = 'percent of total eligible students taking the test' 17 Income = 'Median income of test takers' 18 years = 'Ave of yrs of formal study in social sci, nat sci & humanities' 19 public = 'percentage of takers attending public secondary schools' 20 expend = 'total state expenditure on secondary schools' 21 rank = 'median percentile ranking of test takers in their secondary school classes'; 22 datalines; NOTE: The infile INPUT1 is: File Name=C:\Geaghan\Current\EXST3201\Datasets\ASCII\case1201.csv, RECFM=V,LRECL=256 NOTE: 50 records were read from the infile INPUT1. The minimum record length was 64. The maximum record length was 99. NOTE: The data set WORK.STATESAT has 50 observations and 8 variables. NOTE: DATA statement used (Total process time): 0.01 seconds 0.02 seconds 23 run; PROC PRINT DATA=StateSAT; TITLE2 'Data Listing'; RUN; NOTE: There were 50 observations read from the data set WORK.STATESAT. NOTE: The PROCEDURE PRINT printed page 1. NOTE: PROCEDURE PRINT used (Total process time): 0.26 seconds 0.04 seconds Data Listing Obs state SAT TAKERS INCOME YEARS PUBLIC EXPEND RANK 1 Iowa SouthDakota NorthDakota Kansas Nebraska Montana Minnesota Utah Wyoming Wisconsin Oklahoma Arkansas Tennessee NewMexico Idaho Mississippi Kentucky Colorado

2 EXST3201 Chapter 12a Geaghan Fall 2005: Page 2 19 Washington Arizona Illinois Louisiana Missouri Michigan WestVirginia Alabama Ohio NewHampshire Alaska Nevada Oregon Vermont California Delaware Connecticut NewYork Maine Florida Maryland Virginia Massachusetts Pennsylvania RhodeIsland NewJersey Texas Indiana Hawaii NorthCarolina Georgia SouthCarolina options ps=45 ls=99 nolabel; 28 PROC REG DATA=StateSAT; Title2 'Fit of SAT with REG'; 29 MODEL SAT = TAKERS INCOME YEARS PUBLIC EXPEND RANK / VIF partial; 30 RUN; NOTE: The PROCEDURE REG printed pages 2-9. NOTE: PROCEDURE REG used (Total process time): 0.28 seconds 0.07 seconds Fit of SAT with REG The REG Procedure Model: MODEL1 Dependent Variable: SAT Number of Observations Read 50 Number of Observations Used 50 Model <.0001 Error Parameter Estimates Variance Variable DF Estimate Error t Value Pr > t Inflation Intercept TAKERS INCOME YEARS PUBLIC EXPEND RANK

3 EXST3201 Chapter 12a Geaghan Fall 2005: Page 3 Fit of SAT with REG The REG Procedure Model: MODEL1 Partial Regression Residual Plot SAT Intercept SAT TAKERS SAT INCOME

4 EXST3201 Chapter 12a Geaghan Fall 2005: Page 4 Fit of SAT with REG The REG Procedure Model: MODEL1 Partial Regression Residual Plot SAT YEARS SAT PUBLIC SAT EXPEND

5 EXST3201 Chapter 12a Geaghan Fall 2005: Page 5 Fit of SAT with REG The REG Procedure Model: MODEL1 Partial Regression Residual Plot SAT RANK First, fit and examine the model. Note the VIF and numerous non-significant variables. We want to find a subset of significant variables that describe the variation in SAT scores. The approach we use will also tend to reduce the multicolinearity problem (though not necessarily). The text suggests 3 specific issues related to this process of variable selection. 1) Adjusting for a large set of explanatory variables. There may be a specific hypothesis to be tested, and that variable should be included in the analysis (though not necessarily initially), but variance can be reduced by including other explanatory variables to refine the analysis. 2) Fishing for an explanation. Often there is not clear variable that must be included or any interest in testing a particular variable. The objective is to determine which variable may be correlated and may ultimately be useful explanatory varibles. 3) Prediction. The objective may not be interpretation, but rather prediction. The variable selection procedure can be used to find useful independent variables for future predictions. Variable selection techniques. Stepwise Regression : we know that we should not delete batches of variable from regressions when they are not significant, because we do not know what the effect of adjusting for each other has been (except when order dependent TYPE I SS has been used, as in a pure polynomial). Generally, when we wish to examine the effect of adding and deleting variables, this should be done one variable at a time. There is an algorithm to do this automatically called stepwise regression. Consider p 1 candidate variables where we want to select the best set. One possibility is to fit and examine all possible combinations of the variables. This all possible regressions approach is used, but there are other more automated procedures.

6 EXST3201 Chapter 12a Geaghan Fall 2005: Page 6 1) FORWARD stepwise regression: Start by fitting all simple linear regressions between Y i and the p 1 different X i variables. These would usually be evaluated with the F statistic (TYPE III) to determine which variable is the best predictor. So, out of the p 1 variables, the one with the best F statistic is the one that fits best. Now, that the BEST variable has been selected we want to determine if there is a second variable model which is also useful. There are p 2 variables remaining, so we try our best variable together with each of the p 2 remaining variable to determine if there is a second variable that is also significant. If there is a second best variable then it is included with the first, giving a two factor model as the best model. At this point there are p 3 variables remaining, so we try our best two factors together with each of the p 3 remaining variable to determine if there is a third best variable that is significant. If there is a third best variable then it is included with the first two, giving a three factor model as the best model. This process is repeated until no variable can be found that will enter the model at some predetermined criteria. With Forward stepwise regression, once a variable is entered into the model, it stays in the model. There are no provisions for removing variables once entered. This process is called FORWARD stepwise regression. Sometimes, no single variable will enter and be significant, but some group of 2 or 3, between them, will lower the MSE sufficiently for all to be significant. FORWARD will not catch this. Although the F statistic is the usual criteria, other criteria can be used, such as the R 2. 2) BACKWARDS elimination. This procedure starts with a fit the full model (p 1 variables). Examine the F statistic (TYPE II, fully adjusted for all other variables in the model). We first find the least significant of the p 1 variables, and eliminate this variable only. THIS IS DONE ONE VARIABLE AT A TIME. Now, refit the full model, minus the eliminated variable (p 2 variables). Again find the least significant and eliminate this variable. Repeat until all variables in the model are significant at some predetermine level of significance. Sometimes, a variable added to a model by FORWARD selection is not significant at some later point, after being adjusted for other variables in the model. Since no variable can be removed once entered, backwards elimination can avoid this problem. 3) STEPWISE selection (FORWARD selection with a backward glance). This selection proceeds as does FORWARD, but after each addition all variables in the model are examined to insure that they remain significant. If not significant, they are removed from the model. Selection criteria 2 R p, 2 AdjR p and MSE p can be used to graphically compare and evaluate models. The subscript p refers to the number of parameters in the model Mallow's Cp criterion Use of this statistic presumes no bias in the full model MSE, so the full model should be carefully chosen to have little or no multicolinearity. The Cp statistics will be approximately equal to p if there is no bias in the regression model ˆ ˆ = Cp p ( n p full ) σ σ 2 ˆ σ full

7 EXST3201 Chapter 12a Geaghan Fall 2005: Page 7 SAS application Selection options: FORWARD, BACKWARD, STEPWISE, NONE (full model). These do the selection techniques discussed above. Additional options for working with these are; SLENTRY = alpha value; level of entry for stepwise and forward SLSTAY = alpha value; level of remaining for backward and stepwise INCLUDE = n; automatically puts the first n variables of the list into the regression permanently (they will not be removed) RSQUARE, ADJRSQ, CP : These are not sequential model development techniques. These are more like all possible regressions analysis, but they simply calculate the R 2, AdjR 2 and the C p statistic for all models to find the best ones. Options for controlling these techniques are: START = n STOP = n (will stop all methods) BEST = n 32 options ps=512 ls=111 nolabel; 33 PROC REG DATA=StateSAT; Title2 'Forward selection stepwise regression'; 34 MODEL SAT = TAKERS INCOME YEARS PUBLIC EXPEND RANK / selection=forward; 35 RUN; NOTE: The PROCEDURE REG printed page 10. NOTE: PROCEDURE REG used (Total process time): 0.17 seconds 0.10 seconds Forward selection stepwise regression The REG Procedure Model: MODEL1 Dependent Variable: SAT Number of Observations Read 50 Number of Observations Used 50 Forward Selection: Step 1 Variable RANK Entered: R-Square = and C(p) = Model <.0001 Error Intercept RANK <.0001 Bounds on condition number: 1, 1 -

8 EXST3201 Chapter 12a Geaghan Fall 2005: Page 8 Forward Selection: Step 2 Variable YEARS Entered: R-Square = and C(p) = Model <.0001 Error Intercept YEARS <.0001 RANK <.0001 Bounds on condition number: 1.005, Forward Selection: Step 3 Variable EXPEND Entered: R-Square = and C(p) = Model <.0001 Error Intercept YEARS <.0001 EXPEND RANK <.0001 Bounds on condition number: , Forward Selection: Step 4 Variable PUBLIC Entered: R-Square = and C(p) = Model <.0001 Error Intercept YEARS PUBLIC EXPEND RANK <.0001 Bounds on condition number: ,

9 EXST3201 Chapter 12a Geaghan Fall 2005: Page 9 Forward Selection: Step 5 Variable TAKERS Entered: R-Square = and C(p) = Model <.0001 Error Intercept TAKERS YEARS PUBLIC EXPEND RANK Bounds on condition number: , No other variable met the significance level for entry into the model. Summary of Forward Selection Variable Number Partial Model Step Entered Vars In R-Square R-Square C(p) F Value Pr > F 1 RANK < YEARS < EXPEND PUBLIC TAKERS PROC REG DATA=StateSAT; Title2 'Backward elimination stepwise regression'; 37 MODEL SAT = TAKERS INCOME YEARS PUBLIC EXPEND RANK / selection=backward; 38 RUN; NOTE: The PROCEDURE REG printed page 11. NOTE: PROCEDURE REG used (Total process time): 0.13 seconds 0.05 seconds Backward elimination stepwise regression The REG Procedure Model: MODEL1 Dependent Variable: SAT Number of Observations Read 50 Number of Observations Used 50

10 EXST3201 Chapter 12a Geaghan Fall 2005: Page 10 Backward Elimination: Step 0 All Variables Entered: R-Square = and C(p) = Model <.0001 Error Intercept TAKERS INCOME YEARS PUBLIC EXPEND RANK Bounds on condition number: , Backward Elimination: Step 1 Variable INCOME Removed: R-Square = and C(p) = Model <.0001 Error Intercept TAKERS YEARS PUBLIC EXPEND RANK Bounds on condition number: , Backward Elimination: Step 2 Variable TAKERS Removed: R-Square = and C(p) = Model <.0001 Error Intercept YEARS PUBLIC EXPEND RANK <.0001

11 EXST3201 Chapter 12a Geaghan Fall 2005: Page 11 Bounds on condition number: , Backward Elimination: Step 3 Variable PUBLIC Removed: R-Square = and C(p) = Model <.0001 Error Intercept YEARS <.0001 EXPEND RANK <.0001 Bounds on condition number: , All variables left in the model are significant at the level. Summary of Backward Elimination Variable Number Partial Model Step Removed Vars In R-Square R-Square C(p) F Value Pr > F 1 INCOME TAKERS PUBLIC PROC REG DATA=StateSAT; Title2 'Stepwise regression'; 40 MODEL SAT = TAKERS INCOME YEARS PUBLIC EXPEND RANK / 41 selection=stepwise slentry=0.20 slstay=0.05; 42 RUN; NOTE: The PROCEDURE REG printed page 12. NOTE: PROCEDURE REG used (Total process time): 0.12 seconds 0.04 seconds Stepwise regression The REG Procedure Model: MODEL1 Dependent Variable: SAT Number of Observations Read 50 Number of Observations Used 50

12 EXST3201 Chapter 12a Geaghan Fall 2005: Page 12 Stepwise Selection: Step 1 Variable RANK Entered: R-Square = and C(p) = Model <.0001 Error Intercept RANK <.0001 Bounds on condition number: 1, 1 Stepwise Selection: Step 2 Variable YEARS Entered: R-Square = and C(p) = Model <.0001 Error Intercept YEARS <.0001 RANK <.0001 Bounds on condition number: 1.005, Stepwise Selection: Step 3 Variable EXPEND Entered: R-Square = and C(p) = Model <.0001 Error Intercept YEARS <.0001 EXPEND RANK <.0001 Bounds on condition number: ,

13 EXST3201 Chapter 12a Geaghan Fall 2005: Page 13 Stepwise Selection: Step 4 Variable PUBLIC Entered: R-Square = and C(p) = Model <.0001 Error Intercept YEARS PUBLIC EXPEND RANK <.0001 Bounds on condition number: , Stepwise Selection: Step 5 Variable PUBLIC Removed: R-Square = and C(p) = Model <.0001 Error Intercept YEARS <.0001 EXPEND RANK <.0001 Bounds on condition number: , All variables left in the model are significant at the level. The stepwise method terminated because the next variable to be entered was just removed. Summary of Stepwise Selection Variable Variable Number Partial Model Step Entered Removed Vars In R-Square R-Square C(p) F Value Pr > F 1 RANK < YEARS < EXPEND PUBLIC PUBLIC PROC REG DATA=StateSAT; Title2 'Reduced model with diagnostics'; 44 MODEL SAT = YEARS EXPEND RANK / VIF; 45 output out=next1 r=resid p=yhat lclm=lclm uclm=uclm lcl=lcli ucl=ucli 46 student=student rstudent=rstudent cookd=cookd h=leverage dffits=dffits; 47 RUN; NOTE: The data set WORK.NEXT1 has 50 observations and 19 variables. NOTE: The PROCEDURE REG printed page 13. NOTE: PROCEDURE REG used (Total process time): 0.13 seconds 0.06 seconds

14 EXST3201 Chapter 12a Geaghan Fall 2005: Page 14 Reduced model with diagnostics The REG Procedure Model: MODEL1 Dependent Variable: SAT Number of Observations Read 50 Number of Observations Used 50 Model <.0001 Error Root MSE R-Square Dependent Adj R-Sq Coeff Var Parameter Estimates Variance Variable DF Estimate Error t Value Pr > t Inflation Intercept YEARS < EXPEND RANK < proc print data=next1; 50 var state SAT yhat resid student rstudent cookd leverage dffits; 51 RUN; NOTE: There were 50 observations read from the data set WORK.NEXT1. NOTE: The PROCEDURE PRINT printed page 14. NOTE: PROCEDURE PRINT used (Total process time): 0.07 seconds 0.03 seconds 52 options ps=60 ls=132; 53 proc plot data=next1; TITLE2 'Various plot with state variable'; 54 plot resid * yhat = state / vref=0; 55 RUN; 55! options ps=40 ls=132; NOTE: There were 50 observations read from the data set WORK.NEXT1. NOTE: The PROCEDURE PLOT printed page 15. NOTE: PROCEDURE PLOT used (Total process time): 0.06 seconds 0.00 seconds 56 proc plot data=next1; TITLE2 'Various plot with state variable'; 57 plot student * state / vref=2; 58 plot rstudent * state / vref=2; 59 plot leverage * state / vref=0.16; /* 2p/n = 2*4/50 = 0.16 */ 60 plot cookd * state / vref=1; 61 plot dffits * state / vref=1; 62 RUN; 62! OPTIONS PS=256 ls=132; NOTE: There were 50 observations read from the data set WORK.NEXT1. NOTE: The PROCEDURE PLOT printed pages NOTE: PROCEDURE PLOT used (Total process time): 0.06 seconds 0.00 seconds

15 EXST3201 Chapter 12a Geaghan Fall 2005: Page 15 Reduced model with diagnostics Obs state SAT yhat resid student rstudent cookd leverage dffits 1 Iowa SouthDakota NorthDakota Kansas Nebraska Montana Minnesota Utah Wyoming Wisconsin Oklahoma Arkansas Tennessee NewMexico Idaho Mississippi Kentucky Colorado Washington Arizona Illinois Louisiana Missouri Michigan WestVirginia Alabama Ohio NewHampshire Alaska Nevada Oregon Vermont California Delaware Connecticut NewYork Maine Florida Maryland Virginia

16 EXST3201 Chapter 12a Geaghan Fall 2005: Page Massachusetts Pennsylvania RhodeIsland NewJersey Texas Indiana Hawaii NorthCarolina Georgia SouthCarolina Various plot with state variable Plot of resid*yhat. Symbol is value of state. resid 60 + I T 40 + N S I C K I 20 + V M N N U M M O O H N M D M N A N I R O M W W T V C A W A N C L K G P N A W M S yhat NOTE: 2 obs hidden.

17 EXST3201 Chapter 12a Geaghan Fall 2005: Page 17 Various plot with state variable Plot of student*state. Legend: A = 1 obs, B = 2 obs, etc. student A A A A A A A A A A A A A A A A A A A A A 0 + A A A A A A A A A A A A A A A A A A A A A A A A -2 + A A A A A A A A A C C C D F G H I I I I K K L M M M M M M M M N N N N N N N N O O O P R S S T T U V V W W W W l l r r a o o e l e a d l n o a e o a a a i i i i o e e e e e e o o h k r e h o o e e t e i a e i y a a i k l l n l o o w a l d w n n u i r s c n s s n b v w w w w r r i l e n o u u n x a r r s s s o b s z a i o n a r r a h i i a s t i n y s h n s s t r a H J M Y t t o a g n d t t n a h m g h t c m a k o n f r e w i g i o n a a u s e l a i e i o a a d a e e o h h h o s e h h e s o i i V o i m a n s o a c a d i i o n s c i a c g s s u n s a m r x r C D o n y I C D s n n n i n n a a a r d t r a a i a k a n h a o s r a k p s i k a a m l s a a s t i g r s g s n o i e s y n d u n t i i a s e c r k a v l r k e a t g i i c a s a p h y o o o a a o o e o i n a u e p i l t n n l t n n t t i r i a i d i a i t e n a n a s a a Plot of rstudent*state. Legend: A = 1 obs, B = 2 obs, etc. rstudent A A A A A A A A A A A A A A A A A A A A A 0 + A A A A A A A A A A A A A A A A A A A A A A A A -2 + A A A A A A A A A C C C D F G H I I I I K K L M M M M M M M M N N N N N N N N O O O P R S S T T U V V W W W W l l r r a o o e l e a d l n o a e o a a a i i i i o e e e e e e o o h k r e h o o e e t e i a e i y a a i k l l n l o o w a l d w n n u i r s c n s s n b v w w w w r r i l e n o u u n x a r r s s s o b s z a i o n a r r a h i i a s t i n y s h n s s t r a H J M Y t t o a g n d t t n a h m g h t c m a k o n f r e w i g i o n a a u s e l a i e i o a a d a e e o h h h o s e h h e s o i i V o i m a n s o a c a d i i o n s c i a c g s s u n s a m r x r C D o n y I C D s n n n i n n a a a r d t r a a i a k a n h a o s r a k p s i k a a m l s a a s t i g r s g s n o i e s y n d u n t i i a s e c r k a v l r k e a t g i i c a s a p h y o o o a a o o e o i n a u e p i l t n n l t n n t t i r i a i d i a i t e n a n a s a a state state

18 EXST3201 Chapter 12a Geaghan Fall 2005: Page 18 Various plot with state variable Plot of leverage*state. Legend: A = 1 obs, B = 2 obs, etc. leverage A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A C C C D F G H I I I I K K L M M M M M M M M N N N N N N N N O O O P R S S T T U V V W W W W l l r r a o o e l e a d l n o a e o a a a i i i i o e e e e e e o o h k r e h o o e e t e i a e i y a a i k l l n l o o w a l d w n n u i r s c n s s n b v w w w w r r i l e n o u u n x a r r s s s o b s z a i o n a r r a h i i a s t i n y s h n s s t r a H J M Y t t o a g n d t t n a h m g h t c m a k o n f r e w i g i o n a a u s e l a i e i o a a d a e e o h h h o s e h h e s o i i V o i m a n s o a c a d i i o n s c i a c g s s u n s a m r x r C D o n y I C D s n n n i n n a a a r d t r a a i a k a n h a o s r a k p s i k a a m l s a a s t i g r s g s n o i e s y n d u n t i i a s e c r k a v l r k e a t g i i c a s a p h y o o o a a o o e o i n a u e p i l t n n l t n n t t i r i a i d i a i t e n a n a s a a state Plot of cookd*state. Legend: A = 1 obs, B = 2 obs, etc. 2 + A cookd A A A A 0 + A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A C C C D F G H I I I I K K L M M M M M M M M N N N N N N N N O O O P R S S T T U V V W W W W l l r r a o o e l e a d l n o a e o a a a i i i i o e e e e e e o o h k r e h o o e e t e i a e i y a a i k l l n l o o w a l d w n n u i r s c n s s n b v w w w w r r i l e n o u u n x a r r s s s o b s z a i o n a r r a h i i a s t i n y s h n s s t r a H J M Y t t o a g n d t t n a h m g h t c m a k o n f r e w i g i o n a a u s e l a i e i o a a d a e e o h h h o s e h h e s o i i V o i m a n s o a c a d i i o n s c i a c g s s u n s a m r x r C D o n y I C D s n n n i n n a a a r d t r a a i a k a n h a o s r a k p s i k a a m l s a a s t i g r s g s n o i e s y n d u n t i i a s e c r k a v l r k e a t g i i c a s a p h y o o o a a o o e o i n a u e p i l t n n l t n n t t i r i a i d i a i t e n a n a s a a state

19 EXST3201 Chapter 12a Geaghan Fall 2005: Page 19 Various plot with state variable Plot of dffits*state. Legend: A = 1 obs, B = 2 obs, etc. dffits A A A A A A A A A 0 + A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A -2 + A A A A A C C C D F G H I I I I K K L M M M M M M M M N N N N N N N N O O O P R S S T T U V V W W W W l l r r a o o e l e a d l n o a e o a a a i i i i o e e e e e e o o h k r e h o o e e t e i a e i y a a i k l l n l o o w a l d w n n u i r s c n s s n b v w w w w r r i l e n o u u n x a r r s s s o b s z a i o n a r r a h i i a s t i n y s h n s s t r a H J M Y t t o a g n d t t n a h m g h t c m a k o n f r e w i g i o n a a u s e l a i e i o a a d a e e o h h h o s e h h e s o i i V o i m a n s o a c a d i i o n s c i a c g s s u n s a m r x r C D o n y I C D s n n n i n n a a a r d t r a a i a k a n h a o s r a k p s i k a a m l s a a s t i g r s g s n o i e s y n d u n t i i a s e c r k a v l r k e a t g i i c a s a p h y o o o a a o o e o i n a u e p i l t n n l t n n t t i r i a i d i a i t e n a n a s a a state 63 PROC UNIVARIATE DATA=NEXT1 NORMAL PLOT; VAR resid; RUN; NOTE: The PROCEDURE UNIVARIATE printed page 21. NOTE: PROCEDURE UNIVARIATE used (Total process time): 0.09 seconds 0.04 seconds 64

20 EXST3201 Chapter 12a Geaghan Fall 2005: Page 20 Various plot with state variable The UNIVARIATE Procedure Variable: resid Moments N 50 Sum Weights 50 0 Sum Observations 0 Std Deviation Variance Skewness Kurtosis Uncorrected SS Corrected SS Coeff Variation. Std Error Basic Statistical Measures Location Variability Std Deviation Median Variance Mode. Range Interquartile Range Tests for Location: Mu0=0 Test -Statistic p Value Student's t t 0 Pr > t Sign M 1 Pr >= M Signed Rank S 57.5 Pr >= S Tests for Normality Test --Statistic p Value Shapiro-Wilk W Pr < W Kolmogorov-Smirnov D Pr > D < Cramer-von Mises W-Sq Pr > W-Sq Anderson-Darling A-Sq Pr > A-Sq Quantiles (Definition 5) Quantile Estimate 100% Max % % % % Q % Median % Q % % % % Min Extreme Observations Lowest Highest----- Value Obs Value Obs

21 EXST3201 Chapter 12a Geaghan Fall 2005: Page 21 Stem Leaf Boxplot Normal Probability Plot *+ * * * ** *** **** ** *** *--+--* *** **** **** ** * ** * * * * * Multiply Stem.Leaf by 10** Selection options: FORWARD, BACKWARD, STEPWISE, NONE (full model) These do the selection techniques discussed. Additional options working with these are; SLENTRY = alpha value, level of entry for stepwise and forward SLSTAY = alpha value, level of remaining for backward and stepwise INCLUDE = n, automatically puts the first n variables of the list into the regression (do not remove) RSQUARE, ADJRSQ, CP : These are like all possible regressions. These will simply calculate the R 2, AdjR 2 and C p statistic for all models to find the best ones. Other criteria include Akaike's information criterion (AIC), Schwartz s Bayesian information criterion (SBC), SSE, MSE and others. Additional options START = n STOP = n (will stop all methods) BEST = n eg. with these options we could determine the best 5 models at each level, starting with 3 factors, ending with 6 factors 65 PROC REG DATA=StateSAT; Title2 'RSquare procedure'; 66 MODEL SAT = TAKERS INCOME YEARS PUBLIC EXPEND RANK / 67 selection=rsquare start=1 stop=4 best=5 cp adjrsq aic sbc mse; 68 RUN; 69 NOTE: The PROCEDURE REG printed page 22. NOTE: PROCEDURE REG used (Total process time): 0.12 seconds 0.06 seconds

22 EXST3201 Chapter 12a Geaghan Fall 2005: Page 22 RSquare procedure The REG Procedure Model: MODEL1 Dependent Variable: SAT R-Square Selection Method Number of Observations Read 50 Number of Observations Used 50 Number in Adjusted Model R-Square R-Square C(p) AIC BIC MSE SBC SSE Variables in Model YEARS RANK EXPEND RANK TAKERS YEARS INCOME RANK PUBLIC RANK TAKERS RANK YEARS EXPEND RANK INCOME YEARS RANK TAKERS YEARS RANK YEARS PUBLIC RANK PUBLIC EXPEND RANK TAKERS YEARS EXPEND YEARS PUBLIC EXPEND RANK TAKERS YEARS EXPEND RANK INCOME YEARS EXPEND RANK INCOME YEARS PUBLIC RANK TAKERS INCOME YEARS RANK TAKERS YEARS PUBLIC RANK TAKERS YEARS PUBLIC EXPEND RANK INCOME YEARS PUBLIC EXPEND RANK TAKERS INCOME YEARS EXPEND RANK TAKERS INCOME YEARS PUBLIC RANK TAKERS INCOME PUBLIC EXPEND RANK TAKERS INCOME YEARS PUBLIC EXPEND

23 EXST3201 Chapter 12a Geaghan Fall 2005: Page proc print data=parm1; 72 * var ADJRSQ AIC BIC CP MSE RSQUARE SBC SSE; 73 run; NOTE: There were 24 observations read from the data set WORK.PARM1. NOTE: The PROCEDURE PRINT printed page 23. NOTE: PROCEDURE PRINT used (Total process time): 0.08 seconds 0.03 seconds RSquare procedure Obs _MODEL TYPE DEPVAR RMSE_ Intercept TAKERS INCOME YEARS PUBLIC EXPEND RANK 1 MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT MODEL1 PARMS SAT Obs SAT _IN P EDF SSE MSE RSQ ADJRSQ CP AIC BIC SBC_ PROC REG DATA=StateSAT; Title2 'Cp criterion'; 76 MODEL SAT = TAKERS INCOME YEARS PUBLIC EXPEND RANK / 77 selection=cp start=2 stop=5 best=10; 78 RUN; ods html close; 81 quit; NOTE: The PROCEDURE REG printed page 24. NOTE: PROCEDURE REG used (Total process time): 0.13 seconds 0.03 seconds

24 EXST3201 Chapter 12a Geaghan Fall 2005: Page 24 Cp criterion The REG Procedure Model: MODEL1 Dependent Variable: SAT C(p) Selection Method Number of Observations Read 50 Number of Observations Used 50 Number in Model C(p) R-Square Variables in Model YEARS PUBLIC EXPEND RANK YEARS EXPEND RANK TAKERS YEARS EXPEND RANK INCOME YEARS EXPEND RANK TAKERS YEARS PUBLIC EXPEND RANK INCOME YEARS PUBLIC EXPEND RANK TAKERS INCOME YEARS EXPEND RANK INCOME YEARS RANK INCOME YEARS PUBLIC RANK TAKERS INCOME YEARS RANK