Table of Contents. Logistic Regression- Illustration Carol Bigelow March 21, 2017

Transcription

1 Logistic Regression- Illustration Carol Bigelow March 21, 2017 Table of Contents Preliminary - Attach packages needed using command library( )... 2 Must have installed packages in console window first... 2 Upload data and check structure (continuous v factor, etc)... 2 Create Quartile Groupings of a Continuous Variable age (age_quartile)... 2 Create Medians of Continuous Variable ag, over quartile Groupings (age_qmedian)... 3 Create 0/1 Indicator of Heavy Drinking (alcohol_80plus). Check Create 0/1 Indicator of Heavy Smoking (smoking_30plus). Check Create Interaction of Heavy Drinking and Heavy Smoking (drinker_smoker) Convert agegp to a factor variable and label MODEL 1: Predictors = agegp (R knows it is a factor), heavy drinking MODEL 2: Predictors = agegp (as factor), heavy smoking MODEL 3: Predictors = agegp (as factor), heavy drinking, heavy smoking MODEL 4: Predictors = age (as factor), heavy drinking, heavy smoking, + Interaction Side- by- side Comparison of Models: BETAs Side- by- side Comparison of Models: ODDS RATIOs Likelihood Ratio Test for 2 "Hierarchical" (NULL: Beta for interaction = 0) - Easy Likelihood Ratio Test for 2 "Hierarchical" (NULL: Beta for interaction = 0) - Brute Force REGRESSION DIAGNOSTIC for Model 3: Numerical Measures of Fit REGRESSION DIAGNOSTIC for Model 3: Test of Model Adequacy (Link Test: NULL: beta hatsq=0) REGRESSION DIAGNOSTIC for Model 3: Hosmer Lemeshow GOF Test with 9 bins (NULL: Fit is good) REGRESSION DIAGNOSTIC for Model 3: Plot of ROC Curve (AUC = % Correctly Classified) REGRESSION DIAGNOSTIC for Model 3: Plot of Cook's Distances v Observation Number R Handouts \R for Logistic Regression 2017.docx Page 1 of 14

2 Preliminary - Attach packages needed using command library( ) Must have installed packages in console window first library(car) library(desctools) library(foreign) library(readstata13) library(ggplot2) library(gmodels) library(dplyr) library(mosaic) library(stargazer) library(lmtest) library(proc) library(rms) library(resourceselection) Upload data and check structure (continuous v factor, etc) link <- " dat <- read.dta13(link) glimpse(dat) Observations: 975 Variables: 7 $ case <dbl> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,... $ age <dbl> 42, 45, 35, 78, 45, 64, 76, 42, 48, 42, 37, 45, 60, 54,... $ agegp <dbl> 2, 3, 2, 6, 3, 4, 6, 2, 3, 2, 2, 3, 4, 3, 5, 4, 3, 4, 3,... $ tob <dbl> 0.0, 7.5, 0.0, 0.0, 7.5, 17.5, 2.5, 0.0, 12.5, 25.0, $ tobgp <dbl> 1, 1, 1, 1, 1, 2, 1, 1, 2, 3, 2, 2, 1, 3, 1, 1, 3, 1, 1,... $ alc <dbl> 139, 66, 24, 39, 64, 49, 1, 33, 55, 62, 77, 84, 4, 32, 2... $ alcgp <dbl> 4, 2, 1, 1, 2, 2, 1, 1, 2, 2, 2, 3, 1, 1, 1, 1, 1, 2, 1,... Create Quartile Groupings of a Continuous Variable age (age_quartile) This data set has two continuous variables: age (age, years) and tob (tobacco consumption, gm/day). It also contains two grouped versions: agegp and tobgp. Here we consider the variable age. In this lab session, let s create two new variables. Each is a two step process. age_quartile = Quartile of age, coded 1, 2, 3 or 4 age_qmedian = Median of age, within quartile of age 0. R Handouts \R for Logistic Regression 2017.docx Page 2 of 14

3 #1. Obtain Quartiles using command quantile(data$varibable, c(list )) quantile(dat$age, c(0,.25,.50,.75,1)) 0% 25% 50% 75% 100% #2. Create new variable with values = quartile using command cut( ) dat$age_quartile <- cut(dat$age, breaks=c(0,41,52,63,91), labels=c(1,2, 3,4)) Create Medians of Continuous Variable ag, over quartile Groupings (age_qmedian) #1. Determine median age in each quartile by(data = dat$age, INDICES = dat$age_quartile, FUN = function(x) median (x)) dat$age_quartile: 1 [1] dat$age_quartile: 2 [1] dat$age_quartile: 3 [1] dat$age_quartile: 4 [1] 69 #2. Create new variable with values = median in quartile dat$age_qmedian <- cut(dat$age, breaks = c(0, 41, 52, 63, 91), labels = c(35, 47, 59, 69)) #3. Check tally(age_quartile~age_qmedian,data=dat) age_qmedian age_quartile Ille- et- Vilaine Data: Illustration After creating some new variables for illustration purposes, 4 logistic regression models are fit and then compared side- by- side. Model 1: Predictors = heavy drinking, age Model 2: Predictors = heavy smoking, age Model 3: Predictors = heavy drinking, heavy smoking, age Model 4: Predictors = heavy drinking, heavy smoking, drinking x smoking interaction, age 0. R Handouts \R for Logistic Regression 2017.docx Page 3 of 14

4 Create 0/1 Indicator of Heavy Drinking (alcohol_80plus). Check. dat$alcohol_80plus <- ifelse(as.numeric(dat$alcgp)>=3,1,0) tally(alcgp~alcohol_80plus,data=dat) alcohol_80plus alcgp Create 0/1 Indicator of Heavy Smoking (smoking_30plus). Check. dat$smoking_30plus <- ifelse(as.numeric(dat$tobgp)==4,1,0) tally(tobgp~smoking_30plus,data=dat) smoking_30plus tobgp Create Interaction of Heavy Drinking and Heavy Smoking (drinker_smoker). dat$drinker_smoker <- as.numeric(dat$alcohol_80plus) * as.numeric(dat$s moking_30plus) Convert agegp to a factor variable and label. dat$agegp <- factor(dat$agegp, levels=c(1,2,3,4,5,6), labels=c("25-34","35-44","45-54","55-64","65-74","7 5+")) MODEL 1: Predictors = agegp (R knows it is a factor), heavy drinking. # NAME <- glm(yvariable ~ X1 + X2 + etc, data=dataframename, family=bin omial) model1 <- glm(case ~ agegp + alcohol_80plus, data = dat, family = binom ial) summary(model1) Call: glm(formula = case ~ agegp + alcohol_80plus, family = binomial, data = dat) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) 0. R Handouts \R for Logistic Regression 2017.docx Page 4 of 14

5 (Intercept) e- 07 *** agegp agegp ** agegp *** agegp *** agegp *** alcohol_80plus < 2e- 16 *** Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: on 974 degrees of freedom Residual deviance: on 968 degrees of freedom AIC: Number of Fisher Scoring iterations: 7 exp(cbind(or = coef(model1), confint(model1))) OR 2.5 % 97.5 % (Intercept) e agegp e agegp e agegp e agegp e agegp e alcohol_80plus e MODEL 2: Predictors = agegp (as factor), heavy smoking. model2 <- glm(case ~ agegp + smoking_30plus, data = dat, family = binom ial) summary(model2) Call: glm(formula = case ~ agegp + smoking_30plus, family = binomial, data = dat) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) e- 07 *** agegp agegp *** agegp e- 05 *** agegp e- 05 *** agegp *** smoking_30plus e- 07 *** Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: on 974 degrees of freedom Residual deviance: on 968 degrees of freedom 0. R Handouts \R for Logistic Regression 2017.docx Page 5 of 14

6 AIC: Number of Fisher Scoring iterations: 7 exp(cbind(or = coef(model2), confint(model2))) OR 2.5 % 97.5 % (Intercept) e e- 02 agegp e e+02 agegp e e+02 agegp e e+03 agegp e e+03 agegp e e+03 smoking_30plus e e+00 MODEL 3: Predictors = agegp (as factor), heavy drinking, heavy smoking. model3 <- glm(case ~ agegp + alcohol_80plus + smoking_30plus, data = da t, family = binomial) summary(model3) Call: glm(formula = case ~ agegp + alcohol_80plus + smoking_30plus, family = binomial, data = dat) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) e- 08 *** agegp agegp *** agegp e- 05 *** agegp e- 05 *** agegp e- 05 *** alcohol_80plus < 2e- 16 *** smoking_30plus e- 06 *** Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: on 974 degrees of freedom Residual deviance: on 967 degrees of freedom AIC: Number of Fisher Scoring iterations: 7 exp(cbind(or = coef(model3), confint(model3))) (Intercept) agegp35-44 agegp45-54 agegp55-64 agegp65-74 agegp75+ OR 2.5 % 97.5 % e e e e e e R Handouts \R for Logistic Regression 2017.docx Page 6 of 14

7 alcohol_80plus e+00 smoking_30plus e+00 MODEL 4: Predictors = agegp (as factor), heavy drinking, heavy smoking, + Interaction. model4 <- glm(case ~ agegp + alcohol_80plus + smoking_30plus + drinker_ smoker, data = dat, family = binomial) summary(model4) Call: glm(formula = case ~ agegp + alcohol_80plus + smoking_30plus + drinker_smoker, family = binomial, data = dat) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) e- 08 *** agegp agegp *** agegp e- 05 *** agegp e- 05 *** agegp e- 05 *** alcohol_80plus e- 15 *** smoking_30plus *** drinker_smoker Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: on 974 degrees of freedom Residual deviance: on 966 degrees of freedom AIC: Number of Fisher Scoring iterations: 7 exp(cbind(or = coef(model4), confint(model4))) OR 2.5 % 97.5 % (Intercept) e agegp e agegp e agegp e agegp e agegp e alcohol_80plus e smoking_30plus e drinker_smoker e R Handouts \R for Logistic Regression 2017.docx Page 7 of 14

8 Side- by- side Comparison of Models: BETAs. stargazer(model1, model2, model3, model4,title="logistic Regression of Esophageal Cancer - BETAs",type="text") Logistic Regression of Esophageal Cancer - BETAs ========================================================= Dependent variable: case (1) (2) (3) (4) agegp * 1.841* 1.882* (1.066) (1.065) (1.077) (1.088) agegp *** 3.648*** 3.500*** 3.533*** (1.023) (1.024) (1.037) (1.046) agegp *** 4.177*** 4.029*** 4.063*** (1.018) (1.020) (1.034) (1.043) agegp *** 4.412*** 4.394*** 4.425*** (1.023) (1.026) (1.041) (1.050) agegp *** 4.085*** 4.269*** 4.306*** (1.065) (1.065) (1.081) (1.090) alcohol_80plus 1.654*** 1.634*** 1.613*** (0.189) (0.192) (0.202) smoking_30plus 1.438*** 1.384*** 1.316*** (0.286) (0.307) (0.370) drinker_smoker (0.671) Constant *** *** *** *** (1.009) (1.013) (1.030) (1.038) Observations Log Likelihood Akaike Inf. Crit ========================================================= Note: *p<0.1; **p<0.05; ***p< R Handouts \R for Logistic Regression 2017.docx Page 8 of 14

9 Side- by- side Comparison of Models: ODDS RATIOs. OR <- function(x) exp(x) stargazer(model1, model2, model3, model4,apply.coef=or, title="logistic Regression of Esophageal Cancer - ODDS RATIOs", type="text") Logistic Regression of Esophageal Cancer - ODDS RATIOs ========================================================= Dependent variable: case (1) (2) (3) (4) agegp *** 6.268*** 6.300*** 6.563*** (1.066) (1.065) (1.077) (1.088) agegp *** *** *** *** (1.023) (1.024) (1.037) (1.046) agegp *** *** *** *** (1.018) (1.020) (1.034) (1.043) agegp *** *** *** *** (1.023) (1.026) (1.041) (1.050) agegp *** *** *** *** (1.065) (1.065) (1.081) (1.090) alcohol_80plus 5.228*** 5.122*** 5.018*** (0.189) (0.192) (0.202) smoking_30plus 4.211*** 3.990*** 3.727*** (0.286) (0.307) (0.370) drinker_smoker 1.252* (0.671) Constant (1.009) (1.013) (1.030) (1.038) Observations Log Likelihood Akaike Inf. Crit ========================================================= Note: *p<0.1; **p<0.05; ***p< R Handouts \R for Logistic Regression 2017.docx Page 9 of 14

10 Likelihood Ratio Test for 2 "Hierarchical" (NULL: Beta for interaction = 0) - Easy. It is of interest to know whether the inclusion of extra predictors to a model is statistically significant. The smaller model ( reduced ) contains the control variables. The larger model ( full ) contains the control variables plus the extra variables in question. Models. Reduced: logit[π X,X...,X ] = β +β X +...+β X 1 2 p p p Full: logit[π X,X...,X, X,X,...,X ] = β +β X+...+βX+ β X +...+β X 1 2 p p+1 p+2 p+k p p p+1 p+1 p+k p+ k Null and Alternative Hypotheses: H: β = β =... = β = 0 O p+1 p+2 p+k H: not A Definition Likelihood Ratio Test (LR) LR statistic = DevianceREDUCED - DevianceFULL = [ (- 2) ln (Likelihood) REDUCED ] - [ (- 2) ln (Likelihood) FULL ] Under the null hypothesis, LR is distributed Chi SquareDF=k Ille- et- Vilaine Data: Illustration A likelihood ratio test is performed to assess the stastistical significance of the interaction of heavy drinking and heavy smoking in the model, controlling for age and the main effects of each of heavy drinking and heavy smoking. Thus, Model reduced : Predictors = age, heavy drinking, heavy smoking Model full : Predictors = age, heavy drinking, heavy smoking + (drinking x smoking) 0. R Handouts \R for Logistic Regression 2017.docx Page 10 of 14

11 # lrtest(reduced, FULL) lrtest(model3, model4) Model 1: case ~ agegp + alcohol_80plus + smoking_30plus Model 2: case ~ agegp + alcohol_80plus + smoking_30plus + drinker_smoker L.R. Chisq d.f. P Likelihood Ratio Test for 2 "Hierarchical" (NULL: Beta for interaction = 0) - Brute Force. LR.statistic.1 <- model3$deviance - model4$deviance pchisq(lr.statistic.1, 1, lower.tail = FALSE) [1] LR.statistic.2 <- - 2*logLik(model3)[1] - (- 2*logLik(model4)[1]) pchisq(lr.statistic.2, 1, lower.tail = FALSE) [1] REGRESSION DIAGNOSTIC for Model 3: Numerical Measures of Fit. Now you have a model that is your candidate final model. There are lots of further explorations you can do to assess whether this really is a good final model. Ille- et- Vilaine Data: Illustration Having retained the null hypothesis in our likelihood ratio test of the interaction of heavy smoking and heavy drinking, our candidate final model is model 3, containing: heavy drinking, heavy smoking, and age. lrm(model3) Logistic Regression Model lrm(formula = model3) Model Likelihood Discrimination Rank Discrim. Ratio Test Indexes Indexes Obs 975 LR chi R C d.f. 7 g Dxy Pr(> chi2) < gr gamma max deriv 2e- 09 gp tau- a Brier Coef S.E. Wald Z Pr(> Z ) Intercept < agegp= R Handouts \R for Logistic Regression 2017.docx Page 11 of 14

12 agegp= agegp= < agegp= < agegp= < alcohol_80plus < smoking_30plus < REGRESSION DIAGNOSTIC for Model 3: Test of Model Adequacy (Link Test: NULL: beta hatsq=0). hat <- predict(model3) hatsq <- hat^2 linktest <- summary(glm(case ~ hat + hatsq, data=dat, family=binomial)) linktest Call: glm(formula = case ~ hat + hatsq, family = binomial, data = dat) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) hat e- 13 *** hatsq Signif. codes: 0 '***' '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 (Dispersion parameter for binomial family taken to be 1) Null deviance: on 974 degrees of freedom Residual deviance: on 972 degrees of freedom AIC: Number of Fisher Scoring iterations: 7 WHAT TO LOOK FOR: We expect the p- value for _HAT to be highly significant. Evidence of a GOOD FIT is reflected in a NON- SIGNIFICANT _HATSQ. Here the p- value for _HATSQ is.934. This suggests good model adequacy REGRESSION DIAGNOSTIC for Model 3: Hosmer Lemeshow GOF Test with 9 bins (NULL: Fit is good). hoslem.test(model3$y,fitted(model3),g=9) Hosmer and Lemeshow goodness of fit (GOF) test data: model3$y, fitted(model3) X- squared = , df = 7, p- value = R Handouts \R for Logistic Regression 2017.docx Page 12 of 14

13 WHAT TO LOOK FOR: Evidence of a OVERALL GOODNESS OF FIT is reflected in a NON- SIGNIFICANT p- value Here the Hosmer- Lemeshow test p- value is This suggests good overall fit REGRESSION DIAGNOSTIC for Model 3: Plot of ROC Curve (AUC = % Correctly Classified). roc1 <- roc(dat$case, fitted(model3)) plot.roc(roc1, print.auc=true, legacy.axes=true, identity=true, main="l ogistic Regression of Esophageal Cancer") WHAT TO LOOK FOR: Classification that is no better than a coin toss is reference in the 45 degree line Evidence of GOOD FIT is reflected in an ROC curve that lies above the 45 degree line reference Area under the ROC curve =.812 says that 81% of the observations are correctly classified. REGRESSION DIAGNOSTIC for Model 3: Plot of Cook's Distances v Observation Number. plot(cooks.distance(model3), ylab="cook Distance", xlab="observation #", main="logistic Regression of Esophageal Cancer") 0. R Handouts \R for Logistic Regression 2017.docx Page 13 of 14

14 WHAT TO LOOK FOR: Look for an even ribbon of cook distance values with no spikes. 0. R Handouts \R for Logistic Regression 2017.docx Page 14 of 14