Age 55 (x = 1) Age < 55 (x = 0)

Transcription

1 Logistic Regression with a Single Dichotomous Predictor EXAMPLE: Consider the data in the file CHDcsv Instead of examining the relationship between the continuous variable age and the presence or absence of evidence of coronary heart disease (CHD), we could instead consider a dichotomous predictor: 0 Over55 = 1 if Age < 55 if Age 55 In R: > Over55 = asnumeric(age >=55) > table(over55,chd) CHD Over Find the odds ratio for having CHD associated with being 55 or over: Next, consider our logistic regression model: exp(η0 + η1x i ) E(y i x i ) = θ(x i ) = 1 + exp(η + η x ) 0 1 i If we let xi = our indicator variable, Over55, for each observation, we can construct the following table: Age 55 (x = 1) Age < 55 (x = 0) exp(η0 + η1 ) exp(η0 ) θ(x i = 1) = θ(x i = 0) = CHD = exp(η + η ) 1 + exp(η ) CHD = 0 1- θ(x i 1 = 1) = 1- θ(x 1 + exp(η + η ) 0 1 i 1 = 0) = 1 + exp(η 0 ) 17

2 Estimate the model parameters by hand : Verify the estimates of these model parameters using both R and SAS PROC LOGISTIC: > chdglm <- glm(chd~over55,family="binomial") > summary(chdglm) Call: glm(formula = CHD ~ Over55, family = "binomial") Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std Error z value Pr(> z ) (Intercept) *** Over e-05 *** --- Signif codes: 0 *** 0001 ** 001 * (Dispersion parameter for binomial family taken to be 1) Null deviance: on 99 degrees of freedom Residual deviance: on 98 degrees of freedom AIC: Number of Fisher Scoring iterations: 4 18

3 proc logistic data=chd descending; model CHD = Over55 / link=logit; output out=probs predicted=predicted_probabilities; run; Questions: 1 Use the model parameters to predict the probability of having CHD for a person who is 55 or over and for a person who is younger than 55 2 Given only the estimates of the model parameters, find the odds ratio for having CHD associated with being 55 or over 19

4 Verify the predicted probabilities from both the R and SAS output: > probchd <- fitted(chdglm) > cbind(age,probchd) Age probchd proc print data=probs; run; 20

5 Statistics Measuring Predictive Power Once again, consider the model using the continuous variable age to predict CHD: proc logistic descending; model CHD = age / link=logit; output out=get_values predicted=predicted_probabilities; run; Recall that the p-values shown above are used to test the usefulness of the logistic regression model We can also consider a few other statistics to investigate the model s predictive power: Generalized R 2 Likelihood Ratio Chi - Square This is calculated as follows: 1 exp = n You can also request this quantity from SAS: proc logistic descending; model CHD = age / link=logit rsq; run; Note that the upper-bound of the generalized R 2 is less than 1 Therefore, PROC LOGISTIC also reports a quantity labeled the Max-rescaled R-Square, which divides the original generalized R 2 by its upper bound Ordinal Measures of Association SAS PROC LOGISTIC also reports the following statistics: 21

6 The idea behind these statistics is as follows For the 100 observations in the data set, there exist 100 (99)/2 = 4,950 different ways to pair them up (without pairing an observation with itself) Of these pairs, 2,499 have either both 1s or both 0s for an observed response These are ignored, leaving 2,451 pairs in which one case has a 0 and the other case has a 1 For these pairs, SAS determines whether the observation with a 1 has a higher predicted value (based on the model) than does the observation with a 0 If this is the case, the pair is called concordant If not, the pair is discordant Let C = the number of concordant pairs = D = the number of discordant pairs = T = the number of ties = N = the total number of pairs (before eliminating any) = The four measures of association are given as 1 Somer s D = C D C + D + T 2 Gamma = C D C + D 3 Tau-a = C D N 4 C = 5 (1 + Somer s D) All four measures vary between 0 and 1, with large values corresponding to stronger associations between the predicted and observed values Finally, note that the measure known as C has another familiar interpretation Consider the following programming statements ods html; ods graphics on; proc logistic data=chd descending; model CHD = age / link=logit outroc=roc_data; run; ods graphics off; ods html close; proc print data=roc_data; run; 22

7 These statements request the following output The ROC curve is obtained by changing the classification rule based on the estimated probability Note that the area under the ROC curve is the same as C 23

8 Finding the ROC curve using R You must first install the Deducer package Then, you can create the ROC curve and compute the area under the curve using the following commands > library(deducer) > rocplot(chdglm) 24

9 Finding the Concordance/Discordance Model Fit Measures in R The following function can be used to find Somer s D, Gamma, Kendall s Tau, and the C- statistic in R ( ########################################################### # Function OptimisedConc : for concordance, discordance, ties # The function returns Concordance, discordance, and ties # by taking a glm binomial model result as input # Although it still uses two-for loops, it optimises the code # by creating initial zero matrices ########################################################### OptimisedConc=function(model) { Data = cbind(model$y, model$fittedvalues) ones = Data[Data[,1] == 1,] zeros = Data[Data[,1] == 0,] conc=matrix(0, dim(zeros)[1], dim(ones)[1]) disc=matrix(0, dim(zeros)[1], dim(ones)[1]) ties=matrix(0, dim(zeros)[1], dim(ones)[1]) for (j in 1:dim(zeros)[1]) { for (i in 1:dim(ones)[1]) { if (ones[i,2]>zeros[j,2]) {conc[j,i]=1} else if (ones[i,2]<zeros[j,2]) {disc[j,i]=1} else if (ones[i,2]==zeros[j,2]) {ties[j,i]=1} } } Pairs=dim(zeros)[1]*dim(ones)[1] PercentConcordance=(sum(conc)/Pairs)*100 PercentDiscordance=(sum(disc)/Pairs)*100 PercentTied=(sum(ties)/Pairs)*100 PercentConcordance=(sum(conc)/Pairs)*100 PercentDiscordance=(sum(disc)/Pairs)*100 PercentTied=(sum(ties)/Pairs)*100 N<-length(model$y) Somers_D <-(sum(conc)-sum(disc))/pairs gamma <-(sum(conc)-sum(disc))/(pairs-sum(ties)) k_tau_a <-2*(sum(conc)-sum(disc))/(N*(N-1)) C <-5*(1+Somers_D) return(list("percent Concordance"=PercentConcordance, "Percent Discordance"=PercentDiscordance, "Percent Tied"=PercentTied, "Pairs"=Pairs, "Somer's D"=Somers_D, "Gamma"=gamma, "Kendall's Tau A"=k_tau_a, "C"=C)) } 25

10 To call this function, enter the following: > OptimisedConc(chdglm) $`Percent Concordance` [1] $`Percent Discordance` [1] $`Percent Tied` [1] $Pairs [1] 2451 $`Somer's D` [1] $Gamma [1] $`Kendall's Tau A` [1] $C [1]