Logistic Regressions. Stat 430

Transcription

1 Logistic Regressions Stat 430

2 Final Project Final Project is, again, team based You will decide on a project - only constraint is: you are supposed to use techniques for a solution that are related to material in the class (you can go beyond the content, but it will be part of the project to explain what you are doing) Nov 15 is the first deadline: submit an abstract of the project that you have decided on and a link to the data that you are planning on using

3 Outline Binary Responses Logistic Function Interpretation of Coefficients Comparison of models

4 Situation Response Variable is binary, e.g. Delayed Flight (yes, no) Assumption of normality of response is violated (or of errors - we only have two states for the error) Idea: model probability of response rather than yes/no states

5 Situation Still problematic: linear model does not ensure that predictions are within [0,1] We could change predictions to this range, but that will affect distribution and make inference problematic

6 Logistic Regression Binary Response Y Model should make sure that predictions are in [0,1] logistic function: exp(a+bx)/(1+exp(a+bx)) use logistic function to model probability of success

7 Logistic Regression e x /(1+e x )

8 Logistic Regression use logistic function to model probability of success: logit(πx) = a + Xb logit (πx) = log [πx/(1-πx)] transformed response makes model linear in parameters function in R: glm(y~x, family=binomial())

9 p(x) π(x) = exp(α + β x) /(1 + exp(α + β x) ) for β > 0, π(x) increases, for β < 0, π(x) decreases π(x)/(1 π(x)) = exp(α) exp(β x) for every unit in X, odds increase multiplicatively by exp(β) Since exp(βx) is approx (1 + βx + 0.5β 2 x ) we can interpret β as percentage change for every unit in X inflection point for π(x) = 0.5, for x = α/β (median effect, median lethal dose), steepest slope in inflection point: β/4 x

10 Example: Flights glm(formula = Delayed ~ Wind_SpeedMPH, family = binomial(), data = train) Deviance Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error z value Pr(> z ) (Intercept) < 2e-16 *** Wind_SpeedMPH e-07 *** --- Signif. codes: 0 *** ** 0.01 * (Dispersion parameter for binomial family taken to be 1) Null deviance: on 6389 degrees of freedom Residual deviance: on 6388 degrees of freedom (253 observations deleted due to missingness) AIC: Number of Fisher Scoring iterations: 6

11 Example: Flights as.numeric(delayed) Wind_SpeedMPH

12 Goodness of fit We do not have R 2 (we can still compute it, but it doesn t have the nice distributional properties of the linear model) Use Deviance

13 Deviance Deviance is used for model comparisons: -2 * difference in log likelihood of two models Model M and saturated model: deviance of M, lack of fit of model M Model M and null model (intercept only): null deviance, improvement of model M Deviance difference is asymptotically χ 2 distributed (df given by difference in degrees of freedom)

14 Comparing Models > anova(model.wind, model.dow, test="chisq") Analysis of Deviance Table Model 1: Delayed ~ Wind_SpeedMPH Model 2: Delayed ~ Wind_SpeedMPH + factor(dayofweek) Resid. Df Resid. Dev Df Deviance P(> Chi ) e-05 *** --- Signif. codes: 0 *** ** 0.01 * Deviance difference for models M1, M2 where M1 is a submodel of M2

15 Flights Example > xtabs(~delayed+dayofweek, data=train) DayOfWeek Delayed FALSE TRUE Probability for delay is highest on day 3 Odds for delays on day three is the highest

16 Odds of an event Comparison of probability of an event with its complement odds of an event: P(event) / (1 - P(event)) e.g. fifty-fifty (means P(event) = 0.5) probability of rain = 0.6 gives odds for rain of 3/2

17 Odds Ratio assume we have two groups, want to compare odds of survival: odds ratio phi = ad/(bc) a b c d

18 Example: Flights Compare odds of Delays on Weekends vs Weekdays: odds of delay on weekday: 227/4494 = weekend: 79/1843 = odds ratio of delays on weekends is 0.85 Week day Week end delayed not

19 Bootstrapping Next: