Lecture 10: Introduction to Logistic Regression

Transcription

1 Lecture 10: Introduction to Logistic Regression Ani Manichaikul 2 May 2007

2 Logistic Regression Regression for a response variable that follows a binomial distribution Recall the binomial model And the Binomial Distribution

3 Binomial Model n independent trials (e.g., coin tosses) p = probability of success on each trial (e.g., p = ½ = Pr of Heads) Y = number of successes out of n trials (e.g., Y= number of heads)

4 Binomial Distribution P( Y n p y y = y) = 1 ( ) n y p Example:

5 Why can t we use regular regression (SLR or MLR)?

6 Cannot use Linear Regression The response, Y, is NOT Normally Distributed The variability of Y is NOT constant since the variance, Var(Y)=pq, depends on the expected response, E(Y)=p. The predicted/fitted values must be such that the corresponding probabilities are between 0 and 1.

7 Example Consider phase I clinical trial in which 35 independent patients are given a new medication for pain relief. Of the 35 patients, 22 report significant relief one hour after medication Question: How effective is the drug?

8 Model Y = # patients who get relief n = 35 patients (trials) p = probability of relief for any patient The truth we seek in the population How effective is the drug? What is p? Get best estimate of p given data Determine margin of error -- range of plausible values for p

9 Maximum Likelihood Method The method of maximum likelihood estimation chooses values for parameter estimates which make the observed data maximally likely under the specified model

10 Maximum Likelihood For the binomial model, we have observed Y=y and P( Y n y y = y) = p 1 So for this example ( ) n y p 35 P( Y = y) = p 22 1 p 22 ( ) 13

11 Maximum Likelihood So, estimate p by choosing the value for p which makes observed data maximally likely i.e., choose p that makes the value of Pr (Y=22) maximal The ML estimate is y/n = 22/35 = 0.63 estimated proportion of patients who will experience relief

12 Maximum Likelihood 1.0e-10 Likelihood Function: Pr(22 of 35) Max Likelihood Likelihood 5.0e-11 MLE: p= p=prob(event)

13 Confidence Interval for p Variance of pˆ : Var( pˆ )= ( ) = n n p 1 p pq pˆ Standard Error of : pq n pˆ Estimate of Standard Error of : pˆ q n

14 Confidence Interval for p 95% Confidence Interval for the true proportion, p: pq ˆˆ pˆ ± 1.96 = 0.63± 1.96 n ( 0.63)( 0.37) 35 = (.082), (.082) =(0.47, 0.79)

15 Conclusion Based upon our clinical trial in which 22 of 35 patients experience relief, we estimate that 63% of persons who receive the new drug experience relief within 1 hour (95% CI: 47% to 79%)

16 Conclusion Whether 63% (47% to 79%) represents an effective drug will depend many things, especially on the science of the problem. Sore throat pain? Arthritis pain? Accidentally cut your leg off pain?

17 Aside: Probabilities and Odds The odds of an event are defined as: P(Y P(Y = 1) = 0) odds(y=1) = = P(Y = 1) 1- P(Y = 1) = p 1-p

18 Probabilities and Odds We can go back and forth between odds and probabilities: Odds = p 1-p p = odds/(odds+1)

19 Odds Ratio We saw that an odds ratio (OR) can be helpful for comparisons. Recall the Vitamin A trial: OR = odds(death Vit. A) odds(death No Vit A.)

20 Odds Ratio The OR here describes the benefits of Vitamin A therapy. We saw for this example that: OR = 0.59 An estimated 40% reduction in mortality OR is a building block for logistic regression

21 Logistic Regression Suppose we want to ask whether new drug is better than a placebo and have the following observed data: Relief? No Yes Total Drug Placebo

22 Confidence Intervals for p Placebo ( ) ( ) Drug p

23 Odds Ratio odds(relief Drug) OR = odds(relief Placebo) = Drug)] Placebo)] 0.45/(1-0.45) 0.63/(1-0.63) = = 2.26 P(Relief Drug) / [1- P(Relief P(Relief Placebo) / [1- P(Relief

24 Confidence Interval for OR CI used Woolf s method for the standard error of log( Oˆ R) : se( log( Oˆ R) = = find log( O ˆ R) ± Then (e L,e U ) 1.96se(log( Oˆ R))

25 Interpretation OR = % CI: (0.86, 5.9) The Drug is an estimated 2 ¼ times better than the placebo. But could the difference be due to chance alone?

26 Logistic Regression Can we set up a model for this similar to what we ve done in ANOVA and Regression? Idea: model the log odds of the event, (in this example, relief) as a function of predictor variables

27 Model log [ odds(relief Tx) ] = log P(no relief Tx) = β 0 + β 1 Tx P(relief Tx) where: Tx = 0 if Placebo 1 if Drug

28 Then log( odds(relief Drug) ) = β 0 + β 1 log( odds(relief Placebo) ) = β 0 log( odds(r D)) log( odds(r P)) = β 1

29 And Thus: log = β 1 odds(r D) odds(r P) And: OR = exp(β 1 ) = e β1!! So: exp(β 1 ) = odds ratio of relief for patients taking the Drug-vs-patients taking the Placebo.

30 Logistic Regression Logit estimates Number of obs = 70 LR chi2(1) = 2.83 Prob > chi2 = Log likelihood = Pseudo R2 = y Coef. Std. Err. z P> z [95% Conf. Interval] drug _cons Estimates: log( odds(relief) ) = ˆ βˆ β + 0 Drug 1 = (Drug) Therefore: OR = exp(0.814) = 2.26!

31 It s the same! So, why go to all the trouble of setting up a linear model? What if there is a biologic reason to expect that the rate of relief (and perhaps drug efficacy) is age dependent?

32 Adding other variables What if Pr(relief) = function of Drug or Placebo AND Age We could easily include age in a model such as: log( odds(relief) ) = β 0 + β 1 Drug + β 2 Age

33 Logistic Regression As in MLR, we can include many additional covariates. For a Logistic Regression model with p predictors: log ( odds(y=1)) = β 0 + β 1 X β p X p Pr( Y = 1) 1 Pr( Y = 1) where: odds(y=1) = = Pr( Y Pr( Y = = 1) 0)

34 Logistic Regression Thus: log Pr( Y Pr( Y = = 1) 0) = β 0 + β 1 X β p X p But, why use log(odds)?

35 Logistic regression Linear regression might estimate anything (-, + ), not just a proportion in the range of 0 to 1. Logistic regression is a way to estimate a proportion (between 0 and 1) as well as some related items

36 Linear models for binary outcomes We would like to use something like what we know from linear regression: Continuous outcome = X X 2 + How can we turn a proportion into a continuous outcome?

37 Transforming a proportion The odds are always positive: p odds = [0, + ) 1 p The log odds is continuous: p Log odds = ln (, + ) 1 p

38 Logit transformation Measure Min Max Name Pr(Y = 1) 0 1 probability Pr( Y = 1) 1 Pr( Y = 1) 0 odds Pr( Y = 1) log 1 Pr( Y = 1) - log-odds or logit

39 Logit Function Relates log-odds (logit) to p = Pr(Y=1) 10 logit function log-odds Probability of Success

40 Key Relationships Relating log-odds, probabilities, and parameters in logistic regression: Suppose model: logit(p) = β 0 + β 1 X p i.e. log = β 0 + β 1 X 1-p Take anti-logs 1 p -p = exp(β 0 + β 1 X)

41 Solve for p p = (1 p) exp(β 0 + β 1 X) p = exp(β 0 + β 1 X) p exp(β 0 + β 1 X) p + p exp(β 0 + β 1 X) = exp(β 0 + β 1 X) p {1+ exp(β 0 + β 1 X)} = exp(β 0 + β 1 X) p = exp(β +β X ) exp(β +β X 0 1 )

42 What s the point? We can determine the probability of success for a specific set of covariates, X, after running a logistic regression model.

43 Dependence of Blindness on Age The following data concern the Aegean island of Kalytos where inhabitants suffer from a congenital eye disease whose effects become more marked with age. Samples of 50 people were taken at five different ages and the numbers of blind people were counted.

44 Example: Data Age Number blind / 50 6 / 50 7 / / / / 50

45 Question The scientific question of interest is to determine how the probability of blindness is related to age in this population. Let p i = Pr(a person in age class i is blind)

46 Model 1 logit(p i ) = β 0 * β 0 *= log-odds of blindness for all ages exp(β 0 *) = odds of blindness for all ages No age dependence in this model

47 Model 2 logit(p i ) = β 0 + β 1 (age i 45) β 0 = log-odds of blindness among 45 year olds exp(β 0 ) = odds of blindness among 45 year olds β 1 = difference in log-odds of blindness comparing a group that is one year older than another exp(β 1 ) = odds ratio of blindness comparing a group that is one year older than another

48 Results Model 1 Iteration 0: log likelihood = Logit estimates Number of obs = 250 LR chi2(0) = 0.00 Prob > chi2 =. Log likelihood = Pseudo R2 = y Coef. Std. Err. z P> z [95% Conf. Interval] _cons exp(.08) pˆ pˆ = i i = 1+ exp(.08) logit( ) = or

49 Results Model 2 Logit estimates Number of obs = 250 LR chi2(1) = Prob > chi2 = Log likelihood = Pseudo R2 = y Coef. Std. Err. z P> z [95% Conf. Interval] age _cons logit( pˆ ) = (age i - 45) or i exp pˆ i = 1+ exp { ( age 45) } i { ( age 45) } i

50 Test of significance Is the addition of the age variable in the model important? Maximum likelihood estimates: ˆβ 1 =0.094 s.e.( ˆβ )=0.012 z-test: H 0 : β 1 = 0 z=7.855; p-val= % C.I. (0.07, 0.12) 1

51 What about the Odds Ratio? Maximum likelihood estimates: OR = exp( ˆβ )= 1.10 s.e.( ) = z-test: Ho: exp(β 1 ) = 1 1 z = 7.86 p-val = % C.I. (1.07, 1.13) *(done on log scale) ˆβ 1 It appears that blindness is age dependent Note: exp(0) = 1, where is this fact useful?

52 Model 1: Plot of observed proportion -vspredicted proportions using an intercept only model 1 Observed Prob Blindness.5 Predicted Age

53 Model 2 Plot of observed proportion -vspredicted proportions with age in the model Prob Blindness 1.5 Observed Predicted Age

54 Conclusion Model 2 clearly fits better than Model 1! Including age in our model is better than intercept alone.

55 Summary Logistic regression gives us a framework in which to model binary outcomes Uses the structure of linear models, with outcomes modelled as a function of covariates Many concepts carry over from linear regression Interactions Linear splines Tests of significance for coefficients All coefficients will have different interpretations in logistic regression