Models of Regression type: Logistic Regression Model for Binary Response Variable

Transcription

1 Models of Regression tye: Logistic Regression Model for Binary Resonse Variable Gebrenegus Ghilagaber March 7, 2008 Introduction to Logistic Regression Let Y be a binary (0, ) variable de ned as 8 < if an individual has exerienced the event of interest Y (has the characteristic of interest) : 0; otherwise Let P (Y ), be the robability that a randomly selected individual has the charcteristic of interest. Our goal is to model this robability, as a function of one or more exlanatory variables such as Education, Residence, etc. These variables are usually denoted as X, X 2,... etc. Investigators may be temted to consider using an ordinary linear regression model: 0 + X + 2 X X 3 () where 0 is the constant (intercet), is the e ect of X, 2 is the e ect of X 2, and 3 is the e ect of X 3. The arameters to be estimated ( 0 ; ; 2 and 3 ) may then be obtained by simle regression rocedures, such as Least Squares.Method. If X ; X 2 ; and X 3 are also dichotomous variables taking values 0 and (for instance for Uneducated and Educated women), then the above equation reduces to

2 (2) Any Problem??? However, the right-hand side of equations () or (2) may give largely ositive or largely negative values deending on the estimates. In other words it may be less than 0 or greater than. However, we know that our deendent variable,, is a robability and as such it cannot assume a value outside the range (0, ). To solve, the above aradox we model, instead, the logit of the robability as a function of the exlanatory variables. The logit of is de ned as log it() ln,. (3) where ln() is the natural logarithm of. The logistic regression model is then given by log it() ln If we simlify the above equation, we get 0 + X + ::: + k X k kx i X i (4) ex i X i ) ( ) ex i X i ex i X i ex i X i ) ( + ) i X i i X i 2 (5) (6) (7) (8)

3 or that k P ex i X i k P + ex i X i + ex k P i X i (9) From equation (9) we are guaranteed that lies in the interval (0, ), as it should, because the denominator is always greater than the numerator (since the exonential function cannot be negative). Looking at (4) we notehthat thei estimated coe cients (the i ) are the e ects on the log-odds ln. Accordingly, h ex( i ) are interreted as the odds of having the event of interest i.for individuals with certain characteristics indexed by (say, theheducated women) relative to the odds of having the event of interest 0 of a reference (baseline) grou of individuals (indexed i by 0, say the uneducated women): ex( i ) Re lative Odds Ratio (0) If the event of interest is rare ( is small) the relative odds ratio translates aroximately to a relative risk. For details on Odds Ratio, Relative Odds, etc see Hosmer and Lemeshow. From these coe cients, it is ossible to comute the corresonding robabilities and, hence, ratio of the robabilities corresonding to various categories. The corresonding ratios are called the Odds-Ratio and are closely related to the relative risks in hazard regression. SPSS gives the estimates ( i ); the Relative Odds Ratio ; and the 95% con dence intervals for such ratio. One can run logistic regression in SPSS as follows: 3

4 ANALYSE ) REGRESSION ) BINARY LOGISTIC Deendent Variable Y Covarites Selected Covariates Categorical: (select basline levels and make sure you change) Otions: (95% Con dence Intervals for ex() One can run logistic regression in SAS as follows: Proc logistic data dataname; model binaryvariable covariates; run;. Logistic Regression Model When the Indeendent Variable is Dichotomous Indeendent Variable x Outcome Variable y x x 0 y ex[ 0 + ] +ex[ 0 + ] ex[ 0 ] +ex[ 0 ] y 0 +ex[ 0 + ] +ex[ 0 ] Total The Odds of the outcome being resent among those individuals with x is de ned as () The Odds of the outcome being resent among those individuals with x 0 is de ned as (2) The log of the Odds is known as the logit and is de ned as 4

5 log it(0) ln (3) log it() ln (4) The Odds Ratio, denoted by, is de ned as the ratio of the odds forx to that of the odds for x 0: (5) The log of the Odds Ratio, termed as log-odds ratio or simly log-odds is 2 ln ln ln which is the logit-di erence. 0 ln log it(jx ) log it(jx 0) (6) Now using the exressions in above Table, the Odds Ratio, is 8 < h : h ex(0 + ) +ex( 0 + ) +ex( 0 + ) i9 8 < i h ; : h ex(0 ) +ex( 0 ) +ex( 0 ) i9 i ; ex [ 0 + ] ex [ 0 ] ex [ ] (7) Thus, for logistic regression with a dichotomous indeendent variable the odds ratio is ex [ ] and the logit-di erence, or log-odds is ln( ) ln fex [ ]g. This fact concerning the interretability of the coe cients is the fundamental reason why logistic regression has roven such a owerful analytic tool. 5

6 The Odds Ratio ex [ ] is a measure of the association which has found wide use, esecially in eidemiology, as it aroximates how much more likely (or unlikely) it is for the outcome to be resent among those with x than among those with x 0. For examle, if y denotes the resence (y ) or absence (y 0) of remarital concetion and x denotes whether a woman is educated (x ) or not (x 0), then an estimated ex [ ] 2 indicates that remarital concetion occurs twice as often among educated women than among uneducated women in the study oulation. Similarly, a value of ex [ ] 0.5 would mean that the event of interest (remarital concetion) occurs one half as frequent in individuals with x than among those individuals with x 0. The interretation given for the odds ratio is based on the fact that in many instances it aroximates a quantity called the relative risk (recall Cox PH model). This arameter is equal to It follows from the de nition of : RR (8) 0 that if This aroximation holds when both and are small (the event of interest is rare at both values of x). 2 Demonstration with SPSS and/or SAS 3 Polytomous Logistic Regression (outcome variable with more than 2 levels) (Allison, 995, Chater 7) 6

7 4 Further reading on Logistic Regression Hej Gebre, Inför kommande kurser i överlevnadsanalys som du ska ge kan du (om du vill) hänvisa till: Kleinbaum, Kuer, Muller, and Azhar (998), Alied Regression Analysis and Multivariable Methods (3 rd edition). Denna bok är kurslitteratur å fortsättningskursen. Vi elever brukar ju gilla att gnälla om att kurslitteraturen är så dyr och sällan återkommer i senare kurser. I Kleinbaum med era nns det att läsa: Kaitel 23 Logistisk regression och oddskvoter. Sidorna