Estimation of odds ratios in Logistic Regression models under different parameterizations and Design matrices

Size: px

Start display at page:

Download "Estimation of odds ratios in Logistic Regression models under different parameterizations and Design matrices"

Augustine Richard
6 years ago
Views:

1 Advancs in Computational Intllignc, Man-Machin Systms and Cybrntics Estimation of odds ratios in Logistic Rgrssion modls undr diffrnt paramtrizations and Dsign matrics SURENDRA PRASAD SINHA*, LUIS NAVA PUENTE** AND ELIZABETH TORRES RIVAS* *Instituto d Estadística Aplicada y Computación **Escula d Estadística Facultad d Cincias Económicas y Socials Univrsidad d Los Ands MERIDA VENEZUELA sinha32@yahoo.com, navalu@ula.v, liza@ula.v Abstract: - In this articl an altrnat mthod is proposd for th stimation of odds ratios in logistic rgrssion basd only on th prdictd logit valus without th ncssity of knowing th typ of paramtrization and th dsign matrix usd in th statistical modl. Th nw mthod which is dvlopd is illustratd with svral xampls basd on both artificial and ral data sts. Ths xampls ar also usd to show by comparison th invarianc proprty of dvianc, prdictd logit valus and odds ratios whn th paramtrizations which ar usd imply quivalnt dsign matrics. Th disadvantags of th standard mthod ithr computational or intrprtiv ar both discussd as wll as th advantags of th altrnat mthod which facilitats th comprhnsion of th rsults obtaind in logistic rgrssion analysis. Ky-Words: - Logistic Rgrssion, Equivalnt dsigns, Paramtrization, Odds ratio, Effct coding, Logit. 1 Introduction Logistic rgrssion is a vry usful and powrful tchniqu which is usd to fit modls to data whn th rspons variabl is binary or multinomial, but th modls may bcom difficult to intrprt which must tak into considration th typ of paramtrization and th dsign matrix usd. Th stimation thory for logistic rgrssion is discussd in dtails in [1,3,4]. This articl proposs an altrnat mthod to stimat th odds ratios in th logistic rgrssion analysis which unlik th standard mthod instad of calculating thm through rgrssion cofficints maks us of a slctd st of stimatd valus of X whr X is th dsign matrix and is th vctor of rgrssion cofficints. This facilitats th odds ratios calculation sinc without dpnding on whatvr paramtrization that may b usd, th stimatd valus of Xwill b invariant among quivalnt modls vn if thy hav diffrnt dsign matrics. Th mthod dvlopd in this articl will b illustratd using both artificial and ral data xampls. 2 Problm Formulation W will considr th cas whn vry sampling unit in a study contains data on a binary rspons variabl and p xplanatory variabls which may b continuous or catgorical. Th two outcom catgoris of th binary rspons variabl Y will b codd as Y = 1 which will rprsnt som charactristic of intrst (also known as succss or vnt) whras th outcom codd as Y=0 is without th charactristic of intrst (may also b calld failur or non vnt). Th random variabl will b assumd to follow a Brnoulli distribution with probabilitis (x) and 1- (x) for vnt and non vnt rspctivly whr x = (x 1, x 2,, x p ) is th column vctor of p xplanatory variabls. Th mathmatical modl of th logistic rgrssion can b dfind by th following linar form of th logit of th binomial probability: x Logit x Log x' (1) 1 x Whr is th intrcptand paramtrs to stimat. is a vctor of Th modl (1) can also b xprssd by th following two quivalnt rprsntations n trms of odds and probability of th vnt of intrst. ISBN:

2 Advancs in Computational Intllignc, Man-Machin Systms and Cybrntics x x x' (2) 1 x' x (3) x' 1 In th logistic rgrssion modl (1), whn th x i 's ar continuous variabls and th modl dos not contain any intraction trms among th x i s thn i 's can b intrprtd as an incras (or dcras) in th logodds (also calld logit) for ach unit incras in th corrsponding continuous variabl x i or quivalntly using th modl (2) as an incras (or dcras) in th i odds by amount of for ach unit incras in x. By xponntiating th stimatd modl paramtrs, w obtain th corrsponding stimatd rat of chang in th odds for unit incras in th valu of an xplanatory variabl x i. Howvr unlik as in th cas of continuous xplanatory variabls, th xponntiation of th rgrssion cofficint in th cas of catgorical variabls will not always produc th valu by which th odds ratio will chang sinc it will dpnd on th typ of paramtrization. For this rason th intrprtation in logistic rgrssion analysis may bcom difficult sinc in many situations, th pattrn of th lmnts of th dsign matrix must b xamind in ordr to obtain th odds ratios which ar rquird to intrprt th rsults. Th sam problm occurs whil using th ffct coding which is th dfault paramtrization in Proc Logistic of SAS. In gnral th paramtr stimats of th dummy variabls cannot b dirctly intrprtd without taking into considration th th pattrn of th covariabls in th X matrix. Th objctiv of this articl is to dvlop an altrnat mthod which can b usd to calculat th odds ratios dirctly from a slctd subst of th stimatd logit valus without th ncssity of knowing th typ of paramtrization or th pattrn of th dsign matrix which has bn usd in th logistic rgrssion modl. 3 Problm Solution In what follows, w will prsnt xampls with constructd artificial data to illustrat th application of th standard and th altrnat mthod of calculating odds ratios undr two typs of popular mthods of paramtrization availabl in Proc Logistic of SAS vrsion 9.2. Th ncssary documntation for using logistic procdur is availabl in [6,7]. 3.1 Exampl 1 i Tabl 1 shows th data frquncy tabl of a 2x3 factorial modl for th factors, city and group. Th sam tabl also shows a 6x4 dsign matrix with its columns labld as and which rprsnt th paramtrs of th modl with which th rspctiv columns of th dsign matrix ar associatd. Th typ of coding usd in th construction of th dsign matrix in tabl 1 is calld rfrnc coding dfind in SAS manual and is availabl in Proc logistic as an option [6]. Th last column of Tabl 1 shows th paramtric form of th logit for ach of th obsrvations which is obtaind as a linar combination of th corrsponding lmnts of th row of th dsign matrix and th associatd paramtrs labls: for intrcpt, forcity forpinion W and pinion Z. Th column labld Xb whr b dnots th stimator of shows th stimatd (also calld prdictd) logit valus for ach obsrvation. Tabl 1 Data frquncy tabl, Dsign matrix and stimatd Xb valus for rfrnc coding using city A and opinion V as rfrnc lvls. City group frq ys no Xb Logit A W A V A Z B W B V B Z Tabl 2 prsnts th stimatd valus of th rgrssion cofficints, thir standard rrors, wald s chi-squar and th associatd p-valus. Th valus givn in th last column of this tabl rprsnt th xponntiatd rgrssion cofficints and it should b compard and vrifid that thy ar qual to th odds ratios calculatd by SAS which ar givn in th tabl 2. This shows that whn SAS uss rfrnc coding thn th diffrnt odds ratios can b obtaind vry asily by simply xponntiating th valus of rgrssion cofficints. Howvr th standard mthod although not so asy in this cas but it can also b usd to calculat th odds ratios by th following procdur: Th odds ratio of a lvl h of any factor F with rspct to th rfrnc lvl k dnotd as OR(F,h,k) such that h=1, 2,, g ; hk ; is qual to ( S / T ) whr is th bas of natural logarithm, S and T ar th stimatd valus of logit slctd from Xb such that th diffrnc logit(s) - logit(t) is qual to th paramtr of intrst. For xampl, suppos w wish to calculat th odds ratio of th lvl W of factor opinion which is associatd with th paramtr. So ISBN:

3 Advancs in Computational Intllignc, Man-Machin Systms and Cybrntics in this cas w should pick up valus of S and T n column Xb as and rspctivly. Also quivalntly w can us th valus of S and T as and rspctivly. In both of ths cass w will gt th sam valu of odds ratio which is ( / ) = ( / ) = and which agrs wll with th rsult givn in tabl 2 obtaind by using proc logistic of SAS. Tabl 2 Estimation of paramtrs, Standard rror, Wald-chi squar, p-valus and xponntiatd rgrssion cofficints. Paramtr DF Estimator Standard Errror Wald`s chi-squar Intrcpt < City B 1-0, Group W Group Z Odds ratio stimats: city B vs A group W vs V group Z vs V Dvianc = with 2 DF Pr > ChiSq = Exampl 2 In this xampl w us th sam st of data as in th xampl 1. Th stimation of paramtrs also uss th sam lvls of rfrnc for both factors but this xampl uss a diffrnt typ of coding which is ffct coding. Consquntly som of th rsults ar diffrnt whil othrs ar invariant. Any chang of th rfrnc lvl or typ of coding usd will not chang th vctor Xb, th dvianc and odds ratios which will rmain invariant and hnc th sam conclusions will b obtaind by diffrnt usrs vn though thy us diffrnt typs of coding. This proprty of invarianc is a consqunc of th fact that X is an stimabl function in th logistic rgrssion modl as commntd by [2]. Also w not that although th dsign matrics in xampls 1 and 2 ar diffrnt (givn in tabls 1 and 3) but thy ar quivalnt sinc it is asy to vrify that th column spacs of both ths dsign matrics ar qual. It is a fact which is wll known that quivalnt dsign matrics must produc qual stimation vctors Xb. Howvr it can b obsrvd that th stimatd rgrssion cofficints, thir standard rrors, wald s chi-squar and p-valus ar all diffrnt in xampl 2 in comparison to xampl 1 vn though in both cass th sam st of data is usd. This fact can b vry disturbing and confusing to many usrs unlss thy mak an ffort to undrstand th corrct xplanation from a thortical point of viw basd on th concpt of quivalnt modls. Using such an approach it is possibl to dtrmin th linar rlation which xists btwn th rgrssion cofficint Pr>ChiSq Exp (b) vctors of two diffrnt paramtrization as follows: Lt X 1 and X 2 b th dsign matrics undr two diffrnt paramtrizations with stimatd rgrssion cofficint vctors qual to b 1 and b 2 rspctivly. X 1 and X 2 ar quivalnt dsign matrics C(X 1 ) = C(X 2 ) whrcdnots a column spac X 1 = X 2 A for som A (5) X 1 b 1 = X 2 A b 1 = X 2 b 2 (6) b 2 = Ab 1 (7) which stablishs th rlation btwn b 1 and b 2 through th matrix A which can b obtaind from (5) as whr th suprscript dnots a gnralizd invrs. Using this mthod it can b vrifid that whr b 1 = (b 1j ) and b 2 = (b 2j ) ; j = 1,,4 ar th rgrssion cofficint vctors in xampls 1 and 2 rspctivly. This information will hlp intrprt th rgrssion cofficints in both paramtrizations. Bsids anothr thing w may obsrv in th rsults givn in th tabl 4 of xampl 2 is that using th valus of xponntiatd rgrssion cofficints it is not possibl to obtain th valus of odds ratios as it occurrd in th xampl 1. W can calculat th odds ratios in this xampl by using th standard mthod or rly upon SAS to publish ths valus. Howvr th calculation of odds ratios by standard mthod will rquir that th dsign matrix b known to th usr which gnrally dosn t happn. W can rly upon th publishd valus of odds ratios givn by SAS but ach tim w chang th rfrnc lvl of a factor thn w will nd to run th Proc Logistic again. It may b mor convnint to us a mthod which will allow to obtain th odds ratios in logistic rgrssion in a vry simpl way by only knowing a subst of th valus of th vctor Xb vn if th typ of coding, dsignation of rfrnc lvls of diffrnt factors and th pattrn of dsign matrix usd in th data procssing is not known to th usr. W xplain blow th mthod w propos for this purpos and th computation can b don fairly simply by only using a hand calculator. 3.3 An altrnat mthod for odds ratio stimation For using this mthod w only nd to hav availabl th stimatd logit valus of a subst of all possibl ISBN:

4 Advancs in Computational Intllignc, Man-Machin Systms and Cybrntics combinations of factors usd in th study. In a gnral situation suppos that thr ar k catgorical factors A, B,, Z with numbr of lvls qual to a, b,, z and with a*, b*,, z* as rfrnc lvls rspctivly. Suppos w want to stimat odds ratio of m-th lvl of th factor C (1 m c ; m c*) with rspct to th rfrnc lvl c*, which can b dnotd as OR(C, m, c*) and no information is availabl about th dsign matrix and th typ of paramtrization usd. Lt L(g) = th stimatd logit valu of factorial combination g. Lt L(A p B q C m Z t ) and L(A p B q C c* Z t ) b two stimatd logit valus which diffr from ach othr only in th indics of th factor C such that on of thm has indx of th lvl whos odds ratio will b calculatd and th othr has th indx of th rfrnc lvl of th sam factor. Thr may xist svral such possibl pairs of stimatd logit valus with such a proprty but it can b vrifid that in all ths cass th sam valu of odds ratio will b obtaind which will b qual to LAp BqCm... Zt LAp BqCc... Zt (8) * As an xampl, lt C and G dnot th factors, city and opinion group with lvls A, B and W, V, Z rspctivly. Using th formula (8), th quotint rquird to calculat th odds ratio OR(G, W, V) is = = which agrs wll with th valu calculatd by SAS and prsntd in th tabl 4. W not that th sam valu of odds ratio is obtaind by anothr pair of logit valus by using LCBGW th quotint B V. L C G Tabl 3 Data frquncy tabl, Dsign matrix and stimatd Xb valus for ffct coding using City A and opinion V as rfrnc lvls. City group frq ys no Xb Logit A W A V A Z B W B V B Z It may b obsrvd in th tabl 4 that th xponntiatd valus of th rgrssion cofficints in th last column (with th xcption of th first lmnt which is th intrcpt), ar not qual to th odds ratio valus as it occurrd in th cas of rfrnc paramtrization which was usd in th tabls 1 and 2. Tabl 4 Estimation of paramtrs, Standard rror, Wald-chi squar, p-valus and xponntiatd rgrssion cofficints. Paramtr DF Estimator Standard Errror Wald`s chi-squar Intrcpt < City B Group X Group Z Odds ratio stimats: city B vs A group W vs V group Z vs V Dvianc = with 2 DF Pr > ChiSq = Continuous xplanatory variabls Whn som of th xplanatory variabls ar continuous and th numbr of thir distinct valus is larg thn thr will b only on obsrvation in most of th clls gnratd by th cross-classification of th xplanatory variabl valus. In such a situation th sampl siz rquirmnts for using th goodnss of fit tsts of Parson s chi-squar and liklihood ratio ar not satisfid. In such cass w rcommnd th catgorization of th continuous xplanatory variabls and th odds ratios can b obtaind using th altrnat mthod dscribd in th scción A ral data xampl This xampl is basd on th data st usd in th study publishd in [5] who studid th influnc of 5 catgorical variabls on th prvalnc of mycocardiopathy condition using th information obtaind from a group of 2336 prsons from th stat of Trujillo, Vnzula. A factorial modl 4 2 x2 3 was usd with 5 factors which wr th following: Ag with lvls 0, 1, 2, 3 which rfr to diffrnt groups of ag in ascnding ordr. Wight with lvls 0, 1, 2, 3 which rprsnt diffrnts groups in ascnding ordr. Chimó consumption with lvls 1, 2. Whr 1 indicats chimó consumption and 2 rprsnts no consumption. Smoking with lvls 1, 2. Whr 1 rprsnts smoking and 2 indicats no smoking. Chagas dsas with lvls 1, 2, whr 1 rprsnts th prsnc of th disas and 2 indicats its absnc. Th chagas dsas is causd by Trypanosoma cruzi. Th rfrnc lvls of 0, 0, 2, 1, 1 rspctivly wr usd for ths 5 factors. Th valus of th xponntiatd rgrssion cofficints will not b qual to th odds ratios in this study sinc th ffct coding was usd in this cas by running th proc Pr>ChiSq Exp (b) ISBN:

5 Advancs in Computational Intllignc, Man-Machin Systms and Cybrntics logistic of SAS. Consquntly it bcoms ncssary to xamin th pattrn of th lmnts of th dsign matrix which may not b availabl to th usr. Howvr as w will s blow it is possibl to calculat th stimatd odds ratios by using th altrnat mthod for odds ratio stimation givn in th sction 3.3 which rquirs only a subst of prdictd logit valus givn by SAS. For xampl in ordr to calculat th odds ratios of th diffrnt lvls of th factors: Ag, chimo consuming and smoking with rspct to thir rfrnc lvls, w will nd a subst of th st of stimatd logit valus calculatd by SAS which is givn blow. Tabl 5 A subst of prdictd logit valus calculatd by SAS. Edad Pso chimó Fumar Rsult Estimatd Logit Using th appropriat stimatd logit valus w obtain OR Ag, 1, ; OR Ag, 2, ; OR Ag, 3, ; OR chimó, 2, ; OR fumar, 2, ; Th rmaining odds ratios can b calculatd in a similar way. Th altrnat mthod has th advantag that if w dcid to chang th rfrnc lvl of any factor for xampl in th cas of th factor Ag in this xampl thn w can calculat th nw valus of th odds ratios using th sam subst of stimatd logit valus in th tabl 5 and no additional information is rquird using proc logistic of SAS again. For xampl suppos that th rfrnc lvl of ag has changd from 0 to 3 and w can calculat th following odds ratios using th sam subst of prdictd logit valus givn in th tabl 5: OR OR Ag 1,3 OR Ag 2, , Ag 0, , , Conclusion This articl proposs an altrnat mthod of calculation of odds ratios in logistic rgrssion analysis which is basd only on a subst of th stimatd logit valus and no information about th dsign matrix and th typ of paramtrization usd is rquird. Th illustration of th altrnat mthod using svral xampls in this articl shows its advantags ovr th standard mthod. Th us of th concpt of quivalnc of dsign matrics was found to b a vry usful aid in undrstanding and intrprting th invarianc and th diffrncs among th rsults obtaind whn th sam st of data is usd to calculat stimats in logistic rgrssion analysis undr diffrnt paramtrizations and hnc this tchniqu should b usd for th intrprtation of th rsults obtaind. Rfrncs: [1] Agrsti Alan (2007) An Introduction to Catgorical Data Analysis, 2nd. dition. John Wily &sons Inc. Nw York. [2] Christnsn, Ronald (2002): Plan Answrs to Complx Qustions: Th Thory of Linar Modls. 3rd. dition. Springr-Vrlag Nw York Inc. [3] Hosmr D. W., Lmshow S. Applid logistic rgrssion. Scond dition. Nw York: John Wily & Sons, [4] McCullagh, P. and J.A. Nldr. Gnralizd Linar Modls. Chapman and Hall, London [5] Nava Punt, Luis and Surndra Prasad Sinha (2008): Ajust intrprtación d modlos d rgrsión logística con variabls catgóricas y continuas. Univ. Md. Bogotá (Colombia), 49, [6] SAS/STAT 9.2 Usr s Guid. SAS Institut Inc. Cary, NC, USA, [7] Stoks M.E., Davis C.S., Koch G.G. Catgorical data analysis using th SAS systm. Cary, N.C.: SAS Institut, U.S.A., ISBN:

What are those βs anyway? Understanding Design Matrix & Odds ratios

What are those βs anyway? Understanding Design Matrix & Odds ratios Ral paramtr stimat WILD 750 - Wildlif Population Analysis of 6 What ar thos βs anyway? Undrsting Dsign Matrix & Odds ratios Rfrncs Hosmr D.W.. Lmshow. 000. Applid logistic rgrssion. John Wily & ons Inc.