Consistency of the Maximum Likelihood Estimator in Logistic Regression Model: A Different Approach

Transcription

1 ISSN Joural of Statstcs Volum 16, 9, Cosstcy of th Mamum Lklhood Estmator Logstc Rgrsso Modl: A Dffrt Aroach Abstract Mamuur Rashd 1 ad Nama Shfa hs artcl vstgats th cosstcy of mamum lklhood stmators for th logstc rgrsso modl usg a dffrt aroach. I ths aroach, w vrfy all th rgularty codtos for th logstc rgrsso modl dd for cosstcy of th mamum lklhood stmators. h rformac of th mamum lklhood stmators for cosstcy th logstc rgrsso modl s also amd va stadard Mot Carlo smulato study. Kywords Logstc rgrsso, Cosstcy, Mamum lklhood stmator, Mot Carlo smulato 1. Itroducto A commo am for a rgrsso modl for bary rsos varabls s th logstc rgrsso modl, whch has b wdly usd th hyscal, bomdcal, ad bhavoral sccs (Mhta t al., ). Lt Y b a bary varabl ad lt b th assocatd 1 vctor of laatory varabls. h th stadard logstc rgrsso modl assums th followg modl: P ( Y 1 ) 1 1 Dartmt of Mathmatcs ad Statstcs, Oho Northr Uvrsty, Ada, OH 581, USA E-mal: m-rashd@ou.du Dartmt of Mathmatcs, DPauw Uvrsty, Grcastl, IN 615, USA E-mal: amashfa@dauw.du

2 Rashd ad Shfa whr, s a scal aramtr ad s a 1 vctor of aramtrs. h mamum lklhood stmato rocdur s usd to stmat th ukow aramtrs for th modl (Hosmr ad Lamshow, ). Sc th logstc modl s olar aramtrs, a tratv rocdur such as Nwto-Rahso mthod s ald (McCullagh ad Nldr, 1989). Gvs ad Hotg (5) showd that f l( ) s cotuous ad s a sml root of l ( ), th thr sts a ghborhood of for whch Nwto-Rahso mthod covrgs to wh startd from ay () t, t 1,,... that ghborhood, whr l() s th loglklhood of th fucto. Cosstcy of th mamum lklhood stmators for logstc rgrsso modl was rvously studd by dffrt authors, for aml, Gourrou ad Mofort (1981), Ammya (1985). All thr work was basd uo th fact that th robablty of th stc of th stmator ˆ aroachs 1 as tds to fty ad also assumd that th umbr of laatory varabl s comlld to rma costat whl saml sz crass. Br (1) showd that s a varabl but ddt o ad amd what rlatosh btw ad s cssary ordr ot to dstroy th cosstcy of th stmator ˆ. Howvr, ths artcl focuss o a dffrt aroach to vstgat th cosstcy of th mamum lklhood stmator ˆ for th logstc rgrsso modl. Mor rcsly, w ar gog to show that ˆ covrgs udr crta hyothss to th ral valu f th umbr of obsrvatos ( y, ), whr ( 1,, ), 1,,, td to fty. o show ths, w follow th rocdur dscrbd by Lhma ad Caslla (1998) whch cosstcy of th mamum lklhood stmators hold f crta rgularty codtos ar satsfd. It ds to b otd out that o of th authors (Gourrou ad Mofort, 1981; Ammya, 1985 ad Br, 1) vrfd thr work va th Mot Carlo smulato study. Gourrou ad Mofort (1981) otd, t should b strssd that all ths asymtotc rsults gv lttl dcato o th rorts of th stmators ft saml, ad t would b trstg to clarfy ths ot by mas of Mot Carlo studs. I ths artcl, w rovd a tsv stadard Mot Carlo smulato study showg th cosstcy of th mamum lklhood stmators for th logstc rgrsso modl. hs ar s structurd as follows. I Scto, w rovd cosstcy of th mamum lklhood stmators as dscrbd by Lhma ad Caslla (1998). I scto, w vrfd all th codtos dd for cosstcy otcd Scto

3 Cosstcy of h Mamum Lklhood Estmator Logstc Rgrsso Modl: A Dffrt Aroach for th logstc rgrsso modl. A smulato study s rstd Scto to dmostrat th rformac of cosstcy for th mamum lklhood stmators. Fally, a cocludg rmark s gv Scto 5.. Cosstcy of th Mamum Lklhood Estmator (MLE) Lhma ad Caslla (1998) rovdd th followg rsults of th cosstcy of MLE udr som rgularty codtos. hs ar: (A) h dstrbutos P of th obsrvatos ar dstct (othrws, caot b stmatd cossttly). (A1) h dstrbutos P hav commo suort. (A) h radom varabls ar ( 1,, ), =1,, whr th s ar ddt ad dtcally dstrbutd (d) wth robablty dsty f ( ) wth rsct to robablty masur. (A) hr sts a o subst of cotag th tru aramtr ot such that for almost all, th dsty f ( ) admts all thrd drvatvs f( ) k l for all. (A) h frst ad scod drvatvs of log f satsfy th quatos E log f ( ) for 1,,, ad I k E log f ( ) log f ( ) E k log f ( ) k (A5) Sc th matr I( ) s a covarac matr, t s ostv smdft. W wll assum that I ( ) ar ft ad that th matr k I( ) (( I )),, k 1,,..., s ostv dft for all, ad th Statstcs k

4 Rashd ad Shfa log f ( ),, log f ( ) 1 ar affly ddt wth robablty 1. (A6) Fally, w wll assum that thr sts fucto log f ( ) M kl ( ) for all k l whr m E [ M ( )] for all kl,,. kl kl M kl such that horm 1: Lt,, 1 b d ach wth a dsty f( ) (wth rsct to ) whch satsfs (A)-(A6) abov. h, wth robablty tdg to 1 as, thr st solutos ˆ ˆ (,, ) 1 of th lklhood quatos f ( 1 ) f ( ), 1,,, or, quvaltly, log L( ), 1,,, such that (a) ˆ s cosstt for stmatg. (b) s asymtotcally ormal wth ma (vctor) zro ad 1 covarac matr [ I( ) ], ad (c) ˆ s asymtotcally ffct th ss that 1 ( ˆ L ) N, I( ).. Cosstcy of th MLE Logstc Modl W vrfy all th rgularty codtos udr th logstc rgrsso modl dscussd scto ad th w aly horm 1 to show th cosstcy of MLE for th logstc rgrsso modl.

5 Cosstcy of h Mamum Lklhood Estmator Logstc Rgrsso Modl: A Dffrt Aroach Assumto (A): Lt df th modls as (1) (1) (1) () () () 1 (, 1,..., ) ad (, 1,..., ). W (1) (1) (1) (1) 1 1 P ( Y 1 ) (1) (1) (1) (1) () () () () 1 1 P ( Y 1 ) 1 1 () () () () 1 1 (1) (1) (1) (1) () () () () whr (, 1,..., ), (, 1,..., ) ad (, 1,..., ) If (1) (), th th abov two quatos ar th sam. O th cotrary, w ar gog to show that f th quatos ar qual, th (1) (). (1) () W hav, (1) () 1 1 (1) () (1) () (1) () hs mls, ( ) ( ),..., ( ) hat s, a a1 1,..., a (1), whr (),,1,..., a Sc s ar ddt, so a a 1... a, ths mls that, hs dcats that th dstrbutos ar uqu, thrfor, f 1 dstrbutos P of th obsrvatos ar dstct.. (1) () 5, th th Assumto (A1): h varabls th modl ar 1,,...,, lt 1 (,,..., ) whr ad th aramtr taks valus, =1,,,. hs s tru for ach modl statd th assumto (A). hrfor, th dstrbutos P hav commo suort. Assumto (A): I th logstc modl, w cosdr th obsrvatos of th form (,, ), =1,, whr th ar d wth robablty dsty 1 P ( Y 1 ). Assumto (A): Wh Y=1, w df for th logstc modl s roortoal to f( ) hav th lklhood 1

6 6 Rashd ad Shfa L 1 1 akg log o both sds ad w gt log L log(1 ) 1 Now, takg drvatv wth rsct to, w hav 1 1 log L 1 1 Now th drvatv coms to th form. If w tak th drvatv of kth 1 ordr, th drvatv coms to th form (1 k ), whch ca b rovd by th mathmatcal ducto. hrfor, ot oly dos th drvatv of f( ) thrd ordr st, but th drvatvs of all ordrs st. Assumto (A): h codto (A) s rovd, gral, for th dsty f ( ) udr th codto that th dffrtato udr th tgral sg s allowd. h oly thg w d to chck for th logstc modl s that whthr t rmts th dffrtato udr th tgral sg. o show that art w cosdr th followg thorm, whch s avalabl stadard ral aalyss or robablty books (s, Durrtt, 5). hs thorm allows us to rform th dffrtato udr th tgral sg. horm. Suos w ar gv th followg: A o trval I. A masurabl subst. A fucto H : I A fucto g: [, ] Assum th followg: H ( t, ) g( ) for vry t I ad t g s tgrabl..

7 Cosstcy of h Mamum Lklhood Estmator Logstc Rgrsso Modl: A Dffrt Aroach t H( t, ) s a dffrtabl fucto of t I for vry. H( t, ) s a tgrabl fucto of for vry t I. h th followg hold: H ( t, ) s a tgrabl fucto of for vry t I. t t H ( t, ) d s a dffrtabl fucto of I d H( t, ) d H( t, ) d dt t for vry t I. t. o vrfy th abov assumtos of horm for logstc rgrsso modl, w cosdr th followg fucto wh y =1. H(, ) 1 H(, ) (1 ) 7 H(, ) g( ) (1 ) (1 ) as 1 (1 ) Smlarly, ths ca b show for y =. Sc H(, ) s a dffrtabl fucto of for vry ad tgrabl for for vry. hus th rsults of th horm hold. Assumto (A5): W tak th drvatv of log f( ) wth rsct to 1,,,, w hav log f, =1,,, 1 Now w wrt th vctors th form so that thy ar larly ddt th followg way,

8 8 Rashd ad Shfa hat s, hat s, Sc 1, So, 1 = 1 hus, W hav, P, bcaus th ot dstrbuto of 1 1,,..., s cotuous o. 1 hus, P 1, 1 hs mls that th statstcs ar affly ddt. log L Assumto (A6): W hav, log L ad k 1 k 1 log L (1 ) So, k l k l 1 (1 ) log L (1 ) k l k l 1 (1 )

9 Cosstcy of h Mamum Lklhood Estmator Logstc Rgrsso Modl: 9 A Dffrt Aroach 1 1 (1 ) k l sc 1 M kl k l ( ) k l (1 ) (1 ) (1 ) whr mkl E [ M kl ( )] E k l E k l, whch s 1 1 ft. Sc th logstc modl satsfs all th rgularty codtos (A)-(A6), thrfor, ˆ covrgs to th ral valu by horm 1.. A Smulato Study W ow assss, va stadard Mot Carlo smulato, th ft saml rformac of cosstcy of th mamum lklhood stmators. I th smulato study, w cosdr four laatory varabls 1,,, ad whch ar fd ad th bary rsos varabl y, whch s tratd as a radom varabl th logstc rgrsso modl. For th fd valus of th trct aramtr ad four othr aramtrs 1,,, ad, our am s to comar th rformac of th valus of aramtrs ad thr stadard rrors wh saml sz crass. For fd valus of =.7, 1=1., =1., =.5, ad =.5, th logstc rgrsso modl bcoms: 1 ( ) I th smulato, w cosdr saml szs of 5, 1, 15, ad ad grat 1, ddt sts of radom samls for ach dffrt saml sz. For ach st of radom saml wth artcular saml sz, w stmat, 1,, ad ad thr stadard rrors basd o th logstc rgrsso modl. h fal stmats ad stadard rrors of, 1,,, ad ar th avrag

10 1 Rashd ad Shfa of 1, stmats of, 1,,, ad for that artcular saml sz. h followg abl gvs th rsults of smulato study for dffrt saml szs. abl 1: Estmatd aramtr valus ad thr stadard rrors usg th logstc rgrsso modl for dffrt saml szs of 5, 1, 15, ad Paramtrs Estmat SE Estmat SE Estmat SE Estmat SE SE=Stadard Error As s th abov abl, for saml sz 5, th stmatd valus of aramtrs ar dffrt from th tru valus ( =.7, 1=1., =1., =.5, ad =.5), ad also th stadard rrors bcom larg. Howvr, wh th saml sz crass from 5 to, th stmatd valus of th aramtrs, 1,,, ad ar clos to th tru valus, ad stadard rrors of th stmats ar otcably smallr. hs dcats that smulato study rforms wll showg th cosstcy of mamum lklhood stmators for aramtrs of th logstc rgrsso modl. 5. Cocluso hs ar vstgats a dffrt aroach to show th cosstcy of mamum lklhood stmators th logstc rgrsso modl. I that aroach, w vrfy all th rgularty codtos for cosstcy mtod scto for th logstc rgrsso modl ad coclud that th aramtrs of th logstc rgrsso modl covrg to ts tru valus wh saml sz crass. hs ar also coctrats o Mot Carlo smulato study for showg th cosstcy of th mamum lklhood stmators for th logstc rgrsso modl. Rsults dcat that th smulato study rforms vry wll ad th smulato stadard rrors of th aramtrs gt smallr as th saml sz crass.

11 Cosstcy of h Mamum Lklhood Estmator Logstc Rgrsso Modl: A Dffrt Aroach Rfrcs 1. Ammya,. (1985). Advacd Ecoomtrcs. Harvard Uvrsty Prss, Cambrdg.. Br, M. (1). Asymtotc rorts of th mamum lklhood stmator dchotomous logstc rgrsso modls. Dloma hss Mathmatcs. Uvrsty of Frbourg Swtzrlad.. Durrtt, R. (5). Probablty: hory ad Eamls. hrd Ed., homso, Brooks/Col.. Gvs, G. H. ad Hotg, A. J. (5). Comutatoal Statstcs. Wly Itrscc. 5. Gourrou, C. ad Mofort, A. (1981). Asymtotc rorts of th mamum lklhood stmator dchotomous logt modls. Joural of Ecoomtrcs, 17(1), Hosmr, D. W. ad Lamshow, S. (). Ald Logstc Rgrsso. Joh Wly ad Sos, Nw York. 7. Lhma, E. L. ad Caslla,G. (1998). hory of Pot Estmato. Scod, Ed. Srgr, Nw York. 8. McCullagh, P. ad Nldr, J. A. (1989). Gralzd Lar Modls. Lodo Chama ad Hall. 9. Mhta, C. R., Patl, N. R. ad Schaudhur, P. (). Effct Mot Carlo mthods for codtoal logstc rgrsso. Joural of th Amrca Statstcal Assocato, 95,

12 1 Rashd ad Shfa