Parameter Estimation in Generalized Linear Models through

Size: px

Start display at page:

Download "Parameter Estimation in Generalized Linear Models through"

Winifred Shaw
5 years ago
Views:

1 It. Statstcal Ist.: Proc. 58th World Statstcal Cogress,, Dubl (Sesso CPS3 p.463 Parameter Estmato Geeralzed Lear Models through Modfed Mamum Lkelhood Oral, Evrm Lousaa State Uversty Health Sceces Ceter SPH, Bostatstcs Program 65 Poydras Street Sute:4 New Orleas, LA 7, USA E-mal: Itroducto For most of the geeralzed lear models (GLM, the mamum lkelhood (ML equatos volve olear fuctos of the parameters; thus, they are tractable. Solvg these equatos by teratos ca be problematc for reasos of covergece to wrog values, etremely slow covergece, or o-covergece of the teratos. To allevate these dffcultes, Tku ad Vaugha (997 ad Oral (5, 6a derved modfed mamum lkelhood (MML estmators for logt, log-lear ad proportoal odds models, respectvely. I ths study, we geeralze the estmators gve Tku ad Vaugha (997 ad Oral (5, 6a, ad provde eplct solutos that s applcable for all GLMs whch use caocal lk fuctos. By usg real lfe data sets, we show that the derved estmators are fully effcet for large sample szes ad hghly effcet for small samples. We also study the robustess propertes of these estmators va smulatos.. Modfed Mamum Lkelhood Methodology The method of MML estmato was orgated by Tku (967, 978, 98 ad has bee used etesvely lterature (Ta ad Tabataba, 988; Tku ad Suresh, 99; Oral ad Guay, 4, Oral 6b; Oral ad Kadlar,. The methodology of MML s employed stuatos where the ML estmato s tractable. There are three steps to apply the method: ( epress the lkelhood equatos terms of the order statstcs, ( replace the tractable fuctos by ther lear appromatos such that the dffereces betwee the two coverge to zero as teds to fty, ad ( solve the resultg equatos. The solutos, called MML estmators (MMLEs, have closed forms, ad are therefore easy to compute. A rgorous proof s avalable Vaugha ad Tku ( for the fact that, uder some regularty codtos, MMLEs have eactly the same asymptotc propertes as ML estmators (MLEs, ad for small values they are kow to be essetally as effcet as MLEs. To hghlght the methodology, cosder the famly of skewed dstrbutos ep[ ( µ ] b { + ep[ ( µ ] } b X ( ; µ,, < < ; (. + f where b s the shape parameter. Note that E(X µ + [ ψ(b ψ( ] ad V(X [ ψ (b + ψ ( ], where ψ ( Γ ( Γ( s the dgamma fucto ad ψ ( s ts dervatve. For b <, b ad b >, (. represets egatvely skewed, symmetrc ad postvely skewed dstrbutos, respectvely. Gve a radom sample X,X,..., X from (., we wat to estmate the parameters µ ad. The MLE of µ ad are the solutos of the lkelhood equatos [ ( µ ] [ ( µ ] l L (b + ep µ, (. + ep

2 It. Statstcal Ist.: Proc. 58th World Statstcal Cogress,, Dubl (Sesso CPS3 p.4633 l L + [( µ ] (b + [( µ ] ep[ ( µ ] + ep[ ( µ ], (.3 that clude olear fuctos of the parameters, ad eed to be solved teratvely. However, solvg these equatos by teratos ca be problematc for the reasos gve above. MML methodology proceeds as follows: Sce complete sums are varat to orderg, the lkelhood equatos (.-(.3 ca be re-wrtte terms of the ordered statstcs... ( as ( ( l L (b + g(z ( µ, (.4 l L (b + + z z g(z ( ( ( where (z ep( z ( + ep( z, ( µ g ( ( ( t ( E(z(, aroud, (.5 z ( (. Learzg the tractable fucto g (z ( gves g(z( α βz(, where α [ + ep( t + t ep( t ] [ + ep( t ], ep ( t [ + ep( t ] ( ( b ad t ( s determed from the equato t l( q / ( ( β, (. By corporatg the learzed fucto g (z( to the lkelhood equatos (.4-(.5, the modfed lkelhood equatos are obtaed as ( α β z (, q ( + * l L l L (b + ( µ µ, (.6 ( * l L l L (b + + z z ( ( ( α β z (. (.7 The solutos of (.6-(.7 are the MMLEs whch are eplct fuctos of the observatos as gve below where ˆ ad ˆ ( B + B + 4C ( ( µ K + Dˆ, K m β (, m β, D m α, (b +, ( (b + ( K, C (b + β ( B (. ( K Remark: I practce, the shape parameter b (. may ot be kow. However, oe ca easly determe the value of the shape parameter by costructg several Q Q plots wth the observed values. The Q Q plot that most closely appromates a straght le would be deemed the most approprate.

3 It. Statstcal Ist.: Proc. 58th World Statstcal Cogress,, Dubl (Sesso CPS3 p Modfed Mamum Lkelhood Estmators for Geeralzed Lear Models For a geeralzed lear model, where Y s the outcome ad X j ( j,... p are the eplaatory varables, the radom sample Y,Y,..., has a dstrbuto the epoetal famly Y {( y θ b( θ a( φ + c(y, φ } f (y, θ, φ ep,, (3. for some specfc fuctos a, b ad c. Suppose that φ s kow, so that (3. s a epoetal famly model wth a caocal parameter θ (. If we let E (Y µ µ β, ad p g( j j j the lkelhood equatos for j,... p ca be wrtte as ( y µ Var(Y j g ( µ, or, equvaletly ( y b ( θ j, for j,... p. (3. a( φb ( θ g (b ( θ Whe the caocal lk fucto s used, equatos (3. become ( y b ( θ j, for j,... p, (3.3 a( φ ad do ot have eplct solutos. To derve the MMLEs, we assume the mea of the outcome depeds o a sgle eplaatory covarate X, that s, θ +,. The lkelhood equatos for estmatg ad are wrtte terms of the ordered statstcs θ ( + [] as l L l L where (, [ ] [] ( y b ( [] θ( [( y b ( θ ] [] ( [] y par s the (, y observato (cocomtat whch correspods to ( usg the procedure descrbed Secto, we obta the MMLEs as follows: ˆ ( ˆ a, ad ( δ [] a θ,. By ˆ β (, (3.4 [] a where

4 It. Statstcal Ist.: Proc. 58th World Statstcal Cogress,, Dubl (Sesso CPS3 p.4635 δ m, δ δ δ y α, β, [] m, ( m a β, (3.5 [] α b (t ( t ( β, β b (t (, t ( E( θ(. (3.6 As a eample, for the logstc regresso model, sce θ l [ µ ( µ ], b ( θ ep( θ ( + ep( θ ad ( α ad β ( values (3.6 become b ( θ ep( θ + ep( θ, α [ ep( t ( ( + ep(t ( ] t ( β ad ( ep( t ( + ep(t (, β. (3.7 As aother eample, for the log-lear model, ( ( values (3.6 become ( t α ep( t ( ( ad β ep( t (. θ l µ, b ( θ b ( θ ep( θ ; thus, α ad β Note that, the tal values of t ( ca be take as t ( + [] where ad are the least squares estmators y ad ( (. y After the frst terato, the estmators eeded to be revsed by replacg t ( by t ( ˆ ˆ + [] from (3.4. Ths process may be repeated a few tmes to obta the fal estmates of ˆ ad ˆ. Eamples: To llustrate the methodology, we frst aalyze the coroary heart dsease data gve o page 3 of Hosmer ad Lemeshow (989. Ths data represets the values of coroary heart dsease (CHD status Y ( or, ad the correspodg values of the age X of subjects. We also cosder the data gve o page 8 of Agrest (996, whch s from a study of estg horseshoe crabs where the respose Y s the umber of satelltes that each female crab has, ad the correspodg values of the covarate X s the carapace wdth of 73 crabs. Both studes vestgate the relatoshp betwee Y ad X. For both data, we calculated the MML estmates from equatos (3.4-(3.6. The calculatos gve Table are completely cosstet wth those gve Hosmer ad Lemeshow (989 ad Agrest (996. Table Estmates for CHD ad horseshoe crabs data ML MML CHD Data Horseshoe crabs Data Coeffcet Estmate Coeffcet Estmate Coeffcet Estmate Coeffcet Estmate

5 It. Statstcal Ist.: Proc. 58th World Statstcal Cogress,, Dubl (Sesso CPS3 p Robustess Propertes of the Estmators From a practcal pot of vew, t s very mportat for a estmator to have effcecy robustess. Such a estmator s fully effcet (or early so for a assumed model ad matas hgh effcecy for plausble alteratves. I practce, specfcally outlers are a frequetly ecoutered problem GLM; thus, ths secto we search the robustess propertes of the estmators gve (3.4-(3.6 wth respect to the outlers. For llustrato, we cosder the log-lear model ad perform a Mote-Carlo study as follows: We assume that, ad for three dfferet values of (.,.5, ad., we geerate (-r of the X,X,..., X observatos from the Normal dstrbuto wth parameter, ad the remag r (we do t kow whch from the Normal dstrbuto wth parameter d (d s a postve costat. We calculate r from the formula r [. +.5]. We assume that µ, wthout loss of geeralty ad cosder the models below: a (-r come from N(, ad r come from N(, (No outlers, b (-r come from N(, ad r come from N(,.5, c (-r come from N(, ad r come from N(,, d (-r come from N(, ad r come from N(, 4. The model (a above s the model wthout outlers ad s gve for sake of comparsos. Note that for each model, after geeratg the X values, we calculated θ + ad µ ep( θ for to geerate Y values from Posso(µ. The values obtaed from [/] teratos are gve Table. As ca be see from the table, the bases the estmates are eglgble for all models. The varaces V(ˆ are almost the same for a gve for the models (a, (b, (c ad (d. We coclude that the MMLEs (3.4- (3.6 are farly robust to outlers. Table Smulato results for models (a-(d (a: No Outler (b: d.5 Bas( ˆ V( ˆ Bas( ˆ V( ˆ (c: d. (d: d4. Bas( ˆ V( ˆ Bas( ˆ V( ˆ

6 It. Statstcal Ist.: Proc. 58th World Statstcal Cogress,, Dubl (Sesso CPS3 p Cocludg Remarks ad Future Work The values of the parameters GLM are usually obtaed by ML estmato, whch requre teratve computatoal procedures. There are may teratve methods for ML estmato the geeralzed lear models, of whch the Newto-Raphso ad Fsher scorg methods are amog the most wdely used oes. Usg teratve methods, however, ca be problematc (Vaugha, ; Tku ad Vaugha, 997. I ths study, we geeralze the estmators gve Tku ad Vaugha (997 ad Oral (5, 6a, ad provde eplct solutos that s applcable for all GLMs whch use caocal lk fuctos. We also study the robustess propertes of these estmators va smulatos. We are curretly workg o geeralzg the method to multvarable stuatos ad also to the case where a( φ (3. s also a parameter. REFERENCES Agrest, A., 996, Categorcal Data Aalyss, Joh Wley ad Sos, New York, 558p. Hosmer, D. W. ad Lemeshow, S., 989, Appled Logstc Regresso, Joh Wley ad Sos, New York, 373p. Ta, W.Y., Tabataba, M.A., 988, A modfed Wsorzed regresso procedure for lear models, Joural of Statstcal Computato ad Smulato 3, Tku, M.L., 967, Estmatg the mea ad stadard devato from a cesored ormal sample, Bometrka 54, 55- Tku, M.L., 978, Lear regresso model wth cesored observatos, Commucatos Statstcs: Theory ad Methods A7, 9-3. Tku, M.L., 98, Robustess of MML estmators based o cesored samples ad robust test statstcs, Joural of Statstcal Plag ad Iferece 4, Tku, M.L., Suresh, R.P., 99, A ew method of estmato for locato ad scale parameters. Joural of Statstcal Plag ad Iferece 3, 8-9. Tku, M.L., Vaugha, D.C., 997, Logstc ad o-logstc desty fuctos bary regresso wth ostochastc covarates. Bometrcal Joural 39, Oral, E., 5, Parameter estmato posso regresso va modfed mamum lkelhood method, 5 Proceedgs of the Amerca Statstcal Assocato, Bometrcs Secto [CD-ROM], Aleadra, VA: ASA, Oral, E., 6a, Modfed mamum lkelhood estmato of the proportoal odds model, 6 Proceedgs of the Amerca Statstcal Assocato, Bometrcs Secto [CD-ROM], Aleadra, VA: ASA, Oral, E., 6b, Bary regresso wth stochastc covarates, Commucatos Statstcs: Theory ad Methods 35, Oral E. ad Guay S., 4, Stochastc Covarates Bary Regresso, Hacettepe Joural of Mathematcs ad Statstcs, 33, Oral E. ad Kadlar C.,, Robust rato-type Estmators Smple Radom Samplg, artcle press, Joural of the Korea Statstcal Socety, do:.6/j.jkss..4.. Vaugha D. C. ad Tku M. L. ( Estmato ad hypothess testg for oormal bvarate dstrbuto wth applcatos, Mathematcal ad Computer Modelg 3, Vaugha, D. C.,, The geeralzed secat hyperbolc dstrbuto ad ts propertes, Commucatos Statstcs-Theory ad Methods, 3, 9-38.

A New Family of Transformations for Lifetime Data

A New Family of Transformations for Lifetime Data Proceedgs of the World Cogress o Egeerg 4 Vol I, WCE 4, July - 4, 4, Lodo, U.K. A New Famly of Trasformatos for Lfetme Data Lakhaa Watthaacheewakul Abstract A famly of trasformatos s the oe of several