A ROBUST ADJUSTMENT OF THE PROFILE LIKELIHOOD 1. BY JAMES E. STAFFORD University of Western Ontario

Transcription

1 The Aals of Statistics 1996, Vol., No. 1, A ROBUST ADJUSTMENT OF THE PROFILE LIKELIHOOD 1 BY JAMES E. STAFFORD Uivesity of Weste Otaio Ude mild misspecificatios of model assumptios, maximum likelihood estimates ofte emai cosistet ad asymptotically omal. Asymptotic omality will ofte hold fo the siged oot of the likelihood atio statistic ad the scoe statistic as well. Howeve, stadad estimates of asymptotic vaiace ae usually icosistet. This occus whe Batlett s secod idetity fails. I the mae of McCullagh ad Tibshiai, a vaiace coectio may be used to adjust the pofile likelihood so this idetity obtais. The esultig likelihood yields the obust vesios of the siged oot, Wald ad scoe statistic suggested by Ket ad Royall. Assumig model coectess, asymptotic expasios fo the fist thee cumulats of each obust statistic ae deived. It is see that bias ad skewess ae ot seveely affected by usig a obust statistic. A ivaiat expessio deived fo the asymptotic elative efficiecy of a obust method allows assessmet i umeous examples cosideed. Eve fo modeately lage sample sizes, losses i efficiecy ae sigificat, makig the misuse of a obust vaiace estimate potetially costly. Compute algeba is used i may of the calculatios epoted i this pape. 1. Itoductio. Coside the situatio whee data x 1,..., x ae assumed to be idepedet ealizatios of a distibutio that belogs to the paametic family f Ž X.,, whee f Ž X. is the joit desity fo the data ad may be patitioed ito a scala paamete of iteest ad a uisace paamete. Ifeece fo 0, the tue value of, may be based o the siged squae oot of the likelihood atio statistic Ž., the scoe fuctio už. o the maximum likelihood estimate. ˆ Stadad asymptotic esults show Ž 1. Ž 0. NŽ 0, v., už ' 0. NŽ 0, v u., ' ˆ N 0, v. Ž 0. Ž ˆ. Typically, v 1 ad expessios fo v u, vˆ simplify so they may be cosistetly estimated by quatities that deped solely o the obseved ifomatio. Howeve, ude a mild model misspecificatio, whee v, v ad v do u ˆ Received Novembe 199; evised August Reseach suppoted by the Natual Scieces ad Egieeig Reseach Coucil of Caada. AMS 1991 subject classificatios. 6F35, 6F05 Key wods ad phases. Asymptotic elative efficiecy, Batlett idetities, compute algeba, cumulats, pofile likelihood, obustess. 336

2 ROBUST PROFILE LIKELIHOOD 337 ot simplify, estimates of vaiace based o the obseved ifomatio will be icosistet. Ude model misspecificatio, it is the failue of Batlett s secod idetity, which equates the vaiace of the scoe ad the expected Fishe ifomatio, that does ot allow simplificatio i expessios fo asymptotic vaiace. A obust adjustmet of the pofile log-likelihood to coect this idetity is motivated i the mae of McCullagh ad Tibshiai Ž ad leads to a simple adjustmet ivolvig ˆ v, a sample vesio of v. The esultig obust pofile likelihood is ivaiat ude a boad class of epaameteizatios ad has obseved ifomatio that povides the usual obust estimates of vaiace give i Ket Ž This is discussed i detail i Sectio. The mai esults of this pape may be foud i Sectio 3. Hee, we compae asymptotic expasios of the fist thee cumulats of the siged oot, scoe ad maximum likelihood estimate, stadadized by model-based ad model-obust vaiace estimates. A implicit assumptio of the pape is the pefeece of obust methods whe a model misspecificatio occus. This is sesible give that obust methods emai asymptotically coect while model-based methods typically do ot. Theefoe cumulat compaisos ae made assumig model coectess. This allows us to assess the effect of usig a obust vaiace estimate whe it is uecessay ad povides isight ito the impotace of model assumptios to impove the accuacy of ifeece. If compaisos wee favouable, the obust vaiace estimates could be used at all times. Expessios fo bias ad skewess ae foud to agee to ode 3, ad hece, obust vaiace estimate has a limited effect o these cumulats. Model-obust estimates of vaiace ted to be moe vaiable tha thei model-based coutepats. Fo example, the model-based estimate of vaiace fo the siged oot is simply 1. Oe would the expect this to esult i a moe vaiable test statistic that may fall ito a ejectio egio moe ofte. A measue of asymptotic elative efficiecy, that may be used to make vaiace compaisos, is deived. Numeous examples ae cosideed i Sectio, whee compute algeba methods ae used to evaluate. Fo these examples, obust methods appea to be much less efficiet tha thei modelbased competitos, as oe might suspect. Thoughout this pape, we assume boad coditios like those foud i Hube Ž o White Ž 198. that esue the cosistecy of the maximum likelihood estimate ad the asymptotic omality of Ž., už. ad. ˆ Ude ˆ a model misspecificatio, we deote the limit of by with compoets ad, ad ude model coectess, by 0 with compoets 0 ad 0. I some special cases, ad will coicide Gould ad Lawless Ž ; Li ad Wei Ž Howeve, i geeal this will ot be the case, ad oe must assume that ude a model misspecificatio still has a meaigful scietific itepetatio sice this is the oly quatity fo which we may coduct ifeece. The emaide of this itoductio is devoted to the developmet of otatio. Let lž. deote the usual log-likelihood fuctio with compoets

3 338 J. E. STAFFORD l Ž. i, i 1,...,. We will occasioally use the alteative epesetatio lž,. fo the log-likelihood ad, similaly, l Ž,. fo a compoet. Let ˆ i be the costaied maximum likelihood estimate fo holdig fixed. Whe is eplaced by ˆ i the log-likelihood, the esult is called the pofile log-likelihood l Ž.. Lettig Ž. p, the siged squae oot of the likeli- hood atio statistic ad the scoe fuctio ae defied as Ž. pž. ½ p 5 1 Ž. sg ˆ l ˆ l Ž., už. l pž. l pž.. Let ˆ ad ˆ be the ad compoets of the global maximum likelihood estimate. ˆ The paamete has legth p 1; a compoet of is deoted with the use of a subscipt. Deivatives of lž. o l Ž. i ae also idicated by the use of subscipts. Let Ž., l lž., l l Ž.. st s t i; s s i Expected values of sums of poducts of deivatives of the compoets of the log-likelihood fuctio ae called expected ifomatio quatities, 1 1 I st E l st, I, st E li; l i; st. i Sums that ae ceteed ad scaled so they ae bouded i pobability ae deoted as st st st, s Ý i; i; s, s i1 z l I, z l l I. These ae typically efeed to as O Ž 1. p adom vaiables. Occasioally, i ou otatio we equie explicit depedece of expessios o ad ad, hece, we subscipt tems by these paametes. The uisace paamete is assumed to be a vecto ad hece it is subscipted as well to deote a compoet of that vecto: l l Ž., i; s s i Ý I, E li; l i;, s s 1 1 z l l I. Ý, i; i;, i1 The otatio I s ad I s is used to deote the matix iveses I 1 s ad 1 s I, espectively. The, ad, compoets of I ae deoted I s ad I. O occasio, we will estimate a expected ifomatio quatity by a obseved quatity. Fo this we use the otatio Ý 1 ˆ 1 ˆ ˆ s 1 J l Ž., J l Ž. l Ž., J J., s i; i; s s i Fially, sice we ae i a multipaamete settig we use the summatio covetio to epeset ie poducts, whee a subscipted idex epeated

4 ROBUST PROFILE LIKELIHOOD 339 as a supescipt epesets a implicit sum ove that idex. Fo example, p s s I zs Ý I z s. s1. Batlett s secod idetity ad estimates of vaiace. The Batlett idetities esult fom the desity beig omed, Hf Ž x. dx 1. Successive diffeetiatio of this itegal with espect to the compoets of yields idetities. Let f x f x ad f s Ž x. f Ž x.. The s Ž. I l f Ž x. dx f Ž x. dx f Ž x. dx f Ž x. dx 0 f x H H H H f Ž x. ad similaly, ½ 5 f Ž x. I l f Ž x. dx f Ž x. dx s H s H s f Ž x. f Ž x. f s Ž x. f s Ž x. H f Ž x. dx H f Ž x. dx f Ž x. f Ž x. f Ž x. H l l f Ž x. dx f Ž x. dx I. H s s, s Now coside a model misspecificatio whee the tue desity fo the data gž x. does ot belog to the family f Ž X.,. The cacellatios ecessay to deive the above idetities will ot occu sice f Ž x. f s Ž x. s g Ž x. f Ž x., g Ž x. f Ž x., f Ž x. f Ž x. ad hece the idetities caot be obtaied. ˆ The maximum likelihood estimate will covege to, the value of the paamete that miimizes the KullbackLeible distace betwee f Ž x. ad gž x.: g Ž x. dž f, g. Hlog½ g Ž x. dx f Ž x. 5 H H log g Ž x. g Ž x. dx log f Ž x. g Ž x. dx. Ž. Sice d f, g is miimized at it is the case that H d f, g l g x dx 0 ad so the expected value of the scoe is zeo at, eve if the model is misspecified. Howeve, oly i vey special cases will the secod idetity hold.

5 30 J. E. STAFFORD That is, we usually have H H l g Ž x. dx l l g Ž x. dx,. s s The failue of Batlett s secod idetity has implicatios fo estimates of asymptotic vaiace. Stadad asymptotic calculatios like those give i Ket Ž 198. show Ž. ˆ s s s, s u, s Ž., s v I I I I, v I I I I, v I I I, which simplify to v 1, vu 1I ad vˆ I whe I, s I s. This allows vaiace estimates to be based o the obseved ifomatio aloe. Howeve, whe a model is misspecified ad such simplificatios caot occu, the such estimates will be icosistet. ŽNote that ude model coectess, asymptotic vaiace may also be cosistetly estimated by the expected Fishe ifomatio evaluated at. ˆ Efo ad Hikley Ž show, howeve, that the obseved ifomatio is pefeable.. I developig obust methods that addess model misspecificatios whee the secod Batlett idetity fails, it is atual to coside coectig the idetity. McCullagh ad Tibshiai Ž adjusted the pofile likelihood so the fist two Batlett idetities obtai to highe ode. Thei adjustmet was motivated by a effot to educe the effect of uisace paamete estimatio whe the model is coectly specified ad belogs to a class of adjustmets fo this pupose Badof-Neilse Ž 1983.; Cox ad Reid Ž 1987, which have bee extesively studied by DiCiccio ad Ste Ž 199a, b.. They begi by showig that ude model coectess, E už. OŽ., E už. E u Ž. OŽ.. That is, these idetities hold asymptotically, but ae oly appoximate fo a fiite sample size. To impove these appoximatios, a bias ad vaiace coectio of the pofile scoe is developed by solvig fo ež. ad vž. such that E u Ž. 0, Va u Ž. E u Ž., ˆ a ˆ a ˆ a whee u Ž. vž.už. ež. a. The adjusted pofile log-likelihood is the Ž. l H u Ž t. dt. The solutios ae simply a a Eˆ už. ež. ež. Eˆ už., vž.. Va už. Now athe tha adjustig the pofile likelihood fo the effect of uisace paamete estimatio, we coside the use of the McCullagh ad Tibshiai Ž adjustmet fo obust puposes. I the case of a model misspecificatio, oly the secod Batlett idetity fails ad so we may let ež. 0. Give the tue model is ukow, we caot compute the expected values equied to implemet vž. ad so eplace it by the asymptotically equivalet qua- ˆ

6 ROBUST PROFILE LIKELIHOOD 31 ˆ i1 i tity vž. iž. jž., whee iž. už. Ž. ad j Ý u Ž.. The adjusted pofile likelihood the becomes H ˆŽ v t. už t. dt. To elucidate implemetatio, itegatio ca be avoided by eplacig ˆv Ž. by the asymptotically equivalet quatity ˆv Ž ˆ. without affectig the desied popety that the secod Batlett idetity obtais. The esult is the obust pofile likelihood l Ž. vž ˆ. l Ž. ˆ p which has the followig popeties, which ae demostated below: 1. The secod Batlett idetity obtais asymptotically.. It is ivaiat ude tasfomatios of the fom, Ž., Ž,.. 3. Just as the stadad statistics esult fom l Ž. p, the siged oot, scoe ad Wald statistics based o l Ž. ae simply the obust stadadizatios of Ž., už. ad. ˆ. If thee is o uisace paamete peset, the l Ž. vž ˆ. lž. ˆ, whee Ž. Ž. i l ad j Ý l. i1 i The fist popety esults fom the costuctio of l Ž. ad ca be veified by diffeetiatig l Ž. ad otig the esult. I paticula, ote E 1 l Ž. Ž 1. 1 Ž. 1 Ž. Ž 1 O, ad l ad l diffe by O.. The fouth popety is etiely staightfowad. Ž ˆ. Ž. 1 Ivokig stadad calculatios, we ca show i J ad similaly Ž ˆ. s Ž. j J J J J. Hece we have, s s 1 J J, s J 1 ½ J 5 ˆ v ˆŽ. v ˆ, whee ˆ v is simply a sample vesio of v. So the siged oot, stadad scoe 1 Ž. Ž ' 1 ad Wald statistics based o l ae simply v, u v. ˆ ˆu ad ' ˆ 1 Ž. v, whee ˆˆ s s u, s Ž. ˆˆ, s ˆv J J J J, v J J J. That is, they ae simply the obust statistics suggested by Ket Ž 198. based o the asymptotic esults Ž 1.. The Wald statistic is stadadized by v ˆˆ, the well kow sadwich estimato that occus atually i the theoy of M-estimatio ad was evaluated fo may paametic settigs i Royall Ž Badoff-Nielse ad Soese Ž 199. geealize its use to matigale theoy. To establish the ivaiace popeties of l Ž., coside a epaameteiza- tio to. The log-likelihood fo is lž. ad similaly the log-likelihood fo is lž.. Note ˆ Ž ˆ. ad let ˆi i ˆ i ˆ with ivese ˆ ˆ. Fom this we have ˆi i i ˆi ˆj ˆi ˆj s i j s J Ji j s, J, Ji, j s, J i J j. s s ˆ ˆ

7 3 J. E. STAFFORD Assumig 1 is the paamete of iteest, the we have ude epaameteizatio ½ J 1 i J J j 1 1 i j k l 1, ˆ i J Jj, k J ˆ i j l ˆŽ ˆ1. J 1 1 i j 1 ˆ i J ˆ j Ž ˆ. 1 ˆ i i v, which equals v if is block diagoal; that is, if 0, i 1. A impotat class of tasfomatios ude which l Ž. is ivaiat ae those that peseve ad hece its itepetability. I the cotext of the adjusted pofile likelihood of Cox ad Reid Ž 1987., tasfomatios of this type, that also achieve paamete othogoality, educe the effect of uisace paamete estimatio o ifeece fo. I the peset cotext, the effect of uisace paamete estimatio is ot a coce, but Ket Ž 198. does show that paamete othogoality ude the tue model simplifies the obust vaiace estimates. Howeve, i the case of a model misspecificatio, paamete othogoality typically caot be obtaied because the tue model is ukow. 3. Geeal cumulat expessios. Fo each of the siged oot, scoe ad Wald statistics, we have the choice betwee a model-obust stadadizatio s vesus a simple model-based vesio s m. The cautious use may pefe to use s moe ofte tha is actually ecessay Ž i M-estimatio s is always used.. I this sectio, we study the effect of edudat use of s by compaig, ude model coectess, asymptotic expasios fo the fist thee cumulats of each statistic stadadized by sm ad the by s. The explicit details of how the expasios ae deived ae legthy ad theefoe defeed to the Appedix. A outlie is povided hee. Diffeeces i the expasios ae due to diffeeces i the expasios fo s ad s. I each case we will have m sm s0 1 s11 1 s1 OpŽ 3., s s0 1 Ž s11 s1. 1 Ž s1 s. OpŽ 3., whee s, s,... epeset the coefficiets of the powes of. The expasio 11 1 of s shaes all the tems i the expasio of s m. Hece, the additioal tems s1 ad s i the expasio of s ae of iteest sice they will effect all diffeeces i all subsequet calculatios. Allowig T to be eithe Ž., už. o ˆ, T has a expasio of the fom T T0 1 T1 1 T OpŽ 3.. Lettig T Ts ad Tm Tsm we the have T T s 1 Ž T s T s. m Ž T0 s1 T s0 T1 s11. OpŽ 3., T T0 s0 1 Ž T1 s0 T0 s11 T0 s1. 1 T s T s T s T s T s O p

8 Lettig d T T m, the ROBUST PROFILE LIKELIHOOD 33 d 1 s1t0 1 Ž T0 s T1 s1. OpŽ 3.. Calculatios i the Appedix show ET s fo each pai of competig statistics. Also, T s, T s ivolve odd powes of O Ž p adom vaiables that will have expected values with ode 1. Hece, 3 E d OŽ.. 1 The case of the skewess is almost as simple. Fo ay adom vaiable X, we may wite the skewess i tems of its momets, Fo T, T m we have X 3 i X , i E X. 3 1 OŽ 3., 1 OŽ 1., 3 OŽ 1., E Tm E T0 s0 Ž T0 T1 s0 T0 s0 s11. OŽ., ' 3 Ž E T E T0 s0 E T0 T1 s0 T0 s0 s11 T0 s0 s1 O, ad so 3 Tm T ' E T0 s0 s1 Ž E Tm E Tm E T E T. OŽ. ' 3 3 ' E T0 s0 s1 E d O E T0 s0 s1 O. ' 3 Calculatios i the Appedix show E T s s fo each pai of competig Ž 3 statistics ad hece O. 3. Oe impotat cosequece of this is that Ž 3 the siged oot etais the popety that its skewess is O. ude obust stadadizatio DiCiccio Ž Fo two competig statistics with simila bias ad skewess popeties, a compaiso of vaiaces will povide isight ito elative behaviou. I paticula, the statistics with lage vaiace will typically fall ito a commo ejectio egio moe ofte. Oe caot expect this to always be the case, especially give that othe distibutioal chaacteistics, like kutosis, have a effect as well. Howeve, ufavouable vaiace compaisos ca alet us to iefficiecies. A atual way to make vaiace compaisos is though a measue of asymptotic elative efficiecy Ž ARE. AREŽ T m, T. VaŽ T. VaŽ T m.. Give model coectess, both s ad s ae cosistet fo the same quam tity. Hece it becomes ecessay to examie highe ode tems.

9 3 J. E. STAFFORD We have Va T Va Tm Ž T Tm. Va Tm d Va Tm Va d Cov T m, d. Let Va Tm 1 OŽ.. Ž 1. Ž 1 Sice d O, both Va d ad Cov T, d ae O. p m, ad so Va T Va d Cov T, d Va T m m m ½ 5 ½ 5 1 Va d Cov T m, d OŽ. 1 OŽ. ½1 Va d Cov T m, d 5 ½1 5 OŽ. 1 Va d Cov T, d OŽ. 1 E d E T d OŽ. m m s1t0 s E s T s T s T T s s T s s OŽ.. Calculatios i the Appedix show that fo each pai of competig statistics the asymptotic elative efficiecy is i j i j 1 I I 3I I A I A I I A st u s, t u, s, t u, st u si t u j i,, s t u j k 0 I I s I t u I 6I 8 I I I st u t, su, t, su, st u t, su k0 A si I i j At u j Ii, t I i j Asu j OŽ., whee I I s I t I u ad A I I I. Note possesses the si, si s, i si same ivaiace popeties as the obust pofile likelihood.. Examples. I this sectio, we coside a vaiety of examples whee the above expessio fo the asymptotic elative efficiecy is evaluated ad tems of ode o smalle ae igoed. I most cases, the obust methods suffe losses i efficiecy that ae sevee fo small sample sizes. Some simulatio esults ae also peseted whee we focus o the siged oot of the likelihood atio statistic. Fo these, ad othes ot epoted, the obust methods ted to esult i cofidece itevals that ude-cove. I the examples, the maximum likelihood estimate is eithe a commo summay statistic, like a mea, vaiace o coelatio coefficiet, o it is a egessio coefficiet, whee we assume the developmets of Gould ad Lawless Ž That is, the maximum likelihood estimate is meaigful i the pesece of a model misspecificatio. 1

10 ROBUST PROFILE LIKELIHOOD 35 Compute algeba methods ae used fo the evaluatio of. These methods ivolve opeatos fo evaluatig highly ested sums ad the expected ifomatio quatities that compose the tems i a sum. The elevat code, witte i Mathematica Vesio.0 see Wolfam Ž 1988., is available at the ftp site utstat.tooto.edu i the file ARE.ta. The expasio fo is povided i a fom that may be maipulated i Mathematica, ad istuctios fo the use of the code ae give alog with a example. Fo detailed ifomatio elated to these tools see Adews ad Staffod Ž 1993., Staffod ad Adews Ž ad Staffod, Adews ad Wag Ž EXAMPLE 1 Ž Nomal mea, vaiace ukow.. Fo this example, data x i, i 1,...,, ae assumed to be fom a uivaiate omal distibutio with mea ad vaiace. Fo this case, the obseved ifomatio is diagoal ad so the sadwich estimato is simply J J,. Howeve, J, J Ž. Ýi1 x i x ad hece the obust methods educe to the model-specific methods. That is, the usual methods of ifeece based o the maximum likelihood estimate, scoe o siged oot ae obust i this case. Hee, the above expessio equals 1 as it should. EXAMPLE Ž Nomal egessio.. This example also appeas i DiCiccio ad Ste Ž 199b.. We assume data Ž y, x. i i, i 1,...,, deive fom a egessio model whee the coditioal expectatio of y give x is a liea fuctio of x, x i t ; eos ae omally distibuted with vaiace. The paamete of iteest is ad the esultat expessio fo the asymptotic elative efficiecy is 1 9. Note that this is ot oly idepedet of the paametes i this model, but also of the desig. That is, this case is equivalet to Example 1 whee the paamete of iteest is the stadad deviatio athe tha the mea. Fo this example, we simulated data fom a egessio with two covaiates, 10 ad 1. This value of was tested at 95% level fo 5000 simulatios. The usual siged oot ejected this hypothesis at the ate 0.059, while the siged oot of the obust pofile likelihood ejected this hypothesis at the ate Figue 1 is a plot of the quatiles fo the omal distibutio agaist the simulated values of the siged oot of the likelihood atio statistic ad its obust vesio. The quatiles of the obust statistics ae shifted a geate distace fom the diagoal. The dotted lie epesets whee the theoy pedicts the obust quatiles should lie. The lie passes though the oigi with slope detemied by. Hee, thee is close ageemet betwee theoy ad simulatio. EXAMPLE 3 Ž Expoetial egessio.. I this example, data ae assumed 1 to have expoetial distibutio with mea expž z. i, whee the slope is the paamete of iteest. Fo this case, a compoet of the log-likelihood is zi logž. 1 yi expž z i.. Expessios deived ae simplified by stadadizig the covaiate so Ýz 0. i

11 36 J. E. STAFFORD FIG. 1. Nomal egessio: Q-Q plots fo the simulated values of the siged squae oot of the pofile likelihood atio statistic ad its obust coutepat. The lie gives the locatio of the quatiles of the obust statistic as pedicted by the theoy. This also has the effect of othogoalizig the paametes. Lettig 1 Ýz j, the asymptotic elative efficiecy is i z j 17z 5z 1 z 3 z 1. EXAMPLE Ž Coelatio coefficiet.. Fo this example, we assume bivai- T T ate data x i x 1i, x i, i 1,...,, deive fom N,, whee Ž,. ad 1. A compoet of the log-likelihood is Ž. i i log 1 logž. logž. Ž x. Ž x T 1

12 ROBUST PROFILE LIKELIHOOD 37 The calculatios fo this example ae complicated by the elatively lage umbe of paametes. Fo istace, the deivative li, has 33 tems ad the evaluatio of the above expessio fo asymptotic elative efficiecy ivolves ie poducts of aays with dimesio o 6. This meas that some ie poducts ae ested sums with possibly 5 6 tems that ae expected values of poducts of tems like l i,. This is a extemely compli- cated calculatio. Lawley Ž used popeties specific to the omal distibutio to simplify calculatios i this case. We choose to explicitly evaluate all ie poducts. The use of compute algeba tools geatly simplifies such a calculatio. The value of is simply 1 6. EXAMPLE 5 Nomal egessio Ž evisited.. a omal egessio model whee We agai coside the case of yi x1i x i i ; i NŽ 0,., i 1,...,. The paamete of iteest is the coefficiet of the fist covaiate. The secod covaiate is assumed to cotibute uisace vaiatio i the data. Calculatios ae agai complicated fo this example ad agai compute algeba is of valuable assistace. Lettig ˆ 1 Ý x j x k jk 1i i, the asymptotic elative efficiecy is ž 1 ˆ ˆ ˆ ˆ 10ˆ ˆ 3 ˆ 17ˆ ˆ ˆ 1ˆ 3 ˆ ˆ ˆ 3 0 ˆ 11 ˆ 0 ˆ 0 ˆ 0 ˆ 11 ˆ 0 ˆ 0 11 ˆ 13 ˆ 0 ˆ 3 0 ˆ ˆ 0 / ž 0 Ž / ˆ ˆ 3ˆ ˆ ˆ ˆ ˆ ˆ. Whe the desig is othogoal, this expessio simplifies cosideably to Ž. 1 3ˆ ˆ. 0 EXAMPLE 6 Ž Paamete othogoality.. I this last example, we test the ivaiace popety of by cosideig two examples whee paametes may be othogoalized Cox ad Reid Ž Coside fist, bivaiate data Ž x. Ž. 1i, x i, i 1,...,, fom expoetial distibutios with meas 1 ad 1, espectively. A compoet of the log-likelihood is logž. logž. x Ž. x, 1 ad the paamete of iteest is, the atio of the two meas. Ude both this paameteizatio ad the othogoal paameteizatio give i Cox ad Reid Ž 1987., was calculated to be 1 5.

13 38 J. E. STAFFORD I the case of a Weibull distibutio, whee data assumed to have desity x, i 1,...,, ae i 1 Ž x. Ž x. e, ad iteest lies i the idex, was calculated to be fo both paameteizatios. 5. Fial emaks. I coclusio, the use of a model-obust vaiace estimate fo the siged squae oot, scoe o Wald statistic, while leavig bias ad skewess chaacteistics elatively uchaged, ca icease vaiability cosideably. I the examples cosideed, obust methods ae much less efficiet tha thei model-based coutepats. This suggests cautio should be used whe employig a obust method. It also eflects the impotace of model assumptios i impovig the pecisio of ifeece. Robust methods wee deived fom a adjusted pofile likelihood due to McCullagh ad Tibshiai Ž A impotat aspect i the developmet was to appoximate the itegal l Ž. H ˆv Ž t. u Ž t. dt by l Ž. ˆv Ž ˆ. l Ž. p p fom which the stadad esults of Ket Ž 198. ad Royall Ž ca be obtaied. A questio of iteest is whethe this is wise give l Ž. makes a adaptive adjustmet ove the etie age of while l Ž. does ot. A simple example is illumiatig. Coside data x,..., x fom a NŽ,. 1 distibutio whee model assumptios icoectly specify that 1. The log-likelihood fuctio is lž. l Ž. ÝŽ x., vž. ÝŽ p i ˆ x i. Ž ˆ. Ž. ad ˆv Ý x i x ad the pofile log-likelihood ude the tue model is l Ž. logvž. t p ˆ. What is compellig about this example is that l Ž. is exactly l Ž. while this is ot the case fo l Ž. t p. So eve though l Ž. is computatioally coveiet, a compomise is sometimes made. This is pehaps woth futhe ivestigatio. APPENDIX Below ae the detailed calculatios ivolved i deivig the esults of Sectio 3. Compute algeba tools such as those discussed i Staffod, A- Ž. dews ad Wag 199 wee istumetal. Let k 0 I 1, z I s z s, z I s z ad z I s z s. Expasios of the siged oot, Wald ad scoe s statistic ae well kow Lawley Ž 1956.; DiCiccio ad Ste Ž 199b. ad fom these we have T T c c c c c ˆ s 1 s t z I z s z I I st z z s s t 0 0 s s t s s 1 s t s s 3 st 1 s t 1 s t 3 I I s z z 3 I I z z t s t ' Ž. u z k 1k I z z I I z z z k 1k I z z I z z I I z z 0 1

14 ROBUST PROFILE LIKELIHOOD 39 togethe with coefficiets c 1,..., c5 defied fo late use. To detemie the coefficiets fo the vaious estimates, let Ž. Ž t 1 1 t w u v. A z I z t s s st B z z I z z I I z z I I I z z, s st st u st u v st u v w C s z, s Ž I, st Is, t. z t, t 1 D s z, st zs, t z z t z u Is, t u I, st u I t, su t v u 1 I zu v z I t w I z u z v u v w I, st I s, t, E s I t A I u s, t u F s I t A I u v A I w s I t B I u s, t u v w t u G I C s I s E, H I D I s F E C I s E I E s. s s, s Usig these expessios, we have the expasios J s I, s 1 A s 1 B s OpŽ 3., J, s I, s 1 C s 1 D s OpŽ 3., J s I s 1 E s 1 F s OpŽ 3., J J, s J s I 1 G 1 H OpŽ 3., 1 E F 3 E 1 J k0, ' k k0 8k0 1 G H 3G s 1 J J, s J k0, ' k k0 8k0 ½ 5 1 E F E J k0, ' 3 k k0 8k s, s 0 ' J k0 k 8k J J J G E H F 3G G E k, 0 0 ½ 5 ' s 1, s 1 J k0 k0 J J J G E H F 3G G E E. 8k 0

15 350 J. E. STAFFORD Let G E ad H F. Fom the above expasios fo T, sm ad s ad usig the coefficiets c,..., c, we have T s z k, T s s z k, T s s z k, T s z k, T T s s T z k, T s s c z E k, ž / T s s z k z 3G c G E c E 8k. I paticula, T s s T s s T s z z Ž C s A s. Ž Ct u c5 A t u.. 6 k k 0 0 Fo the calculatios that follow, we make use of umeous idetities, z z A z i, s, s, s si I Ž C A I s I s s. s I s, I si I, si Is, i Ii, s I, s, i 0, s 1 i j s, s, t, u, s, t u si t, u, j E I I z I I A I I, i j 1, s t u, s, t, u, s, t u, s, j t ui E z z I I I I I A OŽ., i j 1, s t u, s, t u s, t u i, s t u j E z z I I I I I A OŽ., E z z z I I I A I I OŽ., i u i j 1 t i s, s, t u s, t u t ui j, t u E z z z I I I I I A OŽ., j u i j 1 s t j, s, t u s, t u j, t u si E z z z z Ž I I I I. Ž I I. OŽ., i i j i j 1, si si i,, js i, j s l t u t u 1 jl t, ju E z z z Ž I I I I. I OŽ., k l t u t u 1 E z z z I I I I OŽ., 3 j j k ad the expected values E z, E z, E z z, E z z z s s s s, j j k j k Ž 1 E z z z, E z I z z ad E z I z z ae all O. s t s t j s t o j k j k smalle. Fom these idetities, we immediately have that ET s 0 1 ad 3 Ž 1. Ž 3 E T s s ae O ad hece E E ae O.. Also, afte

16 legthy calculatios we have ROBUST PROFILE LIKELIHOOD 351 i j E T0 s0 s1 Ž I, s, t, u I, s, t u A si I I t, u, j., ' k0 c u i j E T0T1 s0 s1 Ž I, s, t u I s, t u. I Ij, t u I A si, k 0 i j E z I st u I, st u I, s, t u I s, t u A si I At u j E Ž C A. Ž C c A. 6 k 0 1 s t u st u, st u t, su, t, su I I I I I I I 8 I s s t u 5 t u 6I t, su Ij, t I i j Au si A si I i j A t u j, i j I I Ž c 1. I c I Ž I c I. I A. 6 k 0, s, t, u 5, s, t u 5 s, t u, s, i 5 i, s t u j Combiig all the elevat tems yields the same expessio fo the siged oot, scoe ad Wald statistics: i j i j 1 I I 3I I A I A I I A k 0 st u s, t u, s, t u, st u si t u j i,, s t u j I I s I t u I 6I 8 I I I st u t, su, t, su, st u t, su k0 A si I i j At u j Ii, t I i j Asu j OŽ.. Ackowledgmets. I wish to thak Tom DiCiccio, Athoy Daviso, Rob Tibshiai ad the efeees fo vey useful suggestios that aided the pepaatio of this pape. REFERENCES ANDREWS, D. F. ad STAFFORD, J. E. Ž Tools fo the symbolic computatio of asymptotic expasios. J. Roy. Statist. Soc. Se. B BARNDORFF-NIELSEN, O. E. Ž O a fomula fo the distibutio of the maximum likelihood estimato. Biometika BARNDORFF-NIELSEN, O. E. ad SORENSON, M. Ž A eview of some aspects of asymptotic likelihood theoy fo stochastic pocesses. Iteat. Statist. Rev COX, D. R. ad REID, N. Ž Paametic othogoality ad appoximate coditioal ifeece. J. Roy. Statist. Soc. Se. B COX, D. R. ad REID, N. Ž A ote o the calculatio of adjusted pofile likelihood. J. Roy. Statist. Soc. Se. B DICICCIO, T. J. Ž O paamete tasfomatios ad iteval estimatio. Biometika

17 35 J. E. STAFFORD DICICCIO, T. J. ad STERN, S. E. Ž 199a.. Fequetist ad Bayesia Batlett coectio of test statistics based o adjusted pofile likelihoods J. Roy. Statist. Soc. Se. B DICICCIO, T. J. ad STERN, S. E. Ž 199b.. Costuctig appoximately stadad omal pivots fom siged oots at adjusted likelihood atio statistics. Scad. J. Statist EFRON, B. ad HINKLEY, D. V. Ž Assessig the accuacy of the maximum likelihood estimato: Obseved vesus expected Fishe ifomatio. Biometika GOULD, A. ad LAWLESS, J. F. Ž Cosistecy ad efficiecy of egessio coefficiet estimates i locatio-scale models. Biometika HUBER, P. J. Ž The behaviou of maximum likelihood estimates ude ostadad coditios. Poc. Fifth Bekeley Symp. Math. Statist. Pobab Uiv. Califoia Pess, Bekeley. KENT, T. J. Ž Robust popeties of likelihood atio tests. Biometika LAWLEY, D. N. Ž A geeal method fo appoximatig the distibutio of likelihood atio citeia. Biometika LIN, D. Y. ad WEI, L. J. Ž The obust ifeece fo the Cox popotioal hazads model. J. Ame. Statist. Assoc MCCULLAGH, P. ad TIBSHIRANI, R. Ž A simple method fo the adjustmet of pofile likelihoods. J. Roy. Statist. Soc. Se. B ROYALL, R. M. Ž Model obust cofidece itevals usig maximum likelihood estimatos. Iteat. Statist. Rev STAFFORD, J. E. ad ANDREWS, D. F. Ž A symbolic algoithm fo studyig adjustmets to the pofile likelihood. Biometika STAFFORD, J. E., ANDREWS, D. F. ad WANG, Y. Ž Symbolic computatio: A uified appoach to studyig likelihood. Statistics ad Computig 355. WHITE, H. Ž Maximum likelihood estimatio of misspecified models. Ecoometica WOLFRAM, S. Ž Mathematica: A System fo Doig Mathematics by Compute. Addiso- Wesley, Readig, MA. DEPARTMENT OF STATISTICAL AND ACTUARIAL SCIENCES UNIVERSITY OF WESTERN ONTARIO ROOM 6 WESTERN SCIENCE CENTER LONDON, ONTARIO CANADA N6A 5B7