ON THE USE OF OBSERVED FISHER INFORMATION IN WALD AND SCORE TEST

Transcription

1 N THE USE F BSERVED FISHER INFRMATIN IN WALD AND SCRE TEST Vasudeva Guddattu 1 & Arua Rao Abstract I the recet years there s a large alcato of large samle tests may scetfc vestgatos. The commoly used large samle tests are Lkelhood rato tests, Wald ad Score tests. Wald ad Score tests make use of exected fsher formato comutato of test statstc. I comlex roblems the exressos for fsher formato caot be derved ad t s customary to use the observed fsher formato lace of the exected fsher formato for these tests. The uaswered questos are 1 What haes to the sze ad ower of Wald ad Score test f we use observed fsher formato. What haes f fsher formato s evaluated at restrcted MLE of arameter for Wald test? 3 What s the mlcato of fsher formato evaluated at urestrcted MLE for Score test? Ths aer makes a attemt to aswer these questos. Extesve smulatos have bee carred out to aswer these questos testg for locato arameter θ of Cauchy dstrbuto ad testg for arameter λ of zero Iflated Posso dstrbuto. The results dcate that the test statstc usg observed fsher formato mata ower ad sze whe comared to the test statstc usg exected fsher formato. Further f the arameter uder cosderato s uvarate the t s suggested to use Wald test wth observed fsher formato evaluated at restrcted MLE of θ. If the arameter uder cosderato s bvarate the our vestgato suggests Score test wth observed fsher formato usg restrcted MLE of λ. Key Words Large samle tests, Wald tests.score tests, Exected fsher formato, bserved fsher formato restrcted ad urestrcted MLE 1. Vasudeva guddattu,vasudev.guddattu@gmal.com. Arua Rao, aruaraomu@yahoo.com

2 1. Itroducto I the ast, a lot of work has take lace arametrc tests of sgfcace. I early days the rmary focus was o sgfcace tests whch cosders oly ull hyothess for the develomet of test statstc. It was Neyma ad Pearso (1933 who troduced the cocet of exact arametrc test of hyothess. However, may of the alcatos these exact tests do ot exst. e of the rmary reasos s that suffcecy ad varace rcle s ot satsfed ad the secod roblem s that the exact dstrbuto of the test statstc s ot tractable. As a alteratve, lkelhood rato test develoed by Neyma ad Pearso (198 was favorte amog the scetsts varous dscles. Lkelhood rato test was oular amog ractoers utl two of the comettors Abraham Wald (1941 who troduced Wald test ad C.R.Rao (1945 who troduced Score test emerged to the feld. Lkelhood rato test make use of restrcted ad urestrcted maxmum lkelhood estmators (MLE, Wald test make use of urestrcted MLE ad fsher formato ad Score test makes use of Score fucto, restrcted MLE ad fsher formato. Uder ull hyothess all three tests are asymtotcally equvalet ad have asymtotc ch-square dstrbuto wth same degrees of freedom.the develomet of olear regresso esecally geeralzed lear models(glm troduced by Nedler ad Wedderbur(197 gave rse to the frequet use of these tests. e of the roblem commoly ecoutered these tests s to comute the fsher formato matrx deoted by E log f ( x, j whch was later called as exected fsher formato.the ucertaty the exstece of fsher formato matrx lead to the use of observed fsher formato matrx deoted by log f ( x, j whch was later called as observed fsher formato. A questo whch remaed uaswered for a log tme was o the use of observed fsher formato Wald ad Score test. Secfcally the questo s what haes to the tye I error rates ad ower of the test f observed fsher formato s used stead of exected fsher formato. Further Wald test exected fsher formato s evaluated at urestrcted MLE of arameter where as Score test

3 exected fsher formato s evaluated at restrcted MLE of arameter uder cosderato. Cox ad Hkley (1974 oted out that oe ca use the secfed arametrc values ad urestrcted MLE the evaluato of observed fsher formato for Wald test ad ucodtoal MLE the evaluato of observed fsher formato for Score test. Kale(1998 also rases ths ssues but do ot ursue the ssue further.ths has motvated us to look at these ssues. A attemt has bee made ths aer to aswer the questo usg smulatos. The atteto s restrcted for the locato arameter of the Cauchy dstrbuto ad arameter of Zero flated Posso dstrbuto the resece of the usace arameter.the reaso for choosg these dstrbutos s that exact form of exected fsher formato s kow. For testg locato arameter of Cauchy dstrbuto, the results of smulato exermet suggest the use of observed fsher formato evaluated at restrcted MLE of for Wald test ad observed fsher formato evaluated at urestrcted MLE of for Score test For testg of zero flated Posso dstrbuto the resece of usace arameter,the results of the smulato exermet suggest the use of observed fsher formato evaluated at urestrcted MLE of ad for Wald test ad observed fsher formato wth restrcted MLE of ad for Score test. The aer s orgazed as follows. The small samle comarso of the Wald ad Score test usg observed ad exected fsher formato for locato arameter of Cauchy dstrbuto are reseted secto. Secto.1 cosders estmated tye 1 st error rates of Wald ad Score tests wth the focal ot of atteto of ower comarsos secto.. Secto 3 cosders the estmated tye 1 error rates ad ower for Wald ad Score test usg observed fsher formato ad exected fsher formato evaluated wth urestrcted MLE ad restrcted MLE of resece of usace arameter. I secto 4 we reseted the geeral dscusso. The results are reseted aedx.. Small samle comarsos for Cauchy dstrbuto.

4 .1 Tye 1 error rates. The sze of the test dcates the maxmum robablty of tye I error gve that ull hyothess s true. Geerally we atcate that the robablty of tye I error should ot exceed a gve level ad ths level s kow as level of sgfcace. Whe exact dstrbuto of the test statstc uder ull hyothess s kow, the sze of the test s equal to level of sgfcace. I several stuatos the exact dstrbuto of test statstc s ot tractable. I such cases oe may have to use asymtotcal dstrbuto to determe crtcal value of the test statstcs, where the sze of the test may ot be equal to the level of sgfcace. A test s sad to be lberal f estmated tye I error rates are larger tha level ad strget f t s less tha or equal to level. It s desrable that a test s ether lberal or strget. I ths secto we try to vestgate whether Score ad Wald test mata tye I error rates whe usg observed ad exected fsher formato for small samle sze. We cosdered testg for a locato arameter of Cauchy dstrbuto wth oe sded ad two sded alteratves. The desty fucto of Cauchy dstrbuto s gve by 1 f ( x, - x (1 (1 ( x.further the Maxmum lkelhood (ML equato for Cauchy dstrbuto based o deedet samles from f (x, s gve by the soluto of 1 1 ( x ( x ( The MLE of ca be obtaed by solvg the above ML equato ( by Newto-Rahso rocedure or method of scorg. The exected fsher formato for Cauchy dstrbuto based o observatos s I (.The observed fsher formato s gve by log f ( x, 1 (1 ( x (1 ( x. (3

5 Wald test statstc usg exected fsher formato s gve by W 1 ( (4 Wald test statstc usg observed fsher formato evaluated at urestrcted MLE of θ s gve by W (1- (x ˆ ( ˆ. (5 1 (1 ( x ˆ Wald test statstc usg observed fsher formato wth restrcted MLE of θ s gve by W 3 (1- (x ( ˆ. (6 1 (1 ( x Score test statstc usg exected fsher formato s gve by S 1 1 ( x 1 ( x /. (7 Score test statstc usg observed fsher formato evaluated at urestrcted MLE of θ s gve by S ( x (1 ( x (1- (x ( x ˆ ˆ. (8 Score test statstc usg observed fsher formato evaluated at restrcted MLE of θ s gve by S ( x 1 ( x (1- (x (1 ( x. (9 For estmatg tye I error rates the smulato has bee carred out usg MATLAB.I the smulato study we use the followg cofgurato. Samle sze =, 4, 6, 8,1. =.Number of relcatos =1. We use W 1 W &W 3 for Wald test usg exected fsher formato, observed

6 fsher formato usg urestrcted MLE of arameter ad observed fsher formato usg restrcted MLE of the arameter. Smlarly we use S 1,S,S 3 for Score test usg exected fsher formato, observed fsher formato usg urestrcted MLE of arameter ad observed fsher formato usg restrcted MLE of the arameter. Table.1 gves the estmated tye I error rates for rght taled, left taled ad two taled Wald test. For rght sde alteratve whe samle sze=, we ca observe that W 1 ad W 3 are close to omal level whle W slghtly devates from the omal level. The corresodg estmated error rates are W 1 =.47, W =.58 & W 3 =.45. The estmated tye I error rates for rght taled, left taled ad two taled Score test are reseted Table.3. We ca observe for rght taled alteratve ad samle sze, S 1 ad S are very close to the omal level whle S 3 s slghtly devated from t. The corresodg estmated tye I error rates are S 1 =.5, S =.49& S 3 =.64. Further as we crease the samle sze, the estmated tye I error rates of W ad S 3 coverge to the omal level. The same coclusos are observed for left ad two taled alteratve.. Estmated ower for locato arameter of Cauchy dstrbuto. For the test to be robust ractce we should have the followg roertes. 1. Mata tye I error rates. Has a reasoable ower for alteratve hyothess ot far away from the ull hyothess. For test (x, ower of the test s defed as E 1 ( (x, 1 1. The alteratve arameter sace. Power of the test dcates the robablty of rejectg the ull hyothess gve that the true value of the arameter s 1 I the revous secto we have see that all the sx test statstcs (three for Wald ad three for Score test by ad large mata tye 1 st error rates.therefore the otmal choce of the test ca be acheved by cosderg behavor of ower fucto. Smulato study s carred out to estmate the ower of the tests dscussed revous secto. We use the same cofguratos as revous secto. We take the followg values of locato =-1.5, -1, -.8, -.6, -.4, -.,,.,.4,.6,.8,

7 1, 1.5. We are resetg estmated ower of rght taled Wald ad Score test for samle szes ad 1 ths aer. The fereces of left ad two taled are smlar to rght taled test. I Table.ad.4 we reset the lots of estmated ower curves of rght taled Wald test ad Score test for samle szes ad1.the erformace of Wald test usg observed fsher formato evaluated at restrcted MLE of ad Score test usg observed fsher formato evaluated at restrcted MLE of are better tha other test statstcs. The rate of covergece of ower to 1 s faster as we crease the samle sze to 1..3 Cocluso From the revous secto we ca observe that the erformace of Wald test statstc usg observed fsher formato wth restrcted MLE of s better tha both oe sded ad two sded alteratves tha other tests. Cotrarly Score test the erformace of Score test statstc usg observed fsher formato wth urestrcted MLE of s erformg better tha other test. The over all cocluso s that f the arameter uder cosderato s uvarate, the oe ca use Wald test stead of Score test. Sce most comlex stuatos the comutato of MLE ad exected fsher formato s ot easy ad the use of observed fsher formato stead of exected fsher formato s justfed. The valdty of the coclusos of these tests for testg a arameter the resece of usace arameter s dscussed the ext secto. 3.Small samle comarsos for Zero Iflated Posso dstrbuto. 3.1 Tye 1 error rates. I the revous secto, we comared the small samle tye I error rates ad ower of the Wald ad Score test usg exected ad observed fsher formato for a Cauchy dstrbuto wth oe arameter. I order to check whether the same coclusos are observed for testg a smle hyothess for a arameter the resece of a usace arameter, we use the zero Iflated Posso dstrbuto. The chage the dstrbuto s cosdered to establsh the valdty of the coclusos a

8 wder frame work tha to restrct oly oe dstrbuto. The choce of Iflated Posso dstrbuto s based o wde alcato recet studes (Lambert 199 ad Kale The robablty mass fucto of Zero Iflated Posso dstrbuto s gve by (1 e P( Y,, P y e y! Y Y (1 Cosder a radom samle of sze from the oulato characterzed by the robablty mass fucto P(x,,.Kale(1998 has gve followg maxmum lkelhood equatos. log L ( 1 e ( (1 e (11 log L (1 ( e x ( 1 e. (1 log L ( (1 e (1 e (13 Hece usg equato (13 (1 we wll get log L ( (1 e e x ( 1. (14 Here s the umber of zeros a samle of sze geerated from P(x,. We ca observe that the above equato s urely a fucto of. We ca aly Newto-Rahso rocedure or method of scorg to fd the MLE of.the urestrcted MLE of s obtaed by substtutg the MLE of equato (11 ad solve for P. The restrcted MLE of P s got by secfyg the value of uder ull hyothess equato (11 ad solvg t. The fsher formato matrx s gve by

9 I I I I I (15 where (1 e I (1 e. (16 e I (1 e (17 I = I. I (1 [(1 e e ] e. (18 The observed fsher formato for matrx s gve by (19 where ( [(1 1 e e ] ( 1 ( Y {( e ( 1 [(1 e ((1 e ] e ( e (1 1 ( Y ( [(1 1 e e ] 1 (1 ( Y ( Here ( Y 1f Y elsewhere

10 For the reset vestgato we have cosdered testg a secfed value of whe s take as a usace arameter. We have cosdered both oe sded ad two sded alteratve hyothess. The Wald test statstc s gve by W 11 where I ˆ ( I ( ˆ, ˆ s the secod dagol elemet of I -1 (3 ˆ W I ( ˆ, (4 1 ( W 13 ˆ where ( ( ˆ, ˆ (5 s the secod dagol elemet of -1 ˆ W ( ˆ, (6 14 ( Where ˆ ad ˆ are urestrcted maxmum lkelhood estmators of ad, resectvely. The Score test statstc s gve by ˆ e (1 ˆ e ˆ S 11. (7 I (ˆ, ˆ. ˆ e (1 ˆ e ˆ S 1. (8 I (ˆ,. S 13 (1 e ˆ ˆ e ˆ ( ˆ, ˆ. (9 S 14 (1 e ˆ ˆ e ˆ ( ˆ,.. (3 Where ˆ ad are restrcted maxmum lkelhood estmators of P ad resectvely.

11 We use the followg cofgurato for estmatg tye I error rates. Samle sze =, 4, 6, 8, 1. =15,P=.5.The level of sgfcace s fxed at =.5. For each of the test, there are four test statstcs.for Wald test we have I. The test statstc corresodg to the exected fsher formato usg urestrcted MLE of ad II. The test statstc corresodg to the exected fsher formato usg urestrcted MLE of ad restrcted MLE of. III. The test statstc corresodg to the observed fsher formato usg urestrcted MLE of ad. IV. The test statstc corresodg to the observed fsher formato usg urestrcted MLE of ad restrcted MLE of. We deote these four test statstcs by W 11, W 1, W 13 ad W 14 resectvely.smlarly for Score test we have four test statstcs amely I. The test statstc corresodg to the exected fsher formato usg urestrcted MLE of ad restrcted MLE of. II. The test statstc corresodg to the exected fsher formato usg restrcted MLE of ad restrcted MLE of. III. The test statstc corresodg to the observed fsher formato usg ucodtoal MLE of ad restrcted MLE of. IV. The test statstc corresodg to the observed fsher formato usg restrcted MLE of ad restrcted MLE of. We deote these test statstcs by S 11, S 1, S 13, ad S 14 resectvely. The estmated tye I error rates for oe taled ad two taled Wald ad Score tests are reseted tables 3.1 ad 3.. From table 3.1 we ca observe that for two taled alteratve ad samle sze, the Wald test statstcs amely W 11 W 1 ad W 13 mata tye I error rates whle W 14 devates from the omal level.the estmated error

12 rates for the corresodg test statstcs are.53,.48,.6 ad.45 resectvely. As we crease samle sze all three statstcs coverge to omal level.the same ferece s observed rght taled ad left taled alteratve. For two taled Score test we ca observe table 3. that the test statstc amely S 11 ad S 14 mata tye I error rates tha other two. The corresodg test statstc values are S 11 =.47, S 1 =.58, S 13 =.7, S 14 =.45 resectvely. As we crease the samle sze all the three statstcs coverge to the omal level. 3. Estmated ower of Wald ad Score test for Zero Iflated Posso dstrbuto I revous secto we have see three Wald tests ad two of score test mata tye I error rates. The otmum choce of test ca be acheved by lookg at the ower fucto. Smulato study was carred out to estmate the ower of these tests. We use the same cofgurato as secto 3.1.The values of the arameter λ for geeratg samles uder alteratve hyothess are 5,6,7 9,11,13, 15, 17,19,1, 3,5 ad 3.We are resetg results for two taled alteratve ths aer. The erformace of oe sded alteratve s smlar to that of two taled alteratve. Table 3.3 gves the estmated ower of two sded Wald test for samle sze ad 1. The erformaces of W 1 ad W 13 are better tha other two test statstc for dfferet samle szes. For two sded Score test statstc whe samle sze s, S 1 erforms better followed by S 14 tha other two test statstcs. The corresodg estmated ower are gve table 3.4. We ca observe that eve f S 1 erforms slghtly better tha S 14 for small samle sze the dfferece betwee them s eglsable. For examle whe λ=3 the estmated ower for S 1 s.9 ad for S 14 =.87. As we crease samle sze S 14 erforms equally to S 1. The covergece of estmated ower s faster as we crease the samle sze for both tests. From the above dscusso t s clear that the use of observed fsher formato stead of exected fsher formato s justfed Wald ad Score test statstcs. Also Wald test statstc we ca use observed fsher formato wth the urestrcted MLE of the arameter uder cosderato ad usace arameter. I Score test we ca use observed fsher formato wth the arameter uder ull hyotheses ad restrcted MLE of the usace arameter.

13 4. Dscussos I the comutato of Wald ad Score test statstcs, t s commo ractce to use observed fsher formato. I the Wald test ucodtoal MLE are substtuted for the arameter value the observed fsher formato, whle the score test we use codtoal MLE of the arameter. A roblem that s usettled s what haes f the arameter values secfed uder ull hyothess s used the fsher formato for Wald test ad the effect of usg urestrcted MLE for fsher formato the Score test statstc. Ths aer makes a attemt to look at these ssues. We have cosdered two stuatos amely (a The uderlyg dstrbuto deeds o a real arameter. (b The arameter uder cosderato s vector valued. The dstrbutos that are chose s Cauchy dstrbuto wth oly the locato arameter ad the Zero Iflated Posso dstrbuto. The latter s chose because of the wde use of flated Posso dstrbuto the recet years. For the Cauchy dstrbutos the smulato result dcated that for the Wald test statstc, t s better to use observed fsher formato wth arameter uder ull hyothess (the ull hyothess beg smle ths case.we arrved at ths cocluso by lookg at estmated ad Tye I ad Tye II errors. the cotrary for the Score test we recommed the use of observed fsher formato usg urestrcted MLE. For the flated Posso dstrbuto the smulato results dcate that for Wald test statstc we recommed to use test statstc usg observed fsher formato wth urestrcted MLE of the arameter uder cosderato ad usace arameter. For Score test, we recommed to use the observed fsher formato wth arameter uder ull hyothess ad restrcted MLE. The use of observed ad exected fsher formato wth the substtuto of restrcted ad urestrcted MLE as well as the arameter uder ull hyothess s dscussed Cox ad Hkley (1974 ad Kale(1998.Cox ad Hkley does t rovde ay recommedatos whle Kale refers the

14 use of urestrcted MLE for the Wald test.the reset vestgato justfes the use of observed fsher formato eve the stuatos oe ca derve the exected fsher formato. REFERENCES 1. Neyma, J. ad Pearso, E. (198. The Use ad Iterretato of Certa Test Crtera for Puroses of Statstcal Iferece, Part I. Bometrka, Neyma, J. ad Pearso, E. (1933. the Problem of the most Effcet Tests of Statstcal Hyotheses, Phlosohcal Trasactos of the Royal Socety of Lodo Nelder, J.A ad Wedderbur, R.W.M (197.Geeralzed lear models, J. Roy. Statst. Ser. A Kale, B. K (1998, A frst course o arametrc ferece, Narosa ublcatos. 5. Rao, C. R. (1973, Lear Statstcal Iferece (d ed., New York: Wley. 6. Cox ad Hkley(1974, Theoretcal statstcs, Chama ad Hall 7. Lambert(199. Zero-flated Posso regresso, wth a alcato to defects maufacturg, Techometrcs Bera, A. K., ad Blas. Y. (1. Rao s Score, Neyma s C(α ad Slvey s LM Tests: A Essay o Hstorcal Develomets ad Some New Results, Joural of Statstcal Plag ad Iferece Morga. B. J. T, Palmer. K.J ad Rdout. M. S (7. Negatve Score test statstc, The Amerca Statstca Verbeke, G. ad Moleberghs, G. (7. What Ca Go Wrog Wth the Score Test?, The Amerca Statstca, 61, Freedma. D. A (7. How ca be Score test be Icosstet, The Amerca Statstca

15 Aedx Table.1 Estmated error rates of Wald test for testg Locato arameter Cauchy dstrbuto Samle sze Exected fsher formato observed fsher formato wth ucodtoal MLE of observed fsher formato wth arameter secfed uder ull hyothess Rght taled test Left taled Wald test Two taled Wald test Table. Estmated ower for rght taled Wald test Locato Parameter = =1 W1 W W3 W1 W W

16 Table.3 Estmated error rates of Score test for testg Locato arameter Cauchy dstrbuto Samle sze Exected fsher formato observed fsher formato wth ucodtoal MLE of observed fsher formato wth arameter secfed uder ull hyothess Rght taled Score test Left taled Score test Two taled Score test Table.4 Estmated ower for rght taled Wald test Locato Parameter = =1 S1 S S3 S1 S S

17 Table 3.1 Estmated error rates for Wald test testg λ zero flated Posso dstrbuto Samle sze Exected fsher formato Usg MLE Exected fsher formato Usg arameter uder ull hyothess bserved fsher formato usg arameter uder ull hyothess bserved fsher formato usg MLE Two taled Wald test Left taled Wald test Rght taled Wald test

18 Table 3. Estmated error rates for Score test testg λ zero flated Posso dstrbuto Samle sze Exected fsher formato Usg urestrcted MLE of λ ad restrcted MLE of Exected fsher formato Usg restrcted MLE of λ ad bserved fsher formato usg restrcted MLE of λ ad bserved fsher formato usg urestrcted MLE of λ ad restrcted MLE of Two taled Score test Left taled Score test Rght taled Score test

19 Table 3.3 Estmated ower of two taled Wald test for testg λ of zero flated Posso dstrbuto = =1 λ W11 W1 W13 W14 W11 W1 W13 W Table 3.4 Estmated ower of two taled Wald test for testg λ of zero flated Posso dstrbuto = =1 λ S11 S1 S13 S14 S11 S1 S13 S