Finite-Sample Properties of the Maximum Likelihood Estimator for. the Poisson Regression Model With Random Covariates

Transcription

1 Econometrcs Workn Paper EWP0907 ISSN 85-6 Department of Economcs Fnte-Sample Propertes of the Mamum Lkelhoo Estmator for the Posson Reresson Moel Wth Ranom Covarates Qan Chen School of Publc Fnance an Publc Polcy, Central Unversty of Fnance an Economcs, Bejn, People s Republc of Chna Dav E. Gles Department of Economcs, Unversty of Vctora, Vctora, B.C., Canaa September 009 Abstract: We eamne the small-sample behavour of the mamum lkelhoo estmator for the Posson reresson moel wth ranom covarates. Analytc epressons for the frst-orer bas an secon-orer mean square error for ths estmator are erve, an we unertake some numercal evaluatons to llustrate these results for the snle covarate case. The propertes of the bas-ajuste mamum lkelhoo estmator, constructe by subtractn the estmate frst-orer bas from the ornal estmator, are nvestate n a Monte Carlo eperment. Correctn the estmator for ts frst-orer bas s foun to be effectve n the cases consere, an we recommen ts use when the Posson reresson moel s estmate by mamum lkelhoo wth small samples. Keywors: MSC Coes: Posson reresson moel; bas; mean square error; bas correcton; ranom covarates 6F0; 6J; 6P0 Contact Author: Dav E. Gles, Dept. of Economcs, Unversty of Vctora, P.O. Bo 700, STN CSC, Vctora, B.C., Canaa V8W Y; e-mal: les@uvc.ca; Voce: ; FAX:

2 . Introucton The Posson reresson moel s wely use to stuy count ata n many scplnes, an s enerally ftte usn the mamum lkelhoo estmator MLE. The lkelhoo functon for the Posson moel s strctly concave, an satsfes the usual reularty contons. So, the MLE for the Posson reresson moel has all of the usual esrable asymptotc propertes Gourerou et al., 98. Surprsnly, however, the fnte sample propertes of the MLE for ths moel have been stue only for a lmte number of partcular moels, only for the case of non-ranom covarates, an only usn smulaton methos. Kn 988 an Brännäs 99 use Monte Carlo eperments to eamne the fnte sample propertes of MLE of the Posson reresson moel. The evence prove by the frst of these authors must be treate cautously, as hs Monte Carlo eperment nvolve only 00 replcatons. However, for a moel wth two covarates, Kn 988: 850 reports bases as lare as -.%, 6.% an -.7% for sample szes of n = 0, 50 an 00 respectvely. From more relable eperments, Breslow 990: 568 reports bases n the rane.% to.9% when n = 6, 7; an Brännäs 99: -5 reports bases n the rane -% to % when n = 50. These last two stues nvolve a Posson reresson moel wth a snle covarate. The present paper s the frst to erve analytc epressons for the frst-orer bas an secon-orer mean square error MSE of the MLE for the Posson reresson moel, an the frst to conser these propertes n the contet of ranom covarates. The latter are lkely to arse when the moel s estmate from survey ata, an there s no reason to presume that the relatvely small bases note above wll be applcable n ths case. Now, suppose Y s a seres of count ata, an follows a Posson strbuton wth mean an varance λ: Pr. Y = y = λ y ep λ / y!; y = 0,,, ; =,,, n. The Posson reresson moel arses when we make the mean a non-neatve functon of certan covarates: λ = ep ' β ; =,,,, n.

3 The MLE for the parameter vector n can be erve as the soluton of the follown lo-lkelhoo equatons: an the Hessan matr s: n lo L β / β = y λ = 0 = n lo L β / β β' = λ. = Ths Hessan matr s neatve-efnte for all an β, so there s a unque soluton to the lkelhoo equatons. However, as s non-lnear n the parameters, the MLE has to be solve numercally. Evaluaton of the fnte-sample propertes of the MLE s also complcate by the fact that the estmator cannot be epresse n close form. In ths paper, we apply results from Rlstone et al. RSU 996, as correcte by Rlstone an Ullah 005, to erve analytc epressons for the frst-orer bas an secon-orer MSE functons for the MLE n the Posson reresson moel wth stochastc covarates that follow qute eneral strbutons. We also present some numercal evaluatons of the analytc bas an MSE epressons, an we eplore the effectveness of bas-ajustn the MLE. In the net secton, we apply the methos of RSU to erve the fnte-sample propertes of the MLE for the Posson reresson moel. Secton presents some numercal evaluatons of the rather comple analytc epressons erve n secton, an n secton we scuss the results of a Monte Carlo eperment that focuses on bas-ajustn ths MLE when the moel has a snle covarate. Secton 5 proves our conclusons.. Analytc bas an mean square error epressons RSU 996 prove a eneral framework that allows us to erve the frst-orer bas an secon-orer MSE of a farly we class of nonlnear estmators. There are several well-known methos for eamnn the fnte-sample propertes of statstcs, such as the Eeworth epanson, the bootstrap an the jackknfe methos. Compare wth these methos, that scusse by RSU has some partcular strenths. Frst, t proves us wth

4 analytc, rather than numercal, results. In aton, t s much smpler to erve than the Eeworth epanson, especally for the nonlnear case. RSU s metho focuses on statstcs whch can be epresse as a functon of the ata n the follown way: n ψ n ˆ θ = ˆ θ = 0, 5 n = where ˆ θ s the estmator of nterest; θ = z, θ s a k vector nvolvn the known varables z an the parameters θ ; an E θ = 0 only for the true value of the parameter vector, θ 0. Ths paper apples RSU s metho to erve our analytc results. Recently, corresponn results for the bnary lot moel have been obtane by Chen an Gles 009. For the Posson moel, we set = y λ. So, E = 0 an the law of terate epectatons mples that E = 0. There are certan assumptons about the behavor of θ that are neee n orer for the results of RSU to hol Ullah, 00: : Assumpton The s th orer ervatves of θ est n a nehborhoo of θ 0 an s E θ 0 <, where A = trace / s A A enotes the usual norm of the matr A; an Aθ s the matr of s th orer partal ervatons of the matr A θ wth respect to θ, obtane recursvely. Assumpton For some nehborhoo of θ 0, ψ n θ = O p. Assumpton s s θ θ θ θ0 M for some nehborhoo of 0 0 conton E M C <, =,, K θ, where M satsfes the

5 To smplfy the follown notaton we wll suppress the arument for any functon where ths can be one wthout confuson. So, θ s wrtten as, for eample. The structure of 0 for the Posson reresson moel can easly be shown to satsfy the above three assumptons. We wll use the follown two results, correcte here as ncate by Rlstone an Ullah 005. Lemma Proposton., RSU, 996; Ullah 00: Let Assumptons - hol for some s. Then the bas of θˆ to O n s B ˆ θ = Q V H, 6 n where H j j =, = Q, V =, an = Q. A bar over a functon ncates ts epectaton, so that = E. Lemma Proposton., RSU, 996; Ullah 00: If Assumptons - hol for some s, then the MSE of θˆ to O n s where MSE ˆ θ = Π Π Π Π Π Π, 7 n n n Π Π Π Π = { V H } = Q = Q { V V V V V V }Q { } H Q QH { V V V } H Q Q Q H { V V V }Q { V QV V QV V QV } = Q { V QH V QH V QH } Q

6 5 { } W W W Q { } QV QV QV H Q { } QH QH QH H Q { } QV QV QV H Q { } QH QH QH QH { } 6 H Q 8 an W =. It s realy shown that for the Posson reresson moel we have the follown epressons: ˆ ˆ β β = Ψ n n = λ ; E H = = λ = λ ; E H = = λ = λ ; E H = = λ = = = Q E V λ λ E W = = λ λ = H Q ; Q =. Then, applyn RSU s metho we can erve the follown theorems an corollares. Theorem For the Posson reresson moel, the bas of the MLE to orer n O s ˆ Bas QH vec Q n β =, an the MSE of MLE to orer n O s

7 where Π = Q MSE ˆ β = Π Π Π Π Π Π n n n { λ { λ }} Π = Q E VQXXQ H E vecqxxq XQ { λ } { { λ λ } } Π = Q E VQ X X QV Q QH vecq vecq Q Q Q Q E vec X X vec X X Q Q H Q Π vecqxxq X vecqxxq X } QH vec QX X Q QX X } 6 λλ X H { vec QX X Q X = Q λλ{ QH vec QX X Q QX X QH vec QX X Q QX X QH E Q λλ { vec QXX Q QX X vec QXX Q QX X QHE Q vec QX X Q QX X } When the moel contans only one covarate, the epressons n Theorem smplfy conserably. Corollary reports the eneral result for the one-reressor case, no matter what strbuton the ranom covarate follows. The other corollares report the bas an MSE epressons when the ranom reressor follows a normal, unform or ch-square strbuton. Corollary For the Posson reresson moel, the bas of the MLE to orer O n s E λ Bas ˆ β = n E λ an the MSE of MLE to orer O n s ˆ 9 E λx MSE β = E λ X E λ X n n E λx n E λx E λx Corollary In the Posson reresson moel, when the snle covarate follows a normal strbuton wth mean u an varance σ, the O n orer bas of the MLE s 6

8 μ σ β σ μ σ β Bas ˆ β = n σ μ σ β ep μβ 0.5σ β, an the O n orer MSE s MSE ˆ β = n n σ μ σ β ep μβ 0.5σ β n ep 0.5 σ μ σ β μβ σ β { μ σ β σ μ σ β } ep μβ.5σ β σ 6 σ μ σ β μ σ β 9 σ σ μ σ β μ σ β σ μ σ β 6 Corollary In the Posson reresson moel, when the only covarate follows a unform strbuton on the nterval a, b, the O n orer bas of the MLE s ˆ E Bas β =, n E an the O n orer MSE s λ λ ˆ 9 E λx MSE β = E λ X E λ X n n E λx n E λx E λx where E λ E λ = E λ = b ep bβ a ep aβ bep bβ aep aβ ep bβ ep aβ = β ba β ba β ba b ep bβ a ep aβ β b a E λ β b ep bβ a ep aβ β b a E λ β 7

9 E λ b ep bβ a ep aβ b ep bβ a ep aβ = β b a β b a β β β b ep bβ a ep aβ bep bβ aep aβ ep bβ ep aβ 5 b a b a b a Corollary In the Posson reresson moel, when the only covarate follows a χ strbuton wth r erees of freeom, the O n orer bas of the MLE s r / r β Bas ˆ β =, nr r an the O n orer MSE s r / β MSE ˆ β = n n r r r β r β r 6 n r r β r / 5 r The proofs of Theorem an the varous corollares follow rectly from the results prove by RSU 996, Rlstone an Ullah 005, an Chen an Gles 009. Althouh the recton of the bas of the MLE when the covarate s unformly strbute s ffcult to scern from Corollary, we see mmeately from Corollary that f the reressor s normally strbute then the bas wll be postve f an only f β < μ /σ. From Corollary, f the covarate follows a ch-square strbuton wth r erees of freeom then the bas wll be postve f an only f β > ½ an r s a multple of. Usn the results from Corollares to, two bas-ajuste estmators, βˆ BC an ~ β BC, can be efne as follows: ˆ β ˆ ˆ BC = β Bas β, 9 % β ˆ ˆ BC = β Bas β, 0 8

10 where Bas ˆ β s the bas we erve n the Theorem an the corollares, an Bas ˆ β s the corresponn estmator of the bas epresson. Of course, Bas ˆ β, whch s obtane by substtutn βˆ for β n βˆ BC s an nfeasble estmator, but % β BC s a feasble estmator, an both of these bas-ajuste estmators are unbase to On -. Corresponn to the bas-correcte estmators, we can also obtan the true stanar evaton, s.. βˆ, an the stanar error, s.e. βˆ, to orer O n for the snle covarate case as follows: where MSE βˆ an MSE ˆ β Bas ˆ β. s.. βˆ = MSE ˆ β Bas ˆ β, s.e.βˆ ˆ MSE β Bas ˆ = β, are constructe n a manner smlar to Bas ˆ β an. Numercal evaluatons In ths secton we present some numercal evaluatons usn the analytc results n Corollares to for the one-reressor case. In all of the eperments, the value of β 0 s chosen to control the snal-to-nose ratos to sensble levels, an sample szes of n = 5, 50 an 00 are consere. These numercal evaluatons enable us to check how the fnte sample propertes of the estmator chane as the characterstcs of the moel chane, an they appear n Tables to. Consstent wth Corollares to, we conser a ranom reressor that s ether normally, unformly, or ch-square strbute. We vary the moments of each of these strbutons by chann the parameter values to check the mplcatons for the fnte-sample propertes of the MLE. In all of the cases we have consere, the results show that the bas an MSE epen on the values of the parameters of the reressor strbutons a result that coul be antcpate by fferentatn the epressons n the corollares wth respect to these 9

11 parameters. We note the follown results. No matter whch strbuton the reressor follows an what value β 0 s, the absolute bas an the MSE ecrease as the sample sze ncreases. Ths fnn s further renforce by the results of sample sze of 00, whch are not shown here n orer to conserve space. Ths fnn reflects the mean square consstency of MLE, whch s consstent wth the prevous lterature. Wth respect to the percentae bas, the mantue s qute substantal for certan cases. Amon the cases we consere, when n = 5 an the reressor s normally strbute, the absolute percentae bas can be as lare as 0.8%, epenn on the mean an varance of the ata. Further, for ths sample sze the reporte absolute percentae bases are as lare as 57% for a unformly strbute covarate, an 7.9% for a ch-square strbute covarate, aan epenn on the moments of the reressor s strbuton. It wll be recalle that these absolute bases are conserably reater than those foun by Breslow 990 an Brännäs 99 for moels wth non-ranom covarates.. Bas-correcte estmaton We have also conucte a Monte Carlo eperment to evaluate the relatve performances of the feasble bas-correcte MLE, nfeasble bas-correcte MLE an the MLE tself, an also the performance of s.e. ˆβ as an estmator of s.. ˆβ. Table shows the results of our eperment usn 000 replcatons. Of course, the feasble bas-correcte estmator s the one that we are most ntereste n. In all the cases we have consere, the feasble bas-correcte MLE mproves the performance of the ornal MLE substantally. For eample, when the reressor follows a ch-square strbuton, n = 5 an β 0 = -0., the absolute percentae bas of the feasble bas-correcte MLE s 0.9%, compare wth 0.5% for the MLE. Smlarly, when the reressor follows a unform strbuton, n = 5 an β 0 = -.7, the absolute percentae bas of the feasble bas-correcte MLE s.0%, compare wth.8% for the MLE.Interestnly, there are nelble fferences between the performances of the feasble an nfeasble bas-correcte MLEs. 0

12 Our Monte Carlo eperment also proves nshts nto the performance of the s.e. βˆ as an estmator of the s.. βˆ. The s.e. βˆ tens to overestmate the s.. βˆ n all but three of the cases consere. The larest bas of s.e. βˆ as an estmator of s.. βˆ amon all of the cases tabulate s.77%, whch occurs when the reressor follows a ch-square strbuton, n = 5 an β 0 = Ths bas ecreases to.5%, when the sample sze ncreases to 00. For all the cases we consere, the s.e. βˆ converes to the s.. βˆ an both ecrease monotoncally n value as the sample sze ncreases, as epecte from the consstency of the MLE. 5. Conclun remarks In ths paper, we apply technques evelope by Rlstone et al. 996 to erve analytc epressons for the frst-orer bas an secon-orer mean square error of the mamum lkelhoo estmator for the Posson reresson moel wth ranom reressors. Our stuy s the frst to nvestate the fnte-sample propertes of ths estmator when ths moel has ranom covarates, an the frst to obtan analytc rather than smulaton-base results for the bas an mean square error. Not surprsnly, we fn that the bas of the mamum lkelhoo estmator when the covarates are ranom s reater than that obtane from Monte Carlo eperments by other authors for the non-ranom reressor case. The mantue of the bas an mean square error can be substantal, an ths motvates us to conser bas-ajustn the mamum lkelhoo estmator. Usn a Monte Carlo eperment we are able to confrm that our bas-correcte estmator can substantally reuce the bas of the mamum lkelhoo estmator, an we recommen usn ths bas-ajuste estmator when the sample sze s less than 00.

13 Table : On - Bas an On - MSE of MLE. Normal μ, σ Reressors Normal 0, Normal 5, Normal 0, Normal 0, Normal 0, n = 5 β 0 Bas MSE Bas MSE Bas MSE Bas MSE Bas MSE n = 50 β 0 Bas MSE Bas MSE Bas MSE Bas MSE Bas MSE

14 Table : On - Bas an On - MSE of MLE. Unform a, b Reressors U 0, U 0, 5 U 0, 0 U -5, U -0, n = 5 β 0 Bas MSE Bas MSE Bas MSE Bas MSE Bas MSE n = 50 β 0 Bas MSE Bas MSE Bas MSE Bas MSE Bas MSE

15 Table : On - Bas an On - MSE of MLE. Ch-square.f. reressors Ch-square Ch-square 6 Ch-square 8 n = 5 β 0 Bas MSE Bas MSE Bas MSE n = 50 β 0 Bas MSE Bas MSE Bas MSE

16 Table : Averae of,000 Monte Carlo smulaton replcatons Normal 0, Ch-square β 0 n βˆ ~ β BC βˆ BC s.e. ˆ β s.. ˆ β β 0 n βˆ ~ β BC βˆ s.e. ˆ β s.. ˆ β BC β 0 n βˆ ~ β BC Unform 0, βˆ BC s.e. ˆ β s.. ˆ β β 0 n βˆ ~ β BC βˆ s.e. ˆ β s.. ˆ β BC

17 References Brännäs, K. 99. Fnte sample propertes of estmators an tests n Posson reresson moels. Journal of Statstcal Computaton an Smulaton : 9-. Breslow, N Tests of hypotheses n over-sperse Posson reresson an other quas-lkelhoo moels. Journal of the Amercan Statstcal Assocaton 85: Cameron, A. C., Trve, P. K Econometrc moels base on count ata: comparsons an applcatons of some estmators an tests. Journal of Apple Econometrcs : 9-5. Chen, Q., Gles, D. E The fnte sample propertes of the mamum lkelhoo estmator for the bnary lot moel wth ranom covarates. Unversty of Vctora, Econometrcs Workn Paper EWP0906. Gourerou, C., Monfort, A., Tronon, A. 98. Pseuo mamum lkelhoo methos : Applcatons to Posson moels. Econometrca 5: Kn, G Statstcal moels for poltcal scence event counts: bas n conventonal proceures an evence for the eponental Posson moel. Amercan Journal of Poltcal Scence : R 008 The R Project for Statstcal Computn, Rlstone, P., Srvatsava, V. K., Ullah, A The secon orer bas an MSE of nonlnear estmators. Journal of Econometrcs 75: Rlstone, P., Ullah, A. 00. Sampln bas n Heckman s sample selecton estmator. In: Chaubey, Y. P. e. Recent Avances n Statstcal Methos. Hackensack NJ: Worl Scentfc. Rlstone, P., Ullah, A Correnum to: The secon orer bas an mean square error of non-lnear estmators. Journal of Econometrcs : 0-0. Ullah, A., 00. Fnte Sample Econometrcs. Ofor: Ofor Unversty Press. 6