New Lu Estmators for the Posson Regresson Moel: Metho an Applcaton By Krstofer Månsson B. M. Golam Kbra, Pär Sölaner an Ghaz Shukur,3 Department of Economcs, Fnance an Statstcs, Jönköpng Unversty Jönköpng, Sween Department of Mathematcs an Statstcs, Flora Internatonal Unversty, Mam, Flora, USA 3 Department of Economcs an Statstcs, Lnnaeus Unversty, Växö Sween. Abstract A new shrnkage estmator for the Posson moel s ntrouce n ths paper. Ths metho s a generalzaton of the Lu (993) estmator orgnally evelope for the lnear regresson moel an wll be generalse here to be use nstea of the classcal maxmum lkelhoo () metho n the presence of multcollnearty snce the mean square error (MSE) of becomes nflate n that stuaton. Furthermore, ths paper erves the optmal value of the shrnkage parameter an base on ths value some methos of how the shrnkage parameter shoul be estmate are suggeste. Usng Monte Carlo smulaton where the MSE an mean absolute error (MAE) are calculate t s shown that when the Lu estmator s apple wth these propose estmators of the shrnkage parameter t always outperforms the. Fnally, an emprcal applcaton has been consere to llustrate the usefulness of the new Lu estmators. Key wors: Estmaton; MSE; MAE; Multcollnearty; Posson; Lu; Smulaton. AMS Subect classfcaton: Prmary 6J07, Seconary 6F0.
. Introucton In the fel of health, socal, economcs an physcal scences, the epenent varable often comes n the form of a non-negatve ntegers or counts. In that stuaton one often apply the Posson regresson moel whch s usually estmate by maxmum lkelhoo () where the soluton to a non-lnear equaton s foun by applyng teratve weghte least square (IWLS). Ths metho has been shown n Månsson an Shukur (0a,b) to be senstve to multcollnearty an t becomes ffcult to make a val statstcal nference snce the mean square error (MSE) becomes nflate. In those papers, a rge regresson estmator (RRE) was presente whch was a generalzaton of that propose for lnear regresson by Hoerl an Kennar (970). In both papers t was shown that the RRE outperforme the. The RRE s effectve but as Lu (993) ponte out t has the savantage that the estmate parameters are complcate non-lnear functons of the rge parameter k. Therefore, n ths paper another shrnkage estmator for the Posson moel wll be propose whch s a generalzaton of the metho propose for lnear regresson by Lu (993). The avantage of ths metho s that the estmators are a lnear functon of the shrnkage parameter. For ths reason, ths shrnkage estmator has become more popular urng recent years (see for examples, Akenez an Kacranlar (995), Kacranlar (003) an Alheety an Kbra (009) among others). The purpose of ths paper s to solve the problem of an nflate MSE of the estmator by applyng a Lu estmator. Furthermore, we erve the optmal value of the shrnkage parameter an base on ths value we suggest some methos of how the shrnkage parameter shoul be estmate. In a Monte Carlo stuy we evaluate the performance of the an the Lu estmator apple wth the suggeste estmators of the shrnkage parameter. The performance crtera use n the smulaton stuy s the MSE an mean absolute error (MAE). In our smulaton, factors nclung the egree of correlaton, the sample sze an the number of explanatory varables are vare. Fnally, an emprcal example has been consere to llustrate the beneft of the Lu estmator. In ths applcaton, the effect of the usage of cars an trucks on the number of klle peestrans n fferent countes n Sween s nvestgate
Ths paper s organze as follows: In Secton, the statstcal methoology s escrbe. In Secton 3, the esgn of the Monte Carlo experment s presente an the result from the smulaton stuy s scusse. An applcaton s presente n Secton 4. Fnally, a bref summary an conclusons s gven n secton 5.. Methoology. Posson regresson The Posson regresson moel s a benchmark moel when the epenent varable ( y ) comes n the form of counts ata an strbute as Po, where exp x, x s the th row of X whch s a n p ata matrx wth p explanatory varables an s a p vector of coeffcents. The log lkelhoo of ths moel may be wrtten as: n n l ; y exp x y log exp x log y! n. (.) The most common metho to maxmze the lkelhoo functon s to apply the IWLS algorthm: - X ' WX X ' Wz, (.) where W ag an ẑ s a vector where the th element equals The MSE of ths estmator equals: where y z log. J - E L ' E tr X ' WX, (.3) s the th egenvalue of the X ' WX matrx. When the explanatory varables are hghly correlate the weghte matrx of cross-proucts, X ' WX, s ll-contone whch leas to nstablty an hgh varance of the estmator. In that stuaton t s very har to nterpret the estmate parameters snce the vector of estmate coeffcents s on average too long. By notng that the IWLS algorthm approxmately mnmzes the weghte sum of square error (WSSE) one may apply a generalzaton of the Lu (993) estmator for lnear regresson nstea: - X ' WX I X ' WX I (.4) 3
For ths estmator we have replace the matrx of cross-proucts use n the Lu (993) estmator wth the weghte matrx of cross-proucts an the ornary least square estmator (OLS) of wth the estmator. The MSE of the Lu estmator equals: ' MSE E L E E ' Z ' Z Z ' Z tr Z Z k X WX ki ' ' ' ' J J - (.5) where s efne as the th element of an s the egenvector efne such that X ' WX ', where equals ag. In orer to show that there exst a value of boune between zero an one so that MSE MSE, we start by takng the frst ervatve of equaton (.5) wth respect to : g ' (.6) J J an then by nsertng the value one n equaton (.6) we get: g' J (.7) whch s greater than zero snce 0. Hence, there exsts a value of that les between zero an one so that MSE MSE. Furthermore, the optmal value of any nvual parameter shown that can be foun by settng equaton (.6) to zero an solve for. Then t may be, (.8) correspons to the optmal value of the shrnkage parameter. Hence, the optmal value of s negatve when s less than one an postve when t s greater than one. However, ust as n Lu (993) the shrnkage parameter wll be lmte to take on values only between zero an one. 4
. Estmatng the shrnkage parameter The value of may only take on values between zero an one an there oes not exst a efnte rule of how to estmate t. However, n ths paper some methos wll be propose that are base on the work for lnear rge regresson by for nstance Hoerl an Kennar (970), Kbra (003) an Khalaf an Shukur (005). As n those papers, the shrnkage parameter, wll be estmate by a sngle value. The frst estmator s the followng:, max D max 0,, max max where we efne max an max to be the maxmum element of an X ' WX, respectvely. Replacng the values of the unknown parameters wth the maxmum value of the unbase estmators s an ea taken from Hoerl an Kennar (970). However, for the Lu estmator another maxmum operator s also use that wll ensure that the estmate value of the shrnkage parameter s not negatve. Furthermore, the followng estmators wll be use: D max 0, mean D3 max 0, p J D4 max 0, max D 5 max 0, mn. Usng the average value an the mean s very common when estmatng the shrnkage parameter for rge parameter an the D an D3 estmators has rect counterparts n equaton (3) an (5) n Kbra (003). Usng other quantles such as the maxmum value, was successfully apple n Khalaf an Shukur (005) an the ea behn the D4 an D5 estmator are taken from those papers. 5
.3 Jugng the performance of the estmators To nvestgate the performance of the Lu an the methos, we calculate the MSE usng the followng equaton: MSE R R ', (.9) an the MAE as: R MAE R (.0) where s the estmator of obtane from ether or Lu an R equals 000 whch correspons to the number of replcates use n the Monte Carlo smulaton. 3. The Monte Carlo smulaton Ths secton conssts of a bref escrpton of how the ata s generate together wth a scusson of our fnngs. 3. The Desgn of the Experment The epenent varable of the Posson regresson moel s generate usng pseuo-ranom numbers from the Po exp strbuton where 0 p p x x,,,... n,,,... p. (3.) The parameter values n equaton (3.) are chosen so that p an p. To be able to generate ata wth fferent egrees of correlaton we use the followng formula to obtan the regressors: / p x z z,,,... n,,,... p (3.) where z are pseuo-ranom numbers generate usng the stanar normal strbuton an represents the egree of correlaton (see, Kbra 003 an Munz an Kbra 009 among others). In the esgn of the experment three fferent values of corresponng to 0.85, 0.95 an 0.99 are consere. To reuce eventual start-up value effects we scar the frst 00 observatons. 6
In the esgn of the experment the factors n an p are also vare. Snce the estmators are consstent, ncreasng the sample sze s assume to lower MSE an MAE whle p s assume to ncreases the nstablty of X ' WX an lea to an ncrease of both measures of performance. We use sample szes corresponng to 5, 0, 30, 50 an 00 egrees of freeoms (f=n-p) an number of regressors p equals to an 4. 3. Results Dscusson The estmate MSE an MAE for p= an 4 are presente n Tables an respectvely. It s event from these tables that the egree of correlaton an the number of explanatory varables nflate both the MSE an MAE whle ncreasng the sample sze leas to a ecrease of both measures of performance. We can also see that the MSE ncreases more when conserng the MSE nstea of the MAE crtera. Hence, the gan of applyng Lu s larger n terms of MSE than MAE. Furthermore, when lookng at both measures of performance we can see that the estmator D5 s always ether the shrnakge parameter that mnmzes the MSE an MAE or t s very close to the shrnakge parameter that mnmzes these loss functons. 7
Table : Estmate MSE an MAE of the estmators when p= Estmate MSE Estmate MAE D D D3 D4 D5 D D D3 D4 D5 =0.85 50 0.54 0.44 0.6 0.6 0.5 0.0 0.74 0.57 0.499 0.499 0.50 0.495 75 0.350 0.98 0.83 0.83 0.0 0.79 0.66 0.478 0.468 0.468 0.479 0.465 00 0.8 0.5 0. 0. 0.5 0. 0.459 0.387 0.385 0.385 0.387 0.385 50 0.098 0.080 0.080 0.080 0.080 0.080 0.339 0.308 0.308 0.308 0.308 0.308 00 0.038 0.035 0.035 0.035 0.035 0.035 0.5 0.07 0.07 0.07 0.07 0.07 =0.95 50.755 0.58 0.44 0.44 0.656 0.375.39 0.730 0.655 0.655 0.76 0.6 75.044 0.379 0.30 0.30 0.396 0.79.04 0.633 0.589 0.589 0.644 0.575 00 0.6 0.73 0.50 0.50 0.79 0.39 0.847 0.55 0.538 0.538 0.556 0.53 50 0.34 0.86 0.83 0.83 0.87 0.8 0.65 0.477 0.474 0.474 0.477 0.473 00 0.35 0.03 0.03 0.03 0.03 0.03 0.409 0.359 0.359 0.359 0.359 0.359 =0.99 50 0.49 3.366 3.77 3.77 4.887.453 3.345.439.305.305.748.094 75 6.396.9.764.764.679.356.685.50.009.009.338 0.887 00 3.443.00 0.809 0.809. 0.670.00 0.870 0.770 0.770 0.95 0.7 50.767 0.54 0.445 0.445 0.58 0.44.473 0.78 0.667 0.667 0.74 0.648 00 0.774 0.30 0.98 0.98 0.38 0.90 0.984 0.597 0.590 0.590 0.60 0.585 Table : Estmate MSE an MAE of the estmators when p=4 Estmate MSE Estmate MAE D D D3 D4 D5 D D D3 D4 D5 =0.85 50.80 0.86 0.645 0.57.079 0.465.846.60.09.08.34.049 75 0.96 0.483 0.386 0.374 0.533 0.366.383.05 0.956 0.948.05 0.943 00 0.44 0.86 0.68 0.68 0.90 0.67 0.978 0.85 0.800 0.800 0.88 0.800 50 0.94 0.58 0.57 0.57 0.58 0.57 0.67 0.63 0.6 0.6 0.63 0.6 00 0.07 0.067 0.067 0.067 0.067 0.067 0.47 0.404 0.404 0.404 0.404 0.404 =0.95 50 5.658.9.67.3 3.59 0.695 3.53.878.5.395.60.5 75 3.65.34 0.877 0.753.780 0.606.57.607.337.76.786.07 00.53 0.756 0.544 0.50 0.83 0.50.88.70.33.9.3.09 50 0.68 0.387 0.349 0.349 0.394 0.348.87 0.949 0.90 0.90 0.953 0.90 00 0.6 0.8 0.80 0.80 0.8 0.80 0.733 0.66 0.66 0.66 0.66 0.66 =0.99 50 7.75 9.73 8.94 7.80 0.554.6 7.39 3.530 3.05.80 5.577.553 75 7.44 6.077 4.95 3.854.940.44 5.880.988.446.68 4.74.390 00 7.538.63.740.370 4.30 0.750 4.050.73.699.555.75.79 50 3.47.435 0.870 0.730.795 0.66.79.687.36.97.85.36 00.3 0.693 0.50 0.50 0.78 0.508.739.4.7..63. 8
4. Emprcal Applcaton To llustrate the performance of fferent estmators we conser ata that are taken from the Department of Transport Analyss n Sween. The ata s for fferent countes (the total number s ) n Sween for the year 00. The number of peestrans were klle s use as a epenent varable an the number of klometers rven by cars (x ) an trucks (x ) respectvely are consere as nepenent varables. A lkelhoo rato test has been use to etermne whether the epenent varable follows a Posson strbuton or not. We foun that the test statstc cannot be reecte at the one percent level of sgnfcance. Furthermore, the bvarate correlaton between the two regressors s 0.9, so there s a problem of multcollnearty. The propose fferent Lu estmators are estmate usng IWLS algorthms n R. Furthermore, bootstrap technque was apple n orer to calculate the stanar errors of the estmate parameters. The results are presente n Table 3. From ths table t s clear that the boostratppe stanar errors are the hghest for an the lowest for D5. Ths supporte the smulaton results n secton 3. Moreover, we can also see a postve relatonshp between the explanatory varables an the number of peestrans klle. Ths s expecte snce both regressors show the usage of cars an trucks respectvely whch are suppose to ncrease the number of klle peestrans. But ue to the multcollnearty problem the vector of coeffcents becomes too long an the reuslt obtane from the estmaton metho exagerates ths postve effect. Instee, we shoul look at the much lower estmate values of the coeffcents obtane from D5 snce the smulatons shows that ths estmator has the lowest estmate MSE an the emprcal applcaton shows that t has the lowest bootstrappe stanar errors. Table 3: The estmate parameters an the stanar errors of the fferent estmators D D D3 D4 D5 x x x x x x x x x x x x 3.46 0.37.886 5.349.994 5.683.994 5.683 3.068 9.08 0.99.347 (5.75) (8.79) (0.45) (6.03) (9.7) (5.84) (9.7) (5.84) (3.66) (7.45) (7.6) (4.85) Note: The stanar errors are n parenthess. The ata s publcally avalable on the webpage of the Department of Transport Analyss, www.trafa.se. The ata s avalable from the authors upon request. 9
5. Conclusons In ths paper, the shrnkage estmator evelope by Lu (993) for the lnear regresson moel has been extene for the Posson regresson moel. Ths estmator s propose n orer to reuce the nflaton of the varance of the estmator cause by multcollnearty. The Lu an the estmators are evaluate by means of Monte Carlo smulatons. Both MSE an MAE are use as a performance crtera an factors nclung the egree of correlaton, the sample sze an the number of explanatory varables are vare. Both measures of performance show that the propose Lu estmators are better than n the sense of smaller MSE an MAE. We also observe that the estmator D5 s often the shrnkage parameter that mnmzes the estmates MSE an MAE. The beneft of propose Lu estmators s shown by an example. Both the results from the smulaton stuy an the emprcal applcaton shows that the propose D5 shoul be the estmator of the Lu parameter to recommen for practtoners. Acknowlegements Ths paper was wrtten whle Dr. Kbra was vstng Professor Ghaz Shukur n the Department of Economcs an Statstcs, Lnnaeus Unversty, Växö, Sween urng May- June 0. He acknowleges the excellent research support prove by Lnnaeus Unversty, Växö, Sween. 0
References Akenz, F. an S. Kacranlar (995). On the almost unbase generalze Lu estmator an unbase estmaton of the bas an MSE. Communcatons n Statstcs- Theory an Methos, 4, 789-797. Alheety, M. I. an B. M. G. Kbra (009). On the Lu an almost unbase Lu estmators n the presence of multcollnearty wth heterosceastc or correlate errors. Surveys n Mathematcs an ts Applcatons, 4, 55-67. Alkhams, M. an G. Shukur (007). A Monte-Carlo Stuy of Recent Rge Parameters. Communcatons n Statstcs- Theory an Methos, 36, 535-547. Hoerl, A.E. an R.W. Kennar (970). Rge regresson: base estmaton for non- orthogonal Problems. Technometrcs,, 55-67. Lu, K. (993). A new class of base estmate n lnear regresson. Communcatons n Statstcs-Theory an Methos,, 393-40. Kacranlar, S. (003). Lu estmator n the general lnear regresson moel. Journal of Apple Statstcal Scence, 3, 9-34. Khalaf, G. an Shukur, G. (005). Choosng Rge Parameter for Regresson Problems. Communcatons n Statstcs, Theory an Methos, 34, Issue 5, 77-8. Kbra B.M.G. (003). Performance of some new rge regresson estmators Communcatons n Statstcs, Theory an Methos, 3, 49-435. Munz, G. an Kbra, B. M. G. (009). On some rge regresson estmators: An Emprcal Comparson. Communcatons n Statstcs-Smulaton an Computaton 38:6-630. Månsson, K. an Shukur, G. (0a). A Posson Rge Regresson estmator. to appear n Economc Moelng. Månsson, K. an Shukur, G. (0b). On Rge Parameters n Logstc Regresson, to appear n Communcatons n Statstcs, Theory an Methos.