Liu-type Negative Binomial Regression: A Comparison of Recent Estimators and Applications

Transcription

1 Lu-type Negatve Bnomal Regresson: A Comparson of Recent Estmators and Applcatons Yasn Asar Department of Mathematcs-Computer Scences, Necmettn Erbaan Unversty, Konya 4090, Turey, yasar@onya.edu.tr, yasnasar@hotmal.com Abstract Ths paper ntroduces a new based estmator for the negatve bnomal regresson model that s a generalzaton of Lu-type estmator proposed for the lnear model n []. Snce the varance of the maxmum lelhood estmator (MLE) s nflated when there s multcollnearty between the explanatory varables, a new based estmator s proposed to solve the problem and decrease the varance of MLE n order to mae stable nferences. Moreover, we obtan some theoretcal comparsons between the new estmator and some others va matrx mean squared error (MMSE) crteron. Furthermore, a Monte Carlo smulaton study s desgned to evaluate performances of the estmators n the sense of mean squared error. Fnally, a real data applcaton s used to llustrate the benefts of new estmator. Keywords: Negatve bnomal regresson, Lu-type estmator, multcollnearty, MSE, MLE Mathematcs Subects Classfcaton: Prmary 6J07; Secondary 6J0

2 . Introducton In real lfe contexts, the observatons are not ndependent and dentcally dstrbuted (d) all the tme. The data often comes n the form of non-negatve ntegers or counts whch are not d. The man nterest of a researcher may depend on the covarates whch are assumed to affect the parameters of the condtonal dstrbuton of events, gven the covarates. Ths s generally acheved by a regresson model of count [3]. Thus, count regresson models such as Posson regresson or negatve bnomal (NB) regresson are mostly used n the feld of health, socal, economc and physcal scences such that the nonnegatve and nteger-valued aspect of the outcome plays an mportant role n the analyss. Although twenty two dfferent versons of NB model are mentoned n [5], the tradtonal NB model whch was symbolzed as NB n [] s the man topc of ths paper. Ths model s more useful than Posson regresson model snce NB allows for random varaton n the Posson condtonal mean, h, by lettng h z n where exp x such that x s the th row of the data matrx X of order p wth p explanatory varables, s the coeffcent vector of order p and z s a random varable followng the gamma dstrbuton such that z ~, wth ntercept,,,..., n. The densty functon of the dependent varable y s gven by y y y Pr y y x where the overdsperson parameter s gven as /. The condtonal mean and varance of the dstrbuton are gven respectvely as follows: E y x, Cov y x.

3 The estmaton of the coeffcent vector s usually obtaned by maxmzng the followng loglelhood functon n y L y y y y 0, log log! log log log (.) snce y y log log 0. The estmaton of the parameter s usually obtaned by the method of maxmum lelhood estmaton (MLE) whch can be obtaned by maxmzng the equaton (.) wth respect to, namely, solvng the followng equaton S, n y L x 0. (.) Snce the equaton (.) s non-lnear n, one should use the followng scorng method r r r r I S (.3) where S r s the frst dervatve of the log-lelhood functon evaluated at r and r r L X ; I E X W X, W r obtaned as follows: dag r r evaluated at r. In the fnal step of the algorthm, MLE of s ˆ ˆ ˆ ˆ MLE X WX X WZ (.4) 3

4 where Ẑ s a vector wth the th element equals to log ˆ W r and y ˆ, W ˆ and ˆ are the values of ˆ r at the fnal step respectvely, the hats show the teratve nature of the algorthm. Ths method s also nown as the teratvely re-weghted least squares algorthm (IRLS). However, when the matrx X WX ˆ s ll-condtoned,.e., the correlaton between the explanatory varables are hgh, MLE becomes nstable and ts varance s nflated. Ths problem s called multcollnearty. Although, applyng shrnage estmators s very popular n lnear model to solve the multcollnearty problem (see [7], [9], [], [0] etc.), count models have not been nvestgated n the presence of multcollnearty. Therefore, as an excepton, [3] proposed to use the rdge regresson [6] n negatve bnomal regresson models. The negatve bnomal rdge regresson estmator (RR) s obtaned as follows: where I s the p p ˆ ˆ ˆ X WX I X WZ ˆ, 0 (.5) dentty matrx. The author proposed to use some exstng rdge estmators to estmate the rdge parameter. Moreover, Lu estmator [] s generalzed to the negatve bnomal regresson model n [4] and obtan the followng negatve bnomal Lu estmator (LE) ˆ d X WX ˆ I X WX ˆ di ˆ, 0 d. (.6) MLE Fnally, motvated by the dea that combnng the two estmators mght nhert the advantages of both estmators, a two-parameter estmator whch s a combnaton of RR and LE has been proposed n [8]. 4

5 The purpose of ths paper s to generalze Lu-type estmator [] to the negatve bnomal regresson and dscuss some propertes of the new estmator. The organzaton of the paper s as follows: In secton, Lu-type negatve bnomal estmator () s proposed, matrx mean squared error (MMSE) and mean squared error (MSE) propertes are nvestgated and selecton of the shrnage parameters s dscussed. In order to compare the performances of the estmators MLE, RR, LE and, a Monte Carlo smulaton s desgned and ts results are dscussed n secton 3. A real data applcaton s demonstrated to llustrate the benefts of n secton 4. Fnally, a bref summary and concluson are provded... Constructon of. New Estmator and MSE Propertes Consder the lnear regresson model Y X where X s an n p data matrx, s the p coeffcent vector, s the n random error vector satsfyng ~ N 0, and Y s the n dependent varable. When there s multcollnearty, the matrx X X becomes ll-condtoned and some of the egenvalues of the matrx X X becomes close to zero and the condton number / becomes very hgh such that,,,..., p are the egenvalues of X X. Thus, the ordnary least square estmator (OLS), ˆOLS X X X Y becomes nstable. Therefore, [6] proposed rdge estmator max mn ˆRdge X X I X Y whch s obtaned by augmentng / 0 to the orgnal equaton. However, large values of maes the dstance between / and 0 ncrease and to control the condton number, one should use large values of whch mposes more bas to the rdge estmator. / / Therefore, Lu [] proposed to augment d / ˆ OLS to the orgnal equaton and obtan Lu-type estmator ˆ ˆ where 0, d and ˆ s any, d X X I X X di 5

6 estmator. The results showed that ˆ, d has a better performance than OLS and rdge estmator n the sense of MSE. Therefore, t s a good dea to defne a generalzaton of Lu-type estmator n negatve bnomal model to solve the problem of multcollnearty. In ths study, a generalzaton of Lu-type estmator to the negatve bnomal regresson model s proposed as follows ˆ X WX ˆ I X WX ˆ di ˆ (.) MLE 0, d. From the defnton of, t s easy to see that s a general estmator ncludng MLE, RR and LE as follows, lm ˆ ˆ MLE, d ˆ ˆ lm d 0 whch s the negatve bnomal rdge estmator (RR), ˆ lm LE, d ˆ whch s the negatve bnomal Lu estmator (LE). In order to see the superorty of the estmator, MMSE contanng all the relevant nformaton regardng the estmators can be used as a comparson crteron. MMSE and MSE beng the trace of MMSE of an estmator are respectvely defned by MMSE E Bas Bas var, MSE tr MMSE E Bas Bas tr var. (.) (.3) Thus MSE and MMSE of MLE are gven by the followng equatons respectvely 6

7 p, (.4) tr X WX MSE MLE MMSE MLE X WX (.5) where s the th egenvalue of the matrx X WX. There s a need to mae a transformaton n order to present the explct form of the MMSE and and Q where... p 0 MSE functons. Let Q X WXQ dag,,..., p and Q s the matrx whose columns are the egenvectors of the matrx X WX. functons easly. Now, we obtan the bas and varance functons of the estmators to compute the MMSE and MSE The bas, varance, MMSE and MSE functons of, RR, and LE are obtaned respectvely as follows: b d Q bas, * * var Q d d Q, MMSE Q Q d Q Q, MSE * * d d RR p d d, (.6) b RR Q bas, var RR Q Q, MMSE RR Q Q Q Q, 7

8 MSE RR p b bas LE d Q, LE var LE Q Q, d d, MMSE LE Q Q d Q Q MSE LE d d p d d, where I, di, I and d di. * d After computng the MMSE and MSE functons, s compared to the other estmators n the sense of MMSE n the followng theorems by usng the followng lemma: Lemma. [4]: Let M be a postve defnte (p.d.) matrx, be a vector of nonzero constants and c be a postve constant. Then cm 0 f and only f M c... Comparson of versus MLE The followng theorem presents the condton that s superor to MLE: Theorem.: Let d d MMSE MLE 0,,,..., p MMSE 0 ff b and b bas ˆ * * d d b. Proof: The dfference between the MMSE functons of MLE and s obtaned by. Then 8

9 * * MMSE MLE MMSE Q Q b b d d p d Q dag Q b. b (.7) The matrx * * d d s p.d. f d 0 whch s equvalent to d d 0 d d 0. The proof s fnshed by Lemma... Smplfyng the last nequalty, one gets.3. Comparson of versus RR The followng theorem gves the condton that s superor to RR: Theorem.3: Let and d mn b Q. If * * d d b d d b, then MMSE RR MMSE 0. Proof: The dfference between the MMSE functons of RR and s obtaned by * * MMSE RR MMSE Q Q b b b b d d RR RR p d Qdag Q d d bb p d d dag. Q Q d d bb (.8) 9

10 Snce d d bb s nonnegatve defnte, t s enough to prove that * * d d s p.d. Now let d mn Q Q d d bb Lemma., the proof s fnshed., then usng.4. Comparson of versus LE The followng theorem presents the condton that s superor to LE: Theorem.4: If b * * d d d d b and 0, 0, then MMSE LE MMSE 0 d d d. Proof: The dfference between the MMSE functons of LE and s obtaned by * * MMSE LE MMSE Q Q b b b b d d d d LE LE p d d Qdag Q b LEb LE bb p d d Qdag Q b. LEb LE bb (.9) Smlarly, snce bleb LE s nonnegatve defnte, t s enough to prove that * * d d d d Q Q b b s postve defnte. Lettng d d 0, 0 d, t s easy to see that Lemma. leads to the desred result..5. Estmatng the parameters and d The selecton of shrnage parameters n based estmators has always been an mportant ssue. There are varous numbers of papers suggestng dfferent types of estmaton technques of the rdge 0

11 parameter, Lu parameter and so on. In ths study, motvated by the wor of [6], [9], [5], some methods to select the values of the parameters and d are proposed. Followng [6], dfferentatng the equaton (.6) wth respect to the parameter, t s easy to obtan the followng equaton: p MSE ˆ d d ˆ d ˆ. (.0) 4 4 Smplfyng the numerator of the above equaton and solvng for, one can get the followng ndvdual estmators ˆ ˆ d,,,...,p. (.) ˆ The condton ˆ d restrcton 0 should hold to get a postve value of ˆ. Thus, the followng d ˆ (.) should be satsfed. s proposed: Now, to estmate the parameter, followng [9] and usng mean functon, the followng method ˆ AM p ˆ d (.3) p ˆ whch s the arthmetc mean of ˆ.

12 Moreover, followng [], the maxmum functon s used to obtan the followng estmator: ˆ ˆ d MAX max. ˆ (.4) After choosng the parameter d usng the equaton (.), one can estmate the value of usng one of the methods proposed. By pluggng-n these estmates n, a better performance may be observed. In the followng secton, a Monte Carlo smulaton s desgned to compare the performance of the estmators for dfferent scenaros. To estmate the rdge parameter to be used n RR, Månsson [3] proposed dfferent methods. In ths study, K5 max ˆ ˆ wll be used n the smulaton snce the author reported that K 5 s the best estmator n most of the stuatons nvestgated. Moreover, some methods were proposed n [4] to choose the shrnage parameter d to be used n LE. However, ˆ D5 max 0,mn / ˆ had the lowest MSE value n most of the stuatons. Therefore D 5 s used to estmate d n LE n the smulaton study. 3.. Desgn of the Smulaton 3. Monte Carlo Smulaton Study In the prevous secton, some theoretcal comparsons are provded. In ths secton, an extensve Monte Carlo smulaton study s desgned to evaluate the performances of the estmators. Here s the descrpton of the smulaton. Frstly, the observatons of the explanatory varables are generated usng the followng equaton

13 / x z z (3.) p where,,, n,,,... p, and represents the correlaton between the explanatory varables and z s are ndependent random numbers obtaned from the standard normal dstrbuton. The dependent varable of the NB regresson model s generated usng random numbers followng the negatve bnomal dstrbuton NB, where exp x,,,..., n. The slope parameters are decded such that p, whch s a commonly used restrcton n the feld (see [9]). In the desgn of smulaton, three dfferent values of correspondng to 0.90, 0.95, 0.99 are consdered. The value of s taen to be.0 and.0 due to [3]. Moreover, the followng small, moderate and large sample sze values are consdered: 50, 00 and 00. The numbers of explanatory varables are taen to be 4 and 6. The smulaton s repeated 000 tmes, convergence tolerance s taen to be and the estmated MSE values of the estmators are computed as follows: MSE 000 r r r, (3.) 000 where r s an estmator of at the rth replcaton. 3.. Results of the Smulaton The estmated MSE values obtaned from the Monte Carlo smulaton are presented n Tables -. It s observed from tables that the factors affectng the performance of the estmators are the value of, 3

14 the sample sze n, the number of explanatory varables p and the degree of correlaton between the explanatory varables. Accordng to the tables, ncreasng the value of maes an ncrease n the estmated MSE values. As the degree of correlaton ncreases, MSE of MLE s nflated and MSE of RR s affected negatvely. wth AM and MAX show better performance than MLE and RR snce an ncrease n the degree of correlaton affects wth MAX slghtly,.e., wth MAX s the most stable estmator n the study. LE has also better performance than MLE and RR, however, wth performance n most of the stuatons consdered. AM and MAX has the best Moreover, ncreasng the number of explanatory varables also affects the estmators negatvely,.e., ther estmated MSE ncreases. Although hgh correlaton maes an ncrease n the MSE of wth MAX when p 4, t becomes robust to the correlaton when p 6. Accordng to the results of the smulaton, wth MAX has the best performance among the estmators. 4

15 Table. Estmated MSEs of the estmator when p n (AM) (MAX) RR LE MLE (AM) (MAX) RR LE MLE (AM) (MAX) RR LE MLE

16 Table. Estmated MSEs of the estmator when p n (AM) (MAX) RR LE MLE (AM) (MAX) RR LE MLE (AM) (MAX) RR LE MLE

17 4.. Sweden Traffc Data 4. Real Data Applcatons In ths subsecton, we llustrate the benefts of new estmator usng a real dataset. The dataset s taen from the offcal webste of the Department of Transport Analyss n Sweden ( A smlar dataset s used n [4]. The dependent varable s the number of pedestran lled and the explanatory varables are the number of lometers drven by cars X and trucs X. In ths applcaton, we try to nvestgate the effect of changng the usage of cars and trucs on the number of pedestran lled. There are dfferent countes n Sweden and the data are pooled durng the year 03 for dfferent countes. The condton number beng the square root of the rato of the maxmum egenvalue and the mnmum egenvalue of the data matrx s approxmately showng that there s a moderate multcollnearty. The negatve bnomal regresson model wth ntercept s estmated usng IRLS algorthm for dfferent estmators consdered n ths study. The results are reported n Table 3. Accordng to Table 3, the effect of ncreasng X has a negatve mpact on the number of pedestran lled whch s not expected. It s nown that the sgns of coeffcents may be wrong when there s multcollnearty. Moreover, the effect of ncreasng X s low whle the effect of ncreasng X s hgh. If we use based estmators, the effect of ncreasng X becomes postve whch s expected and the effect of ncreasng s lower when compared to MLE. When we compare the standard errors of estmators, t s observed that wth AM and MAX have lower standard errors than other estmators whch maes them more stable. Thus, the estmator should be preferred snce t has a lower standard errors compared to other estmators and meanngful coeffcents compared to MLE. Moreover, wth AM and MAX have less MSE values than the other estmators. We also plot the MSE values of the estmators and RR for changng values of and LE for changng values of d 7

18 such that 0,0 d. We estmate the parameter d usng (.) for. Accordng to Fgure, we observe that when 0 d 0.6 MSE of LE s smaller than MSE of. Otherwse has the least MSE value. Table 3. Coeffcents, standard errors and MSE values of estmators for Sweden traffc data. Coeffcents (AM) (MAX) RR LE MLE Standard errors MSE Fgure. MSE plot of the estmators 8

19 Fnally, we provde some nformaton to ustfy the theorems gven n Secton. The estmated parameter values of are as follows: AM 0.489, MAX and d To ustfy Theorem., we consder the followngs: d d mn and b * * d d b.8459e 05 and the egenvalues of the dfference matrx MMSE MLE MMSE MLE MMSE are 0.007, and.050 whch are postve. Hence MMSE s postve defnte. Thus, Theorem. s satsfed. Smlarly, we compute the followngs to ustfy Theorem.3, d mn * * d d b d d b 7.3e-08 usng MAX for both RR and. The egenvalues of the dfference MMSE RR MMSE are 0.000,0.003 and whch are all postve, showng that the * * dfference s postve defnte. d d b d d b 4.086e-08 for both RR and and agan the dfference s postve defnte (the egenvalues are , and ). Thus Theorem.3 s satsfed. Agan, we consder the followng computatons to ustfy Theorem.4. We let AM and d s computed usng (.) as 0.09 for both LE and. However, mn d d becomes negatve and does not satsfy the pre-condton of Theorem.4. Thus, we try usng D 5 n both LE and to estmate the parameter d whch s computed as 0.58 and set.. Now, d d mn whch satsfes the precondton of Theorem.4. b * * d d d d b e-05. The 9

20 egenvalues of the dfference matrx MMSE LE MMSE whch are all postve. Hence the dfference matrx s postve defnte. are , and Thus, we observe that Theorems gven n Secton are satsfed. 4.. Football Teams Data In ths subsecton, another data set regardng the football teams competng n the Super League Season n Turey s consdered. A smlar data set s also analyzed n [6] for the 0-03 season. Accordng to [6], the data s approprate for the Posson regresson model. However, we try to ft a negatve bnomal regresson model because the varance (9.76) of the dependent varable s larger than the mean (7.33). Smlar to ther study, we have selected the number of won matches (NWM) as the dependent varable and the followngs are the explanatory varables: the number of red cards (NRC), the number of substtutons (NS), the number of matches endng over.5 goals (NOG), the number of matches completed wth goals (NCG), the rato of the goals scores n number of matches [NGR = NGS/NM], and the rato of goals scores n the sum of goals conceded and goal scores [NGR = NGS/(NGC + NGS)]. The egenvalues of the data matrx are , , , , 0.45 and The condton number s computed as multcollnearty problem whch much larger than 000 and shows that there s In Table 4, we present the coeffcents and the standard errors of estmators. Accordng to Table 4, t s easy to observe that the estmated theoretcal MSE value of wth AM and MAX are smaller than the others. Although, one can see that the varables NRC and NOG have negatve mpacts on NWM when Please see =64 and 0

21 RR, LE or MLE are used, ths s not the case for the estmator. In other words, all the varables have postve but small effects on NWM when s used. Moreover, the estmator has the least standard error values for ths applcaton whch further shows the superorty of over the others. Table 4. Coeffcents, standard errors and MSE values of estmators for football teams data Coeffcents () () RR LE MLE NRC NS NOG NCG NGR NG Standard errors NRC NS NOG NCG NGR NG MSE Concluson In ths study, a new based estmator whch s a generalzaton of Lu-type estmator s proposed for the negatve bnomal regresson models. We also revew some exstng estmators namely, negatve bnomal Lu estmator and negatve bnomal rdge estmator. We obtan some theoretcal comparsons between the estmators usng MMSE and obtan some condtons such that s superor to the others. Moreover, we desgn a Monte Carlo smulaton to understand the effects of the degree of correlaton among the explanatory varables, the sample sze and the number of explanatory varables. has a better performance than the others n the sense of MSE crteron n most of the cases consdered n

22 the smulaton. Fnally, we show that s a better choce and all the theoretcal dervatons are satsfed n real data applcatons and t s recommended to the researchers. 6. References [] Alhams, M., Khalaf, G. and Shuur, G. (006). Some modfcatons for choosng rdge parameters. Communcatons n Statstcs Theory and Methods, 35(), [] Cameron, A. C. and Trved, P. K. (986). Econometrc models based on count data. Comparsons and applcatons of some estmators and tests. Journal of appled econometrcs, (), [3] Cameron, A. C. and Trved, P. K. (03). Regresson analyss of count data (Vol. 53): Cambrdge unversty press. [4] Farebrother, R. (976). Further results on the mean square error of rdge regresson. Journal of the Royal Statstcal Socety. Seres B (Methodologcal), [5] Hlbe, J. (0). Negatve bnomal regresson: Cambrdge Unversty Press. [6] Hoerl, A. E. and Kennard, R. W. (970). Rdge regresson: Based estmaton for nonorthogonal problems. Technometrcs, (), [7] Hoerl, A. E., Kennard, R. W. and Baldwn, K. F. (975). Rdge regresson: Some smulatons. Communcatons n Statstcs-Theory and Methods, 4(), [8] Huang, J. and Yang, H. (04). A two-parameter estmator n the negatve bnomal regresson model. Journal of Statstcal Computaton and Smulaton, 84(), do: 0.080/ [9] Kbra, B. M. G. (003). Performance of some new rdge regresson estmators. Communcatons n Statstcs-Smulaton and Computaton, 3(), [0] Lpovetsy, S. and Conln, W. M. (005). Rdge regresson n two parameter soluton. Appled Stochastc Models n Busness and Industry, (6), [] Lu, K. (993). A new class of based estmate n lnear regresson. Communcatons n Statstcs- Theory and Methods, (), [] Lu, K. (003). Usng Lu-type estmator to combat collnearty. Communcatons n Statstcs-Theory and Methods, 3(5), [3] Månsson, K. (0). On rdge estmators for the negatve bnomal regresson model. Economc Modellng, 9(),

23 [4] Månsson, K. (03). Developng a Lu estmator for the negatve bnomal regresson model: method and applcaton. Journal of Statstcal Computaton and Smulaton, 83(9), [5] Månsson, K. and Shuur, G. (0). A Posson rdge regresson estmator. Economc Modellng, 8(4), [6] Türan, S. and Özel, G. (05). A new modfed Jacnfed estmator for the Posson regresson model. Journal of Appled Statstcs. do: 0.080/