The Bootstrap Maximum Likelihood Method for Estimation of Dispersion Parameter in the Negative Binomial Regression Model

Transcription

1 The Bootstrap Maxmum Lkelhood Method for Estmaton of Dsperson Parameter n the Negatve Bnomal Regresson Model Huang Jewu 1, Xaychalern Kongkeo 1,2,*, L Rong 1 1 College of Data Scence and Informaton Engneerng, Guzhou Mnzu Unversty, Guzhou, Guyang, , Chna 2 Faculty of Educaton, Souphanouvong Unversty, Luangprabang, 06000, Laos *Correspondng author (Xaychalern Kongkeo, E-mal: kkxaychalern@gmal.com) Abstract Estmaton of dsperson parameter n the negatve bnomal regresson model plays an mportant role n varous types of count data analyss. For ths purpose, the maxmum lkelhood method s often used. However, t has been reported n the lterature that the dsperson parameter can be msestmated when usng the maxmum lkelhood method for small sample sze. In addton, several researchers recommended that dsregardng mportant covarates maybe sgnfcantly affected on dsperson estmator. Ths study extends the bootstrap maxmum lkelhood method for estmaton of dsperson parameter n the negatve bnomal regresson model wth covarates for small sample sze. In order to evaluate performances of the estmators n the sense of root mean square error, a Monte Carlo smulaton study s gven. Furthermore, a real dataset s provded to llustrate some of the smulaton results. Keywords: Negatve bnomal regresson model, dsperson parameter, bootstrap maxmum lkelhood estmator, root mean square error 1. Introducton It s often assumed that the dstrbuton of count data at a gven ste or perod follows a Posson dstrbuton, whch only has one parameter and ts mean and varance are the same. The Posson dstrbuton has been shown to be reasonable to handle count data, as dscussed by (Ncholson and Wong, 1993). In realty, count data often dsplays overdsperson, for ths reason, the negatve bnomal (NB) regresson model has become the most commonly used framework for handlng overdsperson n count data, whch has found a wdespread use n the felds of health, socal, economc and physcal scences, (Huang and Yang, 2014). The NB regresson model contans two parameters: the mean and the dsperson parameter. For a random varable Y that NB dstrbuted, the margnal densty functon of Y takes the value y y y s defned as ( ) Pr( y y ) ; 0 ; y 0,1,2,... ; 1,2,..., n (1) ( ) ( 1) where n s the sample sze. For the complete dervaton of the NB regresson model, the readers are referred to (Hlbe, 2011; L et al., 2016). In expresson (1), f we set exp( x), then we get the NB regresson model wth covarates, where x s the th row of the desgn matrx n p y x whch contans p covarates, s the coeffcent vector of order p 1. From the margnal densty functon of the NB regresson model, t can be seen that the dsperson parameter s an essental part of the model, t s used to capture the extra-varaton observed n the data. For example, n transportaton safety analyss, t can be used as an ndcator to determne the varablty observed n the crash counts. The dsperson parameter can also be used as a measure of goodness-of-ft, as documented by (Maou, 2003). More recently, (Wood, 2005) used the dsperson parameter for estmatng the confdence ntervals for the Posson mean, gamma mean and predcted response of the NB regresson model. Gven the mportance of dsperson parameter plays n count data analyss, several estmators have been proposed to estmate the dsperson parameter. The smplest method to estmate s the method of moments estmate (MME), the maxmum lkelhood estmate method (MLE), the maxmum quas-lkelhood estmate (MQLE) (Zhang et al., 2006), etc. As reported n lterature (Hlbe, 2011; Frdstrøm et al., 1995), n many areas, researchers and practtoners are usually unable to collect a large quantty of data, due to prohbtvely hgh costs and other lmted condtons. The problem of small sample sze has been the focus of several studes n many felds. For nstance, (Clark and Perry, 1989) compared the MME and the MQLE. They found that both estmators become based and unstable when the mean value () s less than 3.0 and the sample sze (n) s small. (Pegorsch, 1990) reported that both 579

2 the MLE and the MQLE generated based estmates when the sample sze s 50. In our prevous study (Xaychalern et al., 2017), we proposed the bootstrap maxmum lkelhood estmator (BMLE) for estmaton of dsperson parameter n the NB regresson model wth covarates combned wth small sample sze, we consdered the estmated values of absolute bas as a crteron of method selecton. In ths study, we also extended the BMLE for estmaton of dsperson parameter n the NB regresson model wth covarates combned wth small sample sze whch consdered the root mean square error (RMSE) of dsperson estmators as the crteron of method selecton. We tred to examne that how covarates affect on dsperson estmators; whether the BMLE could be potentally mnmzed the RMSE of dsperson estmators for small sample sze than the MLE. The rest of ths paper s organzed as follows. In secton 2 and secton 3, the commonly used estmaton method as the MLE s frst revewed and the bootstrap maxmum lkelhood estmator (BMLE) s proposed. Monte Carlo smulatons and real data analyss are conducted n secton 4 and secton 5 respectvely. Fnally, some conclusons are gven n secton Estmators of n the nb regresson model wthout covarates 2.1. Maxmum Lkelhood Estmator As shown by Fsher (Bakoban, 2017), the MLE of the dsperson parameter can be found by maxmzng the log-lkelhood functon: n y 1 1 log(1 g) y log( ) log( y!) y log( ) ( y )log(1 ). (2) 1 g0 Then, the MLE can be obtaned by solvng the followng equaton: n y 1 1 log(1 g) y log( ) log( y!) y log( ) ( y )log(1 ) 0. 1 (3) g0 The Eq. 3 can be smplfed as n y 1 1 g ( y ) 2 log(1 ) 0. 1 g 1 (4) 1 g0 Snce the partal dervatve Eq. 4 cannot be further smplfed, we appled the Newton-Raphson teraton for estmatng as follows (1) (0) ( r1) (0) ( ) (0) ( ) ( ), r 1,2,..., m ( ) (0) where s the ntal value of gven by the researchers, s the value obtaned by the rth teraton, and denote the frst dervatve and the second dervatve of log-lkelhood functon respectvely, ( ) and ( ) are the functon s values evaluated at, and m equals 1000 correspondng to the number of replcatons used n the Monte Carlo smulaton. ( r 1) When the convergence s smaller than the convergence tolerance, we consder ( r 1) as the MLE estmator of the dsperson parameter n the NB regresson model wthout covarates Bootstrap Maxmum Lkelhood Estmator The msestmaton of dsperson parameter when usng the MLE can be solved by ncreasng sample sze, but that s usually nfeasble to do n practce as n the case wth crash data. Ths paper ntends to look for an alternatve estmaton of dsperson parameter that performs better wth small sample sze, and we turn to the bootstrap resamplng technque, whch s recommended as a remedy when the sample sze s small. The bootstrap technque was proposed by (Samuels, 1981; Efron and Gong, 1983; Efron, 1982) and further developed by (Efron and Tbshran, 2005). The dea of bootstrap resamplng s ntroduced as follows. Let y1, y2,..., y n be a sample of data, whch are the outcomes of..d. random varable Y1, Y2,..., Y n wth probablty densty functon f and cumulatve dstrbuton functon F. The sample s thus used to make nferences about a populaton characterstc θ. If we put equal probabltes 1/n at each sample value y, then the correspondng estmate of F s the emprcal dstrbuton functon ˆF. The dea of bootstrap resamplng from * * * * ˆF s ntroduced from a real world to the bootstrap world. Drawng a sample Y { Y1, Y2,... Y n } from (5) 580

3 ˆF wth replacements s called the bootstrap resample and resamplng from ˆF s referred to as the nonparametrc bootstrap. In realty, a large number of resamples, such as 500 or more need to be drawn to acheve robust estmates of nferences. In ths way, the dstrbuton of ˆ can be approxmated by the dstrbuton of ˆ*. The bootstrap has several good features n dealng wth small sample problems. Frst, the bootstrap does not rely on theoretcal dstrbutons and thus does not requre strong assumptons of the sample and the dstrbuton. Ths s a nce attrbute snce t s usually dffcult to obtan accurate parameters for a certan dstrbuton gven small samples. Moreover, by bootstrappng, the orgnal sample s duplcated many tmes. Hence, the bootstrap can treat a small sample as the vrtual populaton and generate more observatons from t. Fnally, the bootstrap s a rather smple technque and does not requre a sophstcated mathematcal background for people to use t. Because of those strengths, the bootstrap technques have been wdely appled to many engneerng areas and applcatons, ncludng radar and sonar sgnal processng, geophyscs, bomedcal engneerng, magng processng, control, and envronmental engneerng, etc. The bootstrap technques are also wdely used for estmatng statstcal features ncludng bas, varance, hypothess testng, and confdence nterval. The proposed BMLE method to estmate the fxed values n the NB regresson model under small sample sze condtons has the MLE nested nsde. The BMLE procedure s shown n the followng steps: Step 1: Generate a sample Y { Y1, Y2,... Y n } wth sze n from NB dstrbuton. * * * * Step 2: Draw a random sample of sze n wth replacements from step 1: Y { Y1, Y2,... Y n }. * * Step 3: Calculate the bootstrap estmate of ˆ from Y usng the MLE. Step 4: Repeat steps 2 and 3 for m tmes to obtan m estmates of * * * ˆ * : * * * 1, 2,... m 垐?. Step 5: Calculate the average of the 垐 1, 2,...? m. The average value we obtaned from step 5 s the BMLE estmator of dsperson parameter n the NB regresson model wthout covarates. 3. Estmators of n the NB regresson model wth covarates In the NB regresson model wth covarates, the MLE estmator of dsperson parameter can be found by substtutng exp( x ) to Eq. 2 and repeatng the procedure n secton 2.1. Also, by substtutng exp( x ) to Eq. 2 and repeatng the procedure n secton 2.2 for m tmes, then we can get the BMLE estmator of dsperson parameter n the NB regresson model wth covarates. 4. Monte Carlo smulaton Monte Carlo smulatons are conducted to show the performance and applcablty of the BMLE under small sample sze stuatons for the NB regresson model wthout covarates and NB regresson model wth covarates. Also, estmators of MLE and BMLE are compared to nvestgate ther relatve performances. The smulaton procedure s as follows 4.1. The Desgn of the Experment Observatons of the covarates are generated by where j 2 1/2 j j p z are ndependent standard normal pseudo-random numbers, correlaton between any two covarates gven as 0.72 and p s set to 5. x (1 ) z z, 1,2,..., n, j 1,2,..., p, 2 denotes the theoretcal The response varable y s generated usng pseudo-random numbers from the NB(, ) dstrbuton, where exp( 1x 1 2x 2... pxp ) ; 1,2,..., n ; j 1,2,..., p and the true values of the dsperson parameter correspondng to 0.25, 0.50, 0.75, 1.00 are consdered. The slop parameters are chosen p 2 so that j 1, whch are common restrctons n smulaton studes (Golam Kbra, 2003). j1 The values of n are nvestgated as 30, 50, 100, 150 and 250. And the convergence tolerance s taken to be Method Selecton To compare the MLE estmator and the BMLE estmator, the root mean square error (RMSE) crteron s consdered. The RMSE of an estmator ˆ s calculated as follows 2 581

4 m ˆ 2 ˆ r1 RMSE( ) m Where ˆ s an estmator of dsperson parameter. In the sense of RMSE crteron, 1 ˆ s superor to ˆ 2 f and only f RMSE( 垐 1) RMSE( 2). The smulatons are conducted usng the statstcal program R The results are summarzed n Table 1-2 and Fgure 1-2, where the cons wth subscrpt n denote the results derved from the NB regresson model wth covarates and the cons wth subscrpt non ndcate the results derved from the NB regresson model wthout covarates Results and Dscusson From the smulaton results shown n Table 1, t can be seen that n the majorty cases the RMSE of the MLE obtaned from the NB regresson model wth covarates s smaller than the RMSE of the MLE derved from the NB regresson model wthout covarates. Especally, for small sample (e.g. n 100 ), we have RMSEn < RMSE-non n all cases. Furthermore, t can be seen from Table 1 that the RMSE of the MLE decreases wth an ncrease of sample sze n for fxed n most cases. Smlar results can be seen from Table 2. All these smulaton results show that covarates and sample sze have sgnfcant nfluence on the estmators of dsperson parameter,.e., the MLE-n performs better than the MLE-non and the BMLE-n performs better than the BMLE-non under the RMSE crteron n most cases, especally for small sample. Table 1. Estmated RMSE values of the MLE MLE n RMSE-non RMSE-n 0.25 ˆnon ˆn > > > > < > > > > < > > > > > > > > < < n Table 2. Estmated RMSE values of the BMLE BMLE RMSE-non RMSE-n ˆnon ˆn > > > > < >

5 > > > < > > > > > > > > > < Fgure 1. Barplot of the RMSEs of the MLE-non and BMLE-non In the NB regresson model wthout covarates, we can see from Fgure 1 that the RMSEs of the BMLEnon s slghtly smaller than the RMSEs of the MLE-non for small sample sze (e.g. n 50 ) and the RMSEs of the two estmators decreases wth an ncrease of sample sze n for fxed. For the NB regresson model wth covarates, we can get smlar results from Fgure 2,.e., the RMSEs of the BMLE-n s explct smaller than the RMSEs of the MLE-n for small sample (e.g. n 50 ) and the RMSEs of the two estmators also decreases wth an ncrease of sample sze n for gven. All these smulaton results ndcate that the BMLE-non performs better than the MLE-non and the BMLEn outperforms the MLE-n under the RMSE crteron for small sample. 583

6 Fgure 2. Barplot of the RMSEs of the MLE-n and BMLE-n 5. Real data analyss An accdent dataset s used as an example to show dfferent performances of the two estmators. The sample dataset was taken from (Pornpradtpun, 2009), there are 374 observatons, and the sample has a mean value of and a varance of We assume no spatal or temporal correlaton between those observatons for smplcty and use the dataset to examne the effects of small sample sze and covarates on dfferent estmators. We consder the number of people klled by accdents as the response varable and the accdent occurrence tme, the road characterstcs, the causes of accdents as the covarates. By randomly selectng observatons from the full dataset, we can obtan subsamples wth dfferent sample szes. The two estmators were frstly used to estmate the parameter for the full dataset. Then, the full dataset was treated as the sample populaton and 1000 subsamples wth sample szes of 30, 50, 100, 150 and 250 were generated from t. These subsamples were drawn wth overdsperson to ft NB dstrbutons. For each subsample, the values were estmated by the two estmators. Results of the statstcs are shown n Table 3 and Table 4. Table 3. Estmated RMSE values of the MLE and BMLE (1) n MLE-non MLE-n BMLE-non BMLE-n > > > > > > > < < < Table 4. Estmated RMSE values of the MLE and BMLE (2) n MLE-non BMLE-non MLE-n BMLE-n > > > > > > > < > < For the full dataset, the estmated value s around 1. The estmated values from the MLE and BMLE are close, whch s a concluson smlar to one reached from smulatons wth large sample sze. For the subsamples wth dfferent smaller sample szes as show n Table 3, t s agan found that for the small sample sze (e.g. n 100 ) the RMSEs of both the MLE-n and the BMLE-n are smaller than the RMSEs of the MLEnon and BMLE-non respectvely, ndcatng that estmatng dsperson parameter n the NB regresson model wth consderng covarates s more accurate than dsregardng covarates. 584

7 From Table 4, t can be nferred that n the majorty cases, the RMSEs of the BMLE-non s smaller than the MLE-non and the RMSEs of the BMLE-n s smaller than the MLE-n, ndcatng that the proposed BMLE estmator s superor the MLE estmator when sample sze s small (e.g. n 100 ) for both cases of the NB regresson model wth covarates and wthout covarates. 6. Conclusons In ths paper, we generalzed the BMLE for estmaton of dsperson parameter n the NB regresson model wth covarates combned wth small sample sze. We compared the MLE-non, MLE-n, BMLE-non and BMLE-n under the RMSE crteron. The results show that the MLE-n s superor to the MLE-non and the BMLE-n s superor to the BMLE-non n most cases, whch means covarates have sgnfcant nfluence on the estmators of dsperson parameter. Also, we obtaned the BMLE-non performs better than the MLE-non and the BMLE-n outperforms the MLE-n for small sample sze ( n 100 ) and the RMSE of the MLE-non, MLE-n, BMLE-non and BMLE-n decrease wth ncreasng of sample sze n for gven n most cases. Acknowledgements Ths work was supported by Guzhou Scence and Technology Department (Grant No: Qan Scence [2017]1083), Guzhou Provncal Educaton Department (Grant No: Qan Scence co-jg word LKM [2015]014) and Hgh-level Innovatve Talents Project of Guzhou Provnce. References Bakoban R A. Statstcal Propertes and Fsher Informaton of a Qubt System n Superposton of Negatve Bnomal Dstrbuton,Journal of Computatonal & Theoretcal Nanoscence, Bschoff E, Bschoff H, Gulano F. An Introducton to the Bootstrap/B. Efron, R.J. Tbshran.,2005. Clark S J, Perry J N. Estmaton of the Negatve Bnomal Parameter k by Maxmum Quas - Lkelhood,Bometrcs, 1989, 45(1): Efron B, Gong G. A Lesurely Look at the Bootstrap, the Jackknfe, and Cross-Valdaton,Amercan Statstcan, 1983, 37(1): Efron B. The Jackknfe, the Bootstrap and Other Resamplng Plans, The jackknfe, the bootstrap, and other resamplng plans. Socety for Industral and Appled Mathematcs, Frdstrøm L, Ifver J, Ingebrgtsen S, et al. Measurng the contrbuton of randomness, exposure, weather, and daylght to the varaton n road accdent counts,accdent; analyss and preventon, 1995, 27(1):1-20. Golam Kbra B M. Performance of Some New Rdge Regresson Estmators,Communca- tons n Statstcs - Smulaton and Computaton, 2003, 32(2): Hlbe J. Negatve Bnomal Regresson, second edton, Cambrdge Unversty Press,2011. Huang J, Yang Hu. A two-parameter estmator n the negatve bnomal regresson model,journal of Statstcal Computaton & Smulaton, 2014, 84(1): L R, Chen L, Wang P. Smulaton Study and Analyss of Overdsperson Data,Journal of Quanttatve Economcs, 2016, 33(1): Maou S. Modelng traffc crash-flow relatonshps for ntersectons: dsperson parameter, functonal form, and Bayes versus emprcal Bayes methods,transportaton Research Record Journal of the Transportaton Research Board, 2003, 1840(1840): Ncholson A, Wong Y D. Are accdents posson dstrbuted? A statstcal test,accdent; analyss and preventon, 1993, 25(1):91-7. Pegorsch W W. Maxmum lkelhood estmaton for the negatve bnomal dsperson parameter,bometrcs, 1990, 46(3): Pornpradtpun S. The Dsperson Estmaton under Generalzed Lnear Model wth Negatve Bnomal Dstrbuton[D]. Faculty of Scence and Technology, Thammasath Unversty, Samuels M L. Nonparametrc estmates of standard error: The jackknfe, the bootstrap and other methods,bometrka, 1981, 68(3): Wood G R. Confdence and predcton ntervals for generalzed lnear accdent models,accdent Analyss and Preventon, 2005, 37(2): Xaychalern K, Huang J, Chang G. Study on the Estmaton of Dsperson Parameter n the Negatve Bnomal Regresson Model wth Covarates,Journal of Quanttatve Economcs, 2017, 34(4): 1-6. Zhang Y, Zhru Y, Lord D, et al. Estmatng the Dsperson Parameter of the Negatve Bnomal Dstrbuton for Analyzng Crash Data Usng a Bootstrapped Maxmum Lkelhood Method,Transportaton Research Record Journal of the Transportaton Research Board, 2006,