Joural of Moder Applied Statistical Methods Volume 4 Issue 2 Article 0 --205 New Etropy Estimators with Smaller Root Mea Squared Error Amer Ibrahim Al-Omari Al al-bayt Uiversity, Mafraq, Jorda, alomari_amer@yahoo.com Follow this ad additioal works at: http://digitalcommos.waye.edu/jmasm Part of the Applied Statistics Commos, Social ad Behavioral Scieces Commos, ad the Statistical Theory Commos Recommeded Citatio Al-Omari, Amer Ibrahim (205) "New Etropy Estimators with Smaller Root Mea Squared Error," Joural of Moder Applied Statistical Methods: Vol. 4 : Iss. 2, Article 0. DOI: 0.22237/jmasm/446350940 Available at: http://digitalcommos.waye.edu/jmasm/vol4/iss2/0 This Regular Article is brought to you for free ad ope access by the Ope Access Jourals at DigitalCommos@WayeState. It has bee accepted for iclusio i Joural of Moder Applied Statistical Methods by a authorized editor of DigitalCommos@WayeState.
New Etropy Estimators with Smaller Root Mea Squared Error Cover Page Footote The author thaks the editor ad the referees for their helpful ad valuable commets that substatially improved this paper. This regular article is available i Joural of Moder Applied Statistical Methods: http://digitalcommos.waye.edu/jmasm/vol4/ iss2/0
Joural of Moder Applied Statistical Methods November 205, Vol. 4, No. 2, 88-09. Copyright 205 JMASM, Ic. ISSN 538 9472 New Etropy Estimators with Smaller Root Mea Squared Error Amer Ibrahim Al-Omari Al al-bayt Uiversity Mafraq, Jorda New estimators of etropy of cotiuous radom variable are suggested. The proposed estimators are ivestigated uder simple radom samplig (SRS), raked set samplig (RSS), ad double raked set samplig (DRSS) methods. The estimators are compared with Vasicek (976) ad Al-Omari (204) etropy estimators theoretically ad by simulatio i terms of the root mea squared error (RMSE) ad bias values. The results idicate that the suggested estimators have less RMSE ad bias values tha their competig estimators itroduced by Vasicek (976) ad Al-Omari (204). Keywords: Shao etropy; simple radom samplig, raked set samplig; double raked set samplig; root mea square error. Itroductio The raked set samplig was first suggested by McItyre (952) to estimate a mea of pasture ad forage yields. It is a cost efficiet samplig procedure alterative to the commoly used simple radom samplig scheme. The RSS is useful i situatios where the visual orderig of a set of uits ca be doe easily, but the exact measuremet of the uits is difficult or expesive. Let the variable of iterest X has a probability desity fuctio (pdf) g(x) ad a cumulative distributio fuctio (cdf) G(x), with mea μ ad variace σ 2. Let g (i:) (x) ad G (i:) (x) be the pdf ad cdf of the ith order statistic, X (i:), ( i ) of a radom sample of size. The pdf ad the cdf of X (i:), respectively, are give by i i g( i : ) x G x Gx g x, x i Amer Ibrahim Al-Omari is Faculty of Sciece, i the Departmet of Mathematics. Email at: alomari_amer@yahoo.com. 88
AMER IBRAHIM AL-OMARI ad j j G : x G x G x, x i j j, 2 i: x dx ad variace i: i: x i : g x dx i:. The raked set samplig method ca be describes as follows: with mea x g 2 Step. Radomly select 2 uits from the target populatio. Step 2. Allocate the 2 selected uits radomly ito sets, each of size. Step 3. Without yet kowig ay values for the variable of iterest, rak the uits withi each set with respect to a variable of iterest. This may be based o a persoal professioal judgmet or based o a cocomitat variable correlated with the variable of iterest. Step 4. The sample uits are selected for actual measuremet by icludig the ith smallest raked uit of the ith sample (i =, 2,, ). Step 5. Repeat Steps through 4 for r cycles to obtai a sample of size r for actual measuremet. It is of iterest to ote here that eve if 2 uits are selected from the populatio, but oly of them are measured for compariso with a simple radom samplig of the same size. Let the measured RSS uits are deoted by X (:), X 2(2:),, X (:). The i RSS estimator of the populatio mea is defied as XRSS Xii :. Takahasi ad Wakimoto (968) provided the mathematical theory of the RSS ad showed that 2 i: i: RSS 2 ii: g x g x,, Var X. i i i Al-Saleh ad Al-Kadiri (2000) suggested double raked set samplig (DRSS) method for estimatig the populatio mea to icrease the efficiecy of the estimators for fixed sample size. The DRSS method ca be described as: 2 89
NEW ENTROPY ESTIMATORS Step. Radomly choose 2 samples of size each from the target populatio. Step 2. Apply the RSS method described above o the 2 samples i Step. This step yields samples of size each. Step 3. Reapply the RSS method agai o the samples obtaied i Step 2 to obtai a sample of size from the DRSS data. The cycle ca be repeated r times if eeded to obtai a sample of size r uits. Let X be a cotiuous radom variable with probability desity fuctio gx ( ) ad cumulative distributio fuctio G(x). The etropy H [g(x)] of the radom variable is defied by Shao (948a, 948b) as H g x g x log g x dx. () The problem of etropy estimatio of a cotiuous radom variable is cosidered by may authors. Vasicek's (976) suggested a estimator of etropy based o spacig's as log dg p H g x dp, 0 dp (2) where the estimatio is foud by replacig the distributio fuctio G(x) by the empirical distributio fuctio G (x), ad usig the differece operator istead of d the differetial operator. The the derivative G p is estimated by a dp fuctio of the order statistics. Let X, X 2,, X be a simple radom sample of size from G(x) ad X () < X (2) < < X () be the order statistics of the sample. The Vasicek's (976) estimator of H [g(x)] is defied as HV log X X 2 (3) m i m i m i m 90
AMER IBRAHIM AL-OMARI where m < / 2 is a positive iteger kow as the widow size, X (i - m) = X () if P. i m, ad X (i + m) = X () if i m. He proved that HVm H g x as m, m, ad 0. Va Es (992) suggested a estimator of etropy based o spacigs as m HVEm X X i m i log m log m i m km k (4) ad proved the cosistecy ad the asymptotic ormality of the estimator uder some coditios. Ebrahimi, Pflughoeft, ad Soofi (994) adjusted the weights of Vasicek (976) estimator to have a smaller weights ad proposed a etropy estimator give by where HE X X log (5) m i m i m i im i, i m, m i 2, m i m, i, m i, m where X (i-m) = X () for i m ad X (i+m) = X () for i m. Ebrahimi et al. (994) showed by simulatio that their estimator has a smaller bias ad mea squared error tha Vasicek (976) estimator. Also, they proved that P. HEm H g x m m as,, 0. Noughabi ad Noughabi (203) suggested a ew estimator of etropy of a ukow cotiuous probability desity fuctio as 9
NEW ENTROPY ESTIMATORS HNN log s, m, (6) m i i where ad gˆ X i gˆ X, i m, i 2 m/ si, m, m i m, Xi m X im gˆ X, m i, i Xi X j k, where h is badwidth ad k is a kerel fuctio h h j satisfies k xdx. They proved that HNN P m H g x as, m, m / 0. Note that the kerel fuctio i Noughabi ad Noughabi (203) is selected to be the stadard ormal distributio ad the badwidth h is chose to be h =.06s -/5, where s is the sample stadard deviatio. To estimate the etropy H [g(x)] of a ukow cotiuous probability desity fuctio g(x), Noughabi ad Arghami (200) suggested a etropy estimator give by where HN X X log (7) m i m i m i cim, im, ci 2, m i m,, m i, ad X (i-m) = X () if i m ad X (i+m) = X () for i m. Correa (995) suggested a modified etropy estimator to have smaller mea squared error i the form 92
AMER IBRAHIM AL-OMARI HC m i im j i X X j i j i m log, im X X 2 j i j i m (8) im where X X i j. 2m j i m Al-Omari (204) suggested three estimators of etropy of a ukow cotiuous probability desity fuctio g(x) usig SRS, RSS, ad DRSS methods. Based o SRS his first suggested estimator is defied as AHESRS X X log (9) m i m i m i im where X (i-m) = X () for i m, X (i+m) = X () for i m, ad, i m, 2 i 2, m i m,, m i, 2 (0) The secod ad third estimators suggested by Al-Omari (204), based o RSS ad DRSS respectively, are give by ad AHERSS X X * * log () m i m i m i im AHEDRSS X X ** ** log (2) m i m i m i im * * * * where Xi m X () for i m ad X X im ( ) for i m, ad 93
NEW ENTROPY ESTIMATORS X X ** ** X im () for i m ad i m X for i m. ** ** ( ) For more about etropy estimators, see Choi, Kim, ad Sog (2004), Park, Park (2003), Goria, Leoeko, Mergel, ad Novi Iverardi (2005) ad Choi (2008). The remaiig part of this paper is orgaized as follows. The suggested etropy estimators are give i the sectio, Proposed Estimators. Next, a simulatio study is coducted to compare the ew estimators with their couterparts suggested by Vasicek (976) ad Al-Omari (204). Fially, some coclusios ad suggestios for further works. The proposed estimators The coefficiet of the etropy estimators i Ebrahimi et al. (994), Noughabi ad (0) (0) (0) Arghami (200), ad Al-Omari (204) are adjusted. Let X, X 2,..., X be a simple radom sample of size from G (x). Based o SRS the first suggested estimator is give by where SHESRS X X 0 0 log (3) m i m i m i im, i m, 4 i 2, m i m,, m i, 4 (4) (0) (0) (0) (0) Xi m X () for i m ad X X im ( ) for i m. Comparig (3) with (3), we have 94
AMER IBRAHIM AL-OMARI Let 0 0 SHESRSm log X i m X im i im 2 HVSRSm log (5) () () () (: ) (2: ) ( : ) HVSRS m i 2m 8 log 5 X, X,..., X be a RSS of size, Vasicek (976) etropy estimator usig RSS as cosidered by Mahdizadeh (202) is give by HVRSS log X X 2 i (6) m i m i m i m Based o the RSS uits etropy estimator is X (), (: ) X (), (2: ), X () ( : ), the secod suggested SHERSS X X log (7) m i m i m i im () () () () where i is defied as i (4), ad Xi m X () for i m ad X X im ( ) for i m. Comparig (6) with (7) to have Assume that () () SHERSSm log X i m X im i im 2 HVRSSm Log (8) HVRSS m i 2m 8 log 5 X, X,..., X is a DRSS sample of size. The third (2) (2) (2) (: ) (2: ) ( : ) suggested etropy estimator has the form i 95
NEW ENTROPY ESTIMATORS SHEDRSS X X 2 2 log (9) m i m i m i im (2) (2) (2) (2) where i is defied as i (4), ad Xi m X () for i m ad X X im ( ) for i m. Based o DRSS method Mahdizadeh (202) showed that Vasicek (976) estimator will be SHEDRSS log X X 2 2 2 (20) m i m i m i m Comparig (9) with (20) to get 2 2 SHEDRSSm log X i m X im i im 2 HVDRSSm log (2) HVDRSS ME Remark : The etropy m i 2m 8 log 5 H f of a empirical maximum etropy desity ME f which is related to HVSRS ad SHESRS ca be computed followig Theil (980) as: i ME 2 2log 2 H f HVSRS 2 8 2 2log 2 SHESRS log 5 2 4 SHESRS log 5 (22) ME Remark 2: If i (22), the H f SHESRS. I the followig two theorems, we compared the suggested estimators with Vasicek (967) ad Al-Omari (204). 96
AMER IBRAHIM AL-OMARI Theorem : The suggested estimators have the followig properties: (0) (0) (0) a) Let X, X 2,..., X be SRS of size, the SHESRS m > HVSRS m. () () () b) Let X (), X (2),..., X ( ) be a RSS of size, the SHERSS m > HVRSS m. (2) (2) (2) c) Let X (), X (2),..., X ( ) be a DRSS of size, the SHEDRSS m > HVDRSS m. Proof: The proof of (a), (b), (c), is straightforward by usig (5), (8), (2), respectively, where 2 m log 8 0. 5 I the followig theorem, we compare our suggested etropy estimators with their competitors i Al-Omari (204). Theorem 2: Based o the suggested estimators ad Al-Omari (204) etropy respectively, we have SHEj m > AHEj m, j = SRS, RSS, DRSS. Proof: Compare (9) with (3) based o SRS to obtai SHESRS m 2m 6 AHESRS m log, 5 ad sice 2 m log 6 0, the the case of SRS holds. Also, compare () with (7) 5 based o RSS, ad (2) with (9) usig DRSS to complete the proof of this theorem. The followig theorem proves the cosistecy of the suggested estimators SHESRS m, SHERSS m, ad SHEDRSS m. Theorem 3: Let Ω be the class of cotiuous desities with fiite etropies ad let X, X 2,, X be a radom sample from g Ω. If, m, m/ 0, the SHEj m, (j = SRS, RSS, DRSS) coverges i probability to H [g(x)]. Proof: Based o the simple radom samplig, from (5) we have 97
NEW ENTROPY ESTIMATORS SHESRS m 2m 8 HVSRS m log, 5 ad Vasicek (976) showed that HVSRS m coverges i probability to H [g(x)] ad sice 2 m log 8 coverges to zero as goes to ifiity, the we proved the 5 case of the SRS. Follow the same approach ad use (8) ad (2) to prove the theorem for RSS ad DRSS estimators, respectively. Methodology Simulatio study A simulatio was coducted to ivestigate the performace of the suggested etropy estimators with Vasicek (976) ad Al-Omari (204) etropy estimators usig samplig methods cosidered i this study. The compariso is based o the root mea squared errors (RMSEs) ad bias values of the estimators for 0000 samples geerated from the uiform, expoetial ad the stadard ormal distributios usig SRS, RSS ad DRSS methods. The selectio of the optimal values of the widow size of m for a give value is as yet a ope problem i the etropy estimatio. Therefore, we used the heuristic formula m 0.5 suggested by Wieczorkowski ad Grzegorzewski (999) to select m ad to compute the RMSEs of etropy estimators. I this study, we cosidered the sample ad widow sizes as give i Table. Table. The sample ad widow sizes cosidered i this simulatio Sample size = 0 = 20 = 30 Widow size m 5 m 0 m 5 Also, the performace of the RMSE of the suggested estimators for samples geerated from the uiform, expoetial ad stadard ormal distributios is evaluated based o the quatity Q N HVjm N 00, N SHEj m, AHEj m, j SRS, RSS, DRSS. HVj m 98
AMER IBRAHIM AL-OMARI The results are summarized i Tables 2-6. Also, we compared the suggested estimators of etropy with their competitors suggested by Al-Omari (204) ad the results preseted i Table 7 are take from Al-Omari (204). Based o these results observe the followig. The suggested etropy estimators usig SRS, RSS ad DRSS methods are more efficiet tha their competitors HV m based o the same method for all cases cosidered i this study. As a example, from Table 3, with = 0 ad m = 3 for the expoetial distributio with H [g(x)] = usig RSS method, the RMSE ad bias value of SHERSS m are 0.23042 ad -0.052759 compared to 0.4025 ad -0.332760 the RMSE ad bias of HVRSS m. The SHEDRSS m is superior to the other suggested estimators, SHERSS m ad SHESRS m uder the uiform, expoetial ad ormal distributios. From Table, cosider the case of = 20 ad m = 4 uder the uiform distributio whe H [g(x)] = 0, it ca be oted that the RMSE values of SHEDRSS m, SHERSS m, ad SHESRS m are 0.052373, 0.068747 ad 0.4983, respectively. The ature of the uderlyig distributio as well as the value of H [g(x)] affect o the efficiecy of the estimator usig the same method. As a example, the Q values with = 30 ad m = 3 SHERSS m for the uiform, expoetial, ad the stadard ormal distributios are 95.39025, 3.76442 ad 32.75544, respectively. However, the values of Q for the uiform distributio with H [g(x)] = 0 are SHE m superior to their couterparts for the expoetial ad ormal distributios. Fially, the suggested etropy estimators are foud to be more efficiet tha their competitors i Al-Omari (204) etropy estimators usig SRS, RSS ad DRSS schemes for the same widow ad sample sizes. For illustratio, assume that = 30 ad m = 8 whe the uderlyig distributio is the stadard ormal, from Table 4, the RMSE of SHERSS m is 0.20242 compared to 0.57726 which is the RMSE of AHERSS m as show i Table 7. 99
NEW ENTROPY ESTIMATORS Table 2. The Mote Carlo RMSEs ad bias values of HV m ad SHE m for the uiform distributio with H [g(x)] = 0. SRS HV m SHE m RSS Q SHE m HV m SHE m m Bias RMSE Bias RMSE Bias RMSE Bias RMSE Q SHE m 0-0.59826 0.569537-0.4305 0.490404 3.89427-0.396308 0.443439-0.303703 0.36606 22.63043 2-0.4535 0.452358-0.226627 0.290240 35.83843-0.304078 0.329233-0.695 0.7296 90.3500 3-0.42263 0.45388-0.35797 0.2348 53.03227-0.32768 0.34399-0.04589 0.459 20.3262 4-0.458940 0.487054-0.08005 0.79669 63.07-0.37538 0.38303 0.004574 0.093383 30.24920 5-0.502063 0.52798-0.03273 0.67982 68.8029-0.425903 0.43652 0.042936 0.0550 35.420 20-0.393900 0.48346-0.34992 0.376728 9.94822-0.343340 0.365754-0.294874 0.320679 4.056 2-0.27880 0.29088-0.77492 0.204940 29.5298-0.27937 0.233026-0.256 0.5007 55.33306 3-0.25393 0.270200-0.2786 0.4559 46.4397-0.20532 0.26879-0.063859 0.093348 32.33380 4-0.260596 0.274678-0.074069 0.4983 58.3898-0.24042 0.222524-0.0266 0.068747 223.68540 5-0.276800 0.288985-0.043624 0.095299 67.02286-0.2354 0.24279 0.000439 0.052744 359.5930 6-0.29932 0.30256-0.07934 0.085705 72.37604-0.258899 0.264554 0.022973 0.059480 344.7780 7-0.322084 0.33230 0.005663 0.08233 75.22397-0.28530 0.29056 0.043299 0.06772 328.5490 8-0.348254 0.35790 0.028228 0.087902 75.43958-0.3438 0.3847 0.069 0.0894 292.23460 9-0.374620 0.383864 0.048022 0.09770 74.54567-0.34340 0.3477 0.07994 0.09672 259.49900 0-0.402840 0.474 0.066866 0.08377 73.67836-0.37780 0.375737 0.097578 0.233 235.0860 30-0.352853 0.368369-0.323835 0.34096 7.44037-0.39230 0.333509 0.288992 0.30576 9.2845 2-0.223356 0.235685-0.6288 0.782 24.4242-0.90866 0.20625-0.2749 0.42794 4.999 3-0.9779 0.208362-0.04892 0.24359 40.3589-0.6582 0.73360-0.070574 0.088725 95.39025 4-0.96240 0.205882-0.07025 0.09384 54.4332-0.62899 0.6984-0.038020 0.06304 77.04720 5-0.202003 0.20395-0.04635 0.075603 64.0666-0.7244 0.78293-0.04997 0.046725 28.57950 6-0.23804 0.22385-0.024700 0.063205 7.4509-0.85622 0.90458 0.002250 0.043550 337.3380 7-0.226688 0.23352-0.00794 0.057695 75.29344-0.200036 0.204048 0.08588 0.04506 352.37440 8-0.242599 0.248992 0.007775 0.057090 77.0755-0.27704 0.22309 0.03374 0.0583 326.9890 9-0.25947 0.265356 0.022036 0.060359 77.25358-0.23566 0.238850 0.046793 0.060639 293.88840 Table 2 cotiued o ext page 00
AMER IBRAHIM AL-OMARI 0-0.276934 0.282548 0.03625 0.067383 76.567-0.254437 0.257257 0.058627 0.069646 269.37800-0.295302 0.300725 0.049094 0.074862 75.066-0.273700 0.276336 0.072000 0.08003 24.4290 2-0.33803 0.39255 0.06228 0.085295 73.283-0.293398 0.2959 0.083363 0.09704 222.68060 3-0.332279 0.337432 0.075374 0.095536 7.68733-0.3978 0.340 0.09565 0.02770 23.90720 4-0.35090 0.356205 0.087783 0.06535 70.0966-0.332096 0.33458 0.06272 0.3446 94.86980 5-0.370555 0.37558 0.099545 0.6477 68.9823-0.352077 0.354327 0.856 0.2508 83.27800 Table 3. The Mote Carlo RMSEs ad bias values of HV m ad SHE m for the expoetial distributio with H [g(x)] =. SRS HV m SHE m RSS Q SHE m HV m SHE m m Bias RMSE Bias RMSE Bias RMSE Bias RMSE Q SHE m 0-0.552032 0.67700-0.457584 0.60004.36778-0.430553 0.505229-0.34284 0.432785 4.33884 2-0.442683 0.57820-0.25308 0.442568 22.60362-0.337494 0.404667-0.48595 0.269907 33.3046 3-0.435444 0.56640-0.54607 0.39369 30.3675-0.332760 0.4025-0.052759 0.23042 42.55855 4-0.45545 0.575390-0.07688 0.3720 35.48550-0.348029 0.42067 0.025378 0.233566 44.47062 5-0.469437 0.59776 0.005489 0.37248 37.69784-0.366628 0.445977 0.0893 0.27052 39.34396 20-0.44064 0.49007-0.3607 0.445976 9.00436-0.357765 0.39866-0.3253 0.358752 0.0076 2-0.28577 0.376086-0.9343 0.30495 7.44043-0.234959 0.280262-0.4085 0.207405 25.99603 3-0.260773 0.3534-0.2204 0.272095 22.55530-0.23397 0.2626-0.07287 0.65700 36.57683 4-0.2566 0.35280-0.067569 0.25502 28.746-0.20620 0.259248-0.07564 0.52350 4.23388 5-0.26242 0.358638-0.02244 0.24408 3.95980-0.2422 0.265246 0.02290 0.56584 40.96650 6-0.265650 0.360325 0.06823 0.248330 3.0866-0.28028 0.27235 0.06287 0.74543 35.9040 7-0.266934 0.365008 0.05546 0.256349 29.76894-0.224596 0.28296 0.0360 0.200858 28.82323 8-0.273952 0.37759 0.00674 0.274582 27.2667-0.232629 0.293062 0.45963 0.23970 20.8460 9-0.28023 0.38968 0.43573 0.293999 23.03046-0.23625 0.302083 0.88596 0.267430.4735 0-0.28583 0.39290 0.79545 0.322338 7.627-0.23843 0.30922 0.23203 0.303760 2.30347 30-0.367058 0.423423-0.33206 0.394742 6.77360-0.332526 0.3649-0.303272 0.334033 7.59576 Table 3 cotiued o ext page 0
NEW ENTROPY ESTIMATORS 2-0.233677 0.306086-0.735 0.26206 4.3979-0.203455 0.23600-0.37679 0.82964 22.4732 3-0.202277 0.28503-0.08684 0.2239 20.7452-0.70859 0.207468-0.078000 0.4567 3.76442 4-0.94424 0.275072-0.067472 0.207505 24.56339-0.60246 0.9940-0.036059 0.23278 38.7863 5-0.9705 0.272356-0.033792 0.9778 27.40457-0.5974 0.200465-0.00250 0.22595 38.84469 6-0.86870 0.27296 0.000772 0.9584 28.0548-0.58702 0.202869 0.027994 0.28086 36.86270 7-0.9094 0.275374 0.029066 0.9854 28.0486-0.6705 0.206226 0.05957 0.4042 3.60804 8-0.95662 0.280589 0.056849 0.208607 25.65389-0.64468 0.22265 0.085540 0.60732 24.27767 9-0.96983 0.282040 0.088082 0.22060 2.78060-0.655 0.27222 0.528 0.82796 5.84830 0-0.977 0.283394 0.5949 0.235447 6.9885-0.6752 0.220237 0.4444 0.205632 6.6349-0.98853 0.28624 0.42656 0.253233.5354-0.73076 0.22938 0.72966 0.220033 4.04896 2-0.204089 0.293653 0.7742 0.274080 6.66535-0.7555 0.232740 0.200259 0.2465 7.78766 3-0.202908 0.29808 0.204980 0.228389 23.3877-0.76996 0.240454 0.23487 0.23202 3.47343 4-0.205700 0.300842 0.232277 0.290007 3.6056-0.76922 0.24454 0.262425 0.242 3.65780 5-0.20699 0.305809 0.258234 0.3000.89595-0.77959 0.248760 0.29253 0.2395 3.87723 Table 4. The Mote Carlo RMSEs ad bias values of HV m ad SHE m for the stadard ormal distributio ad H [g(x)] =.49. SRS HV m SHE m RSS Q SHE m HV m SHE m m Bias RMSE Bias RMSE Bias RMSE Bias RMSE Q SHE m 0-0.598925 0.676499-0.499469 0.4347 35.8208955-0.484489 0.549750-0.388446 0.466743 5.09905 2-0.52455 0.59007-0.335907 0.436633 26.205028-0.42269 0.4757-0.238609 0.320258 32.02733 3-0.563002 0.62388-0.275063 0.382983 38.5445484-0.462240 0.504378-0.8597 0.269765 46.553 4-0.6065 0.663364-0.236072 0.35842 46.9609445-0.52309 0.557792-0.49270 0.244690 56.3239 5-0.67777 0.79069-0.200702 0.325688 54.7069892-0.584483 0.64209-0.978 0.28489 64.42758 20-0.435480 0.483459-0.38098 0.4347 0.948666-0.382986 0.42030-0.33552 0.377639 0.5227 2-0.32745 0.375798-0.23087 0.29633 2.988888-0.27576 0.33472-0.82040 0.23472 25.2505 3-0.37948 0.364927-0.7530 0.255 3.0790925-0.268657 0.3048-0.2504 0.8903 37.96057 Table 4 cotiued o ext page 02
AMER IBRAHIM AL-OMARI 4-0.327070 0.372436-0.43556 0.230357 38.485678-0.28533 0.38855-0.09869 0.72598 45.86944 5-0.352658 0.395796-0.7332 0.25233 45.620285-0.305555 0.337744-0.073404 0.60748 52.40537 6 0.375996 0.46964-0.09879 0.204234 5.087930-0.335066 0.36585-0.0592 0.52608 58.2077 7-0.404050 0.442997-0.083445 0.99295 55.0207-0.363782 0.39748-0.036080 0.4838 62.8538 8-0.43968 0.475094-0.06765 0.87822 60.4663498-0.39522 0.42583-0.02065 0.47835 64.93336 9-0.46734 0.500777-0.043230 0.86628 62.732340-0.428042 0.45680-0.006860 0.4459 68.0042 0-0.496926 0.527456-0.029603 0.78984 66.0665534-0.45488 0.47752 0.009882 0.45955 69.42 30-0.378860 0.43455-0.346828 0.384885 6.9006276-0.343626 0.37052-0.33688 0.342854 7.464805 2-0.25905 0.299687-0.96988 0.246877 7.62787-0.22694 0.255947-0.6349 0.20857 2.3328 3-0.236758 0.277238-0.4522 0.203905 26.45280-0.204698 0.234358-0.0857 0.57593 32.75544 4-0.234369 0.275867-0.0865 0.7987 34.875026-0.204765 0.23443-0.08230 0.40863 39.90820 5-0.244288 0.283027-0.088572 0.6605 4.3303324-0.24434 0.243683-0.0568 0.2784 47.80760 6-0.255248 0.293332-0.068084 0.57937 46.575962-0.227340 0.25590-0.038603 0.22294 52.2043 7-0.269724 0.30534-0.048333 0.5084 50.486060-0.24325 0.268228-0.02655 0.20957 54.9056 8-0.28573 0.32039-0.036608 0.594 52.9047873-0.254983 0.282376-0.008427 0.20242 57.4777 9-0.304064 0.337563-0.020683 0.4778 56.2398723-0.274697 0.30420 0.0033 0.23468 59.03789 0-0.32005 0.352764-0.00977 0.48068 58.0263292-0.295057 0.39933 0.0850 0.25482 60.77866-0.3393 0.369866 0.00573 0.47483 60.252886-0.3420 0.3394 0.030498 0.29224 6.89667 2-0.36226 0.392070 0.0635 0.49674 6.8246742-0.33373 0.356224 0.042772 0.33458 62.53537 3-0.382347 0.40463 0.02729 0.52493 62.8485393-0.353582 0.37570 0.053690 0.38200 63.6337 4-0.40068 0.428008 0.0397 0.5554 63.7497430-0.375752 0.397462 0.064272 0.40967 64.5332 5-0.423597 0.449968 0.048426 0.56576 65.2028590-0.394363 0.44605 0.072957 0.47206 64.49488 Table 5. The Mote Carlo RMSEs ad bias values of HV m ad SHE m for the uiform distributio with H [g(x)] = 0 ad expoetial distributio with H [g(x)] = usig DRSS. SRS HV m m SHE SHEm RSS Q HV m m SHE Q SHEm m Bias RMSE Bias RMSE Bias RMSE Bias RMSE Table 5 cotiued o ext page 03
NEW ENTROPY ESTIMATORS 0-0.327408 0.369593-0.230787 0.285326 22.7999448-0.365854 0.425279-0.26738 0.34582 8.68373 2-0.26062 0.27873-0.07592 0.2826 56.2926262-0.288898 0.34068-0.0687 0.207273 39.4796 3-0.29604 0.3066-0.047 0.078474 74.364699-0.300393 0.35750-0.08027 0.8245 48.47335 4-0.346305 0.35272 0.029482 0.073990 79.0225453-0.322839 0.377437 0.05652 0.20436 46.63056 5-0.4042 0.409902 0.065862 0.0952 76.7942093-0.335248 0.39989 0.3478 0.252269 36.80462 20-0.308453 0.329353-0.260588 0.285042 3.4539537-0.32905 0.36324-0.278366 0.37530 2.5842 2-0.8923 0.202666-0.095093 0.956 4.005895-0.204908 0.24036-0.2945 0.68444 29.90729 3-0.82095 0.963-0.04993 0.07976 62.3483624-0.926 0.228320-0.050530 0.33863 4.37044 4-0.97693 0.204342-0.0039 0.052373 74.3699288-0.90904 0.229986-0.003685 0.26728 44.89752 5-0.220876 0.225845 0.027 0.049477 78.092497-0.97900 0.239789 0.036502 0.39896 4.6587 6-0.247733 0.25580 0.03533 0.05678 77.669926-0.207032 0.25002 0.07843 0.673 35.56585 7-0.275808 0.27899 0.053697 0.0680 75.5839509-0.209883 0.25852 0.8656 0.9227 25.545 8-0.303823 0.306608 0.07232 0.082285 73.628007-0.2870 0.27560 0.58069 0.224230 7.42893 9-0.333903 0.336495 0.08949 0.098489 70.730972-0.223692 0.278728 0.20003 0.262984 5.64858 0-0.363272 0.36573 0.06408 0.4566 68.674790-0.22826 0.29043 0.244783 0.283888 2.252859 30-0.298092 0.32767-0.267592 0.28326 9.44824742-0.3080 0.33033-0.278838 0.304383 8.050557 2-0.70745 0.8020-0.07748 0.2262 32.23090-0.8246 0.207785-0.8447 0.54790 25.50473 3-0.463 0.53646-0.05293 0.07039 54.862463-0.52039 0.80708-0.059074 0.4805 36.46933 4-0.4943 0.54886-0.02325 0.047458 69.3593998-0.45325 0.76699-0.09990 0.0239 42.9605 5-0.59888 0.64564-0.003052 0.03857 76.567024-0.46632 0.79028 0.009230 0.05307 4.7847 6-0.7449 0.78204 0.0302 0.03842 78.4398779-0.49443 0.84598 0.038407 0.5953 37.862 7-0.9854 0.94940 0.027534 0.046606 76.092309-0.50245 0.8858 0.068588 0.33307 29.556 8-0.209886 0.22509 0.04087 0.052754 75.756396-0.5344 0.94332 0.095598 0.5225 2.67270 9-0.22900 0.2326 0.052824 0.06955 73.2099230-0.57250 0.99936 0.23844 0.7522 2.4097 0-0.248006 0.249993 0.065446 0.072283 7.0859904-0.62854 0.20889 0.5295 0.98703 4.87785-0.267506 0.26988 0.07763 0.082922 69.955065-0.63540 0.2375 0.8229 0.207543 2.6496 2-0.287408 0.28908 0.08869 0.09339 67.6867877-0.67660 0.22482 0.207757 0.202062 8.768207 3-0.30760 0.308699 0.008 0.0480 66.0507485-0.7024 0.225764 0.239466 0.2883 6.48456 4-0.327370 0.328890 0.085 0.5458 64.8946456-0.70880 0.232977 0.26859 0.20502 9.646875 5-0.346997 0.348439 0.22960 0.26985 63.556033-0.69873 0.23573 0.299068 0.2072 0.397450 04
AMER IBRAHIM AL-OMARI Table 6. The Mote Carlo RMSEs ad bias values of HV m ad SHE m for the stadard ormal distributio ad H [g(x)] =.49. HV m SHE m m Bias RMSE Bias RMSE Q SHE m 0-0.4502 0.47262-0.36672 0.38539 8.43075 2-0.373395 0.42666-0.86378 0.256423 37.8685 3-0.42740 0.4599-0.43329 0.2898 52.30409 4-0.4929 0.58275-0.598 0.20253 60.99503 5-0.55435 0.57728-0.084253 0.800 68.62880 20-0.350703 0.38360-0.30304 0.340790.05804 2-0.245907 0.277809-0.52363 0.20055 27.95230 3-0.246496 0.27694-0.04439 0.6272 4.4468 4-0.262789 0.290545-0.078826 0.4772 49.6037 5-0.29340 0.37967-0.055774 0.38687 56.3832 6-0.3605 0.34597-0.03766 0.3424 60.70984 7-0.349246 0.37332-0.0299 0.32559 64.47397 8-0.384526 0.406764-0.00868 0.3458 67.0822 9-0.465 0.436696 0.006082 0.32054 69.76066 0-0.44590 0.46558 0.023744 0.34764 7.05074 30-0.32940 0.345223-0.29233 0.38084 7.86300 2-0.206709 0.23560-0.43028 0.77006 23.55934 3-0.8763 0.22774-0.094482 0.38090 35.005 4-0.90073 0.25577-0.066854 0.22350 43.24534 5-0.99843 0.224569-0.044224 0.88 50.20773 6-0.24636 0.23902-0.025579 0.08667 54.53663 7-0.2363 0.255278-0.0206 0.08224 57.60543 8-0.247340 0.27084 0.00734 0.09348 59.66269 9-0.268298 0.29044 0.0496 0.3895 60.86674 0-0.286538 0.30866 0.027278 0.88 6.5076-0.30530 0.326485 0.040250 0.23778 62.08769 2-0.324892 0.346062 0.05274 0.29747 62.50759 3-0.343097 0.363236 0.06548 0.35452 62.70964 4-0.369990 0.388586 0.070900 0.40756 63.77739 5-0.387740 0.40608 0.080947 0.4548 64.8990 05
NEW ENTROPY ESTIMATORS Table 7. The Mote Carlo RMSEs ad bias values of AHEj m, j = SRS, RSS, DRSS (Al-Omari, 204). AHESRS m Q AHESRS AHERSS m Q AHERSS AHEDRSS m Q AHEDRSS m Bias RMSE Bias RMSE Bias RMSE Uiform distributio with H [g(x)] = 0 0 2-0.298609 0.350332 22.554260-0.89664 0.228762 30.56686-0.45388 0.7659 36.799638 3-0.249056 0.298944 34.26897-0.54894 0.86380 45.88350-0.2280 0.44286 52.865580 20 4-0.4406 0.67779 38.97933-0.00304 0.8284 46.844385-0.082268 0.096978 52.54328 5-0.3379 0.57805 45.393360-0.09608 0.08584 55.63743-0.077708 0.09093 59.665700 30 7-0.092957 0.09089 53.28544-0.066053 0.07776 6.92883-0.05804 0.067650 65.29705 8-0.089259 0.0588 57.50446-0.06473 0.07688 65.573926-0.05642 0.065369 69.239420 Expoetial distributio with H [g(x)] = 0 2-0.323532 0.483573 5.432654-0.220406 0.35220 22.03853-0.7399 0.25460 26.75364 3-0.26573 0.443276 2.07470-0.59787 0.27697 3.44406-0.28545 0.223802 36.374698 20 4 0.443 0.279706 20.72050-0.098056 0.79990 30.57227-0.075338 0.7977 2.833938 5 0.8697 0.27887 24.8905-0.072456 0.7266 34.905333-0.05275 0.45269 39.47988 30 7-0.058550 0.20526 25.46009-0.02794 0.30283 36.82534-0.046556 0.5023 38.868929 8-0.036080 0.200329 28.6045-0.0063 0.36358 35.760488-0.00239 0.20306 38.092543 Stadard ormal distributio with H [g(x)] =.49 0 2-0.409842 0.496627 5.969354-0.308706 0.375690 20.262250-0.26249 0.36029 23.47728 3-0.386562 0.46847 24.826698-0.2933 0.353844 29.845470-0.254450 0.303820 33.825435 20 4-0.24227 0.279269 25.05573-0.68035 0.29922 3.027583-0.4807 0.94728 32.978368 5-0.205782 0.272804 3.074594-0.60392 0.23700 36.727225-0.45734 0.9755 39.693427 30 7-0.32038 0.96792 35.506368-0.05796 0.58654 40.85067-0.09557 0.43483 43.793433 8-0.2995 0.93509 39.72446-0.02504 0.57726 44.43270-0.094560 0.45579 46.297458 06
AMER IBRAHIM AL-OMARI Coclusio Three etropy estimators are suggested usig SRS, RSS, ad DRSS methods. The cosistecy of these estimators is proved as well as some properties are reported. Based o theoretical ad umerical comparisos the suggested etropy estimators are more efficiet tha Vasicek (976) ad Al-Omari (204) etropy estimators. However, the suggested estimators of etropy i this paper ca be exteded by cosiderig other samplig methods such as the multistage RSS ad media RSS methods. Ackowledgemets The author thaks the referees for their helpful ad valuable commets that substatially improved this paper. Refereces Al-Omari, A. I. (204). Estimatio of etropy usig radom samplig. Joural of Computatio ad Applied Mathematics, 26, 95-02. doi:0.06/j.cam.203.0.047 Al-Saleh, M. F. & Al-Kadiri, M. A. (2000). Double raked set samplig. Statistics ad Probability Letters, 48(2), 205-22. doi:0.06/s067-752(99)00206-0 Choi, B. (2008). Improvemet of goodess of fit test for ormal distributio based o etropy ad power compariso. Joural of Statistical Computatio ad Simulatio, 78(9), 78-788. doi:0.080/009496507029945 Choi, B., Kim, K., & Sog, S. H. (2004). Goodess of fit test for expoetiality based o Kullback-Leibler iformatio. Commuicatio i Statistics-Simulatio ad Computatio, 33(2), 525 536. doi:0.08/sac-20037250 Goria, M. N., Leoeko, N. N., Mergel, V. V., & Novi Iverardi, P. L. (2005). A ew class of radom vector etropy estimators ad its applicatios i testig statistical hypotheses. Joural of Noparametric Statistics, 7(3), 277-297. doi:0.080/0485250420002685 07
NEW ENTROPY ESTIMATORS Correa, J. C. (995). A ew estimator of etropy. Commuicatio i Statistics-Theory Methods, 24(0), 2439-2449. doi:0.080/036092950883626 Ebrahimi, N., Pflughoeft, K., & Soofi, E. S. (994). Two measures of sample etropy. Statistics & Probability Letters, 20(3), 225-234. doi:0.06/067-752(94)90046-9 Mahdizadeh, M. (202). O the use of raked set samples i etropy based test of fit for the Laplace distributio. Revista Colombiaa de Estadística, 35(3), 443-455. McItyre, G. A. (952). A method for ubiased selective samplig usig raked sets. Australia Joural of Agricultural Research, 3(4), 385-390. doi:0.07/ar9520385 Noughabi, H. A. & Noughabi, R. A. (203). O the etropy estimators. Joural of Statistical Computatio ad Simulatio, 83(4), 784-792. doi:0.080/00949655.20.637039 Noughabi, H. A. & Arghami, N. R. (200). A ew estimator of etropy. Joural of the Iraia Statistical Society, 9(), 53-64. Park, S. & Park, D. (2003). Correctig momets for goodess of fit tests based o two etropy estimates. Joural of Statistical Computatio ad Simulatio, 73(9), 685-694. doi:0.080/009496503000070367 Shao, C. E. (948a). A mathematical theory of commuicatios. Bell System Techical Joural 27(3), 379-423. doi:0.002/j.538-7305.948.tb0338.x Shao, C. E. (948b). A mathematical theory of commuicatios. Bell System Techical Joural 27(4), 623-656. doi:0.002/j.538-7305.948.tb0097.x Takahasi, K. & Wakimoto, K. (968). O the ubiased estimates of the populatio mea based o the sample stratified by meas of orderig. Aals of the Istitute of Statistical Mathematics, 20(), -3. doi:0.007/bf029622 Theil, J. (980). The etropy of maximum etropy distributio. Ecoomics Letters, 5(2), 45 48. doi:0.06/065-765(80)90089-0 Va Es, B. (992). Estimatig fuctioals related to a desity by class of statistics based o spacigs. Scadiavia Joural of Statistics, 9(), 6-72. Vasicek, O. (976). A test for ormality based o sample etropy. Joural of the Royal Statistical Society, B, 38, 54-59. 08
AMER IBRAHIM AL-OMARI Wieczorkowski, R. & Grzegorzewsky, P. (999). Etropy estimators - improvemets ad comparisos. Commuicatio i Statistics-Simulatio ad Computatio, 28(2), 54-567. doi:0.080/03609990883564 09