ON THE RESAMPLING METHOD IN SAMPLE MEDIAN ESTIMATION

Size: px

Start display at page:

Download "ON THE RESAMPLING METHOD IN SAMPLE MEDIAN ESTIMATION"

Scot Montgomery
5 years ago
Views:

1 ACTA UNIVERSITATIS LODZIENSIS FOLIA OECONOMICA 3(302), 2014 * ON THE RESAMPLING METHOD IN SAMPLE MEDIAN ESTIMATION Abstract. Bootstrap is oe of the resamplig statistical methods. This method was proposed by B. Efro. The mai idea of bootstrap is to treat the origial sample of values as a stad-i for the populatio ad to resample with replacemet from it repeatedly. Bootstrap allows estimatio of the samplig distributio of almost ay statistics usig oly very simple methods. This paper presets a modificatio of a resamplig procedure based o bootstrap samplig. The proposal leads to samplig from populatio with desity fuctio f(x), where f(x) is estimated based o the kerel estimatio. The properties of the method were aalyzed i the media estimatio i Mote Carlo study. The proposal could be useful for the parameters estimatio i the case of a small sample. This method could be used i quality cotrol procedures such as cotrol charts or i the acceptace samplig. Keywords: bootstrap, kerel estimatio, small sample. I. INTRODUCTION Bootstrap was itroduced by Bradley Efro i 1979 (B. Efro, 1979). The mai idea of bootstrap is to treat the origial sample of values as a stad-i for the populatio ad to resample with replacemet from it repeatedly. Bootstrap allows estimatio of the samplig distributio of almost ay statistic usig oly very simple methods. I the case of small samples the resamplig is based oly o few elemets. K. Pruska (2007) uses bootstrap ad jackkife methods for the estimatio bias ad variace of a sample media ad cocludes that these methods do ot give good results. Aother look at the resamplig method is preseted i the paper. This proposal of resamplig is based o the kerel desity estimatio. It leads to samplig from the populatio with desity fuctio f(x), where f(x) is estimated based o the kerel estimatio. This method could be used i moitorig the productio processes especially of small samples. G.J. Jaacek ad S.E. Meikle (1997) proposed cotrol charts based o * Associate Professor, Departmet of Statistics, Katowice Uiversity of Ecoomics, grzegorz.koczak@ue.katowice.pl. [51]

2 52 a sample media. They suggested moitorig a media characteristic for oormal radom variables. The results of the proposed method have bee compared with the bootstrap method i the Mote Carlo study. II. CLASSICAL BOOTSTRAP PROCEDURE Bootstrap resamplig is usually used for hypothesis testig, variace estimatio or costruct the cofidece itervals. This method is recommeded for the followig situatios: theoretical distributio of a statistic is complicated or ukow, aalytical calculatio could ot be used, the size of the sample is isufficiet for statistical iferece, small sample is available ad power calculatio is expected. Let X 1, X 2,, X be a radom sample of size take from the distributio F. ad x 1, x 2,, x be the realizatio of this sample. Let X B be the radom variable for which the probability distributio fuctio has the followig form (K. Pruska, 2007) 1 P( X B xi ) for i = 1, 2,, (1) The above distributio is called the bootstrap distributio. Let T T( X1, X 2,..., X ) be a estimator of parameter of the populatio. The bootstrap estimatio of the parameter leads to the geerate -elemet sequeces of the pseudoradom umbers from the bootstrap distributio. Let N be a umber of these sequeces. The sequeces are called bootstrap samples ad * * * ca be deoted as X1 k, X 2k,..., X k (k = 1, 2,, N). The realizatio of the * * * bootstrap sample ca be deoted as x1 k, x2k,..., xk. The bootstrap estimator of parameter has the followig form: 1 N * B T k N k1 T (2) where * * k Tk T ( X1k, X 2 *,..., X * k ) The stadard error of estimator T could be estimated by (B. Efro, R. Tibshirai, 1993)

3 O the Resamplig Method i Sample Media Estimatio 53 D( T ) 1 N 1 N k 1 ( T * k T ) B 2 (3) The mai idea of the bootstrap is samplig from the origial sample. The modificatio based o the samplig from the estimated distributio will be cosidered. III. KERNEL ESTIMATION Let X 1, X 2,, X be a radom sample of size take from the populatio with the ukow desity fuctio f(x). Let us assume that f ( x) dx 2. To estimate the desity f(x) the kerel method ca be used. The kerel desity estimator of f(x) ca be writte as follows (S.J. Sheather, 2004, Cz. Domański, K. Pruska, 2000) a f( x; a) K[ a( x X i )] (4) i1 where ( a ) is a sequece of positive umbers divergig to ifiity ad IN a such that lim 0 ad K(x) is a kerel fuctio satisfyig the followig coditios: K( x) 2 dx K( x) K( x) for x (, ) sup x K ( x) dx 1 K( x) A x i K( x) dx 0 for i = 1, 2,, s 1

4 54 x s K ( x) dx 0 x s K( x) dx where s is a fixed atural umber. Let the sequece a be give by the formula 1 a cost, where h > 0 is the smoothig parameter for = 1, 2,. The h kerel estimatio could be writte as follows 1 x X i f( x) K h (5) i1 h Various fuctios K(x) could be used as a kerel. Cz. Domański ad K. Pruska (2000) write of gaussia kerels, triagual kerels, rectagual kerels, Cauchy kerels ad Epaeczikow kerels. The form of the estimated kerel strogly depeds o the form of the K(x). The kerel estimatio for the fixed = 5 elemet samples is show i Fig 1. Figure 1. The kerel estimatio for the fixed dataset ( = 5) for various kerel types (rectagular, triagular ad gaussia) The form of the estimator depeds o the smoothig parameter (h). The kerel estimatio for the fixed = 10 elemet samples for various values of the smoothig parameters is show i Fig 2.

5 O the Resamplig Method i Sample Media Estimatio 55 Figure 2. The kerel estimatio for the fixed dataset ( = 10) for gaussia kerel ad for various smoothig parameter (h = 0.4; 0.8 ad 1.2) IV. ANOTHER WAY OF RESAMPLING Let us cosider a sample of size take from the cotiuous distributio F(x) with the desity fuctio f(x). The mai idea of the bootstrap is samplig from the bootstrap distributio give by (1). Let us cosider aother way of resamplig tha the bootstrap method. Let us cosider the elemet samples take from distributio with desity f (x), where f (x) is the kerel estimatio give by (5) of the desity fuctio f(x). The procedure of samplig from distributio with kow desity f (x) is a idirect method followig S. Ulam ad J. vo Neuma (Y. Rubistei, D.P. Kroese, 2008; R. Wieczorkowski, R. Zieliński, 1997). It is called a acceptace-rejectio method ad it ca be doe usig the followig steps: 1. Geerate radom value x from uiform distributio o the [a, b], where [a, b] is the domai of f(x). 2. Geerate radom value y from the uiform distributio o [0, c] where c max f ( x; a ) x[ a, b] 3. If y f (x) the retur z = x. Otherwise retur to step 1. The bootstrap is based o samplig from the discrete distributio. The proposal is a two step procedure estimatig the desity based o the sample samplig from the estimated desity The procedures will be compared i the sample media estimatio. V. MEDIAN ESTIMATION COMPARING TWO METHODS The above described method of resamplig was compared to the bootstrap i the series of computer simulatios. The problem of the media estimatio was cosidered. Let be the size of a populatio sample. The sample media is the

6 56 observatio o the (+1)/2 positio for the odd i a odecreasig sequece ad the average of two observatios with umbers /2 ad /2 +1 for the eve. There were 4 theoretical populatios cosidered i the Mote Carlo study a) Normal N(10, 1) b) Log-ormal LN (0, 1) c) Beta B(2, 2) d) Beta B(0.2, 0.2) e) Expoetial E(1) f) Uiform[0, 10] Table 1. Parameters of distributios used i Mote Carlo experimets Distributio Simulatio parameters Normal N(μ,) Log-ormal LN(μ,) Beta B(α,β) Beta B(α, β) Expoetial E() Uiform U[a, b] Parameters Mea Media μ = 10, = 1 10 μ = 0, = 1 e 1 α = 10, β = α = 10, β = = 1 1 l2 a = 0, b = Source: Ow preparatio. The details of the aalyzed populatios are described i table 1. The graphical view of desities of these radom variables is preseted i Fig. 2. The study icluded the followig steps: 1. The sample of size ( = 5, 10, 20) was take from the cosidered distributios. 2. The media was estimated usig the bootstrap method ad the above described proposed methods. 3. Steps 1 ad 2 were repeated N sim = 1000 times. The average bias ad stadard error of the aalyzed estimators were calculated. The results of the Mote Carlo study are preseted i table 2.

O the Resamplig Method i Sample Media Estimatio 57 Figure 2.

log-ormal, expoetial, beta, beta ad uiform) Source: Ow preparatio i

Sample media estimators for the right skew distributios (log-ormal

The bias is smaller for the bootstrap method.

7 O the Resamplig Method i Sample Media Estimatio 57 Figure 2. Desities of distributios aalyzed i the Mote Carlo study (ormal, log-ormal, expoetial, beta, beta ad uiform) Source: Ow preparatio i Mathematica. Sample media estimators for the right skew distributios (log-ormal distributio ad expoetial distributio) are biased. The bias is smaller for the bootstrap method. The stadard error of the media estimatio i the proposed methods is sigificatly smaller tha i the bootstrap estimatio i cases of beta(0.2, 0.2) ad uiform (0, 10) distributios.

8 58 Table 2. Bias estimatio ad the stadard error of the estimator Distributio Normal N(, ) Log-ormal LN(0, ) Beta B(2,2) Beta B(0.2,0.2) Expoetial E() Uiform U[0, 10] Sample size Method Bootstrap Kerel Rectagular Triagle Gaussia

9 O the Resamplig Method i Sample Media Estimatio 59 Bootstrap was origially itroduced to estimate the variace of complex estimators. Table 3 presets the average estimated variace of media estimators calculated usig formula (3). The colum Simulatio presets estimated variace of a sample media estimator based o N sim = 1000 samples replicatios. Estimated values of a variace media estimator by the bootstrap method ad three aalyzed proposals are preseted i the last 4 colums of table 3. Table 3. Variace of the media estimator Mote Carlo results Method Distributio Sample size Simulatio Bootstrap Kerel Rectagular Triagle Gaussia Normal N(10, 1) Log-ormal LN(0, ) Beta B(2,2) Beta B(0.2,0.2) Expoetial E() Uiform U[0, 10]

10 60 The bootstrap estimatio of variace of a estimator leads to best results for the ormal, log-ormal ad expoetial distributios. I the case of beta distributio the proposal leads to better variace estimatio. V. CONCLUDING REMARKS Bootstrap is oe of the most commoly used resamplig techiques. This method treats the origial sample of values as a stad-i for the populatio ad results i resamplig with replacemet from it repeatedly. The modificatio of this method was proposed i the paper. The proposal is based o samplig from the estimated distributio. The Mote Carlo study was used to aalyze the properties of the proposal. The results obtaied due to modificatio are similar to the origial bootstrap. I the case of two distributios, the stadard error of a estimator was smaller i the proposed method tha i bootstrap. The proposal could be used i moitorig of the productio processes. The method ca be especially useful i moitorig the media i o-ormal processes where the samples are small. ACKNOWLEDGEMENTS The research was supported by Polish Natioal Sciece Cetre grat DEC- 2011/03/B/HS4/ REFERENCES Domański Cz., Pruska K. (2000) Nieklasycze metody statystycze, PWE Warszawa. Efro B. Bootstrap Methods: Aother Look at the Jackkife, Aals of Statistics 7, 1 26, Efro B., Tibshirai R. (1993) A Itroductio to the Bootstrap, Chapma & Hall. New York. Jaacek, G. J. ad Meikle, S. E. (1997), Cotrol charts based o medias. Joural of the Royal Statistical Society: Series D (The Statisticia), 46: Pruska K. (2007) Estimatios of Bias ad Variace of Sample Media by Jackkife ad Bootstrap Method, [i:] Acta Uiversitatis Lodziesis, Folia Oecoomica, 206. s Rubistei R.Y., Kroese D.P. (2008) Simulatio ad the Mote Carlo Method, Joh Wiley & Sos, Ic. New Jersey. Sheather S.J. (2004) Desity Estimatio, Statistical Sciece vol. 19, o. 4, s Wieczorkowski R., Zieliński R. (1997) Komputerowe geeratory liczb losowych, Wydawictwa Naukowo-Techicze, Warszawa.

11 O the Resamplig Method i Sample Media Estimatio 61 O ESTYMACJI MEDIANY METODĄ REPRÓBKOWANIA DLA MAŁYCH PRÓB Najczęściej wykorzystywaą w badaiach statystyczych metodą repróbkowaia jest bootstrap. Metoda ta prowadzi do wielokrotego zwrotego pobieraia próbki losowej z próby pierwotej. Zaletą tej metody jest fakt, że może być wykorzystaa do wioskowaia o parametrach populacji awet wówczas, gdy ie jest zay jej rozkład. W opracowaiu przedstawioo propozycję modyfikacji metody bootstrap. Repróbkowaie przeprowadza się z rozkładu, który otrzymuje się metodą estymacji jądrowej fukcji gęstości. Propoowaa metoda została porówaa z klasyczą metodą bootstrap z wykorzystaiem symulacji komputerowych. W badaiach porówawczych skocetrowao się a estymacji parametrów populacji a podstawie małych prób.

12 62

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.

Resampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n. Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator