Surveying the Variance Reduction Methods

Size: px

Start display at page:

Download "Surveying the Variance Reduction Methods"

Augustine Park
5 years ago
Views:

Iteratioal Research Joural of Applied ad Basic Scieces 2013 Available olie at www.irjabs.

1 Iteratioal Research Joural of Applied ad Basic Scieces 2013 Available olie at ISSN X / Vol, 7 (7): Sciece Explorer Publicatios Surveyig the Variace Reductio Methods Arash Mirtorabi 1*, Gholamhossei Gholami 2 1. Departmet of Statistics, Sciece ad Research Brach, Islamic Azad Uiversity, Fars, Ira. 2. Departmet of Mathematics, Faculty of scieces, Urmia Uiversity, Urmia, Ira. Correspodig Author A.mirtorabi@hotmail.com ABSTRACT: The Mote Carlo method is the oe used for itegral estimatio accompaied by radom umbers, the mai idea of this method is covertig the itegral to expectacy based o the defiite probable desity fuctio, geeratig radom sample from this desity fuctio ad usig large umbers rule to estimate the expectacy. I Mote Carlo method, is estimated by geeratig sequeces of radom variables whose expectacy is. The efficiecy of this method icreases whe the radom variable is low i variace. Those methods geeratig radom variable by expectacy ad rather low variace are called variace reductio. I this article, we focus o variace reductio methods. Keywords: importacesamplig, Rao-Blackwellisatio, Cotrol variables, Atithetic Variates, commo radom umbers. INTRODUCTION The Mote Carlo method is the oe estimatig the itegrals usig radom umbers. The mai of this method is the covertig of itegrals ti expectacy based o thedefiite probable desity fuctio, geeratig radom sample from desity fuctio, ad usig the large umber rule for estimatio of this expectacy(robert ad Casella, 2004). Suppose, we ted to estimate the followig itegral: θ = g(x)f(x) dx Oe would defie the Mote Carlo algorithm as follows: 1. Geerate oe radom sample from desity fuctio. 2. substitutig these values i g fuctio, calculate the value of g x Accordig to the large umber rule, the abovevalue is the estimatio of expectacy. θ = E g x = 1 g x I Mote Carlo method, we estimated through geeratig sequece of radom variables whoseexpectacy is. The efficiecy of this method icreases whe the radom variable is low i variace. Those methods which are able to geerate radom variable of expectacy ad low variace are called Variace Reductio. VARIANCE REDUCTION METHODS Atithetic Variates Atithetic Variates are the oes icludig egative correlatio. Suppose, Y ad Y are two asymmetric estimators, the we itroduce the ew estimator as follow: θ = Y + Y 2 Var(Y ) + Var(Y ) + 2Cov(Y, Y ) Var θ = 4

2 Itl. Res. J. Appl. Basic. Sci. Vol., 7 (7), , 2013 If Y ad Y are idepedet, the Var θ =. This estimator is lower i variace compared to both Y ad Y estimators. If Y ad Y are depedet, ad if they are positively correlated, the the variace icreases. If they are egatively correlated, the variace correlatio decreases. So, oe would claim that we could geerate asymmetric estimatio usig the Atithetic Variates method which would decrease the variace(hoff, 2009). Example 2.1a fuctio of radom variables i. i. d is the oe of U(0,1) so that θ = E(Y) = E[g(U)] The Mote Carlo method estimates 2 icludig θ sample value: θ = Y = also, we geerate fuctio Y = g(1 U ) for geeratig Atithetic Variates. If U ~U(0,1), the 1 U will be U(0,1), so E[Y ] = E[Y ] = θ. Oe would defie the asymmetric estimator as follow: θ = E[Z ] Where Z =. Accordig to the large umbers rule we coclude that whe teds to ifiite, the θ compare the estimator variace: Var θ = Var Y = Var(Y) 2 2 Var θ = Var Z = Var(Y) 2 = Var θ + Y 2 = Var(Z) + Cov Y, Y 2 Cov(Y, Y) 2 = Var Y + Y 4 equals θ. Now, we So, Var θ, < Var θ if ad oly if Cov Y, Y < 0. Example 2.2. Suppose, g(x) = exp {x }. Also, assume that the radom sample from U(0,1) distributio is available. Theestimatio of such itegral usig Mote Carlo method ad the Atithetic Variates for differet values will be as follow: I = exp {x }d(x) Figure1. IEstimator usig Mote Carlo method (turquoise) ad Atithetic Variates (violet) with 1000 iteratios 428

3 Itl. Res. J. Appl. Basic. Sci. Vol., 7 (7), , 2013 Now, we discuss the cases which guaratee the variace reductio. I case of m = 1, oe sufficiet coditio for guarateeig variace reductio is that u should be oe mootoe distributio o [0,1]. I geeral, whe m > 1, E[g(U)] = θ ad U = (U,, U ), oe would defie the theorem whe X,, X ady = g(x,., X ) is vector of idepedet radom variables ad, E[Y] = θ occurs. If oe is able to use iverse coversio for geeratig X, the it is possible to use Atithetic Variates. Assume F (. ) is a distributio of X. If U ~U(0,1), the F (. ) has equal distributio by X. So, oe would geerate Y sample through geeratig radom variables of i. i. d U,, U by U(0,1) distributio ad the followig attribute Y = g(f (U ),, F (U )) Sice the distributio of each radom variable is o-decreasig, the above theorem(hoff, 2009). Theorem 2.3 if g(u,, u )is a mootoic distributio from each member of it i [0,1] iterval, the for oe set of U = (U,, U )from i. i. d radom variables by U(0,1)distributio, we have the followig Cov h(u), h(1 U) < 0 So that Cov g(u), g(1 U) = Cov g(u,, U ), g(1 U,, 1 U ). Note that the theorem is the sufficiet coditio of variace reductio but ot ecessary which meas the possibility of variace reductio eve if the theorem coditio is ot met. 1. if fis symmetric aroud the (μ) mea, we use Y = 2μ X covertig 2. if X = F (U ), we use Y = F (1 U ) covertig Example 2.4 suppose, we ted to estimate θ = E[X ] so that X~N(2,1) We kow that θ = 5. The Mote Carlo estimatios ad the Atithetic Variates will be as follows for differet values: Figure 2.θEstimator usig Mote Carlo (blue) method ad Atithetic Variates (violet) with 2000 iteratios commo radom umbers This method is usually used whe the aim of estimatio is the differece betwee two depedet quatities, I geeral terms, it is possible to use this method whe several systems accompaied by commo attributes eed simulatio where this simulatio is doe for the purpose of estimatig oe fuctio which depeds o the obtaied resposes from each system(hoff, 2009). Example 2.5 assume a queuig system i which customers eterig follow N(t)Poisso process. The operator of the system eeds the istallatio of oe server for deliver services to customers which ca choose betwee two N ad M possible servers. We show the M service time of customers by i ad the total waitig time i system for all the customers arrivig before S time by X i the evet chose as T which is as follows: X = W 429

4 Itl. Res. J. Appl. Basic. Sci. Vol., 7 (7), , 2013 So that W is the total service time of i customer i system. Cosequetly, W = S + Q so that the Q waitig time before deliverig service to i customer. Similarly, W,S, ad Q are defied for N. The operator teds to estimate θ = E[X ] E[X ] Probably, oe possible way to estimate θ is the estimatio of θ = E[X ]ad θ = E[X ] idepedetly, the obtaiig θ = θ θ The θ variaces is as follow Var θ = Var θ + Var θ It is possible that oe do somethig better through calculatig θ = E[X ]ad θ = E[X ] as depedet. I this case, the estimator variace is as follow: Var θ = Var θ + Var θ 2Cov(θ, θ ) So, if oe is able to arrage ad set is as Cov θ, θ > 0, it leafs to variace reductio. Cotrol variables Assume that we ted to estimate E[g(X)] i which X = (x,, X ). Also, suppose that the f value is kow for oe average f(x) kow fuctio like i E[f(x)] = μ. The, we ca use W = g(x) + a[f(x) μ] For each a costat (fixed) umber as the E[g(X)] estimator. Now, Var(W) = Var[g(X)] + a Var[f(X)] + 2a Cov(f(X), g(x) ) The simple calculatios show that the above value decreases whe Cov[f(X), g(x)] a = Var[F(X)] For such value of a we have Cov[f(X), g(x)] Var(W) = Var[g(X)] Var[F(X)] Which leads to variace reductio(sheldo Ross, 2002). Rao-Blackwellisatio I this method, if we are able to calculate E[E(g(X) Y)] for a radom variabley istead of E[g(X)], we ca coclude from the coditioal variace equatio that Var g(x) = Var[E(g(X) Y)] + E[Var(g(X) Y)] Var g(x) E[Var(g(X) Y)] Sice E[g(X)] = E[E(g(X) Y)], E[g(X)] is better estimator for θ estimatio(gholami, 2008). Example 2.6: We have E[h(X)] = E[exp ( X )] whe X~ (v, μ, σ ). Whe X y~n(v. yσ ) ad Y ~ a(, ), we have δ = 1 m exp ( X ) Oe would improve the experimetal average usig the followig samples (X, Y ),, (X, Y ) So, δ = 1 exp ( X Y ) = 1 1 m m 2σ Y + 1 Is a coditioal expectacy. Also,δ is more accurate tha δ. 430

5 Itl. Res. J. Appl. Basic. Sci. Vol., 7 (7), , 2013 Figure 3. E[exp( X )] Estimator usig Mote Carlo method (δ ),(spot) ad Rao-Blackwellisatio (δ ) (lie) with 1000 iteratios ad (v, μ, σ ) = (4.6,0,1) importace samplig Oe of the methods o Mote Carlo to geerate radom sample is importace samplig. I this method, other desities ad ot the mai desity fuctio are sampled which hare called importacefuctios simulatio from π distributio is ot always optimum ad usig importace distributios lead to better results which ca later lead to lower variaces. The importace samplig is i the way that we geerate (simulate) oe sample from g distributio ad we calculate the followig quatity: h(x ) (1) As we kow, this statistics is based o the geerated sample from g distributio, which is coverget accordig to the large umbers rule E [ h(x)π(x) ] g(x) Ad oe would coclude that 1 π x g x h x. E [h(x)] I the fowlig, we show the ratio of for w x ad we call them importamce poits. Oe fudametal coditio i choosig g is that this selectio leads to fiite variace for estimator(robert ad Casella, 2004). The variace of this estimator is fiite whe E[h (X)w (X)] = E [h (X)w(X)] = h (x) π (x) g(x) dx < oe alterative estimator forμwhich icludes fiite variace ad sometimes leads to more cosistet estimatio is μ which is defied as μ = (2) Sice whe m is 1 accordig to Large umbers rule, this estimator will be coverget to E [h(x)]. Although μ icludes little asymmetrical status, its quadratic mea of error is lower tha the μ asymmetric estimator quadraticmea of error. It also improves the variace, for showig the symmetric status of this estimator we have 431

6 Itl. Res. J. Appl. Basic. Sci. Vol., 7 (7), , 2013 μ = w X h X w X = w X h X w X E(μ) = E w X h X w X = me w X h X μ w X Oe cosiderable advatage of usig symmetric estimator istead of the asymmetric oe is that i asymmetric estimator, we should have the ratio of as accurate while i symmetric estimator it is required that we have this ratio as idefiite to some extet(robert ad Casella, 2004). Example 2.7 suppose, X~ (v, θ, σ ) is accompaied by the followig desity fuctio f(x) = The aim is to calculate the below itegral Γ((v + 1) 2 ) σ vπ Γ(v 2) 1 + (x θ) vσ I = x f(x)dx. Assume θ = 0, σ = 1. We use impotece samplig fromsolvig the above itegral. I doig so, we assume the followig importace fuctios: * (v, 0,1) distributio which is aterior * C(0,1) distributio * U(1, ) distributio. Figure 4. The covergece of three I estimators i iteratios: aterior distributio samplig (dotted) by importace samplig (dot) ad importace samplig of mootoe distributio (lie) have bee show REFERENCES Gholami GH Chage-poit Problems i Regressio: A Bayesia Approach. Ph. D.Thesis. Hoff PD A First Course i Bayesia Statistical Methods. Spriger. Robert CP, Casella G Mote Carlo Statistical Methods. Spriger, SecodEditio Sheldo Ross M A First Course i Probability. 6thEditio. 432

Surveying the Variance Reduction Methods

Surveying the Variance Reduction Methods Available olie at www.scizer.co Austria Joural of Matheatics ad Statistics, Vol 1, Issue 1, (2017): 10-15 ISSN 0000-0000 Surveyig the Variace Reductio Methods Arash Mirtorabi *1, Gholahossei Gholai 2 1.