Surveying the Variance Reduction Methods

Transcription

1 Available olie at Austria Joural of Matheatics ad Statistics, Vol 1, Issue 1, (2017): ISSN Surveyig the Variace Reductio Methods Arash Mirtorabi *1, Gholahossei Gholai 2 1. Departet of Statistics, Sciece ad Research Brach, Islaic Azad Uiversity, Fars, Ira. 2. Departet of Matheatics, Faculty of scieces, Uria Uiversity, Uria, Ira. Correspodig Author eail: A.irtorabi@hotail.co Abstract: The Mote Carlo ethod is the oe used for itegral estiatio accopaied by rado ubers, the ai idea of this ethod is covertig the itegral to expectacy based o the defiite probable desity fuctio, geeratig rado saple fro this desity fuctio ad usig large ubers rule to estiate the expectacy. I Mote Carlo ethod, θ is estiated by geeratig sequeces of rado variables whose expectacy is θ. The efficiecy of this ethod icreases whe the rado variable is low i variace. Those ethods geeratig rado variable by θ expectacy ad rather low variace are called variace reductio. I this article, we focus o variace reductio ethods. Keywords: iportace saplig, Rao-Blackwellisatio, Cotrol variables, Atithetic Variates, coo rado ubers. Itroductio The Mote Carlo ethod is the oe estiatig the itegrals usig rado ubers. The ai of this ethod is the covertig of itegrals ti expectacy based o the defiite probable desity fuctio, geeratig rado saple fro desity fuctio, ad usig the large uber rule for estiatio of this expectacy [1]. Suppose, we ted to estiate the followig itegral: θ = R M g(x)f(x) dx Oe would defie the Mote Carlo algorith as follows: 1. Geerate oe rado saple fro f desity fuctio. 2. substitutig these values i g fuctio, calculate the value of 1 g(x j ) Accordig to the large uber rule, the above value is the estiatio of expectacy. θ = E j g(x j ) = 1 g(x j ) I Mote Carlo ethod, we estiated θ through geeratig sequece of rado variables whose expectacy is θ. The efficiecy of this ethod icreases whe the rado variable is low i variace. Those ethods which are able to geerate rado variable of θ expectacy ad low variace are called Variace Reductio. Variace Reductio Methods Atithetic Variates Atithetic Variates are the oes icludig egative correlatio. Suppose, Y 1 ad Y 2 are two asyetric estiators, the we itroduce the ew estiator as follow: θ = Y 1 + Y 2 2 Var(θ ) = Var(Y 1) + Var(Y 2 ) + 2Cov(Y 1, Y 2 ) 4 If Y 1 ad Y 2 are idepedet, the Var(θ ) = Var(Y). This estiator is lower i variace copared to both Y 2 1 ad Y 2 estiators. If Y 1 ad Y 2 are depedet, ad if they are positively correlated, the the variace icreases. If they are egatively correlated, the variace correlatio decreases. So, oe would clai that we could geerate asyetric estiatio usig the Atithetic Variates ethod which would decrease the variace [2]. Exaple 2.1 a fuctio of rado variables i. i. d is the oe of U(0,1) so that θ = E(Y) = E[g(U)] The Mote Carlo ethod estiates icludig θ saple value:

2 Austria Joural of Matheatics ad Statistics, Vol 1, Issue 1, (2017): θ = Y = Y i i=1 also, we geerate fuctio Ỹ i = g(1 U i ) for geeratig Atithetic Variates. If U i ~U(0,1), the 1 U i will be U(0,1), so E[Y i ] = E[Ỹ i ] = θ. Oe would defie the asyetric estiator as follow: θ = E[Z i ] Where Z i = Ỹ i+y i. 2 Accordig to the large ubers rule we coclude that whe teds to ifiite, the θ equals θ. Now, we copare the estiator variace: Var(θ ) = Var ( i=1 Y i ) = Var(Y) i=1 Z i Var(θ ) = Var ( = Var(Y) Cov(Y, Ỹ) + = Var(θ ) + Cov(Y, Ỹ) ) = Var(Z) = Var(Y + Ỹ) 4 So, Var(θ,a ) < Var(θ ) if ad oly if Cov(Y, Ỹ) < 0. Exaple 2.2. Suppose, g(x) = exp {x 2 }. Also, assue that the rado saple fro U(0,1) distributio is available. Theestiatio of such itegral usig Mote Carlo ethod ad the Atithetic Variates for differet values will be as follow: I = exp {x 2 }d(x) Figure1. Î Estiator usig Mote Carlo ethod (turquoise) ad Atithetic Variates (violet) with 1000 iteratios Now, we discuss the cases which guaratee the variace reductio. I case of = 1, oe sufficiet coditio for guarateeig variace reductio is that u should be oe ootoe distributio o [0,1]. I geeral, whe > 1, E[g(U)] = θ ad U = (U 1,, U ), oe would defie the theore whe X 1,, X ad Y = g(x 1,., X ) is vector of idepedet rado variables ad, E[Y] = θ occurs. If oe is able to use iverse coversio for geeratig X i, the it is possible to use Atithetic Variates. Assue F i (. ) is a distributio of X i. If U i ~U(0,1), the F i 1 (. ) has equal distributio by X i. So, oe would geerate Y saple through geeratig rado variables of i. i. d U 1,, U by U(0,1) distributio ad the followig attribute Y = g(f i 1 (U 1 ),, F i 1 (U )) Sice the distributio of each rado variable is o-decreasig, the above theore [2]. 11

3 Austria Joural of Matheatics ad Statistics, Vol 1, Issue 1, (2017): Theore 2.3 if g(u 1,, u ) is a ootoic distributio fro each eber of it i [0,1] iterval, the for oe set of U = (U 1,, U ) fro i. i. d rado variables by U(0,1) distributio, we have the followig Cov(h(U), h(1 U)) < 0 So that Cov(g(U), g(1 U)) = Cov(g(U 1,, U ), g(1 U 1,, 1 U )). Note that the theore is the sufficiet coditio of variace reductio but ot ecessary which eas the possibility of variace reductio eve if the theore coditio is ot et. 1. if f is syetric aroud the (μ) ea, we use Y i = 2μ X i covertig 2. if X i = F 1 (U i ), we use Y i = F 1 (1 U i ) covertig Exaple 2.4 suppose, we ted to estiate θ = E[X 2 ] so that X~N(2,1) We kow that θ = 5. The Mote Carlo estiatios ad the Atithetic Variates will be as follows for differet values: Figure 2. θ Estiator usig Mote Carlo (blue) ethod ad Atithetic Variates (violet) with 2000 iteratios Coo rado ubers This ethod is usually used whe the ai of estiatio is the differece betwee two depedet quatities, I geeral ters, it is possible to use this ethod whe several systes accopaied by coo attributes eed siulatio where this siulatio is doe for the purpose of estiatig oe fuctio which depeds o the obtaied resposes fro each syste [2]. Exaple 2.5 assue a queuig syste i which custoers eterig follow N(t) Poisso process. The operator of the syste eeds the istallatio of oe server for deliver services to custoers which ca choose betwee two N ad M possible servers. We show the M service tie of custoers by i ad the total waitig tie i syste for all the custoers arrivig before S i tie by X i the evet chose as T which is as follows: N(t) X = W i i=1 So that W i is the total service tie of i custoer i syste. Cosequetly, W i = S i + Q i so that the Q i waitig tie before deliverig service to i custoer. Siilarly, W i, S i, ad Q i are defied for N. The operator teds to estiate θ = E[X ] E[X ] Probably, oe possible way to estiate θ is the estiatio of θ = E[X ] ad θ = E[X ] idepedetly, the obtaiig θ = θ θ The θ variaces is as follow Var(θ ) = Var(θ ) + Var(θ ) It is possible that oe do soethig better through calculatig θ = E[X ] ad θ = E[X ] as depedet. I this case, the estiator variace is as follow: Var(θ ) = Var(θ ) + Var(θ ) 2Cov(θ, θ ) So, if oe is able to arrage ad set is as Cov(θ, θ ) > 0, it leafs to variace reductio. 12

4 Austria Joural of Matheatics ad Statistics, Vol 1, Issue 1, (2017): Cotrol variables Assue that we ted to estiate E[g(X)] i which X = (x 1,, X ). Also, suppose that the f value is kow for oe average f(x) kow fuctio like i E[f(x)] = μ. The, we ca use W = g(x) + a[f(x) μ] For each a costat (fixed) uber as the E[g(X)] estiator. Now, Var(W) = Var[g(X)] + a 2 Var[f(X)] + 2a Cov(f(X), g(x) ) The siple calculatios show that the above value decreases whe Cov[f(X), g(x)] a = Var[F(X)] For such value of a we have Cov[f(X), g(x)] Var(W) = Var[g(X)] Var[F(X)] Which leads to variace reductio [3]. Rao-Blackwellisatio I this ethod, if we are able to calculate E[E(g(X) Y)] for a rado variable Y istead of E[g(X)], we ca coclude fro the coditioal variace equatio that Var(g(X)) = Var[E(g(X) Y)] + E[Var(g(X) Y)] Var(g(X)) E[Var(g(X) Y)] Sice E[g(X)] = E[E(g(X) Y)], E[g(X)] is better estiator for θ estiatio [4]. Exaple 2.6: We have E[h(X)] = E[exp ( X 2 )] whe X~T(v, μ, σ 2 ). Whe X y~n(v. yσ 2 ) ad Y 1 ~Ga( v, v ), we have 2 2 δ = 1 exp ( X j 2 ) Oe would iprove the experietal average usig the followig saples (X 1, Y 1 ),, (X, Y ) So, δ = 1 exp ( X2 Y j ) = 1 1 2σ 2 Y j + 1 Is a coditioal expectacy. Also, δ is ore accurate tha δ. Figure 3. E[exp( X 2 )] Estiator usig Mote Carlo ethod (δ ),(spot) ad Rao-Blackwellisatio (δ ) (lie) with 1000 iteratios ad (v, μ, σ 2 ) = (4.6,0,1) 13

5 Austria Joural of Matheatics ad Statistics, Vol 1, Issue 1, (2017): Iportace saplig Oe of the ethods o Mote Carlo to geerate rado saple is iportace saplig. I this ethod, other desities ad ot the ai desity fuctio are sapled which hare called iportace fuctios siulatio fro π distributio is ot always optiu ad usig iportace distributios lead to better results which ca later lead to lower variaces. The iportace saplig is i the way that we geerate (siulate) oe saple fro g distributio ad we calculate the followig quatity: 1 π(x j) h(x g(x j ) j) (1) As we kow, this statistics is based o the geerated saple fro g distributio, which is coverget accordig to the large ubers rule E g [ h(x)π(x) ] g(x) Ad oe would coclude that 1 π(x j) g(x j ) h(x j) a.s E π [h(x)] I the fowlig, we show the ratio of π(x j) for w(x g(x j ) j)ad we call the iportace poits. Oe fudaetal coditio i choosig g is that this selectio leads to fiite variace for estiator [1]. The variace of this estiator is fiite whe E[h 2 (X)w 2 (X)] = E π [ h 2 (X)w(X)] = h 2 (x) π2 (x) dx < χ g(x) oe alterative estiator for μ which icludes fiite variace ad soeties leads to ore cosistet estiatio is μ IS which is defied as μ IS = w(x j)h(x j ) (2) w(x j ) Sice 1 π(x j) whe is 1 accordig to Large ubers rule, this estiator will be coverget to E π [h(x)]. Although g(x j ) μ icludes little asyetrical status, its quadratic ea of error is lower tha the μ asyetric estiator quadratic ea of error. It also iproves the variace, for showig the syetric status of this estiator we have μ = w(x j)h(x j ) = E(μ ) = h(x j ) E ( w(x j)h(x j ) ) = E ( w(x j)h(x j ) ) μ Oe cosiderable advatage of usig syetric estiator istead of the asyetric oe is that i asyetric estiator, we should have the ratio of π(x) as accurate while i syetric estiator it is required that we have this ratio as idefiite to g(x) soe extet [1]. Exaple 2.7 suppose, X~T(v, θ, σ 2 ) is accopaied by the followig desity fuctio Γ((v + 1) 2 f(x) = ) (x θ)2 (1 + σ vπ Γ(v 2) vσ 2 ) The ai is to calculate the below itegral + (v+1) 2 I = x 5 f(x)dx 2.1 Assue θ = 0, σ = 1. We use ipotece saplig fro solvig the above itegral. I doig so, we assue the followig iportace fuctios: * T(v, 0,1) distributio which is aterior * C(0,1) distributio 1 * U(1, ) distributio

6 Austria Joural of Matheatics ad Statistics, Vol 1, Issue 1, (2017): Figure 4. The covergece of three Î estiators i iteratios: aterior distributio saplig (dotted) by iportace saplig (dot) ad iportace saplig of ootoe distributio (lie) have bee show Refereces Gholai, GH. (2008). Chage-poit Probles i Regressio: A Bayesia Approach. Ph. D.Thesis. Hoff, P. D. (2009). A First Course i Bayesia Statistical Methods. Spriger. Robert, C. P. ad Casella, G. (2004). Mote Carlo Statistical Methods. Spriger, SecodEditio Sheldo Ross, M. (2002). A First Course i Probability. 6thEditio. 15