Clases 7-8: Métodos de reducció de variaza e Mote Carlo * 9 de septiembre de 27 Ídice. Variace reductio 2. Atithetic variates 2 2.. Example: Uiform radom variables................ 3 2.2. Example: Tail probabilities of ormal radom variables..... 3 3. Importace samplig 3 3.. Computatio of the variace.................... 4 3.2. Example: Tail probabilities of ormal radom variables..... 4 4. Cotrol variates 4 4.. Example: The computatio of π................... 5 5. Stratified samplig 5 5.. Example: Itegrals i [, ]...................... 7 5.2. Example: computatio of π..................... 8 6. Coditioal samplig 8 6.. Example: Computig a expectatio................ 9 7. Exercises 9. Variace reductio As we have see, a critical issue i MC method is the quality of estimatio. The questio we face is: ca we devise a method that produces, with the same umber of variates, a more precise estimatio? The aswer is yes, ad the 27. * Notas preparadas por E. Mordecki para el curso de Simulació e procesos estocásticos
geeral idea is the followig: If we wat to estimate µ = EX, to fid Y such that µ = EX = EY, vary < varx. The way to produce a good Y usually departs from the kowledge that we ca have about X. There are several methods to reduce variace, however there does ot to exist a geeral method that always produce gai i the variace, the case is that each problem has its ow good method. 2. Atithetic variates The method is simple, ad cosists i usig a symmetrized variate i the cases this is possible. For istace, if we wat to compute µ = f(x)dx, we would have, with U uiform i [, ], X = f(u), Y = (f(u) + f( U)). 2 We have vary = (varx + cov(f(u), f( U))) varx. 2 If cov(f(u), f( U)) < var(x) we have variace reductio. Propositio If f is ot symmetric (i.e. if f(x) f( x) for some x i case of cotiuity) the vary < varx Proof. As we have see, varx < vary iff cov(f(u), f( U)) < varf(u), ad this holds iff f(x)f( x) < f(x) 2 dx, that is equivalet to 2 that is equivalet to f(x)f( x) < This holds iff f is ot symmetric. f(x) 2 dx + (f(x) f( x)) 2 dx >. f( x) 2 dx, Remark. For ay r.v. with symmetric (w.r.t. µ) distributio, a similar argumet holds: the method reduces the variace i the case the fuctio is ot symmetric w.r.t. µ. 2
2.. Example: Uiform radom variables We compute π = 4 with = 6 variates. Our results x2 dx, Estimate Variace Classical Estimate 3.4379.553 Atithetic Variates 3.4536.25 True value 3.4593 2.2. Example: Tail probabilities of ormal radom variables We wat to compute the probability that a stadard ormal variable is larger tha 3: µ = P(Z > 3), µ = {Z k > 3}, µ A = 2 k= ({Z k > 3} + { Z k > 3}). k= Our results with = 4 variates: Estimate Variace Classical Estimate.2.22 Atithetic Variates.37.6 True value.3499 3. Importace samplig The importace samplig method cosists i chagig the uderlyig distributio of the variable used to simulate. It is specially suited for the estimatio of small probabilities (rare evets). Assumig that X f ad Y g, it is based i the followig idetity h(x)f(x) µ = Eh(X) = h(x)f(x)dx = g(x)dx = EH(Y ), g(x) where we defie H(x) = h(x)f(x) g(x). The mai idea is to achieve that Y poits towards the set where h takes large values. If ot correctly applied, the method ca elarge the variace. 3
3.. Computatio of the variace The variace of the method is varh(y ) = E(H(Y ) 2 ) (EH(Y )) 2 = E(H(Y ) 2 ) µ 2. As µ is fixed, we should miimize (h(x)f(x)) E(H(Y ) 2 2 ) = dx. g(x) 3.2. Example: Tail probabilities of ormal radom variables µ = P(Z > 3) = = 3 3 e x2 /2 2π e (x 3)2 /2 e (x 3)2 /2 dx e 3x+9/2 e (x 3)2 /2 2π dx = Ee 3Y +9/2 {Y >3}, where Y N(3, ). Our results with = 4 variates: Estimate Variace Classical Estimate.2 2.2e-4 Atithetic Variates.37.6e-4 Importace samplig.34.5e-5 True value.3499 4. Cotrol variates Give the problem of simulatig µ = Eh(X) the idea is to cotrol the fuctio h through a fuctio g, close as posible to h, ad such that we kow β = Eg(Y ). We ca add a costat c to better adjustmet. More cocretely, the equatio is µ = Eh(X) = Eh(X) c(eg(x) β) = E(h(X) cg(x)) + cβ. The coefficiet c ca be chose i order to miimize the variace: var(h(x) cg(x)) = varh(x) + c 2 varg(x) 2ccov(h(X), g(x)). This gives a miimum whe c = cov(h(x), g(x)). var(g(x)) 4
As this quatities are usually ukow, we ca first ru a MC to estimate c. obtaiig the followig variace: var(h(x) cg(x)) = var(h(x)) cov(h(x), g(x))2 var(g(x)) = ( ρ(h(x), g(x)) 2 )var(h(x)) As ρ(h(x), g(x)), we usually obtai a variace reductio. 4.. Example: The computatio of π We choose g(x) = x, that is close to x 2 We first estimate c. I this case we kow β = E( U) = /2 ad var( U) = /2. After simulatio we obtiai So we estimate Our results with = 6 : ĉ,7 π = 4E( U 2,7( U /2)). Estimate Variace Classical Estimate 3.4379.553 Atithetic Variates 3.4536.25 Cotrol variates 3.457.25 True value 3.4593 5. Stratified samplig The idea to reduce the variace that this method proposes is to produce a partitio of the probability space Ω, ad distribute the effort of samplig i Adapted from Simulatio 5ed.S. M. Ross, (23) Elsevier 5
each set of the partitio. Suppose we wat to estimate µ = E(X), ad suppose there is some discrete radom variable Y, with possible values y,..., y k, such that, for each i =,..., k: (a) the probability p i = P(Y = y i ), is kow; (b) we ca simulate the value of X coditioal o Y = y i. The proposal is to estimate E(X) = k E(X Y = y i )p i, i= by estimatig the k quatities E(X Y = y i ), i =,..., k. So, rather tha geeratig idepedet replicatios of X, we do p i of the simulatios coditioal o the evet that Y = y i for each i =,..., k. If we let X i be the average of the p i observed values of X Y = y i, the we would have the ubiased estimator µ = k i= X i p i that is called a stratified samplig estimator of E(X). To compute the variace, we first have var( X i ) = var(x Y = y i) p i Cosequetly, usig the precedig ad that the X i, are idepedet, we see that var( µ) = = k p 2 i var( X i ) i= k i= p i var(x Y = y i ) = E(var(X Y )). Because the variace of the classical estimator is var(x), ad var( µ) = E(var(X Y )), we see from the coditioal variace formula that the variace reductio is var(x) = E(var(X Y )) + var(e(x Y )), var(x) E(var(X Y )) = vare(x Y ), 6
That is, the variace savigs per ru is vare(x Y ) which ca be substatial whe the value of Y strogly affects the coditioal expectatio of X. O the cotrary, if X ad Y are idepedet, E(X Y ) = EX ad vare(x Y ) =. Observe that the variace of the stratified samplig estimator ca be estimated by var( µ) = k p 2 i s i, i= if s i is the usual estimator of the sample of X Y = y i. Remark: The simulatio of p i variates for each i is called the proportioal samplig. Alteratively, oe ca choose,..., k s.t. + + k = that miimize the variace. 5.. Example: Itegrals i [, ] Suppose that we wat to estimate We put We have µ = E(h(U)) = h(x)dx. Y = j, if j U < j, for j =,...,. µ = EE(h(U) Y ) = E(h(U (j) )), where U (j) is uiform i j U < j. I this example we have k =, ad we use i = variates for each value of Y. As the resultig estimator is µ = To compute the variace, we have var( µ) = 2 j= U (j) U + j, = ( ) Uj + j h. j= ( ) U + j varh j= j j= j (h(x) µ j ) 2 dx, 7
where µ j = j j h(x)dx. The reductio is obtaied because µ j is closer to h tha µ: var( µ C ) = where µ C stads for the classic MC estimator. 5.2. Example: computatio of π We retur to π = 4 (h(x) µ) 2 dx, x2 dx. Observig that j U U + j U (j), we combie stratified ad atithetic samplig: µ = 2 ( ) 2 ( ) 2 Uj + j j Uj + j= For = 5 we obtai a estimatio with correct digits. µ = 3,45926537 π = 3,459265358979 6. Coditioal samplig Remember the telescopic (or tower ) property of the coditioal expectatio: E(X) = E(E(X θ)). where θ is a auxiliar radom variable. I case we are able to simulate Y = E(X θ), we have the followig variace reductio: var(y ) = var(e(x θ)) = var(x) E(var(X θ)). 8
6.. Example: Computig a expectatio Let U be uiform i [, ] ad Z N(, ). We wat to compute We first compute E(e UZ U = u) = µ = E(e UZ ). so Y = E(e UZ U) = e U 2 /2, ad EY = eu2 /2 du. Our results for two size samples. R e ux 2π e x2 /2 dx = e u2 /2, = 3 = 6 Classical,245 ±,2,95 ±,6 Coditioal,962 ±,4,949 ±,2 True.94958 Note that the classical method requires 2 samples. 7. Exercises Whe ot specified cofidece level is α =,95 ad sample size = 6. The geeral purpose is to estimate usig differet methods of variace reductio the followig quatities µ = 4 µ 2 = x2 dx = π, log xdx = 2 π, µ 3 = P(Z > 4), where Z N(, ) µ 4 = E(e Z 5) +, where Z N(, ) I all exercises use = 6 samples, ad compute the correspodig errors with,95 % cofidece: ɛ =,96s. where s = σ. Exercise. Compute µ with the followig methods: 9
(a) Acceptace rejectio o the square [, ] 2. (b) Sample mea method. (c) Use atithetic variables. (d) Use cotrol variates, with g(x) = x. First fid the optimal c with a small sample ( 3 ) ad the ru your algorithm with this estimatio. (e) Use the stratified method suggested i Lecture 3, i.e. usig, for µ = h(x)dx the formula µ = ( ) Uj + j h. j= You ca combie with atithetic variates (I this case it is o direct to obtai a error estimate). (f) Do you kow a determiistic method to compute this itegral?.for istace, compute a Riema sum, or use the trapezoidal rule: µ = 2 (f() + f()) + f(j/). j= Exercise 2. Compute µ 2 with the followig methods: (a) Ca you use acceptace rejectio method with uiform variables? (b) Use the sample mea method. (c) Use atithetic variables. (d) Use cotrol variates, with g(x) = log x, takig ito accout that log(x)dx =. First fid the optimal c with a small sample ( 3 ) ad the ru your algorithm with this estimatio. Exercise 3. Compute µ 3 with the followig methods: (a) First use crude MC. As the probability is very small, a large is ecessary. (b) Check if the atithetic variates method improves the situatio. (c) Use the importace samplig method, based i the followig ideatity. P(Z > 4) = = where X N(4, ). 4 4 e x2 /2 dx = 2π 4 /2 e x2 /2 e (x 4)2 2π e dx (x 4)2 /2 e 4x+8 e (x 4)2 /2 2π dx = Ee 4X+8 {X>4}. Exercise 4. Compute µ 4 with the followig methods: (a) First use crude MC. (b) Use atithetic variates.
(c) Now we use cotrol variates i the followig way. Check the followig idetity: E(e Z K) + = E(e Z ) K + E(K e Z ) +. (here x + = máx(, x), ad you ca use that x = x + ( x) + ). The, computig Ee Z = e /2, we fid the price of the put optio, give by P (K) = E(K e Z ) +. You ca use also atithetic variates i this situatio. (d) Importace samplig ca be implemeted i the followig way. Check the followig idetity: E(e Z K) + = (e x K) + e x2 /2 dx = e /2 ( Ke x ) + e (x )2 /2 dx. 2π 2π R that is kow as put-call duality. R P (K) = e /2 E( Ke X ) +. where X N(, ). You ca use also atithetic variates i this situatio. Exercise 5. We wat to compute the itegral by simulatio: µ = ( x)e x2 dx I all cases provide the 95 % error of estimatio. (a) Estimate µ usig the acceptace-rejectio method o the square [, ] 2. (b) Estimate µ usig uiform radom variables, by the sample mea method. (c) Estimate µ usig radom variables with desity f(x) of exercise. (d) Use atithetic variables. (e) Use cotrol variates, with g(x) = x. First fid the optimal c with a small sample ( 3 ) ad the ru your algorithm with this estimatio. Exercise 6. We wat to estimate by simulatio: µ = 2 x + x dx = 2 I all cases provide the 95 % error of estimatio. 2 dx. x + x (a) Estimate µ usig uiform radom variables, by the sample mea method. (b) Estimate µ usig radom variables with desity { /(2 x), whe x, f(x) =, i other case.
(c) Use atithetic variables. (d) Use cotrol variates, with g(x) = x. First fid the optimal c with a small sample ( 3 ) ad the ru your algorithm with this estimatio. Exercise 7. Volume of a sphere i R 6.. Evaluate by the method of Mote Carlo the volume of a uit sphere i dimesio 6. (Compare with the true value π 3 /6). 2. We ow use the method of Mote-Carlo with a differet law Let Q be the law i R 6 with desity q(x) = 6 Z(α) exp( α x i ) ad Z(α) is the ormalizig costat. 3. Compute Z(α). i= 4. Plot a code to simulate a poit i R 6 with law Q. 5. Devise a method to compute the volume of the sphere usig the distributio Q usig importace samplig. Compute the error of the method as a fuctio of α 6. Determie a reasoable iterval for α ad propose a value to miimize the error. 2