Bindel, Fall 2011 Intro to Scientific Computing (CS 3220) Week 14: Monday, May 2

Transcription

1 Logistics Week 14: Monday, May 2 Welcome to the last week of classes! Project 3 is due Friday, May 6. Final is Friday, May 13. Lecture Wednesday will be a review, and section this week will be devoted to practice exam problems. Come with questions. Problem du jour What is the expected output of the following code? X = 20 (rand(1,n) 0.5); fx = exp( X.ˆ2/2); gx = 1/20; lx = fx./gx; result = mean(lx); errbar = std(lx)/sqrt(n); Answer: The result should converge to E[l(X)] where X is uniform on [ 10, 10] and l(x) = 20 exp( x 2 /2). The probability density function for X is { 1/20, 10 x 10 g(x) = 0, otherwise. and the expected value is E[l(X)] = R l(x)g(x) dx = exp( x 2 /2) dx, which should be quite close to 2π. We ve created some bias by truncating at ±10, but this error is nowhere near as large as the statistical error due to Monte Carlo.

2 Variance reduction The problem above is a simple example of Monte Carlo integration. We now want to see how to make this simple example more efficient by reducing the variance of the estimator. We will approach this in a few different ways. Importance sampling Let us consider the computation 2π = 2 exp( x 2 /2) dx. 0 Using the idea of the problem du jour, we could estimate 2π by drawing uniform samples on [0, L] for L large enough. But this estimator has rather high variance, and the variance gets larger the larger L is. This is intuitive in that most of the sample points don t really matter to the computation, since exp( x 2 /2) decays very quickly away from zero. The integrand exp( x 2 /2) is largest near the origin, so we get the most contribution to our integral when we have samples near zero. Therefore, it makes sense to use a method that samples more frequently near the origin, rather than sampling uniformly over some large range of x values 2π = 2 0 exp( x 2 /2) exp( x) where X is an exponential random variable. exp( x) dx = 2E[exp( X 2 /2)/ exp( X)] % lec27zmean2(n) % % Compute sqrt(2 pi) by Monte Carlo. Use importance sampling. function [result, err ] = lec27zmean2(n) if nargin < 1, N = 1000; end Y = log(rand(1,n)); fy = exp( Y.ˆ2/2); gy = exp( Y)/2; ly = fy./gy;

3 result = mean(ly); err = std(ly)/sqrt(n); Control variates I want to compute the expectation of l(x) = exp(x x 2 /2), but perhaps I ve decided its too hard. But I know that most of the interesting behavior is near the origin, so perhaps I can approximate l(x) by a polynomial over some interval close to zero. Let s try just interpolating by a quadratic at x = 0, x = 1, and x = 2, and discarding everything past x = 2: { e ( e 1)(x 1) 2, x [0, 2] h(x) = 0, otherwise. While h(x) is not identical to l(x), the two random variables surely are correlated. Furthermore, we can compute E[h(X)] analytically; a somewhat tedious calculus exercise yields E[h(X)] = e(1 e 2 ) ( e 1)(1 5e 2 ). The fact that h(x) and l(x) should be correlated, together with the fact that we can compute E[h(X)] in closed form, makes h(x) an ideal candidate to serve as a control variate with which we can construct a better estimator, as we shall now see. First, note that E[l(X)] = E[l(X) ch(x)] + ce[h(x)]. So ˆl c (X) = l(x) ch(x) + ce[h(x)] has the same expected value that l(x) does; but var[ˆl c (X)] = var[l(x)] 2c cov[l(x), h(x)] + c 2 var[h(x)]. If we choose c = cov[l(x), h(x)]/ var[h(x)], we have var[ˆl c (X)] = var[l(x)] ( 1 corr[l(x), h(x)] 2). If l(x) and h(x) are highly correlated, then ˆl c (X) may have a much lower variance than l(x). Of course, computing the covariance analytically is hard, but we can always do it numerically.

4 % lec27zmean4(n) % % Compute sqrt(2 pi) by Monte Carlo. Use importance sampling + % a control variate. function [result, err ] = lec27zmean4(n) if nargin < 1, N = 1000; end Y = log(rand(1,n)); fy = exp( Y.ˆ2/2); gy = exp( Y)/2; ly = fy./gy; hy = ( sqrt(e) (sqrt(e) 1) (Y 1).ˆ2 ). double( Y<2 ); EhY = ( sqrt(e) (1 eˆ 2) (sqrt(e) 1) (1 5 eˆ 2) ); cs = sum( (ly mean(ly)). (hy EhY) )/sum( (hy EhY).ˆ2 ); W = ly + cs (hy EhY); result = mean(w) err = std(w)/sqrt(n); Antithetic variables Now let s turn to the problem of computing π/4 by throwing darts at [0, 1] 2 and seeing what fraction lie inside the unit circle. Note that if (X i, Y i ) is a uniform random sample from the square, then (1 X i, 1 Y i ) is a correlated sample. It turns out that if φ is the indicator for the unit circle, then φ(x i, Y i ) and φ(1 X i, 1 Y i ) have negative covariance; this makes sense, since only one of them can be outside the unit circle (though both could be the same). Therefore, the estimator φ(x, Y )/2 + φ(1 X, 1 Y )/2 actually has lower variance than φ(x, Y ). This is the method of antithetic variables. % lec27pi mc(n) % % Compute pi by Monte Carlo. Use antithetic variables. function [result, err ] = lec27pi mc2(n,d)

5 if nargin < 1, N = 1000; end if nargin < 2, d = 2; end % Version 2: Antithetic variates XY = rand(d,n); XY2 = 1 XY; trials1 = double(sum(xy.ˆ2,1)<1); trials2 = double(sum(xy2.ˆ2,1)<1); trials = ( trials1 +trials2)/2; result = mean(trials); shat = std(trials ); err = shat/sqrt(n); A concluding note I will not ask you about variance reduction on the final exam, but if you ever find yourself using Monte Carlo methods, it is worth knowing the basic ideas of these methods. Much of the sophistication in getting Monte Carlo methods to run fast is in finding good variance reduction techniques. Accelerating another method to compute π Recall way back at the beginning of the semester, we discussed a way of computing the semiperimeters s k of 2 k -gons as a way of computing π. This turned out to be equivalent to finding a recurrence to compute s k = 2 k sin(2 k π), and when we Taylor expand the sine function, we have s k = π + C 1 4 k + C 2 16 k + C 3 64 k +... where C 1, C 2, C 3, etc. are constants that do not depend on k. The most straightforward estimate, π s k, therefore has a predictable error that is roughly proportional to 4 k. But what if we tried to combine several semiperimeters to get a better estimate?

6 Let s start simple, and try taking a linear combination of s k and s k+1 : αs k + βs k+1 = (α + β)π + (4α + β)c 1 4 k 1 + O(16 k ). If we choose α = 1/3 and β = 4/3, we solve the linear system α + β = 1 and 4α + β = 0. Therefore, 4s k+1 s k 3 = π + O(16 k ). If we take an appropriate combination of s k+2, s k+1, and s k, we can do even better, getting an approximation with error O(64 k ). This is linear convergence, but it is ferociously fast nonetheless.