S6880 #14. Variance Reduction Methods #2: Buffon s Needle Experiment

Transcription

1 S6880 #14 Variace Reductio Methods #: Buffo s Needle Experimet

2 1 Buffo s Needle Experimet Origial Form Laplace Method Laplace Method Laplace Method Variats Outlie 3 Crossed Needles Crossed Needles Crossed Needles, eedles are idistiguishable 4 Use Log Needle Log Needle Log Needle Variats (WMU) S6880 #14 S6880, Class Notes #14 / 30

3 Buffo s Needle Experimet Origial Form The goal of the experimet is to determie the value of π empirically. The Buffo s eedle experimet, i its origial form, is to drop a eedle of legth L at radom o a grid of parallel lies of spacig D with L D. The P(eedle itersects the grid) = L πd. D L Θ X (WMU) S6880 #14 S6880, Class Notes #14 3 / 30

4 Proof for the Origial Form Let X be the distace from the ceter of the eedle to the grid lie below ad Θ be the agle of the eedle to the horizotal. I ay reasoable defiitio of at radom, X U(0, D), Θ U(0, π) ad they are idepedet. So, = π 0 = 1 π P(eedle itersects the grid) P(eedle itersects the grid Θ = θ)f Θ (θ)dθ π 0 P(eedle itersects the grid Θ = θ)dθ. (WMU) S6880 #14 S6880, Class Notes #14 4 / 30

5 Proof for the Origial Form, cotiued p = P(eedle itersects the grid Θ = θ) = P(X L si θ or D X L si θ Θ = θ) L siθ = P(X L si θ or D X L si θ) D X X D X = P(X L si θ) + P(D X L si θ) = L D si θ + L D si θ = L D si θ p = P(eedle itersects the grid) = 1 π L D π 0 si θdθ = L πd. D X L siθ D X (WMU) S6880 #14 S6880, Class Notes #14 5 / 30

6 Proof for the Origial Form, cotiued Let ρ = L D, φ = 1 π (the p = ρφ). Suppose we drop the eedle times ad cout R itersectios. The ˆp = R ˆφ 0 = ˆp ρ, Var( ˆφ 0 ) = p(1 p), Var(ˆp) =, ρφ(1 ρφ) 4ρ = φ ( 1 ρφ 1). ˆφ 0 is most accurate (miimum variace) whe ρ = 1 (i.e., whe L = D, the eedle legth equals the grid spacig). For sufficietly large, ˆφ 0 N(φ, Var( ˆφ 0 )) by CLT. Note that E ˆφ 0 = φ, i.e., it s ubiased. (WMU) S6880 #14 S6880, Class Notes #14 6 / 30

7 Proof for the Origial Form, cotiued If ˆπ 0 = 1/ ˆφ 0, the by Delta Method, ˆπ 0 N(π, Var(ˆπ 0 )), where [ d(1/φ ] ) Var(ˆπ 0 ) dφ φ =φ Var( ˆφ 0 ) = 1 φ 4 Var( ˆφ 0 ) = π 4 Var( ˆφ 0 ). Thus Var(ˆπ 0 ) π 4 Var( ˆφ 0 ) π 4 φ ( φ 1)/ (whe ρ = 1). (WMU) S6880 #14 S6880, Class Notes #14 7 / 30

8 1 Buffo s Needle Experimet Origial Form Laplace Method Laplace Method Laplace Method Variats Outlie 3 Crossed Needles Crossed Needles Crossed Needles, eedles are idistiguishable 4 Use Log Needle Log Needle Log Needle Variats (WMU) S6880 #14 S6880, Class Notes #14 8 / 30

9 Laplace Method Grid of parallel lies replaced by grid of rectagles of sides a ad b. For L mi(a, b), p 1 = P(eedle itersects the grid) = L(a + b) L πab by a similar argumet (why? see below). The variables V, H, Θ are idepedet: V is the distace from the ceter of the eedle to grid lie below ad is U(0, a). H is the distace from the ceter of the eedle to grid lie o left ad is U(0, b). Θ is the agle of the eedle to the horizotal grids ad is U(0, π). Fid 1 p 1. (WMU) S6880 #14 S6880, Class Notes #14 9 / 30

10 Laplace Method, cotiued L(a + b) L Let ρ =, φ = 1 ab π. ˆp 1 = R, Var(ˆp 1) = p 1(1 p 1 ). ˆφ 1 = ˆp 1 ρ, Var( ˆφ 1 ) = φ ( 1 ρφ 1). Var( ˆφ 1 ) is miimized by a = b = L, where p 1 = 3 π (maximum p 1) ad Var( ˆφ 1 ) = 1 3/π 3π ad Var(ˆπ 1 ) Here ˆφ 1 is ubiased ad ˆπ 1 = 1/ ˆφ 1. (WMU) S6880 #14 S6880, Class Notes #14 10 / 30

11 Schuster (1974, Amer. Math. Mothly 81, pp. 6 9) From Laplace, but cout separately the umber of itersectios o the horizotal ad vertical grid lies, with L a = b = D. Let { 1, i X i = th drop itersects a horizotal lie 0, i th drop does ot itersect a horizotal lie { 1, i Y i = th drop itersects a vertical lie 0, i th drop does ot itersect a vertical lie for i = 1,,, ( drops of the eedle). (WMU) S6880 #14 S6880, Class Notes #14 11 / 30

12 Schuster, cotiued p = p H + p V = 1 [ L πd + L ] = L πd πd = ρφ with ρ = L D, φ = 1 π. Here p H = P(eedle itersects the horizotal grid) p V = P(eedle itersects the vertical grid). ˆp = X + Y i=1 = (X i + Y i ) is ubiased. So, ˆφ = ˆp /ρ is a ubiased estimator of φ. (WMU) S6880 #14 S6880, Class Notes #14 1 / 30

13 Schuster, cotiued Now, p = p H = P(X i = 1) = P(Y i = 1) = p V, ad P(X i = 1, Y i = 1) = L πd = ρ π = ρ φ 0 θ π π < θ < π (WMU) S6880 #14 S6880, Class Notes #14 13 / 30

14 So, Schuster, cotiued cov(x i, Y i ) = EX i Y i (EX i )(EY i ) = ρ φ (p ) = ρ φ 4ρ φ = ρ φ(1 4φ) < 0. Var(ˆp ) = 1 [VarX i + cov(x i, Y i )] ( VarX i = VarY i ) = 1 [p (1 p ) + ρ φ(1 4φ)] = 1 [ρφ + 1 ρ φ 4ρ φ ] So, Var( ˆφ ) = 1 [ φ 4ρ + φ 8 φ ] which attais its miimum whe ρ = 1 (i.e., L = a = b = D). This gives Var(ˆπ ) π 4 Var( ˆφ ) (WMU) S6880 #14 S6880, Class Notes #14 14 / 30

15 Suppose Schuster, cotiued p 3 = P(eedle itersects both horizotal & vertical grids) = P(X i = Y i = 1) = ρ φ #{eedle itersects both horizotal & vertical grids} ˆp 3 = = 1 (X i Y i ) i=1 Var(ˆp 3 ) = 1 Var(X 1Y 1 ) = 1 [EX i Y i (EX i Y i ) ] = 1 [EX iy i (EX i Y i ) ] (why?) = ρ φ (ρ φ) = p 3(1 p 3 ) (WMU) S6880 #14 S6880, Class Notes #14 15 / 30

16 Schuster, cotiued ˆφ 3 = ˆp 3 ρ ( p 3 = ρ φ) Var( ˆφ 3 ) = φ( 1 ρ φ 1) which attais its miimum whe ρ = 1 (i.e., L = a = b = D) ad gives Var(ˆπ 3 ) 1.1. Also ote that ˆφ 3 is ubiased. (WMU) S6880 #14 S6880, Class Notes #14 16 / 30

17 Schuster, cotiued Commets ˆφ i, i = 0, 1,, 3 are all ubiased. Perlma & Wichura (1975, Amer. Statist. 9, pp ) shows that the umber of times eedle itersectig oe or both sets of lies is a complete sufficiet statistic for φ. Therefore, Laplace s formulatio i its origial form leads to the miimum variace ubiased estimator of φ. But ˆπ 1 is biased = more efficiet estimator of π i Laplace s set-up. (WMU) S6880 #14 S6880, Class Notes #14 17 / 30

18 1 Buffo s Needle Experimet Origial Form Laplace Method Laplace Method Laplace Method Variats Outlie 3 Crossed Needles Crossed Needles Crossed Needles, eedles are idistiguishable 4 Use Log Needle Log Needle Log Needle Variats (WMU) S6880 #14 S6880, Class Notes #14 18 / 30

19 Crossed Needles Hammersley & Morto (1956) Drop a cross (of perpedicularly crossed eedles at midpoits) of side L o grid of horizotal lies with spacig D, L D. Suppose the two eedles are distiguishable. For i = 1,, let { 1, eedle i itersects a grid lie, X i =, otherwise. The, p 4,i = P(eedle i itersects a grid lie) = P(X i = 1) = L πd = ρφ. Let p 4 = p 4,1 + p 4,. The i=1 ˆp 4 = ˆp 4,1 + ˆp 4, = X 1 + X = (X 1i + X i ) is a ubiased estimator of p 4 sice ˆp 4,i are ubiased. (WMU) S6880 #14 S6880, Class Notes #14 19 / 30

20 Crossed Needles, cotiued Var(ˆp 4 ) = [Var(X 11) + cov(x 11, X 1 )]. EX 11 = EX 1 = ρφ, Var(X 11 ) = EX11 (EX 11) = EX 11 (EX 11 ) = ρφ(1 ρφ). ( ) P(X 11 = X 1 = 1) = 4ρφ 1 (see below ad ext slide). Let V be the vertical distace from the ceter of the cross to grid lie below ad V U(0, D). Deote Θ the agle of eedle 1 to grid ad Θ U(0, π). The variables V ad Θ are idepedet. (WMU) S6880 #14 S6880, Class Notes #14 0 / 30

21 Crossed Needles, cotiued 0 θ π θ π θ + L L π siθ D V ad si(θ + ) D V θ L L π siθ V ad si(θ + ) V π θ + eedle 1 eedle π < θ π θ π θ L L π siθ D V ad si(θ ) D V θ π θ L L π siθ V ad si(θ ) V (WMU) S6880 #14 S6880, Class Notes #14 1 / 30

22 ( cov(x 11, X 1 ) = 4ρφ 1 Var(ˆp 4 ) = Crossed Needles, cotiued ) [ ρφ(1 ρφ) + 4ρφ ( (ρφ) = 4ρφ ( 1 ) 1 ρφ, )]. ρφ ˆφ 4 = ˆp 4 4ρ. Var( ˆφ 4 ) = 1 ( ) 3 φ ρφ which attais miimum 4 ( ) 1 3 φ φ whe ρ = 1 (i.e., L = D) 4 ad leads to Var(ˆπ 4 ) π 4 Var( ˆφ 4 ).4. (WMU) S6880 #14 S6880, Class Notes #14 / 30

23 Crossed Needles, eedles are idistiguishable The eedles are idistiguishable with L = D (i.e., ρ = 1). The there are three possible outcomes correspodig to Z = X 11 + X 1, amely 0, 1, ad with probabilities ( P(Z = ) = 4 1 P(Z = 0) = 1 P(X 11 or X 1 = 1) ) φ (from previous with ρ = 1) = 1 [P(X 11 = 1) + P(X 1 = 1) P(X 11 = 1, X 1 = 1)] [ ( )] = 1 φ + φ 4 1 = 1 φ P(Z = 1) = 1 P(Z = 0) P(Z = ) = 4( 1)φ (WMU) S6880 #14 S6880, Class Notes #14 3 / 30

24 Idistiguishable Crossed Needles, cotiued Let N i = #{Z j = i, 1 j } = umber of Z = i i drops for i = 0, 1,. The N 0, N 1, N joitly have a multiomial distributio, so the likelihood is proportioal to (1 φ) N 0 [4( 1)φ] N 1 [4(1 /)φ] N (1 φ) N 0 φ N 1+N = (1 φ) N 0 φ N 0 Hece, N 0 is sufficiet for φ. So we ca use ˆp 5 = N 0 Z = 0 i drops which is a ubiased estimator of p 5 = P(Z = 0) = 1 φ. = proportio of (WMU) S6880 #14 S6880, Class Notes #14 4 / 30

25 So, Idistiguishable Crossed Needles, cotiued φ = 1 p 5, ad ˆφ = 1 ˆp 5. Var(ˆp 5 ) = p 5(1 p 5 ) Var( ˆφ 5 ) = φ(1 φ) ( ) = φ(1 φ) = φ( 1 φ) = Var(ˆπ 5 ) π 4 Var( ˆφ 5 ) = (WMU) S6880 #14 S6880, Class Notes #14 5 / 30

26 1 Buffo s Needle Experimet Origial Form Laplace Method Laplace Method Laplace Method Variats Outlie 3 Crossed Needles Crossed Needles Crossed Needles, eedles are idistiguishable 4 Use Log Needle Log Needle Log Needle Variats (WMU) S6880 #14 S6880, Class Notes #14 6 / 30

27 Use Log Needle Matel (1953) Allow L > D with horizotal grid lies. Thik of the eedle as beig made up of may small eedles with equal legth D. The E(# of itersectios) = L πd sice each piece has probability of itersectio legth πd. Let N be the umber of itersectios. The Var(N) = Var(E[N Θ]) + E(Var[N Θ]). Now, E[N Θ = θ] = L si θ D = ρ si θ. (WMU) S6880 #14 S6880, Class Notes #14 7 / 30

28 Use Log Needle, cotiued Coditioal o Θ = θ the umber of itersectios is either ρ si θ or ρ si θ + 1, so Var[N Θ = θ] = Var(ρ si θ + (0-1 r.v.)) = Var(0-1 r.v.) 1 4 E(Var[N Θ]) 1 4. Var(E[N Θ]) = Var(ρ si θ ) = ρ ( 1 4 π ) ( 1 Var(N) ρ 4 ) π for large ρ ad gives ( Var(ˆπ 6 ) π4 π4 1 4ρ Var(N) 4 4 ) π.31 for large ρ. (WMU) S6880 #14 S6880, Class Notes #14 8 / 30

29 Log Needle, variat #1 Matel Replace with square grid with side D. Ca show Var(ˆπ 7 ) π4 4L 16ρ Var(E[N Θ]) sice EN = πd. Here ( E[N Θ] = ρ si Θ + ρ cos Θ with variace ρ 1 + π 16 ) π so Var(ˆπ 7 ) (WMU) S6880 #14 S6880, Class Notes #14 9 / 30

30 Log Needle, variat # Matel Let s be the sample variace of N. The Es ρ (1 + π 16/pi ) for large ρ. Suppose we solve to obtai a estimator ˆπ 8 with s ρ = 1 + π 16 π for π ˆπ 8 = c c where c = 1 s ρ. Sice c 1, we obtai ˆπ Coversely, s is bouded above by the case i which half the eedles are parallel to the grid ad half at 45 to the grid, givig s /ρ 3 4 ad ˆπ We ca estimate Var(ˆπ 8 ) usig the large value of L to assume ormality for the umber of itersectios (CLT), so s χ 1. By the delta method, Var(ˆπ 8 ) (WMU) S6880 #14 S6880, Class Notes #14 30 / 30