Chapter 4. Replication Variance Estimation. J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 1 / 28

Transcription

1 Chapter 4 Replication Variance Estimation J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 1 / 28

2 Jackknife Variance Estimation Create a new sample by deleting one observation n 1 n n ( x (k) x) 2 = x (k) = n x x k n 1 x (k) x = x k x n 1 1 n(n 1) n (x k x) 2 = n 1 sx 2 J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 2 / 28

3 Alternative Jackknife Weights n x (k) ψ = ψx k + (1 ψ) x (k) x (k) ψ x = ψ(x k x) + (1 ψ)( x (k) x) x (k) 1 ψ ψ x = (ψ n 1 )(x k x) = ( nψ 1 n 1 )(x k x) ψ (nψ x)2 1)2 n = (n 1) 2 (x k x) 2 ( x (k) If (nψ 1) 2 = n 1 n n (1 f ), then ( x (k) ψ x)2 = 1 n (1 f )s2 x. J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 3 / 28

4 Random Group Jackknife n = mb : m groups of size b, x 1,..., x m x = 1 m x i m i=1 ˆV ( x) = 1 m 1 m 1 x (k) n = n x b x k n b x (k) b x = n x b x k n b ˆV RGJK ( x) m 1 m Unbiased but d.f. = m 1. m ( x (k) m ( x i x) 2 i=1 = m x x k m 1 x = 1 m 1 ( x k x) b x) 2 = 1 m(m 1) m ( x k x) 2 J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 4 / 28

5 Theorem Theorem F N = {y 1,..., y N } : sequence of finite population y i iid(µ y, σ 2 y ) with finite 4 + δ moments g( ) : continuous function with continuous first derivative at µ y n 1 n n {g(ȳ (k) ) g(ȳ)} 2 = [g (ȳ)] 2 ˆV (ȳ) + o p (n 1 ) where ˆV (ȳ) = n 1 s 2 and g (ȳ) = g(ȳ) ȳ. J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 5 / 28

6 Proof of Theorem By a Taylor linearization, g(ȳ (k) ) = g(ȳ)+g (ȳ k )(ȳ (k) ȳ) = g(ȳ)+g (ȳ)(ȳ (k) ȳ)+r nk (ȳ (k) ȳ) for some ȳ k B δ k (ȳ), δ k = ȳ (k) ȳ where R nk = g (ȳ k ) g (ȳ). Thus, n { g(ȳ (k) ) g(ȳ)} 2 = = n { g (ȳk )} 2 (ȳ (k) ȳ) 2 n { g } 2 (ȳ) + R nk (ȳ (k) ȳ) 2 (i) max 1 k n ȳk ȳ 0 in probability. (ii) max 1 k n g (ȳk ) g (ȳ) 0 in probability. n 1 n n {g(ȳ (k) ) g(ȳ)} 2 = [g (ȳ)] 2 ˆV (ȳ) + o p (n 1 ) J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 6 / 28

7 Proof of (i) Pr[max ȳ (k) ȳ > ɛ] k for some c. max k ȳ (k) ȳ 0 in probability. max k ȳ k ȳ max k ȳ (k) ȳ 0 n Pr[ ȳ (k) ȳ > ɛ] n ɛ 2 E[ ȳ (k) ȳ 2 ] cv (ȳ) 0 J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 7 / 28

8 Proof of (ii) Since g (ȳ) g (µ y ) in probability, it suffices to show that max g (ȳk ) g (µ y ) = o p (1). k Now, note that { Pr max g (ȳk ) g (µ y ) } > ɛ k { +Pr max k Pr{max k ȳk µ y > δ} } ȳk µ y < δ, max g (ȳk ) g (µ y ) > ɛ k holds for any δ > 0. Since g (y) is continuous at y = µ y, we can choose δ such that the second term equals to zero. The first term converges to zero by (i). J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 8 / 28

9 Remainder with Second Derivatives If g( ) has continuous second derivatives at µ y, then n 1 (n 1) n [g(ȳ (k) ) g(ȳ)] 2 = [g (ȳ)] 2 ˆV (ȳ) + O p (n 2 ). Proof : [g(ȳ (k) ) g(ȳ)] 2 = [g (ȳ k )]2 (ȳ (k) ȳ) 2 g (ȳk )2 = [g (ȳ)] 2 + 2[g (ȳk (ȳk k ȳ) [g (ȳk )]2 = [g (ȳ)] 2 + K 1 ȳk ȳ for some K 1. Thus, since ȳ k ȳ = O p(n 1 ), we have the result. J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 9 / 28

10 Jackknife Often Larger than Taylor ˆR = x 1 ȳ ˆR (k) ˆR = [ x (k) ] 1 ȳ (k) x 1 ȳ = [ x (k) ] 1 (y k ˆRx k )(n 1) 1 ˆV JK ( ˆR) = n 1 n ( ˆR (k) ˆR) 2 n 1 n = [ x (k) ] 2 (y k n(n 1) ˆRx k ) 2 vs. ˆVL ( ˆR) = 1 n(n 1) E[( x (k) ) 2 ] E[( x) 2 ] n ( x) 2 (y k ˆRx k ) 2 J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 10 / 28

11 Quantiles ξ p = Q(p) = F 1 (p), p (0, 1) where F (y) is cdf ˆξ p = ˆQ(p) = inf { ˆF (y) p}, p = 0.5 for median y ( ) 1 ˆF (y) = w i w i I (y i y) i A i A To reduce the bias, use interpolation: ˆξ p = ˆξ p0 + ˆξ p1 ˆξ p0 ˆF (ˆξ p1 ) ˆF (ˆξ p0 ) {p ˆF (ˆξ p0 )} where ˆξ p1 = inf x1,...,x n {x; ˆF (x) p} & ˆξ p0 = sup x1,...,x n {x; ˆF (x) p} J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 11 / 28

12 Test Inversion for Quantile C.I. Construct acceptance region for H 0 : p = p 0 {p 0 2[ ˆV (ˆp 0 )] 1/2, p 0 + 2[ ˆV (ˆp 0 )] 1/2 } Invert p-interval to give C.I. for ξ p0 { ˆQ(p 0 2[ ˆV (ˆp 0 )] 1/2 ), ˆQ(p 0 + 2[ ˆV (ˆp 0 )] 1/2 )} J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 12 / 28

13 p ζ p Figure: Plot of CDF J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 13 / 28

14 ζ p p Figure: Inverse CDF J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 14 / 28

15 Bahadur representation Let ˆF (x) be an unbiased estimator of F (x), the population CDF of X. For given p (0, 1), we can define ζ p = F 1 (p) to be the p-th population quantile of X. Let ˆζ p = ˆF 1 (p) be the sample estimator of ζ p using ˆF. Also, define ˆp = ˆF (ζ p ). We are interested in applying a Taylor linearization of ˆζ p around p = ˆp. J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 15 / 28

16 Bahadur representation (Cont d) Note that, by Taylor linearization, ζ p = ˆF 1 (ˆp) = ˆF 1 (p) + d ˆF 1 (p) (ˆp p) dp = ˆF 1 (p) + df 1 (p) (ˆp p) dp = ˆζ p + 1 (ˆp p), f (ζ p ) where the last equality follows from dζ p /dp = (dp/dζ p ) 1 with p = F (ζ p ). Therefore, we have which is first established by Bahadur (1966). ˆζ p = ζp + 1 (p ˆp) (1) f (ζ p ) J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 16 / 28

17 From (1), we can use V (ˆζ p ) = {f (ζ p )} 2 V (ˆp), which requires knowledge of density function f ( ), or we need to estimate it. Define Q(p) = F 1 (p). Since 1 f (ζ p ) = dq(p) = dp Q(p + h) Q(p h), p + h (p h) we can use Q ( ˆp + 2 ) ( V (ˆp) Q ˆp 2 ) V (ˆp) ˆp + 2 V (ˆp) {ˆp 2 V (ˆp)} := ˆγ p to estimate 1/f (ζ p ). This is essentially the idea behind the test inversion method of Woodruff. J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 17 / 28

18 Variance Estimation for Quantiles Bahadur Representation ( ) p(1 p) n(ˆξ p ξ p ) N 0, [F (ξ p )] 2 (SRS) V [F (ˆξ b )]. = ( ) F 2 V (ˆξ p ) ξ ˆQ (p) = [ ˆF (ξ p )] 1 = ˆQ(ˆp + 2 V (ˆp)) ˆQ(ˆp 2 V (ˆp)) (ˆp + 2 V (ˆp)) (ˆp 2 V (ˆp)) ˆV (ˆξ p ) = ˆγ 2 ˆV { ˆF (ξ p )} =: ˆγ J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 18 / 28

19 Variance Estimation for Quantiles (2) 1 Jackknife variance estimator is not consistent. Median ˆθ = 0.5(x m + x m+1 ) for n=2m even ˆV JK = 0.25(n 1)[x m+1 x m ] 2 2 Bootstrap and BRR are O.K. 3 Smoothed quantile ˆξ p = ˆξ p0 + ˆγ[p ˆF (ˆξ p0 )] ˆξ (k) p = ˆξ p0 + ˆγ[p ˆF (k) (ˆξ p0 )] ˆV JK (ˆξ p ) = k c k (ˆξ (k) p ˆξ p ) 2 J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 19 / 28

20 Two-Phase Samples ȳ 2p,reg = ȳ 2p,REE = ȳ 2π + ( x 1π x 2π ) ˆβ 2 ˆβ 2 = w 2i (x i x 2π ) (x i x 2π ) i A 2 w 1 2i = π 1i π 2i 1i, w 1i = π 1 1i 1 i A 2 w 2i (x i x 2π ) y i ˆV JK (ȳ 2p,REE ) = L [ ] c k ȳ (k) 2 2p,REE ȳ 2p,REE J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 20 / 28

21 Two-Phase Samples where w (k) 2i ˆβ (k) 2 = = (w (k) 1i )(π 1 2i 1i ) x (k) 1π = ( x (k) 2π, ȳ (k) 2π ) = w (k) i A1 1i w (k) i A2 w (k) 2i (x i x (k) 2π ) (x i x (k) i A2 2i 2π ) 1 1 i A1 w (k) 1i x i w (k) 2i (x i, y i ) i A2 1 w (k) 2i (x i x (k) 2π ) (y i ȳ (k) 2π ) i A 2 J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 21 / 28

22 Theorem Kim, Navarro, Fuller (2006 JASA), Assumptions (i) Second phase is stratified with π 2i 1i constant within group (ii) K L < Nn 1 π 1i < K U for some K L &K U (iii) V { ˆT 1y F} K M V { ˆT 1y,SRS } where ˆT 1y = i A 1 π 1 1i y i (iv) nv { ˆT 1y F} = N N i=1 j=1 Ω ijy i y j where N i=1 Ω ij = O(N 1 ) ( ) 2 (v) E ˆV 1 ( ˆT 1y ) F V ( ˆT 1y F) 1 = o(1) (vi) E{[c k ( ˆT (k) 1y ˆT 1y ) 2 ] 2 F} < K L L 2 [V ( ˆT 1y )] 2 J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 22 / 28

23 Theorem Kim, Navarro, Fuller (2006 JASA), Result ˆV {ȳ 2p,reg } = V (ȳ 2p,reg F) 1 N 2 where κ 2i = π 2i 1i N i=1 κ 1 2i (1 κ 2i )e 2 i + o p (n 1 ) e i = y i ȳ N (x i x)β N [ N ] 1 N β N = (x i x N ) (x i x N ) (x i x N ) y i. i=1 i=1 J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 23 / 28

24 Sketched Proof Extend definition of a i : { 1 if i A2, if i A a i = 1 0 if i / A 2, if i A 1 a i defined throughout the population. Writing ȳ 2p,reg = z 1π + ( x 1π ū 1π ) ˆβ where ( z 1π, ū 1π ) = ( i A 1 a i π 1 2i ) 1 i A 1 π 1 2i (z i, u i ), z i = a i y i, u i = a i x i : conditional on a = (a 1,, a N ), ȳ 2p,reg is a smooth function of ( z 1π, x 1π, ū 1π, ˆβ) ˆβ = π 1 2i (u i ū 1π ) (u i ū 1π ) i A1 1 i A 1 π 1 2i (u i ū 1π ) z i. J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 24 / 28

25 Sketched Proof (Cont d) Variance of ȳ 2p,reg : V [ȳ 2p,reg F] = E[V (ȳ 2p,reg a, F) F] + V [E(ȳ 2p,reg a, F) F] Replication variance estimation: L ȳ (k) 2p,reg = z (k) (k) 1π + ( x 1π ū(k) 1π ) ˆβ (k) c k (ȳ (k) 2p,reg ȳ 2p,reg ) 2 = E[V (ȳ 2p,reg a, F) F] + o p (n 1 ) ˆV (ȳ reg ) = L c k (ȳ (k) 2p,reg ȳ 2p,reg ) 2 where ˆV (ȳ 2p,reg a) is used to emphasize that it is a function of a. =: ˆV (ȳ 2p,reg a), J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 25 / 28

26 Sketched Proof (Cont d) 1 By condition (v), ˆV (ȳ 2p,reg a) = V [ȳ 2p,reg a, F] + o p (n 1 ) conditional on a, F. 2 Show ˆV (ȳ 2p,reg a) = E{V [ [ȳ 2p,reg a, F] F} ] + o p (n 1 ) ˆV (ȳ2p,reg a) Need to show that V = o(1) V (ȳ 2p,reg a, F) 3 ˆV (ȳ 2p,reg a) = V (ȳ 2p,reg F) V (E(ȳ 2p,reg a, F) F) + o p (n 1 ȳ 2p,reg = ȳ 2π + ( x 1π x 2π ) ˆβ = ȳ 2π + ( x 1π x 2π )β N + O p (n 1 ) = ē 2π + x 1π β N + O p (n 1 ) E[ȳ 2p,reg a, F] = 1 N κ 1 2i a i e i + x 1π β N N V [E[ȳ 2p,reg a, F] F] = 1 i=1 N κ 1 2i (1 κ 2i )e 2 i N 2 J. Kim, W. Fuller (ISU) Chapter i=1 4 7/31/11 26 / 28

27 Remark (i) The bias term of ˆV {ȳ 2p,reg } is of order O(N 1 ) and can be negligible if n/n is negligible. (ii) The bias term can be unbiasedly estimated by N 2 i A 2 w 1i π 2 2i 1i (1 π 2i 1i)ê 2 i. (iii) For some designs and replication procedures, it is possible to modify the weights to reduce the bias. (See the textbook.) J. Kim, W. Fuller (ISU) Chapter 4 7/31/11 27 / 28