Chapter 8: Estimation 1

Transcription

1 Chapter 8: Estimation 1 Jae-Kwang Kim Iowa State University Fall, 2014 Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

2 Introduction 1 Introduction 2 Ratio estimation 3 Regression estimator Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

3 Introduction So far, we have discussed various sampling designs and its unbiased estimator. HT estimator is used for each sampling design (except for PPS sampling). No claim for optimality. Definition For parameter θ (y), y = (y 1, y 2,, y N ), an estimator ˆθ (A) is UMVUE (Uniformly unbiased minimum variance estimator) if } 1 Unbiased: E y {ˆθ (A) = θ (y), for all y } } 2 Minimum variance: V y {ˆθ (A) V y {ˆθ (A) for all unbiased estimator ˆθ (A) and for all y. Remark Uniformity is important: Suppose that my estimator is ˆθ 12. If θ = 12, then ˆθ (ˆθ) is unbiased and V = 0. That is, MVUE at θ = 12. But, it is not UMVUE. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

4 Introduction Proposition Consider a noncensus design with π k > 0, (k = 1, 2,, N), then there is no UMVUE of t = N i=1 y i exists. Proof Suppose that there exists ˆQ which is UMVUE of t. Fix any y = (y1,, y N ) R N. Now, consider Q (A) = k A y k y k π k + N yk. The new estimator Q (A) satisfies 1 Unbiased 2 The variance of Q (A) is zero at y = y. Because ˆQ is UMVUE, V y ( ˆQ) V y (Q ). Since V y (Q ) = 0 for y = y, we have V y ( ˆQ) = 0 for y = y. Since y can be arbitrary, we have V y ( ˆQ) = 0 for all y, which means that ˆQ = t for all y, which is impossible for any noncensus design. Therefore, UMVUE cannot exist. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33 i=1

5 Introduction Remark 1 In the proof of the proposition, Q (A) is called the difference estimator. The variance of the difference estimator is V {Q (A)} = k U The variance is small if y k = y k. l U kl y k y k π k y l y l π l. 2 The class of (design) unbiased estimator is too big. We cannot find the best one in this class. 3 If we define the class of the linear estimators as ˆt = w i y i, where w i are constants that are fixed in advance, the HT estimator is the only estimator among the class of linear unbiased estimators. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

6 Introduction Remark (Cont d) 4 We have the following alternative definition of the linear estimator: ˆt = w i (A) y i = w ia y i where w i (A) = w ia are constants that depends on the realized sample. That is, w i (A) = w ia are random variables. 5 One advantage of linear estimator is that it is internally consistent. An estimator is internally consistent if ˆt (y 1 + y 2 ) = ˆt (y 1 ) + ˆt (y 2 ), where ˆt (y) is an estimator of the total of item y. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

7 Ratio estimation 1 Introduction 2 Ratio estimation 3 Regression estimator Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

8 Ratio estimation Ratio estimator Basic Setup : Observe x (auxiliary variable) and y (study variable) in the sample We know X = N i=1 x i or X = N 1 N i=1 x i in advance. ˆX HT = π 1 i x i can be different from X. Ratio estimator : Ŷ r = X ŶHT ˆX HT = X ˆR Ȳ r = X ŶHT ˆX HT = X ˆR Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

9 Ratio estimation Algebraic properties Linear in y (thus it is internally consistent.) If ˆX HT < X, then ŶHT < Ŷr If ˆX HT > X, then ŶHT > Ŷr If y i = x i, then the ratio estimator equals to X. That is, w i x i = X for Ŷ r = w iy i. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

10 Ratio estimation Statistical properties - Bias ( ) It is biased because E ˆR R. Bias ( of ) ˆR = ( Ŷ HT ) / ˆX HT is called the ratio bias. That is, B ˆR = E ˆR R is called the ratio bias. Ratio bias ( ) ( ) Bias ˆR = Cov ˆR, ˆX HT /X. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

11 Ratio estimation Statistical properties - Bias (Cont d) Definition: Bias of ˆθ is negligible R.B.(ˆθ) = Bias(ˆθ) 0 as n. Var(ˆθ) Note: If the bias of ˆθ is negligible, then by CLT, and ˆθ θ Var(ˆθ) = ˆθ E(ˆθ) Var(ˆθ) { MSE(ˆθ) = V (ˆθ) + = V (ˆθ). = V (ˆθ). + Bias(ˆθ) N (0, 1), Var(ˆθ) { 1 + Bias(ˆθ) } 2 [ R.B.(ˆθ) ] 2 } Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

12 Ratio estimation Statistical properties - Bias (Cont d) Ratio bias is negligible. { } 2 V R.B.( ˆR) ( ˆX HT ) X 2 = { ( )} 2 CV ˆX HT 0. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

13 Ratio estimation Statistical properties - Variance Taylor expansion Ȳ r = Ȳ + ( Ȳ HT Ȳ ) R ( XHT X ) X 1 [ ( X HT X ) ( Ȳ HT Ȳ ) R ( X HT X ) 2 ] +o p ( n 1 ) where R = X 1 Y. Variance where E i = y i Rx i. ( ) ).= 1 V (Ŷr V E i π i Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

14 Ratio estimation Remark When the ratio estimator is better than the HT estimator? Note that ) V (Ŷr Thus, ) = V (ŶHT R ˆX HT ) ( ) ( ) = V (ŶHT 2RCov ˆXHT, ŶHT + R 2 V ˆX HT V (Ŷr ) V (ŶHT ) Cov( ˆX HT, ŶHT ) V ( ˆX 1 HT ) 2 R ( ) Corr ˆX HT, Ŷ HT 1 CV ( ˆX HT ) 2 CV (Ŷ HT ) where CV (Ŷ HT ) = {V (Ŷ HT )} 1/2 /E(Ŷ HT ). Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

15 Ratio estimation Variance estimation Variance estimation : Use Êi = y i ˆRx i in the HT (or SYG) variance estimator. Example: SRS V (Ŷr ) = N2 n For variance estimation, use ˆV (Ŷr ) = N2 n ( 1 n ) 1 N N 1 ( 1 n ) 1 N n 1 N (y i Rx i ) 2 i=1 ( y i ˆRx i ) 2. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

16 Ratio estimation Application of the ratio estimator Hajek estimator: Ratio estimator of the mean using x i = 1 Domain estimation : The parameter of interest can take the form of the ratio N i=1 Ȳ d = δ iy i N i=1 δ i where δ i = 1 if i D and δ i = 0 if i / D. Thus, Ȳ HTd = π 1 i π 1 i δ i δ i y i is an (approximately) unbiased estimator of Ȳ d. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

17 1 Introduction 2 Ratio estimation 3 Regression estimator Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

18 Basic Setup Observe x i = (x 1i,, x Ji ) (auxiliary variables) and y i (study variable) in the sample We know X = N i=1 x i or X = N 1 N i=1 x i in advance. Interested in estimating Y = N i=1 y i ˆX HT = π 1 i x i can be different from X. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

19 Motivation - Difference estimator Motivation: Use auxiliary information at estimation stage Use a regression approach: 1 Suppose we have y o k = J b j x jk = b x k, k = 1, 2,, N, j=1 for some known J-dimensional vector b. The y o k is a proxy for y k. 2 Difference estimator : Ŷ diff = N i=1 y o i + y i y o i π i Unbiased (regardless of choice of y o k ) The variance is small if y o k = y k. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

20 Regression estimator How to choose y o k = b x k? - Let s estimate b from the sample. Regression estimator: Ŷ reg = N ŷ i + i=1 y i ŷ i π i, where ŷ i = ˆb x i and ˆb is estimated from the sample. Motivated from the linear regression superpopulation model E ζ (y i ) = x iβ V ζ (y i ) = σ 2. Note that β and σ 2 are superpopulation parameters. (Thus, b is the finite population quantity corresponding to β.) Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

21 Regression estimator How to estimate b? 1 Note that, under census, b can be estimated by solving U (b) N (y i b x i ) x i = 0. i=1 2 Consider an unbiased estimator of U (b): Û (b) = 1 π i (y i b x i ) x i 3 Obtain a solution ˆb by solving Û (b) = 0 for b. The solution is ( ) 1 1 ˆb = x i x 1 i x i y i. π i π i Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

22 Remark We will study that, under some conditions, Ŷ reg is asymptotically equivalent to Ŷ diff. Thus, Ŷ reg is asymptotically unbiased and V (Ŷ reg ) = V (Ŷdiff ) { } 1 = V (y i x i b) π i where ( N ) 1 N b = x i x 1 i x i y i. π i i=1 i=1 Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

23 Regression estimator Regression estimator ( ) Ŷ reg = Ŷ HT + X ˆX HT ˆb where ˆb = ( π 1 i x i x i ) 1 π 1 i x i y i. Note that, if 1 is in the column space of x i, we can write Ŷ reg = N i=1 ŷ i where ŷ i = x i ˆb. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

24 Regression estimator Regression estimator (of the mean): Ȳ reg = Ȳ HT + ( X X HT ) ˆb where ( X HT, Ȳ ) 1 ) HT = (ˆX HT N, Ŷ HT = 1 1 ( ) x N π i, y i. i Note that we can express Ŷ reg = ˆb 0 + ˆb X where ˆb 0 = Ȳ HT ˆb X HT. Thus, it is the predicted value of y at x = X when the linear regression model is used. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

25 Algebraic properties Linear in y: where Also, Ŷ reg = 1 π i g ia y i ( ( g ia = 1 + X ˆX ) 1 HT π 1 i x i x i) x i. Ȳ reg = 1 N 1 π i g ia y i. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

26 Algebraic properties Calibration property 1 π i g ia x i = X. The property (*) is also called benchmarking property. ( ) Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

27 If x i = (1, x i1 ), then Ȳ reg = Ȳ π + ( X 1 X π1 ) ˆb 1 and Ŷ reg = N {Ȳπ + ( X ) } 1 X π1 ˆb 1, where Ȳ π and X π1 are the Hajek estimators of the form ˆb 1 = [ ( X π1, Ȳ π ) = ( π 1 i and X 1 = N 1 N i=1 x i1. π 1 i ) 1 ( xi1 X π1 ) ( xi1 X π1 ) ] 1 π 1 ( ) i x i1, y i, π 1 ( ) i xi1 X π1 yi, Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

28 Weights in Ŷ reg = w iy i can be derived by minimizing Q (w) = π i ( w i 1 π i ) 2 subject to 1 π i g ia x i = X. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

29 Statistical Properties Let s consider the regression estimator of the mean We can express Ȳ reg = Ȳ HT + ( X X HT ) ˆb Ȳ reg = Ȳ HT + ( X ) ) X HT b + ( X X HT (ˆb b). = Ȳ HT + ( X ) X HT b = Ŷ diff where b = ( N i=1 x ix i) 1 N i=1 x iy i. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

30 Taylor linearization Note that Ȳ reg = Ȳ HT + ( X X HT ) ˆb is a (nonlinear) function of ( X HT, Ȳ HT, ˆb). Taylor linearization of Ȳ reg : Ŷ reg = f ( X HT, Ȳ HT, ˆb) { }. = f ( X, Ȳ, b) + Ȳ f ( X, Ȳ, b) (ȲHT Ȳ ) { } + X f ( X, Ȳ, b) ( X HT X ) { } + ˆb f ( X, Ȳ, b) (ˆb b) = Ȳ + ( Ȳ HT Ȳ ) + b ( X ) XHT + 0 Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

31 Statistical Properties (Cont d) Bias : Negligible Variance : { ).= Var (Ŷreg Var π 1 ( i yi x ib )} Variance estimation ˆV (Ŷreg ) = j A ij π ij Ê i Ê j π i π j where Ê i = y i x i ˆb. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

32 Example: SRS Variance of the regression estimator ).= 1 ( V (Ŷreg 1 n ) Se 2 n N where Se 2 = 1 N ( yi x N 1 ib ) 2. i=1 If 1 C(X ), then SST = SSR + SSE. That is, N ( yi Ȳ ) 2 N ( = y o i i=1 i=1 Ȳ ) 2 + N i=1 (y i y o i ) 2, where y o i = x i b. Thus, V (Ŷ reg ). = V (Ŷ HT )(1 R 2 ) V (Ŷ HT ). where R 2 = SSR/SST is the coefficient of determination. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33

33 Remark The regression estimator is derived using a regression model. The validity (i.e. asymptotic unbiasedness) of the regression estimator does not depend on whether the regression model holds or not. However, the variance of the regression estimator is small if the regression model is good. That is, it is model-assisted, not model-dependent. Kim (ISU) Ch. 8: Estimation 1 Fall, / 33