Chapter 8: Estimation 1 Jae-Kwang Kim Iowa State University Fall, 2014 Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 1 / 33
Introduction 1 Introduction 2 Ratio estimation 3 Regression estimator Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 2 / 33
Introduction So far, we have discussed various sampling designs and its unbiased estimator. HT estimator is used for each sampling design (except for PPS sampling). No claim for optimality. Definition For parameter θ (y), y = (y 1, y 2,, y N ), an estimator ˆθ (A) is UMVUE (Uniformly unbiased minimum variance estimator) if } 1 Unbiased: E y {ˆθ (A) = θ (y), for all y } } 2 Minimum variance: V y {ˆθ (A) V y {ˆθ (A) for all unbiased estimator ˆθ (A) and for all y. Remark Uniformity is important: Suppose that my estimator is ˆθ 12. If θ = 12, then ˆθ (ˆθ) is unbiased and V = 0. That is, MVUE at θ = 12. But, it is not UMVUE. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 3 / 33
Introduction Proposition Consider a noncensus design with π k > 0, (k = 1, 2,, N), then there is no UMVUE of t = N i=1 y i exists. Proof Suppose that there exists ˆQ which is UMVUE of t. Fix any y = (y1,, y N ) R N. Now, consider Q (A) = k A y k y k π k + N yk. The new estimator Q (A) satisfies 1 Unbiased 2 The variance of Q (A) is zero at y = y. Because ˆQ is UMVUE, V y ( ˆQ) V y (Q ). Since V y (Q ) = 0 for y = y, we have V y ( ˆQ) = 0 for y = y. Since y can be arbitrary, we have V y ( ˆQ) = 0 for all y, which means that ˆQ = t for all y, which is impossible for any noncensus design. Therefore, UMVUE cannot exist. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 4 / 33 i=1
Introduction Remark 1 In the proof of the proposition, Q (A) is called the difference estimator. The variance of the difference estimator is V {Q (A)} = k U The variance is small if y k = y k. l U kl y k y k π k y l y l π l. 2 The class of (design) unbiased estimator is too big. We cannot find the best one in this class. 3 If we define the class of the linear estimators as ˆt = w i y i, where w i are constants that are fixed in advance, the HT estimator is the only estimator among the class of linear unbiased estimators. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 5 / 33
Introduction Remark (Cont d) 4 We have the following alternative definition of the linear estimator: ˆt = w i (A) y i = w ia y i where w i (A) = w ia are constants that depends on the realized sample. That is, w i (A) = w ia are random variables. 5 One advantage of linear estimator is that it is internally consistent. An estimator is internally consistent if ˆt (y 1 + y 2 ) = ˆt (y 1 ) + ˆt (y 2 ), where ˆt (y) is an estimator of the total of item y. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 6 / 33
Ratio estimation 1 Introduction 2 Ratio estimation 3 Regression estimator Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 7 / 33
Ratio estimation Ratio estimator Basic Setup : Observe x (auxiliary variable) and y (study variable) in the sample We know X = N i=1 x i or X = N 1 N i=1 x i in advance. ˆX HT = π 1 i x i can be different from X. Ratio estimator : Ŷ r = X ŶHT ˆX HT = X ˆR Ȳ r = X ŶHT ˆX HT = X ˆR Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 8 / 33
Ratio estimation Algebraic properties Linear in y (thus it is internally consistent.) If ˆX HT < X, then ŶHT < Ŷr If ˆX HT > X, then ŶHT > Ŷr If y i = x i, then the ratio estimator equals to X. That is, w i x i = X for Ŷ r = w iy i. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 9 / 33
Ratio estimation Statistical properties - Bias ( ) It is biased because E ˆR R. Bias ( of ) ˆR = ( Ŷ HT ) / ˆX HT is called the ratio bias. That is, B ˆR = E ˆR R is called the ratio bias. Ratio bias ( ) ( ) Bias ˆR = Cov ˆR, ˆX HT /X. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 10 / 33
Ratio estimation Statistical properties - Bias (Cont d) Definition: Bias of ˆθ is negligible R.B.(ˆθ) = Bias(ˆθ) 0 as n. Var(ˆθ) Note: If the bias of ˆθ is negligible, then by CLT, and ˆθ θ Var(ˆθ) = ˆθ E(ˆθ) Var(ˆθ) { MSE(ˆθ) = V (ˆθ) + = V (ˆθ). = V (ˆθ). + Bias(ˆθ) N (0, 1), Var(ˆθ) { 1 + Bias(ˆθ) } 2 [ R.B.(ˆθ) ] 2 } Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 11 / 33
Ratio estimation Statistical properties - Bias (Cont d) Ratio bias is negligible. { } 2 V R.B.( ˆR) ( ˆX HT ) X 2 = { ( )} 2 CV ˆX HT 0. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 12 / 33
Ratio estimation Statistical properties - Variance Taylor expansion Ȳ r = Ȳ + ( Ȳ HT Ȳ ) R ( XHT X ) X 1 [ ( X HT X ) ( Ȳ HT Ȳ ) R ( X HT X ) 2 ] +o p ( n 1 ) where R = X 1 Y. Variance where E i = y i Rx i. ( ) ).= 1 V (Ŷr V E i π i Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 13 / 33
Ratio estimation Remark When the ratio estimator is better than the HT estimator? Note that ) V (Ŷr Thus, ) = V (ŶHT R ˆX HT ) ( ) ( ) = V (ŶHT 2RCov ˆXHT, ŶHT + R 2 V ˆX HT V (Ŷr ) V (ŶHT ) Cov( ˆX HT, ŶHT ) V ( ˆX 1 HT ) 2 R ( ) Corr ˆX HT, Ŷ HT 1 CV ( ˆX HT ) 2 CV (Ŷ HT ) where CV (Ŷ HT ) = {V (Ŷ HT )} 1/2 /E(Ŷ HT ). Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 14 / 33
Ratio estimation Variance estimation Variance estimation : Use Êi = y i ˆRx i in the HT (or SYG) variance estimator. Example: SRS V (Ŷr ) = N2 n For variance estimation, use ˆV (Ŷr ) = N2 n ( 1 n ) 1 N N 1 ( 1 n ) 1 N n 1 N (y i Rx i ) 2 i=1 ( y i ˆRx i ) 2. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 15 / 33
Ratio estimation Application of the ratio estimator Hajek estimator: Ratio estimator of the mean using x i = 1 Domain estimation : The parameter of interest can take the form of the ratio N i=1 Ȳ d = δ iy i N i=1 δ i where δ i = 1 if i D and δ i = 0 if i / D. Thus, Ȳ HTd = π 1 i π 1 i δ i δ i y i is an (approximately) unbiased estimator of Ȳ d. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 16 / 33
1 Introduction 2 Ratio estimation 3 Regression estimator Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 17 / 33
Basic Setup Observe x i = (x 1i,, x Ji ) (auxiliary variables) and y i (study variable) in the sample We know X = N i=1 x i or X = N 1 N i=1 x i in advance. Interested in estimating Y = N i=1 y i ˆX HT = π 1 i x i can be different from X. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 18 / 33
Motivation - Difference estimator Motivation: Use auxiliary information at estimation stage Use a regression approach: 1 Suppose we have y o k = J b j x jk = b x k, k = 1, 2,, N, j=1 for some known J-dimensional vector b. The y o k is a proxy for y k. 2 Difference estimator : Ŷ diff = N i=1 y o i + y i y o i π i Unbiased (regardless of choice of y o k ) The variance is small if y o k = y k. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 19 / 33
Regression estimator How to choose y o k = b x k? - Let s estimate b from the sample. Regression estimator: Ŷ reg = N ŷ i + i=1 y i ŷ i π i, where ŷ i = ˆb x i and ˆb is estimated from the sample. Motivated from the linear regression superpopulation model E ζ (y i ) = x iβ V ζ (y i ) = σ 2. Note that β and σ 2 are superpopulation parameters. (Thus, b is the finite population quantity corresponding to β.) Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 20 / 33
Regression estimator How to estimate b? 1 Note that, under census, b can be estimated by solving U (b) N (y i b x i ) x i = 0. i=1 2 Consider an unbiased estimator of U (b): Û (b) = 1 π i (y i b x i ) x i 3 Obtain a solution ˆb by solving Û (b) = 0 for b. The solution is ( ) 1 1 ˆb = x i x 1 i x i y i. π i π i Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 21 / 33
Remark We will study that, under some conditions, Ŷ reg is asymptotically equivalent to Ŷ diff. Thus, Ŷ reg is asymptotically unbiased and V (Ŷ reg ) = V (Ŷdiff ) { } 1 = V (y i x i b) π i where ( N ) 1 N b = x i x 1 i x i y i. π i i=1 i=1 Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 22 / 33
Regression estimator Regression estimator ( ) Ŷ reg = Ŷ HT + X ˆX HT ˆb where ˆb = ( π 1 i x i x i ) 1 π 1 i x i y i. Note that, if 1 is in the column space of x i, we can write Ŷ reg = N i=1 ŷ i where ŷ i = x i ˆb. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 23 / 33
Regression estimator Regression estimator (of the mean): Ȳ reg = Ȳ HT + ( X X HT ) ˆb where ( X HT, Ȳ ) 1 ) HT = (ˆX HT N, Ŷ HT = 1 1 ( ) x N π i, y i. i Note that we can express Ŷ reg = ˆb 0 + ˆb X where ˆb 0 = Ȳ HT ˆb X HT. Thus, it is the predicted value of y at x = X when the linear regression model is used. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 24 / 33
Algebraic properties Linear in y: where Also, Ŷ reg = 1 π i g ia y i ( ( g ia = 1 + X ˆX ) 1 HT π 1 i x i x i) x i. Ȳ reg = 1 N 1 π i g ia y i. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 25 / 33
Algebraic properties Calibration property 1 π i g ia x i = X. The property (*) is also called benchmarking property. ( ) Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 26 / 33
If x i = (1, x i1 ), then Ȳ reg = Ȳ π + ( X 1 X π1 ) ˆb 1 and Ŷ reg = N {Ȳπ + ( X ) } 1 X π1 ˆb 1, where Ȳ π and X π1 are the Hajek estimators of the form ˆb 1 = [ ( X π1, Ȳ π ) = ( π 1 i and X 1 = N 1 N i=1 x i1. π 1 i ) 1 ( xi1 X π1 ) ( xi1 X π1 ) ] 1 π 1 ( ) i x i1, y i, π 1 ( ) i xi1 X π1 yi, Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 27 / 33
Weights in Ŷ reg = w iy i can be derived by minimizing Q (w) = π i ( w i 1 π i ) 2 subject to 1 π i g ia x i = X. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 28 / 33
Statistical Properties Let s consider the regression estimator of the mean We can express Ȳ reg = Ȳ HT + ( X X HT ) ˆb Ȳ reg = Ȳ HT + ( X ) ) X HT b + ( X X HT (ˆb b). = Ȳ HT + ( X ) X HT b = Ŷ diff where b = ( N i=1 x ix i) 1 N i=1 x iy i. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 29 / 33
Taylor linearization Note that Ȳ reg = Ȳ HT + ( X X HT ) ˆb is a (nonlinear) function of ( X HT, Ȳ HT, ˆb). Taylor linearization of Ȳ reg : Ŷ reg = f ( X HT, Ȳ HT, ˆb) { }. = f ( X, Ȳ, b) + Ȳ f ( X, Ȳ, b) (ȲHT Ȳ ) { } + X f ( X, Ȳ, b) ( X HT X ) { } + ˆb f ( X, Ȳ, b) (ˆb b) = Ȳ + ( Ȳ HT Ȳ ) + b ( X ) XHT + 0 Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 30 / 33
Statistical Properties (Cont d) Bias : Negligible Variance : { ).= Var (Ŷreg Var π 1 ( i yi x ib )} Variance estimation ˆV (Ŷreg ) = j A ij π ij Ê i Ê j π i π j where Ê i = y i x i ˆb. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 31 / 33
Example: SRS Variance of the regression estimator ).= 1 ( V (Ŷreg 1 n ) Se 2 n N where Se 2 = 1 N ( yi x N 1 ib ) 2. i=1 If 1 C(X ), then SST = SSR + SSE. That is, N ( yi Ȳ ) 2 N ( = y o i i=1 i=1 Ȳ ) 2 + N i=1 (y i y o i ) 2, where y o i = x i b. Thus, V (Ŷ reg ). = V (Ŷ HT )(1 R 2 ) V (Ŷ HT ). where R 2 = SSR/SST is the coefficient of determination. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 32 / 33
Remark The regression estimator is derived using a regression model. The validity (i.e. asymptotic unbiasedness) of the regression estimator does not depend on whether the regression model holds or not. However, the variance of the regression estimator is small if the regression model is good. That is, it is model-assisted, not model-dependent. Kim (ISU) Ch. 8: Estimation 1 Fall, 2014 33 / 33