Robust Small Area Estimation Using a Mixture Model

Transcription

1 Robust Small Area Estmaton Usng a Mxture Model Jule Gershunskaya U.S. Bureau of Labor Statstcs Partha Lahr JPSM, Unversty of Maryland, College Park, USA ISI Meetng, Dubln, August 23, 2011

2 Parameter of Interest: Small Area Means y : value of a characterstc of nterest for the jth unt j n area ( = 1,...,m;j = 1,...,N ) Parameter of nterest: N, Y N y f y (1 f ) Y 1 j r j1 n y n y ; f n N 1 j j1 and sample sze for area ; N and n are the populaton sze 2

3 Estmator of Small Area Means ˆr Y ˆ f y (1 f ) Y ˆ r Y s a model-dependent predctor of the mean of the non-sampled part of area ( 1,, m). If 0 Let f, Y ˆ Y ˆ n m 1 r n and N m N 1. 3

4 The Nested Error Regresson Model (Battese, Harter, Fuller, 1988) For = 1,...,m;j = 1,...,N, y =x β +v +ε T j j j xjs a vector of known auxlary β s the correspondng vector of parameters; v are random effects are errors n ndvdual observatons j d ~ (0, ) 2 d ~ N(0, ), 2 v N and j We assume that samplng s non-nformatve 4

5 EBLUP BLUP of Y r : N T 1 T xr N n x j jn 1 ( ), ˆ T Y x β ˆ vˆ, r r ˆβ s the BLUE of β, T vˆ ( ) ( ˆ n y x β). s the BLUP of v 2 EBLUP of Y r : Plug n estmates of and 2. 5

6 A Robust Unt-Level Model: An Extenson of the BHF Model For j = 1,...,N ; = 1,...,m, d 2 v ~ N(0, ), d zj Bn ~ (1; ), T y x β v, j j j d 2 2 j zj zj N 1 zjn 2 ~ (1 ) (0, ) (0, ), : probablty of belongng to mxture part 2. 6

7 Emprcal Best Predctor (EBP) ˆ T Y x β ˆ vˆ 1 m n n T T w ˆ j j j wj j yj v 1 j1 j1 ˆ β xx x( ) ˆ (1 ˆ ) ˆ ˆ, ˆ,, ˆ 2 2 wj 1 zj 2 zj zj E z yj xj θ 1 2 n ˆ ˆ ˆT ˆ 2 x β j D ˆ j1 1 1 n n n n, ˆ T j j j x j jx j j1 j1 j1 j1 vˆ ( y ), D w yˆ w w y w w, r r 7

8 Overall Bas-corrected REB m n REB ˆ ˆ e REBOBC REB 1 REB j Yr Yr n s b, REB 1 j1 s REB s : a robust measure of scale for the set of resduals REB e ; j 1,..., n, 1,..., m j e.g., REB REB REB s med e med( e ) j j b : a bounded Huber s functon wth the tunng parameter b = 5. 8

9 Smulaton set up (Chambers et al 2009) y x v, 1,...,40 j j j j 37,. Base model: v ~ N 0,3, ~ 0,6 Outly ng areas: v ~ N 9,20,..,40 Indvdual outlers: ~ 20,150 j Scenaros: 1) No contamnaton, [0,0] 2) Outlyng areas, [0,v] 3) Indvdual outlers, [e,0] 4) Indvdual outlers and outlyng areas, [e,v] SRSWOR N =100; n =5; 9

10 Table: Smulaton Results Scenaros 1-4 (250 runs), N =100, n = 5 No outlers Indvdual outlers only Area outlers Indvdual and area outlers Estmator / Scenaro [0,0] [0,u]/1 36 [e,0] [e,u]/1 36 [0,u]/37 40 [e,u]/37 40 Medan values of Relatve Bas (expressed as a percentage) EBLUP REBLUP (SR) MQ N SR+BC MQ+BC N2+OBC N2+OBC* Medan values of Relatve Root MSE (expressed as a percentage) EBLUP REBLUP (SR) MQ N SR+BC MQ+BC N2+OBC N2+OBC*

11 Estmaton of Crop Indcaton USDA-NASS has been publshng county level crop and lvestock estmates snce 1917 County ndcatons of crops such as harvested yeld are needed to assst farmers, agrbusnesses and government agences n local agrcultural decson makng. Most NASS Feld Offces conduct a separate County Estmates Survey every year. Data from multple sample surveys are used to estmate harvested yeld for varous crops at the county level. 10

12 Estmators Compared For seven md-western states n the year 2007, we compared the followng estmates, treatng the 2007 agrculture census as the gold standard. EBLUP under the BHF model EBP under NER Mxture Model [N2] Kott-Busselberg Model-Based Drect [KB] USDA-NASS offcal estmates 11

13 Crtera for Evaluaton AAD: the mean of absolute devatons between county estmates and correspondng 2007 census (PC) values ASD: the mean of squared devatons between estmates and PC values AARD: the mean of ratos between absolute devatons and PC values ASRD: the mean of squared ratos between absolute devatons and PC values PBC: the proporton of countes wth estmate less than the correspondng PC value. 12

14 Results The BHF and N2 estmates are clearly superor to the drect estmates for all the states consdered. EBPs are also better than the offcal estmates n all but one state (Mnnesota.) The OBC correcton to N2 provdes smlar results for most of the seven states. However, t provdes slghtly better results for Iowa, but slghtly worse results for Mnnesota. 13

15 Level 2 Regresson for Harvested Yeld: Mnnesota 14

16 Table: Estmaton Accuracy Measures for Harvested Yeld* State Estmator Metrc AAD ASD AARD ASRD PBC Illnos EBLUP KB N N2+OBC Offcal Iowa EBLUP KB N N2+OBC Offcal Mnnesota EBLUP KB N N2+OBC Offcal

17 Resdual Plots for BHF model for Soybeans yeld: Indana 16

18 Resdual Plots for BHF model for Soybeans yeld: Mnnesota 17

19 Parametrc Bootstrap Confdence Interval Defne the pvot vector: ˆ ˆ ˆ where ˆ 1 Y ˆ r Yr ˆ, ˆ Y Y Y 2 2 1,..., m,,..., 1 Y ˆ ˆ ˆ r Y1 r,... Ymr, r r mr 2 dag ˆ 2 m, ˆ D ˆ. 2 D ˆ 19

20 Generate v * N(0, ˆ 2 ) and * ˆ z ~ Bn(1; ). * 2 * Generate ej N(0, ˆ1 ), f zj 0 and e * (0, ˆ 2 j N 2 ), f j z 1. * j * A set of bootstrap data y j s obtaned as y ˆ v e x β, where 1,..., * T * * j j j j n, 1,..., m. Let 20

21 Y x β ˆ v be bootstrap versons of the true * T * r r populaton means. From the bootstrap data y * j, obtan the bootstrap estmates of the parameters ˆ, ˆ, ˆ, ˆ, ˆ β usng * * * * * 1 2 the same method as s used for the estmates 1 2 ˆ, β ˆ, ˆ, ˆ, ˆ. 21

22 ˆ ˆ ˆ * T * * Let Y x β v be a bootstrap estmate of Y *. r r r The vector ˆ * ˆ * 1 Y * ˆ * r Yr s a bootstrap approxmaton of. ˆ In the above, ˆ vˆ ( ) *2 * ˆ* ˆ* ˆ * y * *2 D ˆ x β and the estmated parameters nvolved are bootstrap 22

23 versons of the estmates of exactly the same form as the estmates based on the orgnal sample. The nterval estmate for Y r : Y ˆ q ˆ, Y ˆ q ˆ, r 1 r 2 where q 1 and q 2 are quantles of the dstrbuton of the bootstrap estmates * ˆ. 23

24 Table: Average coverage and length of dfferent confdence ntervals [0,0] 94.0 (3.6) 94.6 (3.6) 94.6 (3.6) [e 0,0] 92.0 (4.5) 95.9 (4.1) 95.8 (4.1) [e,0] 82.9 (52.7) 90.4 (3.9) 92.9 (3.9) 24

25 References: Battese, G. E., Harter, R. M. and Fuller, W. A. (1988). An error-components model for predcton of county crop areas usng survey and satellte data, Journal of the Amercan Statstcal Assocaton, 83, Bellow, M.E. (2007), Comparson of Methods for Estmatng Crop Yeld at the County Level, Unted States Department of Agrculture, Natonal Agrcultural Statstcs Servce, RDD Research Report. Gershunskaya, J. (2010), Robust Small Area Estmaton Usng a Mxture Model, Proc. SRMS Gershunskaya, J. and Lahr, P., (2008). Robust Estmaton of Monthly Employment Growth Rates for Small Areas n the Current Employment Statstcs Survey. Proceedngs of the Secton on Survey Research Methods, Amercan Statstcal Assocaton. Iwg, W.C. (1993), The Natonal Agrcultural Statstcs Servce County Estmates Program, Natonal Agrcultural Statstcs Servce. Kott, P.S. (2008), Some deas for a New Set of County-Estmates Crop Indcatons: an Update. Prusack, J. (2008), County Estmates/Small Area Estmaton. Rao, J.N.K. (2003). Small Area Estmaton, New-York, John Wley & Sons, Inc. Stasny, E. A., Goel, P.K., & Rumsey, D.J. (1991). County Estmates of Wheat Producton. Survey Methodology, 17,