Small Area Estimation: Methods, Applications and New Developments. J. N. K. Rao. Carleton University, Ottawa, Canada

Transcription

1 Small Area Estmaton: Methods, Applcatons and New Developments J. N. K. Rao Carleton Unversty, Ottawa, Canada Paper presented at the NTTS 2013 Conference, Brussels, March

2 Introducton Censuses and admnstratve records have lmted scope. Sample surveys can provde relable current statstcs for large areas or subpopulatons (domans) wth large sample szes. Growng demand for relable small area statstcs but sample szes are too small to provde drect (or area specfc) estmators wth acceptable accuracy. 2

3 Examples of small areas: county, muncpalty, three dgt occupaton counts wthn a provnce, health regons, even a state by age-sex-race groups. Doman or subpopulaton s called a small area f the doman-specfc sample sze s small. It s necessary to borrow strength from related areas through lnkng models based on auxlary data such as recent census and admnstratve records. Ths leads to ndrect estmators. Major applcaton (SAIPE): Estmaton of counts and rates for school age chldren under poverty n USA at the 3

4 county and school dstrct level. More than 15 bllon dollars of federal funds are allocated annually on the bass of model-based ndrect estmators constructed from the Amercan Communty Survey (ACS) data, admnstratve records and recent census. ACS s a large survey that samples all countes each month and drect annual county ACS estmates are used n constructng model-based estmates. Before the ACS, data from the Current Populaton Survey (CPS) were used; only 1000 of the 3000 countes sampled n CPS. 4

5 Desgn ssues Can we mnmze the use of ndrect estmators by takng preventve measures at the desgn and/or estmaton stage? At the desgn stage, possble actons: (a) Replace clusterng: use lst frames (b) Use many strata (c) Compromse sample allocaton: Canadan LFS used a two-step allocaton: optmzed at the provncal level and at UI level. Maxmum coeffcent of varaton (CV) reduced from 18% to 9% at UI level wth slght ncrease n CV at the provncal level. (d) Optmal allocaton: Mnmze total sample sze subject to desred tolerances on the CVs of drect (or ndrect) estmators of 5

6 areas and aggregate of areas (Choudhry, Rao and Hdroglou 2012) (e) Integraton of surveys, multple frames, rollng samples. Estmaton stage: use effcent drect estmators. The clent wll always requre more than s specfed at the desgn stage (Fuller 1999). So we cannot avod unplanned small domans. Methods for ndrect small area estmaton Parameters of nterest: Area means, totals, proportons and quantles. Complex measures: Poverty ndcators 6

7 (for example, poverty rate, gap and severty used by the World Bank). Tradtonal post-stratfed synthetc estmator: Populaton post-stratfed nto subgroups g that cut across small areas and cell populaton counts Ngare known. Relable drect estmates Yˆ +g and N ˆ + g of the post-strata totals Y + gand N + g are used to get a poststratfed synthetc estmator of small area total Y as = ˆ PS Y ( ˆ / ˆ = g Ng Y+ g N+ g ) Implct assumpton: Y g = Y+ g 7

8 Area-level Fay-Herrot model: Drect area estmates and area-level auxlary data avalable. Consder smple random samplng wthn areas ( = 1,..., m) and sample area mean y as drect estmator of the populaton area mean Y. Some areas of nterest may not be sampled at all. Samplng model: y = Y + e, e ~ nd N(0, ψ ), ψ known Lnkng model: Y = z β + v v ~ N(0, σ ) d 2 v 8

9 z = Vector of area-level covarates obtaned from census and admnstratve sources Combned model: y = z β + v + e, = 1,..., m Specal case of a lnear mxed model Optmal estmaton of Y Best (Bayes) estmator under the assumed model s the condtonal expectaton of Y gven the survey estmator 2 y and model parameters β and σ v : 9

10 Yˆ B = γ y + ( 1 γ ) z β, γ = σ /( σ + ψ ) 2 v 2 v Best estmator s a weghted combnaton of drect estmator yand regresson synthetc estmatorz β. More weght s gven to drect estmator when the samplng varance s small relatve to total varance and more weght to the synthetc estmator when samplng varance s large or model varance s small. In practce, we need to estmate model parameters: Fay and Herrot (1979) method of moments (MM), maxmum lkelhood (ML) or restrcted ML (REML). 10

11 Resultng estmator s called emprcal best or emprcal Bayes (EB): ˆ EB Y = ˆ γ y + (1 ˆ γ ) z ˆ β MM does not requre normalty assumpton and the resultng EB estmator s also EBLUP (emprcal best lnear unbased predcton) estmator. Customary weghted least squares estmator of β may not be the best from a predcton pont of vew unless the mean model z β s correctly specfed. Jang, Nguyen and J. S. Rao (2011) proposed alternatve 11

12 estmator of β that makes the EB estmator perform well under msspecfcaton of the mean model. 2 Estmaton of σ v often leads to dffcultes due to negatve estmates whch are truncated to zero. Ths s an area of actve research and one recent method, called adjusted ML method under normalty, gves strctly postve estmates (L and Lahr, 2010). Prelmnary test estmaton (Datta et al. 2010): Test 2 for the hypothess H0 : σ v = 0 usng sgnfcance level α = 0.2. If H 0 s not rejected, use regresson synthetc 12

13 estmator under the model wthout the random effects, otherwse use the EB estmator. Assumpton of known samplng varances ψ s also a problem. Proposed solutons nclude smoothng estmated samplng varances usng generalzed varance functon (GVF) method. For non-sampled areas wth known z, regresson synthetc estmator z βˆ s used. A more realstc lnkng model s to replace Y by h Y ) for some sutable functon h (.). For example, ( 13

14 use a logstc transformaton f the mean s a proporton. In ths case, samplng model and lnkng model are msmatched and hence cannot be combned nto a lnear mxed model. More sophstcated methods are needed (You and Rao 2002). Mohadjer et al. (2012) appled msmatched models to estmate the county proportons of adults at the lowest lteracy level usng herarchcal Bayes (HB) methods. Samplng model s modfed to match the lnkng model by takng t as h ( y ) = h( Y ) + v + e~, but the mean of nduced samplng error e ~ may not be zero. Fnd the EB estmator of h Y ) and then back ( 14

15 transform to get the estmator of Y whch s not EB. SAIPE uses ths estmator after bas adjustment. Benchmarkng: (a) Adjust the EBLUP estmators to make the estmators agree wth a relable drect estmator at an aggregate level. For example, use rato benchmarkng. (b) Self-benchmarkng: Use a dfferent estmator of β (You, Rao and Hdroglou 2012) or augment the model usng wψ as the addtonal covarate where w s the weght used for aggregaton (Wang et al. 2008). 15

16 Mean squared predcton error MSPE ˆ EB 2 ( Y ) = ˆ EB E( Y Y ) s used as a measure of precson of the EB estmator. If the number of areas m s moderately large then MSPE ˆ EB ( Y ) g ( σ ) + g ( σ ) + g ( σ ) 1 v 2 v 3 v 2 Leadng term g1 ( σ v ) = γ ψ shows large reducton n MSPE can be acheved relatve to the drect estmator y f γ s small. Second term s due to the 16

17 estmaton of β and the last term s due to 2 estmaton of σ v and they are of lower order. MSPE estmator s obtaned by replacng σ v by ts estmator and multplyng the last term by the factor 2. Alternatve methods use resamplng ncludng jackknfe and bootstrap whch are more computer ntensve but more wdely applcable. MSPE estmaton s an area of actve research. For the prelmnary test estmator, use the MSPE estmator under the model wthout random effects 2 17

18 when H 0 s not rejected, otherwse use the usual MSPE estmator. Propertes are under study (Molna, Rao and Datta 2013). Model selecton and checkng Varable selecton: Fence method (Jang et al. 2008), Condtonal Akake Informaton Crteron (AIC) for predctve performance (Han 2011) 18

19 Model checkng: Resdual analyss (weghted Q-Q plots, nfluental dagnostcs: Rao 2003, Chapters 6 and 7) Applcatons of area level model SAIPE for countes Drect county estmates of total poor obtaned from ACS and log (total poor) taken as the response varable n the model. Predcton varables nclude log of the followng: food stamps, poor from tax forms, number of exemptons, last census poor. 19

20 Extensve nternal model checkng gnorng the random area effects and usng fxed effects model. External evaluatons usng census estmates (Chapter 7, Rao 2003). Occupatonal proportons Three dgt codes nested n two dgt code categores (Canadan Labor Force Survey). Here nterest s n estmatng three dgt code proportons or counts wthn each provnce. Ths leads to an extenson of the Fay-Herrot model to vector case (Berg and Fuller 2012). 20

21 Lnkng model: True proporton for category k and provnce s wrtten as p k = ~ pk + vk where p~ k s based on census proportons and a logstc model and v k s the model error. Unt level models: Nested error regresson model Suppose we select smple random samples of szes n from the areas wth szes N and observe unt responses {( y 1,..., y ); = 1,..., m} and assocated n covarates xjwth known area means X. Objectve s to estmate the small area means Y. 21

22 Model yj = xj β + v + ej, j = 1,..., n ; = 1,..., m v 2 2 σ v ), ej ~ d N(0, σ e ), ~ d N(0, v e j Area mean Y X β + v f n s small relatve to where the area populaton means X known from census or admnstratve records. EB estmator of the mean Y s a weghted average of the sample regresson estmator y + ˆ( β X x ) and the regresson synthetc estmator X βˆ wth weghts ˆ γ = ˆ σ v /( ˆ σ v + ˆ σ e / n ) and 1 γˆ respectvely. EB estmator can be wrtten as N 22

23 EB Yˆ = X ˆ β + vˆ = j U yˆ j Desgn consstency and benchmarkng Pseudo-EBLUP estmator obtaned from an aggregated model based on desgn weghts ensures desgn consstency and self-benchmarkng to a relable drect estmator of a larger area coverng small areas (You and Rao 2002). Informatve samplng: Populaton model does not hold for the sample. Current research provdes tools to handle ths case. Pfeffermann and Sverchkov (2007) proposed a bas- 23

24 adjusted estmator of a small area mean. Augmentng the nested error model by ncludng desgn weghts as addtonal covarates and then dong EB s a smple method and seems to work well (Verret, Hdroglou and Rao 2012). But t assumes that the sum of the populaton weghts for each area s known. Impact of samplng desgn on model-based estmators (Burgard,, Munnch and Zmmermann 2012) Desgn-based smulaton study under stratfed random samplng and unequal probablty samplng wthn strata, based on synthetc busness data resemblng small and medum enterprses from the Italan busness regster. Small 24

25 areas of nterest: 103 Italan provnces, 927 strata. Parameters of nterest are the means of value added and auxlary varables used are number of employees and revenue. What can go wrong? (1) Outlers n random effects v and /or unt errors e j : Use robust EBLUP assumng mean zero random effects and unt errors (Snha and Rao 2009). (2) Relax the mean assumpton by replacng mean functon x n the model by some smooth functon and β j 25

26 approxmate t by a P-splne. Resultng model has a lnear mxed model form so use EBLUP (Opsomer et al. 2008). Robust EBLUP verson to handle outlers: Rao et al. (2010). (3) What f the specfed model s wrong? Use a model asssted approach by treatng the model as a workng model and then dong desgn-based bas correcton (Lehtonen and Vejanen 1999). Under smple random samplng wthn areas, estmator of mean Y s gven by Yˆ EB, c 1 = N { j U yˆ j + ( N / n ) j s( ) ( y j yˆ j )} 26

27 Note that f n s small the bas-corrected estmator may have large CV because the bas correcton s a drect estmator based only on the sample s () n area EB estmaton of small area poverty ndcators E j s a welfare measure for ndvdual j n area and z s the poverty lne. World Bank (WB) famly of poverty ndcators: α j j j < 1 = j Fα = {( z E ) / z} I( E z) F N α U F αj 27

28 α = 0: Poverty ncdence, α =1: poverty gap Also called FGT poverty measures (Foster et al. 1984) Transform Ejto yj = log ( E j ) and express F α as a functon of the y j, say h α ( y ). EB estmator of F α = condtonal expectaton of h α ( y ) wth respect to the estmated predctve dstrbuton of non-sampled y r gven the sample y s. 28

29 Implementaton (Monte Carlo approxmaton) ( y l ) Generate Lnon-sampled r, l = 1,..., L from the estmated predctve dstrbuton. Attach the sampled elements to form smulated census ( vectors y l ), l = 1,..., L. Calculate the desred poverty measure wth each ( l) ( l) populaton vector: Fα = h α ( y ), l = 1,..., L. Take the average over the Lsmulated censuses as an approxmaton to the EB estmator: Under the nested error model on y j and normalty, we can generate values from the estmated predctve 29

30 dstrbuton usng only unvarate normal dstrbutons (Molna and Rao 2010). F EB α L 1 L l= 1 F ( l) α Bootstrap s used for MSE estmaton. WB uses another smulated census method but the proposed EB method can be consderably more effcent than the WB method for sampled areas. 30

31 Current Research Relax normalty assumpton by assumng a famly of skew normal dstrbutons for the random effects and the errors n the nested error model. WB s also studyng EB method wthout normalty assumpton, usng a mxture of normal dstrbutons approach. Herarchcal Bayes estmaton (Molna, Nandram and Rao 2013): Computatonally smpler than the EB as t avods bootstrap MSE estmaton. Permts nterval estmaton. 31

32 M-quantle method Uses unt level data and assumes that the quantles of the condtonal dstrbuton of y gven x are lnear n x. Random small area effects are not drectly ncorporated nto the model (Chambers and Tzavds 2006). Estmators of area means are essentally synthetc f the area samplng fracton s small. Bas adjusted M- quantle estmators (Tzavds et al. 2010) can lead to sgnfcant ncrease n MSE. Provdes estmators that are robust to outlers. Extends to other parameters such as the FGT poverty measures (Gust, Marchett, Prates and Salvat 2012). Not clear 32

33 how to handle more complex poverty ndcators such as the Fuzzy Monetary Index. It s not clear how to estmate parameters of nonsampled areas usng M-quantle method or how to use area level data. Recommendatons (1) Preventve measures at the desgn stage may reduce the need for ndrect estmators sgnfcantly. (2) Good auxlary nformaton related to the varables of nterest plays a vtal role n model-based estmaton. Expanded access to auxlary data, such as census and 33

34 admnstratve data, through coordnaton and cooperaton among federal agences s needed. (3) Model selecton and checkng plays an mportant role. External evaluatons are also desrable whenever possble. (4) Area-level models have wder scope because area-level data are more readly avalable. But assumpton of known samplng varance s restrctve. (5) HB approach s powerful and can handle complex modelng, but cauton should be exercsed n the choce of prors on model parameters. Practcal ssues n mplementng 34

35 HB paradgm should be addressed (Rao 2003, Secton ). (6) Model-based estmates of area totals and means not sutable f the objectve s to dentfy areas wth extreme populaton values or to dentfy areas that fall below or above some pre-specfed level (Rao 2003, Secton 9.6). (7) Sutable benchmarkng s desrable. (8) Model-based estmates should be dstngushed clearly from drect estmates. Errors n small area estmates may be more transparent to users than errors n large area estmates. 35

36 (9) Proper crteron for assessng qualty of model-based estmates s whether they are suffcently accurate for the ntended uses. Even f they are better than drect estmates, they may not be suffcently accurate to be acceptable. (10) Overall program should be developed that covers ssues related to sample desgn and data development, organzaton and dssemnaton, n addton to those pertanng to methods of estmaton for small areas. 36

37 References Berg, E.J. and Fuller, W. A. (2012). Estmaton of error covarance matrces for small area predcton. Computatonal Statstcs and Data Analyss, 56, Burgard, J. P., Munnch, R. and Zmmermann, T. (2012). The mpact of samplng desgn on small area estmates for busness data. Techncal Report. Chambers, R. and Tzavds, N. (2006). M-quantle models for small area estmaton. Bometrka, 73, Choudhry, G. H., Rao, J. N. K. and Hdroglou, M. A. (2012)). On sample allocaton for effcent doman estmaton. Survey Methodology, 38, Datta, G. S., Hall, P. and Mandal, A. (2011). Model selecton for the presence of small-area effets and applcatons to area-level data. Journal of the Amercan Statstcal Assocaton, 106, Fay, R.E. and Herrot, R. A. (1979). Estmates of ncome for small places: an applcaton of James- Sten procedures to census data. Journal of the Amercan Statstcal Assocaton, 74, Foster, J., Greer, J. and Thorbecke, E. (1984). A class of decomposable poverty measures. Econometrca, 52,

38 Fuller, W. A. (1999). Envronmental surveys over tme. Journal of the Agrcultural, Bologcal and Envronmental Statstcs, 4, Gust, C., Marchett, S., Prates, M. and Salvat, N. (2012). Robust small area estmaton and oversamplng n the estmaton of poverty ndcators. Survey Research Methods, 6, Jang, J., Rao, J. S., Gu, Z and Nguyen, T. (2008). Fence methods for mxed model selecton. Annals of Statstcs, Jang, J., Nguyen, T. and Rao, J. S. (2011). Best predctve small area estmaton. Journal of the Amercan Statstcal Assocaton, 106, Lehtonen, R., Sarndal, C. E. and Vejanen, A. (2003). The effect of model choce n estmaton for domans. Survey Methodology, 29, L, H. and Lahr, P. (2010). An adjusted maxmum lkelhood method for solvng small area estmaton problems. Journal of Multvarate Analyss, 101, Mohadjer, L. et al. (2012). Herarchcal Bayes small area estmaton of adult lteracy usng unmatched samplng and lnkng models. Journal of the Indan Socety of Agrcultural Statstcs, 66,

39 Molna, I. and Rao, J. N. K. (2010). Small area estmaton of poverty ndcators. Canadan Journal of Statstcs, 38, Molna, I, Nandram, B. and Rao, J. N. K (2013). Small area estmaton of general parameters wth applcaton to poverty ndcators: a herarchcal Bayes approach. Techncal Report. Opsomer, J.D., Claeskens, G., Ranall, M. G. Kauemann, G. and Bredt, F. J. (2008). Nonparametrc small area estmaton usng penalzed splne regresson. Journal of the Royal Statstcal Socety, seres B, 70, Pfeffermann, D. and Sverchkov, M. (2007). Small-area estmaton under nformatve probablty samplng. Journal of the Amercan Statstcal Assocaton, 102, Rao, J. N. K. (2003). Small Area Estmaton. Wley, Hoboken, New Jersey. Rao, J. N. K., Snha, S. K. and Roknossadat, M. (2009). Robust small area estmaton under penalzed splne mxed models. In Proceedngs of the Survey Research Secton, Amercan Statstcal Assocaton, pp Snha, S. K. and Rao, J. N. K. (2009). Robust small area estmaton. Canadan Journal of Statstcs, 37,

40 Tzavds, N., Marchett, S. and Chambers, R. (2010). Robust estmaton of small-area means and qunatles. Australan and New Zealand Journal of Statstcs, 52, You, Y. and Rao, J. N. K. (2002). A pseudo-emprcal best lnear unbased predcton approach to small area estmaton usng survey weghts. Canadan Journal of Statstcs, 30, You, Y., Rao, J. N. K. and Hdroglou, M. (2012). On the performance of self benchmarked small area estmators under the Fay-Herrot area level model. Survey Methodology, 38 (n press). Verret, F., Hdroglou, M. A. and Rao, J. N. K. (2012). Model-based small area estmaton under nformatve samplng. Techncal Report. Wang, J., Fuller, W. A. and Qu, Y. (2008). Small area estmaton under a restrcton. Survey Methodolgy, 34,