Small Area Estimation in R with Application to Mexican Income Data

Transcription

1 with Application to Mexican Income Data Ann-Kristin Kreutzmann 1, Sören Pannier 1 Natalia Rojas-Perilla 1, Timo Schmid 1 Matthias Templ 2 Nikos Tzavidis 3 1 Freie Universität Berlin 2 Zürcher Hochschule für Angewandte Wissenschaften 3 University of Southampton New Techniques and Technologies for Statistics (NTTS) March 16, 2017 Ann-Kristin Kreutzmann 1 (18) - NTTS 2017

2 Why Using Small Area Estimation Population of interest (or target population): population for which the survey is designed, here the State of Mexico (EDOMEX) Direct estimators should be reliable for the target population Gini - Direct Figure: Gini coefficient for EDOMEX using direct estimation. Ann-Kristin Kreutzmann 2 (18) - NTTS 2017

3 Introduction to Small Area Estimation Why Using Small Area Estimation Population of interest: Sub-population of the population (domain), planned or not in the survey design,here municipalities in EDOMEX, Direct estimators may be unreliable due to small sample sizes Gini - Direct Gini - SAE High High Low Low Figure: Gini coefficients for municipalities in EDOMEX using direct estimation (left) and a Small Area Estimation (SAE) method (right). Municipalities filled with grey color represent areas with zero sample size. Ann-Kristin Kreutzmann 3 (18) - NTTS 2017

4 Combining Different Data Sources In order to provide reliable estimates in all subdomains, efficient ways of combining information are required Main idea: 1. Usage of observed/collected data to fit a suitable model Y X 2. Produce predictions in all the domains using available covariates Types of covariates: - Aggregate data provide covariates for every domain - Individual data provide covariates for every individual in every domain Ann-Kristin Kreutzmann 4 (18) - NTTS 2017

5 R Packages for SAE CRAN Task View: Official Statistics & Survey Methodology nlme and lme4 for mixed-effects models hbsae for basic area- and unit-level models fitted by restricted maximum likelihood or hierarchical Bayes rsae for robust basic unit- and area-level models JoSAE for unit-level models and the generalized regression estimator (main purpose: documentation of function used in publications) Ann-Kristin Kreutzmann 5 (18) - NTTS 2017

6 R Packages for SAE Other packages BayesSAE for area-level models in Bayesian context saerobust for robust area level models saery and sae2 for area-level models with time effects sae for a wide variety of SAE methods including area-level and unit-level models for the mean as well as models for the estimation of non-linear parameters emdi for the estimation and visualization of non-linear indicators Ann-Kristin Kreutzmann 6 (18) - NTTS 2017

7 Gini coefficients for municipalities in EDOMEX How does a statistical institute receive the map of Gini coefficients for the municipalities in EDOMEX? We assume that survey and census data is available for EDOMEX SAE methods that enable the estimation of non-linear indicators as the Gini coefficient are implemented in the sae package and in package emdi Since package emdi supports the whole process from the estimation over model evaluation and visualization functions from this package are used for the following example Ann-Kristin Kreutzmann 7 (18) - NTTS 2017

8 Estimation The method that is used to receive indicators for the municipalities is the Empirical Best Prediction (EBP) approach by Molina and Rao (2010). 1 modelfit <- ebp ( fixed = ictpc ~ pcocup + jnived + 2 clase _ hog + pcpering + bienes + 3 actcom, 4 pop _ data = census, pop _ domains = " domain _id", 5 smp _ data = survey, smp _ domains = " domain _id", 6 transformation = " box. cox ", 7 MSE = TRUE, 8 custom _ indicator = 9 list (my_max = function (y, pov _ line ){ max (y)}, 10 my_min = function (y, pov _ line ){ min (y)}), 11 na. rm = TRUE ) Ann-Kristin Kreutzmann 8 (18) - NTTS 2017

9 Data and Model Diagnostics (1) 1 > summary(modelfit) 2 Empirical Best Prediction 3 4 Out -of - sample domains : 67 5 In - sample domains : Sample sizes : 8 Units in sample : Units in population : Min. 1 stqu. Median Mean 3 rdqu. Max. 12 Sample _ domains Population _ domains Ann-Kristin Kreutzmann 9 (18) - NTTS 2017

10 Data and Model Diagnostics (2) 1 Explanatory measures : 2 Marginal _ R2 Conditional _ R Residual diagnostics : 6 Skewness Kurtosis Shapiro _ W Shapiro _ p 7 Error e Random _ effect e ICC : Transformation : 13 Transformation Method Optimal _ lambda Shift _ parameter 14 box. cox reml Ann-Kristin Kreutzmann 10 (18) - NTTS 2017

11 Graphical Diagnostics 1 > plot(modelfit) Error term Random effect Quantiles of pearson residuals Theoretical quantiles Quantiles of random effects Theoretical quantiles Density Density Pearson residuals Pearson residuals Density Density Standardized random effects Standardized random effects Cook's Distance Plot Box Cox REML Cook's Distance Log Likelihood Index λ Ann-Kristin Kreutzmann 11 (18) - NTTS 2017

12 Selection of Indicators Function estimators helps to select the indicators the user is interested in Additionally, the coefficient of variation can be received The user can choose single indicators or groups of indicators 1 > head(estimators(object = modelfit, indicator = "Gini", CV = TRUE)) 2 Domain Gini Gini _ CV 3 1 Acambay Acolman Aculco Almoloya de Alquisiras Almoloya de Juárez Almoloya del Río Ann-Kristin Kreutzmann 12 (18) - NTTS 2017

13 Introduction to Small Area Estimation Visualization 1 > map table <- data.frame(domain = unique(census$domain id), 2 mun = sort(shp mex$mun)) 3 4 > map plot(object = modelfit, CV = TRUE, map obj = shp mex, 5 indicator = "Gini", map dom id = "mun", map tab = map table) Gini Ann-Kristin Kreutzmann Gini CV High High Low Low 13 (18) - NTTS 2017

14 Export to excel 1 > write.excel(modelfit, file ="excel output.xlsx", indicator = "Gini", CV = TRUE) Ann-Kristin Kreutzmann 14 (18) - NTTS 2017

15 Official statistics are interested in disaggregated indicators SAE methods enable the estimation of disaggregated indicators and several packages in R provide these methods Non-linear indicators like the Gini coefficient or the At-risk-of-poverty rate are of special interest Package sae and package emdi enable the estimation of these indicators As shown, package emdi provides an overall package for the user from the estimation to the visualization and export of results Ann-Kristin Kreutzmann 15 (18) - NTTS 2017

16 References Bates, D., Maechler, M., Bolker, B. and Steve Walker (2015). Fitting Linear Mixed-Effects. Models Using lme4. Journal of Statistical Software, 67(1), doi: /jss.v067.i01. Boonstra, H.J. (2012). hbsae: Hierarchical Bayesian Small Area Estimation. R package version 1.0., URL: Breidenbach, J. (2015). JoSAE: Functions for some Unit-Level Small Area Estimators and their Variances. R package version , URL: Chengchun Shi and with contributions from Peng Zhang (2013). BayesSAE: Bayesian Analysis of Small Area Estimation. R package version , URL: Ann-Kristin Kreutzmann 16 (18) - NTTS 2017

17 References Esteban Lefler, M. D., Morales Gonzalez, D. and Perez Martin, A. (2014). saery: Small Area Estimation for Rao and Yu Model. R package version 1.0., URL: Fay, R.E. and Diallo, M. (2015). sae2: Small Area Estimation: Time-series Models. R package version , URL: Kreutzmann, A., Rojas-Perilla, N., Schmid, T., Templ, M. and Tzavidis, N. (2017). emdi: Estimating and Mapping Disaggregated Indicators. R package version Molina, I. and Marhuenda, Y. (2015). sae: An R Package for Small Area Estimation. The R Journal 7(1), Ann-Kristin Kreutzmann 17 (18) - NTTS 2017

18 References Molina, I. and Rao, J.N.K. (2010). Small area estimation of poverty indicators. The Canadian Journal of Statistics 38(3), Pinheiro, J., Bates, D., DebRoy, S., Sarkar, D. and R Core Team (2016). nlme: Linear and Nonlinear Mixed Effects Models. R package version , URL: Rao, J.N.K. & Molina, I. (2015), Small Area Estimation. John Wiley & Sons Schoch, T. (2014). rsae: Robust Small Area Estimation. R package version Warnholz, S. (2016). saerobust: Robust Small Area Estimation. R package version , URL: Ann-Kristin Kreutzmann 18 (18) - NTTS 2017