Development of methodology for the estimate of variance of annual net changes for LFS-based indicators

Transcription

1 Development of methodology for the estimate of variance of annual net changes for LFS-based indicators Deliverable 1 - Short document with derivation of the methodology (FINAL) Contract number: Subject: Version: QTM 4 - for specific contract n ESTAT/ under framework contract n ESTAT/ Methodology for estimate of variance of annual net changes for LFS-based indicators V1 Date of first version: 13/05/2015 Date of updated version: 18/05/2015 Written by: Diffusion: Guillaume OSIER, Virginie RAYMOND Laurent Jacquet, Gwenaëlle Le Coroller, Virginie Raymond (SOGETI) Guillaume Osier (Expert) Fabrice Gras, Pepa Marinova, Hannah Kiiver (EUROSTAT)

2

3 TABLE OF CONTENTS 1. Introduction Variance estimation approach Case of linear indicators Variance estimation approach Case of non-linear indicators Extension to annual changes in the EU-LFS indicators Conclusion References... 17

4

5 1. Introduction Monitoring changes or trends in indicators over time is of key importance in many areas of economic and social sciences. For example, since the launch of the Europe 2020 (EU 2020) strategy for smart, sustainable and inclusive growth ( the data from the European Labour Force Survey (EU-LFS) have been used for policy targeting, as one of the EU 2020 headline targets, in relation to employment, is to raise the employment rate so to ensure that 75% of year-olds in the EU is in employment. The European Union Labour Force Survey (EU-LFS) is a large household sample survey, which is conducted in the 28 Member States of the European Union, 2 candidate countries and 3 countries of the European Free Trade Association (EFTA) in accordance with Council Regulation (EEC) No. 577/98 of 9 March The survey provides quarterly results on labour participation of people aged 15 and over as well as on persons outside the labour force. Following International Labour Organisation (ILO) guidelines and using common internal classifications (NACE, ISCO, ISCED, NUTS) in each country, the EU Labour Force Survey has become the reference source for comparable statistics on activity, employment and unemployment in the European Union (EU), covering a wide range of topics such as population, employment, unemployment, working time, permanency of the job, working conditions, professional status or educational background. In this paper, we describe possible approaches to estimate the variance for annual changes in LFSbased indicators, taking into account the overlap between quarterly and annual data. We will focus on the total number of employed (or unemployed) people and the employment (or unemployment) rate, both of them being key EU-LFS indicators. The indicators based on EU-LFS are sample estimates. This makes the interpretation of the difference between cross-sectional estimates misleading, unless an estimate for the variance is calculated along with the point estimate in order to judge whether or not the observed difference is statistically significant. Basically, if a confidence interval for the difference does not include 0, then we can conclude that the difference is statistically significant. Variance estimation for measures of change would be relatively straightforward if the cross-sectional estimates were based upon independent samples. In that case, we would only need to calculate the variance for each cross-sectional estimate. However, nearly all the countries involved in the EU-LFS have adopted a rotating design 2. The most commonly used design is the 2-(2)-2 scheme, where 1 See the LFS dedicated section on Eurostat s website: 2 In 2011, only two countries (Belgium and Luxembourg) did not implement a rotating design.

6 sampled units are interviewed for two consecutive quarters, than stay out of the sample for the next two quarters and are included again two more times afterwards. Figure 1: The 2-(2)-2 rotating design Quarter Another widely used rotation patterns consists of interviewing the panel members for five or six consecutive quarters before removing them from the sample permanently. Thus, 1/5 (five-wave rotating design) or 1/6 (six-wave rotating design) of the EU-LFS sample is refreshed every quarter. Thus, we cannot assume that the EU-LFS quarterly or annual estimates are time-independent and, therefore, temporal correlations between cross-sectional estimates also need to be reflected in variance calculations. Actually, several variance estimators for measures of change between two time points have been proposed in the statistical literature 3. Yet, none of those estimators seem, in our view, accurate as well as easy and flexible enough to be implemented in the context of an EUwide undertaking such as EU-LFS. Ideally, the methodology to be implemented for the estimation of variance of annual net changes should: 3 For instance: Kish (1965), Munnich and Zins (2011), Nordberg (2000), Qualité (2009), Qualité and Tillé (2008), Tam (1984) and Wood (2008) 6

7 take into account the overlap between quarterly data and between annual data (where annual data is derived from the average of quarterly data) reflect most of the complex design features adopted in the EU-LFS countries: stratification, unequal probabilities of selection, multi-stage selection, weighting adjustments for unit non-response and calibration to external data sources be able to deal with changes in levels, ratios and rates Figure 2: The six-wave rotating design Quarter In what follows, we present an estimator proposed by Berger and Priam (2013) and Berger and Oguz Alper (2013). This estimator has also been recommended by the Second Network for the analysis of the EU-SILC data ( Net-SILC2 project 4 ) to deal with measures of changes between the EU-SILC reference indicators of poverty, inequality and social exclusion (Eurostat, 2013) The proposed estimator has several advantages. It is theoretically justified. It is general enough to be valid under most of the EU-LFS sampling designs, which is actually a challenge considering the 4 7

8 important differences in sampling design between countries. As we ll see, the estimator can be implemented using standard statistical software, such as SAS, SPSS, Stata or R, and should require minimal computing power. In the next section, we describe the variance estimation approach in the case of linear indicators (i.e. population totals, means or proportions), then we elaborate on non-linear indicators such as ratio indicators. Finally, we explain how the approach can be extended to deal with variance estimation for annual changes in LFS-based indicators. 2. Variance estimation approach Case of linear indicators Suppose we wish to estimate the change between two population totals and of wave 5 1 and 2, where and denote respectively the values of the variable of interest at 1 and 2. For example, may be the total number of persons who are employed (unemployed, resp.) at 1 and may be the same number calculated at 2. In that case, (, resp.) for employed at 1 (employed at 2, resp.), 0 otherwise. The change is estimated by: Where and are the estimates of and based on the cross-sectional samples and at wave 1 and 2. and denote respectively the sampling weights at 1 and 2. Using these notations, the variance of is given by: ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) The cross-sectional variances ( ) and ( ) may be calculated using any traditional designbased variance estimator. For example, we may use the estimator proposed by Net-SILC2 (Eurostat, 2013), which relies on the ultimate cluster approximation: ( ) ( ) { } 5 In most cases, survey waves refer to quarters or years. 8

9 Where and ( ) The symbol is the stratum label and the number of strata. If there is no stratification, the whole target population can be regarded as a single stratum ( = 1). The symbol is the label of the primary sampling units (PSU). We have PSUs within stratum. The symbol is the household label within PSU of stratum, with a total of households. For single stage sampling designs, each household can be regarded as a PSU. The quantity is the value of the study variable for household in PSU of stratum. The quantity is the sampling weight for household in PSU of stratum. The correlation ( ) between the cross-sectional estimators and is certainly the most difficult part to estimate. In most cases, it cannot be ignored as the survey waves in EU-LFS are timecorrelated. Ignoring the correlation term would result in over-estimating the variance of and then under-interpreting the observed differences. Kish (1965) proposes to estimate the correlation by that calculated from the common sample at 1 and 2. Kish s estimator is biased as the correlation tends to be stronger among units from the common sample. As a result, the variance of may be underestimated and then the observed differences over-interpreted. The estimator proposed by Berger and Priam (2013) and Berger and Oguz Alper (2013) is based upon the residual matrix of a multivariate regression model. The model includes covariates which specify the stratification and interaction terms which specify the rotation of the sampling designs: ( ) ( ) ( ) ( ) ( ( ) ( ) ( ) ) ( ) Where and the residuals ( ) follow a bivariate normal distribution with mean zero and an unknown variance-covariance matrix. The and are (dummy) design variables which specify the stratification: { { ( ) ( ) ( ) ( ) ( ) and ( ) are regression parameters that need to be included into the model. 9

10 Assuming that the sampling design has a large entropy, the correlation between the estimated regression residuals and gives an estimate of the correlation ( ) between and. Berger and Priam (2013) showed that the estimator is also design-consistent. Simulation studies 6 have shown that the proposed estimator outperforms the traditional estimators by Kish (1965), Tam (1984) and Qualité and Tillé (2008). Furthermore, the approach can be easily extended to cope with multi-stage sampling designs by using the ultimate cluster approximation (e.g. Särndal et al., 1992). Yet, this approximation is based on the assumption that sampling fractions are negligible, which may not hold in practice. 3. Variance estimation approach Case of non-linear indicators The variance estimator introduced in the previous section can be extended to deal with measures of change between complex non-linear indicators. The idea is to apply the linearisation method (Särndal et al. (1992), Deville (1999), Denmati and Rao (2004), Wolter (2007), Osier (2009)) in order to approximate a complex non-linear statistic with a linear function of the observations, justified by asymptotic properties of the estimator. Then, a variance estimator of the linear approximation gives an estimator of the variance of the non-linear statistic. Let be a non-linear statistic and let assume that can be expressed as a smooth function of estimated population totals, : is the value of the study variable ( ) for and the are the sampling weights. Firstorder Taylor series approximation of leads to: ( ) Where the remainder tends towards zero as the sample size tends towards infinity. As a result, the variance of is asymptotically equal to that of the linear part ( ) ( ) is called the linearised" variable for 7. 6 See Andersson et al. (2011) and the deliverables of the DACSEIS project on 7 For examples see Osier (2009) 10

11 The linearisation approach based on influence functions (Deville, 1999) is general enough to cope with most of the commonly used indicators, including those which cannot be expressed as a smooth function of totals (e.g. quantiles, Gini coefficient). Suppose now that we wish to estimate the change between two non-linear parameters and of wave 1 and 2 (years or quarters). An estimator of is given by: Where and are the standard estimators of and based on the cross-sectional samples and of wave 1 and 2. For instance, if is the ratio of two population totals (such as the employment or the unemployment rate) at and the same ratio at, then we have: Let be the linear approximation of and the linear approximation of. For example, in case of ratio indicators, we obtain the following formula: ( ) ( ) Hence, the variance of is given by: ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) The cross-sectional variances ( ) and ( ) may be calculated using the estimator proposed by Net-SILC2 (see previous section). The correlation ( ) between and is estimated through the multivariate regression model proposed by Berger and Priam (2013), where the study variables and are substituted with their linearised counterparts and : ( ) ( ) ( ) ( ) ( ( ) ( ) ( ) ) ( ) 11

12 The estimated regression residuals and gives an estimate of the correlation ( ) between and. 4. Extension to annual changes in the EU-LFS indicators a. Case of the total number of employed (or unemployed) people Suppose we want to estimate the variance of the annual change in the total number of employed (or unemployed) people between years 1 and 2: Where and are the averages of the quarterly estimates at 1 and 2: ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) is the estimated number of employed people at the quarter of year 1 ( { }) ( ) ( ) ( ) is the estimated number of employed people at the quarter of year 2. These estimates are based on the quarterly EU-LFS samples and at 1 and 2. denote the sampling weights for the quarterly samples and, respectively, and ( ) and ( ) ( ) and ( ) is the 0/1 dummy indicator for the employed (1 if employed at the quarter of year 1 (of year 2 resp.), 0 otherwise). Using these notations, the variance of the annual change can be written as: ( ) ( ) ( ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ) 12

13 ( ( ) ( ) ( ) ( ) ) ( ( ) ( ) ( ) ( ) ) ( ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ) [ ( ( ) ) ( ( ) ) ] [ ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ] Where the matrices X and A contain the quarterly variances and covariances, respectively: ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ) 13

14 ( ) All the covariance terms in the matrix A can be estimated using the approach proposed by Berger and Priam (2013) and described in the previous sections. b. Case of the employment (or unemployment) rate The annual employment (unemployment, resp.) rate is defined as the ratio of the number of employed (unemployed, resp.) people in the population to the total number of active people (following the guidelines from the International Labour Organisation). Using the same notations as previously, the employment rate at year 1 (at year 2, respectively) can be written as: ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) is the estimated number of employed people at the quarter of year 1. ( ) ( ) is the estimated number of active people at the quarter of year 1. ( ) ( ) is the estimated number of employed people at the quarter of year 2. ( ) ( ) is the estimated number of active people at the quarter of year 2. Let ( ) ( ) ( ) be the estimated employment rate at the quarter of year ( { } { }). Assuming that the size of the active population remains (almost) unchanged from one quarter to another within the same year, we obtain: ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) 14

15 ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) Thus, the same approach as previously can be implemented to deal with annual changes in the employment rate, substituting the estimated quarterly employment rate ( ) for the estimated number of employed people ( ). Thus, suppose we want to estimate the change in the employment rate between years 1 and 2: Then we obtain: ( ) Where the matrices X and A are given by: ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ) ) ( ( ( ) ) ) ( ) All the variance and covariance terms in the above matrices can be calculated combining the linearisation procedure with the appropriate variance estimators. Using the previous notations, the linearised variable for the employment rate is: 15

16 ( ) ( ) ( ) ( ( ) ( ) ( ) ) ( ) ( ) ( ) ( ( ) ( ) ( ) ) ( { } ). 5. Conclusion The proposed variance estimator is both easy and flexible to cope with annual net changes in LFSbased estimators. The approach is also able to cope with domain estimators and can be applied to a large class of sampling designs and estimators using standard statistical techniques. The variance estimator can be implemented with SAS using, for instance, the matrix operations as provided in the SAS IML module. However, the implementation should be pretty cumbersome as we need to estimate the variance and covariance terms for eight quarterly estimates. There are a few approximations and limitations in this approach: The variance estimator proposed by Berger and Priam relies on the assumption of a high entropy design, which may not hold in practice. As the linearization procedure is justified on the basis of asymptotic properties, variance estimates may not be reliable if the sample size is not sufficiently large. This should be problematic with small domain estimation. In case of multi-stage sampling designs, we must suppose that sampling fractions at first stage are negligible for the ultimate cluster approximation to be valid. This assumption may not hold in practice. In order for the approach to get implemented, accurate sample design variables are needed: o When applicable, id codes for strata, Primary Sampling Units (PSUs), households and individuals o Sample weights o Each sampling unit in the database must receive a unique identifier which, above all, must remain consistent across time and across waves. In any case, any numerical results based on this approach should be taken with caution given the potential lack of sample design variables in the EU-LFS datasets and potential quality problems with the existing sampling design information. 16

17 References Andersson C., Andersson K. and Lundquist P. (2011), Variansskattningar avseende förändringsskattningar i panelundersökningar (Variance Estimation of change in panel surveys.), Methodology reports from Statistics Sweden. Berger, Y. G. and Oguz Alper, M. (2013), Variance estimation of change of poverty based upon the Turkish EU-SILC survey, paper presented at the NTTS (New Techniques and Technologies for Statistics) Conference, Brussels, 5-7 March Berger, Y. G. and Priam, R. (2013), A simple variance estimator of change for rotating repeated surveys: an application to the EU-SILC household surveys, University of Southampton, Statistical Sciences Research Institute. Available at Demnati, A. and Rao, J. N. K. (2004), Linearization variance estimators for survey data, Survey Methodology, 30: Deville, J-C. (1999), Variance estimation for complex statistics and estimators: linearization and residual techniques, Survey Methodology, 25, 2: Eurostat (2013), Standard error estimation for the EU-SILC indicators of poverty and social exclusion, Eurostat Methodologies and Working papers. Munnich, R. and Zins, S. (2011), Variance estimation for indicators of poverty and social exclusion, Work-package of the European project on Advanced Methodology for European Laeken Indicators (AMELI). Available at [Online; accessed 4 Jan. 2013] Nordberg, L. (2000), On variance estimation for measures of change when samples are coordinated by the use of permanent random numbers, Journal of Official Statistics, 16: Available at: URL = Osier, G. (2009), Variance estimation for complex indicators of poverty and inequality using linearization techniques, Survey Research Methods, 3, 3: Available at Qualité, L. (2009), Unequal probability sampling and repeated surveys, PhD Thesis, University of Neuchatel, Switzerland. Qualité, L. and Tillé, Y. (2008), Variance estimation of changes in repeated surveys and its application to the Swiss survey of value added, Survey Methodology, 34: Särndal, C-E., Swensson, B. and Wretman, J. (1992), Model Assisted Survey Sampling, Springer Series in Statistics. 17

18 Tam, S. M. (1984), On covariances from overlapping samples, The American Statistician, 38: Wolter, K. M. (2007), Introduction to Variance Estimation, 2 nd ed. New York: Springer. Wood, J. (2008), On the covariance between related Horvitz-Thompson estimators, Journal of Official Statistics, 24: Available at: URL = 18