Combining Time Series and Cross-sectional Data for Current Employment Statistics Estimates 1

Transcription

1 JSM015 - Surey Research Methos Section Combining Time Series an Cross-sectional Data for Current Employment Statistics Estimates 1 Julie Gershunskaya U.S. Bureau of Labor Statistics, Massachusetts Ae NE, Suite 4985, Washington, DC, 01 Abstract Estimates from the Current Employment Statistics (CES) Surey are prouce base on the ata collecte each month from the sample of businesses that is upate once a year. In some estimation cells, where the sample is not large enough, the Fay-Herriot moel is use to improe the estimates. Uner the current approach, the moel combines information from a set of areas an is estimate inepenently eery month. Gien the esign of the surey, it may be beneficial to borrow information not only cross-sectionally but also oer time. This paper explores the feasibility of applying such a moel. The results are ealuate base on historical "true" employment ata aailable on a lagge basis. Key Wors: small area estimation, Fay-Herriot moel, Current Employment Statistics Surey 1. Introuction Estimation for omains where the traitional irect sample base estimator lacks precision requires strengthening the estimator by using moeling assumptions. In the past seeral ecaes, the methoology for estimation in such unplanne omains has grown into a fiel of Small Area Estimation (SAE). The literature on the subject is rich an it is still growing (see Rao 003; Pfeffermann 00, 013) The quality of the result in SAE epens on the amount an releance of the information summone by the moel. Sometimes, the parsimoniousness of the moel an the ability to inclue more imensions of the aailable ata are at os. This paper consiers application of alternatie moels in estimation of employment from the Current Employment Statistics (CES) surey conucte by the U.S. Bureau of Labor Statistics (BLS). Gien the esign of the surey, it is reasonable to expect that it is beneficial to base the moel on information aailable not only across areas but also oer time. This paper explores the feasibility of applying such a moel. The results are ealuate base on historical "true" employment ata, aailable to CES on a lagge basis. Contrary to our expectations, the empirical results show that, in the case of the CES series consiere in our research, the classical Fay-Herriot moel that borrows information across areas at a gien point in time works about as well as a more sophisticate Rao-Yu moel that combines information oer areas an time. One reason the results were so close is that both the Fay-Herriot an Rao-Yu moels use in this research inclue the same preictor that capture most useful information regaring the estimates. Still, we were perplexe by the 1 Any opinions expresse in this paper are those of the author an o not constitute policy of the Bureau of Labor Statistics. 1085

2 JSM015 - Surey Research Methos Section obseration that in a number of cases the simpler Fay-Herriot moel performe slightly better than the more complete Rao-Yu moel. We inestigate possible reasons of this phenomena using the simulation stuy. The paper is organize as follows. In Section, we introuce the CES setup: the ata an the CES estimator. We talk about the reasons why borrowing information across time might be beneficial an iscuss the coariance structure of the sampling errors in the CES series. We introuce the moels in Section 3. In Section 4, we present results from the real ata analysis. In Section 5, we use simulate ata to stuy the effect of arious alues of the moel parameters on the results of the moel fit. The ata is generate from moels similar to the ones that are assume to goern the real ata.. Employment estimator in CES Eery month, CES computes estimates of the relatie change in employment from the preious to current month. The estimation is performe for arious omains efine by intersections of inustry an geography. The estimator of the employment leel YT, in omain at month T has the following form: Yˆ Y Rˆ, (1) T,,0 0, T where Y,0 is a known true employment leel at month 0 (also referre to as the benchmark leel) an R ˆ 0, T is an estimate of the relatie employment change from the base perio 0 to T, the latter being the prouct of estimates of monthly trens Rˆ 1,, t t t 1,..., T, Rˆ Rˆ Rˆ... Rˆ 0, 0,1 1, 1,. () T T T (To aoi hinering the narratie with unnecessary etails, (1) an () present a slightly simplifie ersion of the estimator compare to what actually is use in prouction.) We note that the finite population parameters of interest in omain are both the employment leels Y t, at months t 1,..., T (it also can be iewe as the cumulatie change from the base perio to month t ) an employment changes oer m months, Ytm, t Yt, Yt, m. Specifically, at a gien month t, the target finite population quantity of interest is the one-month relatie change R t1,t jp jp y y jt jt, 1, (3) 1086

3 JSM015 - Surey Research Methos Section where y jt is the employment of business j at time t ; in the omain. The sample base estimator of R t 1, t is ( P ) is the set of population units t1, t js t Rˆ, jst wy j j wy jt j, t1 (4) where w j is the sampling weight of unit j an s t is a set of units sample in the omain an use in the estimation at month t (generally, the sets of responing units use in the monthly estimation iffer from month to month.) The estimator of leels is consiere approximately unbiase: Yˆ Y e, t, t, t, where t, E e. e is the sampling error, uncorrelate across omains, with t, 0 Since the sets of responents s t largely oerlap uring the estimation perio, sampling errors are correlate oer time. Let us assume the following stationary autoregressie moel for the sampling errors: et, eet, 1 t,, e 1, t 1,..., T (5) where Et, Vart, E t, s, 0; ; 0 for t s. The moel implies that the ariance of Y ˆ, t is Var Yˆ 1 t e t, 1 e an for large t it nears V. 1 e Coariance between the leel estimates at times t m an t is ˆ ˆ m co( Yt,, Yt, m) ev. Preious research shows that correlations between the leel estimates in consecutie months are high, in the icinity of 0.8 to 0.9. For estimates of monthly changes, Yˆ ˆ ˆ t, Yt, Yt, 1 Yt, et,, the ariance is 1087

4 JSM015 - Surey Research Methos Section t, t, t, 1 1 e Var Y Var e e V an the coariance is ˆ,, ˆ, 1,, 1, 1, 1 Co Y Y E e e e e V. t t t t t t e Correlation between changes in the ajacent months is ˆ ˆ 1 t,, t, 1 1 e Corr Y Y. (6) (See empirical results in Scott et al. 01, Scott an Serchko 005.) Barring the noise in the irect estimates of correlations between sampling errors in the estimates of changes, the preious research, generally, supports the conclusion that correlations between the ajacent months are negatie, approximately Due to the noisy estimates, it is een more ifficult to iscern a efinitie pattern in correlations between perios that are more than 1 month apart. For this paper, we assume that moel (5) for the sampling errors hols. 3. The Rao-Yu Moel for the CES Series It is a common assumption that the relatie oer-the-month changes from the same month in preious years sere as goo preictors for the current relatie oer-the-month changes. True alues for historical employment counts are aailable from the Quarterly Census of Employment an Wages (QCEW), another BLS program. Auxiliary ariable X t, is the relatie oer-the-month change in employment at month t in cell as forecaste from the historical QCEW ata. The moels below are formulate for relatie monthly changes. Note that Rt 1, t is usually close to 1. Thus, we hae the following approximate formulas. 1 Variance: Var Rˆ 1, ˆ t t Var Y, t. Y t, 1 Coariance: Co R ˆ ˆ 1 1,, 1 ˆ ˆ t t, tt t,, t, 1 R Co Y Y. Y Y t, 1 t, 1 Correlation: Corr Rˆ ˆ 1,, 1 ˆ ˆ t t, R tt Corr Yt,, Yt, 1 1 e. To simplify notation in the moels formulation, we enote: y Rˆ. t, t 1, t 1088

5 JSM015 - Surey Research Methos Section The Fay-Herriot (FH) moel that is currently use for select CES series at the statewie inustrial supersector leel is formulate inepenently for each month. At month t, for omains 1,..., M, y X u e (7) t, t t, t, t,, where the ranom terms u t, an e t, are mutually inepenent an ii u ~ N 0, t, ut, in et N an, ~ 0,, with ariances of the sampling errors consiere known. The Rao-Yu (RY) moel for the CES case is formulate for omains 1,..., M as y X u e u t, t t, t, t, u, 1. t, t, 1 t,, (8) where ranom terms, e, t,, t are mutually inepenent; ii ~ N 0, are ranom effects representing ariation between areas; ii u t, ~ N 0, ; is the correlation between ranom effects ut, 1 an u t, at two consecutie time points. The coariance matrix for the sampling errors is assume known. It has the block-iagonal structure. The block corresponing to omain has the following structure: Co e B, where e e,1, t T,..., e, is the ariance for e t,, 1089

6 JSM015 - Surey Research Methos Section ij 1 B is a T T symmetric matrix haing 1 on the iagonal an 0.5e 1 e at the off-iagonal position j, i j. Parameter t reflects ifferences between the history-base moements X t, an the current tenency. Besies sering as ajustment to historical moements base on the most current CES ata, t also acts as the correction factor for the ifferences in seasonality between the CES an QCEW series. This is the main reason for haing the month specific coefficient, as inicate by subscript t. Coariance matrices for the time an area ranom effects u t, an epen on unknown, u parameters, an. As note aboe, the coariance matrix of sampling errors is consiere known. This is require for moel to be ientifiable. In practice, it is populate by ariances an coariances obtaine base on preious research (an approach often inoles fitting a generalize ariance function.) For sureys where the same sample or a portion of the sample is use repeately uring the estimation perio, as in CES, the sample base estimates in a gien area are correlate oer time. Ability to account for the correlate sampling errors is one point supporting the use of the Rao-Yu moel instea of the cross-sectional moel Fay-Herriot. It is note, base on the results of Rao an Yu (1994) simulation stuy, that the smaller the ariance associate with the time ranom effect u an the larger the ariance associate with the area ranom effect, the stronger the gains from using the Rao-Yu moel oer the cross-sectional Fay-Herriot moel. Gien the structure of the CES ata, the use of information both across time an omains looks appealing. On the other han, the Rao-Yu moel is more complicate: it contains more parameters that nee to be estimate from the ata; in aition, it has parameters that nee to be use as known in practice, this requires further assumptions. Motiate by results from the CES real ata example, we are trying to explore some of the conitions justifying the use of the Rao-Yu moel oer a simpler, Fay-Herriot, moel. 4. Results for the CES Series States within ifferent inustries efine the sets of omains to which we fit our moels. The estimation is performe for each of the 1 months of the estimation perio. For example, at month 5 after the starting point, we fit the Rao-Yu moel to estimate relatie change at month 5 base on the information aailable from all preceing months, 1 through 5; at month 1 after the starting point, we can use information aailable from months 1 through 1. Estimates for the first two months are obtaine using only the Fay-Herriot moel. We use Small Area Estimation: Time-series Moels sae R package ( to fit the Rao-Yu moel. The true population alues are aailable from QCEW program seeral months after the actual estimation. This enables us to compare results of estimation with the population target. Due to ifferences in seasonality between the CES series an the QCEW aministratie ata source, the most meaningful sets of comparison is after 1 months of estimation. Results from 4 years of estimation are presente in Tables

7 JSM015 - Surey Research Methos Section Table 1: October September 011 estimation perio Inustry M Mean Absolute NAICS Reision RY Parameter Estimates an stanar errors coe FH RY rho sig_u sig_ ,019 1, (0.13) 0.65 (0.11) 0.00 (0.0) ,894 3, (0.09) 1.41 (0.16) 0.16 (0.09) ,550 3, (0.08) 0.44 (0.09) 0.00 (1.03) ,51 1, (0.16) 0.46 (0.10) 0.08 (0.04) ,361 1, (0.3) 0.17 (0.07) 0.10 (0.03) ,614, (0.1) 0.8 (0.08) 0.00 (0.0) ,751 1, (0.74) 0.07 (0.07) 0.03 (0.0) ,83 1, (0.11) 0.74 (0.11) 0.0 (0.03) ,807, (0.19) 0.1 (0.07) 0.04 (0.1) ,174 3, (0.14) 0.69 (0.14) 0.0 (0.05) ,018 3, (0.7) 0.05 (0.11) 0.00 (0.09) ,994 1, (0.13) 1.38 (0.4) 0.07 (0.08) ,99 5, (0.40) 0.09 (0.10) 0.00 (0.17) ,717, (0.13) 1.1 (0.0) 0.00 (0.05) ,18 3, (133.1) 0.00 (0.05) 0.01 (.15) ,14, (0.36) 0.19 (0.1) 0.00 (0.04) ,686 3, (0.66) 0.1 (0.10) 0.01 (0.0) ,391, (0.5) 0.3 (0.08) 0.04 (0.0) Table : October September 01 estimation perio Inustry M Mean Absolute NAICS Reision RY Parameter Estimates an stanar errors coe FH RY rho sig_u sig_ (0.7) 0.09 (0.07) 0.00 (0.09) ,386 3, (0.3) 0.8 (0.09) 0.03 (0.0) ,819 1, (0.46) 0.03 (0.05) 0.00 (0.3) ,96 1, (1.05) 0.01 (0.05) 0.00 (0.3) ,44 1, (1.41) 0.0 (0.06) 0.05 (0.05) ,976, (57.87) 0.00 (0.06) 0.0 (0.01) ,808 1, (.97) 0.00 (0.04) 0.00 (0.97) ,007 1, (0.07) 1.68 (0.17) 0.00 (0.04) ,773 1, (0.81) 0.06 (0.07) 0.0 (0.01) ,515, (.07) 0.01 (0.05) 0.00 (0.75) ,953 4, (1.4) 0.0 (0.09) 0.00 (0.11) ,54, (5.58) 0.00 (0.05) 0.00 (16.55) ,556 8, (0.30) 0.35 (0.14) 0.00 (0.03) ,933, (0.16) 0.75 (0.16) 0.00 (0.04) ,65 4, (9.37) 0.01 (0.09) 0.08 (0.03) ,700, (0.7) 0.10 (0.10) 0.00 (0.0) ,56, (3.5) 0.0 (0.09) 0.00 (0.0) ,409 1, (35.19) 0.00 (0.06) 0.01 (0.01) 1091

8 JSM015 - Surey Research Methos Section Table 3: October 01 - September 013 estimation perio Inustry NAICS M Mean Absolute Reision RY Parameter Estimates an stanar errors coe FH RY rho sig_u sig_ (507.46) 0.00 (0.07) 0.08 (0.03) ,40, (0.17) 0.41 (0.10) 0.00 (0.0) ,783, (0.55) 0.0 (0.04) 0.00 (0.61) ,303 1, (1.7) 0.01 (0.04) 0.00 (0.80) ,48 1, (133.09) 0.00 (0.05) 0.03 (0.15) ,409, (4.77) 0.00 (0.06) 0.0 (0.01) ,871 1, (3.39) 0.01 (0.06) 0.03 (0.0) ,11 1, (0.08) 1.5 (0.16) 0.00 (0.04) ,436 1, (0.45) 0.1 (0.07) 0.0 (0.0) ,139, (36.83) 0.00 (0.04) 0.01 (1.99) ,37 3, (7.41) 0.00 (0.07) 0.10 (0.19) ,675 1, (1.35) 0.06 (0.10) 0.05 (0.03) ,747 3, (0.61) 0.08 (0.11) 0.00 (0.05) ,367 1, (0.14) 0.50 (0.10) 0.00 (0.0) ,989 5, (0.33) 0.17 (0.08) 0.01 (0.0) , (51.93) 0.00 (0.08) 0.00 (0.0) ,0, (0.4) 0.30 (0.10) 0.00 (0.0) ,99 1, (46.81) 0.00 (0.06) 0.00 (0.01) Table 4: October September 014 estimation perio Inustry NAICS M Mean Absolute Reision RY Parameter Estimates an stanar errors coe FH RY rho sig_u sig_ (489.45) 0.00 (0.07) 0.01 (0.01) ,045 3, (0.14) 0.60 (0.11) 0.03 (0.03) ,699 1, (85.08) 0.00 (0.03) 0.01 (9.86) (31.77) 0.00 (0.03) 0.0 (10.01) , (0.43) 0.1 (0.07) 0.03 (0.0) ,671 3, (0.14) 0.48 (0.09) 0.00 (0.0) ,34 1, (0.5) 0.3 (0.08) 0.0 (0.0) (0.3) 0.5 (0.08) 0.01 (0.0) ,311 1, (315.16) 0.00 (0.06) 0.04 (0.0) ,131, (0.34) 0.30 (0.13) 0.00 (0.03) ,91, (0.55) 0.1 (0.09) 0.0 (0.0) ,97 1, (0.) 0.35 (0.11) 0.00 (0.03) ,036 4, (3.90) 0.0 (0.08) 0.01 (0.01) ,875 1, (0.14) 0.50 (0.10) 0.00 (0.0) ,90 3, (0.55) 0.10 (0.07) 0.0 (0.0) ,544 1, (0.78) 0.07 (0.08) 0.00 (0.01) ,894, (0.16) 0.50 (0.11) 0.00 (0.03) ,777 1, (0.18) 0.36 (0.08) 0.00 (0.0) 109

9 JSM015 - Surey Research Methos Section The results show no clear aantage of using the Rao-Yu moel oer the Fay-Herriot moel: mean absolute reisions after 1 months of estimation are generally close. There are inustries where the Rao-Yu moel results are somewhat better in all 4 years (e.g., Transportation, Eucation, Accommoation an Foo Serices, Other Serices), in other inustries, one moel is better than the other in one year while the opposite is true in another year; in inustry (Arts, Entertainment, an Recreation), the Fay-Herriot moel worke better in all 4 years. One reason why there was no clear benefit from using the Rao-Yu moel is that the ariance of the area ranom effects was small relatie to the sampling error or to the ariance of the time effect. Possible misspecification of the sampling error matrix may also contribute to the result. Inee, by the efine setup of the cross-sectional Fay-Herriot moel case, sampling errors o not correlate oer time. Thus the sampling ariance matrix, the known component of the moel, is iagonal, which is simpler than the block-iagonal structure of the known matrix when one ecies to inclue the knowlege of the oertime correlation in the moel. To test the aboe conjectures, we performe simulations (presente in the next section). 5. Inestigation Base on Simulate Data In this section, we use simulate ata to stuy the effect of the moel parameters an errors in the sampling error ariances on the results of the moel fit. As can be seen from the preious section, the ariance of the area ranom effect is close to zero. This is the worst scenario if one counts on taking aantage from using information oer time with the Rao-Yu moel. Still, een in this case, it is possible to benefit from accounting for the sampling error correlation. Our simulations, inee, show that this is the case. Howeer, one must remember that the sampling error coariance structure is known only in theory. In practice, we use some estimate alues an assumptions about the coariance structure as if they were true an known. We generate ata from the following moel: y u e, (9) t, t, t, for 1,...,0 areas an t 1,...,1 time points. Ranom terms, e, t, u, t are generate inepenently: ii u, ~ N 0, with t u u 0.5 ii ~ N 0, with two choices for the alues of a. b

10 JSM015 - Surey Research Methos Section Sampling error structure: E e t, 0 Var et, 1. The employment leel error correlation between ajacent months is assume to be e 0.7. Then employment one-month change error correlation is 0.51 e 0.15 ; the coariance matrix for errors of employment changes is block-iagonal; each block is T T symmetric matrix haing 1 on the iagonal an i j at off-iagonal positions ji, j. e 1 e We consier seeral ersions of the assume error structure as use at the time we fit the moel. First, we may erroneously assume that the sampling errors are inepenent oer time; secon, we may use the true, correct ariance structure, the same as was use to generate the moel. In aition, we consier the situation where the ariances of the sampling errors are estimate with error. To moel this, we assume that the ariance estimates are gamma-istribute Gamma k, with shape k 13 an scale 3. Thus, this correspons to the unbiase ariance estimates (the expectation is 1) with the ariance of the ariance estimates equal 3. The situation where ariances are estimate with sizable errors is plausible with the employment ata. The employment numbers hae a highly skewe istribution; the employment changes are concentrate aroun zero with smaller proportion of businesses haing significant changes in employment while yet smaller proportion haing extreme large positie or negatie changes. The simulation stuy is base on 500 simulation runs for t 1,..., T, where T 3,...,1. We present results for moels using T 6 an T 1 points of history. To fit the Rao-Yu moel, we use the metho of moments as gien in Rao an Yu (1994). This metho proie approximately the same results as the REML-base sae R package that we use for the real ata. The aantage of using this metho rather than REML was that it works significantly faster. Instea of estimating the moel correlation parameter, we assume it to be 0, i.e., equal to the true moel parameter, which in the case of simulation is known to us. Since all the areas are equally istribute, the empirical mean square error was calculate by both aeraging the errors across areas an simulations. Thus, the simulation error is base on the actual simulation size of = 10,000 trials: s, s, for E = Direct, FH, or RY base estimate. s11 MSE E E The relatie efficiency of RY oer FH was compute as RE MSE RY MSE FH 100% MSE FH. 1094

11 JSM015 - Surey Research Methos Section As can be seen from Table 5, when there is no error in the ariance estimates, the Rao-Yu moel is more efficient than the Fay-Herriot moel. This is true een for the case where the area ranom effect is absent ( 0 ), een for the case where the sampling errors are wrongly assume to be inepenent. With the existing area ranom effect, the efficiency of Rao-Yu oer Fay-Herriot increases to oer 30%. Table 5: Mean square error base on 500 simulation runs for ifferent moel parameters an assumptions on coariance structure of the sampling errors Sampling Error Error in Direct FH RY RE,% Correlation Sampling True Assume Variances T=6 T=1 T=6 T=1 T=6 T=1 T=6 T=1 0.5, 0 u None None Gamma Gamma , 0.5 u None None Gamma Gamma The situation is rastically ifferent when the known sampling error ariances are generate from the Gamma 13,3 istribution. This results in the increase of the mean square error in both Rao-Yu an Fay-Herriot base estimates; yet the MSE of the FHbase estimates is lower than the MSE of the RY-base estimates. It is also interesting to note that the assumption of the iagonal sampling error coariance structure leas to lower MSE in the RY-base results as compare with the results base on the correct assumption that the matrix is block-iagonal. 6. Summary We explore aantages of using the Rao-Yu moel that utilizes information from time as well as cross-sectionally, as compare to the cross-sectional-only Fay-Herriot moel. The empirical results showe that, in the case of the CES ata, there is no clear aantage from applying the Rao-Yu moel. In the attempt to unerstan the nature of these mixe results, we performe the simulation stuy. We showe that misspecification in the estimate sampling ariances, orinarily consiere fixe an known in both moels, affects the results in such a way that the Fay-Herriot-base moel may become more efficient compare to the Rao-Yu moel. 1095

12 JSM015 - Surey Research Methos Section References Fay, R.E., an Herriot, R.A. (1979), Estimates of Income for Small Places: An Application of James-Stein Proceures to Census Data. Journal of the American Statistical Association, 74, Pfeffermann, D. (00). Small area estimation - new eelopments an irections. Int. Statist. Re Pfeffermann, D. (013). New Important Deelopments in Small Area Estimation. Statistical Science Rao, J.N.K. (003), Small Area Estimation, John Wiley & Sons, Hoboken, NJ. Rao, J.N.K. an Yu, M. (1994), Small Area Estimation by Combining Time Series an Cross-Sectional Data. Canaian Journal of Statistics,, Scott, S. an Serchko, M. (005), Variance Measures for X-11 Seasonal Ajustment: A Summing Up of Empirical Work. ASA Proceeings of the Joint Statistical Meetings. Scott, S., Pfeffermann, D., an Serchko, M. (01). Estimating Variance in X-11 Seasonal Ajustment. In Economic Time Series: Moeling an Seasonality, eite by William R. Bell, Scott H. Holan, an Tucker S. McElroy, Lonon: Chapman an Hall. 1096