A comparison of small area estimators of counts aligne with irect higher level estimates Giorgio E. Montanari, M. Giovanna Ranalli an Cecilia Vicarelli Abstract Inirect estimators for small areas use auxiliary variables to borrow strength from relate areas through a linking moel. Precision of inirect estimators epens on the valiity of such a moel. To protect against possible moel failures, benchmarking proceures make the total of small area estimates match a esign consistent estimate for a larger area. This is also particularly important for National Institutes of Statistics to ensure coherence between small area estimates an irect estimates prouce at higher level planne omains. We investigate a self-benchmarke estimator in the case of a unit level logistic mixe moel for a binary response, propose an estimator for its mean square error an compare its performance with competing estimators through a simulation stuy. Key wors: Small area estimation, Logistic mixe moel, Penalize Quasi Likelihoo, Unit level moel, MSE estimation. 1 Introuction Sample surveys are esigne to provie reliable estimates of finite population parameters for large omains. In fact, esign-unbiase or approximately esignunbiase irect estimators are use for those omains whose sample size is sufficiently large. However, local an central governments increase their eman of etaile statistical information on small omains, either geographical areas that are smaller than those planne, or specific subpopulations, as those given by a fine classification on socio-emografic variables. Since such omains are not planne at the esign stage, their sample size coul be very small an even zero. Consequently, irect estimators of small area parameters may have very large variance an some- Giorgio E. Montanari, M. Giovanna Ranalli an Cecilia Vicarelli Dipartimento i Economia, Finanza e Statistica, Università egli Stui i Perugia, e-mail: giovanna.ranalli@stat.unipg.it 1
2 Giorgio E. Montanari, M. Giovanna Ranalli an Cecilia Vicarelli times cannot be calculate ue to lack of observations. This makes it necessary to fin inirect estimators that connect relate small areas through a linking moel that is base on auxiliary information to borrow strength an increase the effective sample size. Precision of inirect estimators epens on the valiity of such a moel. To protect against possible moel failures, benchmarking proceures make the total of small area estimates match a esign consistent estimate for a larger area. This is also particularly important for National Statistical Institutes to ensure coherence between small area estimates an irect estimates prouce at higher level planne omains. There are two kins of benchmarke estimators: estimators that are internally benchmarke (or self-benchmarke) an those that are externally benchmarke. For a recent review see Wang et al. (2008). Self benchmarke preictors are, for example, the pseuo-eblup introuce by You an Rao (2002) an the augmente estimator propose by Wang et al. (2008). A rawback of this type of self benchmarke estimators is that they force the use of the same auxiliary information use for the irect usually GREG-type estimator also for the moel-base small area preictors, whereas it coul be very profitable to allow for ifferent auxiliary variables at the small area level. Externally benchmarke preictors are obtaine through an a-posteriori ajustment of moel-base preictors. Among the others, Pfeffermann an Barnar (1991) propose an externally restricte benchmarke estimator of small area means. This is constructe uner an area linear mixe moel for a continuous response variable. In this work, we investigate an extension of this approach to the case of a unit level logistic mixe moel for a binary response. This has particular relevance any time the small area estimate takes the form of a count. You et al. (2004) aress this issue by using a ratio type ajustment of Hierarchal Bayes estimators. The paper is organize as follows. Section 2 escribes the small area estimator propose by Pfeffermann an Barnar (1991) an then Section 3 illustrates the propose methoology to tackle situations in which the variable of interest is binary. An algorithm to actually compute parameter estimates an ranom variables preictions is propose together with an analytic estimator for its mean square error. The results of a limite simulation stuy that compares the performance of ifferent estimators are then presente in Section 4. 2 Some small area estimators with the benchmarking property Consier a finite population U = {1,2,..,N} an a partition of U in omains (small areas) U i mae of N i units, with i = 1,2,..,, such that U i = U an N i = N. We are intereste in estimating small area totals Y i = j Ui y i j of a binary y variable. To this purpose, we assume that a (K + 1)-imensional rowvector x i j = [1,x i j1,..,x i jk ] of an auxiliary variable x is known for each element j U i, for i = 1,2,..,. Let X N = {x i j } j Ui ;,2,.., be the matrix of population values x i j. Suppose we also know the -imensional row-vector z i j that takes value 1 in its i-th position an 0 otherwise, for all units j U i, for i = 1,2,..,. Let
Benchmarke small area estimators of counts 3 Z N = {z i j } j Ui ;,2,.., be the matrix of population values z i j. To estimate the population an omains total, a sample s of size n is rawn from the finite population U using a probabilistic sampling esign p(s). Let s i = s U i enote the sample with n i elements realize in the i-th omain, for i = 1,2,..,. Estimators Ŷi bench have the benchmarking property if Ŷ i bench = Ŷ, where Ŷ is a consistent estimator, usually a GREG or Calibration type estimator of the population total Y = j Ui y i j. Moel base estimators the most wiely use for small area parameters usually o not have this property when p(s) is a complex sampling esign. Given a small area estimator Ŷ i, that oesn t show the benchmarking property, a first simple way of achieving benchmarking is by a ratio type ajustment, i.e. Ŷi bench Ŷ = Ŷ i. (1) Ŷi Pfeffermann an Barnar (1991) consier an area level moel for farmlan values an moify the optimal preictor uner a linear mixe moel to have the benchmarking property on the sample mean. To this en, they compute the moel parameters an ranom effects as the penalize generalize least squares solution uner the benchmark constraint. This proceure gives ajuste estimates that are equivalent to the (un-benchmarke) small area estimates uner the linear mixe moel plus a correction term. This correction term is given for each small omain estimate by a proportion, a i, of the ifference between the esign consistent estimate an the sum of the moel-base small area estimates. In particular, Ŷ bench i = Ŷ i + a i (Ŷ Ŷ i ), (2) with a i = 1. These proportions take the form a i = cov(ŷ i, Ŷi)/var( Ŷi). In this work we consier a similar approach to eal with variables of interest that are binary an small area totals that are, therefore, counts. This is illustrate in the next section. 3 Benchmarke small area estimates of counts We assume that the y i j s are inipenent bernoulli ranom variables with P(y i j = 1 x i j ) = ( p i j ) pi j logit(p i j ) = log = η i j = x i j b + z i j u, (3) 1 p i j u N(0,σu 2 I) for j U i, i = 1,2,...,. We want to erive a benchmarke estimator Ŷi bench of Y i for i = 1,2,.., uner moel (3) that takes the following form
4 Giorgio E. Montanari, M. Giovanna Ranalli an Cecilia Vicarelli Ŷ bench i = y i j + ˆp i j = y i j + j s i j U i \s i j s i j U i \s i exp(x i j ˆb + z i jû) 1 + exp(x i j ˆb + z i j û) (4) (see e.g. Saei an Chambers, 2003, or Rao, 2003), but for which the benchmarking property hols, i.e. such that Ŷ bench i = Ŷ. (5) To this en, we consier the Penalize Quasi log-likelihoo (PQL) of moel (3) uner the benchmark constraint in equation (5). The PQL approach consiers a loglikelihoo function of the sample values y i j for i s i an i = 1,..., conitional on u with the aition of a penalty function ue to the presence of the ranom effect, given by the logarithm of the probability ensity function of u (for a more etaile escrition of PQL methos see e.g. McCulloch an Searle, 2001). We moify such PQL metho by aing the benchmark constraint through the metho of Lagrange multipliers. As a result, the loglikelihoo is mae of three parts. In particular, let l 1 be the loglikelihoo function of the sample values y i j conitional on u, let l 2 be the penalization coming from the marginal istribution of the ranom effects u, an let l 3 be the penalization ue to the benchmarking constraint. Therefore, the benchmarke PQL is given by l = l 1 + l 2 + l 3, with l 1 = y t sη s 1 t n log[1 + exp(η s )], l 2 = 1 2 [σ 2 l 3 = λ (Ŷ u u t u + log(σu 2 )], ) ( Ŷi bench = λ Ŷ 1 t ny s 1 t exp(η r ) N n 1 + exp(η r ) where y s enotes the vector of observe values y i j for j s i an i = 1,2,..,; η s is the linear preictor vector for j s i, i = 1,2,..,, an η r is the linear preictor vector for j U i \s i, i = 1,2,..,. To obtain maximum benchmarke PQL estimates of b, u, λ an the ML estimate of the variance component σ 2 u we use an algorithm base on Fisher Scoring. Saei an Chambers (2003) escribe the proceure to obtain small area estimates from a logistic mixe moel. Such proceure is aapte to accommoate l 3 an escribe below. 1. Assign initial values to b,u, λ, an σ 2 u. 2. Upate b, u an λ using Fisher scoring, i.e. by b (r+1) b (r) ), u (r+1) = u (r) + I(b (r),u (r),λ (r) ) 1 S(b (r),u (r),λ (r) ) λ (r+1) λ (r) where I( ) an S( ) are the Information matrix an the score of l, respectively. 3. Calculate η s an η N. 4. Calculate B s = ( 2 l 1 / η s η t s) an C r = ( 2 l 3 / η r η t r).
Benchmarke small area estimators of counts 5 5. Calculate T = (σu 2(r) I + Z t sb s Z s + Z t rc r Z r ) 1, where Z s = {z i j } j si ;,2,.., an Z r is its analogous on non-sample units. 6. Upate σu 2(r+1) = 1 (trace(t) + u t(r+1) u (r+1) ). 7. Return to step 2 an repeat the proceure until convergence of the loglikelihoo. It is recommene that ifferent starting values are consiere to check whether the proceure got trappe in some local maximum. Goo starting values for b an u are those coming from a fixe logistic moel, for σu 2 is the variance of these u estimates. The choice for λ, on the other sie, seems less critical; in the simulation stuies a value of 1 has been use. Using the results of the algorithm we obtain estimates of parameters to be use in (4). To calculate the mean square error (MSE) of the propose estimator we use an approximation that is an application of Prasa an Rao (1990) analytic form of MSE after a Taylor linearization of the moel. In particular, let ŷ bench = (Ŷ1 bench,...,ŷ bench ) t, MSE(ŷ bench ) = G 1 (σ 2 u ) + G 2 (σ 2 u ) + G 3 (σ 2 u ) + G 4 (σ 2 u ) (6) where, if matrices X r an B r are the analogous of X s an B s for nonsample units, an A r is a (N n) matrix of ones, G 1 (σ 2 u ) = A t rh r Z r TZ t rh t ra r, G 2 (σ 2 u ) = A t rh r (X r Z r TZ t sb s X s )(X t sv 1 s X s ) 1 (X t r X t sb s Z s TZ t r)h t ra r, G 3 (σ 2 u ) = var( ˆσ 2 u )[h t k Zt sb s V s B s Z s h l ] k,l=1,...,, an G 4 (σ 2 u ) = A t rb r A r, where H r = iag[ ˆp i j (1 ˆp i j )] j Ui \s i for i = 1,..,, V s = σu 2 Z s Z t s + B 1 s, h t k = (h k1,..,h k ) = a t rk Z rt, a t rk enotes the k-th row of At rh r, an h t k = ( h k1/ σu 2,..., h k / σu 2 ). Observe that the approximation in (6) epens on how well the linear mixe moel in the linearize response variable approximates the original generalize linear mixe moel, an how goo is the first-orer Taylor approximation. An estimator for MSE(ŷ bench ) can then be obtaine by mse(ŷ bench ) = Ĝ 1 ( ˆσ u 2 ) + Ĝ 2 ( ˆσ u 2 )+2Ĝ 3 ( ˆσ u 2 )+Ĝ 4 ( ˆσ u 2 ), where hats enote substitution with estimate values from the output of the algorithm. For more etails in the un-benchmarke case see Saei an Chambers (2003) an González-Manteiga et al. (2007). 4 Simulation stuy an some concluing remarks We conuct a simulation stuy (i) to investigate the effect of benchmarking on the performance of the estimators, (ii) to compare the performance of estimators base on linear mixe moels an that of those base on logistic mixe moels to estimate small area counts an (iii) to compare the performance of estimators base on alternative algorithms that estimate logistic mixe moels. To these ens we start from real ata coming from the Italian Multipurpose househol survey. The sample
6 Giorgio E. Montanari, M. Giovanna Ranalli an Cecilia Vicarelli from the Region Umbria has been use as our finite population mae of N = 4,879 units ivie into = 51 small area given by municipalities. To simulate a complex sampling esign, units are ranomly assigne to four strata of imension 500, 800, 1,500 an 2,079, respectively. The response variable is simulate an constructe using age, sex an stratum membership as covariates in the following linear preictor η i j = 0.4str 1i j + 0.1str 2i j 1.7str 3i j 2str 4i j 0.16age i j + 0.9sex i j + u i, with = 51 ranom effects u i rawn from a zero mean normal variable with variance 3. This moel allows to buil an informative stratification. A bernoulli experiment is then rawn once with probabilities p i j = exp(η i j )/(1 exp(η i j )) to obtain binary response variable y i j. An ajustment is then mae to have at least 3 values equal to 1 for each small area. This allows to use overall measures of performance base on relative bias (see later in this section) to ease the comparison of the ifferent estimators. The overall mean of the response variable is 7.8%. Five hunres stratifie ranom samples of imension n = 500 have been selecte using a isproportionate allocation. In particular, we select 112 elements from the first stratum, 120 from the secon, 112 from the thir an 156 from the fourth. This gets sampling rates within strata of 22.4%, 15.0%, 7.5% an 7.5%, respectively. The benchmark is the Horvitz-Thompson estimator of the total. For each sample several small area estimators are compute. All use moels base on the same two covariates age an sex an a normal ranom area effect. Firstly, a set of three estimators base on a unit level linear mixe moel are consiere: the classical Battese et al. (1988) estimator BHF that is not benchmarke, then its ratio ajustment as in equation (1) BHFr an, finally, the Pfeffermann an Barnar (1991) estimator PB as in equation (2). Seconly, a set of seven estimators base on a logistic mixe moel is consiere. It is well known that, ifferently from the normal case, maximum likelihoo estimates of the parameters an of the variance components in a logistic-normal mixe moel is hinere by the presence of a imensional integral, so that irect calculation is intractable an well known computational issues arise. Therefore, more attention shoul be pai when computing these estimators by, possibly, comparing results coming from ifferent softwares that use ifferent algorithms. Consequently, we consier the estimator RPQL obtaine using the function glmmpql of R, an its ratio ajustment RPQLr, the estimator RML obtaine using the function glmmml of R, an its ratio ajustment RMLr. Finally, three estimators are consiere that are obtaine using the Saei an Chambers (2003) algorithm: the first one is the un-benchmarke one SC, then its ratio ajustment SCr an, finally, the propose self-benchmarke estimator SCsb. Let Ŷi r be the value of a small area estimator of the total at replicate r, for r = 1,...,500; then the following evaluation criteria have been compute an reporte in Table 1: % Relative Bias for small area i: RB i = 1 R [ R r=1 ] Ŷi r Y i 100; Y i
Benchmarke small area estimators of counts 7 Average Absolute RB: AARB = 1 RB i ; Maximum Absolute RB: MARB = max RB i ; i % Relative Root MSE for small area i: RRMSE i = 1 R Average RRMSE: ARRMSE = 1 RRMSE i ; Maximum RRMSE: MRRMSE = maxrrmse i. i [ R r=1 (Ŷ ) r 2 ] i Y i 100; Y i Table 1 Simulation results: Average an Maximum Absolute % Relative Bias, Average an Maximum % Relative Root Mean Square Error for all estimators. BHF BHFr PB RPQL RPQLr RML RMLr SC SCr SCsb AARB 47.8 40.8 39.3 56.3 48.4 51.4 39.0 49.5 42.5 39.5 MARB 221.5 185.8 178.3 291.3 246.1 232.8 173.9 230.9 192.9 168.3 ARRMSE 71.5 63.3 65.7 80.9 70.6 80.8 64.2 69.9 61.7 63.1 MRRMSE 243.3 207.9 205.4 327.4 281.4 266.1 208.3 255.8 218.3 199.6 First we can note that estimators with the benchmarking property all have a smaller bias an MSE than their corresponing un-benchmarke ones. This can be explaine by the fact that benchmarking moel base estimates to the unbiase Horvitz-Thompson estimator helps in ecreasing the bias of the estimates. Seconly, we compare the performance of estimators base on linear mixe moels an that of those base on a logistic mixe moel. In fact, if in classical statistics using normal moels for a binary variable instea of the appropriate logistic ones is usually eprecate, for small area statistics there is not a clear-cut evience of the superiority in terms of performance of the latter over the former. In fact, in this case the performance is similar, both in terms of bias an error. However, note that normal base moels prouce negative estimates for the areas with a low population count. This has happene 60 times for BHF an BHFr an 268 times for PB in the 500 51 estimates. Thirly, when comparing estimators base on a logistic mixe moel compute using ifferent algorithms, we can note that the Saei an Chambers (2003) algorithm has a goo performance with respect to the two others, both in terms of bias an error. Finally, it can be note that the self-benchmarke propose estimator shows a very goo performance in terms of overall bias, an it also ecreases significantly the bias for the most biase area it has the smallest MARB value. This is true also for the MSE. Decrease of bias shown by the benchmarke estimators can be likely explaine by the fact that the sampling esign is, in this case, informative. It will be interesting, therefore, to evaluate the effect of weighting the likelihoo equations provie in Section 3 to account for the sampling esign. In aition, we envision the stuy of
8 Giorgio E. Montanari, M. Giovanna Ranalli an Cecilia Vicarelli the effect of benchmarking to GREG-type estimators base on auxiliary information that may or may not coincie with that use in the small area moels. Also, it will be of interest to compare the performance of the ifferent estimators for ifferent values of the population proportion of the attribute an of the variance of the area effects. Finally, evaluation of the analytic MSE estimator is also on the agena. Acknowlegements Work supporte by the project PRIN 2007 Efficient use of auxiliary information at the esign an at the estimation stage of complex surveys: methoological aspects an applications for proucing official statistics. References 1. Battese, G.E., Harter, R.M., Fuller, W.A., An error-components moel for preiction of county crop areas using survey an satellite ata. J Am Stat Ass. 83, 28 36 (1988). 2. McCulloch, C.E., Searle, S.R., Generalize, Linear, an Mixe Moel, Wiley, New York (2001) 3. González-Manteiga, W., Lombaría, M.J., Molina, I., Morales, D., Santamaría, L., Estimation of the mean square error of preictors of small area linear parameters uner a logistic mixe moel, Comput Stat Data An. 51, 2720 2733 (2007) 4. Pfeffermann, D., Barnar, C.H., Some New Estimators for Small-Area Means With Application to the Assessment of Farmlan Values, J Bus Econ Stat. 90, 73 84 (1991) 5. Prasa, N.G.N., Rao, J.N.K., The estimation of the mean square errror of small area estimators. J Am Stat Ass. 85, 163 171 (1990) 6. Rao, J.N.K., Small area estimation. Wiley, New Jersey (2003) 7. Saei, A., Chambers, R., Small Area Estimation Uner Linear an Generalize Linear Moel With Time an Area Effects. Working Paper M03/15, Southampton Statistical Sciences Research Institute, University of Southampton (2003). 8. Wang, J., Fuller, W.A., Qu, Y., Small area estimation uner a restriction. Surv Methool. 34, 29 36 (2008) 9. You, Y., Rao, J.N.K., A pseuo-empirical Best Linear Unbiase Preiction Approach to Small Area Estimation Using Survey Weights. Can J Stat. 30, 431 439 (2002). 10. You, Y., Rao, J.N.K., Dick, P., Benchmarking hierarchical bayes small area estimators in the Canaian census unercoverage estimation. Statistics in Transition, 6, 631 640 (2004)