Small Area Estimation: A Review of Methods Based on the Application of Mixed Models. Ayoub Saei, Ray Chambers. Abstract

Transcription

1 Small Area Estimation: A Review of Methos Base on the Application of Mixe Moels Ayoub Saei, Ray Chambers Abstract This is the review component of the report on small area estimation theory that was prepare as part of Southampton s involvement in the EURAREA Enhancing small area estimation techniques to meet European Nees project IST in the European Union s Fifth Research An Technological Development Framework Programme. S 3 RI Methoology Working Paper M03/16

2 Small Area Estimation: A Review of Methos Base on the Application of Mixe Moels Ayoub Saei an Ray Chambers Southampton Statistical Sciences Research Institute University of Southampton Highfiel, Southampton SO17 1BJ Unite Kingom September 2003 Abstract This is the review component of the report on small area estimation theory that was prepare as part of Southampton s involvement in the EURAREA Enhancing small area estimation techniques to meet European Nees project IST in the European Union s Fifth Research An Technological Development Framework Programme

3 1 Introuction Efficient estimation of population characteristics for subnational omains is an important aim for many statistical agencies. Such omains are efine as any subivision of the population for the variable of the interest. However, geographically-base omains, like regions, states, counties, wars an metropolitan areas are typically of most interest. A traitional approach to estimation for such omains is base on application of classical esign-base survey sampling methos. Estimates base on this approach are often calle irect estimates in the literature. However, sample sizes are typically small or even zero within the omains/areas of interest. This results in the irect estimators having large variances. When there are no sample observations in some of the small areas of interest, irect estimators cannot even be calculate. Small area estimation theory is concerne with resolving these problems. Often there is auxiliary information that can be use to efine estimators for small areas. In some cases these are values of the variable of interest in other, similar, areas, or past values of this variable in the same area or values of other variables that are relate to the variable of interest. Estimation an inference approaches base on using this auxiliary information are calle inirect or moel-base. Methos base on the use of auxiliary information have been characterize in the statistical literature as borrowing strength from the relationship between the values of the response variables an the auxiliary information. Moel-base methos have long history, but have only receive attention in the last few ecaes as efining an approach to estimating small area characteristics. In this context, two main ieas have been use in eveloping moels for use in small area estimation. These either assume that the interomain variability in the response variable can be explaine entirely in terms of corresponing variability in the auxiliary information, leaing to so-calle fixe effect moels, or require the assumption that "unexplaine" omain specific variability remains even after accounting for the auxiliary information, leaing to mixe moels incorporating omain specific ranom effects. Fixe effect moels explain inter-omain variation in the response variable of interest entirely in terms of variation in known factors. Such moels have been the mainstay of statistical analysis since the pioneering work of Legenre (1806). Estimates of small area characterics base on fixe effect moels are referre to as synthetic estimators (Levy an 2

4 French, 1977), composite estimators (Schaible et al, 1977), an preiction estimators (Holt et al, 1979; Sarnal, 1984; an Marker, 1999). Mixe moels also have a long history, but have receive special interest only in the last few ecaes. This is partly ue to the heavy computational buren of estimation methos use with such moels. Recent evelopments in computing harware, software an estimation methos have however le to increase attention being pai to the use of mixe moels for ata analysis. Linear mixe moels have a wie range of applications. In particular, the ability to preict linear combination of fixe an ranom effects is one the more attractive properties of such moels. In a series of papers, Henerson (1948, 1949, 1959, 1963, 1973, 1975) evelope the best linear unbiase preiction (BLUP) metho for mixe moels. In this case "best" stans for minimum mean square error among all linear unbiase preictors, "linear" means that the preictor is a linear combination of the response variable values an "unbiase" means that expecte value of the preiction error (preicte value of variable - actual value of variable) is zero. The BLUP metho has become a powerful an wiely use proceure for fitting moels for genetic trens in animal populations base on traits measure on both the continuous an the categorical scale. However, the BLUP methos escribe in Henerson ( ) assume that the variances associate with ranom effects in the mixe moel (the variance components) are known. In practice of course such variance components are unknown an have to be estimate from the ata. There are several methos for estimating variance components. Harville (1977) reviews these methos, incluing maximum likelihoo an resiual maximum likelihoo an three other methos suggeste by Henerson. The preictor obtaine from the BLUP when unknown variance components are replace by associate estimators is calle the empirical best linear unbiase preictor (EBLUP) an is escribe in Harville (1990), Robinson (1990) an Harville (1991). Over the last ecae several important approaches have been evelope for estimating/preicting the value of a linear combination of fixe an ranom effects in iscrete response ata. In virtually all of these ranom effects are assume to be normally istribute. Schall (1991), Breslow an Clayton (1993), McGilchrist an Aisbett (1991) an McGilchrist 3

5 (1994) have extene the empirical best linear preictor (EBLUP) to generalize linear moels. Wolfinger (1993) an Wolfinger an O'Connell (1993) evelope essentially the same computational algorithm but via a ifferent approach. Three likelihoo expansion techniques incluing the Solomon an Cox (1992) approach were compare in Breslow an Lin (1995) an Lin an Breslow (1996). Zeger an Karim (1991) introuce a Gibbs sampling approach for generalize linear mixe moels. Monte Carlo EM (MCEM) an Monte Carlo Newton-Raphson (MCNR) computation approaches are escribe in McCulloch (1994) an McCulloch (1997) respectively. Mixe moels have been use to improve estimation of small area characteristics of small area base on survey sampling or census ata by Fay an Herriot (1979), Ghosh an Rao (1994), Rao (1999) an Pfeffermann (1999). In these applications, the mixe moel erives from the concept that the vector of finite population values is a realisation of a superpopulation. In this context, estimation of a small area mean is equivalent to preiction of the realization of the unobservable ranom area effect in a linear mixe moel for the superpopulation istribution of the variable efining this mean. See Valliant et al (2000) for a iscussion of the istinction between the traitional interpretation of moel-base preiction an its application in survey sampling. In aition to EBLUP, empirical Bayes (EB) an hierarchical Bayes (HB) estimation an inference methos have been also applie to small area estimation. Uner the EB approach, Bayes estimation an inferential approaches are use in which posterior istributions are estimate from ata. Uner the HB approach, unknown moel parameters (incluing variance components) are treate as ranom, with values rawn from specifie prior istributions. Posterior istributions for the small area characteristics of interest are then obtaine by integrating over these priors, with inferences base on these posterior istributions. Ghosh an Rao (1994) review the application of these estimation methos in small area estimation. Maiti (1998) has use non-informative priors for hyperparameters when applying HB methos. You an Rao (2000) have use HB methos to estimate small area means uner ranom effect moels. Many surveys or censuses are repeate over time, an this means that auxiliary information from past values of a variable of interest can be use to improve current estimates 4

6 of this variable. This "borrowing of strength over time" has been use to improve estimators for small areas. Starting with the work of Scott an Smith (1974), Pfeffermann an Burck (1990), Tiller (1991) Rao an You (1992), Singh et al (1994), an Ghosh et al (1996) have use time series moels an associate estimation methos to improve estimators for small areas. In particular, Datta et al (1999) have use times series moels in estimating State-level unemployment rates for the US. Often the survey or census ata from which small area estimates are neee are iscrete or categorical. A general approach for small area estimation base on generalize linear moels is escribe in Ghosh et al (1998). Malec et al (1999) have extene the moels escribe in Malec et al (1997) by incluing an oversampling component in the likelihoo. Farrell et al (1997) exten the mixe logistic moel of MacGibbon an Tomberlin (1989). Moura an Migon (2001) further exten this moel, introucing a component to account for spatially correlate structure in the binary response ata. The naïve mean square error estimator suggeste by Henerson (1975) unerestimates the true mean square error of small area estimators base on mixe effect moels. Kacker an Harville (1984) introuce an estimator for the mean square error of an estimator of a small area mean base on an approximation to its true mean square error uner such moels. Prasa an Rao (1990) evelope an approximation to the Kacker an Harville estimator. Harville an Jeske (1992) also evelope approximations to the mean square error of preictors. Rivest an Belmonte (2000) have evelope the conitional mean square error of a small area estimator. The aim of this report is to review small area estimation an inference methos base on the application of mixe moels within a frequentist environment. Section 2 starts by reviewing the mixe moels that have been use in small area estimation. Section 3 then escribes the ifferent estimation an inference methos that have been use with these moels. Details of the BLUP approach are set out in subsection 3.1, while subsection 3.2 reviews the application of empirical best linear unbiase preiction (EBLUP) methos to small area estimation. Extensions to Generalize Linear Mixe Moels (GLMMs) are reviewe in subsection 3.3. Section 4 conclues the report with a review of moern methos for estimating the mean square error of small area estimators base on mixe moels. 5

7 2 Moels for Small Area Means Let y ti represent the i th population value for a characteristic of interest within a "cell" t within an area (i = 1,2,, N t ; t = 1,2,,T; = 1,2,,D). In general the "cells" can be efine quite freely, an in many cases will inclue a time component. Similarly, we efine vectors X ti which contain corresponing auxiliary population information (covariates). The population an sample sizes for the area i are assume known an are given by T N = N t t= 1 an T n = n t t= 1 respectively. The objective is to preict the value of the small area mean y = N T Nt 1 t= 1 i= 1 y ti. Many ifferent types of moels have been put forwar as a basis for this preiction. In this review we focus on two basic classes of moels, corresponing to fixe effect an ranom effect specifications. The fixe effect specification assumes that all systematic variability between iniviuals from ifferent areas can be explaine in terms of variability in covariates. In contrast the ranom effects specification assumes that significant systematic variation between small areas remains after covariates are accounte for in the moel, an moels this via the aition of area specific ranom coefficients. Moels that o not inclue such ranom coefficients are referre to as fixe effect below, while those that contain ranom coefficients (in aition to fixe effects) are referre to as mixe below. 2.1 Linear Fixe Effect Moels Let y = { y ti }be the population vector of response variable values, with y s enoting the values observe in the sample, an let X = { X ti } be a corresponing matrix of covariate values known for all units in the population, with enoting the corresponing sample component of this quantity. A large class of fixe effect moels for a population can then be represente by the linear form (2.1.1) y = Xβ +e 6

8 where β is a vector of regression coefficients an e enotes a population vector of inepenent ranom "noise" components. A funamental assumption is then that the moel (2.1.1) also applies to the sample ata. That is, one can write y s = β + e s where the components of e s are still inepenent ranom "noise" values. It is clear that the sample ata can therefore be use to calculate an efficient estimator of β. Denote this estimator by ˆ β. An obvious preictor of y ti for a non-sample unit in area is corresponing preictor of the overall population mean of y in area is X tiβˆ an the (2.1.2) yˆ T nt T Nt 1 = N yti + X tiβ ˆ. t= 1 i= 1 t= 1 i= nt + 1 Note that (2.1.2) oes not require that there be any sample units in area. Its basic assumption is that the relationship between y an pecifie in (2.1.1) above remains the same irrespective of area. This preiction-base approach to small area estimation is iscusse further in Holt et al (1979). Ghosh an Rao (1994) review this approach an Ericksen (1974 an 1975) apply the preiction approach to obtain estimates of population change from 1960 to 1970 for 2586 counties in the US. Levy (1971) an Gonzalez an Hoza (1978) use a regression moel-base preiction approach to estimate unemployment in small areas. Alternatively, classical esign-base sampling techniques can be applie to obtain an estimate for the small area mean. Such estimation methos are calle irect in the literature. This approach assumes that there are sample units in each small area of interest. The estimator for the mean of y in small area i is then T n T n (2.1.3) yˆ = wti wti yti = yw t= 1 i= 1 t= 1 i= 1 1 where w ti is the "sample weight" associate with the sample unit inexe by ti. From a moelling perspective, using (2.1.3) correspons to replacing (2.1.1) by the more restrictive ANOVA-type moel (2.1.4) y ti = β + eti 7

9 for each =1,2,,D an Var(e ti ) = σ 2 1 w ti. This is a simple form of the fixe effect moel (2.1.1) with the matrix X efine by vectors that "pick out" the ifferent small areas. There are two basic problems with this approach. First, from a moel-base perspective it assumes that all units in a small area have the same expecte value of y. This ignores the variability in y "explaine" by the ifferent values in X. Secon, it requires that the sample size in any small area be large enough so that the sample weighte mean (2.1.3) above can (i) be calculate, an (ii) be a reasonably efficient estimator of the small area population mean. Neither of these conitions hol in practice. Consequently, irect estimators are of little interest in small area estimation. An extension of the irect estimation approach that "briges the gap" between (2.1.2) an (2.1.3) is base on the moel-assiste approach to esign-base sampling theory. This estimates the regression parameter β in (2.1.1) using the sample weights w ti to obtain a weighte estimate β ˆ w (which is not always efficient) an then estimates the small area mean of interest by (2.1.5) yˆ = N = = y 1 T + N t= 1 i= 1 1 w T t= 1 i= 1 nt T Nt w ti Nt t= 1 i= 1 + ( X X βˆ ti 1 T w t= 1 i= 1 w + nt w T t= 1 i= 1 w tiβ ˆ X w X ) βˆ ti T nt y ti nt t= 1 i= 1 w w ti ti 1 1 T t= 1 i= 1 T nt nt t= 1 i= 1 w w ti ti ( y ti X βˆ ti X βˆ w ti w ) The estimator efine by (2.1.5) is often referre to as a generalise regression estimator or GREG estimator. Technically it still requires that there is sample in every small area of interest, but this requirement is often relaxe an a slightly moifie version of (2.1.5) is calculate, efine by y ˆ = X βˆ in small areas where n is zero (or very small). Clearly this w correspons to using an inefficient version of the moel-base estimator (2.1.2) in such areas an the GREG estimator (2.1.5) where the sample size is "reasonable". See Sarnal (1984) an Hiiroglou an Sarnal (1985). Early evelopments in small area estimation were base on simple apportionment methos. That is, reasonably accurate means for large subgroups in the population of interest 8

10 were calculate, an these were then apportione within a small area of interest accoring to the proportional representation of the ifferent subgroups in the small area. (Such methos were calle synthetic when they were first introuce in NCHS (1968), but now this name is use to escribe any moel-base small area estimate base on a fixe effects moel.) In our notation an apportionment-base estimator for the mean of y in small area is T N (2.1.6) yˆ = t= 1 N t x x t yˆ t where ˆ y t enotes the estimate of the population average of y in subgroup t an it is assume that the covariate x is scalar. When x is ientically equal to one it is easy to see that use of (2.1.6) essentially correspons to the assumption of an ANOVA moel for y within subgroups (not areas) in the population. A popular choice however is where x is the value of y from a previous census. In this case this estimator oes not have a straightforwar linear moel interpretation. For example, suppose it is reasonable to assume that E(y ti ) = β x ti an y ˆ t is unbiase for y t uner this moel. Then (2.1.6) is biase, since E( yˆ y ) = βn T 1 t= 1 N t x t x x t 1 is not zero in general. Design-base properties of these estimators are evelope in papers by Gonzalez an Waksberg (1978) an Levy an French (1977). Assuming that a esignunbiase irect estimator of y is available, Gonzalez an Waksberg (1973) evelope a esign-unbiase estimator of the esign mean square error (p-mse) of the apportionment estimator. The performance of this estimators is investigate further in Levy (1971), who uses this approach to estimate eath rates, in Namekata et al (1975), an in Schaible et al (1977) who use it to estimate regional unemployment rates. Laake (1979) compares the preiction approach with the apportionment approach. Marker (1999) erives the apportionment estimator as form of GREG estimator. A stanar approach in small area estimation is to construct a linear combination of estimators suggeste by ifferent approaches. Such estimators are referre to as composite estimators. Schaible et al (1977) propose a composite estimator base on a linear combination of the unbiase irect estimator an the moel-base preiction estimator. This is an estimator of the form 9

11 ˆ ˆ (1 ) ˆ y = γ y1 + γ y2 where ˆ an yˆ 2 are the irect an moel-base estimators of the small area mean y. An y 1 obvious issue is choice of the coefficient γ. Purcell an Kish (1980) also consier this approach, but use a common value of γ when efining the composite estimator. 2.2 Linear Mixe Moels The linear moel (2.1.1) assumes that all the systematic variability in the population values making up y is explaine by the variation in the values in X. This valiates the assumption that the error vector e is "noise". However, in practice the observe sample resiuals r s = y s ˆ β o not look like noise, an often contain significant between area variation. This implies that the moel (2.1.1) is misspecifie, an that there are "missing" covariates in the moel, whose values vary from one area to another. If there are relatively few small areas making up the population, so that the sample size within any one is reasonable, then this misspecification is easily hanle. If X contains an intercept term, then this can be replace by an area factor. A little consieration shows that this leas to the estimator yˆ + ( X X ) βˆ = y s s where now X an β exclue the intercept term. If X oes not contain an intercept term (so the regression is through the origin) then (2.1.1) can be replace by (2.2.1) y = X ( β + γ ) + e where the parameter γ represents the eviation from the overall moel (2.1.1) in small area, an the γ sum to zero across all such areas. This type of constraine moel forms the basis of the small area moelling approach aopte for unerenumeration estimation in the 2001 UK Census (Abbott et al, 2000). The moel itself can be fitte to the sample ata in the small areas using stanar regression software (e.g. SAS). In many cases, however, there are many small areas, with any particular area containing only a small number of sample units. In such cases introucing a fixe effect for each small area in the moel can lea to consierable loss of efficiency (the inefficient irect estimator is, of course, the archetype such area effect-base estimator). In these cases therefore the basic approach replaces fixe effects for the small areas by ranom effects, 10

12 working on the assumption that the small areas can be consiere an exchangeable set of population subgroups once unit an area specific covariate information is taken into account. In practice this means (2.1.1) is replace by the linear mixe moel (2.2.2) y s = β + Z s u + e s where Z s is a matrix of known covariates (often area specific rather than unit specific) an u is a ranom vector with zero mean an unknown covariance matrix Ω. This vector of "ranom effects" is assume to be uncorrelate with the error vector e s, which is still assume to be "white noise" with a zero mean vector an a covariance matrix proportional to a known iagonal matrix W s, with unknown constant of proportionality σ 2. It immeiately follows from these assumptions that (2.2.3) E(y s ) = β 2 (2.2.4) Var( y s X s ) = Z sω Z s + σ Ws. The components of the matrix Ω an σ 2 are calle variance components an the moel efine by (2.2.3) an (2.2.4) is often referre to as a variance components moel. Note that the same variance components structure applies to the population as a whole (an, by subtraction, to the non-sample units). All one nees to o is to elete the "s" inex in (2.2.3) an (2.2.4). Typically, specialist software is neee to fit moels like (2.2.2)-(2.2.3). Variance components moels have a long history, ating back to Airy (1861). However, their application to small area estimation is relatively recent, with interest focussing on preiction of finite population quantities (e.g. small area means) uner these moels. In this context Royall (1976) gives a very general result that allows one to write own the Best Linear Unbiase Preictor (BLUP) of any linear function θ = a y of the population y-values uner the moel (2.1.1) with arbitrary correlation of sample an non-sample units. Here a is a known vector of constants of the same imension as the vector y of population y-values. Using Royall's result, the BLUP of θ is (2.2.5) ˆ θ = a s y s + a r [X ˆ r β + V rs V 1 ss (y s ˆ β )] where a subscript of s enotes restriction to the sample units, a subscript of r enotes restriction to the non-sample units, an V ss = Var(y s ), V rs = Cov(y r,y s X r, ). The vector β ˆ in this formula is the Best Linear Unbiase Estimator (BLUE) of the parameter β in (2.1.1), efine by the GLS estimator 11

13 (2.2.6) β ˆ = ( X s V 1 ss ) 1 ( X s V 1 ss y s ). In the context of preiction of the mean of y in small area i the quantity θ is efine by a-values that are zero for population units not in small area i an equal to 1/N for units in this small area. Note that calculation of (2.2.5) requires specification of the correlation structure of the resiuals associate with (2.1.1), so this moel nees to be extene appropriately. Also, since in any practical situation the coefficients of V rs an V ss will be unknown (or at least epen on unknown moel parameters), the BLUP (2.2.5) cannot be use in practice. When estimates of these quantities are substitute in (2.2.5) the resulting estimator is often referre to as an Empirical BLUP, or EBLUP. In orer to motivate use of (2.2.5) in the context of a variance components moel, we focus on a very simple form of (2.2.2) that has been applie quite wiely in small area estimation. This is where there are no subgroup ifferences (so we can rop the "t" subscript) X = 1 n, u is a ranom vector of imension equal to the number (m) of small areas with Ω = ω 2 I m, Z s = iag(1 s ) is a matrix that "picks out" the component of u corresponing to any particular small area an W s = σ 2 I n. Here I a enotes the ientity matrix of orer a an 1 a enotes a vector of ones of length a. In this case (2.2.2) is equivalent to (2.2.7) y i = β + u + ei. This moel correspons to the assumption of a common overall mean β, but where area specific sample means vary inepenently aroun this common mean, with variance equal to ω n σ 2. Here ω 2 is the variance of the ranom area effect u. It is straightforwar to see that the BLUE of β is then a weighte average of these area-specific means, given by ˆ β = ( ) 1 (ω 2 + n 1 σ 2 ) 1 y s (ω 2 + n 1 σ 2 ) 1. In orer to construct the BLUP for the mean of y in area we apply Royall's formula (2.2.5), noting that, uner (2.2.7) It follows 2 V rs = iag( ω 1r1 2 2 Vss = iag( ω 1s1 s + σ I s ). s ) V ( [ rsvss = iag ω 1r1s ω 1s1s + σ I s ] 1 1 where straightforwar manipulations can be use to show ) 12

14 ω ω 1 s1 s + σ I s ] = σ [ I s 1 ] 2 s1 s. σ + n ω [ 2 Substituting these results in (2.2.5) an taking the components of the vector a there as being zero for populations units outsie area i an of the area i mean of y is (2.2.8) yˆ = N 1 n y s + ( N n 1 N for units in the area, we see that the BLUP ) βˆ + ( N n 2 nω ) 2 σ + nω 2 ( y s βˆ). Note that when ω 2 is zero (there is no area effect) ˆ β reuces to the overall sample mean of y an (2.2.8) reuces to the "stanar" preiction estimator of y uner a common mean an variance moel for the istribution of the population values of y. Note also that implicit in (2.2.8) is the BLUP of the ranom area effect u i. 2 n ω u ˆ = ( βˆ) 2 2 σ ω ys. + n This is sometimes referre to as the "shrunken" area i resiual. Extening the basic ieas set out in Ericksen (1973, 1974) an Maow an Hansen (1975), Fay an Herriot (1979) use (2.2.2), with Z just containing an intercept term (so the moel only inclues a ranom area effect), to estimate average incomes for small areas (population less than 1000) using 1970 US Census ata. In particular, their moel assume that both the error term an the ranom effects followe inepenent normal istributions. This moel was also use by Ericksen an Kaane (1985) to prouce small area population prouce estimates. Battese an Fuller (1981) an Battese et al. (1988) applie the same moel to estimate mean crop acreage for 12 counties (small areas) in Iowa. A similar moel was applie to estimation of population changes for local areas by Ericksen (1973), while Kott (1989) use it to investigate robust small omain estimation. Prasa an Rao (1990) use (2.2.2) with X = Z to evelop estimators of the mean square error of the BLUP. Datta an Ghosh (1991) extene the moels consiere in Prasa an Rao (1990) an in Battese et al. (1988) to cross-classifie ata an to two stage sampling. In the latter case this was accomplishe by the introuction of separate ranom effects for the ifferent stages of sampling. Hulting an Harville (1991) iscuss all these moels in their comparison of Bayesian an frequentist approaches to the small area estimation. 13

15 In the context of estimating farmlan values in the Unite States, Pfeffermann an Barnar (1991) use the moel (2.2.2). Their response variable is the value reporte the Agricultural an Lan Values Survey (ALVS) an they assume that lan values reporte by farmers resiing in the same county are istribute ranomly aroun the county mean. The county means themselves were moelle as functions of auxiliary variables, representing known county characteristics an ranom state an county effects. Ghosh an Rao (1994) give an excellent review of all these evelopments. The general moel (2.2.2) can be interprete as a multilevel moel (Golstein, 1995) an Moura (1994) an Moura an Holt (1999) use this framework to evelop estimates for small area means. Lahiri an Rao (1995) also consier the Fay-Herriot version of (2.2.2) an show that an estimator of the mean square error of the BLUP, erive by Prasa an Rao (1990) uner an assumption of normality of small area means, is robust to this assumption. Datta an Lahiri (1995) also stuy the Fay-Herriot version of (2.2.2) an propose a robust hierarchical Bayes technique for estimating small area means when one or more outliers are present. The performance of Bayes an frequentist methos for estimating the mean square error of small area estimators base on the moel (2.2.2) is iscusse in Singh et al (1998). Prasa an Rao (1999) evelop a moel-assiste estimator of a small area mean uner the simple moel (2.2.7) an Stukel an Rao (1999) have obtaine estimators for small area means uner neste error regression moels. Hierarchical Bayes estimation of small area means uner multilevel moels is escribe in You an Rao (2000), while Rao (2001) iscusses small area estimation with applications to agriculture base on the use of linear mixe moels. The impact of outliers on small area estimation uner linear mixe moels has been investigate by Li (2000), who erives robust methos for estimating small area means uner the version of (2.2.2) investigate by Battese et al. (1988). Small area estimation methos that "borrow strength over time" have also been investigate. Beginning with the seminal work of Scott an Smith (1974), times series moels have been fitte to aggregate survey ata in the context of estimating corresponing population means. Scott et al (1977), Smith (1978), Jones (1980), Biner an Dick (1986, 1989), Biner an Hiiroglou (1986), Tiller (1989), Bell an Hillmer (1990) an Pfeffermann (1991) are relevant references. 14

16 Tiller (1991) uses a time series moelling approach to estimate state level unemployment rates from the Current Population Survey. This approach is base on a time series moel for the true unemployment rate that inclues auxiliary variables an an ARMA moel for the sampling errors for the survey-base estimates of these rates. Pfeffermann an Burck (1991) iscuss a general class of moels that combine time series an cross-sectional ata in orer to estimate small area means. Their approach assumes a ranom walk moel for the moel regression coefficients an uses the Kalman filter to obtain estimates of these coefficients as well as the associate variance components. Singh at al (1994) exten the moel consiere by Fay-Herriot (1979) to repeate survey ata an use EBLUP methos to obtain estimates of small area means. Their simulation results supporte a combine crosssectional an time series approach to moelling for small area estimation. Rao an Yu (1994) also exten the Fay-Herriot moel to cross-sectional an time series ata. Their propose moel is of the form y + t = X tβ + Z tu1 + u2t 1t e t where u 1 is a cross-sectional ranom effect that oes not vary with time, while u 2t is the outcome of a AR(1) process. Pfeffermann an Burck (1990) an Singh et al (1991) consier similar moels, but assume specific moels for the sampling error. Ghosh et al (1996) also consier a time specific ranom effects moel with the time effects following a ranom walk. Datta et al (1999) use hierarchical Bayes methos to estimate State level unemployment rates in the US using the Rao an Yu (1994) moel an generalise the Fay-Herriot (1979) moel to time series ata. Pfeffermann (1999), Rao (1999) an Marker (1999) are reviews of these evelopments. So far, the emphasis has been on estimating a single characteristic (e.g. a small area mean). However, estimation of multiple characteristics is also of interest in small area estimation. Datta et al (1999) consier corn an soybeans output as two relate characteristics an use neste regression moels to evelop estimates of multivariate small area means base on EBLUP an EB methos. They also obtain a secon orer approximation (base on Prasa an Rao, 1990) to the mean square error of the EBLUP estimators. Their approach is base on the moel consiere by Battese et al (1988) an their results emonstrate the superiority of multivariate moels over univariate moels. Datta et al (1996) apply multivariate area level 15

17 moels to obtain estimates of meian income for four-person families in the U.S. Univariate shrinkage (composite) methos for estimating small area means an rates are extene to multivariate shrinkage methos by Longfor (1999), who also illustrates the avantages of multivariate shrinkage over univariate shrinkage methos. Finally, Fuller an Harter (1987) iscuss methos for estimating multivariate small area means. 2.3 Non-Linear Mixe Moels Extension of small area estimation methos to nonlinear moels has largely been carrie out via Bayesian methos. MacGibbon an Tomberlin (1989) use EB methos to estimate small area proportions. Their analysis uses simulate ata base on the binomial-logistic mixe moel corresponing to (2.2.2). This is efine by (2.3.1) logit( π ) = X β Z u + where π is a vector of probabilities associate with the units in small area i an the iniviual values in small area i are assume to be inepenent Bernoulli outcomes base on these probabilities. In the absence of population microata, Farrell et al (1997) use a secon orer approximation to the EB metho to evelop estimates of small area proportions base on MacGibbon an Tomberlin's moel. They use bootstrap methos to account for uncertainty cause by estimation of prior parameters. Malec et al (1997) apply HB an EB methos to the moel (2.3.1) to estimate the proportion of iniviuals who visit a octor at least once within 12 months. Farrell (1997) extens the EB methos in Farrell et al (1997) to multinomial an orinal moels. A general HB approach to small area estimation base on the generalize linear moel is escribe in Ghosh et al (1998). Their approach also applies to moels with spatial correlation. Maiti (1998) uses non-informative priors for hyperparameters when applying HB estimation methos to estimation of mortality rates for isease mapping. Malec et al (1999) exten the hierarchical moel of Malec et al (1997) by incluing an oversampling component in the likelihoo. They then use EB estimation methos to estimate overweight prevalence in U.S. You an Rao (2000) have evelop HB methos for estimating small area means uner multilevel moels. Moura an Migon (2001) exten the mixe effects logistic moel of 16

18 MacGibbon an Tomberlin (1989) by introucing a further component to account for spatial correlation in the binary response ata. They then use the moel selection methoology propose by Gelfan an Ghosh (1998) to select a best moel. 3 Estimation Methos for Moels with Ranom Effects There are three stanar approaches to small area estimation base on moels with ranom effects. These are Empirical Best Linear Unbiase Preiction (EBLUP), Empirical Bayes (EB) an Hierarchical Bayes (HB). The EBLUP metho usually requires maximum likelihoo (ML) or resiual (restricte) maximum likelihoo (REML) estimation of parameters associate with ranom area effects, an is ientical to the EB an HB approaches in some situations. In this report we focus on the frequentist approach to small area estimation an so we escribe the ML an REML methos use in the EBLUP approach in more etail in the following sections. Althogh we o not treat Bayesian methos in any etail in this review, Empirical Bayes (EB) an Hierarchical Bayes (HB) methos form an important an wiely use class of inferential techniques use in small area estimation. Consequently it is appropriate that we briefly escribe how they are typically applie. In the EB metho, the joint prior an posterior istributions of the small area quantities of interest are first estimate from the sample ata. Typically, these epen on unknown moel parameters, which are then estimate via the usual techniques eg. ML or REML. These estimates are "plugge into" the posterior istribution of the small area quantities of interest, with inference about any particular small area quantity base on the marginal posterior istribution of this quantity. A basic problem with the EB metho is the necessity to account for the uncertainty in estimating the prior/posterior. Several approaches have been propose to account for this uncertainty, with the elta an bootstrap methos being the most popular. Deely an Linley (1981), Morris (1983a, b) an Kass an Steffey (1989) iscuss the elta methos, while Lair an Louis (1987) propose a bootstrap metho. For more etail on the EB metho see Morris (1983). In the HB metho, the fixe effect parameter β an the variance components are treate as ranom. The metho assumes a joint prior istribution for these parameters, an 17

19 Bayes Theorem is use to efine the joint posterior istribution of the small area quantities of interest. This is usually one in several stages. For example Fay-Herriot (1979) first erive the posterior istribution of β given the variance component σ 2 u an then the posterior istribution of σ 2 u. For specifie variance components the HB estimate of a small area mean typically coincies with the BLUP an EB estimate of this quantity The HB metho has the avantage that because moelling is carrie out in stages, each stage can be relatively simple an easy to unerstan, even though the entire moel fitting process can be rather complicate. Furthermore, uner some assumptions concerning the prior istribution of the moel parameters, the HB estimators have smaller MSE than corresponing BLUP-base estimators (Ghosh an Rao, 1994). However, the HB metho also has two major isavantages. The first is its lack of robustness to specification of the prior istribution. The other is its computational complexity. Typically, application of the HB metho requires simulation-base methos such as Markov Chain Monte Carlo (MCMC) to be use in orer to approximate the posterior istribution an hence the posterior means an variances of the small area quantities of interest. 3.1 Best Linear Unbiase Preiction (BLUP) In section 2.2 we introuce the linear mixe moel (2.2.2) as a way of characterising between area variation in the values of the characteristic Y. In particular, the aim is to preict/estimate a linear function a y of the population y-values given this moel. We assume that the vector y can be partitione as y = [ y s, y r ] where subscripts of s an r corresponing to sample an non-sample population units. These subscripts will be use below to enote conformable partitions of other vectors an matrices. Thus, the vector a is partitione conformably as a = [ a s, a r ]. The linear function a y can then be written (3.1.1) a y = a y + a y. s s r r The first term in (3.1.1) epens only on the sample values an is known after the sample is observe. The secon term, which epens on the non-sample values, is unknown. The problem of using the sample values y s to preict the linear function a y therefore becomes the problem of preicting the linear function a r y r uner (2.2.2). In turn, this is a special case of 18

20 the more general problem of preicting the linear function τ = a r (X r β + Z r u) given the linear mixe moel (3.1.2) y s = β + Z s u + e s where the vector of ranom effects u are partitione into m subvectors u u, u, K, u ], = [ 1 2 m with Z s partitione conformably as Z s = [Z s1,..., Z sm ], an Var(u) = σ 2 Ω = σ 2 blk-iag(ϕ j Ω j ) with ϕ j = σ j 2 /σ 2. The ranom errors e s are assume to be inepenent of u with variance σ 2 W s. The variance-covariance matrix of y s is therefore Var(y s ) = σ 2 ( W + Z ΩZ ) = σ 2 Σ s. y s, i.e., Following Henerson (1963), we consier preictors of τ that are linear functions of (3.1.3) τ = s y s + where an are vector an scalar respectively. Unbiaseness of τ uner (3.1.2) implies that (3.1.4) E( τ ) = s β + =E(τ) = a r X r β. For this to hol in general we must have = 0 an (3.1.5) s = a r X r. Put C s = Cov( y s, u). The vector s is then efine by solving the constraine optimisation problem (3.1.6) Minimise Var( s y s τ) = σ 2 s Σ s s + Var(τ) 2 s C s Z r a r subject to: s = a r X r. The Lagrangian for this problem is (3.1.7) L( s,λ) = σ 2 s Σ s s 2 s C s Z r a r 2( s a r X r )λ. Differentiating (3.1.7) with respect to s an λ an equating to zero yiels the estimating s s s equations (3.1.8) σ 2 Σ s X s 0 s λ = C s Z r a r X r a r. Using Searle (1982, pg ), the inverse of the matrix of coefficients on the left han sie of (3.1.8) is given by (3.1.9) σ 2 1 Σ s σ 2 Σ I s 1 Using (3.1.9), the optimum value of s is therefore ( σ 2 X ' s ) 1 σ 2 X ' 1 [ s Σ s I]. 19

21 (3.1.10) s opt = σ 2 C s Z r a r + ( X s ) 1 ( X r a r σ 2 X s C s Z r a r ) = ( X s ) 1 X r a r + σ 2 [I ( X s ) 1 X s ]C s Z r a r an so the BLUP of τ is (3.1.11) τ = a r [X ˆ r β + σ 2 Z r C s (y s ˆ β )] where β ˆ = ( X s ) 1 X s y s is Aitken's generalize least squares (GLS) estimator of β uner (3.1.2). It immeiately follows that the BLUP of θ = a y is (3.1.12) θ = a s y s + a r [X ˆ r β + σ 2 Z r C s (y s ˆ β )]. Substituting C rs = Cov(y r,y s ) for Z r C s in (3.1.12) leas to the BLUP for θ erive by Royall (1976), while setting X r an a s to zero, a r to one an Z r to the ientity matrix in (3.1.11) leas to the BLUP u ˆ = σ 2 C s (y s ˆ β ) obtaine by Henerson (1963). The BLUP of linear combination of fixe an mixe effects set out in Hoerson (1975) is also a special case of (3.1.11). 3.2 Empirical Best Linear Unbiase Preiction (EBLUP) The BLUP evelopment set out in the preceing section assumes variance components are known. In practice of course, this is harly ever the case. We therefore nee to estimate these variance components from the sample ata. The empirical best linear unbiase preiction (EBLUP) metho replaces the unknown variance components in BLUP by these estimates. Henerson (1963) introuce three methos (I, II an III) of estimating variance components. These were in common use up to the 1970s, when avances in computing power le to the introuction of maximum likelihoo (ML) an resiual maximum likelihoo (REML) estimators. In this section we escribe these estimation methos. Henerson (1975) showe that substituting estimate values of variance components in the BLUP le to biase preictions. However, Kackar an Harville (1981) showe that a two-stage approach (first estimate variance components, then use these to estimate an preict fixe parameters an ranom components) leas to unbiase preictors provie the istribution of the ata vector is symmetric about its expecte value an provie the variance 20

22 component estimators are translation invariant an are even functions of the ata vector. They showe that the ML an REML variance component estimators have these properties Maximum Likelihoo Estimation of Variance Components (ML) The ML approach requires parametric specification of the istribution of the ranom component in a mixe moel. A stanar assumption is that this component is normally istribute. With this extra assumption, we can write own the log-likelihoo function generate by the observation vector y s uner the linear mixe moel (3.1.2) as l = (1/2)[nln(2πσ 2 ) + ln Σ s +σ 2 (y s β ) (y s β)] Differentiation of this log-likelihoo function with respect to the parameters β,σ 2 an ϕ j leas to the ML score functions ( ) l/ β = σ 2 X s (y s β) l/ σ 2 = (1/2)[nσ 2 σ 4 (y s β ) (y s β) l/ ϕ j = (1/2)[tr( Z sj Ω j Z sj ) σ 2 (y s β ) Z sj Ω j Z sj (y s β)] Equating these score functions to zero yiels the ML estimating equations for β,σ 2 an ϕ j. Given the MLEs for σ 2 an ϕ j, an hence the MLE ˆ Σ s for Σ s, it is clear that the MLE for β is just the GLS estimator of this parameter efine by Σ ˆ s. That is, β ˆ ML = ( X ˆ s ) 1 X ˆ s y s. However, the estimating equations for the variance components have no analytic solution an so have to be solve numerically Resiual Maximum Likelihoo of Variance Components (REML) For the linear mixe moel (3.1.2), the expectations of the component score functions for σ 2 an ϕ j efine by ( ) are zero. However, if ˆ β ML is substitute for β in these functions, then these expectations are no longer zero. For example, the expecte value of the score function for σ 2 when β is replace by β ˆ ML is -(p/2)σ 2 an so the ML estimator for this parameter is biase. On the other han, an unbiase estimator for σ 2 is efine by ( ) (n p)σ 2 σ 4 (y s ˆ β ML ) (y s β ML ) = 0 21

23 where n is sample size an p is the rank of the matrix. Similarly, we can show that when β is replace by ˆ β ML the expectation of the score function for ϕ j in ( ) is not zero but (1/2) (ln X s / ϕ j. Consequently, an unbiase estimator for ϕ j is efine by ( ) ln X s + ln Σ + 1 ϕ j ϕ j σ (y X 2 s sˆ β ML ) Σ 1 s (y s ˆ β ML ) = 0. ϕ j The Resiual Maximum Likelihoo (REML) approach uses the above iea to reuce bias when estimating variance components. In particular, this approach starts by first transforming y s into two inepenent vectors y s1 = K 1 y s an y s2 = K 2 y s. The y s1 vector has a istribution that oes not epen on the fixe effect β an hence satisfies E(K 1 y s ) = 0, i.e. K 1 = 0, while the y s2 vector is inepenent of y s1 an satisfies K 1 Σ s K 2 = 0. The matrix K 1 is chosen to have maximum rank, i.e. n p, an so the rank of K 2 is p. The likelihoo function of y s is then the prouct of the likelihoos of y s 1 an y s2. REML estimators of variance components are maximum likelihoo estimators base on y s1. That is, the REML metho estimates variance components by maximising the log-likelihoo function efine by y s1 = K 1 y s, l REML = (1/2)[(n p)ln2πσ 2 + K 1 Σ s K 1 + σ 2 y s K 1 (K 1 Σ s K 1 ) 1 K 1 y s ] where the matrix K 1 = W 1 s W 1 s ( X s W 1 s ) 1 X s W 1 s. Note that if K 1 Σ s K 1 is not of full rank, then K 1 Σ s K 1 must be interprete as the eterminant of its linearly inepenent rows an columns. Given this efinition of K 1, the matrix K 2 is efine as K 2 = X s. The loglikelihoo function efine by y s2 = K 2 y s is l L = (1/2)[ pln2πσ 2 + lnk 2 Σ s K 2 + σ 2 (y s2 E(y s2 )) (K 2 Σ s K 2 ) 1 (y s2 E(y s2 ))] = (1/2)[ pln2πσ 2 + ln X s + σ 2 (y s β ) ( X s ) 1 (y s β)] For given values of the variance components, β is estimate by maximizing l L, leaing to β ˆ = ( X s ) 1 X s y s. To gain insight into the REML metho consier the simple stanar linear regression/anova moel, i.e. y s = β + e. Assuming that the ranom error vector e is a multivariate normal with zero mean an variance of σ 2 I, the log-likelihoo functions l L an l REML simplify to 22

24 l L = (1/2)[ pln2πσ 2 + ln X s + σ 2 (y s β ) ( X s ) 1 (y s β)] l REML = (1/2)[(n p)ln2πσ 2 + lnk 1 + σ 2 y s K 1 y s ] where K 1 = I ( X s ) 1 X s. In this special case, the REML estimator of σ 2 is the unbiase 2 σ ˆ REML = y s K 1 y s /(n p) = ( y s [I ( X s ) 1 ]y s )/(n p). Following Patterson an Thompson (1971), the resiual maximum likelihoo estimates of the variance components σ 2 an ϕ j are the values of these parameters that maximise l REML. Differentiation of l REML with respect to these variance components yiels the REML estimating equations l REML / σ 2 = (1/2)[(n p)σ 2 σ 4 y s K 1 (K 1 Σ s K 1 ) 1 K 1 y s ] l REML / ϕ j = (1/2)[{tr(K ϕ j ) σ 2 y s K ϕ j K 1 (K 1 Σ s K 1 ) 1 K 1 y s ] where K ϕ j = K 1 (K 1 Σ s K 1 ) 1 K 1 Σ s / ϕ j. The estimating equations for the variance components have no analytic solution an so have to be solve numerically. In orer to motivate use of the REML metho in the context of small area estimation, we use the simple moel (2.2.7) to illustrate the steps in the iterative calculation of the variance components. In this case ϕ j = ϕ = ω 2 /σ 2 an the steps in the iterations for this special case are as follows: 1. Set k = 0 an assign an initial value ϕ 0 to the variance component ϕ. 2. Set k = k Put γ (k ) = n ˆ ϕ k 1 (1+ n ˆ ϕ k 1 ) Calculate ˆ (k ) β k = γ [ ] 1 γ (k) y s vector of sample means for the m small areas. ( k ) an uˆ (iag( ))( ˆ k = γ y s 1D β k ). Here y s is the 5. Calculate σ ˆ 2 k = (n 1) 1 y s (y s 1 ˆ n β k Z sˆ u k ), where Z s is a n D matrix that "picks out" the small areas. 6. Calculate ϕ ˆ k = (D r k ) 1 σ ˆ k 2ˆ u k ˆ where r k = ϕ 1 k 1 tr[g k + G k Z s 1 n ( 1 n Σ 1 k 1 n ) 1 1 n Z s G k ] u k G k ( k ) (k = iag( γ / n ) an Σ k = blk iag(i n γ ) /n 1 n 1 n ). 7. Return to step 2 an repeat the proceure until the estimates converge. 8. The Empirical BLUP (EBLUP) of the area i mean of Y is then 23

25 y ˆ = N 1 ī [n y s + (N n ) β ˆ + (N n )ˆ γ (y s ˆ β )] where ˆ γ = n ˆ ϕ (1+ n ˆ ϕ ) Generalize Linear Mixe Moels (GLMM) The ata unerpinning small area estimates are often categorical an generalize linear moels (GLMs), see Neler an Weerburn (1972), are a stanar technique for analysing such ata. In the case of small area estimation we use the Generalize Linear Mixe Moel (GLMM) extension of the GLM to allow for correlation within the small areas of interest. In the GLMM E(y s u) = h(η s ) for a specifie function h. Here u enotes the vector of ranom area effects an η s = β + Z s u. As usual, we assume u is normally istribute with zero mean vector an variance-covariance matrix Var(u) = Ω = blk-iag(ϕ j Ω j). Let f(y s ;β u) be the probability ensity function of y s conitional on u. The log-likelihoo of y s conitional on u is then l 1 = ln f (y s ;β u), while the logarithm of the probability ensity function of u is J l 2 = (1/2) [ν j lnπϕ j + lnω j + ϕ 1 j u j Ω 1 j u j ]. j=1 where v j is the rank of matrix Z sj. The function l 2 can be thought of as a penalty function, an the parameter values that maximise l 1 +l 2 are sometimes referre to as penalise likelihoo (PL) estimates. Key references for evelopment GLMM theory are Schall (1991), Solomon an Cox (1992), Wolfinger (1993), Wolfinger an O Connell (1993), Breslow an Clayton (1993), McGilchrist (1994) an Engel an Keen (1994). Breslow an Lin (1995) an Lin an Breslow (1996) review the ifferent estimation approaches introuce by these authors. A typical example of iscrete ata is a binomial response. In this case the vector of small area counts y s = [y s1, y s2,...,y sd ] is assume to follow a binomial istribution with loglikelihoo conitional on the ranom components u given by l = constant + [ y lnθ + ( n - y )ln(1- )] 1 s s θ where y s is istribute as binomial with parameters n an θ. The probabilities θ are assume to satisfy θ = exp( η ) /[1 + exp( η )] where η = X β + Z u. This is usually calle a mixe logistic regression moel. The X an Z are vectors of area level auxiliary covariates. 24