Updating Small Area Welfare Indicators in the Absence of a New Census

Transcription

1 Updating Small Area Welfare Indicators in the Absence of a New Census DRAFT Johannes G. Hoogeveen Thomas Emwanu Paul Okiira Okwi* November 13, 2003 Abstract Elbers, Lanjouw and Lanjouw (2003a) show how, for census years, welfare estimates for small target populations can be derived through the combination of sample survey household information with unit record population census information. Estimates for non-census years are less reliable and are, for that reason, typically not generated. This leaves small area welfare estimators often dated. This paper extends the Elbers et al. methodology by showing how reliable, updated small area welfare estimators can be generated in the absence of a new census. The approach requires panel data and the estimation of a relation between per capita consumption from the year of interest and household characteristics from the census year. The method is illustrated for Uganda. It is demonstrated that updated welfare estimates are plausible (in that they match well stratum level estimates calculated directly from the household survey) satisfactorily precise (at a level of disaggregation below that allowed by the household survey), and obtainable at low cost. Key Words: Welfare measurement, poverty, dynamics, Uganda JEL Classification Numbers: I32, O18, C53 * Johannes is with the World Bank (jhoogeveen@worldbank.org), Thomas is employed by the Uganda Bureau of Statistics (tom_emwanu@hotmail.com) and Paul works at the Economics Department, Makerere University, Kampala (pokiira@yahoo.com). We are grateful to the Uganda Bureau of Statistics, Entebbe, for their help with the provision of the survey and census data, and to Simon Appleton for providing expenditure aggregates for 1992 and 1999/2000. The financial support from ILRI, DFID and the World Bank research department is gratefully acknowledged. For useful comments and other forms of help we would like to thank: Johan Mistiaen, Qinghua Zhao, Jenny Lanjouw, Peter Lanjouw, Berk Ozler and Chris Elbers. None of the views expressed in this paper should be taken to represent those of the World Bank or its affiliated organizations. The responsibility for all errors is ours.

2 1. Introduction Indicators of poverty and inequality are available from sample surveys, but they are only representative for the strata identified in the survey. Censuses on the other hand can provide representative information at the lowest administrative levels, but only for welfare correlates such as household size, educational attainment or access to clean water. Small area welfare estimation combines the information in surveys and censuses. In doing so it makes it possible to generate welfare estimates for small target populations. 1 For instance, if the typical household survey yields welfare estimates at the regional level (1 st administrative level), small area welfare estimates typically are able to replicate the results at this level and to also provide the same information for districts (2 nd administrative level), counties (3 rd administrative level) and in some instances subcounties (4 th administrative level). There are various approaches to small area estimation. 2 A method that recently has attracted considerable attention for its ability to arrive at small area welfare estimates and their standard errors, is described in Elbers, Lanjouw and Lanjouw (2003a). For this method to work household characteristics collected in the survey have to be representative of the census. Especially in quickly changing economic environments it follows that survey and census data have to be collected contemporaneously. As a result reliable disaggregated welfare estimates are only available for the census year. And as it takes time to process census data and as censuses are typically implemented once every decade, there is a demand for updated small area welfare estimates. This paper proposes an extension to the Elbers et al. methodology that allows to generate local welfare estimates for non-census years. It does so by combining the population census with a representative household panel survey. The method is inexpensive and requires collection of up to date per capita consumption information for (a subset of) households included in the sample survey held contemporaneously with the census. As long as, for the set of panel survey households, the characteristics collected in the census 1 In this paper we use the terms small area welfare estimation and poverty mapping to refer to all welfare estimates derived for small target populations. 2 For surveys see Ghosh and Rao (1994) and Rao (1999). 2

3 year are representative of the census, and the up to date per capita expenditures are representative of the expenditures from the full (and recent) cross section, updated small area welfare estimators may be produced by combining up to date expenditure information with household characteristics that are common to the survey and the census and that have been collected at the census year. Updating not only permits to arrive at contemporary small area welfare estimates, if the expenditure aggregates collected in the two sample surveys are comparable, small area estimates for the census year can be compared with those for the recent period. The method to update poverty maps is illustrated for Uganda, a country where poverty reduction and decentralization are high on the policy agenda. Uganda has one of the most developed poverty programs in Africa and has, since the early 1990s, been attempting to devolve greater authority to lower administrative levels to fortify the campaign against poverty. Disaggregated, small area, estimates of poverty and inequality are available at the (sub)-county level (Okiira Okwi, Emwanu and Hoogeveen 2003) but the information only reflects the situation for 1992, making it of less relevance for current policy decisions. Household survey information is more up to date, but poverty indicators are only available for regions comprising up to 800,000 households. They show how consumption poverty declined from 56% in 1992 to 32% in 1999/2000. This decline is not uniformly distributed. Poverty fell most in the Central region (from 54 % in 1992 to 26% in 1999/2000) and least in the Northern region (from 73% in 1992 to 67% in 1999/2000). Not only is the relative decline in poverty greatest in the Central region, this is also the region with the lowest poverty incidence in The reverse holds for the North where the decline in poverty is lowest and poverty incidence in 1992 highest. The pattern to which the survey data point is one where regions with highest initial levels of poverty experience least in terms of poverty reduction, leading to greater inequality between the various regions. Yet the few data points are insufficient to investigate in detail the causes for the observed divergence in development or to make inferences about changes in poverty since 1992 below the survey stratum level. In the remainder of this paper small area welfare estimators for 1999 are derived for rural 3

4 Uganda using the 1991 population census and the panel element in the /2000 household surveys. The paper demonstrates that despite structural changes to the Ugandan economy during the 1990s, it is possible to estimate a model for 1999/2000 per capita expenditure using household characteristics from The model is acceptable in part because of the accuracy of its coefficients and its R 2. More importantly the welfare estimates derived from it are plausible in that they closely replicate stratum level estimates calculated directly from the household survey. The welfare estimates are satisfactorily precise as well. For instance 1999/2000 headcount rates of poverty for subcounties (4 th administrative level) have 95 percent confidence intervals of approximate the same width as those of stratum level estimates in the household survey. The paper is organized as follows. In the next section the methodology is outlined. Section three briefly describes the data, after which, in section four, updated small area welfare estimators for rural Uganda are derived for 1999/2000. Section five discusses the accuracy of the results, the ability of updated census based small area welfare estimates to replicate, strata level, sample survey welfare estimates and presents a geographic profile of poverty for 1992 and 1999/2000. The section also compares small area estimates for 1992 derived using the subset of panel survey households with small area welfare estimates derived using the complete 1992 cross section (i.e. the conventional poverty map). A summary of the findings concludes the paper. 2. Methodology The methodology used here to derive small area welfare estimators and their standard errors was first described in Hentchel, Lanjouw, Lanjouw and Poggi (1998) and has been refined in Elbers, Lanjouw and Lanjouw (2002, 2003a). Briefly it comprises regressing household survey per capita consumption on a set of control variables that are common to the survey and the census. Out of sample prediction on unit record census data is then used to yield predicted per capita consumption for each household. Instead of calculating one prediction for each household, a number of simulations (typically 100) is run in which the coefficient vector is perturbed and errors are attributed to the predicted per capita consumption. This yields (100) per capita consumption predictions for each 4

5 household from which point estimates of various welfare indicators and their standard errors are calculated. Below follows a more formal overview of the method as well as a description on how updated welfare estimates can be derived. 2.1 Deriving small area welfare estimators For a household h in location c the (natural logarithm of) household per capita consumption, ln y ch, can be written as the expected value of per capita consumption conditional on a set of household characteristics, X ch, that are common to both the survey and the census and an error term ν ch. If there are more households within one location -as is common for household surveys, the error term can be thought to consist of a location component, η c, and an idiosyncratic household component, ε ch, and be written as: ν = η + ε ch c ch. (1) ych = E[ ln ych X ch ] +η c + ε ch ln. Using a linear approximation to the conditional expectation in (1), the household s logarithmic per capita expenditure can then be modeled as: T (2) ln ych = X ch β + ηc + ε ch, which is estimated using GLS allowing for heteroskedasticity in ε ch latter a logistic model is estimated of the variance of ε ch regressors, comprising of ln ŷch. 3 To deal with the with a set of variables z ch as, X ch, their squares and all potential interactions. The log of the variance is rewritten such that its prediction is bound between zero and a maximum A, set equal to 1.05*max( ε ch ) 2 : 3 In theory it is possible to also allow for heterogeneity in c ηˆ. In practice the number of observations is too small (namely the number of clusters in stratum) to do so. 5

6 2 ε ch T (3) ln = zchα + ρ ch A ε. 2 ch Estimation of (2) and (3) yields the coefficient vectorsαˆ and βˆ. In combination with household characteristics X ch from the census a prediction for the log consumption for each household in the census ln ŷ h can be made. The accuracy of this predicted per capita consumption depends on the properties of the regression model and especially on the precision of the model s coefficients and its explanatory power. As the interest is in the welfare estimates and their standard error, instead of one, a number of predictions is generated by drawing a set of β ~ coefficients along with location and idiosyncratic disturbances. The β ~ coefficients are drawn from the multivariate normal distributions described by their respective point estimates, βˆ, and the associated variance covariance matrix. The idiosyncratic error term, ~ ε ch, is drawn from a household specific normal 4 2 distribution with variance ~ σ which is derived by combining the αˆ coefficients with ε, ch the census data. 5 The location error term, ~ η c is drawn from a normal distribution with variance ~ 2 σ η which itself is drawn from a gamma distribution defined so as to have mean 2 2 σ ˆη and variance V( σ ˆη ). The drawn coefficients ~ β, ~ η c and ~ ε ch are used to arrive at the simulated predicted per capita expenditure: T ~ (4) ln ~ ych = X ch β + ~ η ~ c + ε ch. By repeating this process typically a hundred times, a full set of simulated household per capita expenditures is derived. 4 We experimented with various t and non-parametric distributions and found that the results are robust to the choice of distribution. exp 5 Letting ( z T ) B ch = αˆ and using the delta method, the model implies a household specific variance 6

7 Welfare estimates are based on individuals rather than on households and this has to be accounted for. If household h has m h family members then the welfare measure can be written as W(m, y h, u), where m is the vector of household sizes, y h is household per capita expenditure and u is a vector of disturbances. Disturbances for households in the target population are unknown by definition and cannot be estimated. What can be estimated is the expected value of the welfare indicators given the predicted household per capita expenditures from the census. This expectation is denoted as: (5) ~ = E [ W m, ~ ] µ. y h Based on (5) welfare measures (and their standard errors) can be calculated for different target populations. The performance of these census based welfare estimators may be judged by their ability to replicate the sample survey s welfare estimates (at the lowest level of representative disaggregation attainable) and the size of the standard error of the census based welfare estimators for smaller target populations. The prediction error depends mostly on the accuracy with which the model s coefficients have been estimated (model error) and the explanatory power of the expenditure model (idiosyncratic error). 6 Determined by the properties of the expenditure model and the sensitivity of the welfare estimator to deviations in expenditure, the variance attributable to model error is independent of the size of the target population. The variance due to idiosyncratic error falls approximately proportionately in the number of households in the target population (Elbers et al. 2003a). That is, the smaller the target population, the greater is this component of the prediction error. This puts a limit to the degree of disaggregation feasible. There is also a limit to which aggregation will increase precision. As location is related to household consumption, it is plausible that some of the effect of location remains unexplained even estimator of ˆ 2 σ ε, ch AB 1 AB(1 B) + Var( ˆ) ρ 1+ B 2 (1 + B) = 3 6 Simulation introduces another source of error in the process: computational error. Its magnitude depends on the method of computation and the number of repetitions. It can be made as small as desired with sufficient resources. 7

8 with a rich set of household specific regressors. The greater the fraction of the total disturbance that can be attributed to a common location component, the less one benefits in precision from aggregating over more households. 2.2 Updating small area welfare estimators in the absence of a new census At the core of the small area welfare estimation is an out-of-sample prediction of per capita expenditure using a set of representative household variables that is common to the survey and the census. A close correspondence between census and survey household characteristics is a pre-requisite to yield reliable welfare estimates. Much attention is therefore devoted to identifying common variables by assuring that variable definitions are identical between the census and the survey, that questions are phrased the same way, that coding and enumerator instructions are identical and that the survey and census are fielded contemporaneously. When the latter condition is not met -and this is more of a problem in rapidly changing economic environments, changes in the economic situation will be reflected in household characteristics. As a result, survey variables identified as common to the census, are actually not representative of the census and small area welfare estimates can not be derived. The need for common, representative, regressors effectively closes the possibility to update poverty maps through the use of a household survey from a non-census year. 7 In the presence of panel survey data however, for which one of the waves has been collected at the time of the census, this problem can be avoided. The representativeness of the common survey variables with the census can be maintained by relying on household characteristics collected during the census year. Updated welfare estimates can then be based on expenditures obtained for the more recent period. More formally, and denoting time with subscript t, in the presence of panel data equation (1) can be re-written to: (1 * ) ln y, + 1 E[ ln y, + 1 X, ] +, ε, + 1 ch t = ch t ch t c t ch t η. 7 In reality survey and census are rarely administered at the same time, but the period between both is never long. And always much attention is devoted to assuring that household characteristics obtained from the 8

9 Simulated log per capita expenditure is now derived from (4 * ) instead of (4): (4 * ) ~ T ~ ln y ~ ~, + 1 X, + η + ε, + 1 ch t = ch t c c t β, and welfare estimates are based on: (5 * ) ~ E[ W m, ~ y ] µ. = t+ 1 t+ 1 t h, t+ 1 This changes the original small area welfare estimation methodology in that instead of a contemporaneous association between per capita household expenditure and household characteristics, per capita household expenditure from a different time period is made conditional on household characteristics collected in the census year. To implement the method three conditions have to be met: (i) the survey has to be reweighted, (ii) a set of common census-survey variables has to be identified and (iii) a sufficiently accurate expenditure model has to be estimated. Reweighting the survey is required because at the census based prediction stage only information on household size from the census year is available so that welfare estimates for year t+1 have to be based on information on household size from year t. To assure a close association between census and survey based welfare estimates for year t+1, it is needed to replicate the cross sectional per capita consumption distribution for year t+1 (based on y h,t+1 and m h, t+1 ) using y h,t+1 and m h,t. This implies reweighting the survey. Reweighting the survey in one dimension (expenditure) may have consequences for its representativeness in other dimensions. Hence even if a set of representative variables has been identified between the survey and the census to make a poverty map for year t, it needs to be tested whether, with new weights, these common variables remain representative. After a set of common variables has been identified, a model for year t+1 per capita expenditure can be estimated with household characteristics from year t as survey are representative of those in the census. 9

10 regressors. Estimating a model of future expenditure on past household characteristics is unusual (though less so for permanent income adherents), but recall that the objective of equation (4 * ) is to estimate the conditional expectation of expenditure (from (1 * )) and not a causal relation. The model is only usable if its coefficients are estimated accurately (to limit the variance attributable to model error) and if a reasonably high R 2 (to assure disaggregation for small target populations) is obtained. If these conditions are met, updating small area welfare estimates is feasible without the need for a new census. 3. Data Three data sets are used to arrive at updated small area welfare estimators for Uganda: unit record data from the population census and information from two household surveys. The census was administered in January 1991 and covers 450,000 urban households and 3.0 million rural households. It comprises, for all household members, information on household composition, ethnic background, marital status and educational attainment. For urban households and a 10% sample of the rural households a long form was administered which additionally collected information on activity status, housing conditions, types of fuel used and sources of water. In rural areas, the long form is representative at the district level. The surveys used for this paper are the Integrated Household Survey (IHS) administered between January and December 1992 and the 1999/2000 Uganda National Household Survey (UNHS). Both surveys are of the LSMS type. They are representative in 9 strata: rural and urban areas in Central, East, North and West Uganda plus Kampala, the capital city. Each survey comprises information for about 10,000 households, collects information on household and individual characteristics and contains a consumption module. Uganda s official poverty lines (and hence poverty) are based on IHS information (Appleton 1998). Aggregate consumption is comparable across the IHS and UNHS (Appleton 2002) ensuring that poverty measures can be compared over time. The UNHS and IHS comprise a small number of households (1263) that are common to both; for these panel households their 1992 characteristics along with their expenditures in 1992 and 1999/2000 are available. 10

11 4. Preliminaries Okiira Okwi et al. (2003) describe in detail how, using the IHS and the population census, small area welfare estimators are derived for Uganda for This section draws upon this paper, but focuses primarily on the derivation of updated welfare estimators. Updated small area welfare indicators are derived for rural areas. For urban areas updated estimates are not calculated because only 161 of the survey panel households are urban. This is too small a number to represent the diversity of Uganda s urban households well. Consequently, updated small area welfare estimates are derived for rural areas alone, using information for 1071 rural panel households one observation is dropped because its expenditure looked wild. The identification of a set of representative variables common to both the census and the IHS is crucial in small welfare estimation. For updating, this identification is especially elaborate and proceeds in three steps. First, and using the existing the household survey weight, common IHS variables representative of census household characteristics are identified. Next the household weights of the panel households are adjusted. This is required because (i) the panel households have been identified in an ad hoc, nonrepresentative, manner and (ii) to allow welfare estimates for 1992 and 1999/2000 derived from the full IHS and UNHS to be replicated by the subset of panel households using as population weights the adjusted household weights times 1992 household size. Finally and using the new household weights, it is tested whether common variables initially identified as representative, remain representative. 4.1 Identifying identical variables between census and IHS After a comparison of wording, coding and instructions in the enumerator manual a total of 162 common variables is identified. Whether survey characteristics are actually representative of their census counterparts is checked, at the stratum level, by testing the equality of survey and census means. For continuous variables, the census and survey distributions are compared as well. Despite being identified as common, household size did not pass the distribution 11

12 comparison. It differed consistently between the census and the survey in that small households are underrepresented in the survey. For instance in the Central rural region the fraction of one person households is 18.4%. The corresponding number in the survey is 16.3%. As household size is crucial when deriving per capita estimates, it was less of an option to drop it from the set of common variables. And fed by the suspicion that small households are underrepresented because of non-compliance and a replacement procedure that ignores the size of the non-responding household (Hoogeveen and Schipper 2003), it was decided to reweigh the IHS. The reweighting strategy followed is known as poststratification adjustment (Lessler and Kalsbeek 1992). It ensures that the weighted relative frequency distribution among mutually exclusive and exhaustive categories in the survey correspond precisely to the relative distribution among those same categories in the census. In total 13 different household size categories are distinguished, reflecting households of size 1-12 with category 13 reflecting households of size 13 and over. Reweighting is done at the stratum level. A danger of reweighting along one dimension household size in this case, is that survey variables that are representative using the old weights become unrepresentative once the weights have been adjusted. On the other hand, if the adjustment corrects for a genuine sampling error, the survey s representativeness of the census should improve in all dimensions. As a check on the appropriateness of reweighting we therefore compare, for the set of variables that are considered common on the basis of wording, coding and enumerator instructions, how many survey variables are actually representative of the corresponding variable in the census before and after reweighting. Apart from ensuring representativity for all household size related variables, reweighting increases the fraction of common variables that are representative of the census in all rural strata from 23% to 43%. Having corrected for non-participation due to household size, another concern may be that survey participation varies with household wealth. Mistiaen and Ravallion (2003) demonstrate how such a wealth effect on survey compliance can be estimated using data on non-response across geographic areas. Using information on non-response rates per 12

13 expenditure quintile at the district level (38 districts) we therefore also tested for wealth related non-compliance. Estimates for the linear model of non-compliance on per capita expenditure yielded insignificant results, whereas a quadratic specification turned out to be significant (at the 90% level). It shows an inverted-u shaped compliance-expenditure pattern with people in middle quintile groups more likely to comply than either the richest or the poorest. The difference in compliance rates is only marginal 8, and we therefore only adjust for non-compliance related to household size. 4.2 Replicating survey based welfare estimates using the panel For the panel households additional reweighting is required such that the welfare estimates for 1992 and 1999/2000 derived from the complete sample surveys can be reproduced with the subset of panel households. Also when the panel households are representative of the cross sections from which they originate, adjusting the panel household weights is required because cross-sectional welfare estimates have to be replicated by the panel households using as population weights, household weight times 1992 household size. In this case the panel is not representative of the cross sections so that other, data set specific, adjustments have to be made. This lead to the decision to opt for further reweighting such that at the stratum and national level poverty estimates could be replicated. 9 Post-stratification adjustments are made along three dimensions: household size, expenditure and stratum. Household size is corrected in the way outlined previously. 10 For expenditure, mutually exclusive expenditure groups are constructed at the stratum level for the IHS and the UNHS and adjustment factors calculated as the fraction of panel households in each of the groups. Similarly, stratum adjustments are made by defining four categories: Central, East, North and West. Adjustment factors are calculated as the 8 After correcting for wealth related non-compliance we estimate for the poorest quintile which shows the largest divergence, that the true population proportion is (instead of 0.20); for the wealthiest quintile it is If separate expenditure models are estimated at the stratum level (as was done for the 1992 poverty map) representativity at the national level is not a prerequisite. In this case however, only one expenditure model will be estimated and an adjustment that deals with different response rates per stratum is needed. 10 Being resampled in 1999/2000, household weights have to be adjusted for another round of household non-participation. 13

14 fraction of panel households in each of the groups. A complicating factor is that panel household have two different weights: one pertaining to 1992 and another for1999/2000. As a result, the described procedure yields two adjustment factors per stratum: one for 1992 and another for 1999/2000. To arrive at final adjustment factors a weighted average of the adjustment factors is taken. The weights are determined iteratively and based on the ability of the re-weighted subset of panel households to replicate, at the stratum level, 1992 and 1999/2000 poverty incidence point estimates derived for the full sample surveys. 4.3 Identifying identical variables between census and panel As check on the appropriateness of the derived weights it is tested whether the set of common variables identified as representative of the census in the 1992 poverty mapping exercise, are also representative if we limit ourselves to the subset of panel households and use the newly derived weights. Out of a total of 162 candidate variables 138, 148, 153 and 146 passed the means comparison test in respectively Central, East, North and West rural Uganda. 113 variables passed the test in all four rural strata. This is better than what was attained for the 1992 poverty map when respectively 143, 130, 128 and 130 variables passed the means comparison test in Central, East, North and West rural Uganda and when 92 variables passed the test in all four rural strata. 4.4 Empirical modeling Estimation of an empirical model that explains per capita consumption well is another important aspect of poverty mapping. To capture differences between strata, stratum level models are usually estimated. This was the case for the 1992 poverty map. But with only 1071 rural panel households available, estimating separate models for each stratum could easily lead to over-fitting. In the North for instance as few as 160 panel households were interviewed. So for 1999/2000 one model is estimated with interaction terms for each region except Central which is subsumed in the constant term. To enhance the accuracy of the model (and to keep the model error low) only variables that passed the means comparison test in all four rural strata are considered in the model selection stage and 14

15 only those variables whose p-values are 0.05 or less are maintained. Failing to account for spatial correlation in the disturbances would result in underestimated standard errors on poverty estimates. Sampling in the IHS and UNHS and household surveys is stratified into four regions (divided by rural and urban) and within each region primary sampling units (PSUs) are selected from the list of all census enumeration areas. Within the selected PSUs a number of households (typically 10) is randomly selected for inclusion in the survey. In the IHS, the PSU is therefore the level at which the cluster is defined and this is also the level at which the 1992 poverty map controls for location effects (Okiira Okwi et al. 2003). In the panel it often occurred that no, or only one panel household, was interviewed in a given PSU. So for the updated poverty map, the cluster is defined two administrative areas up from the PSU, at the county level Since unexplained location effects reduce the precision of the poverty estimates, an important goal is to explain the variation in consumption due to location as far as possible with the choice and construction of explanatory variables. With respect to the household model, this is tackled in three ways: 1. Regional dummies (and interaction terms with other household characteristics) are included. 2. Census means of household characteristics such as ethnic origin, household size and composition and the gender, age and average level of education of household heads are calculated at the enumeration area and interactions with household characteristics are created and included. 3. For the long form information collected for 10% of the rural households (and representative at the district level) on housing characteristics, use of fuel, access to water sources etc. district means are calculated and interacted with household characteristics. To select location variables, the common component in the household specific error 15

16 terms is determined and this is regressed on cluster and district means. A limited number of variables (5 at most) that best explain the variation in the cluster fixed effects estimates are then included in the final model. The number of explanatory variables is limited to prevent over-fitting. The selected location variables are included in the household regression model after which a combined model is estimated comprising of household specific and location variables. 11 The adjusted R 2 of the model is This is not particularly high, which may be attributed to at least two reasons. First, variables in the census short forms are limited to mostly household composition, education and ethnic origin. Though this information is correlated to say, family labor or ability to understand extension information, other variables of obvious importance to rural households are not available such as plot size, presence of livestock, soil quality or access to markets. Secondly, household composition and education only change slowly over time. The returns to agriculture are variable and depend on rainfall, illness of family laborers, incidence of pests and diseases and prices. Again some of this variation may be captured if, for instance, the age of the head of household and proneness to disease are correlated, but much of the cross sectional variation attributable to any of these sources will remain unexplained and get subsumed in the error term. Despite not being high, the explanatory levels are comparable to those attained elsewhere in Africa. For Uganda, the R 2 s of the rural models for 1992 vary from 0.30 to 0.44 (Okiira Okwi et al. 2003). In rural Madagascar the adjusted R 2 s range from 0.24 to 0.46 (Mistiaen et al. 2002) and in Malawi they range from 0.25 to 0.45 (Machinjili and Benson 2002). Considering that for the panel households there are 7 years between the collection of the household characteristics and the gathering of consumption information, an R 2 of 0.31 is not particularly low and in fact quite encouraging. A Hausman test described in Deaton (1997) is used to determine whether to estimate with 11 Annex 1 comprises the panel expenditure model for 1999/

17 household weights. 12 To model heteroskedasticity in the household specific part of the residual we choose a small number of variables that best explain its variation out of all selected regressors and predicted per capita expenditure, their squares and all interactions. Before proceeding to simulation, the estimated variance-covariance matrix is used to obtain final GLS estimates of the first stage consumption model. For the panel model, no location errors are drawn. The standard error calculated from the survey data was negative and set to zero, excluding the location error from the simulations. At this point a full model of consumption has been obtained that can be used to simulate any expected welfare measures with associated prediction errors. For a description of different approaches to simulation see Elbers et al. (2001 and 2003a). Table 1: Comparison of Summary Statistics for First Stage Regression Models Number of observations IHS (1992 poverty map) IHS/UNHS panel (updated map) Central East North West All rural strata rural rural rural rural Number of observations Number of clusters Hausman test for weights Regression weighted? Yes Yes Yes No No Adjusted R 2 without location means Adjusted R 2 with location means Note: In the IHS the cluster is defined by the census enumeration area; for the panel by the sub-county. In the panel, the predicted variance of the cluster effect is negative, and set to zero. Consequently in the predication stage cluster errors are not included for panel households. Information on the IHS is from Okiira Okwi et al. (2003). 5. Results This section presents the welfare indicators derived from the out of sample predictions on the unit record census data. Mean per capita expenditure is presented along with measures of poverty. To this end the Foster-Greer-Torbecke measures (FGT(α)) are 12 Compare this to sections 4.1 and 4.2 where population weights are required to identify survey variables that are representative of their census counterparts. Here whether to weigh the regression depends on the additional explanatory power that weighting adds to the regression. 17

18 reported with α-values of 0, 1 and 2 reflecting respectively poverty incidence, the poverty gap and its square. As benchmark the official monthly per capita poverty lines (in 1989 prices) are used of Shs for rural Central, for rural East, for rural North and for rural West Uganda (Appleton 1998). To reflect inequality the Ginicoefficient is presented. 5.1 How well do (reweighted) panel and survey estimates match at stratum level? Table 2 presents stratum-level welfare estimates for respectively 1992 and 1999/2000 derived from respectively the IHS and UNHS and from the reweighted panel households. For the IHS official estimates are presented and estimates obtained after adjusting the household weights for non-participation. The table also presents, in the last column, census based predictions for 1999/2000. For the various FGT(α) poverty measures, the table illustrates a number of points. First reweighting the IHS to adjust for household non-response does not affect the poverty estimates in a significant way. It should not be inferred from this that reweighting is superfluous. 13 This depends on the research question. For instance, if the interest is in the fraction of non-poor living in small households then reweighting makes a significant difference (at the 95% level of confidence) by increasing the fraction from 39.3 to 45.1%. Secondly both for 1992 and for 1999/2000 we cannot reject at the 95% confidence level that the stratumlevel poverty estimates derived for the panel households are the same as those derived for the complete surveys. This provides confidence in the post-stratification reweighting procedure that was followed to assure the representativeness of the panel households. In combination with the large number of variables that passed the means comparison test, it provides a solid basis for deriving census based poverty estimates from the panel households. Unsurprisingly given the small number of panel observations, the strata-level standard errors based on panel data are considerably larger than those reported for either the IHS or the UNHS. 13 The absence of any impact of reweighing on the poverty indicators can be traced to two aspects: (i), the fraction of poor one and two person households is small; and (ii) even after reweighing, members from small households make up only between 8% and 9% of the total population. 18

19 Table 2: Poverty estimates for 1992 and 1999/2000 IHS, official FGT(0) Urban 27.8 Poverty (2.4) Incidence Central rural 54.3 (2.2) East rural 60.6 (2.3) North rural 73.0 (2.9) West rural 54.3 (2.4) FGT(1) Urban 8.3 Poverty Gap (0.8) Central rural 18.7 (1.2) East rural 23.0 (1.3) North rural 29.0 (2.0) West rural 19.2 (1.3) FGT(2) Urban 3.5 Poverty Gap (0.4) squared Central rural 8.8 (0.7) East rural 11.4 (0.8) North rural 14.8 (1.3) West rural 9.3 (0.8) Gini Coefficient Urban (0.03) Central rural (0.01) East rural (0.01) North rural (0.02) West rural (0.01) Per capita Urban consumption (2063) Central rural (638) East rural (480) North rural (632) West rural (500) /2000 IHS, IHS, panel UNHS UNHS panel reweighted reweighted official reweighted (2.4) (5.3) (1.6) (2.8) (2.2) (4.0) (1.4) (3.3) (2.3) (4.2) (1.6) (3.9) (2.9) (5.9) (3.8) (5.2) (2.6) (4.0) (1.9) (4.1) (0.8) (1.6) (0.3) (0.6) (1.2) (1.9) (0.4) (0.9) (1.3) (2.5) (0.6) (1.5) (1.9) (3.4) (2.9) (2.6) (1.4) (2.4) (0.6) (1.2) (0.4) (0.9) (0.1) (0.2) (0.7) (1.1) (0.2) (0.3) (0.8) (1.6) (0.3) (0.6) (1.3) (2.3) (2.0) (1.8) (0.9) (1.8) (0.2) (0.5) (0.03) (0.04) (0.02) (0.07) (0.01) (0.02) (0.02) (0.07) (0.01) (0.02) (0.01) (0.05) (0.01) (0.04) (0.01) (0.03) (0.01) (0.02) (0.01) (0.02) (1699) (2895) (3560) (8702) (629) (770) (862) (2972) (486) (817) (605) (1908) (636) (1054) (773) (809) (537) (780) (528) (932) Panel & census (1.5) 34.3 (2.4) 66.5 (2.2) 31.7 (1.5) (0.5) 9.5 (0.9) 27.2 (1.5) 10.8 (1.0) (0.3) 3.9 (0.4) 14.3 (1.1) 5.7 (0.9) (0.01) (0.01) (0.03) (0.01) (784) (701) (972) (550) Notes: The columns IHS (and UNHS) official present welfare estimates as released by UBOS. IHS reweighted adjusts the IHS sampling weights for household non-response. The columns IHS (and UNHS) panel reweighted provide welfare estimates derived for the set of 1071 panel households. The column panel & census presents updated small area welfare estimates. Standard errors are in parentheses. Standard errors for sample surveys are corrected for survey design. For urban households small area welfare estimators are not derived. 19

20 Turning to the updated, census based, welfare estimates for 1999/2000, the last column of table 2 shows that the stratum level sample survey estimates of poverty, the poverty gap and the poverty gap squared are closely replicated by the updated census based estimates. 14 The size of the standard errors is smaller than the standard errors derived for IHS and is of similar magnitude as to those reported for the UNHS. The updated estimates not only replicate the poverty estimates well, the imputed per capita household expenditure is in all strata within the 95% confidence interval of the household survey. This reflects the fact that the distribution of explanatory variables is similar in the IHS panel and the 1991 census, and pays tribute to the care with which comparable variables have been identified. 5.2 How low can we go? As discussed in section two, the precision of the small area welfare estimates declines with the degree of disaggregation. This because the idiosyncratic error component increases as the number of households in the target population falls. For how small a target population estimates can be reported is an empirical matter. For Uganda one would prefer estimates at the sub-county as this is the lowest administrative level with full time professional staff. Whether this is feasible has to be judged by what is an acceptable level of statistical precision. As benchmark, the precision attained in the household survey is taken, whereby a distinction is made between the absolute magnitude of the standard error and its magnitude relative to the point estimate. 15 According to first the criterion and for poverty incidence, in 91% of the sub-counties the standard error is less than the maximum standard error reported in the IHS (of 5.4% for North rural). For the poverty gap and poverty gap squared the corresponding figures are respectively 82% and 84%. Figures 1-3 consider the ratio of the standard error to the point estimate. The value of this 14 An exception holds for West rural where the poverty gap and its square differ significantly from the survey estimate. 15 Either benchmark is strict. The rural poverty estimates derived from the Ugandan household survey have small standard errors, varying from 1.4 to 3.8. Compare this to Madagascar where rural standard errors vary between 3.5 and 6.5 (Mistiaen et al. 2002) or Kenya [ to be obtained...] 20

21 ratio is represented by the vertical axis, and sub-counties are ranked from lowest to highest along the horizontal axis. This is overlayed by the ratio from the survey estimates for the strata covering rural and urban Uganda. If the zone of acceptability is up to the highest ratio from the survey estimates, then it may be concluded that estimating poverty at this level of disaggregation does not result in particularly noisy estimates. Figure 1 present the ratio for poverty incidence. In almost all instances (90%) are the estimates more accurate than the IHS estimate for urban poverty in the Central region (indicated by the dashed line). The sub-counties for which the ratio exceeds this threshold typically have low poverty incidence; 28% on average as opposed to a national average of 56%. In selected instances are the standard errors very high, the largest being 11.1%. Figure 2 present the poverty gap. Here the accuracy of the estimates is lower. The percentage of sub-counties with lower ratios than observed from the household survey is about 85%. Finally figure 3 presents the poverty gap squared. The percentage of sub-counties with lower ratios than observed from the household survey is again about 90%. Figure 1. Sub-county ratios of standard error and poverty incidence 0.30 sub-county ratio from census highest ratio in survey Percentage rural stratum ratio from IHS urban stratum ratio from IHS Ranking by (s.e. / point estimate) 21

22 Figure 2. Sub-county ratios of standard error and poverty gap sub-county ratio from census 0.25 Highest ratio in survey Percentage urban stratum ratio from IHS 0.05 rural stratum ratio from IHS Ranking by (s.e. / point estimate) Figure 3. Sub-county ratios of standard error and poverty gap squared 0.5 Percentage highest ratio from survey rural stratum ratio from IHS sub-county ratio from census urban stratum ratio from IHS Ranking by (s.e. / point estimate) 22

23 5.3 Comparing the 1992 panel poverty map with the official map for 1992 The updated census based welfare estimates derived for the subset of panel households match the IHS and UNHS stratum estimates well. Comparisons at lower levels of disaggregation cannot be made, because the surveys are not representative at that level. One way to check the accuracy of estimates derived from the panel at lower administrative levels is by comparing welfare estimates for 1992 derived using the subset of panel IHS households with the official poverty map for 1992 derived using the full IHS and reported in Okiira Okwi et al. (2003). This requires estimation of an expenditure model for 1992 using the subset of panel households only and the generation of a set of disaggregated welfare estimates. Thus derived, census based, panel estimates for 1992 can be compared with the census based estimates resulting from the complete 1992 IHS. Table 3 presents, at the stratum level, the 1992 IHS benchmark poverty estimates along with the census based estimates reported by Okiira Okwi et al. (2003) and the estimates derived with the updating methodology using the subset of panel households from the IHS. The expenditure regression underlying the latter estimates is weighted (unlike the regression for 1999/2000), comprises 38 variables (34 for 1999/2000) and has an adjusted R 2 that is identical to that for 1999/2000: T-tests cannot reject (at the 95% confidence level) that that the census based stratum level welfare estimates calculated using the subset of panel households are identical to the estimates of the official poverty map for Likewise, can it not be rejected that the census based stratum level welfare estimates calculated using the subset of panel households have the same mean as the stratum estimates from the IHS sample survey. In comparison to the 1999/2000 panel based welfare estimates, the 1992 panel estimates replicate the, stratum level, survey results less well. Also and unlike the panel based small area standard errors for 1999/2000, the 1992 stratum level, small area panel household based standard errors are somewhat larger than those reported for the IHS sample survey. Using the official small area estimates for 1992 derived using the full IHS from Okiira Okwi et al. (2003), the census based welfare indicators derived for the subset of IHS panel households can be compared at lower levels of disaggregation. Table 4 does so for 23

24 district poverty incidence. Most point estimates are close to each other, but in some instances (e.g. Kalangala, Gulu) the differences are substantial. A t-test on the hypothesis of equality of district means does not rejected the null hypothesis for 28 of the 38 districts at the 95% critical value. For 10 districts it is rejected. Table 3: Comparison of census based welfare indicators for 1992 with those obtained from the household survey (IHS) IHS reweighted IHS & census Panel & census FGT(0) Poverty Incidence Central rural 54.1 (2.2) 54.2 (1.7) 49.3 (2.6) East rural 60.6 (2.3) 63.7 (1.6) 65.6 (2.5) North rural 72.3 (2.9) 74.5 (1.8) 70.1 (3.4) West rural 55.0 (2.6) 55.5 (1.7) 57.0 (3.0) FGT(1) Poverty Gap Central rural 18.6 (1.2) 18.0 (0.8) 16.0 (1.2) East rural 23.0 (1.3) 23.8 (0.9) 25.8 (1.6) North rural 28.3 (1.9) 30.3 (1.1) 27.6 (2.3) West rural 19.8 (1.4) 20.2 (1.0) 19.0 (1.5) FGT(2) Poverty Gap squared Central rural 8.8 (0.7) 8.1 (0.7) 7.2 (0.7) East rural 11.4 (0.8) 11.6 (0.6) 13.3 (1.1) North rural 14.4 (1.3) 15.6 (0.7) 14.0 (1.5) West rural 9.6 (0.9) 9.9 (0.9) 8.9 (0.9) Gini Coefficient Central rural (0.01) (0.02) (0.01) East rural (0.01) (0.01) (0.01) North rural (0.01) (0.01) (0.01) West rural (0.01) (0.02) (0.01) Per capita consumption Central rural (629) (565) (714) East rural (486) (381) (600) North rural (636) (370) (731) West rural (537) (507) (684) Notes: The column IHS reweighted presents benchmark welfare indicators for 1992 derived from the full IHS sample survey. The column IHS & census replicates small area welfare estimates for 1992 as presented in Okiira Okwi et al. (2003). The column Panel & census comprises updated small area welfare estimates for Standard errors are in parentheses. Standard errors for the IHS sample survey are corrected for survey design 24

25 Table 4: Census based rural poverty indicators (1992) for districts Region District Poverty incidence Poverty gap Poverty gap squared Per capita Consumption Panel Full Panel Full Panel Full Panel Full households IHS households IHS households IHS households IHS Central region Kalangala East Uganda (3.6) (3.6) (1.5) (1.4) (0.8) (0.7) (945) (1545) Kiboga (3.2) (2.7) (1.6) (1.5) (0.9) (0.9) (692) (617) Luwero (3.1) (1.9) (1.6) (0.9) (0.9) (0.6) (727) (486) Masaka (3.1) (2.2) (1.5) (1.0) (0.8) (0.7) (708) (571) Mpigi (2.8) (3.2) (1.1) (1.9) (0.6) (1.7) (972) (1068) Mubende (2.8) (4.2) (1.4) (2.3) (0.9) (1.5) (618) (1135) Mukono (2.6) (2.5) (1.1) (1.3) (0.6) (0.9) (1094) (826) Rakai (3.3) (2.2) (1.8) (1.1) (1.0) (0.6) (745) (475) Iganga (3.5) (2.2) (2.0) (1.3) (1.2) (0.8) (819) (488) Jinja (7.4) (4.6) (3.1) (2.0) (1.6) (1.1) (2748) (1654 Kamuli (3.7) (3.0) (2.8) (2.0) (1.9) (1.3) (831) (648) Kapchorwa (3.9) (5.9) (2.3) (3.0) (1.4) (1.7) (811) (1325) Kumi (7.6) (3.7) (5.5) (3.6) (4.1) (2.6) (2911) (844) Mbale (2.8) (3.0) (1.8) (1.5) (1.1) (0.9) (654) (707) Pallisa (3.8) (4.0) (2.2) (2.3) (1.3) (1.4) (777) (868) Soroti (2.1) (2.7) (3.8) (2.0) (3.7) (1.4) (725) (621) Tororo (3.1) (3.8) (2.2) (2.3) (1.5) (1.4) (645) (849) 25

26 Region District Poverty incidence Poverty gap Poverty gap squared Per capita Consumption North Uganda West Uganda Panel households Full IHS Panel households Full IHS Panel households Full IHS Panel households Full IHS Apac (3.8) (3.2) (2.7) (1.9) (1.8) (1.1) (819) (718) Arua (6.1) (6.5) (3.5) (3.2) (2.1) (1.7) (1280) (1039) Gulu (6.9) (3.1) (4.4) (2.3) (2.7) (1.6) (1466) (804) Kitgum (2.9) (1.3) (3.0) (2.0) (2.4) (1.8) (680) (400) Kotido (4.0) (1.3) (2.7) (1.9) (1.7) (1.6) (800) (372) Lira (4.1) (2.6) (2.5) (1.6) (1.5) (1.0) (848) (614) Moroto (5.2) (2.9) (3.0) (2.6) (1.9) (2.0) (1099) (1145) Moyo (3.7) (3.2) (3.5) (1.9) (2.6) (1.2) (820) (648) Nebbi (3.2) (2.2) (3.2) (2.1) (2.5) (1.5) (761) (495) Bundibugyo (3.5) (3.2) (1.9) (1.9) (1.1) (1.3) (654) (947) Bushenyi (3.5) (3.1) (1.6) (1.4) (0.8) (0.7) (936) (729) Hoima (2.8) (7.6) (2.4) (5.0) (1.6) (4.6) (611) (2672) Kabale (4.8) (3.5) (2.7) (2.0) (1.6) (1.2) (989) (937) Kabarole (3.2) (2.6) (1.5) (1.7) (0.8) (1.3) (696) (684) Kasese (4.2) (4.9) (2.7) (2.7) (1.7) (1.7) (827) (1497) Kibaale (2.7) (7.7) (2.0) (2.2) (1.3) (3.2) (592) (1613) Kisoro (4.4) (3.3) (2.4) (2.4) (1.5) (1.6) (877) (700) Masindi (2.7) (7.7) (1.9) (2.8) (1.2) (3.5) (611) (1738) Mbarara (6.0) (2.7) (2.7) (1.2) (1.4) (0.7) (1501) (734) Rukungiri (4.6) (2.9) (2.5) (2.0) (1.5) (1.3) (944) (666) Notes: Standard errors in parentheses. The Full IHS column is from Okiira Okwi et al. (2003). Estimates in the Panel households column are based on an expenditure regression for 1071 rural panel households.

27 Figure 4 takes the comparison one administrative level below the district. It presents census based estimates for poverty incidence at the county level derived using the full IHS and using the subset of panel households. The figure shows how, in a number of cases, the means differ substantially and lie far from the 45 degree line. A t-test on the hypothesis of equality of county poverty incidence means is not rejected for 101 of the 149 counties at the 95% critical value. For 48 counties it is rejected. Although in many instances the differences between the 1992 census based estimates derived using the IHS panel households and the official 1992 small area estimates are substantial, the estimates are closely correlated (a, population weighted, correlation coefficient of 0.77). Figure 4 also suggests the absence of systematic divergence. The latter is confirmed by regressing the panel derived estimates for poverty incidence on the official small area estimates. If the regression is run without a constant term, the regression coefficient is 1.01 and not significantly different from 1. Similar results are obtained for per capita consumption, the poverty gap and the poverty gap squared. Figure 4: Small area welfare estimates of 1992 poverty incidence at county level "Official' small area estimate Panel based small area estimate 27

28 5.4 Comparing poverty in 1992 and 1999/2000 Table 5 presents district level welfare estimates for the Central region for 1992 and 1999/ The 1992 estimates are copied from Okiira Okwi et al. (2003) and the 1999/2000 estimates are derived with the updating methodology. 17 From the discussion in the previous section it is clear that the 1999/2000 district estimates have to be interpreted with care. Though the 1999/2000 expenditure model performs better than the model estimated for 1992 in that the standard errors on the welfare estimates are smaller and that the stratum level estimates from the survey are more closely replicated, considerable divergence from the actual (but unknown) estimates is a real possibility. Keeping this caveat in mind but realizing that the results are correct on average, one could still consider changes and trends. This is possible because the expenditure aggregates that were calculated using the 1992 IHS and the 1999/2000 UNHS are comparable (Appleton, 2002). The results confirm the sample survey results mentioned in the introduction, that the drop in poverty incidence was highest in the Central region (where it dropped by 30 percentage points), and lowest in Northern Uganda where poverty dropped 8 percent points. The census based estimates allow, unlike the survey, to consider changes in poverty at administrative levels below the stratum. So whereas the survey presents evidence that poverty declined in all regions, table 5 illustrates how the drop in poverty was widely distributed across districts: poverty dropped in almost all districts as well. There are three districts (Apac, Moyo and Kasese, not reported in table 5) in which poverty might have increased. The three districts have all been affected by insurgency in the period so that it is plausible that poverty did not decline during the 1990s (the increases are not significantly different from zero). Gulu is another district affected by insecurity 16 A full set of estimates for all regions and at the sub-county level can be obtained by sending an to jhoogeveen@worldbank.org. 17 An issue requiring further investigation is that the standard errors for the 1992 and 1999 estimates are not independent as they are derived from the same census. Correlation may come through the ˆβ ' s, ε ch s and the cluster effects η c s. To control for this, simultaneous estimation of the 1992 and 1999 would be needed. The importance of correlation is likely to be limited, however, because the panel households are a small subset of the full IHS, because the various consumption models comprise different variables and because the cluster is defined at the PSU for 1992 and the county for

29 and the reported drop in poverty may be incorrect. 18 There is considerable within region variation in poverty incidence and reduction. For instance, in the Central region there are districts where the drop in poverty between 1992 and 1999/2000 was only 15 percentage points, but there are also districts such as Rakai where the drop was close to 40 percent points. Table 5: Census based district level welfare indicators for 1992 and 1999, Central region Region District Population FGT(0) FGT(1) FGT(2) Consumption (p.c.) Central Kalangala 0.1% (3.6) (1.9) (1.4) (0.6) (0.7) (0.3) (1545) (904) Kiboga 0.9% (2.7) (1.7) (1.5) (0.6) (0.9) (0.3) (617) (626) Luwero 2.8% (1.9) (1.6) (0.9) (0.5) (0.6) (0.2) (486) (681) Masaka 5.2% (2.2) (1.5) (1.0) (0.5) (0.7) (0.2) (571) (662) Mpigi 5.3% (3.2) (2.9) (1.9) (0.9) (1.7) (0.4) (1068) (2808) Mubende 3.2% (4.2) (2.1) (2.3) (0.7) (1.5) (0.3) (1135) (627) Mukono 5.0% (2.5) (3.7) (1.3) (1.4) (0.9) (0.7) (826) (1011) Rakai 2.5% (2.2) (1.8) (1.1) (0.5) (0.6) (0.2) (475) (862) Notes: The 1992 estimates are from Okiira Okwi et al. (2003). The 1999 estimates are derived using the updating methodology. Standard errors are in parentheses. Figure 5 returns to the question raised in the introduction. Has poverty declined most in areas with lower initial levels of poverty? The figure presents a scatter plot with the proportional decline in poverty at the county level on the vertical axis and initial poverty on the horizontal axis. The scatter plot shows little in terms of a relation between the changes in poverty and initial poverty levels. The line however, which is a locally weighted smoothed function of the decline in poverty, suggests that areas with the highest levels of initial poverty did worst in terms of poverty decline. This is an alarming result 18 The unexpected result for Gulu is possibly brought about by the fact that 1999/2000 UNHS was only administered in secure areas, leading to an overestimation of consumption. Also note that the 1992 estimate for Gulu derived using the panel households differed significantly (65%) from the official census based estimate (76%) for

30 as it means a growing divergence between Uganda s poorest and better off regions. The finding is, however, indicative at best. The negative slope may, for instance, have been brought about by measurement error. In the absence of any real correlation between poverty reduction and initial levels of poverty, a negative correlation would be found if the 1992 level of poverty was measured with error. Even if the negative slope is not brought about by measurement error, one still has to investigate whether the relation is statistically significant. Such an investigation is beyond the scope of this paper, as it requires taking into account that both right and left hand side variables are estimates with an standard error (but see Elbers, Lanjouw and Lanjouw 2003b on this issue). Figure 5: Decline in poverty (as fraction of initial poverty) and initial poverty in Proportional decline in poverty Poverty incidence in Geographic profiles of poverty for 1992 and 1999 The previous subsections have shown that unbiased small area welfare estimates can be produced for 1999/2000 that are closely correlated to the true, but latent, welfare estimates. Since it is known where the various districts, counties and sub-counties for which the welfare estimates are derived are located, it is possible to present a geographic poverty profile in the form of maps in which different colors depict different levels of poverty. Figure 6 does so and depicts poverty incidence at the county level for It illustrates 30

31 dramatically the heterogeneity in welfare levels in the Northern region, with households living in the North West facing less poverty than those in the North East. The strata level survey estimates, suggest that households in the Central district are better off. The detail provided by the county map, shows that within the Central region especially households in Wakiso and Mukono along with household living in the South (some of which live in the Western Uganda) between Lake Edward and Lake Victoria. Figure 7 shows poverty incidence for 1999/2000. It illustrates how the pattern of wealth did not change much during the 1990s. In comparison to the map for 1992, the greening of the 1999/2000 map illustrates dramatically how almost all rural areas in Uganda benefited from the growth that took place during the 1990s. Poverty reduced in all districts, with the exception of North Eastern Uganda where poverty reduction stagnated. Finally, two notes of warning about putting small area welfare estimates on the map. This paper has placed considerable emphasis on the fact the census based poverty estimates are associated with a standard error. The maps do not reflect this, and in various instances counties that are classified differently on the map, have means for which a t-test cannot reject that they are identical. Next, poverty incidence is just one way to report poverty. Instead of reporting the fraction of poor, a geographic profile of welfare could also take into account land area and report poverty density i.e. the number of poor per square kilometer. If one were to do so the geographic poverty profile becomes very different, with poverty being least an issue in the North and being most urgent near the Rwandan border in the South West and South of Mt Elgon in the South East (see figure 8). 31

32 6. Conclusion By combining census with survey data, welfare estimates can be derived for target populations that are much smaller than what is feasible using sample survey data alone. In Uganda for instance, the household survey program generates welfare indicators for regions comprising 600,000 to more than 800,000 households. The poverty map on the other hand provides estimates for counties with as few as 2000 households. A disadvantage of the latter is that the small area estimates are only available for the census year. For Uganda it means that the most recent local welfare estimates are for The methodology outlined in this paper shows how in the presence of panel data, a poverty map can be updated to a non-census year. Updating requires reweighting the household survey to ensure that welfare estimates for the year of interest can be replicated using population information from the census year. It also requires regressing year t+1 per capita expenditures on household characteristics from the census year, t. The method is illustrated for rural Uganda. Updated local welfare estimates are generated for 1999/2000 using data for 1071 rural households for which expenditure and other household information was collected in both 1992 and 1999/2000. The findings are threefold. First generating an updated poverty map is theoretically straightforward and empirically feasible. It is possible to reweigh the survey such that the stratum level welfare estimates from the household survey for year t+1 can be replicated using expenditure from year t+1 for the sub-set of panel households and using population information from the census year t. Additionally, estimating a sufficiently precise expenditure model required to update a poverty map, is feasible even though the number of years between the surveys is substantial (8 years in this illustration), the economic environment changed considerably and the census questionnaire asked only limited information. The derived updated welfare estimates are accurate in that they are able to replicate stratum level survey results and precise: for about 90 percent of the sub-counties three administrative levels below which the household survey is representative, the standard error of the poverty estimates is smaller than that of the urban strata in the household survey. 32

33 Updating requires panel data and estimation of an updated poverty map and will typically be done on a smaller survey data set than the one used to generate the poverty map for the census year. In the case of Uganda, the 1992 rural poverty map is based on a survey with 6,396 observations, whereas the updated map is based on 1,171 observations. This has implications. Updated welfare estimates for urban areas are not derived and the estimation procedure had to be adjusted. For instance one expenditure model with regional interaction terms was estimated instead of one for each of the four rural strata; district dummies could not be used because not all districts were represented in the panel and indicators of ethnicity obtained from the census were used instead. These deviations from the preferred poverty mapping methodology require careful scrutiny of the generated welfare estimates. Fortunately, in a typical case where a poverty map is updated, small area estimates already exist for the census year. The second important result from this exercise is that one should not only estimate an updated poverty map for the year of interest, but an updated map for the census year should also be generated. The comparison of the updated census year map, with the actual poverty map for the census year, allows to check the accuracy of the method. Together with the R 2 of the updated expenditure model and the accuracy with which stratum level welfare estimates from the sample survey are replicated, it guides the decision on how to use updated small area results. Comparing the poverty map results for 1992 with updated results for 1992 reveals that in about 30% of the cases the updated estimates are significantly different (at the 95% confidence level) from the official estimates. Though the performance of the 1999 updated model is better than that for 1992 (in terms of standard errors and the accuracy with which stratum welfare estimates from the survey are replicated by the census method), it nevertheless raises the issue whether the updated results are suited for use as point estimates for particular administrative areas. The conclusion that we reached is that without further verification the updated results should not be used as indicators for the welfare in specific sub-counties, counties or districts. 33

34 It has also been shown that the updated estimates are unbiased, and closely correlated, estimates of the true welfare estimates for 1999/2000. So for statistical applications the updated welfare estimates for 1999/2000 can very well be used. And as the welfare estimates for 1992 and 1999 are derived from identical expenditure aggregates, the 1992 and 1999 welfare estimates represent one of the first data sets with comparable welfare estimates for two points in time for a substantial number of observations 149 if county level estimates; 732 sub-county estimates. This opens many possibilities for research to better understand areas poverty dynamics, pro-poor growth etc.. Finally updating requires information on household characteristics from the census year and per capita expenditure information from the year for which the update is made. In the absence of panel data or if insufficient panel observations are available, it may be considered to collect the necessary information. This is relatively inexpensive as only information on per capita household consumption is needed. There is no need for questionnaire design (as the original survey modules should be used) nor is there a need for an elaborate sampling procedure as households included in the initial survey have to be revisited. If the interest is in regional updated welfare estimates this can be attained by implementing a survey that only collects in the region of interest per capita expenditure information. 34

35 Annex 1: First stage regression model for panel households Intercept Number of males aged 10 years or less Squared fraction of females aged Fraction of spouses with secondary education D-East Uganda Number of household members with O-level or above (ea) Interactions for all regions Coeff. Maximum number of years of education * number of Bagisu per household (ea) Number of males with secondary education * number of Buganda per household (ea) Log household size * living in servants quarters (d) Log household size * household goes without a toilet (d) Households (%) with less than three members * number of Batoro per household (ea) Mean years of education of adult household members * number of Madi per household (ea) Adult equivalent household size * iron roof (d) * number of Lugbare per household (ea) Adult equivalent household size * iron roof (d) * electric lighting (d) Head is divorced, separated or widowed * age of head squared * electric lighting (d) Interactions for North Uganda Age of head of household squared * number of Bagisu per household (ea) Log household size * head of household is involved in unspecified activity (ea) Households (%) with less than three members * number of Lugbara per household (ea) Households (%) with less than three members * head is crafts person (d) Mean years of education of adult household members * cooking on charcoal (d) Age of head of household squared * mud walls (d) Head is divorced, separated or widowed * age of head squared * electric lighting (d) Interactions for East Uganda Households (%) with less than three members * number of Karimojong per household (ea) Households (%) with less than three members * number of Madi per household (ea) Interactions for West Uganda Log household size * number of Japadhola per household (ea) Number of males with secondary education * number of Acholi household members (ea) Age of head squared * number of Buganda household members (ea) Log household size * head is unpaid family worker (d) Maximum number of years of education * wall made of stone Mean years of education of adult household members * number of Bakonjo per household (ea) Number of males with secondary education * cooking on gas (d) Age of head of household squared * number of Karimojong per household (ea) Age of head of household squared * household has an exclusive bathroom (ea) Adult equivalent household size * iron roof (ea?) * number of Bakiga per household (ea) Head is divorced, separated or widowed * age of head squared * number of Alur per hh (ea) Notes: The dependent variable is log of household per capita expenditure obtained from the 1999/2000 UNHS. Explanatory variables are from 1992 IHS. (ea) indicates a mean taken for the enumeration area calculated from unit record census data. (d) indicates a census mean calculated per district from the 10% long forms administered in each district. The total number of observations is The adjusted R 2 is T-stat 35

36 References Appleton S. Changes in poverty in Uganda , CSAE, WPS/98-15 (July 1998). Appleton S. Regional or National Poverty Lines? (2002). The Case of Uganda in the 1990s. Unpublished manuscript. Deaton, A. (1997). The analysis of household surveys: A microeconometric Approach to development policy. Baltimore, MD: Johns Hopkins University Press. Elbers, C., Lanjouw, J.O. and P. Lanjouw (2003a). Micro level estimation of poverty and inequality. Econometrica 71(1): Elbers, C., Lanjouw, J.O. and P. Lanjouw (2003b). Imputed Welfare Estimates in Regression Analysis. Unpublished manuscript. Ghosh M. and J.N.K. Rao (1994). Small Area Estimation: An Appraisal. Statistical Science 9(1): Hentchel, J., Lanjouw, J. O. Lanjouw P. and Poggi, J. (1998). Combining Census and survey data to study spatial dimensions of poverty. Policy Research working Paper No Washington D.C: The World Bank Hoogeveen J. and Y. Schipper (2003). Survey Non-Response and Household Size in Uganda: Implications for Poverty and Inequality. Unpublished manuscript. Lessler J. T. and W. D. Kalsbeek. (1992). Nonsampling Error in Surveys. (John Wiley & Sons: New York). Machinjili C. and T. Benson (2002). Poverty Mapping Malawi. Results of the fourth iteration of the analysis. February Unpublished manuscript. Minot N. (2000). Generating disaggregated poverty maps: An application to Vietnam. World Development 28 (2): Mistiaen J. and M. Ravallion (2003). Survey Compliance and the Distribution of Income. The World Bank: Policy Research Working Paper Mistiaen J. A., B. Ozler, T. Razafimanantena and J. Razafindravonona (2002). Putting 36

37 Welfare on the Map in Madagascar. The World Bank: African Region Working Paper Series no. 34. Okiira Okwi P., T. Emwanu and J.G. Hoogeveen (2003) Poverty and Inequality in Uganda: Evidence from Small Area Estimation Techniques. Unpublished manuscript. Rao J.N.K. (1999). Some Recent Advances in Model-Based Small Area Estimation. Survey Methodology 25(2):

38 Figure 6 Rural poverty incidence at county level for

39 Figure 7 Rural poverty incidence at county level for 1999/