Survey nonresponse and the distribution of income

Size: px

Start display at page:

Download "Survey nonresponse and the distribution of income"

Anne Foster
5 years ago
Views:

1 Survey nonresponse and the distribution of income Emanuela Galasso* Development Research Group, World Bank Module 1. Sampling for Surveys

2 1: Why are we concerned about non response? 2: Implications for measurement of poverty and inequality 3: Evidence for the US Estimation methods Results 4: An example for China 2

3 1: Why do we care? 3

4 Types of nonresponse Item-nonresponse (participation to the survey but non-response on single questions) Imputation methods using matching Lillard et al. (1986); Little and Rubin (1987) 4

5 Types of nonresponse Item-nonresponse Imputation methods using matching Lillard et al. (1986); Little and Rubin (1987) The idea: Observations with complete data X Yes Y Yes missing data Yes No For sub-sample with complete data: Then impute missing data using: Y = M (X ) Y ˆ = Mˆ ( X ) 5

6 Types of nonresponse Unit-nonresponse ( non-compliance ) (non-participation to the survey altogether) 6

7 Unit-nonresponse: possible solutions Ex-ante: Replace non respondents with similar households Increase the sample size to compensate for it Using call-backs, monetary incentives: Van Praag et al. (1983), Alho (1990), Nijman and Verbeek (1992) Ex-post: Corrections by re-weighting the data Use imputation techniques (hot-deck, cold-deck, warm-deck, etc.) to simulate the answers of nonrespondents 7

8 Unit-nonresponse: possible solutions Ex-ante: Replace nonrespondents with similar households Increase the sample size to compensate for it Using call-backs, monetary incentives: Van Praag et al. (1983), Alho (1990), Nijman and Verbeek (1992) Ex-post: Corrections by re-weighting the data Use imputation techniques (hot-deck, cold-deck, warm-deck, etc.) to simulate the answers of nonrespondents None of the above 8

9 The best way to deal with unit-nonresponse is to prevent it Lohr, Sharon L. Sampling: Design & Analysis (1999)

10 Qualification Motivation Training Work Load Data collection method Interviewers Total Nonresponse Availability Socio-economic Type of survey Respondents Burden Economic Motivation Demographic Proxy 10 Source: Some factors affecting Non-Response. by R. Platek Survey Methodology

11 Rising concern about unitnonresponse High nonresponse rates of 10-30% are now common LSMS: 0-26% nonresponse (Scott and Steele, 2002) UK surveys: 15-30% US: 10-20% Concerns that the problem might be increasing 11

12 Nonresponse is a choice, so we need to understand behavior Survey participation is a matter of choice nobody is obliged to comply with the statistician s randomized assignment There is a perceived utility gain from compliance the satisfaction of doing one s civic duty But there is a cost too An income effect can be expected 12

13 Nonresponse bias in measuring poverty and inequality Compliance is unlikely to be random: Rich people have: higher opportunity cost of time more to hide (tax reasons) more likely to be away from home? multiple earners Poorest might also not comply: alienated from society? homeless 13

14 2: Implications for poverty and inequality measures 14

15 Implications for poverty F(y) is the true income distribution, density f(y) Fˆ ( y) is the observed distribution, density fˆ ( y) Note: ˆ ( F( y ) F y = and F( y ) F y ) = 0 p = p ) 0 r ˆ ( = r 15

16 Implications for poverty F(y) is the true income distribution, density f(y) Fˆ ( y) is the observed distribution, density fˆ ( y) Note: ˆ ( F( y ) F y = and F( y ) F y ) = 0 p = p ) 0 r ˆ ( = r Definition: correction factor w(y) such that: f ( y) = w( y) fˆ( y) y F ( y) = w( x) fˆ( x) dx y p 16

17 Implications for poverty cont., If compliance falls with income then poverty is overestimated for all measures and poverty lines. i.e., first-order dominance: if w (y) > 0 for all y (y P, y R ), F ( y) < Fˆ ( y) then for all y (y P, y R ) 17

18 First-order dominance w (y) > observed corrected

19 Example Poor Non-poor Estimated distribution (%) However, Response rate (%) True distribution of population (%) Correction factors

20 Implications for inequality If compliance falls with income (w (y) > 0) then the implications for inequality are ambiguous Lorenz curves intersect so some inequality measures will show higher inequality, some lower 20

21 Example of crossing Lorenz Curves

22 3: Evidence for the U.S. 22

23 Current Population Survey Source: CPS March supplement, , Census Bureau 3 types of non-interviews: type A: individual refused to respond or could not be reached what we define as non-response type B: housing unit vacant; type C: housing unit demolished we ignore type B/C in our analysis Year total number of households Type A households rate of nonresponse (%) ,574 4, % ,103 4, % ,763 3, % ,932 4, % ,831 6, % All years 303,203 23, % 23

24 Dependence of response rate on income Response rate and average per-capita income for 51 US states, CPS March supplement 2002 State Response Rate Average Income Maryland 86.77% $31,500 District Of Columbia 87.21% $34,999 Alaska 88.16% $26,564 New York 88.61% $26,013 New Jersey 88.71% $28,746 California 89.66% $26,822 Mississippi 95.08% $17,821 Indiana 95.21% $23,909 North Dakota 95.36% $20,154 Georgia 95.66% $23,893 West Virginia 96.65% $18,742 Alabama 97.24% $21,155 24

25 Dependence of response rate on income Response rate and average per-capita income for 51 US states, CPS March supplement % 96% probability of compliance 94% 92% 90% 88% 86% average state income 25

26 Estimation method In survey data, the income of non-responding households is by definition unobservable. However, we can observe the survey compliance rates by geographical areas. The observed characteristics of responding households, in conjunction with the observed compliance rates of the areas in which they live, allow one to estimate the household-specific probability of survey response. Thus we can correct for selective compliance by re-weighting the survey data. 26

27 Estimation method cont., {(X ij, m ij )} set of households in state j s.t. m ij households each carry characteristics X ij, where X ij includes e.g. ln(y ij ), a constant, etc. total number of households in state j: M j representative sample S j in state j with sampled households m j = Σ m ij for each sampled household ε there s a probability of response D εij {0,1} P ( D = 1 X, θ ) = P = logistic( X θ ) εij ij i i 27

28 Estimation method cont., The observed mass of respondents of group i in state/area j is E ( m ) = obs ij obs mij E [ ] = mij Pi Then summing up for a given j yields: Now let s define: i m obs ij P i = i=1 obs obs obs mij mij mij ψ j ( θ ) { E[ ]} = { m j} P P P i i m i ij w ij P i = m These are the individual weights j i i This is known! 28

29 Estimation method cont., obs obs obs mij mij mij ψ j ( θ ) { E[ ]} = { m j} P P P i i i i i where obviously [ ( θ )] = 0 E ψ j Then we can estimate θ = arg min Ψ( θ ) ψ W ( θ )' ψ ( θ ) ˆ 1 ( θ ) 29

30 Estimation method cont., Optimal weighting matrix W = Var(ψ(θ)) Hansen (1982) Assume for single state j: [ ] 2 ψ ( θ ) m σ Var = j j This can be estimated as Finally, Var ( ˆ) 2 θ = ˆ σ [ G' NG] 1 ˆ σ 2 = where ψ j ( θ ) w j 2 ψ G = θ ( θ ) 30

31 Alternative Specifications Specification Ψ(θ) min θ 1 θ 2 θ 3 1: P = logit (θ 1 + θ 2 ln (y ) + θ 3 ln (y ) 2 ) (85.95) (-15.90) (0.7320) 2: P = logit (θ 1 + θ 2 ln (y )) (3.51) (-0.329) 3: P = logit (θ 1 + θ 3 ln (y ) 2 ) (1.646) ( ) 4: P = logit (θ 2 ln (y ) + θ 3 ln (y ) 2 ) (0.289) ( ) 5: P = logit (θ 1 + θ 2 y ) : P = logit (θ 1 + θ 2 y + θ 3 y 2 ) 7: P = logit (θ 1 + θ 2 y + θ 3 ln (y )) (0.202) ( ) (0.463) ( ) ( ) (12.13) ( ) (-1.229) 31

32 Results From Specification 2 P = logit(θ 1 + θ 2 ln(y)) Year Ψ(θ) min θ 1 θ 2 Gini uncorr Gini corr ΔGini % 50.92% 5.43% (4.58) (-0.43) % 49.03% 3.82% (4.42) (-0.418) % 47.67% 3.37% (4.46) (-0.413) % 49.47% 4.48% (3.82) (-0.355) % 48.02% 3.66% (3.51) (-0.329) All % 49.07% 4.24% (1.89) (-0.177) 32

33 Graph of specification 2: Probability of compliance as a function of income 1 probability of compliance $0 $50,000 $100,000 $150,000 $200,000 $250,000 Y... income/capita 33

34 Empirical and Corrected Cumulative Income Distribution observed corrected 0 $0 $50,000 $100,000 $150,000 $200,000 $250,000 34

35 Income Distribution: Magnification observed corrected 0 $0 $2,500 $5,000 $7,500 $10,000 35

36 Correction by Percentile of Income 50% 40% implied correction of income 30% 20% 10% 0% population percentile 36

37 Empirical and Corrected Lorenz Curve observed corrected % total income % population 37

38 Lorenz Curves: Magnification observed corrected % population 38

39 Specifications with Other Variables Specifications 10 18, P = logit(θ 1 + θ 2 ln(y)+ θ 3 X 1 + θ 4 X 2 ): Specification Ψ(θ) min θ 1 θ 2 θ 3 θ 4 Gini corrected ΔGini 2: (baseline) % 4.24% (1.89) (-0.177) 10: X 1 = age X 2 = age 2 11: X 1 = age (3.52) (2.61) (-0.188) (-0.201) ( ) ( ) ( ) 49.09% 49.23% 4.26% 4.40% 12: X 2 = age (2.31) (0.198) ( ) 49.26% 4.43% 13: X 1 = (age>64) 14: X 1 = edu X 2 = edu 2 15: X 1 = edu (2.06) (7.59) (-0.188) (-0.334) ( ) (-1.447) ( ) 49.23% 48.52% 48.68% 4.40% 3.69% 3.85% (2.48) (-0.333) ( ) 16: X 1 = (edu>39) (1.93) (-0.233) (-1.187) 48.53% 3.70% 17: X 1 = sex (1.92) (-0.187) ( ) 48.84% 4.01% 18: X 1 = race % 3.43% (1.96) (-0.183) (0.1592) 19: X 1 = size % 4.32% (2.12) (-0.189) ( ) 20: X 1 = race % 3.55% X 2 = size (2.16) (-0.189) (0.1574) ( ) 39

40 4: China 40

41 Example for China Urban Household Survey of NBS Two stages in sampling Stage 1: Large national random sample with very short questionnairre and high repsonse rate Stage 2: Random sample drawn from Stage 1 sample, given very detailed survey, including daily diary, regular visits etc Use Stage 1 data to model determinants of compliance Then re-weight the data 41

42 Further reading Korinek, Anton, Johan Mistiaen and Martin Ravallion, An Econometric Method of Correcting for unit Nonresponse Bias in Surveys, Journal of Econometrics, (2007), 136: Korinek, Anton, Johan Mistiaen and Martin Ravallion, Survey Nonresponse and the Distribution of Income. Journal of Economic Inequality, (2006), 4:

Survey Nonresponse and the Distribution of Income

Survey Nonresponse and the Distribution of Income Public Disclosure Authorized Surve Nonresponse and the Distribution of Income Anton Korinek, Johan A. Mistiaen, and Martin Ravallion 1 Development Research Group, World Bank, 1818 H Street NW, Washington