Sampling: What you don t know can hurt you. Juan Muñoz

Size: px

Start display at page:

Download "Sampling: What you don t know can hurt you. Juan Muñoz"

Imogen McCoy
5 years ago
Views:

1 Sampling: What you don t know can hurt you Juan Muñoz

2 Outline of presentation Basic concepts Scientific Sampling Simple Random Sampling Sampling Errors and Confidence Intervals Sampling error and sample size Sample size and population size Non-sampling errors Two-stage sampling and cluster effect Sample design Total size, number of PSUs and cluster size Analytic domains and stratification Implementation The sample frame Selecting the PSUs Planning the survey Household listing Documenting

3 Random Sampling Random Sampling (a.k.a. Scientific Sampling) is a procedure that gives each element of the population a known, positive probability of being included in the sample Random Sampling permits establishing Sampling Errors and Confidence Intervals Other sampling procedures (purposive sampling, quota sampling, etc.) cannot do that Other sampling procedures can also yield biased conclusions

4 Simple Random Sampling

5 Simple Random Sampling A simple random sample would be hard to implement... No list of households available to select from High transportation costs Difficult management...but can be used to illustrate some basic facts about sampling Sampling Errors and Confidence Intervals The relationship between sampling error and sample size The relationship between sample size and population size Sampling vs. non-sampling errors

6 Sampling error and sample size Sampling error e when estimating a proportion p in a sample of size n taken from an infinite population e = p( 1 p) n

7 Confidence intervals In a sample of 1,000 households, 280 households (28 percent) have preschool children e = = ,000 Sampling error is 1.42 percent.

8 Confidence intervals In a sample of 1,000 households, 280 households (28 percent) have preschool children. Sampling error is 1.42 percent. Sampling error percent confidence interval: 28 ± percent confidence interval: 28 ±

9 Sampling error and sample size Sampling error To halve sampling error......sample size must be quadrupled Sample size

10 Sample size and population size Sampling error e when estimating a proportion p in a sample of size n taken from a population of size N e = 1 n N p( 1 p) n finite population correction

11 Sample size and population size Sample size needed for a given precision Population size

12 Sampling vs. non-sampling errors Sampling error Sample size

13 Sampling vs. non-sampling errors Non-sampling error Sampling error Sample size

14 Sampling vs. non-sampling errors Total error Non-sampling error Sampling error Sample size

15 Two-stage sampling The country is divided

16 Two-stage sampling The country is divided into small Primary Sampling Units (PSUs)

17 Two-stage sampling The country is divided into small Primary Sampling Units (PSUs)

18 Two-stage sampling The country is divided into small Primary Sampling Units (PSUs) In the first stage, PSUs are selected

19 Two-stage sampling The country is divided into small Primary Sampling Units (PSUs) In the first stage, PSUs are selected In the second stage, households are chosen within the selected PSUs

20 Two-stage sampling Solves the problems of Simple Random Sampling Provides an opportunity to link communitylevel factors to household behavior The sample can be made self-weighted if In the first stage, PSUs are selected with Probability Proportional to Size (PPS) In the second stage, a fixed number of households are chosen within the selected PSUs The price to pay is cluster effect

21 Cluster effect Sampling error grows when the sample of size n is drawn from k PSUs, with m households in each PSU (n=k m) Intra-cluster correlation coefficient e 2 = e 2 [ 1+ ρ( m 1)] corrected Cluster effect

22 Cluster effects For a total sample size of 12,000 households Number of PSUs Number of households per PSU Intra-cluster correlation coefficient

23 Cluster effects For a total sample size of 12,000 households Number of PSUs Number of households per PSU Intra-cluster correlation coefficient

24 Cluster effects For a total sample size of 12,000 households Number of PSUs Number of households per PSU Intra-cluster correlation coefficient

25 Cluster effects For a total sample size of 12,000 households Number of PSUs Number of households per PSU Intra-cluster correlation coefficient ,

26 Total size, analytic domains and strata Distribution of Nepali households by ecological belt, development region and location (x 1,000) East Center West Mid- West Far- West Total Mountains Urban hills Rural hills Urban Terai Rural Terai Total

27 Total size, analytic domains and strata Distribution of a non-stratified self-weighted sample Urban of 3,300 households in Nepal Mountain sample sample too too small! East Center West Total small! Mid- West Far- West Mountains Urban hills Rural hills Urban Terai Rural Terai Total

28 Total size, analytic domains and strata Distribution of a 3,300 households sample in four strata for Nepal Mountains Mountains 96 Urban Hills Urban hills 24 Rural Hills Rural hills 204 Terai Urban Terai 36 Rural Terai , , ,402, ,501, Stratum Households Sample East Center West Midin Nepal size West Far- West Total Total

29 Total size, analytic domains and strata Distribution of a 3,300 households sample in four strata for Nepal Mountains Mountains 96 Urban Hills Urban hills 24 Rural Hills Rural hills 204 Terai Urban Terai 36 Rural Terai , , ,402, ,501, Stratum Households Sample East Center West Midin Nepal size West Raising Far- West factor , , Total Total

30 Excluded strata Parts of the country may need to be excluded from the sample for security or other reasons

31 Conditions to use the census as a sample frame Exhaustive Unambiguous Linked with cartography Measure of size (for PPS selection) Up to date (?) Area Units of adequate size

32 Planning the survey Selected PSUs should be allocated Among teams During the survey period

33 What is involved? Household listing How long does it take? How much earlier than the survey? Is it always needed? Dwellings or households? household listing Who draws the sample? Asking extra questions during listing Can new technologies help? There will be differences between PSU sizes in the frame and the This will require adjusting the raising factors on a per-psu basis Training, organization, supervision, forms households per enumerator/day As close as possible One of the many issues involved in survey Yes (almost) A dwelling listing is more permanent Ideally, central staff documentation Not recommended Yes (GPS)

34 Most statistical software assume Simple Random Sampling, but Samples are almost always 2-stage or more Samples are almost always stratified It s not unusual to sample households, then subsample families and/or persons within the selected households practically, most samples will not be Simple Random Samples Therefore, sampling errors produced by standard packages most likely are under estimated

35 Deff (Design Effect) is the ratio of variance of a complex survey design to that of an SRS of equal size Deff considers clustering, stratification and weighting) Effective_ Sample_ Size= Sample_ Size_( n) Deff E.g. An estimate from a 3,000 household complex design survey with a deff of 1.5 has a precision equivalent to that of an SRS with 2,000 households

36 Sampling Errors of selected variables from the 1991 Pakistan LSMS (4,783 households) Description Estimate Std. Error (assuming SRS) True Standard Error Deff Effective Sample Size Access to electricity 70.0% 0.7% 1.4% ,123 Household Size ,628 Per Capita food expenditure (rupees) Per Capita total expenditure (rupees) , ,881

37 End of presentation

38 Outline of presentation Basic concepts Sampling design Sample implementation

39 Selecting with PPS PSU No. of households Region Province Cumulative No. of hh Select measure of size Number of households Compute cumulative size k = No. of PSUs to select k = 750 Get sampling step S = Total size / k S = 560,039 / 750 = 747 Get random start 1<R<S R = 358 Compute the sequence R, R+S, R+2S,

40 Two-stage sampling The country is divided

41 Variations from standard practice Random stops Quota sampling Stratification from the listing The FSU case

42 Tilahun

Sample size and Sampling strategy

Sample size and Sampling strategy Dr. Abdul Sattar Programme Officer, Assessment & Analysis Do you agree that Sample should be a certain proportion of population??? Formula for sample N = p 1 p Z2 C 2