Module 4 Approaches to Sampling. Georgia Kayser, PhD The Water Institute

Transcription

1 Module 4 Approaches to Sampling Georgia Kayser, PhD 2014 The Water Institute

2 Objectives To understand the reasons for sampling populations To understand the basic questions and issues in selecting a sample. These include sample size and logistic cost, bias, representativeness, and external validity To understand the advantages and disadvantages of different sampling methods

3 Sampling

4 Purpose Most often, it is not practical or cost effective to study the whole population of interest A sample Is a smaller collection of units from the population Is used to determine truths about the population Allows you to generalize to the larger population Reduces costs Saves time for data collection Gives accurate results that can be calculated mathematically (Field, 2005)

5 Sample A sample is a smaller collection of units from the larger population Sample Population

6 Definitions Population: the group to which you want to generalize Sample: collection of observations Probability sampling: random collection of observations Nonprobability sampling: nonrandom collection of observations Sample Frame: The list of units (subjects) of the population from which the sample is selected

7 Definitions External Validity: The degree to which you can generalize to the population of interest Sampling Error: The precision of statistical estimates

8 Steps in Sampling 1. State the research or evaluation question(s) and variables of interest 2. Determine research design and methodology 3. Identify the population of interest to whom do you want to generalize results 4. Determine the sample frame

9 Steps in Sampling (con t) 5. Decide on sampling method 6. Determine the sample size (discussed in next module) 7. Implement the sample plan 8. Sample the population 9. Review the response rate and the sampling process

10 External Validity The degree to which your sample is generalizable to the other populations, settings, and contexts Threats to External Validity Hawthorne effect: subjects know they are participating in an experiment and respond to expectations or perceived expectations Order Effects: When multiple treatments are studied one after the other, a benefit might accrue from the first treatments

11 External Validity Improving External Validity Replicate study in many different places, with different people and at different times Include a control group or multiple groups

12 Types of Sampling Probability Sampling: a sample in which each element has an equal chance of selection independent of any other Nonprobability Sampling: a sample where each element does not have an equal chance of selection and some elements have no chance of being selected (Babbie, 2013)

13 Types of Sampling Probability Sampling Simple random Systematic Stratified Clustered Multistage Probability Proportion to Size Sampling Nonprobability Sampling Convenience Purposive Snowball Quota Expert Sampling

14 Nonprobability Sampling Convenience Sampling Nonrandom selection Easiest method of collecting a sample Participants selected in the most convenient possible way

15 Convenience Sampling Advantages Convenient Disadvantages Cannot generalize to the population Is only representative of the units (subjects) selected The degree that the sample is similar or different from the population is unknown Low external validity Sample bias is introduced Need to describe limitations of the sample

16 Example of Convenience Sampling Ex. 1 A questionnaire is being piloted in the population where it will be used and the closest group is selected Ex. 2 A water-point survey where only those who visit between 9 and 9:30 AM are interviewed.

17 Convenience Sampling Source: NewsAsiaOne

18 Nonprobability Sampling Purposive Sampling Nonrandom selection Units selected based on researchers judgment Units selected because researcher thinks they will be most useful

19 Purposive Sampling Advantages Judgment based sample Disadvantages Cannot generalize to the population Is only representative of the units (subjects) selected The degree that the sample is similar or different from the population is unknown

20 Example of Purposive Sampling Someone is standing at a waterpoint talking to people that pass by that fit into a particular category (women yrs old.) Identify people passing by, and ask then to participate

21 Nonprobability Sampling Snowball Sampling Nonrandom sample Used in hard to reach groups Units (subjects) selected are asked to nominate other units (subjects) Sample increases in size like a rolling snowball

22 Snowball Sampling Recommends Recommends Recommends Sample Possible participants recommended by sampled participants Source:

23 Snowball Sampling Advantages Can investigate hard to reach groups Disadvantages Cannot generalize to the population Is only representative of the units (subjects) selected The degree that the sample is similar or different from the population is unknown

24 Example of Snowball Sampling Ex. 1 Want to study the water, sanitation and hygiene of the homeless in a specific geographic area and no lists of the homeless exist Go to the geographic area of interest Find one or two homeless people and interview them Ask them for recommendations of other homeless people at the end of the interview Interview the homeless people who are recommended to you

25 Nonprobability Sampling Quota Nonrandom Population divided into specified number and type of units (subjects) A predetermined number of units (subjects) are selected from each group

26 Quota Sampling Advantages Can divide the population into groups and select a certain number from each group Disadvantages Cannot generalize to the population Is only representative of the units (subjects) selected The degree that the sample is similar or different from the population is unknown

27 Example of Quota Sampling Collect proportional information from 40% female and 60% male high school students about their WaSH services because the population of students is 40% female and 60% male.

28 Nonprobability Sampling Expert Sampling Nonrandom sample Used to reach experts

29 Expert Sampling Advantages Can investigate a group of highly knowledgeable experts about a subject area May provide validity for a subsequent sampling approach Disadvantages Cannot generalize to the population Is only representative of the units (subjects) selected The degree that the sample is similar or different from the population is unknown Experts could be wrong

30 Ex. Expert Sampling Want to understand the main challenges in the WaSH sector in country X, a country that you will be working in the coming year Interview experts in the WaSH sector in country X Note that experts come in all forms.e.g. handpump mechanics are experts in their business!

31 Probability Sampling Simple Random Sample Random Each unit (subject) has an equal probability of selection Each unit is numbered and a predetermined number of units are sampled, randomly Here is a link to a video that explains how to create a random sample in excel.

32 Simple Random Sampling Source:

33 Simple Random Sampling Advantages Can generalize to the population Representative of the population The degree that the sample is similar or different from the population is known by calculating sample error Disadvantages More expensive and time intensive than a nonprobability sample May not be practical if sample frame is large

34 Probability Sampling Systematic Random Sample Random Individuals are selected at regular intervals from a list of the whole population The intervals are selected to ensure an adequate sample size It is important that the start point is not automatically the first in the list

35 Systemic Random Sampling Start Source:

36 Systematic Random Sampling Advantages Can generalize to the population Representative of the population Easy to select and evenly spread over population The degree that the sample is similar or different from the population is known Disadvantages Expensive and time consuming if the population of interest is large Is only as random as the mix of the population sampled

37 Steps in a Systematic Random Sample Number the units of the population 1 to N Calculate the sample size you need Decide on the interval size = k Take a random start Sample every k th unit

38 Probability Sampling Stratified Sampling Population divided into subgroups (strata) based on a characteristic(s) Sample is obtained by taking samples from each stratum Probability of inclusion varies according to known characteristics Is taken into account in analysis Examples of strata include: male/female, smoking/nonsmoking, rural/urban, program area/non-program area

39 Stratified Sampling Source:

40 Stratified Sampling Advantages Can generalize to the population Can improve the representativeness of the sample The degree that the sample is similar or different from the population is known Can analyze data according to strata Different sampling approaches can be applied to each stratum Less expensive than a simple random sample Less time intensive than a simple random sample Disadvantages If there are a lot of strata, there may be relationships between the strata if these are not considered in the design, it may bias the results

41 Example of a Stratified Sample You are an organization that mainly works with rural and urban communities and want to understand the WaSH services of the population you serve compared to a comparison group that you do not serve. Population can be divided into rural and urban (1st strata) Rural and urban population can be divided into the group you serve and the group you do not serve (2 nd strata) You can then sample the strata

42 Probability Sampling Cluster Sampling A multistage sampling in which groups (clusters) are sampled in the first stage with each selected group (cluster) sampled in the second stage Population divided into clusters based on geography Stage 1: A sample of clusters is taken Stage 2: Everyone in the cluster is sampled

43 Cluster Sampling Source:

44 Cluster Sampling Advantages Can generalize to a large population Representative of population Fewer costs than a simple random sample Less travel Disadvantages Complex design Requires geographic division of sample frame into small clusters Enumeration Areas Sample size can be larger than a simple random sample for the same precision Greater sampling error than a simple random sample To reduce the sampling error, a large number of clusters must be sampled

45 Ex. Of Cluster Sampling Want to understand the portion of people in USA with access to treated piped water and sanitation services in the household Divide USA into clusters or census tracks Randomly sample clusters Interview households within each selected cluster Map Source : Trochim, 2006

46 Probability Sampling Multi-stage Cluster Sampling A multistage sampling in which groups (clusters) are sampled in the first stage with each selected group (cluster) sampled in the second stage and then those groups are sampled again in a third stage Population divided into clusters based on geography

47 Probability Sampling Stage 1: A sample of clusters is taken if relatively equal in size Ex. A sample of villages taken If not equal in size, the clusters can be selected probability proportionate to size Stage 2: A sample of each selected cluster in stage 1 is taken Ex. Villages are divided into smaller subgroups and a sample is taken Stage 3: A sample of the cluster selected in stage 2 is taken Ex. Households in the sample taken in stage 2 are listed and a sample taken

48 Multi-stage Cluster Sampling Advantages Can generalize to the population Representative of population Fewer costs than a simple random sample Less travel Sample size is larger than simple random sample for the same cost Disadvantages Complex design Requires geographic division of sample frame into small clusters Enumeration Areas

49 Multi-stage Cluster Sampling Ex. WHO wants to facilitate a survey on WaSH at the household level in country x. Population divided into enumeration areas (EAs) based on geography

50 Multi-stage Cluster Sampling Stage 1: A sample of EAs is taken If not equal in size, the EAs can be selected probability proportionate to size Stage 2: If EAs are too large, a sample can be taken of the EAs. Stage 3: A sample of households in the selected EA from stage 2 is taken and the subjects are interviewed about WASH in the household Households in the selected EA are listed before the sample is taken

51 Multistage Cluster Sampling: A more sophisticated form of cluster sampling Goal: is to have each unit (number of households) to have an equal chance of selection If clusters are of differing sizes, give each cluster a chance of selection that is proportionate to its size (number of household)

52 Sampling methods can be combined Stratification in Multistage Cluster Sampling Step 1: Stratify your sample Ex. Stratify by geography (interested in the rural and urban portions of the population) Step 2: A sample of clusters is taken in each strata Step 3: A sample of the households in the clusters selected in stage 2 is taken Ex. Households in the sample taken in stage 2 are listed before the sample is taken

53 Bias Source: Dogbert.com

54 Sources of Bias Change from sample plan Hard to reach units (subjects) are eliminated Replacement of units (subjects) with others Response rate is lower than calculated Sample frame is out of date or does not include all units (subjects)

55 Resources Babbie, E. (2013). The Practice of Social Research. 13th edition. Wadsworth Press. Estrella M. and Gaventa J. Who Counts Reality? Participatory Monitoring and Evaluation: A Literature Review. IDS Working Paper 70. International Workshop on Participatory Monitoring and Evaluation. International Institute for Rural Reconstruction Pp1-27

56 Resources Fitzpatrick JL, Sanders JR, Worthen, BR. (2011). Program Evaluation: Alternative Approaches and Practical Guidelines. 4th Edition. Pearson, Allyn & Bacon ISBN 10: Glanz, K., Rimer, B., Viswanath, K. (2008). Health Behavior and Health Education Theory, Research and Practice. (4 th Edition). John Wile and Sons, Inc.

57 Resources Rossi, P., Libsey, M., Freeman, H., (2004).Evaluation, A Systematic Approach (7 th edition). Sage Publications. Trochim WMK, Donnelly JP. The Research Methods Knowledge Base (3rd Edition). Cengage Learning, Trochim. Research Methods Knowledge Base. dex.php

58 Resources Trochim. Research Methods Knowledge Base. dex.php