Lecturer: Dr. Adote Anum, Dept. of Psychology Contact Information:

Transcription

1 Lecturer: Dr. Adote Anum, Dept. of Psychology Contact Information: College of Education School of Continuing and Distance Education 2014/ /2017

2 Session Overview In this Session we will discuss Sampling in Psychological Research and sample size determination. Sampling is the process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen. We will describe probability and non-probability methods and the different types of each method. At the end of the session you will be to explain the difference between probability and nonprobability sampling, and describe the major types of both sampling methods.

3 Session Outline The key topics to be covered in the session are as follows: Topic One: What is Sampling? Topic Two: Types of Sampling - Probability Topic Three: Types of Sampling Non-Probability Topic Four: Determining Sample Size

4 Reading List Cozby, P. C. (2004). Methods in behavioral research (8 th Ed.). Mayfield Pub. Co. CA. (Chapter 9, pages ). Please refer to Sakai for the PDF version of this textbook.

5 Topic One WHAT IS SAMPLING?

6 SAMPLING A sample is a smaller collection of units from a population used to determine truths about that population (Field, 2005) Why do we sample? Lack of Resources (time, money) & workload Gives results with known accuracy that can be calculated mathematically What is a sampling frame? The list from which the potential respondents are drawn

7 Steps in Sampling Process Definition of target population Selection of a sampling frame (list) Probability or Nonprobability sampling Sampling Unit Error Random sampling error (chance fluctuations) Nonsampling error (design errors)

8 Step 1 - Target Population Who has the information/data you need? How do you define your target population? - Geography/location - Demographics - Use - Awareness

9 Step 2 - Sampling Frame List of elements Sampling Frame error Error that occurs when certain sample elements are not listed or available and are not represented in the sampling frame

10 Step 3 - Probability or Nonprobability Probability Sample A sampling technique in which every member of the population will have a known, nonzero probability of being selected Non-Probability Sample Units of the sample are chosen on the basis of personal judgment or convenience There are NO statistical techniques for measuring random sampling error in a non-probability sample generalizability is never statistically appropriate

11 SAMPLING 3 factors that influence sample representativeness Sampling procedure Sample size Participation (response rate) When might you sample the entire population? When your population is very small When you have extensive resources When you don t expect a very high response

12 Topic Two TYPES OF SAMPLING PROBABILITY SAMPLING

13 Probability Sampling Methods Simple Random Sampling the purest form of probability sampling. Assures each element in the population has an equal chance of being included in the sample Random number generators Probability of Selection = Sample Size Population Size

14 Simple random sampling

15 Advantages Minimal knowledge of population needed External validity high Internal validity high Easy to analyze data

16 Disadvantages High cost; low frequency of use Requires sampling frame Not applicable when the population is large Likelihood of exclusion minority or sub groups Does not use researchers expertise Larger risk of random error than stratified

17 SYSTEMATIC SAMPLING Systematic sampling relies on arranging the target population according to some ordering scheme and then selecting elements at regular intervals through that ordered list. Systematic sampling involves a random start and then proceeds with the selection of every kth element from then onwards. In this case, k=(population size/sample size).

18 SYSTEMATIC SAMPLING It is important that the starting point is not automatically the first in the list, but is instead randomly chosen from within the first to the kth element in the list. A simple example would be to select every 10th name from the telephone directory (an 'every 10th' sample, also referred to as 'sampling with a skip of 10').

19 Systematic sampling

20 ADVANTAGES: Systematic Sampling Sample easy to select Suitable sampling frame can be identified easily Sample evenly spread over entire reference population DISADVANTAGES: Sample may be biased if hidden periodicity in population coincides with that of selection. Difficult to assess precision of estimate from one survey.

21 Stratified Sampling If the population has identifiable subgroups sample selection is selected based on the subgroup (stratum). Every unit in a stratum has same chance of being selected. Using same sampling fraction for all strata ensures proportionate representation in the sample. Adequate representation of minority subgroups of interest can be ensured by stratification & varying sampling fraction between strata as required.

22 Stratified Sampling Identify variable(s) as an efficient basis for stratification. Must be known to be related to dependent variable. Usually a categorical variable Complete list of population elements must be obtained Use randomization to take a simple random sample from each stratum

23 Stratified Sampling Types of Stratified Samples Proportional Stratified Sample: The number of sampling units drawn from each stratum is in proportion to the relative population size of that stratum Disproportional Stratified Sample: The number of sampling units drawn from each stratum is allocated according to analytical considerations e.g. as variability increases sample size of stratum should increase

24 Stratified Sampling Advantages Assures representation of all groups in sample population needed Characteristics of each stratum can be estimated and comparisons made Reduces variability from systematic

25 Stratified Sampling Limitations First, sampling frame of entire population has to be prepared separately for each stratum Requires accurate information on proportions of each stratum Stratified lists costly to prepare Finally, in some cases (such as designs with a large number of strata, or those with a specified minimum sample size per group), stratified sampling can potentially require a larger sample than would other methods

26 Cluster Sampling The primary sampling unit is not the individual element, but a large cluster of elements. Either the cluster is randomly selected or the elements within are randomly selected Frequently used when no list of population available or because of cost Is the cluster as heterogeneous as the population? Can we assume it is representative?

27 Cluster Sampling Cluster sampling is an example of 'two-stage sampling'. First stage a sample of areas is chosen; Second stage a sample of respondents within those areas is selected. Population divided into clusters of homogeneous units, usually based on geographical contiguity. Sampling units are groups rather than individuals. A sample of such clusters is then selected. All units from the selected clusters are studied.

28 Cluster Sampling Two types of cluster sampling methods. One-stage sampling. All of the elements within selected clusters are included in the sample. Two-stage sampling. A subset of elements within selected clusters are randomly selected for inclusion in the sample.

29 Cluster Sampling Advantages Low cost/high frequency of use Requires list of all clusters, but only of individuals within chosen clusters Can estimate characteristics of both cluster and population For multistage, has strengths of used methods Often used to evaluate vaccination coverage in EPI

30 Cluster Sampling Disadvantages Larger error for comparable size than other probability methods Multistage very expensive and validity depends on other methods used

31 Topic Three TYPES OF SAMPLING NON- PROBABILITY SAMPLING

32 Quota Ssampling The population is first segmented into mutually exclusive sub-groups, just as in stratified sampling. Then judgment used to select subjects or units from each segment based on a specified proportion. For example, an interviewer may be told to sample 200 females and 300 males between the age of 45 and 60. It is this second step which makes the technique one of non-probability sampling.

33 QUOTA SAMPLING It is this second step which makes the technique one of non-probability sampling. In quota sampling the selection of the sample is nonrandom. For example interviewers might be tempted to interview those who look most helpful. The problem is that these samples may be biased because not everyone gets a chance of selection.

34 Convenience Sampling Sometimes known as grab or opportunity sampling or accidental or haphazard sampling. The researcher using such a sample cannot scientifically make generalizations about the total population from this sample because it would not be representative enough. For example, if the interviewer was to conduct a survey at a shopping center early in the morning on a given day, the people that he/she could interview would be limited to those given there at that given time, which would not represent the views of other members of society in such an area, if the survey was to be conducted at different times of day and several times per week. This type of sampling is most useful for pilot testing.

35 Snowball Snowball sampling is a technique, in which existing study subjects are used to recruit more subjects into the sample Useful when the respondents are difficult to recruit

36 Judgmental or Purposive sampling The researcher chooses the sample based on who they think would be appropriate for the study. This is used primarily when there is a limited number of people that have expertise in the area being researched

37 Topic four DETERMINATION OF SAMPLE SIZE

38 Is Sample Size Important? Sample size calculations are important to ensure that estimates are obtained with required precision or confidence. In experiments concerned with detecting an effect if an effect deemed to be clinically or biologically important exists, then there is a high chance of it being detected, i.e. that the analysis will be statistically significant. If the sample is too small, then even if large differences are observed, it will be impossible to show that these are due to anything more than sampling variation.

39 Importance of Sample Size calculation Scientific reasons Ethical reasons Economic reasons

40 Scientific Reasons In a trial with negative results and a sufficient sample size, the result is concrete In a trial with negative results and insufficient power (insufficient sample size), may mistakenly conclude that the treatment under study made no difference

41 Ethical Reasons An undersized study can expose subjects to potentially harmful treatments without the capability to advance knowledge An oversized study has the potential to expose an unnecessarily large number of subjects to potentially harmful treatments Or lead to wrong conclusions

42 Economic Reasons Undersized study is a waste of resources due to its inability to yield useful results Oversized study may result in statistically significant result with doubtful clinical importance leading to waste of resources

43 Classic Approaches to Sample Size Precision analysis Bayesian Calculation Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available Frequentist a type of statistical inference that draws conclusions from sample data by emphasizing the frequency or proportion of the data Power analysis Most common

44 What Is Statistical Power? Essential concepts The null hypothesis H o Significance level, α Type I error Type II error

45 Statistical Hypothesis Testing When you perform a statistical hypothesis test, there are four possible outcomes Whether the null hypothesis (H o ) is true or false Whether you decide either to reject, or else to retain, provisional belief in H o

46 Statistical Hypothesis Testing Decision Retain H o H o is really true i.e., there is really no effect to find correct decision: prob = 1 - α H o is really false i.e., there really is an effect to be found Type II error: prob = β Reject H o Type I error: prob = α correct decision: prob = 1 - β

47 Type I Error- When H o Is True & It is Rejected When there really is no effect, but the statistical test comes out significant by chance, you make a Type I error. When H o is true, the probability of making a Type I error is called alpha (α). This probability is the significance level associated with your statistical test.

48 Type II Error- When H o is False but You Fail To Reject It When, in the population, there really is an effect, but your statistical test comes out non-significant, due to inadequate power and/or bad luck with sampling error, you make a Type II error. When H o is false, (so that there really is an effect there waiting to be found) the probability of making a Type II error is called beta (β).

49 The Definition Of Statistical Power Statistical power is the probability of not missing an effect, due to sampling error, when there really is an effect to be found. Power is the probability (prob = 1 - β) of correctly rejecting H o when it really is false.

50 Calculating Statistical Power Calculating Statistical Power Depends On 1. The sample size 2. The level of statistical significance required 3. The minimum size of effect that it is reasonable to expect.

51 Sample Size Equations There are several equations for calculating sample size but we will discuss one common example here

52 Determining The Sample Size With a Specified Level Of Precision Calculate an initial sample size using the following equation: n Z 2 2 s 2 B recall 2 2 x z z n x n 2 B 2 2 n Z α s The uncorrected sample size estimate. The standard normal coefficient from the statistical table The standard deviation.

53 Determining Sample Size With a Specified Level Of Precision Calculate an initial sample size using the following equation: n Z 2 2 s 2 B B The desired precision level expressed as half of the maximum acceptable confidence interval width. This needs to be specified in absolute terms rather than as a percentage.

54 Determining Sample Size With a Specified Level Of Precision Confidence level Alpha (α) level Z α 80% % % %

55 References Cozby, P. C. (2004). Methods in behavioral research (8 th Ed.). Mayfield Pub. Co. CA. ods/ (Chapter 9, pages ). Please refer to Sakai for the PDF version of this textbook.

56 Thank You