Sampling Distributions

Transcription

1 Sampling and Variability Sampling Distributions Ken Kelley s Class Notes 1 / 44

2 Sampling and Variability Lesson Breakdown by Topic 1 Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example 2 Visualization 3 Notation Glossary What You Learned A Worked Example 2 / 44

3 Sampling and Variability What You Will Learn from this Lesson You will learn: The idea of sampling from a population to obtain point estimates. The idea of the variability of point estimates. How the standard deviation of a mean decreases as the sample size increases (specifically at a rate of 1/ n). How to find probability values for a mean (rather than a single score, as was done when discussing the normal distribution). How the Central Limit Theorem allows the normal distribution to be used when interest concerns a mean (even if the distribution of scores is not normal). 3 / 44

4 Motivation Sampling and Variability If the mean effectiveness of a blood pressure medication is estimated to be a 15 point reduction in blood pressure after a week (i.e., X = 15), does this mean that µ = 15? 4 / 44

5 Motivation Sampling and Variability If the mean effectiveness of a blood pressure medication is estimated to be a 15 point reduction in blood pressure after a week (i.e., X = 15), does this mean that µ = 15? What if you discovered that X = 15 when n = 20? 5 / 44

6 Motivation Sampling and Variability If the mean effectiveness of a blood pressure medication is estimated to be a 15 point reduction in blood pressure after a week (i.e., X = 15), does this mean that µ = 15? What if you discovered that X = 15 when n = 20? What if three other studies, each with n = 20, reported the following: X 1 = 1, X 2 = 3, and X 3 = 5? 6 / 44

7 Motivation Sampling and Variability If the mean effectiveness of a blood pressure medication is estimated to be a 15 point reduction in blood pressure after a week (i.e., X = 15), does this mean that µ = 15? What if you discovered that X = 15 when n = 20? What if three other studies, each with n = 20, reported the following: X 1 = 1, X 2 = 3, and X 3 = 5? What if you discovered that X = 4.5 when n = 250, 000? 7 / 44

8 Motivation Sampling and Variability If the mean effectiveness of a blood pressure medication is estimated to be a 15 point reduction in blood pressure after a week (i.e., X = 15), does this mean that µ = 15? What if you discovered that X = 15 when n = 20? What if three other studies, each with n = 20, reported the following: X 1 = 1, X 2 = 3, and X 3 = 5? What if you discovered that X = 4.5 when n = 250, 000? The estimated mean is sample specific, but it represents a fixed population value... understanding the variability of estimates is important for making decisions based on a sample. 8 / 44

9 Sampling Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Simple random sample: when all members of a population have the same probability of being selected. 9 / 44

10 Sampling Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Simple random sample: when all members of a population have the same probability of being selected. Unless otherwise stated, we will assume simple random sampling. 10 / 44

11 Sampling Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Simple random sample: when all members of a population have the same probability of being selected. Unless otherwise stated, we will assume simple random sampling. This is not the way all samples are collected so be aware of data collection methods. 11 / 44

12 Sampling Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Simple random sample: when all members of a population have the same probability of being selected. Unless otherwise stated, we will assume simple random sampling. This is not the way all samples are collected so be aware of data collection methods. There are other sampling methods, which we will discuss at a later time. 12 / 44

13 Estimation Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Point estimate: the estimated value of a population parameter of interest based on a sample from the population of interest. 13 / 44

14 Estimation Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Point estimate: the estimated value of a population parameter of interest based on a sample from the population of interest. Point estimates are (essentially) always wrong. 14 / 44

15 Estimation Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Point estimate: the estimated value of a population parameter of interest based on a sample from the population of interest. Point estimates are (essentially) always wrong. A sample estimate will differ (usually) from the population parameter it estimates. 15 / 44

16 Estimation Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Point estimate: the estimated value of a population parameter of interest based on a sample from the population of interest. Point estimates are (essentially) always wrong. A sample estimate will differ (usually) from the population parameter it estimates. Correspondingly, if new or a different sample had been collected, a different point estimate would likely result. 16 / 44

17 Sampling Variability Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Multiple samples can be taken and a point estimate calculated in each instance. The sample-to-sample variability of the point estimates is sampling variability. Sampling error is the random difference between an estimate and the parameter it estimates. 17 / 44

18 Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Sampling Variability: A Demonstration The effects of sampling variability on a normal distribution: A Demonstration (Link). 18 / 44

19 Thought Question Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Does Variability Matter in the Market? 19 / 44

20 Thought Question Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Does Variability Matter in the Market? Yes! 20 / 44

21 Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Does Variability Matter in the Market? Volatility: the standard deviation of the continuously compounded returns of a financial instrument (over a specified period of time). Volatility is the standard deviation! Implied volatility, an annualized standard deviation. The VIX (SP 500 market volatility index). The VXN (Nasdaq 100 volatility index). The VXD (DJIA volatility index). 21 / 44

22 Financial Prospectus From Fidelity (Link).

23 Financial Prospectus From Fidelity (Link).

24 Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Does Variability Matter in the Market Example of the effect of volatility (i.e., the standard deviation) on returns Demonstration (Link). Example performance measures from Fidelity (note the standard deviation) (Link). 24 / 44

25 Sampling and Variability The Standard Deviation of the Mean Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Often, interest is not literally in individual scores, but rather the mean of a set of scores (e.g., months, groups, teams). The variability of the mean is different from the variability of scores. The mean is based on more information than individual scores and is more stable. 25 / 44

26 Sampling and Variability The Standard Deviation of the Mean Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example The population standard deviation of the mean is denoted σ x, which equals σ x = σ n. 26 / 44

27 Sampling and Variability The Standard Deviation of the Mean Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example The population standard deviation of the mean is denoted σ x, which equals σ x = σ n. Because n is in the denominator, as the sample size increases, the variability of the mean decreases. 27 / 44

28 Sampling and Variability The Standard Deviation of the Mean Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example The population standard deviation of the mean is denoted σ x, which equals σ x = σ n. Because n is in the denominator, as the sample size increases, the variability of the mean decreases. This fact is extremely important, as it shows how increasing sample size leads to more precise (i.e., less variable) estimates. Increasing sample size decreases noise and magnifies signal. 28 / 44

29 Sampling and Variability Sampling Distributions Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Sampling Distribution: the distribution of a statistic of interest when when that statistic is calculated from random samples of data under the same set of conditions. This can be for any statistic (e.g., x, p, s, s 2, r, etc.). Each statistic has a sampling distribution with its own properties. The sampling distribution always depends on sample size. The larger the sample size, the less variable the statistic. 29 / 44

30 Sampling and Variability Sampling Variability: 12 Ounce Cans Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Beverage manufacturers must ensure that the stated amount of liquid is close to the actual amount. Manufactures know that the process of filling a can is not exact (e.g., temperature, barometric pressure, viscosity). The manufacturing specifications for a bottler is such that the machines attempt to fill each can with ounces of liquid. Suppose that data shows that the standard deviation is 0.25 of ounces of liquid across the cans. 30 / 44

31 Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Recall, the Standard Normal Distribution A normal distribution with a mean of µ = 0 and a standard deviation of σ = 1 is the standard normal distribution. Such a distribution is standardized because z-scores are formed. Recall, a z-score for individual i is defined as z i = x i µ σ Any normal distribution can be converted into a standard normal distribution by transforming scores into z-scores. The distribution shape (e.g., if it is skewed) does not change by converting to z-scores (but the mean and standard deviation does). 31 / 44

32 Sampling and Variability Sampling Variability: An Example Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example What is the probability that a can has less than 12 ounces? z = x µ σ = =.2.25 =.8 Thus, P(Z.8) = / 44

33 Sampling and Variability Sampling Variability: An Example Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example What about the mean of a 6 pack being less than 12 ounces? z = x µ σ/ = n.25/ 6 =.20 = 1.96; P(Z 1.96) = / 44

34 Sampling and Variability Using Excel Instead of the z-table Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example The NORM.S.DIST formula in Excel can be used instead of the z-table. NORM.S.DIST requires one to specify the z value and TRUE (for cumulative). That is: NORM.S.DIST(z, TRUE) For the example of a can being less than 12 ounces, in which z =.80:NORM.DIST(X, 0, 1, TRUE). The formula returns: (which for our purposes can be rounded to four decimal places: P = Recall formulas in Excel require an = sign in order to return the results. Thus, =NORM.DIST(X, 0, 1, TRUE) would be entered in an Excel cell. 34 / 44

35 Sampling and Variability Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Using Excel More Generally: For Any Normal Distribution The NORM.DIST formula in Excel can be used for any normal distribution (notice there is not S, which denoted standardized). NORM.DIST requires one to specify the X value, µ, σ, and TRUE (for cumulative). That is: NORM.DIST(X, µ, σ, TRUE) For a standard normal distribution (i.e., z-distribution): NORM.DIST(X, 0, 1, TRUE) For the example of a six-pack being less than 12 ounces: NORM.DIST(12, 12.20,.25/sqrt(6), TRUE). The formula returns: (which for our purposes can be rounded to four decimal places: P = / 44

36 Sampling and Variability Sampling Variability: An Example Sampling/Variability Demonstration Standard Deviation of the Mean Sampling Variability Example Rather than using the z-table, the NORM.DIST formula in Excel can be used: 36 / 44

37 Sampling Variability Visually (Equal X -Axis) Distribution of the Weight of Individual Cans Distribution of Sample Mean for the Weight of Cans in a 6 Pack Density Density Weight of One Cans Sample Mean Weight of the Cans in a 6 Pack Distribution of Sample Mean for the Weight of Cans in a 12 Pack Distribution of Sample Mean for the Weight of Cans in a Case Density Density Sample Mean Weight of the Cans in a 12 Pack Sample Mean Weight of the Cans in a Case

38 Sampling Variability Visually (Equal X and Y Axes) Distribution of the Weight of Individual Cans Distribution of Sample Mean for the Weight of Cans in a 6 Pack Density Density Weight of One Can Sample Mean Weight of the Cans in a 6 Pack Distribution of Sample Mean for the Weight of Cans in a 12 Pack Distribution of Sample Mean for the Weight of Cans in a Case Density Density Sample Mean Weight of the Cans in a 12 Pack Sample Mean Weight of the Cans in a Case

39 Sampling and Variability Visualization 39 / 44

40 Sampling and Variability Visualization Random samples of size n drawn from some population will have a sampling distribution for x that can be approximated by a normal distribution as the sample size becomes large. 40 / 44

41 Sampling and Variability Visualization Random samples of size n drawn from some population will have a sampling distribution for x that can be approximated by a normal distribution as the sample size becomes large. Note that nothing is stated about the shape of the distribution in the population! The distribution of x will have mean µ and standard deviation σ x = σ n. This is one of the most important theorems in all of statistics! 41 / 44

42 Histogram of Uniform Distribution Histogram of Uniform Distribution Density Score

43 Sampling Distribution of X from a Uniform Distribution Sample Means from Uniform Distribution when n = 2 Sample Means from Uniform Distribution when n = 5 Density Density Value Observed for Sample Mean Value Observed for Sample Mean Sample Means from Uniform Distribution when n = 10 Sample Means from Uniform Distribution when n = 25 Density Density Value Observed for Sample Mean Value Observed for Sample Mean

44 Histogram of Log Normal Distribution Histogram of Log Normal Distribution Density Score

45 Sampling Distribution of X from Log Normal Distribution Sample Means from Log Normal Distribution When n = 2 Sample Means from Log Normal Distribution When n = 5 Density Density Sample Mean Sample Mean Sample Means from Log Normal Distribution When n = 25 Sample Means from Log Normal Distribution When n = 50 Density Density Sample Mean Sample Mean

46 Histogram of Mixed Normal Distribution Histogram of Mixed Normal Distribution Density Score

47 Sampling Distribution of X from a Mixed Normal Sample Means from Mixtured Normal Distributions When n = 2 Sample Means from Mixtured Normal Distributions When n = 5 Density Density Sample Mean Sample Mean Sample Means from Mixtured Normal Distributions When n = 10 Sample Means from Mixtured Normal Distributions When n = 25 Density Density Sample Mean Sample Mean

48 Sampling and Variability Visualization Said another way, with regards to the sampling distribution of the sample mean, the shape of the distribution from which scores are sampled is irrelevant, because as sample size grows large the sampling distribution of the sample mean will be normally distributed! Because the distribution of the sample means is normal, provided sample size is not too small, inferences about the mean can legitimately be made using a normal distribution! 48 / 44

49 Sampling and Variability The Normality Assumption Notation Glossary What You Learned A Worked Example The standard normal distribution (i.e., the z-table) assumes that the normally distributed. If the normality assumption is violated, the probabilities obtained will not be correct. If interest concerns a mean, which is often the case, the Central Limit Theorem tells us that the distribution of sample means will be normally distributed (if sample size is not too small). Thus, the z-distribution can be used when interest concerns means, regardless of the shape of the distribution of the scores. However, the z-distribution should not be used for probabilities of individual scores if the distribution from which the scores are sampled is not normal. We have also assumed that σ is known. We relax this assumptions in the next topic (by using a t-distribution instead of z). 49 / 44

50 Sampling Error Sampling and Variability Notation Glossary What You Learned A Worked Example The random discrepancy between a statistic (from a sample) and the parameter (the population value). Sampling error is the reason for inferential statistics. If there were no sampling error, the estimate from a particular sample would equal the population value. Because sample error (essentially) always exists, we need to quantify the uncertainty of an estimate. This is why confidence intervals and hypothesis tests are so important (much more to come on these procedures later). 50 / 44

51 Sampling and Variability Finite Sample Calculations Notation Glossary What You Learned A Worked Example For small populations in which the population size (N) is known, there is a correction factor that can be used when calculating the standard error of the mean. Rather than using σ x = σ n as would be typical,using ) σ x = provides a more appropriate value of the N n N 1 ( σ n standard error of the sample mean in situations in which the population is small. In general, this might be used only if the sample size is 5% or more of the population size. Usually we assume that the data come from an infinite population or process (or we acknowledge we do not know the population size), and thus this correction is not often needed. 51 / 44

52 Notation Glossary Sampling and Variability Notation Glossary What You Learned A Worked Example σ x - population standard deviation of the sample mean (σ x = σ n ) n - sample size N - population size x - the mean of a sample µ - the mean of a population z - z-score for a score of interest (z = x µ σ ) z - z-score when the sample mean is of interest (z = x µ σ/ n ) 52 / 44

53 Sampling and Variability Notation Glossary What You Learned A Worked Example What You Learned from this Lesson You learned: The idea of sampling from a population to obtain point estimates. The idea of the variability of point estimates. Understanding how the standard deviation of a mean decreases as the sample size increases (i.e., σ X = σ/ n). How to find probability values for a mean. How the Central Limit Theorem allows for normal distribution to be used when interest concerns a mean (even if the distribution of scores is not normal). How the Central Limit Theorem allows for the assumption of normality not to be met, but still allows the use of the normal distribution when finding probabilities for a mean. 53 / 44

54 A Worked Example Sampling and Variability Notation Glossary What You Learned A Worked Example From a statement made by a automotive dealership in a court proceeding, it was stated that 40% of the time they sell plus or minus 8 vehicles per month around the population mean. That is, p(µ ± 8) =.40. Assume that the statement is based on the relevant population values and the vehicle sales follow a normal distribution. What is σ, which you would find, as a competitor to this dealership, very valuable for comparison purposes? 54 / 44

55 Sampling and Variability A Worked Example, Continued Notation Glossary What You Learned A Worked Example First, we need to determine the relevant z-values based on what is known. If there is a 40% chance of selling plus/minus 8 vehicles around mean, we know can find the corresponding z-scores; 60% of the time the dealership falls outside of the limits (30% less and 30% more). There thus being 30% of the time on either size: p(z z) =.30 is for z =.524; meaning for this problem the relevant quantiles (i.e., z-values) are ±.524 (which can be found using qnorm(.30) and qnorm(.70)). The relevant z-values are thus ± / 44

56 Sampling and Variability A Worked Example, Continued Notation Glossary What You Learned A Worked Example Recalling that z = X µ σ, and from the problem we have ±.524 = ±8 σ, we are solve for σ (because we know everything else): σ = 8/.524 = Thus, we have found that the population standard deviation is around cars per month. 56 / 44

57 Sampling and Variability A Worked Example, Continued Notation Glossary What You Learned A Worked Example What is the probability of the dealership having a summer mean (i.e, of the three months of summer) that is within 10 vehicles of the overall mean? Here we use X instead of X, and thus σ X instead of σ: z = X µ σ/ n. From earlier we already know that σ = and we know the other values: z = ± / 3 = ± The z-values of interest are ± Thus, p(± 10 Cars around µ) = p(z & Z 1.135) = The above probability can be obtained from 1 2 pnorm( 1.135) or, equivalently yet more formally, 1-[pnorm(-1.135)+1-pnorm(1.135)]. 57 / 44