1 Estimating a population statistic from a sample statistic

Size: px

Start display at page:

Download "1 Estimating a population statistic from a sample statistic"

Mitchell Taylor
6 years ago
Views:

1 Political Sciece 100a/200a Fall 2001 The cetral limit theorem ad the law of large umbers 1 1 Estimatig a populatio statistic from a sample statistic Questio 1: You wat to predict the outcome of a upcomig presidetial electio, but you lack the time ad moey to iterview every sigle possible voter about his or her vote itetios. So you decide to iterview of a radom sample of registered (possible) voters. Ca you use the results of this sample to geeralize about the populatio, ad how accurate is this method likely to be? Questio 2: Suppose you wat to kow the average ad stadard deviatio of life expectacy across coutries i a particular year, but the oly way to get the data is a paiful ad difficult process of cosultig records coutry-by-coutry. You do ot have time ad research moey to do this for all coutries i a give year. What should or ca you do? Suppose you decide istead to draw a radom sample of 25 coutry ames, ad to collect life expectacy data for these. Ca you use this sample to estimate the average ad stadard deviatio of life expectacy for the populatio? Sice we have the actual data o life expectacy for (almost) the full set of coutries i 1994(?), we are i positio to evaluate how accurate ad reliable this procedure would be. So let s try it: use lifeexp, sample 16, sum, graph lifeexp. Let s preted that this is the sample of 25 coutries that you costruct. Note that the distributio of the sample values looks sort of like the distributio of the populatio values. The idea is to use this sample to make ifereces about the populatio (or radom process) from which it is draw. So use sum to get basic descriptive statistics x ad sd o the sample ad the compare to populatio values. How do they compare? I reality, we oly ever see the sample we are workig with, so we ca t kow how far our estimates are from the truth. 1 Notes by James D. Fearo, Dept. of Political Sciece, Staford Uiversity, November

2 But we ca perform a experimet with Stata to see how good is the sample mea, x, i geeral as a estimate for the populatio mea µ. We ca write a program i Stata that will 1. Draw a radom sample of 25 coutries ad compute the mea of life expectacy for the coutries i this sample. 2. Record this result i a ew data file as a observatio (a case). 3. Repeat this procedure as may times as we like, thus buildig up a data set that cosists of (say) the mea life expectacies for each of the 25 coutry samples we drew. 4. We ca the look at the distributio of these estimates based o 25-coutry samples, to see how well they match up with the true mea i the populatio. The spread (or stadard deviatio) of this distributio will give us a idicatio of the likelihood that a estimate based o a 25 coutry sample is way off. Ok? NB: Of course, i reality, you will oly ever have a sample, ad ot the populatio or direct access to the social or political process you are tryig to draw a iferece about. Here we are doig a experimet where we imagie that we have the populatio ad the see how good are estimates of the populatio are based o smaller samples. Clear? Show xmea.do, discuss... ru simul xmea, reps(.) for 10, 50, 100, 1000, replicatios. Compare the distributio of our 25-coutry sample meas to the distributio of lifeexp i the origial sample. What do you otice? Several features of this experimet tur out (ad we will show) to be etirely geeral they would occur (almost) o matter what the distributio of the populatio variable of iterest. These features are: 1. The distributio of the sample mea is cetered o the true mea i the populatio. This is called ubiasedess, ad it is oe atural criterio for a sample statistic to be a good estimate of a populatio statistic. Def : A sample statistic b is a ubiased estimator for a populatio parameter β if E(b) = β. 2. The distributio of the sample mea is approximately ormal. This is immesely useful because it allows us to easily figure how much ucertaity attaches to the particular estimate we get we make use of the stadard ormal curve. The approximate ormality of the sample mea is explaied by the cetral limit theorem, which says that the sum of a large umber of of radom variables has a approximately ormal distributio almost o matter what is the distributio of the compoet radom variables. (That is, as you add more ad more radom variables 2

3 together, the distributio of their sum more ad more closely approximates a ormal distributio.) 3. The stadard deviatio of the sample mea is smaller tha the stadard deviatio of the populatio variable. I fact, we will show that the stadard deviatio of the sample mea is σ/, where is ow the size of the sample beig draw ad σ is the sd of the populatio. (Show that this works for cases above, where = 25.) So, as we icrease the sample size, our estimate of the populatio mea becomes much more precise, ad this does t deped i ay way o the size of the populatio (discuss relevace to pollig...). 2 Explaiig the simulatio results Suppose we draw a radom sample of values x 1, x 2, x 3,..., x from some populatio (e.g., life expectacies). Recall that the sample mea is defied as x = i=1 x i = 1 x 1 + x x = x 1 + x x. The experimets above suggested how we ca thik of the sample mea x as a radom variable. How? (This is very importat to uderstad.) I ay give social sciece research problem, we oly see ONE sample mea, but we ca thik about it as though it were a draw from a distributio of possible values, i.e., a radom variable. Examples: 1. I the life expectacy example above, there were 157 some coutries with data o life expectacy i the populatio. How may possible 25-coutry samples ca be draw from this set? ( ) So let our sample space S be the set of all possible 25-coutry samples from the populatio of 157. With radom samplig all draws are equally likely. Each differet draw implies a sample mea x(s), where s S is a particular sample. Thus, the sample mea is a radom variable, a fuctio that assigs a umber to each poit i a sample space. Clear? (this is importat). Keep i mid that we oly observe oe sample, the realized value for our particular draw. 2. There are some z millio registered voters (let s suppose). We draw a radom sample of 2500 ad ask them about their vote itetios. There are ( z 2500) possible samples (a huge umber), ad these comprise our sample space. The sample mea of (say) itetio to vote for Gore (1 or 0) is a thus radom variable, x(s). 3. You wat to estimate the probability that a oddly shaped die will tur up a ace (a 1). You roll it 100 times ad use the average umber of aces (the sample mea) as a estimate. Note that here the populatio ad the sample space from which 3

4 the sample is draw is ifiite. But o matter: Here the sample mea is a radom variable i the sese of beig the result of well-defied chace process. 4. You wat to estimate the probability that a 40-year-old smoker has cacer, ad to do so you draw a radom sample of 40-year-old smokers. What is the populatio? All curret 40-yr-old smokers? All curret ad future? If we ca thik of the sample mea x as a radom variable, we ca also thik about its expectatio or expected value. What is E( x)? E( x) = E( x 1 + x x ). But each of these particular values, x 1, x 2,..., is a draw from the populatio, i.e., a sample of size 1. So we ca treat each particular value as a particular realizatio of the radom variable that is the value of oe draw from the populatio. Call this radom variable X i for the ith draw. Next, what is the expectatio of a sum of radom variables multiplied by a costat? E( x) = 1 (E(X 1) + E(X 2 ) E(X )). But what is E(X i )? What is the expected value of a sigle draw from the populatio? Just the expectatio of the radom variable X, or E(X) = µ, the populatio mea. Thus, E( x) = 1 (E(X 1) + E(X 2 ) E(X )) = 1 µ = µ. This explais oe of the simulatio results above: the expectatio of the sample mea is the populatio mea, or i other words, the distributio of the sample mea is cetered o the true, uderlyig mea of the populatio. I other words still: the sample mea is a ubiased estimate of the populatio mea. Notice that this argumet did ot deped i ay way o the particular distributio of values i the populatio (the radom variable X). So thikig about the sample mea x as a radom variable we have established that E( x) = µ. We ca also ask about the variace ad stadard deviatio of this radom variable: var( x) = var( X 1+X X ) = var( X 1 + X X ) i=1 = 1 2 var(x 1 ) var(x 2 ) var(x ) 4

5 Be sure you uderstad what happeed i the last step. Two properties of the variace of a radom variable were used here. What were they? Why could we take variace through i this maer? What is var(x i )? Each idividual value is a draw from the populatio box, so it has variace equal to the populatio variace σ 2. Thus... var( x) = 1 2 var(x 1 ) var(x 2 ) var(x ) = 1 2 σ σ σ 2 = 1 2 i=1 σ2 = 1 2 σ 2 = σ2. So (very importat), we coclude that the variace of the sample mea x is σ 2 /, ad the stadard deviatio of the sample mea is σ/. illustrate that this is approximately correct with simulatio results from Stata... So what happes to the stadard deviatio (dispersio) of the sample mea as the size of the sample () gets larger? Why is this happeig? What is the ituitio? Friedma has a ice way of thikig about it i terms of the sum of draws from box model (i.e., radom variable), istead of the mea (average) of the draws: The expectatio of the sum is E(X 1 + X X ) = µ. The variace of the sum of draws is var(x 1 +X 2 + +X ) = var(x 1 )+var(x 2 ) var(x ) = σ 2. So the variace of the sum of the draws icreases liearly with. But stadard deviatio is the square root of variace, so the stadard deviatio of the sum of the draws (which Friedma calls the stadard error ) is σ. Thus the stadard error of the sum of the draws icreases oly with the square root of the umber of draws. Ad accordigly, the stadard deviatio of the average of the sum of the draws is σ/ = σ/, as show above. e.g.: flip a coi 100 times ad cout the umber of heads. The expected umber is 50, ad the stadard error is 100 p(1 p) = = 5. But the expected percetage of heads is thus 50/100 = 1/2, ad the stadard error as a percetage of the umber of flips is 5/100 =.05. We could have gotte the last umber directly from σ/ =.5.5/ 100 =.5/10 =.05. Ok? 5

6 Problem-solvig ote: Sometimes a questio/problem requires you to thik about the sum of a buch of radom draws, ad sometimes about the mea. Note the differece. This last example is of a form that is worth developig i the more geeral case: May problems take the form of estimatig a percetage or proportio. e.g.: Public opiio pollig o vote itetios try to estimate the proportio of voters who will vote for Gore, Bush, Nader ad Buchaa. Schematically, we are tryig to estimate the mea ad sd of the proportio of 1 s i a box with a ukow umber of 1 s ad 0 s i it. From the above geeral results, we have for a sample of size, E( x) = µ var( x) = σ 2 / = p(1 p) sd( x) = p(1 p). e.g.: Suppose that the truth is that 48% of those who will vote will vote for Gore, ad 52% for someoe else. What is the stadard error of the estimate of this populatio proportio if we have a radom sample of 100 voters? 1000? 2500? 10,000? ,000 s.e..48(.52)/100 =.05.48(.52)/1000 = (.52)/2500 =.01.48(.52)/2500 =.005 Be sure to read the FPP chapters o the Gallup poll ad the uemploymet statistics to get a sese of the various obstacles that make the reality of doig this much trickier (though the theory ad core ideas behid it are i the above). Very importat poit to uderstad: Notice that i the above aalysis the accuracy of the estimates (as idicated by the stadard error of the sample proportio above) depeds o the sample size, but ot the size of the populatio. Why is this? See FPP chapter 20, part 4 for a very ice expositio. Ituitio says that if we sample 2500 voters i New Mexico ad 2500 i Texas, our results i New Mexico should be much more accurate because we sampled 1 i 500 voters there ad oly 1 i about 5,000 i Texas. But this is a ituitio that is just wrog. 6

7 Whe the size of the sample is small relative to the size of the populatio (usually the case i survey research), what matters is the size of the sample. Compare to estimatig the amout of salt dissolved i a cotaier of water, by samplig a cubic cetimeter worth obviously it wo t matter whether the cubic cetimeter is draw from a cup or a jug, if the solutio is well mixed. Whe the size of the sample is large relative to the size of the populatio, we do eed a correctio factor to get a good estimate of the stadard error: sd( x) = σ N N 1. The reaso is that i samplig without replacemet from the populatio, each draw leaves the compositio of the box slightly differet, ad thus chages the probability of gettig a 1 o the ext draw. These differeces are trivial whe there are may tickets i the box ad the sample is small. (Note what happes above whe N is large relative to.) Illustrate with Stata results above... 3 Iterlude: the law of large umbers Recall that the frequetist defiitio of probability wet like this: The probability of a evet is the relative frequecy that the evet would occur if the experimet or trial were ru may, may times, over ad over agai. We are ow i a positio to show that this defiitio at least has a certai iteral, mathematical coherece. Th m : (The Law of Large Numbers.) Let X 1, X 2,..., X be a sequece of idepedet radom variables with E(X i ) = µ ad var(x i ) = σ 2. Let X = 1 i=1 X i. The for ay umber ɛ > 0, P ( X µ > ɛ) 0 as. Explai i words: As the umber of draws from the box gets very large, the probability that average of the draws (the sample mea) is farther tha ɛ away from the average of the umbers i the box (the true mea) gets very, very small. Illustrate with Stata: toss a coi 1000 times, compute the ruig percetage of heads, ad plot the ruig percetage agaist the umber of tosses. set obs 1000, ge =, ge x = uiform() >.5, ge heads = sum(x), ge pctheads = heads/ graph pctheads, s(.) ylie(.5) yla(0,.5,1) ti(pct heads i tosses). Where is this comig from? Ca already kid of see it from the result that the stadard deviatio of the sample mea x is σ/, which approaches zero as gets large. But this 7

8 does t say aythig directly yet about the probability of gettig a result far from the sample mea, sice we have t said aythig yet about the distributio of the sample mea. I fact we do t eve eed to: We ca prove the theorem usig Chebyshev s iequality: 1. Chebyshev s iequality said that for ay X, at least 1 1/k 2 percet of the observatios lie withi k s.d. s of the mea, X. 2. This implies that that for ay X, at most 1/k 2 percet of the observatios lie outside of k s.d. s from the mea, X. 3. Restated i radom variable terms, this meas that P ( X µ > kσ) 1/k 2. e.g., for ay radom variable, the probability of drawig a value at least two s.d.s from the mea is at most 1/4. (3 s.d.s, at most 1/9, etc.) 4. Look back at the Law of Large Numbers: We eed a ɛ i the P (..) part. So let s try lettig ɛ = kσ. This implies that k = ɛ/σ. So we ca write the iequality i (3) as P ( X µ > ɛ) 1/(ɛ/σ) 2 = σ 2 /ɛ SO, if we are cosiderig the sample mea X, we have P ( X µ > ɛ) σ 2 X /ɛ 2 = σ2 ɛ Ad what happes to this as gets very large? Goes to zero. Review that this proves the claim. It is NOT crucial that you uderstad the logic or math of this proof. It is importat that you uderstad the idea or ituitio of the law of large umbers, which FPP calls the law of averages. Oe way to put is this: As oe s sample gets large, the probability that the sample mea is very differet from the mea of the populatio gets very small. Aother way to put is this: If a evet occurs with probability p, the the proportio of times the evet occurs i trials will approach p with a very high degree of cofidece as gets very large. Oe further cocept illustrated i the law of large umbers: Cosider some statistic that ca be computed for a sample of size, ot ecessarily the sample mea. Call it Z. Let α be the parameter value that Z is tryig to estimate (like the populatio mea re the sample mea). The methods types say that Def : Z coverges i probability to α if it is true that P ( Z α < ɛ) approaches zero as gets very large, for ay ɛ > 0. 8

9 e.g.: Suppose you wat to kow what the percetage of owhites is i the U.S. cogressioal district that has the largest such percetage. (Assume this data is t readily available). Suppose you draw a sample of Cogressioal districts ad use the maximum value for the sample as your estimator (Z = max{x 1, x 2,..., x }). Is this estimator biased or ubiased? (i.e., will it be systematically differet from the true value) Does it coverge i probability to the true value as the sample size grows? If ad whe you get to more advaced statistical procedures such as regressio aalysis with a dichotomous depedet variable (e.g., war or ot, democracy or odemocracy, etc.), you will ecouter may estimators that are ot ubiased like the sample mea, but which do have the weaker ice property of covergig i probability to the right value. 4 Retur to problem of estimatig a populatio statistic from a sample statistic We saw i the simulatios that the distributio of the sample mea looked approximately ormal. This is ot a accidet. I the first part of the 18th cetury, de Moivre showed that the distributio of the umber of heads i tosses of a coi that lads heads with probability p is approximately Normal as gets large, with mea equal to p ad variace p(1 p). Subsequetly it was realized that this was a reflectio of a much more geeral truth, which is depicted i a variety of versios of cetral limit theorems. Here is a loose versio of a very geeral oe: Th m : Suppose that radom variables X 1, X 2,..., X are idepedet ad that some techical coditios (o the third cetral momet) hold. The the sum of these radom variables has a distributio that is approximately ormal with the approximatio gettig very good as gets large. Here is a more particular versio, cocerig the case of the sum of a sequece of radom variables that all have the same probability distributio. Th m : Let X i be radom variable with mea µ ad variace σ 2. Let Z = X 1 + X X be the radom variable formed by addig together a sequece of idepedet draws of X i. The as grows large, Z a N(µ, σ 2 ) ( a reads has a distributio that is approximately....) 9

10 Notice that this theorem says othig about how the radom variables that are the compoets of the sum are distributed themselves this does t matter, which is kid of amazig. It is true, though, that how rapidly the covergece to Normality takes place depeds o the shape of the distributios of the compoets of the sum. The more ormal lookig they are, the more rapid the covergece. How rapid is it? Illustrate with sum of 12 uiform distributios i Stata: set obs 1000, for um 1/12: ge xx = uiform(), ge z = 0, for um 1/12: ge z = z + xx, sum, graph z,bi(25) ormal. 12 is t very may, a uiform distributio is t very ormal, but look at that. By cotrast, do same but with for um 1/12: ge xx = uiform() >.9. The geeral poit is this: How rapid the covergece is depeds o how Normal lookig are the distributio of the variables that are compoets of the sum. The uiform distributio has fatter tails tha the ormal distributio, but is symmetric. The skewed Beroulli distributios are both asymmetric ad ot distributed with much mass at the ceter. How does the cetral limit theorem bear o the distributio of the sample mea? Recall that the sample mea is based o a sum of radom variables (i.e., draws from the populatio box ). This is why the sample mea has a distributio that is approximately ormal. This implies our mai result: Th m : If the radom variables (x 1, x 2, x 3,..., x ) form a radom sample of size from a give distributio with mea µ ad variace σ 2 <, the as the size of the sample grows large, the sample mea x a N(µ, σ 2 /). I other words, the sample mea is a ubiased estimator for the populatio mea, ad has a approximately ormal distributio with variace σ 2 /. Recall (from properties of Normal distributios) that the sum Z of two ormally distributed radom variables X ad Y has a ormal distributio also. This implies further that: Th m : If a radom variable X has a ormal distributio, the the distributio of the sample mea x for ay sized sample from X is exactly ormal. Ca you see why? The cetral limit theorem helps explai why so may quatities i ature ad eve i social sciece have a approximately ormal distributio: I brief, ay variable whose 10

11 value is produced by the sum of a fairly large umber of other factors or iflueces should have a approximately ormal distributio. e.g.: legth of tree leaves, heights ad weights of people (see FPP), measuremet errors (for repeated scietific measuremets) of all sorts, golfers 18 scores, total weeked receipts at a restaurat or other store, total targets hit by US bombers i Afghaista by week, etc... I short, whe you see a variable i social sciece that has a roughly ormal distributio, it is good ad potetially fruitful idea to ask if you ca imagie it beig produced as the sum of umber of other factors or compoets. 11

Understanding Samples

Understanding Samples 1 Will Moroe CS 109 Samplig ad Bootstrappig Lecture Notes #17 August 2, 2017 Based o a hadout by Chris Piech I this chapter we are goig to talk about statistics calculated o samples from a populatio. We