Discrete Mathematics for CS Spring 2008 David Wagner Note 22
|
|
- Bennett Tucker
- 5 years ago
- Views:
Transcription
1 CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig a small radom sample. How large does our sample have to be to guaratee that our estimate will be withi (say) ±1 percetage poits (i absolute terms) of the true value with probability at least 0.95? This is perhaps the most basic statistical estimatio problem, ad it shows up everywhere. We will develop a simple solutio that uses oly Chebyshev s iequality. More refied methods ca be used to get sharper results. Let s deote the size of our sample by (to be determied), ad the umber of Democrats i it by the radom variable S. (The subscript just remids us that the r.v. depeds o the size of the sample.) The our estimate will be the value A = 1 S. Now as has ofte bee the case, we will fid it helpful to write S = X 1 + X 2 + +X, where { 1 if perso i i sample is a Democrat; X i = 0 otherwise. Note that each X i ca be viewed as a coi toss, with Heads probability p (though of course we do ot kow the value of p). Ad the coi tosses are idepedet. 1 [We ca say that the X i s are idepedet ad idetically distributed, or just i.i.d. for short.] What is the expectatio of our estimate? E[A ] = E[ 1 S ] = 1 E[X 1 + X 2 + +X ] = 1 (p) = p. So for ay value of, our estimate will always have the correct expectatio p. [Such a r.v. is ofte called a ubiased estimator of p.] Now presumably, as we icrease our sample size, our estimate should get more ad more accurate. This will show up i the fact that the variace decreases with : i.e., as icreases, the probability that we are far from the mea p will get smaller. To see this, we eed to compute Var(A ). Ad sice A = 1 i=1 X i, we eed to figure out how to compute the variace of a sum of radom variables. Theorem 22.1: For ay radom variable X ad costat c, we have Var(cX) = c 2 Var(X). Ad for idepedet radom variables X,Y, we have Var(X +Y) = Var(X)+Var(Y). 1 We are assumig here that the samplig is doe with replacemet ; i.e., we select each perso i the sample from the etire populatio, icludig those we have already picked. So there is a small chace that we will pick the same perso twice. CS 70, Sprig 2008, Note 22 1
2 Proof: From the defiitio of variace, we have Var(cX) = E[(cX E[cX]) 2 ] = E[(cX ce[x]) 2 ] = E[c 2 (X E[X]) 2 ] = c 2 Var(X). The proof of the secod claim is left as a exercise. Note that the secod claim does ot i geeral hold uless X ad Y are idepedet. Usig this theorem, we ca ow compute Var(A ). Var(A ) = Var( 1 i=1 X i ) = ( 1 )2 Var( i=1 X i ) = ( 1 )2 Var(X i ) = σ 2 i=1, where we have writte σ 2 for the variace of each of the X i, i.e., σ 2 = Var(X i ). So we see that the variace of A decreases liearly with. This fact esures that, as we take larger ad larger sample sizes, the probability that we deviate much from the expectatio p gets smaller ad smaller. Let s ow use Chebyshev s iequality to figure out how large has to be to esure a specified accuracy i our estimate of the proportio of Democrats p. A atural way to measure this is for us to specify two parameters, ε ad δ, both i the rage (0,1). The parameter ε cotrols the error we are prepared to tolerate i our estimate, ad δ cotrols the cofidece we wat to have i our estimate. A more precise versio of our origial questio is the the followig: Questio: For the Democrat-estimatio problem above, how large does the sample size have to be i order to esure that Pr[ A p ε] δ? I our origial questio, we had ε = 0.01 ad δ = 0.05: we wated to kow how large eeds to be so that Pr[p 0.01 < A < p ] 0.95, which is equivalet to askig how large eeds to be so that Pr[ A p 0.01] Notice that, i this example, ε measures the absolute error, i.e., the differece betwee the estimate A ad the true value p. I may applicatios, the relative error is a better measure of error, but i the case of pollig, it s usually eough to boud the absolute error. 2 Let s apply Chebyshev s iequality to aswer our more precise questio above. Sice we kow Var(A ), this will be quite simple. From Chebyshev s iequality, we have Pr[ A p ε] Var(A ) ε 2 = σ 2 ε 2. To make this less tha the desired value δ, we eed to set σ 2 ε 2 δ. (1) 2 I may other applicatios, the absolute error is a poor measure, because a give absolute error (say, ±0.01) might be quite small i the cotext of measurig a large value like p = 0.5, but very large whe measurig a small value like p = For this reaso, i most real-life applicatios, it is more useful to examie the relative error, i.e., to measure the error as a ratio of the target value p. (Thus the absolute error of the estimate A is A p, while the relative error is A p /p = A p 1.) The relative error has the advatage of treatig all values of p equally. However, pollig is a special case where it is ofte sufficiet to use the absolute error, ad the mathematics for the absolute error is slightly simpler, so we will cotiue to use the absolute error i this example, cofidet i the kowledge that we could modify these calculatios to use a relative error measure if we wated. CS 70, Sprig 2008, Note 22 2
3 Now recall that σ 2 = Var(X i ) is the variace of a sigle sample X i. Sice X i is a 0/1-valued r.v., we have σ 2 = p(1 p). It is easy to see usig a bit of calculus that p(1 p) 1 4 (sice 0 p 1). As a result, iequality (1) becomes 1 4ε 2 δ. Pluggig i ε = 0.01 ad δ = 0.05, we see that a sample size of = 50,000 is sufficiet. Oe amazig corollary is that ecessary sample size depeds oly upo the desired margi of error (ε) ad cofidece level (δ), but ot o the size of the uderlyig populatio. We could be pollig the state of Wyomig, the state of Califoria, the whole of the US, or the etire world ad the same sample size is sufficiet. This is perhaps a bit couterituitive. Estimatig a geeral expectatio What if we wated to estimate somethig a little more complex tha the proportio of Democrats i the populatio, such as the average wealth of people i the US? The we could use exactly the same scheme as above, except that ow the r.v. X i is the wealth of the ith perso i our sample. Clearly E[X i ] = µ, the average wealth of people i the US (which is what we are tryig to estimate). Ad our estimate will agai be A = 1 i=1 X i, for a suitably chose sample size. Oce agai the X i are i.i.d. radom variables, so we agai have E[A ] = µ ad Var(A ) = σ 2, where σ 2 = Var(X i ) is the variace of the X i. (Recall that the oly facts we used about the X i was that they were idepedet ad had the same distributio. Actually it would be eough for them to be idepedet ad all have the same expectatio ad variace do you see why?) I this case, we probably wat to use the relative error: we wat to choose to esure that Pr[(1 ε)µ < A < (1+ε)µ] 1 δ], i.e., to esure that Pr[ A µ εµ] δ. Applyig Chebyshev s iequality much as before, we fid Pr[ A µ εµ] Var(A ) (εµ) 2 = σ 2 ε 2 µ 2. Hece it is eough for the sample size to satisfy (2) σ 2 µ 2 1 ε 2 δ. (3) Here ε ad δ are the desired error ad cofidece respectively, as before. Now of course we do t kow the other two quatities, µ ad σ 2, appearig i equatio (3). I practice, we would try to fid some reasoable lower boud o µ ad some reasoable upper boud o σ 2 (just as we used a upper boud o p(1 p) i the Democrats problem). Pluggig these bouds ito equatio (3) will esure that our sample size is large eough. For example, i the average wealth problem we could probably safely take µ to be at least (say) $20k (probably more). However, the existece of people such as Bill Gates meas that we would eed to take a very high value for the variace σ 2. Ideed, if there is at least oe idividual with wealth $50 billio, the assumig a relatively small value of µ meas that the variace must be at least about ( ) 2 = (Check this.) However, this idividual s cotributio to the mea is oly = 200. There is really o way aroud this problem with simple uiform samplig: the ueve distributio of wealth meas that the variace is iheretly very large, ad we will eed a huge umber of samples before we are likely to fid aybody who is immesely wealthy. But if we do t iclude such people i our sample, the our estimate will be way too low. CS 70, Sprig 2008, Note 22 3
4 As a further example, suppose we are tryig to estimate the average rate of emissio from a radioactive source, ad we are willig to assume that the emissios follow a Poisso distributio with some ukow parameter λ of course, this λ is precisely the expectatio we are tryig to estimate. Now i this case we have µ = λ ad also σ 2 = λ (see the previous lecture otes). So σ 2 = 1 µ 2 λ. Thus i this case a sample size of = 1 suffices. (Agai, i practice we would use a lower boud o λ.) λε 2 δ The Law of Large Numbers The estimatio method we used i the previous two sectios is based o a priciple that we accept as part of everyday life: amely, the Law of Large Numbers. This asserts that, if we observe some radom variable may times, ad take the average of the observatios, the this average will coverge to a sigle value, which is of course the expectatio of the radom variable. I other words, averagig teds to smooth out ay large fluctuatios, ad the more averagig we do the better the smoothig. Theorem 22.2: [Law of Large Numbers] Let X 1,X 2,...,X be i.i.d. radom variables with commo expectatio µ = E[X i ]. Defie A = 1 i=1 X i. The for ay α > 0, we have Pr[ A µ α] 0 as. We will ot prove this theorem here. Notice that it says that the probability of ay deviatio α from the mea, however small, teds to zero as the umber of observatios i our average teds to ifiity. Thus by takig large eough, we ca make the probability of ay give deviatio as small as we like. [Note, however, that the Law of Large Numbers does ot say aythig about how large has to be to achieve a certai accuracy. For that, we eed Chebyshev s iequality or some other quatitative tool.] Actually we ca say somethig much stroger tha the Law of Large Numbers: amely, the distributio of the sample average A, for large eough, looks like a bell-shaped curve cetered about the mea µ. The width of this curve decreases with, so it approaches a sharp spike at µ. This fact is kow as the Cetral Limit Theorem. To say this precisely, we eed to defie the bell-shaped curve. This is the so-called Normal distributio, ad it is the first (ad oly) o-discrete distributio we will meet i this course. For radom variables that take o cotiuous real values, it o loger makes sese to talk about Pr[X = a]. As a example, cosider a r.v. X that has the uiform distributio o the cotiuous iterval [0,1]. The for ay sigle poit 0 a 1, we have Pr[X = a] = 0. However, clearly it is the case that, for example, Pr[ 1 4 X 3 4 ] = 1 2. So i place of poit probabilities Pr[X = a], we eed a differet otio of distributio for cotiuous radom variables. Defiitio 22.1 (desity fuctio): For a real-valued r.v. X, a real-valued fuctio f(x) is called a (probability) desity fuctio for X if Pr[X a] = a f(x)dx. Thus we ca thik of f(x) as defiig a curve, such that the area uder the curve betwee poits x = a ad x = b is precisely Pr[a X b]. Note that we must always have f(x)dx = 1. (Do you see why?) As a example, for the uiform distributio o [0,1] the desity would be 0 for x < 0; f(x) = 1 for 0 x 1; 0 for x > 1. [Check you agree with this. What would be the desity for the uiform distributio o [ 1,1]?] CS 70, Sprig 2008, Note 22 4
5 Expectatios of cotiuous r.v. s are computed i a aalagous way to those for discrete r.v. s, except that we use itegrals istead of summatios. Thus Ad also E[X] = x f(x)dx. Var(X) = E[X 2 ] (E[X]) 2, where E[X 2 ] = x2 f(x)dx. [You should check that, for the uiform distributio o [0,1], the expectatio is 1 2 Now we are i a positio to defie the Normal distributio. Defiitio 22.2 (Normal distributio): distributio with desity fuctio f(x) = 1 σ 2π e (x µ)2 /2σ 2. ad the variace is 1 12.] The Normal distributio with mea µ ad variace σ 2 is the 1 [The costat factor σ comes from the fact that 2π e (x µ)2 /σ 2 dx = σ 2π. So, we have to ormalize by this costat factor to esure that f(x)dx = 1. If you like calculus, you might like to do the itegrals to check that the expectatio x f(x)dx is ideed µ ad that the variace is ideed σ 2.] If you plot the above desity fuctio f(x), you will see that it is a symmetrical bell-shaped curve cetered aroud the mea µ. Its height ad width are determied by the stadard deviatio σ as follows: 50% of the mass is cotaied i the iterval of width 0.67σ either side of the mea, ad 99.7% i the iterval of width 3σ either side of the mea. (Note that, to get the correct scale, deviatios are o the order of σ rather tha σ 2.) Put aother way, if we sample a radom value from a Normal distributio, the 50% of the time, the value we get will be withi 0.67 stadard deviatios of the mea (i.e., i the rage [µ 0.67σ, µ σ]); ad 99.7% of the time, it will be withi 3 stadard deviatios of the mea. Now we are i a positio to state the Cetral Limit Theorem. Because our treatmet of cotiuous distributios has bee rather sketchy, we shall be cotet with a rather imprecise statemet. This ca be made completely rigorous without too much extra effort. Theorem 22.3: [Cetral Limit Theorem] Let X 1,X 2,...,X be i.i.d. radom variables with commo expectatio µ = E[X i ] ad variace σ 2 = Var(X i ) (both assumed to be fiite). Defie A = 1 i=1 X i. The as, the distributio of A approaches the Normal distributio with mea µ ad variace σ 2. Note that the variace is σ 2 of as icreases. (as we would expect) so the width of the bell-shaped curve decreases by a factor The Cetral Limit Theorem is actually a very strikig fact. What it says is the followig. If we take a average of observatios of absolutely ay r.v. X, the the distributio of that average will be a bell-shaped curve cetered at µ = E[X]. Thus all trace of the distributio of X disappears as gets large: all distributios, o matter how complex, 3 look like the Normal distributio whe they are averaged. The oly effect of the origial distributio is through the variace σ 2, which determies the width of the curve for a give value of, ad hece the rate at which the curve shriks to a spike. Oe useful cosequece is that the Biomial distributio ca be approximated by the Normal distributio, sice the Biomial(, p) distributio is obtaied as the sum of i.i.d. (0/1-valued) radom variables. I particular, if we hold p fixed ad let X be a radom variable with the distributio X Biomial(, p), the 3 We do eed to assume that the mea ad variace of X are fiite. CS 70, Sprig 2008, Note 22 5
6 as, 1 X approaches the Normal distributio with mea p ad variace p(1 p). This meas that if 1 is sufficietly large, we ca approximate the r.v. X as a Normal distributio (with mea p ad variace p(1 p) ). Or, i other words, if is sufficietly large, we ca approximate the r.v. X as a Normal distributio with mea p ad variace p(1 p). This meas that, for large, the Biomial distributio ca ofte be approximated as a Normal distributio. This ca be a helpful tool for approximate computatios about Biomially distributed radom variables. What is amazig about the Cetral Limit Theorem is it shows that the same kid of approximatios apply ot oly to sums of 0/1-valued radom values (i.e., ot oly to the Biomial distributio) but also to sums of ay other kid of i.i.d. r.v.s, as log as is sufficietly large. CS 70, Sprig 2008, Note 22 6
Discrete Mathematics and Probability Theory Spring 2014 Anant Sahai Lecture 16
EECS 70 Discrete Mathematics ad Probability Theory Sprig 2014 Aat Sahai Lecture 16 Variace Questio: Let us retur oce agai to the questio of how may heads i a typical sequece of coi flips. Recall that we
More informationStatistics 511 Additional Materials
Cofidece Itervals o mu Statistics 511 Additioal Materials This topic officially moves us from probability to statistics. We begi to discuss makig ifereces about the populatio. Oe way to differetiate probability
More information7-1. Chapter 4. Part I. Sampling Distributions and Confidence Intervals
7-1 Chapter 4 Part I. Samplig Distributios ad Cofidece Itervals 1 7- Sectio 1. Samplig Distributio 7-3 Usig Statistics Statistical Iferece: Predict ad forecast values of populatio parameters... Test hypotheses
More informationn outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n,
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 9 Variace Questio: At each time step, I flip a fair coi. If it comes up Heads, I walk oe step to the right; if it comes up Tails, I walk oe
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More informationHypothesis Testing. Evaluation of Performance of Learned h. Issues. Trade-off Between Bias and Variance
Hypothesis Testig Empirically evaluatig accuracy of hypotheses: importat activity i ML. Three questios: Give observed accuracy over a sample set, how well does this estimate apply over additioal samples?
More informationExpectation and Variance of a random variable
Chapter 11 Expectatio ad Variace of a radom variable The aim of this lecture is to defie ad itroduce mathematical Expectatio ad variace of a fuctio of discrete & cotiuous radom variables ad the distributio
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationStatistical Inference (Chapter 10) Statistical inference = learn about a population based on the information provided by a sample.
Statistical Iferece (Chapter 10) Statistical iferece = lear about a populatio based o the iformatio provided by a sample. Populatio: The set of all values of a radom variable X of iterest. Characterized
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationFACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING. Lectures
FACULTY OF MATHEMATICAL STUDIES MATHEMATICS FOR PART I ENGINEERING Lectures MODULE 5 STATISTICS II. Mea ad stadard error of sample data. Biomial distributio. Normal distributio 4. Samplig 5. Cofidece itervals
More informationDiscrete Mathematics for CS Spring 2005 Clancy/Wagner Notes 21. Some Important Distributions
CS 70 Discrete Mathematics for CS Sprig 2005 Clacy/Wager Notes 21 Some Importat Distributios Questio: A biased coi with Heads probability p is tossed repeatedly util the first Head appears. What is the
More informationApril 18, 2017 CONFIDENCE INTERVALS AND HYPOTHESIS TESTING, UNDERGRADUATE MATH 526 STYLE
April 18, 2017 CONFIDENCE INTERVALS AND HYPOTHESIS TESTING, UNDERGRADUATE MATH 526 STYLE TERRY SOO Abstract These otes are adapted from whe I taught Math 526 ad meat to give a quick itroductio to cofidece
More informationFrequentist Inference
Frequetist Iferece The topics of the ext three sectios are useful applicatios of the Cetral Limit Theorem. Without kowig aythig about the uderlyig distributio of a sequece of radom variables {X i }, for
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationDiscrete Mathematics and Probability Theory Spring 2016 Rao and Walrand Note 19
CS 70 Discrete Mathematics ad Probability Theory Sprig 2016 Rao ad Walrad Note 19 Some Importat Distributios Recall our basic probabilistic experimet of tossig a biased coi times. This is a very simple
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationEstimation of a population proportion March 23,
1 Social Studies 201 Notes for March 23, 2005 Estimatio of a populatio proportio Sectio 8.5, p. 521. For the most part, we have dealt with meas ad stadard deviatios this semester. This sectio of the otes
More informationChapter 8: Estimating with Confidence
Chapter 8: Estimatig with Cofidece Sectio 8.2 The Practice of Statistics, 4 th editio For AP* STARNES, YATES, MOORE Chapter 8 Estimatig with Cofidece 8.1 Cofidece Itervals: The Basics 8.2 8.3 Estimatig
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationStatisticians use the word population to refer the total number of (potential) observations under consideration
6 Samplig Distributios Statisticias use the word populatio to refer the total umber of (potetial) observatios uder cosideratio The populatio is just the set of all possible outcomes i our sample space
More information1 Inferential Methods for Correlation and Regression Analysis
1 Iferetial Methods for Correlatio ad Regressio Aalysis I the chapter o Correlatio ad Regressio Aalysis tools for describig bivariate cotiuous data were itroduced. The sample Pearso Correlatio Coefficiet
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationOutput Analysis and Run-Length Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More informationResampling Methods. X (1/2), i.e., Pr (X i m) = 1/2. We order the data: X (1) X (2) X (n). Define the sample median: ( n.
Jauary 1, 2019 Resamplig Methods Motivatio We have so may estimators with the property θ θ d N 0, σ 2 We ca also write θ a N θ, σ 2 /, where a meas approximately distributed as Oce we have a cosistet estimator
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationBinomial Distribution
0.0 0.5 1.0 1.5 2.0 2.5 3.0 0 1 2 3 4 5 6 7 0.0 0.5 1.0 1.5 2.0 2.5 3.0 Overview Example: coi tossed three times Defiitio Formula Recall that a r.v. is discrete if there are either a fiite umber of possible
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationDiscrete Mathematics and Probability Theory Summer 2014 James Cook Note 15
CS 70 Discrete Mathematics ad Probability Theory Summer 2014 James Cook Note 15 Some Importat Distributios I this ote we will itroduce three importat probability distributios that are widely used to model
More information2 Definition of Variance and the obvious guess
1 Estimatig Variace Statistics - Math 410, 11/7/011 Oe of the mai themes of this course is to estimate the mea µ of some variable X of a populatio. We typically do this by collectig a sample of idividuals
More informationDS 100: Principles and Techniques of Data Science Date: April 13, Discussion #10
DS 00: Priciples ad Techiques of Data Sciece Date: April 3, 208 Name: Hypothesis Testig Discussio #0. Defie these terms below as they relate to hypothesis testig. a) Data Geeratio Model: Solutio: A set
More informationEstimation for Complete Data
Estimatio for Complete Data complete data: there is o loss of iformatio durig study. complete idividual complete data= grouped data A complete idividual data is the oe i which the complete iformatio of
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationUnderstanding Samples
1 Will Moroe CS 109 Samplig ad Bootstrappig Lecture Notes #17 August 2, 2017 Based o a hadout by Chris Piech I this chapter we are goig to talk about statistics calculated o samples from a populatio. We
More informationTopic 6 Sampling, hypothesis testing, and the central limit theorem
CSE 103: Probability ad statistics Fall 2010 Topic 6 Samplig, hypothesis testig, ad the cetral limit theorem 61 The biomial distributio Let X be the umberofheadswhe acoiofbiaspistossedtimes The distributio
More informationStat 421-SP2012 Interval Estimation Section
Stat 41-SP01 Iterval Estimatio Sectio 11.1-11. We ow uderstad (Chapter 10) how to fid poit estimators of a ukow parameter. o However, a poit estimate does ot provide ay iformatio about the ucertaity (possible
More informationSimulation. Two Rule For Inverting A Distribution Function
Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationBig Picture. 5. Data, Estimates, and Models: quantifying the accuracy of estimates.
5. Data, Estimates, ad Models: quatifyig the accuracy of estimates. 5. Estimatig a Normal Mea 5.2 The Distributio of the Normal Sample Mea 5.3 Normal data, cofidece iterval for, kow 5.4 Normal data, cofidece
More informationMATH/STAT 352: Lecture 15
MATH/STAT 352: Lecture 15 Sectios 5.2 ad 5.3. Large sample CI for a proportio ad small sample CI for a mea. 1 5.2: Cofidece Iterval for a Proportio Estimatig proportio of successes i a biomial experimet
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationCHAPTER 8 FUNDAMENTAL SAMPLING DISTRIBUTIONS AND DATA DESCRIPTIONS. 8.1 Random Sampling. 8.2 Some Important Statistics
CHAPTER 8 FUNDAMENTAL SAMPLING DISTRIBUTIONS AND DATA DESCRIPTIONS 8.1 Radom Samplig The basic idea of the statistical iferece is that we are allowed to draw ifereces or coclusios about a populatio based
More informationLecture 2: Monte Carlo Simulation
STAT/Q SCI 43: Itroductio to Resamplig ethods Sprig 27 Istructor: Ye-Chi Che Lecture 2: ote Carlo Simulatio 2 ote Carlo Itegratio Assume we wat to evaluate the followig itegratio: e x3 dx What ca we do?
More informationDiscrete Mathematics and Probability Theory Spring 2012 Alistair Sinclair Note 15
CS 70 Discrete Mathematics ad Probability Theory Sprig 2012 Alistair Siclair Note 15 Some Importat Distributios The first importat distributio we leared about i the last Lecture Note is the biomial distributio
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationSTA Learning Objectives. Population Proportions. Module 10 Comparing Two Proportions. Upon completing this module, you should be able to:
STA 2023 Module 10 Comparig Two Proportios Learig Objectives Upo completig this module, you should be able to: 1. Perform large-sample ifereces (hypothesis test ad cofidece itervals) to compare two populatio
More informationBasis for simulation techniques
Basis for simulatio techiques M. Veeraraghava, March 7, 004 Estimatio is based o a collectio of experimetal outcomes, x, x,, x, where each experimetal outcome is a value of a radom variable. x i. Defiitios
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationChapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationMA131 - Analysis 1. Workbook 3 Sequences II
MA3 - Aalysis Workbook 3 Sequeces II Autum 2004 Cotets 2.8 Coverget Sequeces........................ 2.9 Algebra of Limits......................... 2 2.0 Further Useful Results........................
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationMath 140 Introductory Statistics
8.2 Testig a Proportio Math 1 Itroductory Statistics Professor B. Abrego Lecture 15 Sectios 8.2 People ofte make decisios with data by comparig the results from a sample to some predetermied stadard. These
More informationChapter 6. Sampling and Estimation
Samplig ad Estimatio - 34 Chapter 6. Samplig ad Estimatio 6.. Itroductio Frequetly the egieer is uable to completely characterize the etire populatio. She/he must be satisfied with examiig some subset
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationLecture 1 Probability and Statistics
Wikipedia: Lecture 1 Probability ad Statistics Bejami Disraeli, British statesma ad literary figure (1804 1881): There are three kids of lies: lies, damed lies, ad statistics. popularized i US by Mark
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationCS434a/541a: Pattern Recognition Prof. Olga Veksler. Lecture 5
CS434a/54a: Patter Recogitio Prof. Olga Veksler Lecture 5 Today Itroductio to parameter estimatio Two methods for parameter estimatio Maimum Likelihood Estimatio Bayesia Estimatio Itroducto Bayesia Decisio
More informationLesson 10: Limits and Continuity
www.scimsacademy.com Lesso 10: Limits ad Cotiuity SCIMS Academy 1 Limit of a fuctio The cocept of limit of a fuctio is cetral to all other cocepts i calculus (like cotiuity, derivative, defiite itegrals
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationModule 1 Fundamentals in statistics
Normal Distributio Repeated observatios that differ because of experimetal error ofte vary about some cetral value i a roughly symmetrical distributio i which small deviatios occur much more frequetly
More informationTests of Hypotheses Based on a Single Sample (Devore Chapter Eight)
Tests of Hypotheses Based o a Sigle Sample Devore Chapter Eight MATH-252-01: Probability ad Statistics II Sprig 2018 Cotets 1 Hypothesis Tests illustrated with z-tests 1 1.1 Overview of Hypothesis Testig..........
More informationMBACATÓLICA. Quantitative Methods. Faculdade de Ciências Económicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS
MBACATÓLICA Quatitative Methods Miguel Gouveia Mauel Leite Moteiro Faculdade de Ciêcias Ecoómicas e Empresariais UNIVERSIDADE CATÓLICA PORTUGUESA 9. SAMPLING DISTRIBUTIONS MBACatólica 006/07 Métodos Quatitativos
More informationChapter 8: STATISTICAL INTERVALS FOR A SINGLE SAMPLE. Part 3: Summary of CI for µ Confidence Interval for a Population Proportion p
Chapter 8: STATISTICAL INTERVALS FOR A SINGLE SAMPLE Part 3: Summary of CI for µ Cofidece Iterval for a Populatio Proportio p Sectio 8-4 Summary for creatig a 100(1-α)% CI for µ: Whe σ 2 is kow ad paret
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationProblems from 9th edition of Probability and Statistical Inference by Hogg, Tanis and Zimmerman:
Math 224 Fall 2017 Homework 4 Drew Armstrog Problems from 9th editio of Probability ad Statistical Iferece by Hogg, Tais ad Zimmerma: Sectio 2.3, Exercises 16(a,d),18. Sectio 2.4, Exercises 13, 14. Sectio
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationExponential Families and Bayesian Inference
Computer Visio Expoetial Families ad Bayesia Iferece Lecture Expoetial Families A expoetial family of distributios is a d-parameter family f(x; havig the followig form: f(x; = h(xe g(t T (x B(, (. where
More informationAMS570 Lecture Notes #2
AMS570 Lecture Notes # Review of Probability (cotiued) Probability distributios. () Biomial distributio Biomial Experimet: ) It cosists of trials ) Each trial results i of possible outcomes, S or F 3)
More informationLast Lecture. Wald Test
Last Lecture Biostatistics 602 - Statistical Iferece Lecture 22 Hyu Mi Kag April 9th, 2013 Is the exact distributio of LRT statistic typically easy to obtai? How about its asymptotic distributio? For testig
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationExam II Covers. STA 291 Lecture 19. Exam II Next Tuesday 5-7pm Memorial Hall (Same place as exam I) Makeup Exam 7:15pm 9:15pm Location CB 234
STA 291 Lecture 19 Exam II Next Tuesday 5-7pm Memorial Hall (Same place as exam I) Makeup Exam 7:15pm 9:15pm Locatio CB 234 STA 291 - Lecture 19 1 Exam II Covers Chapter 9 10.1; 10.2; 10.3; 10.4; 10.6
More informationPSYCHOLOGICAL RESEARCH (PYC 304-C) Lecture 9
Hypothesis testig PSYCHOLOGICAL RESEARCH (PYC 34-C Lecture 9 Statistical iferece is that brach of Statistics i which oe typically makes a statemet about a populatio based upo the results of a sample. I
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More information(7 One- and Two-Sample Estimation Problem )
34 Stat Lecture Notes (7 Oe- ad Two-Sample Estimatio Problem ) ( Book*: Chapter 8,pg65) Probability& Statistics for Egieers & Scietists By Walpole, Myers, Myers, Ye Estimatio 1 ) ( ˆ S P i i Poit estimate:
More information1 Approximating Integrals using Taylor Polynomials
Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................
More informationAnalysis of Experimental Data
Aalysis of Experimetal Data 6544597.0479 ± 0.000005 g Quatitative Ucertaity Accuracy vs. Precisio Whe we make a measuremet i the laboratory, we eed to kow how good it is. We wat our measuremets to be both
More information5. INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY
IA Probability Let Term 5 INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY 51 Iequalities Suppose that X 0 is a radom variable takig o-egative values ad that c > 0 is a costat The P X c E X, c is
More informationIf, for instance, we were required to test whether the population mean μ could be equal to a certain value μ
STATISTICAL INFERENCE INTRODUCTION Statistical iferece is that brach of Statistics i which oe typically makes a statemet about a populatio based upo the results of a sample. I oesample testig, we essetially
More informationLecture 1 Probability and Statistics
Wikipedia: Lecture 1 Probability ad Statistics Bejami Disraeli, British statesma ad literary figure (1804 1881): There are three kids of lies: lies, damed lies, ad statistics. popularized i US by Mark
More informationSection 11.8: Power Series
Sectio 11.8: Power Series 1. Power Series I this sectio, we cosider geeralizig the cocept of a series. Recall that a series is a ifiite sum of umbers a. We ca talk about whether or ot it coverges ad i
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More informationAP Statistics Review Ch. 8
AP Statistics Review Ch. 8 Name 1. Each figure below displays the samplig distributio of a statistic used to estimate a parameter. The true value of the populatio parameter is marked o each samplig distributio.
More information