Distribution of Random Samples & Limit theorems
|
|
- Allan Barber
- 5 years ago
- Views:
Transcription
1 STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to demostrate the digital trasformatio i the US populatio over the past 5 years. Oe feature of this techological revolutio is for istace the time spet o smartphoes. Let us deote X that radom variable for some US idividual. If oe is iterested i E[X] the mea time spet o smartphoes by the US populatio, that quatity is exactly E[X] = 1 N x i N where N 320 millio, is the size of the US populatio ad x i is the time spet by idividual i o his/her smartphoe. However, i most statistical studies, it is rare to have access to the whole populatio. Therefore, we eed some pricipled guidace to estimate E[X] from a sample of the populatio of size with geerally N. Defiitio 1.1 i.i.d. Sample). Let X 1,..., X be collectio of N radom variables o probability space Ω, A, P). X 1,..., X ) is a radom sample of size if ad oly if: 1) X 1,..., X are mutually idepedet 2) X 1,..., X are idetically distributed, that is, each X i comes from the same distributio. We say that X 1,..., X are idepedet ad idetically distributed abbreviated i.i.d.). A good statistical sample is such that the picked idividuals are represetative of the populatio. The latter is esured by the mutual idepedece of the idividuals. Upo existece, the populatio mea is deoted µ = E[X 1 ] ad the variace of the populatio σ 2 = VarX 1 ). Defiitio 1.2 Sample mea). Let X 1,..., X ) be a radom sample.the the radom variable X = 1 X i 1) is called the sample mea. 1
2 We are iterested i the distributio of X. We eed to defie formally the distributio of more tha 2 radom variables. You should see that most of the followig defiitios/properties are atural extesios of the case = 2. Defiitio 1.3 Joit cumulative distributio fuctio). Let X 1,..., X be N rrvs o probability space Ω, A, P). The joit cumulative) distributio fuctio c.d.f.) of X 1,..., X is defied as follows : for x 1,..., x ) R F X1...X x 1,..., x ) = P X 1 x 1,..., X x ), 2) Defiitio 1.4 Discrete rrvs). Let X 1,..., X be N discrete rrvs o probability space Ω, A, P). The joit probability mass fuctio pmf) of X 1,..., X, deoted is defied as follows : p X1...X x 1,..., x ) = P X 1 = x 1,..., X = x ), 3) for x 1,..., x ) X 1 Ω)... X Ω). Property 1.1. Let X 1,..., X be N discrete rrvs o probability space Ω, A, P) with joit pmf p X1...X, the the followig holds : p X1...X x 1,..., x ) 0, for x 1,..., x ) X 1 Ω)... X Ω) x 1 X 1Ω)... x X Ω) p X 1...X x 1,..., x ) = 1 Defiitio 1.5 Joit probability desity fuctio). Let X 1,..., X be N rrvs o probability space Ω, A, P). X 1,..., X are said to be joitly cotiuous if there exists a fuctio f X1...X such that, for ay subset B R : PX 1,..., X ) B) = f X1...X x 1,..., x ) dx 1... dx 4) B The fuctio f X1...X is called the joit probability desity fuctio of X 1,..., X. Property 1.2. Let X 1,..., X be N rrvs o probability space Ω, A, P) with joit pdf f X1...X, the the followig holds : f X1...X x 1,..., x ) 0, for x 1,..., x ) R R f X1...X x 1,..., x ) dx 1... dx = 1 Defiitio 1.6 Margial distributios). Let X 1,..., X be N rrvs o probability space Ω, A, P). Discrete case) If X 1,..., X have joit pmf p X1...X. The the margial probability mass fuctio of X i is obtaied by summig over all X j, j i. For istace, p X1 x 1 ) =... p X1...X x 1,..., x ), for x 1 X 1 Ω) x 2 X 2Ω) x X Ω) 5) 2
3 Cotiuous case) If X 1,..., X have joit pdf f X1...X. The the margial probability desity fuctio of X i is obtaied by itegratig over all X j, j i. For istace, f X1 x 1 ) = f X1...X x 1,..., x ) dx 2... dx, for x 1 R 6) R 1 Defiitio 1.7 Idepedece). Let X 1,..., X be N rrvs o probability space Ω, A, P). Discrete case) If X 1,..., X have joit pmf p X1...X with respective margial pmfs p X1,..., p X. The X 1,..., X are said to be idepedet if ad oly if : p X1...X x 1,..., x ) = p Xi x i ), for all x 1,..., x ) X 1 Ω)... X Ω) 7) Cotiuous case) If X 1,..., X have joit pdf f X1...X with respective margial pdfs f X1,..., f X. The X 1,..., X are said to be idepedet if ad oly if : f X1...X x 1,..., x ) = f Xi x i ), for all x 1,..., x ) R 8) Property 1.3 Distributio of iid sample). Let X 1,..., X ) be a radom sample of size N o probability space Ω, A, P). The Discrete case) p X1...X x 1,..., x ) = p X1 x i ), for all x 1,..., x ) X 1 Ω)... X Ω) 9) Cotiuous case) f X1...X x 1,..., x ) = f X1 x i ), for all x 1,..., x ) R 10) Defiitio 1.8 Expected Value). Let X 1,..., X be N rrvs o probability space Ω, A, P) ad let g : R R. Discrete case) If X 1,..., X have joit pmf p X1...X. The, the mathematical expectatio of gx 1... X ), if it exists, is : E[gX 1... X )] =... gx 1,..., x )p X1...X x 1,..., x ) x 1 X 1Ω) x X Ω) 11) 3
4 Cotiuous case) If X 1,..., X have joit pdf f X1...X. The, the mathematical expectatio of gx 1... X ), if it exists, is : E[gX 1... X )] = gx 1,..., x )f X1...X x 1,..., x ) 12) R Theorem 1.1. Let X 1,..., X be N idepedet rrvs o probability space Ω, A, P) ad let g 1,..., g be real-valued fuctios o R. The, E[g 1 X 1 )... g X )] = E[g 1 X 1 )]... E[g X )] 13) provided that the expectatios exist. Theorem 1.2 Variace of idepedet rrvs). Let X 1,..., X be N idepedet rrvs o probability space Ω, A, P). The, ) Var X i = VarX i ) 14) provided that the variaces exist. Property 1.4. Let X 1,..., X ) be a radom sample of size N o probability space Ω, A, P) with mea µ = E[X 1 ] ad variace σ 2 = VarX 1 ) <. The E[X ] = µ 15) ad VarX ) = σ2 2 Covergece of radom variables 2.1 Covergece i probability 16) Defiitio 2.1. Let X ) N be a sequece of rrvs o probability space Ω, A, P) ad X be a rrv o the same probability space. Sequece X ) is said to coverge i probability towards X if, for all ε > 0 : Covergece i probability is deoted as follows : lim P X X > ε) = 0 17) X P X Example 1. Let X be a discrete rrv with pmf p X defied by : 1/3 if x = 1 p X x) = 2/3 if x = 0 0 otherwise 4
5 ad let X = )X. Show that X P X. Aswer. We have: X X = X + X X = X = X sice X ca oly take oegative values. The, for ay ε > 0, P X X > ε) = P X > ε). Note that the evet { X > ε} ca oly occur whe X = 1 ad ε < 1 sice ε > 0. Therefore, we get: { p X 1) = 1/3 < 1/ε P X X > ε) = 0 > 1/ε It ow becomes obvious that P X X > ε) coverges to 0, because it is idetically equal to zero for all > 1/ε, which etails the desired result. Example 2. For 1, let X ) be a sequece of radom variables where X follows a expoetial distributio with parameter. Show that X ) coverges i probability to 0. Aswer. The probability desity fuctio of X is give by : f X x) = e x 1 [0, ) x). Let ε > 0 be a arbitrary costat, we have P X 0 > ε) = PX > ε) = Hece, the result holds. ε e x dx = e ε 0 sice ε > 0 give that a expoetial rrv ca oly take o oegative values 2.2 Almost sure covergece Defiitio 2.2. Let X ) N be a sequece of rrvs o probability space Ω, A, P) ad X be a rrv o the same probability space. Sequece X ) is said to coverge almost surely or almost everywhere or with probability 1 or strogly towards X if: ) P lim X = X = 1 18) Almost sure covergece is deoted as follows : X a.s. X 5
6 From Equatio 18, we ote that almost sure covergece is a slightly modified versio of the cocept of poitwise covergece of fuctios recall that a radom variable is formally a mappig from the sample space Ω to R. That is, ω Ω, X ω) Xω) Requirig covergece for all ω Ω is actually too striget. To defie almost sure covergece, we relax the above statemet ad allow that covergece might ot be reached for some outcomes i Ω. Rigorously, let E be the followig evet : E = {ω Ω : X ω) does ot coverge to Xω)} ad F be a evet with zero probability but F is ot the impossible evet, i.e. F. The we say that the sequece X ) coverges almost surely to X if E F. Almost sure covergece is a widely spread cocept i the Probability ad Statistics literature but provig almost sure covergece requires tools from measure theory, which is out of scope of this course. 2.3 Covergece i mea Defiitio 2.3. Let X ) N be a sequece of rrvs o probability space Ω, A, P) ad X be a rrv o the same probability space. Give a real umber r 1, sequece X ) is said to coverge i the r-th mea or i the L r -orm towards X if: lim E[ X X r ] = 0 19) provided that for all, E[ X r ] ad E[ X r ] exist. Covergece i the r-th mea is deoted as follows : X L r X The most importat cases of covergece i the r-th mea are: Whe Equatio 19) holds for r = 1, we say that X ) coverges i mea to X Whe Equatio 19) holds for r = 2, we say that X ) coverges i mea square to X 2.4 Covergece i distributio Defiitio 2.4. Let X ) N be a sequece of rrvs o probability space Ω, A, P). For ay, the distributio fuctio of X is deoted by F. Let X be a rrv with distributio fuctio F X. Sequece X ) is said to coverge i distributio or coverge weakly towards X if: lim F x) = F X x) 20) 6
7 for all x R at which F X is cotiuous. Covergece i distributio is deoted as follows : X D X The first fact to otice is that covergece i distributio, as the ame suggests, oly ivolves the distributios of the radom variables. Thus, the radom variables eed ot eve be defied o the same probability space that is, they eed ot be defied for the same radom experimet), ad ideed we do t eve eed the radom variables at all. This is i cotrast to the other modes of covergece we have studied. Example 3. Let X ) N be a sequece of rrvs with cdf F F x) = ) x ) 1 0, ) x) What is the asymptotic distributio of X )? Aswer. Note that for x, 0), we trivially have that F x) = 0 0. Now, let x [0, ), a result i calculus gives : lim 1 1 ) x = e x Therefore, F x) = 1 1 ) 1 x 1 e x. We recogize the cumulative distributio fuctio of a expoetial distributio with parameter 1 for those who are ot coviced, you ca differetiate the expressio o the right-had side. We coclude that the sequece X ) coverges to a expoetial distributio with parameter 1. Theorem 2.1. Let X ) N be a sequece of rrvs o probability space Ω, A, P) with respective mgfs M. Let X be a rrv with mgf M X. If the followig holds : lim M x) = M X x) 21) for all x R where M x) ad M X x) exist, the sequece X ) coverges i distributio to X. 2.5 Implicatios betwee modes of covergece The followig summary gives the implicatios for the various modes of covergece; o other implicatios hold i geeral. 7
8 Propositio For s > r 1, covergece i the s-th mea implies covergece i r-th mea. 2. Covergece i mea implies covergece i probability. 3. Almost sure covergece implies covergece i probability. 4. Covergece i probability implies covergece i distributio. 3 The Laws of Large Numbers Property 3.1 Markov s Iequality). Let X be a rrv that takes oly o oegative values. The, for ay a > 0, we have : PX a) E[X] a 22) Property 3.2 Bieaymé-Chebyshev s Iequality). Let X be a rrv that has expectatio ad variace. The, for ay α > 0, we have : P X E[X] α) VarX) α 2 23) Theorem 3.1 Weak law of large umbers). Let X ) N be a sequece of i.i.d. rrvs, each havig fiite expectatio. The weak law of large umbers also called Khitchie s law) states that the sample mea X coverges i probability towards E[X 1 ], that is, for all ɛ > 0 : lim P X E[X 1 ] > ɛ ) = 0 24) Theorem 3.2 Strog law of large umbers). Let X ) N be a sequece of i.i.d. rrvs, each havig fiite expectatio. The strog law of large umbers SLLN) also called Kolmogorov s strog law, states that the sample mea X coverges almost surely towards E[X 1 ], that is: ) P lim X = E[X 1 ] = 1 25) Fudametal implicatio. Let X ) be a sequece of idepedet Beroulli radom variables with parameter p, that is X = 1 whe some evet E occurs with probability p = PE) ad X = 0 with probability 1 p whe E does ot occur. Accordig to the Strog Law of Large Numbers, X i a.s. E[X 1 ] = p I words, X i is the umber of times that E occurs over trials. The SLLN thus states that the frequecy of observig E coverges to PE) as the size of the sample gets larger ad larger. This justifies the frequetist school that sees the probability of a evet as the theoretical frequecy of observig that evet. 8
9 4 The Cetral Limit Theorem Property 4.1. Let X 1,..., X be N idepedet rrvs o probability space Ω, A, P) with respective momet geeratig fuctios M 1,..., M. The the momet geeratig fuctio of S = is : for all x R where M x) exist. M S x) = X i M i x) 26) Corollary 4.1 Mgf of iid sample). Let X 1,..., X ) be a radom sample of size N o probability space Ω, A, P) with momet geeratig fuctio M = M X1. The the momet geeratig fuctio of S = is : for all x R where Mx) exists. X i M S x) = Mx)) 27) Theorem 4.2 Cetral Limit Theorem). Let X ) N be a sequece of i.i.d. rrvs, each havig expectatio E[X 1 ] = µ ad fiite variace VarX 1 ) = σ 2 <. The Cetral Limit Theorem states that the sequece of variables Z ) N defied by: Z = X µ σ 2 coverges i distributio towards Z followig a stadard ormal distributio N 0, 1), that is: lim F Z x) = Φx), for all x R 28) Equivaletly, the CLT ca be rewritte as: ) D X N µ, σ2 Applicatios of the CLT. With the Strog Law of Large Numbers, the CLT is the other most importat result i Probability ad Statistics. I words, the CLT states that the distributio of the sum or mea) of ay iid radom variables coverges to a ormal distributio provided that the populatio distributio has fiite variace. As a cosequece, you ca use the ormal distributio to approximate probabilities as log as the sample size is large eough. How large is large eough? The aswer depeds o two factors. 9
10 Requiremets for accuracy. The more closely the samplig distributio eeds to resemble a ormal distributio, the more sample poits will be required. The shape of the uderlyig populatio. The more closely the origial populatio resembles a ormal distributio, the fewer sample poits will be required. Empirical evidece shows that a sample size of 30 is large eough whe the populatio distributio is roughly bell-shaped. Some statisticias may recommed a sample size of at least 40 though. But if the origial populatio is distictly ot ormal, the sample size should be eve larger. Example 4. Let X 1,..., X 15 ) be a radom sample with probability desity fuctio : fx) = 3 2 x2 1 1,1) x) What is the approximate probability that the sample mea X 15 falls betwee -2/5 ad 1/5? Aswer. The CLT states that X 15 follows approximately a ormal distributio N E[X 1 ], VarX 1 )/15). Let us compute E[X 1 ] ad VarX 1 ) : E[X 1 ] = 1 1 x 3 2 x2 dx = 0 VarX 1 ) = E[X 2 1 ] E[X 1 ] 2 = = x x2 dx 0 2 Therefore, VarX 1 )/15 = 3/75 = 1/25 D X 15 N 0, 1 ) 25 Hece, P 2 5 X 15 1 ) ) 2/5 0 = P X / /25 1/25 1/25 P 2 Z 1) Z is a stadard ormal radom variable) Φ1) Φ 2) Φ is the cdf of N 0, 1)) Φ1) + Φ2)
This section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationAMS570 Lecture Notes #2
AMS570 Lecture Notes # Review of Probability (cotiued) Probability distributios. () Biomial distributio Biomial Experimet: ) It cosists of trials ) Each trial results i of possible outcomes, S or F 3)
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationLecture 18: Sampling distributions
Lecture 18: Samplig distributios I may applicatios, the populatio is oe or several ormal distributios (or approximately). We ow study properties of some importat statistics based o a radom sample from
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationLimit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).
Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More informationProbability and statistics: basic terms
Probability ad statistics: basic terms M. Veeraraghava August 203 A radom variable is a rule that assigs a umerical value to each possible outcome of a experimet. Outcomes of a experimet form the sample
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationECONOMETRIC THEORY. MODULE XIII Lecture - 34 Asymptotic Theory and Stochastic Regressors
ECONOMETRIC THEORY MODULE XIII Lecture - 34 Asymptotic Theory ad Stochastic Regressors Dr. Shalabh Departmet of Mathematics ad Statistics Idia Istitute of Techology Kapur Asymptotic theory The asymptotic
More informationLarge Sample Theory. Convergence. Central Limit Theorems Asymptotic Distribution Delta Method. Convergence in Probability Convergence in Distribution
Large Sample Theory Covergece Covergece i Probability Covergece i Distributio Cetral Limit Theorems Asymptotic Distributio Delta Method Covergece i Probability A sequece of radom scalars {z } = (z 1,z,
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationElements of Statistical Methods Lots of Data or Large Samples (Ch 8)
Elemets of Statistical Methods Lots of Data or Large Samples (Ch 8) Fritz Scholz Sprig Quarter 2010 February 26, 2010 x ad X We itroduced the sample mea x as the average of the observed sample values x
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More information5. INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY
IA Probability Let Term 5 INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY 51 Iequalities Suppose that X 0 is a radom variable takig o-egative values ad that c > 0 is a costat The P X c E X, c is
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationAn Introduction to Asymptotic Theory
A Itroductio to Asymptotic Theory Pig Yu School of Ecoomics ad Fiace The Uiversity of Hog Kog Pig Yu (HKU) Asymptotic Theory 1 / 20 Five Weapos i Asymptotic Theory Five Weapos i Asymptotic Theory Pig Yu
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationGeneralized Semi- Markov Processes (GSMP)
Geeralized Semi- Markov Processes (GSMP) Summary Some Defiitios Markov ad Semi-Markov Processes The Poisso Process Properties of the Poisso Process Iterarrival times Memoryless property ad the residual
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 6 9/24/2008 DISCRETE RANDOM VARIABLES AND THEIR EXPECTATIONS
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 6 9/24/2008 DISCRETE RANDOM VARIABLES AND THEIR EXPECTATIONS Cotets 1. A few useful discrete radom variables 2. Joit, margial, ad
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationGlivenko-Cantelli Classes
CS28B/Stat24B (Sprig 2008 Statistical Learig Theory Lecture: 4 Gliveko-Catelli Classes Lecturer: Peter Bartlett Scribe: Michelle Besi Itroductio This lecture will cover Gliveko-Catelli (GC classes ad itroduce
More informationCEE 522 Autumn Uncertainty Concepts for Geotechnical Engineering
CEE 5 Autum 005 Ucertaity Cocepts for Geotechical Egieerig Basic Termiology Set A set is a collectio of (mutually exclusive) objects or evets. The sample space is the (collectively exhaustive) collectio
More informationThe variance of a sum of independent variables is the sum of their variances, since covariances are zero. Therefore. V (xi )= n n 2 σ2 = σ2.
SAMPLE STATISTICS A radom sample x 1,x,,x from a distributio f(x) is a set of idepedetly ad idetically variables with x i f(x) for all i Their joit pdf is f(x 1,x,,x )=f(x 1 )f(x ) f(x )= f(x i ) The sample
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationEECS564 Estimation, Filtering, and Detection Hwk 2 Solns. Winter p θ (z) = (2θz + 1 θ), 0 z 1
EECS564 Estimatio, Filterig, ad Detectio Hwk 2 Sols. Witer 25 4. Let Z be a sigle observatio havig desity fuctio where. p (z) = (2z + ), z (a) Assumig that is a oradom parameter, fid ad plot the maximum
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationChapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
More informationJoint Probability Distributions and Random Samples. Jointly Distributed Random Variables. Chapter { }
UCLA STAT A Applied Probability & Statistics for Egieers Istructor: Ivo Diov, Asst. Prof. I Statistics ad Neurology Teachig Assistat: Neda Farziia, UCLA Statistics Uiversity of Califoria, Los Ageles, Sprig
More informationLecture 3. Properties of Summary Statistics: Sampling Distribution
Lecture 3 Properties of Summary Statistics: Samplig Distributio Mai Theme How ca we use math to justify that our umerical summaries from the sample are good summaries of the populatio? Lecture Summary
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationLecture 6: Coupon Collector s problem
Radomized Algorithms Lecture 6: Coupo Collector s problem Sotiris Nikoletseas Professor CEID - ETY Course 2017-2018 Sotiris Nikoletseas, Professor Radomized Algorithms - Lecture 6 1 / 16 Variace: key features
More informationn outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n,
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 9 Variace Questio: At each time step, I flip a fair coi. If it comes up Heads, I walk oe step to the right; if it comes up Tails, I walk oe
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationUnderstanding Samples
1 Will Moroe CS 109 Samplig ad Bootstrappig Lecture Notes #17 August 2, 2017 Based o a hadout by Chris Piech I this chapter we are goig to talk about statistics calculated o samples from a populatio. We
More informationDirection: This test is worth 150 points. You are required to complete this test within 55 minutes.
Term Test 3 (Part A) November 1, 004 Name Math 6 Studet Number Directio: This test is worth 10 poits. You are required to complete this test withi miutes. I order to receive full credit, aswer each problem
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationApproximations and more PMFs and PDFs
Approximatios ad more PMFs ad PDFs Saad Meimeh 1 Approximatio of biomial with Poisso Cosider the biomial distributio ( b(k,,p = p k (1 p k, k λ: k Assume that is large, ad p is small, but p λ at the limit.
More information5 Birkhoff s Ergodic Theorem
5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More informationBasics of Probability Theory (for Theory of Computation courses)
Basics of Probability Theory (for Theory of Computatio courses) Oded Goldreich Departmet of Computer Sciece Weizma Istitute of Sciece Rehovot, Israel. oded.goldreich@weizma.ac.il November 24, 2008 Preface.
More informationCentral limit theorem and almost sure central limit theorem for the product of some partial sums
Proc. Idia Acad. Sci. Math. Sci. Vol. 8, No. 2, May 2008, pp. 289 294. Prited i Idia Cetral it theorem ad almost sure cetral it theorem for the product of some partial sums YU MIAO College of Mathematics
More informationLecture 33: Bootstrap
Lecture 33: ootstrap Motivatio To evaluate ad compare differet estimators, we eed cosistet estimators of variaces or asymptotic variaces of estimators. This is also importat for hypothesis testig ad cofidece
More informationUnbiased Estimation. February 7-12, 2008
Ubiased Estimatio February 7-2, 2008 We begi with a sample X = (X,..., X ) of radom variables chose accordig to oe of a family of probabilities P θ where θ is elemet from the parameter space Θ. For radom
More informationMA Advanced Econometrics: Properties of Least Squares Estimators
MA Advaced Ecoometrics: Properties of Least Squares Estimators Karl Whela School of Ecoomics, UCD February 5, 20 Karl Whela UCD Least Squares Estimators February 5, 20 / 5 Part I Least Squares: Some Fiite-Sample
More informationStatistical Theory; Why is the Gaussian Distribution so popular?
Statistical Theory; Why is the Gaussia Distributio so popular? Rob Nicholls MRC LMB Statistics Course 2014 Cotets Cotiuous Radom Variables Expectatio ad Variace Momets The Law of Large Numbers (LLN) The
More informationSome Basic Probability Concepts. 2.1 Experiments, Outcomes and Random Variables
Some Basic Probability Cocepts 2. Experimets, Outcomes ad Radom Variables A radom variable is a variable whose value is ukow util it is observed. The value of a radom variable results from a experimet;
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More informationLecture 3 : Random variables and their distributions
Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}
More informationQuick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter 2 & Teachig
More informationPRACTICE PROBLEMS FOR THE FINAL
PRACTICE PROBLEMS FOR THE FINAL Math 36Q Fall 25 Professor Hoh Below is a list of practice questios for the Fial Exam. I would suggest also goig over the practice problems ad exams for Exam ad Exam 2 to
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationLecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
More informationQuick Review of Probability
Quick Review of Probability Berli Che Departmet of Computer Sciece & Iformatio Egieerig Natioal Taiwa Normal Uiversity Refereces: 1. W. Navidi. Statistics for Egieerig ad Scietists. Chapter & Teachig Material.
More information2.1. Convergence in distribution and characteristic functions.
3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For
More information