7.1 Convergence of sequences of random variables
|
|
- Candace McCarthy
- 5 years ago
- Views:
Transcription
1 Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite sequece of distributios it is possible to costruct a probability space with a correspodig sequece of radom variables is a otrivial fact, whose proof is due to Kolmogorov (see for example Billigsley). 7.1 Covergece of sequeces of radom variables Defiitio 7.1 The sequece (X ) is said to coverge to X almost-surely (or, w.p. 1) if We write X a.s X. ({ }) P ω : lim X (ω) = X(ω) = 1. It is ofte easier to express this mode of covergece usig its complemet. X (ω) fails to coverge to X(ω) if there exists a ɛ> 0 such that X (ω) X(ω) ɛ holds for ifiitely may values of. Let us deote the followig family of evets, B ɛ = {ω : X (ω) X(ω) ɛ}. Thus, X (ω) does ot coverge almost-surely to X(ω) is there exists a ɛ> 0 such that P(B ɛ i.o.) > 0,
2 126 Chapter 7 ad iversely, it does coverge almost-surely to X(ω) if for all ɛ> 0 ( ) P(B ɛ i.o.) = P lim sup B ɛ = 0. Defiitio 7.2 The sequece (X ) is said to coverge to X i the mea-square if We write X m.s X. lim E X X 2 = 0. Defiitio 7.3 The sequece (X ) is said to coverge to X i probability if for every ɛ> 0, lim P ({ω : X (ω) X(ω) >ɛ}) = 0. We write X Pr X. Defiitio 7.4 The sequece (X ) is said to coverge to X i distributio if for every a R, lim F X (a) = F X (a), i.e., if the sequece of distributio fuctios of the X coverges poit-wise to the D distributio fuctio of X. We write X X. Commet: Note that the first three modes of covergece require that the sequece (X ) ad X are all defied o a joit probability space. Sice covergece i distributio oly refers to distributio, each variable could, i priciple, belog to a separate world. The first questio to be addressed is whether there exists a hierarchy of modes of covergece. We wat to kow which modes of covergece imply which. The aswer is that both almost-sure ad mea-square covergece imply covergece i probability, which i tur implies covergece i distributio. O the other had, almost-sure ad mea-square covergece do ot imply each other. Propositio 7.1 Almost-sure covergece implies covergece i probability.
3 Limit theorems 127 Proof : As we have see, almost sure covergece meas that for every ɛ> 0 Defie the family of evets X X almost-surely if for every ɛ> 0 P ({ω : X (ω) X(ω) >ɛ i.o.}) = 0. B ɛ = {ω : X (ω) X(ω) >ɛ}. ( ) P lim sup B ɛ = 0. By the Fatou lemma, ( ) lim sup P(B ɛ ) P lim sup B ɛ = 0, from which we deduce that lim P(Bɛ ) = lim P ({ω : X (ω) X(ω) >ɛ}) = 0, i.e., X X i probability. Propositio 7.2 Mea-square covergece implies covergece i probability. Proof : This is a immediate cosequece of the Markov iequality, for P ( X X >ɛ) E X X 2 ɛ 2, ad the right-had side coverges to zero. Propositio 7.3 Mea-square covergece does ot imply almost sure covergece.
4 128 Chapter 7 Proof : All we eed is a couter example. Cosider a family of idepedet Beroulli variables X with atomistic distributios, 1/ x = 1 p X (x) = 1 1/ x = 0, ad set X = 0. We claim that X X i the mea square, as E X X 2 = E[X ] = 1 0. O the other had, it does ot coverge to X almost surely, as for ɛ = 1/2, P( X X >ɛ) = =1 ad by the secod lemma of Borel-Catelli, =1 P( X X >ɛ i.o.) = 1. 1 =, Propositio 7.4 Almost-sure covergece does ot imply mea square covergece. Proof : Agai we costruct a couter example, with 1/ 2 x = 3 p X (x) = 1 1/ 2 x = 0, ad agai X = 0. We immediately see that X does ot coverge to X i the mea square, sice E X X 2 = E[X 2 ] = 6 2 =. It remais to show that X X almost-surely. For every ɛ> 0, ad sufficietly large, P( X >ɛ) = 1/ 2, i.e., for every ɛ> 0, P( X X >ɛ) <, =1
5 Limit theorems 129 ad by the first lemma of Borel-Catelli, P( X X >ɛ i.o.) = 0. Commet: I the above example X X i probability, so that the latter does ot imply covergece i the mea square either. Propositio 7.5 Covergece i probability implies covergece i distributio. Proof : Let a R be give, ad set ɛ> 0. O the oe had F X (a) = P (X a, X a + ɛ) + P (X a, X > a + ɛ) = P (X a X a + ɛ) P (X a + ɛ) + P (X a, X > a + ɛ) P (X a + ɛ) + P (X < X ɛ) F X (a + ɛ) + P ( X X >ɛ), where we have used the fact that if A implies B the P(A) P(B)). By a similar argumet F X (a ɛ) = P (X a ɛ, X a) + P (X a ɛ, X > a) Thus, we have obtaied that = P (X a ɛ X a) P (X a) + P (X a ɛ, X > a) P (X a) + P (X < X ɛ) F X (a) + P ( X X >ɛ), F X (a ɛ) P ( X X >ɛ) F X (a) F X (a + ɛ) + P ( X X >ɛ). Takig ow we have F X (a ɛ) lim if F X (a) lim sup F X (a) F X (a + ɛ). Fially, sice this iequality holds for ay ɛ> 0 we coclude that lim F X (a) = F X (a). To coclude, the various modes of covergece satisfy the followig scheme:
6 130 Chapter 7 almost surely i the mea square i probability i distributio Exercise 7.1 Prove that if X coverges i distributio to a costat c, the X coverges i probability to c. Exercise 7.2 Prove that if X coverges to X i probability the it has a subsequece that coverges to X almost-surely. 7.2 The weak law of large umbers Theorem 7.1 (Weak law of large umbers) Let X be a sequece of idepedet idetically distributed radom variables o a probability space (Ω, F, P). Set E[X i ] = µ ad Var[X i ] = σ 2. Defie the sequece of cummulative averages, S = X X. The, S coverges to µ i probability, i.e., for every ɛ> 0, lim P ( S µ >ɛ) = 0. Commets: ➀ The assumptio that the variace is fiite is ot required; it oly simplifies the proof. ➁ Take the particular case where 1 ω A X i (ω) = 0 ω A.
7 Limit theorems 131 The, S = fractio of times ω A. The weak law of large umbers states that the fractio of times the outcome is i a give set coverges i probability to E[X 1 ], which is the probability of this set, P(A). Proof : This is a immediate cosequece of the Chebyshev iequality, for by the additivity of the expectatio ad the variace (for idepedet radom variables), The, E[S ] = µ ad Var[S ] = σ2. P ( S µ >ɛ) Var[S ] ɛ 2 = σ2 ɛ 2 0. Commet: the first proof is due to Jacob Beroulli (1713), who proved it for the particular case of biomial variables. 7.3 The cetral limit theorem Theorem 7.2 (Cetral limit theorem) Let (X ) be a sequece of i.i.d. radom variables with E[X i ] = 0 ad Var[X i ] = 1. The, the sequece of radom variables S = X X coverges i distributio to a radom variables X N(0, 1). That is, lim P (S a) = 1 a e y2 /2 dy. 2π Commets:
8 132 Chapter 7 ➀ If E[X i ] = µ ad Var[X i ] = σ 2 the the same applies for S = X X µ σ = 1 X i µ σ. ➁ The cetral limit theorem (CLT) is about a ruig average rescaled by a factor of. If we deote by Y the ruig average, Y = X X, the the CLT states that P (Y a ) Φ(a), i.e., it provides a estimate of the distributio of Y at distaces O( 1/2 ) from its mea. It is a theorem about small deviatios from the mea. There exist more sophisticated theorems about the distributio of Y far from the mea, part of the so-called theory of large deviatios. ➂ There are may variats of this theorem. Proof : We will use the followig fact, which we wo t prove: if the sequece of momet geeratig fuctios M X (t) of a sequece of radom variables (X ) coverges for every t to the momet geeratig fuctio M X (t) of a radom variable X, the X coverges to X i distributio. I other words, M X (t) M X (t) for all t implies that X D X. Thus, we eed to show that the momet geeratig fuctios of the S s teds as to exp(t 2 /2), which is the momet geeratig fuctio of a stadard ormal variable. Recall that the PDF of a sum of two radom variables is the covolutio of their PDF, but the momet geeratig fuctio of their sum is the product of the their momet geeratig fuctio. Iductively, M X1 +X ,+X (t) = M Xi (t) = [M X1 (t)],
9 Limit theorems 133 where we have used the fact that they are i.i.d., Now, if a radom variable Y has a momet geeratig fuctio M Y, the M Y/a (t) = e ty f Y/a (y) dy, but sice f Y/a (y) = af Y (ay) we get that M Y/a (t) = a e ty f Y (ay) dy = e aty/a f Y (ay) d(ay) = M Y (t/a), R R R from which we deduce that M S (t) = [ M X1 ( t )]. Take the logarithm of both sides, ad write the right had side explicitly, log M S (t) = log e tx/ f X1 (x) dx. Taylor expadig the expoetial about t = 0 we have, log M S (t) = log R ( 1 + tx + t2 x 2 R 2 + t3 x 3 6 ) = log ( t2 2 + O( 3/2 ) ( ) t 2 = 2 + O( 3/2 ) t2 2. ) eξx/ 3/2 f X1 (x) dx Example: Suppose that a experimetalist wats to measure some quatity. He kows that due to various sources of errors, the result of every sigle measuremet is a radom variable, whose mea µ is the correct aswer, ad the variace of his measuremet is σ 2. He therefore performs idepedet measuremets ad averages the results. How may such measuremets does he eed to perform to be sure, withi 95% certaity, that his estimate does ot deviate from the true result by σ/4?
10 134 Chapter 7 The questio we re askig is how large should be i order for the iequality P µ σ 4 1 X k µ + σ to hold. This is equivalet to askig what should be for P 4 1 X k µ σ k=1 k=1 By the cetral limit theorem the right had side is, for large, approximately 2 /4 e y2 /2 dy, 2π which turs out to be larger tha 0.95 for The problem with this argumet that it uses the assumptio that is large, but it is ot clear what large is. Is = 62 sufficietly large for this argumet to hold? This problem could have bee solved without this difficulty but resortig istead to the Chebyshev iequality: P , k=1 X k µ σ ad the right had side is larger tha 0.95 if 4 = 1 P 1 k=1 X k µ σ = 320. Example: The umber of studets X who are goig to fail i the exam is a Poisso variable with mea 100, i.e, X Poi(100). I am goig to admit that the exam was too hard if more tha 120 studet fail. What is the probability for it to happe? We kow the exact aswer, P (X 120) = e 100 k= k k!,
11 Limit theorems 135 which is a quite useless expressio. Let s base our estimate o the cetral limit theorem as follows: a Poisso variable with mea 100 ca be expressed as the sum of oe hudred idepedet variables X k Poi(1) (the sum of idepedet Poisso variables is agai a Poisso variable), that is X = 100 k=1 X k. Now, P (X 120) = P k=1 X k 1 1 which by the cetral limit theorem equals approximately, 20 10, P (X 120) 1 e y2 /2 dy π 2 Example: Let us examie umerically a particular example. Let X i Exp (1) be idepedet expoetial variable ad set S = 1 (X i 1). A sum of expoetial variables has distributio Gamma (, 1), i.e., its pdf is The desity for this sum shifted by is with x > ad after dividig by, x 1 e x. Γ() (x + ) 1 e (x+), Γ() f S (x) = ( x + ) 1 e ( x+), Γ() with x >. See Figure 7.1 for a visualizatio of the approach of the distributio of S toward the stadard ormal distributio.
12 136 Chapter 7 1 =1 0.6 = = = Figure 7.1: The approach of a ormalized sum of 1, 2, 4 ad 16 expoetial radom variables to the ormal distributio.
13 Limit theorems The strog law of large umbers Our ext limit theorem is the strog law of large umber, which states that the ruig average of a sequece of i.i.d. variables coverges to the mea almostsurely (thus stregtheig the weak law of large umber, which oly provides covergece i probability). Theorem 7.3 (Strog law of large umbers) Let (X ) be a i.i.d. sequece of radom variables with E[X i ] = 0 ad Var[X i ] = σ 2, the with probability oe, X X 0. Proof : Set ɛ> 0, ad cosider the followig sequece of evets: A = max k X i 1 k i >ɛ. From Komogorov s iequality, P(A ) 1 ɛ 2 k=1 σ 2 k 2. The (A ) are a icreasig sequece hece, by the cotiuity of the probability, ( ) lim P(A ) = P lim A = P max k X i 1 k i >ɛ Cσ2 ɛ, 2 or equivaletly, P max k 1 k X i i ɛ 1 Cσ2 ɛ 2 Sice this holds for all ɛ> 0, takig ɛ results i P max k X i 1 k i < = 1.
14 138 Chapter 7 from which we ifer that P X i i < = 1, It is the a cosequece of a lemma due to Kroecker that X i i 1 < implies lim X k = 0. k=1
7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationIntroduction to Probability. Ariel Yadin. Lecture 7
Itroductio to Probability Ariel Yadi Lecture 7 1. Idepedece Revisited 1.1. Some remiders. Let (Ω, F, P) be a probability space. Give a collectio of subsets K F, recall that the σ-algebra geerated by K,
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationApplication to Random Graphs
A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let
More informationProbability for mathematicians INDEPENDENCE TAU
Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationLecture 3: August 31
36-705: Itermediate Statistics Fall 018 Lecturer: Siva Balakrisha Lecture 3: August 31 This lecture will be mostly a summary of other useful expoetial tail bouds We will ot prove ay of these i lecture,
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationExercise 4.3 Use the Continuity Theorem to prove the Cramér-Wold Theorem, Theorem. (1) φ a X(1).
Assigmet 7 Exercise 4.3 Use the Cotiuity Theorem to prove the Cramér-Wold Theorem, Theorem 4.12. Hit: a X d a X implies that φ a X (1) φ a X(1). Sketch of solutio: As we poited out i class, the oly tricky
More information1+x 1 + α+x. x = 2(α x2 ) 1+x
Math 2030 Homework 6 Solutios # [Problem 5] For coveiece we let α lim sup a ad β lim sup b. Without loss of geerality let us assume that α β. If α the by assumptio β < so i this case α + β. By Theorem
More informationECE534, Spring 2018: Final Exam
ECE534, Srig 2018: Fial Exam Problem 1 Let X N (0, 1) ad Y N (0, 1) be ideedet radom variables. variables V = X + Y ad W = X 2Y. Defie the radom (a) Are V, W joitly Gaussia? Justify your aswer. (b) Comute
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationAMS570 Lecture Notes #2
AMS570 Lecture Notes # Review of Probability (cotiued) Probability distributios. () Biomial distributio Biomial Experimet: ) It cosists of trials ) Each trial results i of possible outcomes, S or F 3)
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationAxioms of Measure Theory
MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/
More informationMATH 112: HOMEWORK 6 SOLUTIONS. Problem 1: Rudin, Chapter 3, Problem s k < s k < 2 + s k+1
MATH 2: HOMEWORK 6 SOLUTIONS CA PRO JIRADILOK Problem. If s = 2, ad Problem : Rudi, Chapter 3, Problem 3. s + = 2 + s ( =, 2, 3,... ), prove that {s } coverges, ad that s < 2 for =, 2, 3,.... Proof. The
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationThe Central Limit Theorem
Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx
More information5. INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY
IA Probability Let Term 5 INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY 51 Iequalities Suppose that X 0 is a radom variable takig o-egative values ad that c > 0 is a costat The P X c E X, c is
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationIt is often useful to approximate complicated functions using simpler ones. We consider the task of approximating a function by a polynomial.
Taylor Polyomials ad Taylor Series It is ofte useful to approximate complicated fuctios usig simpler oes We cosider the task of approximatig a fuctio by a polyomial If f is at least -times differetiable
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More informationSolution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1
Solutio Sagchul Lee October 7, 017 1 Solutios of Homework 1 Problem 1.1 Let Ω,F,P) be a probability space. Show that if {A : N} F such that A := lim A exists, the PA) = lim PA ). Proof. Usig the cotiuity
More informationAssignment 5: Solutions
McGill Uiversity Departmet of Mathematics ad Statistics MATH 54 Aalysis, Fall 05 Assigmet 5: Solutios. Let y be a ubouded sequece of positive umbers satisfyig y + > y for all N. Let x be aother sequece
More informationProbability: Limit Theorems I. Charles Newman, Transcribed by Ian Tobasco
Probability: Limit Theorems I Charles Newma, Trascribed by Ia Tobasco Abstract. This is part oe of a two semester course o measure theoretic probability. The course was offered i Fall 2011 at the Courat
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationOutput Analysis and Run-Length Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationn outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n,
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 9 Variace Questio: At each time step, I flip a fair coi. If it comes up Heads, I walk oe step to the right; if it comes up Tails, I walk oe
More informationLimit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).
Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.
More informationMATH 320: Probability and Statistics 9. Estimation and Testing of Parameters. Readings: Pruim, Chapter 4
MATH 30: Probability ad Statistics 9. Estimatio ad Testig of Parameters Estimatio ad Testig of Parameters We have bee dealig situatios i which we have full kowledge of the distributio of a radom variable.
More informationRandom Variables, Sampling and Estimation
Chapter 1 Radom Variables, Samplig ad Estimatio 1.1 Itroductio This chapter will cover the most importat basic statistical theory you eed i order to uderstad the ecoometric material that will be comig
More informationLaw of the sum of Bernoulli random variables
Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationIntroduction to Probability. Ariel Yadin. Lecture 2
Itroductio to Probability Ariel Yadi Lecture 2 1. Discrete Probability Spaces Discrete probability spaces are those for which the sample space is coutable. We have already see that i this case we ca take
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationSince X n /n P p, we know that X n (n. Xn (n X n ) Using the asymptotic result above to obtain an approximation for fixed n, we obtain
Assigmet 9 Exercise 5.5 Let X biomial, p, where p 0, 1 is ukow. Obtai cofidece itervals for p i two differet ways: a Sice X / p d N0, p1 p], the variace of the limitig distributio depeds oly o p. Use the
More informationSTAT 350 Handout 19 Sampling Distribution, Central Limit Theorem (6.6)
STAT 350 Hadout 9 Samplig Distributio, Cetral Limit Theorem (6.6) A radom sample is a sequece of radom variables X, X 2,, X that are idepedet ad idetically distributed. o This property is ofte abbreviated
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More information6 Infinite random sequences
Tel Aviv Uiversity, 2006 Probability theory 55 6 Ifiite radom sequeces 6a Itroductory remarks; almost certaity There are two mai reasos for eterig cotiuous probability: ifiitely high resolutio; edless
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationNotes 27 : Brownian motion: path properties
Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X
More information2 Banach spaces and Hilbert spaces
2 Baach spaces ad Hilbert spaces Tryig to do aalysis i the ratioal umbers is difficult for example cosider the set {x Q : x 2 2}. This set is o-empty ad bouded above but does ot have a least upper boud
More informationLecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
More informationMa 530 Introduction to Power Series
Ma 530 Itroductio to Power Series Please ote that there is material o power series at Visual Calculus. Some of this material was used as part of the presetatio of the topics that follow. What is a Power
More informationSimulation. Two Rule For Inverting A Distribution Function
Simulatio Two Rule For Ivertig A Distributio Fuctio Rule 1. If F(x) = u is costat o a iterval [x 1, x 2 ), the the uiform value u is mapped oto x 2 through the iversio process. Rule 2. If there is a jump
More information2.1. Convergence in distribution and characteristic functions.
3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For
More information10/31/2018 CentralLimitTheorem
10/31/2018 CetralLimitTheorem http://127.0.0.1:8888/bcovert/html/cs237/web/homeworks%2c%20labs%2c%20ad%20code/cetrallimittheorem.ipyb?dowload=false 1/10 10/31/2018 CetralLimitTheorem Cetral Limit Theorem
More information1 The Haar functions and the Brownian motion
1 The Haar fuctios ad the Browia motio 1.1 The Haar fuctios ad their completeess The Haar fuctios The basic Haar fuctio is 1 if x < 1/2, ψx) = 1 if 1/2 x < 1, otherwise. 1.1) It has mea zero 1 ψx)dx =,
More informationChapter 2 The Monte Carlo Method
Chapter 2 The Mote Carlo Method The Mote Carlo Method stads for a broad class of computatioal algorithms that rely o radom sampligs. It is ofte used i physical ad mathematical problems ad is most useful
More informationChapter 6 Sampling Distributions
Chapter 6 Samplig Distributios 1 I most experimets, we have more tha oe measuremet for ay give variable, each measuremet beig associated with oe radomly selected a member of a populatio. Hece we eed to
More informationMathematical Statistics - MS
Paper Specific Istructios. The examiatio is of hours duratio. There are a total of 60 questios carryig 00 marks. The etire paper is divided ito three sectios, A, B ad C. All sectios are compulsory. Questios
More informationMIT Spring 2016
MIT 18.655 Dr. Kempthore Sprig 2016 1 MIT 18.655 Outlie 1 2 MIT 18.655 Beroulli s Weak Law of Large Numbers X 1, X 2,... iid Beroulli(θ). S i=1 = X i Biomial(, θ). S P θ. Proof: Apply Chebychev s Iequality,
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More information