7.1 Convergence of sequences of random variables
|
|
- Beatrice Atkins
- 5 years ago
- Views:
Transcription
1 Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite sequece of distributios it is possible to costruct a probability space with a correspodig sequece of radom variables is a otrivial fact, whose proof is due to Kolmogorov (see for example Billigsley). 7.1 Covergece of sequeces of radom variables For every poit!, (X (!)) is a umber sequece ad X(!) is a umber. It might be that! is such that the sequece (X (!)) coverges to X(!), but it might also be that this sequece does ot coverge at all, or that it does ot coverge to X(!). The set! lim X (!) = X(!) is a evet; as such, it has a probability, which, i priciple, could be either zero, oe or ay itermediate umber. The followig defiitio provides a termiology to oe of those cases: Defiitio 7.1 The sequece of radom variable (X ) is said to coverge to the radom variable X almost-surely ($*/; )3/,) (or, w.p. 1) if We write X a.s X. P! lim X (!) = X(!) = 1.
2 164 Chapter 7 We ca write this mode of covergece i more explicit form. (X (!)) exists ad equals X(!) if ad oly if The limit of > 0, N N, > N, X (!) X(!) <. Note that we ca replace this coditio by the equivalet coditio k N, N N, > N, X (!) X(!) 1 k. Equivaletly, X(!) is ot the limit of (X (!)) if ad oly if k N, N N, > N, X (!) X(!) > 1 k. It follows that the coditio X a.s X ca be reformulated as P! k N, N N, > N, X (!) X(!) > 1 = 0, k or equivaletly, N=1 =N+1 P This ca further be writte as {! X (!) X(!) > 1k} = 0. P lim sup {! X (!) X(!) > 1k} = 0. Note that the sequece of evets {! X (!) X(!) > 1k} is icreasig as a fuctio of k, hece also their lim-sups. Thus, this equality is equal to lim k P lim sup {! X (!) X(!) > 1k} = 0. But if the limit of a icreasig o-egative sequece teds to zero, it must be a.s that the sequece is idetically zero, amely X X if ad oly if P lim sup {! X (!) X(!) > 1k} = 0 k N. a.s I words, X X if for every k, the probability that X (!) deviates from X(!) by more tha 1k for ifiitely may s is zero.
3 Limit Theorems 165 Just like sequeces of fuctios ca coverge to a limitig fuctios i more tha oe way (e.g., poitwise versus uiformly), so a sequece of radom variables ca coverge to a limitig radom variable i may di eret ways. Sayig that X coverges to X is like sayig that the sequece of radom variables (X X) coverges to the (costat) radom variable zero. The zero radom variable has the property that its secod momet is zero. This leads us to the followig defiitio: Defiitio 7.2 The sequece (X ) is said to coverge to X i the mea-square (;-(&;") if lim E X X 2 = 0. We write X m.s X. I words: the sequece (X X) coverges to zero i the mea-square if its secod momets coverge to zero. A third mode of covergece higes of the fact that we would relate the fact that (X ) coverges to X with the fact that for every > 0, the probability that X X > should teds to zero as : Defiitio 7.3 The sequece of radom variables (X ) is said to coverge to the radom variable X i probability (;&9";2%") if for every > 0, We write X Pr X. lim P({! X (!) X(!) > }) = 0. You might thik that this coicides with the mode of covergece we have already defied almost-sure covergece. We will see that this is ot the case. Covergece i probability di ers substatially from almost-sure covergece. Also, here too, we might replace > 0 by 1k for k N. Fially, we might say that the sequece (X ) coverges to X is the distributio of X coverges to the distributio of X. I this case, we do t eve eed all the radom variables to be defied o the same probability space; each variable could, i priciple, belog to a separate world. Defiitio 7.4 The sequece of radom variables (X ) is said to coverge to the radom variable X i distributio (;&#-5;%") if for every cotiuity poit a R of
4 166 Chapter 7 F X, lim F X (a) = F X (a), i.e., if the sequece of distributio fuctios of the X coverges poit-wise to the D distributio fuctio of X at all poits where F X is cotiuous. We write X X. The first questio to be addressed is whether there exists a hierarchy of modes of covergece. We wat to kow which modes of covergece imply which. The aswer is that both almost-sure ad mea-square covergece imply covergece i probability, which i tur implies covergece i distributio. O the other had, almost-sure ad mea-square covergece do ot imply each other. Propositio 7.1 Almost-sure covergece implies covergece i probability. Proof : If X a.s X, the By the Fatou lemma, hece P lim sup {! X (!) X(!) > } = 0 > 0. lim sup P (A ) P lim sup A, lim sup P ({! X (!) X(!) > }) 0 > 0. Sice the sequece P ({! X (!) X(!) > }) is o-egative, it follows as oce that lim P ({! X (!) X(!) > }) = 0 > 0, i.e., X Pr X. Propositio 7.2 Mea-square covergece implies covergece i probability.
5 Limit Theorems 167 m.s Proof : This is a immediate cosequece of the Markov iequality. Let X X, the for every > 0, P({! X (!) X(!) > }) = P({! X (!) X(!) 2 > 2 }) EX X 2 2, i.e., which implies that X lim P({! X (!) X(!) > }) = 0, Pr X. Propositio 7.3 Mea-square covergece does ot imply almost-sure covergece. Proof : All we eed is a couter example. Cosider a family of idepedet Beroulli variables X with atomistic distributios, 1 x= 1 p X (x) = 1 1 x= 0. The larger, the more it is likely that X(!) = 0. Thus, it seems sesible to guess that the sequece of radom variables (X ) coverges to the (costat) radom variable X = 0. The questio is i what sese does this covergece occur. m.s First, we show that X X. Ideed, E[X X 2 ] = E[X 2 ] = 1 0. O the other had, (X ) does ot coverge to X almost-surely. Sice for = 12, =1 P({! X (!) X(w) > 12}) = it follows from the secod lemma of Borel-Catelli that 1 =1 =, P lim sup({! X (!) X(w) > 12} = 1.
6 168 Chapter 7 Propositio 7.4 Almost-sure covergece does ot imply mea-square covergece. Proof : Agai, we costruct a couter example, this time takig 1 p X (x) = 2 x = x = 0., Oce agai, the larger, the more it is likely that X(!) = 0. We immediately see that X does ot coverge to X i the mea-square, sice Yet, X a.s X. For every > 0, =1 EX X 2 = E[X 2 ] = 6 2. P({! X (!) X(!) > }) = hece by the first lemma of Borel-Catelli, =1 1 2 <, P lim sup({! X (!) X(!) > } = 0. Corollary 7.1 Covergece i probability does ot imply either almost-sure covergece ot covergece i the mea-square. Proof : Suppose, for example, that covergece i probability implies almostsure covergece. This would mea that covergece i the mea-square implies almost-sure covergece, which cotradicts the last propositio. Fially, we show that covergece i probability implies covergece i distributio, hece both almost-sure covergece ad covergece i the mea-square imply covergece i distributio.
7 Limit Theorems 169 Propositio 7.5 Covergece i probability implies covergece i distributio. Proof : Let a R be give, ad set > 0. O the oe had F X (a) = P (X a) + P (X a, X a + ) + P (X a, X > a + ) = P (X ax a + ) P (X a + ) + P (X a, X > a + ) P (X a + ) + P (X < X ) F X (a + ) + P (X X > ), where we have used the fact that if A B the P(A) P(B). By a similar argumet F X (a ) = P (X a,x a) + P (X a,x > a) Thus, we have obtaied that = P (X a X a) P (X a) + P (X a,x > a) P (X a) + P (X < X ) F X (a) + P (X X > ), F X (a ) P (X X > ) F X (a) F X (a + ) + P (X X > ). Takig ow we have F X (a ) lim if F X (a) lim sup F X (a) F X (a + ). Fially, sice this iequality holds for ay > 0 we coclude that provided that a is a cotiuity poit of F X, lim F X (a) = F X (a). To coclude, the various modes of covergece satisfy the followig scheme:
8 170 Chapter 7 almost surely i the mea square i probability i distributio. Exercise 7.1 Prove that if X coverges i distributio to a costat c, the X coverges i probability to c.. Exercise 7.2 Prove that if X coverges to X i probability the it has a subsequece that coverges to X almost-surely. 7.2 The weak law of large umbers Theorem 7.1 (Weak law of large umbers (.*-&$#%.*952/% -: :-(% 8&(%)) Let X be a sequece of idepedet idetically-distributed radom variables o a probability space (, F, P) ad let µ = E[X i ]. Defie the sequece of ruig averages, S = X 1 + +X. The, S coverges to µ i probability, i.e., for every > 0, lim P ({! S (!) µ > }) = 0. Commet: Take the particular case where X 1, X 2,... are i.i.d., A R ad The, S = 1 i=1 1 X i (!) A Y i = I Xi A = 0 X i (!) A. Y i = fractio of times X i (!) A for 1 i.
9 Limit Theorems 171 The weak law of large umbers states that the fractio of times the outcome is i a give set coverges i probability to E[Y 1 ], which is the probability of this set, P X1 (A), amely, for every > 0, lim P({! S (!) P X1 (A) > }) = 0. Proof : We will prove the weak law uder the additioal assumptio that the radom variables have fiite variace 2 = Var[X i ]. The, the weak law of large umbers is a immediate cosequece of the Chebyshev iequality: by the additivity of the expectatio ad the variace (for idepedet radom variables), The, E[S ] = µ ad Var[S ] = 2. P ({! S (!) µ > }) Var[S ] = 2 which teds to zero as. 2 2, Commet: the first proof is due to Jacob Beroulli (1713), who proved it for the particular case of biomial variables. 7.3 The strog law of large umbers Our ext limit theorem is the strog law of large umber, which states that the ruig average of a sequece of i.i.d. variables coverges to the mea almostsurely (thus stregtheig the weak law of large umbers, which oly provides covergece i probability). Theorem 7.2 (Strog law of large umbers (.*-&$#%.*952/% -: 8'(% 8&(%)) Let X be a sequece of idepedet, idetically distributed radom variables with fiite expectatio µ = E[X i ]. Defie the sequece of ruig averages, S = X 1 + +X. The, S coverges to µ almost-surely, i.e., for every > 0, P({! lim S (!) = µ}) = 1.
10 172 Chapter 7 Proof : We have to prove that for every > 0, P lim sup{! S (!) µ > } = 0, which by the lemma of first Borel-Catelli lemma holds if =1 P ({! S (!) µ > }) <. We will prove the theorem uder the additioal assumptio that the radom variables are bouded, i.e., there is M < such that X i M. Set Y k = X k µ 2M, The reverse relatio is X k = 2MY k + µ. Clearly E[Y k ] = 0, ad by the boudedess assumptio, Y k 1. Hece, we ca use Hoe dig s iequality ad get that for every ad a By our defiitio of Y k, P Y k a exp a2 2 {! S (!) µ > } =! 1 X k µ > =! 1 =! 2M! =! 2M (X k µ) > Y k > Y k! 2M Y k 2M! By Hoe dig s iequality (oce for (Y k ) ad oce for ( Y k )), Y k Y k 2M. which is ideed summable. P ({! S (!) µ > }) 2 exp 2 8M 2,
11 7.4 The cetral limit theorem Limit Theorems 173 Theorem 7.3 (Cetral limit theorem (*',9/% -&"#% )5:/)) Let (X ) be a sequece of i.i.d. radom variables with E[X i ] = 0 ad Var[X i ] = 1. The, the sequece of radom variables S = X 1 + +X coverges i distributio to a radom variable X N (0, 1). That is, for every a R, lim P (S 1 a a) = e y22 dy. 2 Commets: If E[X i ] = µ ad Var[X i ] = 2 the the same applies for S = X 1 + +X µ = 1 X i µ. i=1 The cetral limit theorem (CLT) is about a ruig average rescaled by a factor of. If we deote by Y the ruig average, the the CLT states that Y = X 1 + +X, P Y a (a), i.e., it provides a estimate of the distributio of Y at distaces O( 12 ) from its mea. It is a theorem about small deviatios from the mea. There exist more sophisticated theorems about the distributio of Y far from the mea, part of the so-called theory of large deviatios. There are may variats of this theorem.
12 174 Chapter 7 Proof : We will use the followig fact, which we wo t prove: if the sequece of momet geeratig fuctios M X (t) of a sequece of radom variables (X ) coverges for every t to the momet geeratig fuctio M X (t) of a radom variable X, the X coverges to X i distributio. I other words, M X (t) M X (t) for all t implies that X D X. Thus, we eed to show that the momet geeratig fuctios of the S s teds as to exp(t 2 2), which is the momet geeratig fuctio of a stadard ormal variable. Recall that the pdf of a sum of two radom variables is the covolutio of their pdf, but the momet geeratig fuctio of their sum is the product of the their momet geeratig fuctio. Iductively, M X1 +X ,+X (t) = i=1 M Xi (t) = [M X1 (t)], where we have used the fact that they are i.i.d., Now, if a radom variable Y has a momet geeratig fuctio M Y, the but sice f Ya (y) = af Y (ay) we get that M Ya (t) = R e ty f Ya (y) dy, M Ya (t) = a R e ty f Y (ay) dy = R e atya f Y (ay) d(ay) = M Y (ta), from which we deduce that M S (t) = M X1 t. Take the logarithm of both sides, ad write the left had side explicitly, log M S (t) = log R e tx f X1 (x) dx. Taylor expadig the expoetial about t = 0 we have, log M S (t) = log R 1 + tx + t2 x t3 x 3 6 = log t2 2 + O( 32 ) = t2 2 + O( 32 ) t2 2. e x f 32 X1 (x) dx
13 Limit Theorems 175 Example: Suppose that a experimetalist wats to measure some quatity. He kows that due to various sources of errors, the result of every sigle measuremet is a radom variable, whose mea µ is the correct aswer, ad the variace of his measuremet is 2. He therefore performs idepedet measuremets ad averages the results. How may such measuremets does he eed to perform to be sure, withi 95% certaity, that his estimate does ot deviate from the true result by 4? The questio we re askig is how large should be i order for the iequality P µ 4 1 X k µ to hold. This is equivalet to askig what should be for P 4 1 X k µ By the cetral limit theorem the right had side is, for large, approximately which turs out to be larger tha 0.95 for 62. e y2 2 dy, The problem with this argumet that it uses the assumptio that is large, but it is ot clear what large is. Is = 62 su cietly large for this argumet to hold? This problem could have bee solved without this di culty but resortig istead to the Chebyshev iequality: P 4 1 X k µ 4 = 1 P 1 X k µ , ad the right had side is larger tha 0.95 if = 320.
14 176 Chapter 7 Example: The umber of studets X who are goig to fail i the exam is a Poisso variable with mea 100, i.e, X Poi(100). I am goig to admit that the exam was too hard if more tha 120 studet fail. What is the probability for it to happe? We kow the exact aswer, P (X 120) = e k k!, which is a quite useless expressio. Let s base our estimate o the cetral limit theorem as follows: a Poisso variable with mea 100 ca be expressed as the sum of oe hudred idepedet variables X k Poi(1) (the sum of idepedet Poisso variables is agai a Poisso variable), that is X = 100 X k. Now, P (X 120) = P X k , which by the cetral limit theorem equals approximately, P (X 120) e y2 2 dy Example: Let us examie umerically a particular example. Let X i Exp (1) be idepedet expoetial variable ad set S = 1 (X i 1). i=1 A sum of idepedet expoetial variables has distributio Gamma (, 1), i.e., its pdf is x 1 e x (). The desity for this sum shifted by is with x > ad after dividig by, (x + ) 1 e (x+), () f S (x) = ( x + ) 1 e ( x+), ()
15 Limit Theorems =1 0.6 = = = Figure 7.1: The approach of a ormalized sum of 1, 2, 4 ad 16 expoetial radom variables to the ormal distributio.
16 178 Chapter 7 with x >. See Figure 7.1 for a visualizatio of the approach of the distributio of S toward the stadard ormal distributio.
7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationEE 4TM4: Digital Communications II Probability Theory
1 EE 4TM4: Digital Commuicatios II Probability Theory I. RANDOM VARIABLES A radom variable is a real-valued fuctio defied o the sample space. Example: Suppose that our experimet cosists of tossig two fair
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationAMS570 Lecture Notes #2
AMS570 Lecture Notes # Review of Probability (cotiued) Probability distributios. () Biomial distributio Biomial Experimet: ) It cosists of trials ) Each trial results i of possible outcomes, S or F 3)
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationThe Central Limit Theorem
Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationProbability for mathematicians INDEPENDENCE TAU
Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet
More informationApplication to Random Graphs
A Applicatio to Radom Graphs Brachig processes have a umber of iterestig ad importat applicatios. We shall cosider oe of the most famous of them, the Erdős-Réyi radom graph theory. 1 Defiitio A.1. Let
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More informationLimit Theorems. Convergence in Probability. Let X be the number of heads observed in n tosses. Then, E[X] = np and Var[X] = np(1-p).
Limit Theorems Covergece i Probability Let X be the umber of heads observed i tosses. The, E[X] = p ad Var[X] = p(-p). L O This P x p NM QP P x p should be close to uity for large if our ituitio is correct.
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationLecture 7: Properties of Random Samples
Lecture 7: Properties of Radom Samples 1 Cotiued From Last Class Theorem 1.1. Let X 1, X,...X be a radom sample from a populatio with mea µ ad variace σ
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationJanuary 25, 2017 INTRODUCTION TO MATHEMATICAL STATISTICS
Jauary 25, 207 INTRODUCTION TO MATHEMATICAL STATISTICS Abstract. A basic itroductio to statistics assumig kowledge of probability theory.. Probability I a typical udergraduate problem i probability, we
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More information5. INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY
IA Probability Let Term 5 INEQUALITIES, LIMIT THEOREMS AND GEOMETRIC PROBABILITY 51 Iequalities Suppose that X 0 is a radom variable takig o-egative values ad that c > 0 is a costat The P X c E X, c is
More information2.1. Convergence in distribution and characteristic functions.
3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More informationExercise 4.3 Use the Continuity Theorem to prove the Cramér-Wold Theorem, Theorem. (1) φ a X(1).
Assigmet 7 Exercise 4.3 Use the Cotiuity Theorem to prove the Cramér-Wold Theorem, Theorem 4.12. Hit: a X d a X implies that φ a X (1) φ a X(1). Sketch of solutio: As we poited out i class, the oly tricky
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More information1 The Haar functions and the Brownian motion
1 The Haar fuctios ad the Browia motio 1.1 The Haar fuctios ad their completeess The Haar fuctios The basic Haar fuctio is 1 if x < 1/2, ψx) = 1 if 1/2 x < 1, otherwise. 1.1) It has mea zero 1 ψx)dx =,
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationLecture 3: August 31
36-705: Itermediate Statistics Fall 018 Lecturer: Siva Balakrisha Lecture 3: August 31 This lecture will be mostly a summary of other useful expoetial tail bouds We will ot prove ay of these i lecture,
More informationThe natural exponential function
The atural expoetial fuctio Attila Máté Brookly College of the City Uiversity of New York December, 205 Cotets The atural expoetial fuctio for real x. Beroulli s iequality.....................................2
More informationSDS 321: Introduction to Probability and Statistics
SDS 321: Itroductio to Probability ad Statistics Lecture 23: Cotiuous radom variables- Iequalities, CLT Puramrita Sarkar Departmet of Statistics ad Data Sciece The Uiversity of Texas at Austi www.cs.cmu.edu/
More informationProbability: Limit Theorems I. Charles Newman, Transcribed by Ian Tobasco
Probability: Limit Theorems I Charles Newma, Trascribed by Ia Tobasco Abstract. This is part oe of a two semester course o measure theoretic probability. The course was offered i Fall 2011 at the Courat
More information6 Infinite random sequences
Tel Aviv Uiversity, 2006 Probability theory 55 6 Ifiite radom sequeces 6a Itroductory remarks; almost certaity There are two mai reasos for eterig cotiuous probability: ifiitely high resolutio; edless
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationECE534, Spring 2018: Final Exam
ECE534, Srig 2018: Fial Exam Problem 1 Let X N (0, 1) ad Y N (0, 1) be ideedet radom variables. variables V = X + Y ad W = X 2Y. Defie the radom (a) Are V, W joitly Gaussia? Justify your aswer. (b) Comute
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationSolutions: Homework 3
Solutios: Homework 3 Suppose that the radom variables Y,...,Y satisfy Y i = x i + " i : i =,..., IID where x,...,x R are fixed values ad ",...," Normal(0, )with R + kow. Fid ˆ = MLE( ). IND Solutio: Observe
More informationIntroduction to Extreme Value Theory Laurens de Haan, ISM Japan, Erasmus University Rotterdam, NL University of Lisbon, PT
Itroductio to Extreme Value Theory Laures de Haa, ISM Japa, 202 Itroductio to Extreme Value Theory Laures de Haa Erasmus Uiversity Rotterdam, NL Uiversity of Lisbo, PT Itroductio to Extreme Value Theory
More informationn outcome is (+1,+1, 1,..., 1). Let the r.v. X denote our position (relative to our starting point 0) after n moves. Thus X = X 1 + X 2 + +X n,
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 9 Variace Questio: At each time step, I flip a fair coi. If it comes up Heads, I walk oe step to the right; if it comes up Tails, I walk oe
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2016 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationTopic 9: Sampling Distributions of Estimators
Topic 9: Samplig Distributios of Estimators Course 003, 2018 Page 0 Samplig distributios of estimators Sice our estimators are statistics (particular fuctios of radom variables), their distributio ca be
More informationChapter 6 Principles of Data Reduction
Chapter 6 for BST 695: Special Topics i Statistical Theory. Kui Zhag, 0 Chapter 6 Priciples of Data Reductio Sectio 6. Itroductio Goal: To summarize or reduce the data X, X,, X to get iformatio about a
More informationMAS111 Convergence and Continuity
MAS Covergece ad Cotiuity Key Objectives At the ed of the course, studets should kow the followig topics ad be able to apply the basic priciples ad theorems therei to solvig various problems cocerig covergece
More informationfor all x ; ;x R. A ifiite sequece fx ; g is said to be ND if every fiite subset X ; ;X is ND. The coditios (.) ad (.3) are equivalet for =, but these
sub-gaussia techiques i provig some strog it theorems Λ M. Amii A. Bozorgia Departmet of Mathematics, Faculty of Scieces Sista ad Baluchesta Uiversity, Zaheda, Ira Amii@hamoo.usb.ac.ir, Fax:054446565 Departmet
More informationIntroduction to Probability. Ariel Yadin. Lecture 7
Itroductio to Probability Ariel Yadi Lecture 7 1. Idepedece Revisited 1.1. Some remiders. Let (Ω, F, P) be a probability space. Give a collectio of subsets K F, recall that the σ-algebra geerated by K,
More informationRandom Models. Tusheng Zhang. February 14, 2013
Radom Models Tusheg Zhag February 14, 013 1 Radom Walks Let me describe the model. Radom walks are used to describe the motio of a movig particle (object). Suppose that a particle (object) moves alog the
More informationOutput Analysis and Run-Length Control
IEOR E4703: Mote Carlo Simulatio Columbia Uiversity c 2017 by Marti Haugh Output Aalysis ad Ru-Legth Cotrol I these otes we describe how the Cetral Limit Theorem ca be used to costruct approximate (1 α%
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationMATH 472 / SPRING 2013 ASSIGNMENT 2: DUE FEBRUARY 4 FINALIZED
MATH 47 / SPRING 013 ASSIGNMENT : DUE FEBRUARY 4 FINALIZED Please iclude a cover sheet that provides a complete setece aswer to each the followig three questios: (a) I your opiio, what were the mai ideas
More informationParameter, Statistic and Random Samples
Parameter, Statistic ad Radom Samples A parameter is a umber that describes the populatio. It is a fixed umber, but i practice we do ot kow its value. A statistic is a fuctio of the sample data, i.e.,
More informationEcon 325 Notes on Point Estimator and Confidence Interval 1 By Hiro Kasahara
Poit Estimator Eco 325 Notes o Poit Estimator ad Cofidece Iterval 1 By Hiro Kasahara Parameter, Estimator, ad Estimate The ormal probability desity fuctio is fully characterized by two costats: populatio
More information1+x 1 + α+x. x = 2(α x2 ) 1+x
Math 2030 Homework 6 Solutios # [Problem 5] For coveiece we let α lim sup a ad β lim sup b. Without loss of geerality let us assume that α β. If α the by assumptio β < so i this case α + β. By Theorem
More informationAxioms of Measure Theory
MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationUniversity of Colorado Denver Dept. Math. & Stat. Sciences Applied Analysis Preliminary Exam 13 January 2012, 10:00 am 2:00 pm. Good luck!
Uiversity of Colorado Dever Dept. Math. & Stat. Scieces Applied Aalysis Prelimiary Exam 13 Jauary 01, 10:00 am :00 pm Name: The proctor will let you read the followig coditios before the exam begis, ad
More informationd) If the sequence of partial sums converges to a limit L, we say that the series converges and its
Ifiite Series. Defiitios & covergece Defiitio... Let {a } be a sequece of real umbers. a) A expressio of the form a + a +... + a +... is called a ifiite series. b) The umber a is called as the th term
More informationLecture 6 Simple alternatives and the Neyman-Pearson lemma
STATS 00: Itroductio to Statistical Iferece Autum 06 Lecture 6 Simple alteratives ad the Neyma-Pearso lemma Last lecture, we discussed a umber of ways to costruct test statistics for testig a simple ull
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationEFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS
EFFECTIVE WLLN, SLLN, AND CLT IN STATISTICAL MODELS Ryszard Zieliński Ist Math Polish Acad Sc POBox 21, 00-956 Warszawa 10, Polad e-mail: rziel@impagovpl ABSTRACT Weak laws of large umbers (W LLN), strog
More informationSolution to Chapter 2 Analytical Exercises
Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationSeunghee Ye Ma 8: Week 5 Oct 28
Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value
More informationA Proof of Birkhoff s Ergodic Theorem
A Proof of Birkhoff s Ergodic Theorem Joseph Hora September 2, 205 Itroductio I Fall 203, I was learig the basics of ergodic theory, ad I came across this theorem. Oe of my supervisors, Athoy Quas, showed
More informationLaw of the sum of Bernoulli random variables
Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible
More informationSolutions to HW Assignment 1
Solutios to HW: 1 Course: Theory of Probability II Page: 1 of 6 Uiversity of Texas at Austi Solutios to HW Assigmet 1 Problem 1.1. Let Ω, F, {F } 0, P) be a filtered probability space ad T a stoppig time.
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More information5. Likelihood Ratio Tests
1 of 5 7/29/2009 3:16 PM Virtual Laboratories > 9. Hy pothesis Testig > 1 2 3 4 5 6 7 5. Likelihood Ratio Tests Prelimiaries As usual, our startig poit is a radom experimet with a uderlyig sample space,
More information