MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
|
|
- Imogen May
- 5 years ago
- Views:
Transcription
1 MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece 2. Fuctioal Strog Law of Large Numbers 3. Existece of Wieer measure (Browia motio) 1 Additioal techical results o weak covergece Give two metric spaces S 1, S 2 ad a measurable fuctio f : S 1 S 2, suppose S 1 is equipped with some probability measure P. This iduces a probability measure o S 2 which is deoted by Pf 1 ad is defied by Pf 1 (A) = P(f 1 (A)) for every measurable set A S 2. The for ay radom variable X : S 2 R, its expectatio E Pf 1 [X] is equal to E P [X(f)]. (Covice yourself that this is the case by lookig at the special case whe f is a simple fuctio). Theorem 1 (Mappig Theorem). Suppose P P for a sequece of probability measures P, P o S 1 ad suppose f : S 1 S 2 is cotiuous. The P f 1 Pf 1 o S 2. Proof. We use Portmetau theorem, i particular weak covergece characterizatio usig bouded cotiuous fuctios. Thus let g : S 2 R be ay bouded cotiuous fuctio. Sice it is cotiuous, it is also measurable, thus it is also a radom variable defied o (S 2, B 2 ), where B 2 is the Borel σ-field o S 2. We have, E Pf 1 [g] = E P [g(f)]. Sice g is a bouded cotiuous, the the compositio is also bouded cotiuous. Therefore, by Portmateau theorem E P [g(f)] E P [g(f)] = E Pf 1 [g] 1
2 Defiitio 1. A sequece of probability measures P o metric space S is defied to be tight if for every E > 0 there exists 0 ad a compact set K S, such that P (K) > 1 E for all > 0. Theorem 2 (Prohorov s Theorem). Suppose sequece P is tight. The it cotais a weakly coverget subsequece P (k) P. The coverse of this theorem is also true, but we will ot eed this. We do ot prove Prohorov s Theorem. The proof ca be foud i [1]. Recall that Arzela-Ascoli Theorem provides a characterizatio of compact sets i C[0, T ]. We ca use it ow for characterizatio of tightess. Propositio 1. Suppose a sequece of measures P o C[0, T ] satisfies the followig coditios: (i) There exists a 0 such that lim P ( x(0) a) = 0. (ii) For each E > 0, lim δ 0 lim sup P ({x : w x (δ) > E}) = 0. The the sequece P is tight. Proof. Fix E > 0. From (i) we ca fid â ad 0 large eough so that P ( x(0) > â) < E for all > 0. For every m 0 we ca also fid a m large eough so that P m ( x(0) > a m ) < E. Take a = max(â, a m ). The P ( x(0) > a) < E for all. Let B = {x : x(0) a}. We just showed P (B) 1 E. Similarly, for every k > 0 we ca fid δˆk ad k such that P (w x (δˆk) > E/2 k ) < E/2 k for all > k. For every fixed k we ca fid a small eough δ > 0 such that P (w x (δ ) > E/2 k ) < E/2 k sice by uiform cotiuity of x we have δ>0 {w x (δ) > E/2 k } = Ø a.s. Let δ k = mi(δˆk, mi k δ ). Let B k = {x : w x (δ k ) E}. Sice P (B c ) < E/2 k the P ( k Bc k k ) < E ad P ( k B k ) 1 E, for all. Therefore P (B k B k ) 1 2E for all. The set K = B k B k is closed (check) ad satisfies the coditios of Arzela-Ascoli Theorem. Therefore it is compact. 2 Fuctioal Strog Law of Large Numbers (FSLLN) We are about to establish two very importat limit results i the theory of stochastic processes. I probability theory two corerstoe theorems are (Weak or Strog) Law of Large Numbers ad Cetral Limit Theorem. These theorems have direct aalogue i the theory of stochastic processes as Fuctioal Strog Law of Large Numbers (FSLLN) ad Fuctioal Cetral Limit Theorem (FCLT) also kow as Dosker Theorem. The secod theorem cotais i it the fact that Wieer Measure exists. 2
3 We first describe the setup. Cosider a sequece of i.i.d. radom variables X 1, X 2,..., X,.... We assume that E[X 1 ] = 0, E[X 2 1 ] = σ 2. We ca view each realizatio of a ifiite sequece (X (ω)) as a sample i a product space R equipped with product type σ-field F ad probability measure P i.i.d., iduced by the probability distributio of X 1. Defie S = 1 X k. Fix a iterval [0, T ] ad for each 1 ad t [0, T ] cosider the followig fuctio S t (ω) X t +1(ω) N (t) = + (t t ). (1) This is a piece-wise liear cotiuous fuctio i C[0, T ]. Theorem 3 (Fuctioal Strog Law of Large Numbers (FSLLN)). Give a i.i.d. sequece (X ), 1 with E[X 1 ] = 0, E[ X 1 ] <, for every T > 0, the sequece of fuctios N : [0, T ] R coverges to zero almost surely. Namely P(IN (ω)i T 0) = P( sup N (t, ω) 0) = 1 0 t T As we see, just as SLLN, the FSLLN holds without ay assumptios o the variace of X 1, that is eve if σ =. Here is aother way to state FSLLN. We may cosider fuctios N defied o etire [0, ) usig the same defiig idetity (1). Recall that sets [0, T ] are compact i R. A equivalet way of sayig FSLLN is N coverges to zero almost surely uiformly o compact sets. Proof. Fix E > 0 ad T > 0. By SLLN we have that for almost all realizatios ω of a sequece X 1 (ω), X 2 (ω),..., there exists 0 (ω) such that for all > 0 (ω), S (ω) E < T We let M(ω) = max 1 m 0 (ω) S m (ω). We claim that for > M(ω)/E, there holds sup N (t) < E. 0 t T We cosider two cases. Suppose t [0, T ] is such that t > 0 (ω). The ( S t (ω) S t +1(ω) ) N (t) max,. 3
4 We have S t (ω) S t (ω) t ɛ = t T t ɛ. Usig a similar boud o S t +1 (ω), we obtai N (t) ɛ. Suppose ow t is such that t 0 (ω). The M(ω) N (t) < ɛ, sice, by our choice > M(ω)/ɛ. We coclude sup 0 t T N (t) < ɛ for all > M(ω)/ɛ. This cocludes the proof. 3 Weier measure FSSLN was a simpler fuctioal limit theorem. Here we cosider istead a Gaussia scalig of a radom walk S ad establish existece of the Weier measure (Browia motio) as well as FCLT. Thus suppose we have a sequece of i.i.d. radom variables X 1,..., X with mea zero but fiite variace σ 2 <. Istead of fuctio (1) cosider the followig fuctio S t (ω) X t +1 (ω) N (t, ω) = + (t t ) σ σ, 1, t [0, T ]. (2) This is agai a piece-wise liear cotiuous fuctio. The for each we obtai a mappig ψ : R C[0, T ]. Of course, for each, the mappig ψ depeds oly o the first T + 1 coordiates of samples i R. Lemma 1. Each mappig ψ is measurable. Proof. Here is where it helps to kow that Kolmogorov field is idetical to Borel field o C[0, T ], that is Theorem 1.4 from the previous lecture. Ideed, ow it suffices to show that that ψ 1 (A) is measurable for each set A of the form 4 4
5 A = π 1 t ψ 1 (π 1 t (, y], as these sets geerate Kolmogorov/Borel σ-field. Each set (, y]) is the set of all realizatios N (ω) such that 1 k N m X k(ω) (t, ω) = + (t m)xm+1 (ω) y. σ where m = t. This defies a measurable subset of R m+1 R. Oe way to see this is to observe that the fuctio f : R m+1 R defied by x f(x 1,..., x m ) = 1 k m k σ + (t m)x m+1 is cotiuous ad therefore is measurable. We coclude that ψ is measurable for each. Thus each ψ iduces a probability measure o C[0, T ], which we deote by P. This probability measure is defied by P (A) = P i.i.d. (ψ 1 (A)) = P i.i.d. (N (ω) A). We ow establish the pricipal result of this lecture existece of Weier measure, amely, the existece of a Browia motio. Theorem 4 (Existece of Wieer measure). A sequece of measures P has a weak limit P which satisfies the property of Wieer measure o C[0, T ]. The proof of this fact is quite ivolved ad we give oly its scheme, skippig some techical results. First let us outlie the mai steps i the proof. I the previous lecture we cosidered projectio mappigs P t : C[0, T ] R. Similarly, for ay collectio 0 t 1 < < t k we ca cosider π t1,...,t k (x) = (x(t 1 ),..., x(t k )) R k. 1. We first show that the sequece of measures P o C[0, T ] is tight. We use this to argue that there exists a subsequece P (k) which coverges to some measure π. 2. We show that π satisfies the properties of Wieer measures. For this purposes we look at the projected measures π t1,...,t k (π ) o R k ad show that these give a joit Gaussia distributio, the kid arisig i a Browia motio (that is the joit distributio of (B(t 1 ), B(t 2 ),..., B(t k ))). At this poit the existece of Wieer measure is established. 3. We the show that i fact the weak covergece π π holds. Proof sketch. We begi with the followig techical ad quite delicate result about radom walks. 5
6 Lemma 2. The followig idetity holds for radom walks S = X 1 + +X : lim lim sup λ 2 P(max S k λσ ) = 0. (3) λ Note, that this is ideed a very subtle result. We could try to use submartigale iequality, sice S k is sub-martigale. It will give E[S 2 ] 1 P(max S k λσ ) =. λ 2 σ 2 λ 2 So by takig a product with λ 2 we do ot obtai covergece to zero. O the other had ote that if the radom variables have a fiite fourth momet E[X 4 ] <, the the result follows from the sub-martigale iequality by co siderig S 4 i place of S 2 (exercise). The proof of this lemma is based o the followig fact: Propositio 2 (Etemadi s Iequality,[1]). For every α > 0 P(max S k 3α) 3 max P( S k α) Proof. Let B k be the evet S k 3α, S j < 3α, j < k. The P(max S k 3α) P( S α) + P(B k S < α) P( S α) + P(B k S S k > 2α) = P( S α) + P(B k )P( S S k > 2α) P( S α) + max P( S S k 2α) P( S α) + max (P( S α) + P( S k α)) 3 max P( S k α). Now we ca prove Lemma 2. Proof of Lemma 2. Applyig Etemadi s Iequality P(max S k λσ ) 3 max P( S k (1/3)λσ ) 6
7 Fix E > 0. Let Φ deote the cumulative stadard ormal distributio. Fid λ 0 large eough so that 2λ 2 (1 Φ(λ/3)) < E/3 for all λ λ 0. Fix ay such λ. By the CLT we ca fid 0 = 0 (λ) large eough so that P( S (1/3)λσ ) 2(1 Φ(λ/3)) + E/(3λ 2 ) for all 0, implyig λ 2 P( S (1/3)λσ ) 2E/3. Now fix ay 27 0 /E ad ay k. If k 0, the from the derived boud we have λ 2 P( S k (1/3)λσ ) λ 2 P( S k (1/3)λσ k) 2E/3. O the other had, if k 0 the λ 2 E[S 2 ] σ 2 k k λ 2 P( S k (1/3)λσ ) = E/3. (λ 2 /9)σ 2 (1/9)σ 2 We coclude that for all 27 0 /E, λ 2 max P( S k (1/3)λσ ) E/3, 1 from which we obtai λ 2 lim sup P(max S k λσ ) E Sice E > 0 was arbitrary, we obtai the result. The ext result which we also do ot prove says that the property (3) implies tightess of the sequece of measures P o C[0, T ]. Lemma 3. The followig covergece holds for every E > 0. lim lim sup P(w N (δ) E) = 0. (4) δ 0 As a result the sequece of measures P is tight. Proof. Observe that X k i k j w N (δ) max. i j T :j i δ σ Exercise 1. Use this to fiish the proof of the lemma. Hit: partitio iterval [0, T ] ito legth δ itervals ad use Lemma 2. 7
8 Let us ow see how Lemma 2 implies tightess. We use characterizatio give by Propositio 1. First for ay positive a, P ( x(0) a) = P( N (0) > 0) = 0 sice N (0) = 0. Now P (w x (δ) > ɛ) = P(w N (δ) > ɛ). From the first part of the lemma we kow that the double covergece (4) holds. This meas that coditio (ii) of Propositio 1 holds as well. We ow retur to the costructio of Wieer measure. Lemma 3 implies that the sequece of probability measures P o C[0, T ] is tight. Therefore, by Prohorov s Theorem, it cotais a weakly coverget subsequece P (k) P. Propositio 3. P satisfies the property of the Wieer measure. Proof. Sice P is defied o the space of cotiuous fuctios, the cotiuity of every sample is immediate. We eed to establish idepedece of icremets ad the fact that icremets are statioary Gaussia. Thus we fix 0 t 1 < < t k ad y 1,..., y k R k. To preserve the cotiuity, we still deote elemets of C[0, T ] by x, x(t) or x(ω, t) whereas before we used otatios ω, B(ω), B(t, ω). Cosider the radom vector π t1 (N ) = N (t 1 ). This is simply the radom variable S t 1 ( ω) X ω) t + (t ( t 1 ) σ σ The secod term i the sum coverges to zero i probability. The first term we rewrite as S (ω) t t 1 σ 1. t 1 ad by CLT it coverges to a ormal N(0, t 1 ) distributio. Similarly, cosider t X m(ω) 1<m t X t (ω) N (t 2) N (t 1) = (t 2 t2 ) σ σ X t 1 +1(ω) (t1 t 1 ) σ Agai by CLT we see that it coverges to ormal N(0, t 2 t 1 ) distributio. Moreover, the joit distributio of (N (t 1 ), N (t 2 ) N (t 1 )) coverges to a joit distributio of two idepedet ormals with zero mea ad variaces t 1, t 2 t 1. Namely, the (N (t 1 ), N (t 2 )) coverges i distributio to 8
9 (Z 1, Z 1 + Z 2 ), where Z 1, Z 2 are idepedet ormal radom variables with zero mea ad variaces t 1, t 2 t 1. By a similar toke, we see that the distributio of the radom vector π t1,...,t k (N ) = (N (t 1 ), N (t 2 ),..., N (t k )) coverges i distributio to (Z 1, Z 1 + Z 2,..., Z Z k ) where Z j, 1 j k are idepedet zero mea ormal radom variables with variaces t 1, t 2 t 1,..., t k t k 1. O the other had the distributio of π t1,...,t k (N ) is P π 1 t 1,...,t k = P i.i.d. ψ 1 πt 1 1,...,t k. Sice P (k) P ad π is cotiuous, the, applyig mappig theorem (Theorem 1) we coclude that P (k) πt 1 P π 1 1,...,t k t 1,...,t k. Combiig, these two facts, we coclude that the probability measure P πt 1 1,...,t k is the probability measure of (Z 1, Z 1 + Z 2,..., Z Z k ) (where agai Z j are idepedet ormal, etc...). What does this mea? This meas that whe we select x C[0, T ] accordig to the probability measure P ad look at its projectio π t1,...,t k (x) = (x(t 1 ),..., x(t k )), the probability distributio of this radom vector is the distributio of (Z 1, Z 1 + Z 2,..., Z Z k ). This meas that x has idepedet icremets with zero mea Gaussia distributio ad variaces t 1, t 2 t 1,..., t k t k 1. This is precisely the property we eeded to establish for P i order to argue that it is ideed Wieer measure. This cocludes the proof of Propositio 3 the fact that P is the Weier measure. We also eed to show that the covergece P P holds. For this purpose we will show that P is uique. I this case the covergece holds. Ideed, suppose otherwise, there exists a subsequece (k) such that P (k) P. The we ca fid a bouded cotiuous r.v. X, such that E P(k) X E P X. The we ca fid E 0 ad a subsequece (k i ) of (k) such that E P(ki ) X E P X E 0 for all i. By tightess we ca fid a further subsequece (k ij ) of (k i ) which coverges weakly to some limitig probability measure P. But we have see that every such weak limit has to be a Wieer measure which is uique. Namely P = P. This is a cotradictio sice E P X E P X E 0. (k ij ) It remais to show the uiqueess of Wieer measure. This follows agai from the fact that the Kolmogorov σ-field coicides with the Borel σ-field o C[0, T ]. But the properties of Wieer measure (idepedet icremets with variaces give by the legth of the time icremets) uiquely defie probability o geeratig sets obtaied via projectios π t1,...,t k : C[0, T ] R k. This cocludes the proof of uiqueess. 9
10 4 Applicatios Theorem 4 has applicatios beyod the existece of Wieer measure. Here is oe of them. Theorem 5. The followig covergece holds max 1 S k sup B(t) (5) σ 0 t T where B is the stadard Browia motio. As a result, for every y max 1 S k lim P( y) = 2(1 Φ(y)), (6) σ where Φ is stadard ormal distributio. Proof. The fuctio g(x) = sup 0 t T x(t) is a cotiuous fuctio o C[0, T ] (check this). Sice by Theorem 4, P P the, by Mappig Theorem, g(n ) g(b), where B is a stadard Browia motio radom sample max 1 S from the Wieer measure P. But g(n k ) = sup 0 t T N (t) =. σ We coclude that (5) holds. To prove the secod part we ote that the set A = {x C[0, T ] : sup 0 t T x(t) = y} has P (A) = 0 recall that the maximum sup 0 t T B(t) of a Browia motio has desity. Also ote that A is the boudary of the set {x : sup 0 t T x(t) y}. Therefore by Portmetau theorem, (6) holds. 5 Additioal readig materials Billigsley [1] Chapter 2, Sectio 8. Refereces [1] P. Billigsley, Covergece of probability measures, Wiley-Itersciece publicatio,
11 MIT OpeCourseWare J / 6.265J Advaced Stochastic Processes Fall 2013 For iformatio about citig these materials or our Terms of Use, visit:
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationNotes 27 : Brownian motion: path properties
Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More information1 The Haar functions and the Brownian motion
1 The Haar fuctios ad the Browia motio 1.1 The Haar fuctios ad their completeess The Haar fuctios The basic Haar fuctio is 1 if x < 1/2, ψx) = 1 if 1/2 x < 1, otherwise. 1.1) It has mea zero 1 ψx)dx =,
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationDonsker s Theorem. Pierre Yves Gaudreau Lamarre. August 2012
Dosker s heorem Pierre Yves Gaudreau Lamarre August 2012 Abstract I this paper we provide a detailed proof of Dosker s heorem, icludig a review of the majority of the results o which the theorem is based,
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationSolutions to HW Assignment 1
Solutios to HW: 1 Course: Theory of Probability II Page: 1 of 6 Uiversity of Texas at Austi Solutios to HW Assigmet 1 Problem 1.1. Let Ω, F, {F } 0, P) be a filtered probability space ad T a stoppig time.
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013. Large Deviations for i.i.d. Random Variables
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 2 9/9/2013 Large Deviatios for i.i.d. Radom Variables Cotet. Cheroff boud usig expoetial momet geeratig fuctios. Properties of a momet
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More informationAn Introduction to Randomized Algorithms
A Itroductio to Radomized Algorithms The focus of this lecture is to study a radomized algorithm for quick sort, aalyze it usig probabilistic recurrece relatios, ad also provide more geeral tools for aalysis
More information6a Time change b Quadratic variation c Planar Brownian motion d Conformal local martingales e Hints to exercises...
Tel Aviv Uiversity, 28 Browia motio 59 6 Time chage 6a Time chage..................... 59 6b Quadratic variatio................. 61 6c Plaar Browia motio.............. 64 6d Coformal local martigales............
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More information5 Birkhoff s Ergodic Theorem
5 Birkhoff s Ergodic Theorem Amog the most useful of the various geeralizatios of KolmogorovâĂŹs strog law of large umbers are the ergodic theorems of Birkhoff ad Kigma, which exted the validity of the
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationThis exam contains 19 pages (including this cover page) and 10 questions. A Formulae sheet is provided with the exam.
Probability ad Statistics FS 07 Secod Sessio Exam 09.0.08 Time Limit: 80 Miutes Name: Studet ID: This exam cotais 9 pages (icludig this cover page) ad 0 questios. A Formulae sheet is provided with the
More informationMath 341 Lecture #31 6.5: Power Series
Math 341 Lecture #31 6.5: Power Series We ow tur our attetio to a particular kid of series of fuctios, amely, power series, f(x = a x = a 0 + a 1 x + a 2 x 2 + where a R for all N. I terms of a series
More information2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationThe Central Limit Theorem
Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx
More informationSolutions to home assignments (sketches)
Matematiska Istitutioe Peter Kumli 26th May 2004 TMA401 Fuctioal Aalysis MAN670 Applied Fuctioal Aalysis 4th quarter 2003/2004 All documet cocerig the course ca be foud o the course home page: http://www.math.chalmers.se/math/grudutb/cth/tma401/
More informationSolution. 1 Solutions of Homework 1. Sangchul Lee. October 27, Problem 1.1
Solutio Sagchul Lee October 7, 017 1 Solutios of Homework 1 Problem 1.1 Let Ω,F,P) be a probability space. Show that if {A : N} F such that A := lim A exists, the PA) = lim PA ). Proof. Usig the cotiuity
More informationLecture 3 : Random variables and their distributions
Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}
More informationMath 525: Lecture 5. January 18, 2018
Math 525: Lecture 5 Jauary 18, 2018 1 Series (review) Defiitio 1.1. A sequece (a ) R coverges to a poit L R (writte a L or lim a = L) if for each ǫ > 0, we ca fid N such that a L < ǫ for all N. If the
More informationf n (x) f m (x) < ɛ/3 for all x A. By continuity of f n and f m we can find δ > 0 such that d(x, x 0 ) < δ implies that
Lecture 15 We have see that a sequece of cotiuous fuctios which is uiformly coverget produces a limit fuctio which is also cotiuous. We shall stregthe this result ow. Theorem 1 Let f : X R or (C) be a
More informationIntroduction to Probability. Ariel Yadin. Lecture 7
Itroductio to Probability Ariel Yadi Lecture 7 1. Idepedece Revisited 1.1. Some remiders. Let (Ω, F, P) be a probability space. Give a collectio of subsets K F, recall that the σ-algebra geerated by K,
More informationChapter 7 Isoperimetric problem
Chapter 7 Isoperimetric problem Recall that the isoperimetric problem (see the itroductio its coectio with ido s proble) is oe of the most classical problem of a shape optimizatio. It ca be formulated
More information2.1. Convergence in distribution and characteristic functions.
3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For
More information1 Introduction to reducing variance in Monte Carlo simulations
Copyright c 010 by Karl Sigma 1 Itroductio to reducig variace i Mote Carlo simulatios 11 Review of cofidece itervals for estimatig a mea I statistics, we estimate a ukow mea µ = E(X) of a distributio by
More informationREAL ANALYSIS II: PROBLEM SET 1 - SOLUTIONS
REAL ANALYSIS II: PROBLEM SET 1 - SOLUTIONS 18th Feb, 016 Defiitio (Lipschitz fuctio). A fuctio f : R R is said to be Lipschitz if there exists a positive real umber c such that for ay x, y i the domai
More informationNotes 19 : Martingale CLT
Notes 9 : Martigale CLT Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall
More informationLaw of the sum of Bernoulli random variables
Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible
More informationProbability for mathematicians INDEPENDENCE TAU
Probability for mathematicias INDEPENDENCE TAU 2013 28 Cotets 3 Ifiite idepedet sequeces 28 3a Idepedet evets........................ 28 3b Idepedet radom variables.................. 33 3 Ifiite idepedet
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationM17 MAT25-21 HOMEWORK 5 SOLUTIONS
M17 MAT5-1 HOMEWORK 5 SOLUTIONS 1. To Had I Cauchy Codesatio Test. Exercise 1: Applicatio of the Cauchy Codesatio Test Use the Cauchy Codesatio Test to prove that 1 diverges. Solutio 1. Give the series
More informationLecture Notes for Analysis Class
Lecture Notes for Aalysis Class Topological Spaces A topology for a set X is a collectio T of subsets of X such that: (a) X ad the empty set are i T (b) Uios of elemets of T are i T (c) Fiite itersectios
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationANSWERS TO MIDTERM EXAM # 2
MATH 03, FALL 003 ANSWERS TO MIDTERM EXAM # PENN STATE UNIVERSITY Problem 1 (18 pts). State ad prove the Itermediate Value Theorem. Solutio See class otes or Theorem 5.6.1 from our textbook. Problem (18
More informationHomework 4. x n x X = f(x n x) +
Homework 4 1. Let X ad Y be ormed spaces, T B(X, Y ) ad {x } a sequece i X. If x x weakly, show that T x T x weakly. Solutio: We eed to show that g(t x) g(t x) g Y. It suffices to do this whe g Y = 1.
More informationSolution to Chapter 2 Analytical Exercises
Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationPrecise Rates in Complete Moment Convergence for Negatively Associated Sequences
Commuicatios of the Korea Statistical Society 29, Vol. 16, No. 5, 841 849 Precise Rates i Complete Momet Covergece for Negatively Associated Sequeces Dae-Hee Ryu 1,a a Departmet of Computer Sciece, ChugWoo
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More informationBIRKHOFF ERGODIC THEOREM
BIRKHOFF ERGODIC THEOREM Abstract. We will give a proof of the poitwise ergodic theorem, which was first proved by Birkhoff. May improvemets have bee made sice Birkhoff s orgial proof. The versio we give
More information32 estimating the cumulative distribution function
32 estimatig the cumulative distributio fuctio 4.6 types of cofidece itervals/bads Let F be a class of distributio fuctios F ad let θ be some quatity of iterest, such as the mea of F or the whole fuctio
More informationAxioms of Measure Theory
MATH 532 Axioms of Measure Theory Dr. Neal, WKU I. The Space Throughout the course, we shall let X deote a geeric o-empty set. I geeral, we shall ot assume that ay algebraic structure exists o X so that
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationMathematics 170B Selected HW Solutions.
Mathematics 17B Selected HW Solutios. F 4. Suppose X is B(,p). (a)fidthemometgeeratigfuctiom (s)of(x p)/ p(1 p). Write q = 1 p. The MGF of X is (pe s + q), sice X ca be writte as the sum of idepedet Beroulli
More informationIntroduction to Probability. Ariel Yadin. Lecture 2
Itroductio to Probability Ariel Yadi Lecture 2 1. Discrete Probability Spaces Discrete probability spaces are those for which the sample space is coutable. We have already see that i this case we ca take
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationThe Boolean Ring of Intervals
MATH 532 Lebesgue Measure Dr. Neal, WKU We ow shall apply the results obtaied about outer measure to the legth measure o the real lie. Throughout, our space X will be the set of real umbers R. Whe ecessary,
More informationProbability and Random Processes
Probability ad Radom Processes Lecture 5 Probability ad radom variables The law of large umbers Mikael Skoglud, Probability ad radom processes 1/21 Why Measure Theoretic Probability? Stroger limit theorems
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationCentral Limit Theorem using Characteristic functions
Cetral Limit Theorem usig Characteristic fuctios RogXi Guo MAT 477 Jauary 20, 2014 RogXi Guo (2014 Cetral Limit Theorem usig Characteristic fuctios Jauary 20, 2014 1 / 15 Itroductio study a radom variable
More informationLecture 4. We also define the set of possible values for the random walk as the set of all x R d such that P(S n = x) > 0 for some n.
Radom Walks ad Browia Motio Tel Aviv Uiversity Sprig 20 Lecture date: Mar 2, 20 Lecture 4 Istructor: Ro Peled Scribe: Lira Rotem This lecture deals primarily with recurrece for geeral radom walks. We preset
More informationSeunghee Ye Ma 8: Week 5 Oct 28
Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value
More informationSieve Estimators: Consistency and Rates of Convergence
EECS 598: Statistical Learig Theory, Witer 2014 Topic 6 Sieve Estimators: Cosistecy ad Rates of Covergece Lecturer: Clayto Scott Scribe: Julia Katz-Samuels, Brado Oselio, Pi-Yu Che Disclaimer: These otes
More informationJournal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula
Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials
More informationsin(n) + 2 cos(2n) n 3/2 3 sin(n) 2cos(2n) n 3/2 a n =
60. Ratio ad root tests 60.1. Absolutely coverget series. Defiitio 13. (Absolute covergece) A series a is called absolutely coverget if the series of absolute values a is coverget. The absolute covergece
More informationAn alternative proof of a theorem of Aldous. concerning convergence in distribution for martingales.
A alterative proof of a theorem of Aldous cocerig covergece i distributio for martigales. Maurizio Pratelli Dipartimeto di Matematica, Uiversità di Pisa. Via Buoarroti 2. I-56127 Pisa, Italy e-mail: pratelli@dm.uipi.it
More informationMATH301 Real Analysis (2008 Fall) Tutorial Note #7. k=1 f k (x) converges pointwise to S(x) on E if and
MATH01 Real Aalysis (2008 Fall) Tutorial Note #7 Sequece ad Series of fuctio 1: Poitwise Covergece ad Uiform Covergece Part I: Poitwise Covergece Defiitio of poitwise covergece: A sequece of fuctios f
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall Midterm Solutions
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.65/5.070J Fall 0 Midterm Solutios Problem Suppose a radom variable X is such that P(X > ) = 0 ad P(X > E) > 0 for every E > 0. Recall that the large deviatios rate
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationEntropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP
Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.
More informationb i u x i U a i j u x i u x j
M ath 5 2 7 Fall 2 0 0 9 L ecture 1 9 N ov. 1 6, 2 0 0 9 ) S ecod- Order Elliptic Equatios: Weak S olutios 1. Defiitios. I this ad the followig two lectures we will study the boudary value problem Here
More informationMeasure and Measurable Functions
3 Measure ad Measurable Fuctios 3.1 Measure o a Arbitrary σ-algebra Recall from Chapter 2 that the set M of all Lebesgue measurable sets has the followig properties: R M, E M implies E c M, E M for N implies
More informationExercise 4.3 Use the Continuity Theorem to prove the Cramér-Wold Theorem, Theorem. (1) φ a X(1).
Assigmet 7 Exercise 4.3 Use the Cotiuity Theorem to prove the Cramér-Wold Theorem, Theorem 4.12. Hit: a X d a X implies that φ a X (1) φ a X(1). Sketch of solutio: As we poited out i class, the oly tricky
More informationProperties of Fuzzy Length on Fuzzy Set
Ope Access Library Joural 206, Volume 3, e3068 ISSN Olie: 2333-972 ISSN Prit: 2333-9705 Properties of Fuzzy Legth o Fuzzy Set Jehad R Kider, Jaafar Imra Mousa Departmet of Mathematics ad Computer Applicatios,
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationLecture 27. Capacity of additive Gaussian noise channel and the sphere packing bound
Lecture 7 Ageda for the lecture Gaussia chael with average power costraits Capacity of additive Gaussia oise chael ad the sphere packig boud 7. Additive Gaussia oise chael Up to this poit, we have bee
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationTHE STRONG LAW OF LARGE NUMBERS FOR STATIONARY SEQUENCES
THE STRONG LAW OF LARGE NUMBERS FOR STATIONARY SEQUENCES Debdeep Pati Idia Statistical Istitute, Kolkata Jue 26, 2006 Abstract The traditioal proof of the strog law of large umbers usig idepedet ad idetically
More informationTENSOR PRODUCTS AND PARTIAL TRACES
Lecture 2 TENSOR PRODUCTS AND PARTIAL TRACES Stéphae ATTAL Abstract This lecture cocers special aspects of Operator Theory which are of much use i Quatum Mechaics, i particular i the theory of Quatum Ope
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More informationMath 61CM - Solutions to homework 3
Math 6CM - Solutios to homework 3 Cédric De Groote October 2 th, 208 Problem : Let F be a field, m 0 a fixed oegative iteger ad let V = {a 0 + a x + + a m x m a 0,, a m F} be the vector space cosistig
More informationLecture 27: Optimal Estimators and Functional Delta Method
Stat210B: Theoretical Statistics Lecture Date: April 19, 2007 Lecture 27: Optimal Estimators ad Fuctioal Delta Method Lecturer: Michael I. Jorda Scribe: Guilherme V. Rocha 1 Achievig Optimal Estimators
More informationACO Comprehensive Exam 9 October 2007 Student code A. 1. Graph Theory
1. Graph Theory Prove that there exist o simple plaar triagulatio T ad two distict adjacet vertices x, y V (T ) such that x ad y are the oly vertices of T of odd degree. Do ot use the Four-Color Theorem.
More informationSTAT331. Example of Martingale CLT with Cox s Model
STAT33 Example of Martigale CLT with Cox s Model I this uit we illustrate the Martigale Cetral Limit Theorem by applyig it to the partial likelihood score fuctio from Cox s model. For simplicity of presetatio
More informationHOMEWORK I: PREREQUISITES FROM MATH 727
HOMEWORK I: PREREQUISITES FROM MATH 727 Questio. Let X, X 2,... be idepedet expoetial radom variables with mea µ. (a) Show that for Z +, we have EX µ!. (b) Show that almost surely, X + + X (c) Fid the
More informationGeneralized Semi- Markov Processes (GSMP)
Geeralized Semi- Markov Processes (GSMP) Summary Some Defiitios Markov ad Semi-Markov Processes The Poisso Process Properties of the Poisso Process Iterarrival times Memoryless property ad the residual
More information