Notes 19 : Martingale CLT
|
|
- Norah Gallagher
- 5 years ago
- Views:
Transcription
1 Notes 9 : Martigale CLT Math : Theory of Probability Lecturer: Sebastie Roch Refereces: [Bil95, Chapter 35], [Roc, Chapter 3]. Sice we have ot ecoutered weak covergece i some time, we first recall a umber of defiitios ad results that will be useful to keep i mid throughout this lecture. DEF 9. (Distributio fuctio) Let X be a RV o a triple (Ω, F, P). The law of X is L X = P X, which is a probability measure o (R, B). By the Uiqueess of Extesio lemma, L X is determied by the distributio fuctio (DF) of X F X (x) = P[X x], x R. DEF 9.2 (Covergece i distributio) A sequece of DFs (F ) coverges i distributio (or weakly) to a DF F if F (x) F (x), for all poits of cotiuity x of F. If F ad F are the DFs of X ad X respectively, we write X X. THM 9.3 (Covergece i distributio v. i probability) Suppose that (X ) is a sequece of RVs with X F. The followig holds:. If X P X, the X X. 2. If X c, where c is a costat, the X P c. LEM 9.4 (Covergig together lemma) If X X ad Z X 0, the Z X. LEM 9.5 (Multiplyig by a covergig sequece) If X X ad Y c with Y 0 ad c > 0, the X Y cx.
2 Lecture 9: Martigale CLT 2 DEF 9.6 (Characteristic fuctio) The characteristic fuctio (CF) of a RV X is φ X (t) = E[e itx ] = E[cos(tX)] + ie[si(tx)], where the secod equality is a defiitio ad the expectatios exist because they are bouded. EX 9.7 (Gaussia distributio) Let X N(0, ). The φ X (t) = e t2 /2. THM 9.8 If X ad X 2 are idepedet the φ X +X 2 (t) = φ X (t)φ X2 (t). LEM 9.9 If DFs F ad G have the same CF, the they are equal. THM 9.0 (Lévy s Cotiuity Theorem) Let µ, be probability measures (PM) with CFs φ.. If µ µ the φ (t) φ (t) for all t. 2. If φ (t) coverges poitwise to a limit φ(t) that is cotiuous at 0 the the associated sequece of PMs µ coverges weakly to a PM µ with CF φ. THM 9. (CLT) Let (X ) be IID with E[X ] = µ ad Var[X ] = σ 2 < +.The if S = k= X k S µ σ N(0, ). Martigale CLT We have see that MGs are, i some ways, geeralizatios of ubiased RWs. Hece, oe might expect a CLT to hold uder appropriate coditios (amely, Lideberg-type coditios). That turs out to be the case, but the situatio is more complex tha for sums of idepedet RVs, as the ext example suggests. I brief, mixtures of ubiased RWs are also MGs, i which case the limit may be a mixture of Gaussias.
3 Lecture 9: Martigale CLT 3 EX 9.2 (Mixtures of RWs) Recall that, if the bouded process {C } is predictable (i.e. C F for all ) ad {S } 0 is a MG, the the martigale trasform {(C S) } 0 with M = (C S) i C i (S i S i ), is also a MG. Suppose S is SRW o the itegers started at 0, that is, S = i= X i where the X i s are IID uiform i {, +}. Hece S i S i = X i. Let also X 0 be idepedet uiform i {, 2} ad set C i = X 0 for all i. The, with respect to the filtratio F = σ(x 0,..., ) for 0, {S } 0 is a MG (because E[X i ] = 0) ad {C } is bouded ad predictable. Hece {M }, as defied above, is a MG. Now ote that {M } is a mixture of two RWs, i the followig sese. O {X 0 = }, which occurs with probability /2, we have M = i= X i ad THM 9. implies that M / N(0, ). O the other had, o {X 0 = 2}, which also occurs with probability /2, we have istead M = i= 2X i ad THM 9. implies this time that M / 4 N(0, ) or M / N(0, 4). To hadle this issue, we itroduce the coditioal variace. Let {M } be a MG i L 2 with correspodig MG differece Z = M M ad let σ 2 = E[Z 2 F ]. (Note that this is a RV.) Cosider the stoppig time (ST) { } m τ = if m 0 : σi 2. () This is ideed a ST sice {τ m} = { m i= σ2 i } F m F m. Uder the assumptio that σ2 = + with probability, we the have τ + a.s. whe +. THM 9.3 (A CLT for MGs with Bouded Icremets) Let {M } be a MG with correspodig MG differece Z = M M. Assume there is a costat K < + such that Z K with probability for all. Let σ 2 = E[Z 2 F ] ad assume that σ2 = + with probability. Let τ be as defied i (). The M τ N(0, ), as +. i=
4 Lecture 9: Martigale CLT 4 Goig back to the example above: EX 9.4 (Mixtures of RWs cotiued) The coditios of THM 9.3 are satisfied with K = 4 by otig that σ 2 a.s. for all. O {X 0 = } we have τ = for all, while o {X 0 = 2} we have τ = /4 for all. To prove THM 9.3, we establish a geeralizatio of the Lideberg-Feller CLT. The settig ivolves a sequece of MGs {M,m } m 0 for with respect to filtratios {F,m } m 0. (If the origial MG is oly defied for 0 m r, the oe ca exted it by lettig M,m = M,r ad F,m = F,r for all m > r. More geerally, this settig accommodates a MG stopped at a sequece of stoppig times divergig to ifiity. See proof of THM 9.3 below.) THM 9.5 (Martigale CLT) For each, let {M,m } m 0 be a MG i L 2 with respect to filtratio {F,m } m 0 with correspodig MG differece Z,m = M,m M,m ad coditioal variace σ,m 2 = E[Z,m 2 F,m ]. Assume that, for each, M,m ad Γ,m m r= σ2,r coverge a.s. to a fiite limit whe m +. Suppose that The. Γ, + m= σ2,m i probability as + 2. ε > 0, lim + m= E[Z2,m; Z,m > ε] = 0 as +. M, + m= Z,m N(0, ), Before givig the proof of the MG-CLT, we use it to establish THM 9.3. Proof:(of THM 9.3) We costruct a sequece of MGs as follows. Let F,m = F m ad M,m = M m τ so that Z,m = Z m {τ m}. This is ideed a MG for each sice τ is a ST (ad multiplyig by a costat
5 Lecture 9: Martigale CLT 5 does ot affect the MG property). Moreover [ (Zm ] 2 σ,m 2 = E {τ m}) F,m = {τ m} E[Z 2 m F m ] = σ2 m {τ m}, where we used that {τ m} F m. Summig over m ad usig the boud o Z m, we get τ σm 2 Γ, = + K2, m= by the defiitio of τ. So as +, we have Γ,. Also, for ay ε > 0, Z,m = Z m {τ m} K < ε, m wheever is large eough so that lim + m= E[ Z,m 2 ; Z,m > ε] = 0. Therefore, the result follows from the MG-CLT after otig that M, = M τ, where we used the fact that τ < + a.s. by the assumptio σ2 = +. We move o to the proof of the MG-CLT. Proof:(of THM 9.5) We first assume that there is a costat c such that Γ, c, for all. We seek to prove that the followig is goig to 0 as + : E[e itm, ] e t2 /2 E[e itm, e itm, e t2 /2 e t2 Γ, /2 ] +E[e itm, e t2 /2 e t2 Γ, /2 e t2 /2 ] E[ e t2 /2 e t2 Γ, /2 ] + E[e itm, e t2 Γ, /2 ]. The first term above 0 by (BDD) ad the assumptio that Γ, c.
6 Lecture 9: Martigale CLT 6 To boud the secod term we use a telescopig argumet. Write e itm, e t2 Γ, /2 = {e } itm,m e t2 Γ,m/2 e itm,m e t2 Γ,m /2 m = m e itm,m e t2 Γ,m/2 { e itz,m e t2 σ 2,m/2 }. Hece, E[e itm, e t2 Γ, /2 ] [ { }] E e itm,m e t2 Γ,m/2 e itz,m e t2 σ,m/2 2 m [ [ ]] E e itm,m e t2 Γ,m/2 E e itz,m e t2 σ,m 2 /2 F,m m e [ [ ]] E ct2 E e itz,m e t2 σ,m/2 2 F,m m where we used that M,m, Γ,m F,m ad the assumptio that Γ, c. We will use the followig lemmas proved i a previous lecture. LEM 9.6 It holds that E [ e itx] ( + it E[X] t2 ]) 2 E[X2 E [ mi{ tx 3, tx 2 } ]. LEM 9.7 If z is a complex umber the e z ( + z) z 2 e z. By the MG property, E[Z,m F,m ]. Also by defiitio E[Z 2,m F,m ] = σ 2,m. By LEM 9.6, E [ e ] itz,m F,m ( t2 2,m) σ2 E [ tz,m 3 tz,m 2 F,m ] E[ tz,m 3 ; Z,m ε F,m ] + E[ tz,m 2 ; Z,m > ε F,m ] ε t 3 E[Z 2,m; Z,m ε F,m ] + t 2 E[Z 2,m; Z,m > ε F,m ] ε t 3 σ 2,m + t 2 E[Z 2,m; Z,m > ε F,m ].
7 Lecture 9: Martigale CLT 7 By LEM 9.7, ( ) σ,m/2 2 e t2 t2 σ,m 2 2 (t2 σ,m/2) 2 2 e t2 σ,m/2 2 t 4 σ,me 2 ct2 Pluggig the previous two displays ito the equatio above, we get E[e itm, e t2 Γ, /2 ] e [ [ ]] E ct2 E e itz,m e t2 σ,m/2 2 F,m m e ct2 { ε t 3 c + t 2 + m= ( ) sup σ,m 2. m [ E[Z,m; 2 Z,m > ε] + t 4 ce ct2 E sup σ,m] } 2. m (Note that the partial sum i the telescopig argumet above, e itm,m e t2 Γ,m/2, is bouded ad therefore we ca take the expectatio iside by (BDD).) To boud the supremum, we ote that so σ 2,m ε 2 + E[Z 2,m; Z,m > ε F,m ] ε i= E[Z 2,i; Z,i > ε F,i ] [ ] + E sup σ,m 2 ε 2 + E[Z,i; 2 Z,i > ε]. m i= Takig + ad ε > 0 arbitrarily ad usig the secod coditio of the statemet, we get E[e itm, e t2 Γ, /2 ] 0, for all t. That cocludes the proof i the case Γ, c. To remove the last assumptio, multiply the differece Z,m by the idicator that {Γ,m c} F,m for some c > ad apply the argumet above. The ote that P[Γ, c] as + by the first coditio of the statemet ad use the covergig together lemma. See the details i [B]. 2 A applicatio: autoregressive processes Let {W } be a sequece of IID RVs with W c for all for some costat c < +. Assume that E[W ] = 0 ad Var[W ] = ν 2. For ρ (0, ), cosider
8 Lecture 9: Martigale CLT 8 the followig autoregressive process (of order ) X = ρx + W, ad X 0 = 0. Assume that ρ is ukow ad that we oly observe the sequece {X }. To estimate ρ, we observe that E[X X ] = E[ρX 2 + W X ] = ρe[x 2 ], where we used the idepedece of W ad X σ(w,..., W ). Hece, a atural estimator for ρ from {X } is ˆρ k= X kx k k= X2 k We use the MG-CLT to establish the asymptotic ormality of ˆρ. CLAIM 9.8 We have where Ψ ν2. ρ 2 Ψ ν 2 (ˆρ ρ) N(0, ), Proof: Our startig poit is the followig observatio: (ˆρ ρ) Xk 2 = k= = = X k X k ρ k=. k= X 2 k (X k ρx k )X k k= W k X k. k= The latter is ot a sum of idepedet RVs, but it turs out to be a MG with respect to the filtratio F = σ(w,..., W ), as we show ext. LEM 9.9 The process is a MG. M = W k X k, 0 k=
9 Lecture 9: Martigale CLT 9 LEM 9.20 We have Ψ Xk 2 P k= ν 2 ρ 2 Ψ. Before provig the two claims, it will be helpful to write by iductio X = ρ k W k, (2) k= ad defie H (x) = k= x k. The Var[X ] = E[X 2 ] = ν 2 H (ρ 2 ), where we used that the W k s are idepedet ad have mea 0 ad variace ν 2. Note also that by (2) ad the boud o W, we have the followig boud X ρ k W k ch (ρ). (3) k= Proof:(of LEM 9.9) The process {M } is adapted by (2) ad itegrable by the bouds o W ad X. Observe further that, sice X k F k, [ ] E W k X k F = W k X k + X E [W F ] = W k X k. k= k= That cocludes the proof of the claim. Proof:(of LEM 9.20) We use Chebyshev s iequality. We kow that [ ] E Xk 2 = ν 2 H k (ρ 2 ) ν2 ρ 2, (4) k= k= as H (ρ 2 ) ρ 2. We eed a boud o the variace. Trivially Var[X 2 k ] E[X4 k ] {ch k(ρ)} 4. Moreover, for k < l, we write X l ρ l k X k +Y l,k where Y l,k σ(w k+,..., W l ) k=
10 Lecture 9: Martigale CLT 0 ad H l (ρ 2 ) = ρ 2(l k) H k (ρ 2 ) + H l k (ρ 2 ). Hece, Cov [ Xk 2, ] X2 l = E [ (Xk 2 ν2 H k (ρ 2 ))(Xl 2 ν2 H l (ρ 2 )) ] = E[(X 2 k ν2 H k (ρ 2 )) ((ρ l k X k + Y l,k ) 2 ν 2 [ρ 2(l k) H k (ρ 2 ) + H l k (ρ 2 )])] = E[(X 2 k ν2 H k (ρ 2 )) (ρ 2(l k) X 2 k + 2ρl k X k Y l,k + Y 2 l,k ν2 [ρ 2(l k) H k (ρ 2 ) + H l k (ρ 2 )])] = ρ 2(l k) E[(X 2 k ν2 H k (ρ 2 )) 2 ] ρ 2(l k) {ch k (ρ)} 4. Therefore Var [ k= X 2 k ] = 2 2 = 0 k l 2c 4 2 ( ρ) 4 2c 4 2 ( ρ) 4 ρ 2(l k) {ch k (ρ)} 4 0 k l 0 m + 2c 4 ( ρ) 4 ( ρ 2 ) ρ 2(l k) ρ 2m k= K. By Chebyshev s iequality, [ [ ] P Xk 2 E X 2 k ] δ k= as +. Together with (4), that proves the claim. We apply the MG-CLT with ad F,m = F m. The Z,m = M m M m Ψ ν 2 M,m = M m Ψ ν 2, K/ δ 2 0, {m } = W mx m {m }, Ψ ν2
11 Lecture 9: Martigale CLT ad ad σ 2,m = E[Z 2,m F,m ] = {m } X2 m Ψ ν 2 E[W 2 m F m ] = {m } X2 m Ψ ν 2 ν2, Γ, = Ψ m= X 2 m = Ψ Ψ P, as + by LEM To check the secod coditio i the MG-CLT, we ote that Z,m = W m X m Ψ ν {m } c(ch (ρ)) 2 Ψ ν c 2 2 ( ρ) Ψ ν ε, 2 for all large eough. The MG-CLT the says that or put differetly, that (ˆρ ρ) M N(0, ) Ψ ν2 Ψ ν 2 k= X 2 k By LEM 9.5 ad LEM 9.20, we fially have Ψ ν 2 (ˆρ ρ) N(0, ). That cocludes the proof of CLAIM 9.8. Refereces N(0, ). [Bil95] Patrick Billigsley. Probability ad measure. Wiley Series i Probability ad Mathematical Statistics. Joh Wiley & Sos Ic., New York, 995. [Roc] Sebastie Roch. Moder Discrete Probability: A Essetial Toolkit. Book i preparatio. roch/mdp/.
2.2. Central limit theorem.
36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard
More informationNotes 5 : More on the a.s. convergence of sums
Notes 5 : More o the a.s. covergece of sums Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces: Dur0, Sectios.5; Wil9, Sectio 4.7, Shi96, Sectio IV.4, Dur0, Sectio.. Radom series. Three-series
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.436J/15.085J Fall 2008 Lecture 19 11/17/2008 LAWS OF LARGE NUMBERS II THE STRONG LAW OF LARGE NUMBERS
MASSACHUSTTS INSTITUT OF TCHNOLOGY 6.436J/5.085J Fall 2008 Lecture 9 /7/2008 LAWS OF LARG NUMBRS II Cotets. The strog law of large umbers 2. The Cheroff boud TH STRONG LAW OF LARG NUMBRS While the weak
More informationConvergence of random variables. (telegram style notes) P.J.C. Spreij
Covergece of radom variables (telegram style otes).j.c. Spreij this versio: September 6, 2005 Itroductio As we kow, radom variables are by defiitio measurable fuctios o some uderlyig measurable space
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit Theorems Throughout this sectio we will assume a probability space (, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationLecture 19: Convergence
Lecture 19: Covergece Asymptotic approach I statistical aalysis or iferece, a key to the success of fidig a good procedure is beig able to fid some momets ad/or distributios of various statistics. I may
More information1 = δ2 (0, ), Y Y n nδ. , T n = Y Y n n. ( U n,k + X ) ( f U n,k + Y ) n 2n f U n,k + θ Y ) 2 E X1 2 X1
8. The cetral limit theorems 8.1. The cetral limit theorem for i.i.d. sequeces. ecall that C ( is N -separatig. Theorem 8.1. Let X 1, X,... be i.i.d. radom variables with EX 1 = ad EX 1 = σ (,. Suppose
More informationNotes 27 : Brownian motion: path properties
Notes 27 : Browia motio: path properties Math 733-734: Theory of Probability Lecturer: Sebastie Roch Refereces:[Dur10, Sectio 8.1], [MP10, Sectio 1.1, 1.2, 1.3]. Recall: DEF 27.1 (Covariace) Let X = (X
More informationLecture 10 October Minimaxity and least favorable prior sequences
STATS 300A: Theory of Statistics Fall 205 Lecture 0 October 22 Lecturer: Lester Mackey Scribe: Brya He, Rahul Makhijai Warig: These otes may cotai factual ad/or typographic errors. 0. Miimaxity ad least
More information7.1 Convergence of sequences of random variables
Chapter 7 Limit theorems Throughout this sectio we will assume a probability space (Ω, F, P), i which is defied a ifiite sequece of radom variables (X ) ad a radom variable X. The fact that for every ifiite
More informationProduct measures, Tonelli s and Fubini s theorems For use in MAT3400/4400, autumn 2014 Nadia S. Larsen. Version of 13 October 2014.
Product measures, Toelli s ad Fubii s theorems For use i MAT3400/4400, autum 2014 Nadia S. Larse Versio of 13 October 2014. 1. Costructio of the product measure The purpose of these otes is to preset the
More informationDistribution of Random Samples & Limit theorems
STAT/MATH 395 A - PROBABILITY II UW Witer Quarter 2017 Néhémy Lim Distributio of Radom Samples & Limit theorems 1 Distributio of i.i.d. Samples Motivatig example. Assume that the goal of a study is to
More informationSTA Object Data Analysis - A List of Projects. January 18, 2018
STA 6557 Jauary 8, 208 Object Data Aalysis - A List of Projects. Schoeberg Mea glaucomatous shape chages of the Optic Nerve Head regio i aimal models 2. Aalysis of VW- Kedall ati-mea shapes with a applicatio
More informationSequences and Series of Functions
Chapter 6 Sequeces ad Series of Fuctios 6.1. Covergece of a Sequece of Fuctios Poitwise Covergece. Defiitio 6.1. Let, for each N, fuctio f : A R be defied. If, for each x A, the sequece (f (x)) coverges
More informationLECTURE 8: ASYMPTOTICS I
LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1 Defiitio: (Weak covergece) A sequece
More informationChapter 3. Strong convergence. 3.1 Definition of almost sure convergence
Chapter 3 Strog covergece As poited out i the Chapter 2, there are multiple ways to defie the otio of covergece of a sequece of radom variables. That chapter defied covergece i probability, covergece i
More informationThis section is optional.
4 Momet Geeratig Fuctios* This sectio is optioal. The momet geeratig fuctio g : R R of a radom variable X is defied as g(t) = E[e tx ]. Propositio 1. We have g () (0) = E[X ] for = 1, 2,... Proof. Therefore
More informationLecture 8: Convergence of transformations and law of large numbers
Lecture 8: Covergece of trasformatios ad law of large umbers Trasformatio ad covergece Trasformatio is a importat tool i statistics. If X coverges to X i some sese, we ofte eed to check whether g(x ) coverges
More informationSeunghee Ye Ma 8: Week 5 Oct 28
Week 5 Summary I Sectio, we go over the Mea Value Theorem ad its applicatios. I Sectio 2, we will recap what we have covered so far this term. Topics Page Mea Value Theorem. Applicatios of the Mea Value
More informationST5215: Advanced Statistical Theory
ST525: Advaced Statistical Theory Departmet of Statistics & Applied Probability Tuesday, September 7, 2 ST525: Advaced Statistical Theory Lecture : The law of large umbers The Law of Large Numbers The
More informationECE 330:541, Stochastic Signals and Systems Lecture Notes on Limit Theorems from Probability Fall 2002
ECE 330:541, Stochastic Sigals ad Systems Lecture Notes o Limit Theorems from robability Fall 00 I practice, there are two ways we ca costruct a ew sequece of radom variables from a old sequece of radom
More informationLet us give one more example of MLE. Example 3. The uniform distribution U[0, θ] on the interval [0, θ] has p.d.f.
Lecture 5 Let us give oe more example of MLE. Example 3. The uiform distributio U[0, ] o the iterval [0, ] has p.d.f. { 1 f(x =, 0 x, 0, otherwise The likelihood fuctio ϕ( = f(x i = 1 I(X 1,..., X [0,
More informationLecture 01: the Central Limit Theorem. 1 Central Limit Theorem for i.i.d. random variables
CSCI-B609: A Theorist s Toolkit, Fall 06 Aug 3 Lecture 0: the Cetral Limit Theorem Lecturer: Yua Zhou Scribe: Yua Xie & Yua Zhou Cetral Limit Theorem for iid radom variables Let us say that we wat to aalyze
More information1 Convergence in Probability and the Weak Law of Large Numbers
36-752 Advaced Probability Overview Sprig 2018 8. Covergece Cocepts: i Probability, i L p ad Almost Surely Istructor: Alessadro Rialdo Associated readig: Sec 2.4, 2.5, ad 4.11 of Ash ad Doléas-Dade; Sec
More informationLaw of the sum of Bernoulli random variables
Law of the sum of Beroulli radom variables Nicolas Chevallier Uiversité de Haute Alsace, 4, rue des frères Lumière 68093 Mulhouse icolas.chevallier@uha.fr December 006 Abstract Let be the set of all possible
More informationLecture 20: Multivariate convergence and the Central Limit Theorem
Lecture 20: Multivariate covergece ad the Cetral Limit Theorem Covergece i distributio for radom vectors Let Z,Z 1,Z 2,... be radom vectors o R k. If the cdf of Z is cotiuous, the we ca defie covergece
More informationLecture 2: Concentration Bounds
CSE 52: Desig ad Aalysis of Algorithms I Sprig 206 Lecture 2: Cocetratio Bouds Lecturer: Shaya Oveis Ghara March 30th Scribe: Syuzaa Sargsya Disclaimer: These otes have ot bee subjected to the usual scrutiy
More informationMATH301 Real Analysis (2008 Fall) Tutorial Note #7. k=1 f k (x) converges pointwise to S(x) on E if and
MATH01 Real Aalysis (2008 Fall) Tutorial Note #7 Sequece ad Series of fuctio 1: Poitwise Covergece ad Uiform Covergece Part I: Poitwise Covergece Defiitio of poitwise covergece: A sequece of fuctios f
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 21 11/27/2013 Fuctioal Law of Large Numbers. Costructio of the Wieer Measure Cotet. 1. Additioal techical results o weak covergece
More information4. Partial Sums and the Central Limit Theorem
1 of 10 7/16/2009 6:05 AM Virtual Laboratories > 6. Radom Samples > 1 2 3 4 5 6 7 4. Partial Sums ad the Cetral Limit Theorem The cetral limit theorem ad the law of large umbers are the two fudametal theorems
More informationDefinition 4.2. (a) A sequence {x n } in a Banach space X is a basis for X if. unique scalars a n (x) such that x = n. a n (x) x n. (4.
4. BASES I BAACH SPACES 39 4. BASES I BAACH SPACES Sice a Baach space X is a vector space, it must possess a Hamel, or vector space, basis, i.e., a subset {x γ } γ Γ whose fiite liear spa is all of X ad
More informationIntegrable Functions. { f n } is called a determining sequence for f. If f is integrable with respect to, then f d does exist as a finite real number
MATH 532 Itegrable Fuctios Dr. Neal, WKU We ow shall defie what it meas for a measurable fuctio to be itegrable, show that all itegral properties of simple fuctios still hold, ad the give some coditios
More informationEntropy Rates and Asymptotic Equipartition
Chapter 29 Etropy Rates ad Asymptotic Equipartitio Sectio 29. itroduces the etropy rate the asymptotic etropy per time-step of a stochastic process ad shows that it is well-defied; ad similarly for iformatio,
More informationThe natural exponential function
The atural expoetial fuctio Attila Máté Brookly College of the City Uiversity of New York December, 205 Cotets The atural expoetial fuctio for real x. Beroulli s iequality.....................................2
More informationMath Solutions to homework 6
Math 175 - Solutios to homework 6 Cédric De Groote November 16, 2017 Problem 1 (8.11 i the book): Let K be a compact Hermitia operator o a Hilbert space H ad let the kerel of K be {0}. Show that there
More informationAdvanced Stochastic Processes.
Advaced Stochastic Processes. David Gamarik LECTURE 2 Radom variables ad measurable fuctios. Strog Law of Large Numbers (SLLN). Scary stuff cotiued... Outlie of Lecture Radom variables ad measurable fuctios.
More informationNotes on Snell Envelops and Examples
Notes o Sell Evelops ad Examples Example (Secretary Problem): Coside a pool of N cadidates whose qualificatios are represeted by ukow umbers {a > a 2 > > a N } from best to last. They are iterviewed sequetially
More informationSolutions to HW Assignment 1
Solutios to HW: 1 Course: Theory of Probability II Page: 1 of 6 Uiversity of Texas at Austi Solutios to HW Assigmet 1 Problem 1.1. Let Ω, F, {F } 0, P) be a filtered probability space ad T a stoppig time.
More informationSolution to Chapter 2 Analytical Exercises
Nov. 25, 23, Revised Dec. 27, 23 Hayashi Ecoometrics Solutio to Chapter 2 Aalytical Exercises. For ay ε >, So, plim z =. O the other had, which meas that lim E(z =. 2. As show i the hit, Prob( z > ε =
More information6.3 Testing Series With Positive Terms
6.3. TESTING SERIES WITH POSITIVE TERMS 307 6.3 Testig Series With Positive Terms 6.3. Review of what is kow up to ow I theory, testig a series a i for covergece amouts to fidig the i= sequece of partial
More informationAn alternative proof of a theorem of Aldous. concerning convergence in distribution for martingales.
A alterative proof of a theorem of Aldous cocerig covergece i distributio for martigales. Maurizio Pratelli Dipartimeto di Matematica, Uiversità di Pisa. Via Buoarroti 2. I-56127 Pisa, Italy e-mail: pratelli@dm.uipi.it
More informationLecture 3 : Random variables and their distributions
Lecture 3 : Radom variables ad their distributios 3.1 Radom variables Let (Ω, F) ad (S, S) be two measurable spaces. A map X : Ω S is measurable or a radom variable (deoted r.v.) if X 1 (A) {ω : X(ω) A}
More informationAsymptotic distribution of products of sums of independent random variables
Proc. Idia Acad. Sci. Math. Sci. Vol. 3, No., May 03, pp. 83 9. c Idia Academy of Scieces Asymptotic distributio of products of sums of idepedet radom variables YANLING WANG, SUXIA YAO ad HONGXIA DU ollege
More informationSequences A sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece 1, 1, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet
More informationProblem Set 2 Solutions
CS271 Radomess & Computatio, Sprig 2018 Problem Set 2 Solutios Poit totals are i the margi; the maximum total umber of poits was 52. 1. Probabilistic method for domiatig sets 6pts Pick a radom subset S
More informationRead carefully the instructions on the answer book and make sure that the particulars required are entered on each answer book.
THE UNIVERSITY OF WARWICK FIRST YEAR EXAMINATION: Jauary 2009 Aalysis I Time Allowed:.5 hours Read carefully the istructios o the aswer book ad make sure that the particulars required are etered o each
More information1+x 1 + α+x. x = 2(α x2 ) 1+x
Math 2030 Homework 6 Solutios # [Problem 5] For coveiece we let α lim sup a ad β lim sup b. Without loss of geerality let us assume that α β. If α the by assumptio β < so i this case α + β. By Theorem
More informationlim za n n = z lim a n n.
Lecture 6 Sequeces ad Series Defiitio 1 By a sequece i a set A, we mea a mappig f : N A. It is customary to deote a sequece f by {s } where, s := f(). A sequece {z } of (complex) umbers is said to be coverget
More informationSTAT331. Example of Martingale CLT with Cox s Model
STAT33 Example of Martigale CLT with Cox s Model I this uit we illustrate the Martigale Cetral Limit Theorem by applyig it to the partial likelihood score fuctio from Cox s model. For simplicity of presetatio
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 3 9/11/2013. Large deviations Theory. Cramér s Theorem
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 3 9//203 Large deviatios Theory. Cramér s Theorem Cotet.. Cramér s Theorem. 2. Rate fuctio ad properties. 3. Chage of measure techique.
More informationSTAT Homework 1 - Solutions
STAT-36700 Homework 1 - Solutios Fall 018 September 11, 018 This cotais solutios for Homework 1. Please ote that we have icluded several additioal commets ad approaches to the problems to give you better
More informationDiscrete Mathematics for CS Spring 2007 Luca Trevisan Lecture 22
CS 70 Discrete Mathematics for CS Sprig 2007 Luca Trevisa Lecture 22 Aother Importat Distributio The Geometric Distributio Questio: A biased coi with Heads probability p is tossed repeatedly util the first
More informationDifferent kinds of Mathematical Induction
Differet ids of Mathematical Iductio () Mathematical Iductio Give A N, [ A (a A a A)] A N () (First) Priciple of Mathematical Iductio Let P() be a propositio (ope setece), if we put A { : N p() is true}
More informationANSWERS TO MIDTERM EXAM # 2
MATH 03, FALL 003 ANSWERS TO MIDTERM EXAM # PENN STATE UNIVERSITY Problem 1 (18 pts). State ad prove the Itermediate Value Theorem. Solutio See class otes or Theorem 5.6.1 from our textbook. Problem (18
More informationEcon 325/327 Notes on Sample Mean, Sample Proportion, Central Limit Theorem, Chi-square Distribution, Student s t distribution 1.
Eco 325/327 Notes o Sample Mea, Sample Proportio, Cetral Limit Theorem, Chi-square Distributio, Studet s t distributio 1 Sample Mea By Hiro Kasahara We cosider a radom sample from a populatio. Defiitio
More informationUniversity of Colorado Denver Dept. Math. & Stat. Sciences Applied Analysis Preliminary Exam 13 January 2012, 10:00 am 2:00 pm. Good luck!
Uiversity of Colorado Dever Dept. Math. & Stat. Scieces Applied Aalysis Prelimiary Exam 13 Jauary 01, 10:00 am :00 pm Name: The proctor will let you read the followig coditios before the exam begis, ad
More informationEntropy and Ergodic Theory Lecture 5: Joint typicality and conditional AEP
Etropy ad Ergodic Theory Lecture 5: Joit typicality ad coditioal AEP 1 Notatio: from RVs back to distributios Let (Ω, F, P) be a probability space, ad let X ad Y be A- ad B-valued discrete RVs, respectively.
More informationChapter 5. Inequalities. 5.1 The Markov and Chebyshev inequalities
Chapter 5 Iequalities 5.1 The Markov ad Chebyshev iequalities As you have probably see o today s frot page: every perso i the upper teth percetile ears at least 1 times more tha the average salary. I other
More information1 Approximating Integrals using Taylor Polynomials
Seughee Ye Ma 8: Week 7 Nov Week 7 Summary This week, we will lear how we ca approximate itegrals usig Taylor series ad umerical methods. Topics Page Approximatig Itegrals usig Taylor Polyomials. Defiitios................................................
More informationDiscrete Mathematics for CS Spring 2008 David Wagner Note 22
CS 70 Discrete Mathematics for CS Sprig 2008 David Wager Note 22 I.I.D. Radom Variables Estimatig the bias of a coi Questio: We wat to estimate the proportio p of Democrats i the US populatio, by takig
More information1 The Haar functions and the Brownian motion
1 The Haar fuctios ad the Browia motio 1.1 The Haar fuctios ad their completeess The Haar fuctios The basic Haar fuctio is 1 if x < 1/2, ψx) = 1 if 1/2 x < 1, otherwise. 1.1) It has mea zero 1 ψx)dx =,
More informationThe Central Limit Theorem
Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx
More information1 Review and Overview
DRAFT a fial versio will be posted shortly CS229T/STATS231: Statistical Learig Theory Lecturer: Tegyu Ma Lecture #3 Scribe: Migda Qiao October 1, 2013 1 Review ad Overview I the first half of this course,
More informationProbability and Statistics
ICME Refresher Course: robability ad Statistics Staford Uiversity robability ad Statistics Luyag Che September 20, 2016 1 Basic robability Theory 11 robability Spaces A probability space is a triple (Ω,
More informationINFINITE SEQUENCES AND SERIES
11 INFINITE SEQUENCES AND SERIES INFINITE SEQUENCES AND SERIES 11.4 The Compariso Tests I this sectio, we will lear: How to fid the value of a series by comparig it with a kow series. COMPARISON TESTS
More informationKolmogorov-Smirnov type Tests for Local Gaussianity in High-Frequency Data
Proceedigs 59th ISI World Statistics Cogress, 5-30 August 013, Hog Kog (Sessio STS046) p.09 Kolmogorov-Smirov type Tests for Local Gaussiaity i High-Frequecy Data George Tauche, Duke Uiversity Viktor Todorov,
More informationBerry-Esseen bounds for self-normalized martingales
Berry-Essee bouds for self-ormalized martigales Xiequa Fa a, Qi-Ma Shao b a Ceter for Applied Mathematics, Tiaji Uiversity, Tiaji 30007, Chia b Departmet of Statistics, The Chiese Uiversity of Hog Kog,
More informationMixingales. Chapter 7
Chapter 7 Mixigales I this sectio we prove some of the results stated i the previous sectios usig mixigales. We first defie a mixigale, otig that the defiitio we give is ot the most geeral defiitio. Defiitio
More informationCharacteristic functions and the central limit theorem
Characteristic fuctios ad the cetral limit theorem Saifuddi Syed, #0387835 Uiversity of Waterloo Submitted to Dr. Kathry Hare for PMATH 800 April 9, 03 Abstract The cetral limit theorem is oe of the corerstoes
More informationMath 25 Solutions to practice problems
Math 5: Advaced Calculus UC Davis, Sprig 0 Math 5 Solutios to practice problems Questio For = 0,,, 3,... ad 0 k defie umbers C k C k =! k!( k)! (for k = 0 ad k = we defie C 0 = C = ). by = ( )... ( k +
More information17. Joint distributions of extreme order statistics Lehmann 5.1; Ferguson 15
17. Joit distributios of extreme order statistics Lehma 5.1; Ferguso 15 I Example 10., we derived the asymptotic distributio of the maximum from a radom sample from a uiform distributio. We did this usig
More informationA Proof of Birkhoff s Ergodic Theorem
A Proof of Birkhoff s Ergodic Theorem Joseph Hora September 2, 205 Itroductio I Fall 203, I was learig the basics of ergodic theory, ad I came across this theorem. Oe of my supervisors, Athoy Quas, showed
More informationLearning Theory: Lecture Notes
Learig Theory: Lecture Notes Kamalika Chaudhuri October 4, 0 Cocetratio of Averages Cocetratio of measure is very useful i showig bouds o the errors of machie-learig algorithms. We will begi with a basic
More informationDetailed proofs of Propositions 3.1 and 3.2
Detailed proofs of Propositios 3. ad 3. Proof of Propositio 3. NB: itegratio sets are geerally omitted for itegrals defied over a uit hypercube [0, s with ay s d. We first give four lemmas. The proof of
More informationMath 341 Lecture #31 6.5: Power Series
Math 341 Lecture #31 6.5: Power Series We ow tur our attetio to a particular kid of series of fuctios, amely, power series, f(x = a x = a 0 + a 1 x + a 2 x 2 + where a R for all N. I terms of a series
More informationChapter 6 Infinite Series
Chapter 6 Ifiite Series I the previous chapter we cosidered itegrals which were improper i the sese that the iterval of itegratio was ubouded. I this chapter we are goig to discuss a topic which is somewhat
More informationA sequence of numbers is a function whose domain is the positive integers. We can see that the sequence
Sequeces A sequece of umbers is a fuctio whose domai is the positive itegers. We ca see that the sequece,, 2, 2, 3, 3,... is a fuctio from the positive itegers whe we write the first sequece elemet as
More informationIntroduction to Probability. Ariel Yadin
Itroductio to robability Ariel Yadi Lecture 2 *** Ja. 7 ***. Covergece of Radom Variables As i the case of sequeces of umbers, we would like to talk about covergece of radom variables. There are may ways
More informationECE534, Spring 2018: Final Exam
ECE534, Srig 2018: Fial Exam Problem 1 Let X N (0, 1) ad Y N (0, 1) be ideedet radom variables. variables V = X + Y ad W = X 2Y. Defie the radom (a) Are V, W joitly Gaussia? Justify your aswer. (b) Comute
More informationLecture Chapter 6: Convergence of Random Sequences
ECE5: Aalysis of Radom Sigals Fall 6 Lecture Chapter 6: Covergece of Radom Sequeces Dr Salim El Rouayheb Scribe: Abhay Ashutosh Doel, Qibo Zhag, Peiwe Tia, Pegzhe Wag, Lu Liu Radom sequece Defiitio A ifiite
More informationCentral Limit Theorem using Characteristic functions
Cetral Limit Theorem usig Characteristic fuctios RogXi Guo MAT 477 Jauary 20, 2014 RogXi Guo (2014 Cetral Limit Theorem usig Characteristic fuctios Jauary 20, 2014 1 / 15 Itroductio study a radom variable
More informationThe standard deviation of the mean
Physics 6C Fall 20 The stadard deviatio of the mea These otes provide some clarificatio o the distictio betwee the stadard deviatio ad the stadard deviatio of the mea.. The sample mea ad variace Cosider
More information7 Sequences of real numbers
40 7 Sequeces of real umbers 7. Defiitios ad examples Defiitio 7... A sequece of real umbers is a real fuctio whose domai is the set N of atural umbers. Let s : N R be a sequece. The the values of s are
More informationLecture 3 The Lebesgue Integral
Lecture 3: The Lebesgue Itegral 1 of 14 Course: Theory of Probability I Term: Fall 2013 Istructor: Gorda Zitkovic Lecture 3 The Lebesgue Itegral The costructio of the itegral Uless expressly specified
More informationMASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/15.070J Fall 2013 Lecture 6 9/23/2013. Brownian motion. Introduction
MASSACHUSETTS INSTITUTE OF TECHNOLOGY 6.265/5.070J Fall 203 Lecture 6 9/23/203 Browia motio. Itroductio Cotet.. A heuristic costructio of a Browia motio from a radom walk. 2. Defiitio ad basic properties
More information2.1. Convergence in distribution and characteristic functions.
3 Chapter 2. Cetral Limit Theorem. Cetral limit theorem, or DeMoivre-Laplace Theorem, which also implies the wea law of large umbers, is the most importat theorem i probability theory ad statistics. For
More informationLecture 12: September 27
36-705: Itermediate Statistics Fall 207 Lecturer: Siva Balakrisha Lecture 2: September 27 Today we will discuss sufficiecy i more detail ad the begi to discuss some geeral strategies for costructig estimators.
More informationJournal of Multivariate Analysis. Superefficient estimation of the marginals by exploiting knowledge on the copula
Joural of Multivariate Aalysis 102 (2011) 1315 1319 Cotets lists available at ScieceDirect Joural of Multivariate Aalysis joural homepage: www.elsevier.com/locate/jmva Superefficiet estimatio of the margials
More informationProbability 2 - Notes 10. Lemma. If X is a random variable and g(x) 0 for all x in the support of f X, then P(g(X) 1) E[g(X)].
Probability 2 - Notes 0 Some Useful Iequalities. Lemma. If X is a radom variable ad g(x 0 for all x i the support of f X, the P(g(X E[g(X]. Proof. (cotiuous case P(g(X Corollaries x:g(x f X (xdx x:g(x
More information(A sequence also can be thought of as the list of function values attained for a function f :ℵ X, where f (n) = x n for n 1.) x 1 x N +k x N +4 x 3
MATH 337 Sequeces Dr. Neal, WKU Let X be a metric space with distace fuctio d. We shall defie the geeral cocept of sequece ad limit i a metric space, the apply the results i particular to some special
More informationEmpirical Processes: Glivenko Cantelli Theorems
Empirical Processes: Gliveko Catelli Theorems Mouliath Baerjee Jue 6, 200 Gliveko Catelli classes of fuctios The reader is referred to Chapter.6 of Weller s Torgo otes, Chapter??? of VDVW ad Chapter 8.3
More informationFall 2013 MTH431/531 Real analysis Section Notes
Fall 013 MTH431/531 Real aalysis Sectio 8.1-8. Notes Yi Su 013.11.1 1. Defiitio of uiform covergece. We look at a sequece of fuctios f (x) ad study the coverget property. Notice we have two parameters
More informationLecture 3: August 31
36-705: Itermediate Statistics Fall 018 Lecturer: Siva Balakrisha Lecture 3: August 31 This lecture will be mostly a summary of other useful expoetial tail bouds We will ot prove ay of these i lecture,
More informationSingular Continuous Measures by Michael Pejic 5/14/10
Sigular Cotiuous Measures by Michael Peic 5/4/0 Prelimiaries Give a set X, a σ-algebra o X is a collectio of subsets of X that cotais X ad ad is closed uder complemetatio ad coutable uios hece, coutable
More informationInfinite Sequences and Series
Chapter 6 Ifiite Sequeces ad Series 6.1 Ifiite Sequeces 6.1.1 Elemetary Cocepts Simply speakig, a sequece is a ordered list of umbers writte: {a 1, a 2, a 3,...a, a +1,...} where the elemets a i represet
More informationLecture 2. The Lovász Local Lemma
Staford Uiversity Sprig 208 Math 233A: No-costructive methods i combiatorics Istructor: Ja Vodrák Lecture date: Jauary 0, 208 Origial scribe: Apoorva Khare Lecture 2. The Lovász Local Lemma 2. Itroductio
More informationSelf-normalized deviation inequalities with application to t-statistic
Self-ormalized deviatio iequalities with applicatio to t-statistic Xiequa Fa Ceter for Applied Mathematics, Tiaji Uiversity, 30007 Tiaji, Chia Abstract Let ξ i i 1 be a sequece of idepedet ad symmetric
More informationMath 113 Exam 3 Practice
Math Exam Practice Exam will cover.-.9. This sheet has three sectios. The first sectio will remid you about techiques ad formulas that you should kow. The secod gives a umber of practice questios for you
More informationREAL ANALYSIS II: PROBLEM SET 1 - SOLUTIONS
REAL ANALYSIS II: PROBLEM SET 1 - SOLUTIONS 18th Feb, 016 Defiitio (Lipschitz fuctio). A fuctio f : R R is said to be Lipschitz if there exists a positive real umber c such that for ay x, y i the domai
More informationLecture 11 October 27
STATS 300A: Theory of Statistics Fall 205 Lecture October 27 Lecturer: Lester Mackey Scribe: Viswajith Veugopal, Vivek Bagaria, Steve Yadlowsky Warig: These otes may cotai factual ad/or typographic errors..
More information