MIT 18.655 Dr. Kempthore Sprig 2016 1 MIT 18.655
Outlie 1 2 MIT 18.655
Beroulli s Weak Law of Large Numbers X 1, X 2,... iid Beroulli(θ). S i=1 = X i Biomial(, θ). S P θ. Proof: Apply Chebychev s Iequality, Khitchi s Weak Law of Large Numbers The X 1, X 2,... iid E [X 1 ] = µ, fiite 1 / S = i=1 X i S P µ. 3 MIT 18.655
Khitchi s WLLN Proof: If Var(X 1 ) <, apply Chebychev s Iequality. If Var(X 1 ) =, apply Levy-Cotiuity Theorem for characteristic fuctios. i φ (t) = E [e itx ] = E [e t X i X i=1 ] = i=1 φ X ( t ) = [φ X ( t )] = [1 + iµt + o( t )] e itµ. (characteristic fuctio of costat radom variable µ) So D P X µ, which implies X µ. 4 MIT 18.655
Limitig Momet-Geeratig Fuctios Cotiuity Theorem. Suppose X 1,..., X ad X are radom variables F 1 (t),..., F (t), ad F (t) are the correspodig sequece of cumulative distributio fuctios M X1 (t),..., M X (t) ad M X (t) are the correspodig sequece of momet geeratig fuctios. The if M (t) M(t) for all t i a ope iterval cotaiig zero, the F (x) F (x), at all cotiuity poits of F. i.e., L X X 5 MIT 18.655
Limitig Characteristic Fuctios Levy Cotiuity Theorem. Suppose X 1,..., X ad X are radom variables ad φ t (t),..., φ (t) ad φ X (t) are the correspodig sequece of characteristic fuctios. The L X X if ad oly if lim φ (t) = φ X (t), for all t R. Proof: See http://wiki.math.toroto.edu/torotomathwiki/images/ 0/00/MAT1000DaielRuedt.pdf 6 MIT 18.655
Outlie 1 7 MIT 18.655
De Moivre-Laplace Theorem If {S } is a sequece of Biomial(, θ) radom variables, (0 < θ < 1), the S θ L - Z, θ(1 θ) where Z has a stadard ormal distributio. Applyig the Cotiuity Correctio : P[k S m] = P k 1 S m + 1 [ 2 2 1 k 1 θ m+ θ 2 S θ 2 θ(1 θ) θ(1 θ) θ(1 θ) = P [ - [ - 1 m+ θ k 1 θ 2 2 Φ Φ θ(1 θ) θ(1 θ) 8 MIT 18.655
Cetral Limit Theorem X 1, X 2,... iid E [X 1 ] = µ, ad Var[X 1 ] = σ 2, both fiite (σ 2 > 0). = S i=1 X i The S µ L Z, σ where Z has a stadard ormal distributio. Equivaletly: ( 1 ṉ S µ) L Z. σ 2 / 9 MIT 18.655
: Cetral Limit Theorem Limitig Distributio of X X 1,..., X iid with µ = E [X ], ad mgf M X (t). i=1 X i The mgf of X = is M (t) = E[e tx ] = E [e 1 t X i X ] = E [e t X i ] = 1 1 M X ( t ) = [M X ( t )] Applyig Taylor s expasio, there exists t 1 : 0 < t 1 < t/: M X ( t ) = M X (0) + M ' (t 1 ) t X [M ' (t 1 ) M ' (0)]t = 1 + µt X + So t µt [M X ' (t 1 ) M ' (0)]t lim [M X ( )] = lim [1 + + ] = e µt P So X µ. 10 MIT 18.655
: Cetral Limit Theorem X µ (X µ) S µ Defie Z = = = σ/ σ σ where S = 1 X i. Note: Z : E [Z ] 0 ad Var[Z ] 1. M Z (t) = E [e tz ] = E exp{ t X i µ } i=1 σ t = i=1 M Y ( ) D X i µ ' '' σ Y Y where Y =, E [Y ] = 0 = M (0), ad E [Y 2 ] = 1 = M (0), ad M Y (t) = E [e ty ]. By Taylor s expasio, t 1 (0, t/ ) : t ' t '' t M Y ( ) = M Y (0) + M (0)( ) + 1 M (t 1 )( ) 2 Y 2 Y [MY yy (t 1 ) 1]t = 1 + t2 + 2 2 2 11 MIT 18.655
: Cetral Limit Theorem '' Y Sice lim [M (t 1 ) 1] = 1 1 = 0, '' t 2 [M Y (t 1 ) 1]t 2 lim M Z (t) = lim [1 + + ] 2 2 2 + t L = e 2, the mgf of a N(0, 1) so Z N(0, 1). 12 MIT 18.655
Classic Limit Theorem Examples Poisso Limit of Biomials {X }: X Biomial(, θ ) where lim θ = 0 lim θ = λ, with 0 < λ < L X Poisso(λ). 13 MIT 18.655
Classic Limit Theorem Examples Sample Mea of Cauchy Distributio X 1,..., X i.i.d. Cauchy(µ, γ) r.v.s; µ R, γ > 0 1 1 f (x θ) = ( ), < x <. πγ 1+( x µ ) γ 2 The characteristic fuctio of the Cauchy is φ X (t) = exp{iµt γ t } The characteristic fuctio of the sample mea is: t X i ] φ X (t) = E [e i 1 = i=1 φ t X ( t ) = [exp{iµ γ t }] = exp{iµt γ t } D So X = X 1 for every(!). 14 MIT 18.655
Berry-Essee Theorem If X 1,..., X iid with mea µ ad variace σ 2 > >>> 0, [ the for all, - > S µ >>> > P C E [ X 1 µ 3 ] sup t Φ(t). σ σ 3 t C = 33 4 : B&D, A.15.12. C = 0.4748: http://e.wikipedia.org/wiki/berry-essee_theorem, Shevtsova (2011) 15 MIT 18.655
Asymptotic Order Notatio U = o P (1) iff U P 0. U = O P (1) iff E > 0, M <, such that P[ U M] E U = o P (V ) iff U V = o P (1) U = O P (V ) iff U V = O P (1) 16 MIT 18.655
Asymptotic Order Notatio Example: Z 1,..., Z are iid as Z If E [ Z ] <, the by WLLN: Z = µ + o P (1). where µ = E [Z ]. If E [ Z 2 ] <, the by CLT: Z = µ + O P ( 1 ). 17 MIT 18.655
MIT OpeCourseWare http://ocw.mit.edu 18.655 Mathematical Statistics Sprig 2016 For iformatio about citig these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.