2.2. Central limit theorem.

Transcription

1 36.. Cetral limit theorem. The most ideal case of the CLT is that the radom variables are iid with fiite variace. Although it is a special case of the more geeral Lideberg-Feller CLT, it is most stadard ad its proof cotais the essetial igrediets to establish more geeral CLT. Throughout the chapter, Φ( is the cdf of stadard ormal distributio N(0,. (i. Cetral limit theorem (CLT for iid r.v.s. The followig lemma plays a key role i the proof of CLT. Lemma. For ay real x ad, e ix (ix 0! ( x + mi ( +!, x.! Cosequetly, for ay r.v. X with characteristic fuctio ψ ad fiite secod momet, ψ(t [ + ite(x t E(X ] t 6 E(mi( t X 3, 6 X. (. Proof. The proof relies o the idetity e ix (ix! 0 i+! x 0 (x s e is ds i (! x 0 (x s (e is ds, which ca be show by iductio ad by takig derivatives. x + /( +!, ad the last bouded by x /!. The middle term is bouded by Theorem. Suppose X, X,..., X,... are iid with mea µ ad fiite variace σ > 0. The, S µ σ N(0, i distributio. Proof. Without loss of geerality, let µ 0. Let ψ be the commo characteristic fuctio of X i. Observe that, by domiated covergece E(mi( t X 3, 6 X 0 as t 0 The characteristic fuctio of S / σ is, by applyig the above lemma, E(e its / σ E(e it S E(e itx / σ ψ t ( σ [ + it E(X t σ σ E(X + o( ] [ t + o( ] e t /, which is the characteristic fuctio of N(0,. The, Levy s cotiuity theorem implies the above CLT. I the case the commo variace is ot fiite, the partial sum, after proper ormalizatio, may or may ot coverge to a ormal distributio. The followig theorem provides sufficiet ad ecessary coditio. The key poit here is whether there exists appropriate trucatio, which is a trick that we have used so may times before.

2 Theorem.3 Suppose X, X, X,... are iid odegeerate. The, (S a /b coverges to a ormal distributio for some costats a ad 0 < b, if ad oly if x P ( X > x E(X 0, as x. (. { X x} The proof is omitted. We ote that (. holds if X i has fiite variace σ > 0, i which case CLT of Theorem. holds with a E(X ad b σ. Theorem.3 is of iterest whe E(X. I this case, oe ca choose to trucate the X i s at With some calculatio, coditio (. esures c sup{c : E( X { X c} /c } P ( X > c 0 ad E( X { X c}/c. Separate S ito two parts, oe with X i beyod ±c ad the other bouded by ±c. The former takes value 0 with chace goig to. The latter, whe stadardized by a E(X { X c } ad b E(X { X c } c. coverges to N(0,, which ca be show by repeatig the proof of Theorem. or by citig Lideberg-Feller CLT. We ote that b var(x { X c} by (.. 37 Example.5 Recall Example.3, i which X, X, X,... are iid symmetric such that P ( X > x x α for some α > 0 all large x. The, Theorem.3 implies (S a /b N(0, if ad oly if α. Ideed, whe α >, the commo variace is fiite ad CLT applies. Whe α, for some σ. S /( log / N(0, σ Whe α <, the coditio i Theorem caot hold. As to be see i Sectio.3, S whe properly ormalized shall coverge to o-ormal distributio. (ii. The Lideberg-Feller CLT. Theorem.4 Lideberg-Feller CLT. Suppose X,..., X,... are idepedet r.v.s with mea 0 ad variace σ. Let s σ deote the variace of partial sum S X + + X. If, for every ɛ > 0, E(X { X >ɛs } 0, (.3 s the S /s N(0,. Coversely, if max σ /s 0 ad S /s N(0,, the (.3 holds. Proof. The Lideberg coditio (.3 implies ( σ max s ɛ + s max E(X { X >ɛs } 0, (.4 by lettig ad the ɛ 0. Observe that for every real x > 0, e x +x x /. Moreover, for complex z ad w with z ad w, z w z w, (.5

3 38 which ca be proved by iductio. With Lemma., it follows that, for ay ɛ > 0, E(e itx /s e t σ /s E ( + itx (tx s ( t σ ( t X ( tx 3 E s { X >ɛs } + E 6s 3 t s The, for ay fixed t, s [ ( t X + E mi { X ɛs } E(X { X >ɛs } + t 3 ɛ s E(X + t4 σ s E(e its /s e t / E(e itx /s e t σ /s s + t4 σ 4 8s 4 σk max k s E(e itx/s e t σ /s by (.5 ( t ( t s s E(X { X >ɛs } + t 3 ɛ s E(X + t4 σ s E(X { X >ɛs } + ɛ t 3 + t 4 max σ s ɛ t 3, as, by (.3 ad (.4., tx 3 ] 6s 3 + t4 σ 4 8s 4 σ max s Sice ɛ > 0 is arbitrary, it follows that E(e its/s e t / for all t. Levy s cotiuity theorem implies S /s N(0,. Let ψ be the momet geeratig fuctio of X. The asymptotic ormality is equivalet to ψ (t/s e t /. Notice that (. implies Write, as, [ψ (t/s ] + t / ψ (t/s t σ s (.6 [ψ (t/s log ψ (t/s ] + [log ψ (t/s ] + t / ψ (t/s log ψ (t/s + +o( ψ (t/s + o( max ψ k(t/s k t σk 4 max k s ψ (t/s + o( t σ s + o( by (.6 o(, by the assumptio max σ /s 0.

4 39 O the other had, by defiitio of characteristic fuctio, the above expressio is, as, o( [ψ (t/s ] + t / E(e itx /s + t / E(cos(tX /s + t / + i E(si(tX /s E{(cos(tX /s { X >ɛs }} + +imagiary part (immaterial. E{(cos(tX /s { X ɛs }} + t / Sice cos(x x / for all real x, s E(X {X >ɛs } t E( t X s {X ɛs } ( t t + E{(cos(tX /s { X ɛs }} t ( E{(cos(tX /s { X >ɛs }} + o( t P ( X > ɛs + o( 4 t 4 t ɛ + o(. σ (ɛs + o( Sice t ca be chose arbitrarily large, Lideberg coditio holds. by Chebyshev iequality Remark. Sufficiecy is proved by Lideberg i 9 ad ecessity by Feller i 935. Lideberg- Feller CLT is oe of the most far-reachig results i probability theory. Nearly all geeralizatios of various types of cetral limit theorems spi from Lideberg-Feller CLT, such as, for example, CLT for martigales, for reewal proceses, or for weakly depedet processes. The isights of the Lideberg coditio (.3 are that the wild values of the radom variables, compared with s, the stadard deviatio of S as the ormalizig costat, are isigificat ad ca be trucated off without affectig the geeral behavior of the partial sum S. Example.6. Suppose X are idepedet ad P (X P (X α /4 ad P (X 0 α /, with 0 < α < 3. The, σ E(X α / ad s α /, which icreases to at the order of 3 α. Note that Lideberg coditio (.3 is equivalet to / 3 α 0, i.e., 0 < α <. O the other had, max σ /s 0. Therefore, it follows from Theorem.4 that S /s N(0, if ad oly if 0 < α <. Example.7 Suppose X are idepedet ad P (X / P (X 0. The, [S log(]/ log( N(0, i distributio. It s clear that E(X / ad var(x ( //. So, E(S i i /i, ad var(s i ( /i/i log(. As X are all bouded by ad var(s, the Lideberg

5 40 coditio is satisfied. Therefore, by the CLT, S i /i [ N(0,, i distributio. i ( /i/i]/ The, [S log(]/ log( N(0, i distributio sice log( i /i ad var(s / log(. Theorem. as well as the followig Lyapuov CLT are both special cases of the Lideberg-Feller CLT. Nevertheles they are coveiet for applicatio. Corollary (Lyapuov CLT Suppose X are idedet with mea 0 ad E( X δ /s δ 0 for some δ >, the S /s N(0,. Proof. For ay ɛ > 0, as, s E(X { X >ɛs } E( X s { X /s >ɛ} ɛ δ E( Xδ s δ 0. Lideberg coditio (.3 holds ad hece CLT holds. I Example.6, for ay δ >, E X δ δ α / which icreasig at the order δ α+, while s δ icreases at the order of (3 αδ/. Simple calculatio shows, whe 0 < α <, Lypuov CLT holds. (iii. CLT for arrays of radom variables. Very ofte Lideberg-Feller CLT is preseted i the form of arrays of radom variables as give i the textbook. Theorem.5 (CLT for arrays of r.v.s Let X,,..., X, be idepedet radom variables with mea 0 such that, as, var(x, ad The, S X, + + X, N(0,. E(X, { X, >ɛ} 0, for ay ɛ > 0. This theorem is slightly more geeral tha Lideberg-Feller CLT, although the proof is idetical to that of the first part of Theorem.4. Theorem.4. is a special case of Theorem.5 by lettig X,i X i /s. Thus X,k are udertood as the usual r.v.s ormalized by the stadard deviatio of the partial sums. Thus S i this theorem is already stadardized. DIY Exercises Exercise.5 Suppse X are idepedet with P (X α P (X α β ad P (X 0 β with α > β. Show that the Lideberg coditio holds if ad oly if 0 β <. Exercise.6 Suppose X are iid with mea 0 ad variace. Let a > 0 be such that s a i ad a /s 0. Show that i a ix i /s N(0,. Exercise.7 Suppose X, X... are idepedet ad X Y + Z, where Y takes values ad with chace / each, ad P (Z ± /( ( P (Z 0/ Show that Lideberg coditio does ot hold, yet S / N(0,. Exercise.8 Suppse X, X,... are iid oegative r.v.s with mea ad fiite variace σ > 0. Show that ( S N(0,.