The Central Limit Theorem

Transcription

1 Chapter The Cetral Limit Theorem Deote by Z the stadard ormal radom variable with desity 2π e x2 /2. Lemma.. Ee itz = e t2 /2 Proof. We use the same calculatio as for the momet geeratig fuctio: exp(itx 2 x2 )dx = e t2 /2 exp( 2 (x it)2 )dx = 2π Theorem.2. Suppose {X } is i.i.d. with mea m ad variace 0 < σ 2 <. Let S = X + + X. The S m σ D Z This is oe of the special cases of the Lideberg theorem. Notice that i geeral ϕ S/ (t) is a complex umber. For example, whe X are expoetial, the coclusio says that which is ot so obvious to see. e it ( i t ) e t2 /2 Here is a simple iequality that will suffice for the proof i the geeral case. Lemma.3. If z,..., z m ad w,..., w m are complex umbers of modulus at most the (.) z... z m w... w m m z k w k 7

2 8. The Cetral Limit Theorem Proof. Write the left had side of (.) as a telescopig sum: m z... z m w... w m = z... z k (z k w k )w k+... w m (Omitted i 208) Example.. We show how to complete the proof for the expoetial distributio. e it ( ) ( ) e t2 /2 i t = e it/ i t (e t2 /(2) ) e it/ i t e t2 /(2) it/ + t 2 /(2) + it 3 /(6 )... = i t + t 2 /(2) t 4 /(6 2 ) +... ( = it ) ( t2 2 it i t ) t t2 2 t = ( t2 + t2 2 + i t t2 2 t C(t) 0. Proof of Theorem.2. Without loss of geerality we may assume m = 0 ad σ =. We have ϕ S/ (t) = ϕ X (t/ ). For a fixed t R choose large eough so that t2 2 >. For such, we ca apply (.) with z k = ϕ X (t/ ) ad w k = t2 2. We get ϕ S / (t) ( t2 2 ) ϕ X (t/ { } ) t2 t X 3 2 t2 E mi, X 2 Notig that lim mi{ t X 3 /, X 2 } = 0, by domiated covergece theorem (the itegrad { t X 3 } is domiated by the itegrable fuctio X 2 ) we have E mi, X 2 0 as. So lim ϕ S / (t) ( t2 = 0. 2 ) It remais to otice that ( t2 2 ) e t2 /2.. Lideberg ad Lyapuow theorems The settig is of sums of triagular arrays: For each we have a family of idepedet radom variables X... X,r ad we set S = X + + X,r. For Theorem.2, the triagular array ca be X,k = X k m σ. Or oe ca take X,k = X k m σ...

3 . Lideberg ad Lyapuow theorems 9 Through this sectio we assume that radom variables are square-itegrable with mea zero, ad we use the otatio (.2) E(X k ) = 0, σ 2 k = E(X2 k ), s2 = r σk 2 Defiio. (The Lideberg coditio). We say that the Lideberg coditio holds if r (.3) lim dp = 0 for all ε > 0 s 2 X k >εs X 2 k (Note that strict iequality X k >εs Xk 2 dp ca be replaced by X k εs Xk 2 dp ad the resultig coditio is the same.) Remark.4. Uder the Lideberg coditio, we have (.4) lim max σk 2 k r s 2 Ideed, So = 0 σk X 2 = Xk 2 dp + Xk 2 dp εs2 + Xk 2 dp k εs X k >εs X k >εs σk 2 max k r s 2 ε + s 2 max Xk 2 dp ε + k r X k >εs s 2 r X k >εs X 2 k dp Theorem.5 (Lideberg CLT). Suppose that for each the sequece X... X,r is idepedet with mea zero. If the Lideberg coditio holds for all ε > 0 the S /s D Z. Example.2 (Proof of Theorem.2). I the settig of Theorem.2, we have X,k = X k m σ ad s =. The Lideberg coditio is lim X k m >εσ (X k m) 2 σ 2 dp = lim σ 2 X m >εσ (X m) 2 = 0 by Lebesgue domiated covergece theorem, say. (Or by Corollary 6.6 o page 70.) Proof. Without loss of geerality we may assume that s 2 = so that r σk 2 =. Deote ϕ k = E(e itx k). From (0.3) we have (.5) ϕ k(t) ( 2 t2 σk 2 ) E ( mi{ tx k 2, tx k 3 } ) tx k 3 dp + tx k 2 dp t 3 εσk 2 + t2 Xk 2 dp X k <ε X k ε X k ε Usig (.), we see that (.6) ϕ S (t) ( 2 t2 σk 2 ) εt 3 σk 2 + t 2 X k >ε X 2 k dp

4 20. The Cetral Limit Theorem This shows that lim ϕ S (t) ( 2 t2 σk 2 ) = 0 It remais to verify that lim e t 2 /2 ( 2 t2 σ 2 k ) = 0. To do so, we apply the previous proof to the triagular array σ,k Z k of idepedet ormal radom variables. Note that So r ϕ Z k (t) = r e t2 σ 2 k /2 = e t2 /2 We oly eed to verify the Lideberg coditio for {Z k }: Z k >ε Z 2 k dp r σ 2 k Z k >ε Z 2 k dp = σ2 k x >ε/σ k x 2 f(x)dx x >ε/σ k x 2 f(x)dx max k r x 2 f(x)dx x >ε/σ k x 2 f(x)dx x >ε/ max k σ k The right had side goes to zero as, because by max k r σ k 0 by (.4). 2. Lyapuov s theorem Theorem.6. Suppose that for each the sequece X... X,r is idepedet with mea zero. If the Lyapuov s coditio (.7) lim s 2+δ E X k 2+δ = 0 holds. The S /s D Z Proof. We use the followig boud to verify Lideberg s coditio: r Xk 2 dp r X k >εs ε δ s 2+δ s 2 X k >εs X k 2+δ dp ε δ s 2+δ E X k 2+δ Corollary.7. Suppose X k are idepedet with mea zero, variace σ 2 ad that sup k E X k 2+δ <. The S / D σz. Proof. Let C = sup k E X k 2+δ The s = ad Lyapuov s coditio is satisfied. s 2+δ E( X k 2+δ ) C/ δ/2 0, so Corollary.8. Suppose X k are idepedet, uiformly bouded, ad have mea zero. If Var(X ) =, the S / Var(S ) D N(0, ).

5 Required Exercises 2 Proof. Suppose X C for a costat C. The s 3 E X 3 C s2 s 3 = C 0 s 3. Strategies for provig CLT without Lideberg coditio Oe basic idea is trucatio: X = X I X a +X I X >a. Oe wats to show that s Xk I Xk a Z ad that s Xk I Xk a P 0 Example.3. Let X, X 2,... be idepedet radom variables with the distributio (k ) Pr(X k = ±) = /4, Pr(X k = k k ) = /4 k, Pr(X k = 0) = /2 /4 k. The σk 2 = 2 +( ) k k 4 ad s /4 D. But S /s 0 ad i fact we have S / D Z/ 2. To see this, ote that Y k = X k I X k are idepedet with mea 0, variace 2 ad P (Y k X k ) = /4 k so by the first Borel Catelli Lemma (Theorem 3.7) (Y k X k ) U 0 with probability oe. Required Exercises Exercise.. Suppose a k is a array of umbers such that a2 k = ad max k a k 0. Let X j be i.i.d. with mea zero ad variace. Show that a kx k D Z. Exercise.2. Suppose that X, X 2,... are i.i.d., E(X ) =, E(X 2) = σ2 <. Let X = j= X j. Show that for all k > 0 ( ) D Xk N(0, kσ) as. Exercise.3. Suppose X, X 2,... are idepedet, X k = ± with probability 2 ( k 2 ) ad X k = ±k with probability 2 k 2. Let S = X k (i) Show that S / D N(0, ) (ii) Is the Lideberg coditio satisfied? Exercise.4. Suppose X, X 2,... are idepedet radom variables with distributio Pr(X k = ) = p k ad Pr(X k = 0) = p k. Prove that if V ar(x k ) = the (X k p k ) p k( p k ) D N(0, ).

6 22. The Cetral Limit Theorem Exercise.5. Suppose X k are idepedet ad have desity for x >. Show that x 3 N(0, ). S log Hit: Verify that Lyapuov s coditio (.7) holds with δ = for trucated radom variables. Several differet trucatios ca be used, but techical details differ: Y k = X k I Xk k is a solutio i [Billigsley]. To show that log (X k Y k ) P 0 use L -covergece. Triagular array Y k = X k I Xk is simpler computatioally Trucatio Y k = X k I Xk k log k leads to asymptotically equivalet sequeces. Exercise.6 (May 208). To be added latter Some previous prelim problems Exercise.7 (Aug 207). Let {X } N be a collectio of idepedet radom variables with P(X = ± 2 ) = 2 β ad P(X = 0) = β, N, where β (0, ) is fixed for all N. Cosider S := X + + X. Show that S D N (0, σ 2 γ ) for some σ > 0, γ > 0. Idetify σ ad γ as fuctios of β. You may use the formula k θ θ+ θ + for θ > 0, ad recall that by a b we mea lim a /b =. Exercise.8 (May 207). Let {X } N be idepedet radom variables with P(X = ) = / = P(X = 0). Let S := X + + X be the partial sum. (i) Show that (ii) Prove that ES Var(S ) lim = ad lim log log =. S log log D N (0, ) as. Explai which cetral limit theorem you use. State ad verify all the coditios clearly. Hit: recall the relatio lim /k =. log Exercise.9 (May 206).(a) State Lideberg Feller cetral limit theorem.

7 Some previous prelim problems 23 (b) Use Lideberg Feller cetral limit theorem to prove the followig. Cosider a triagular array of radom variables {Y,k } N,,..., such that for each, EY,k = 0, k =,...,, ad {Y,k },..., are idepedet. I additio, with σ := ( EY 2,k )/2, assume that Show that lim σ 4 EY,k 4 = 0. Y, + + Y, σ D N (0, ). Exercise.0 (Aug 205). Let {U } N be a collectio of i.i.d. radom variables with EU = 0 ad EU 2 = σ 2 (0, ). Cosider radom variables {X } N defied by X = U + U 2, N, ad the partial sum S = X + + X. Fid appropriate costats {a, b } N such that S b a D N (0, ). Exercise. (May 205). Let {U } N be a collectio of i.i.d. radom variables distributed uiformly o iterval (0, ). Cosider a triagular array of radom variables {X,k },...,, N defied as X,k = { Uk }. Fid costats {a, b } N such that X, + + X, b a D N (0, ). Exercise.2 (Aug 204). Let X, X 2,... be idepedet ad idetically distributed radom variables with Prove that P (X i = ) = P (X i = ) = / D kx k N(0, ) (You may use formulas j= j2 = 6 ( + )(2 + ) ad j= j3 = 4 2 ( + ) 2 without proof.) Exercise.3 (May 204). Let {X k : k =,...,, N} be a family of idepedet radom variables satisfyig ( P X k = k ) ( = P X k = k ) = P (X k = 0) = /3 Let S = X + +X. Prove that S /s coverges i distributio to a stadard ormal radom variable for a suitable sequece of real umbers s.

8 24. The Cetral Limit Theorem Some useful idetities: k = ( + ) 2 k 2 = ( + )(2 + ) 6 k 3 = 4 2 ( + ) 2 Exercise.4 (Aug 203). Suppose X, Y, X 2, Y 2,..., are idepedet idetically distributed with mea zero ad variace. For iteger, let 2 2 U = X j + Y j. j= Prove that lim P (U u) = e u/2 for u > 0. Exercise.5 (May 203). Suppose X,, X,2,... are idepedet radom variables cetered at expectatios (mea 0) ad set s 2 = E ( (X,k ) 2). Assume for all k that X,k M with probability ad that M /s 0. Let Y,i = 3X,i + X,i+. Show that Y, + Y, Y, s coverges i distributio ad fid the limitig distributio. j=

9 Bibliography [Billigsley] P. Billigsley, Probability ad Measure IIIrd editio [Durrett] R. Durrett, Probability: Theory ad Examples, Editio 4. (olie) [Gut] A. Gut, Probability: a graduate course [Resik] S. Resik, A Probability Path, Birkhause 998 [Proscha-Shaw] S M. Proscha ad P. Shaw, Essetial of Probability Theory for Statistitcias, CRC Press 206 [Varadha] S.R.S. Varadha, Probability Theory, (olie pdf from 2000) 35

10

11 Idex L metric, L 2 metric, L p-orm, 59, 76 λ-system, 25 π-system, 25 σ-field, 6 σ-field geerated by X, 4 distributio of a radom variable, 42 Beroulli radom variables, 46 Biomial distributio, 7, 73 bivariate cumulative distributio fuctio, 30 Boferroi s correctio, 8 Boole s iequality, 8 Borel σ-field, 4 Borel sigma-field, 6 Catelli s iequality, 65 cardiality, 9 Cauchy distributio, 4 Cauchy-Schwarz iequality, 58 cetered, 63 Cetral Limit Theorem, 7 characteristic fuctio, characteristic fuctio cotiuity theorem, 4 Characteristic fuctios uiqueess, 3 Characteristic fuctios iversio formula, 3 Chebyshev s iequality, 57 complex umbers, 08 cojugate expoets, 60 cotiuity coditio, 4 Cotiuous mappig theorem, 03 coverge i L p, 6 coverge i mea square, 6 covergece i distributio, 49, 99 coverges i distributio, 25 coverges i probability, 46 coverges poitwise, 7 coverges uiformly, 7 coverges with probability oe, 47 covex fuctio, 58 correlatio coefficiet, 59 coutable additivity, 4 covariace matrix, 28 cumulative distributio fuctio, 26, 43 cylidrical sets, 32, 33 cyllidircal sets, 32 de Moivre formula, 08 DeMorga s law, 8 desity fuctio, 29, 74 diadic iterval, 33 discrete radom variable, 73 discrete radom variables, 45 equal i distributio, 43 evets, 3, 7 expected value, 54, 69 Expoetial distributio, 74 expoetial distributio, 29 Fatou s lemma, 7 field, 3 fiite dimesioal distributios, 32 fiitely-additive probability measure, 4 Fubii s Theorem, 82 Geometric distributio, 73 Hölder s iequality, 60, 76 iclusio-exclusio, 8 idepedet σ-fields, 35 idepedet evets, 35 idepedet idetically distributed, 46 idepedet radom variables, 44 idicator fuctios, 9 iduced measure, 42 ifiite umber of tosses of a coi, 33 itegrable, 69 itersectio, 8 Jese s iequality, 58 joit cumulative distributio fuctio, 30 joit distributio of radom variables, 43 Kolmogorov s maximal iequality, 9

12 38 Idex Kolmogorov s oe series theorem, 9 Kolmogorov s three series theorem, 92 Kolmogorov s two series theorem, 92 Kolmogorov s zero-oe law, 90 Kolmogorov-Smirov metric, 0, Kroecker s Lemma, 94 Lévy distace, 06 law of X, 42 Lebesgue s domiated covergece theorem, 7, 72 Lebesgue s domiated covergece theorem used, 73, 89, 02, 5 Levy s metric, Levy s theorem, 93 Lideberg coditio, 9 Lyapuov s coditio, 20 Lyapuov s iequality, 58 margial cumulative distributio fuctios, 30 Markov s iequality, 57 maximal iequality, Etemadi s, 95 maximal iequality,kolmogorov s, 9 mea square covergece, 76 measurable fuctio, 4 measurable rectagle, 8 metric, 0 metric space, 0 Mikowki s iequality, 59 Mikowski s iequality, 76 momet geeratig fuctio, 56, 77 momets, 55 Mootoe Covergece Theorem, 69 multivariate ormal, 27 multivariate ormal distributio, 28 multivariate radom variable, 4 simple radom variables, 45 Skorohod s theorem, 0 Slutsky s Theorem, 0 Stadard ormal desity, 74 stochastic process with cotiuous trajectories, 43 stochastic processes, 42 stochastically bouded, 5 symmetric distributio, 97 tail σ-field, 36 Tail itegratio formula, 84 Taylor polyomials, 07 tight, 5 tight probability measure, 9 Toelli s theorem, 82 total variatio metric, trucatio of r.v., 50 ucorrelated, 63 uiform cotiuous, 28 Uiform desity, 74 uiform discrete, 28 uiform sigular, 28 uiformly itegrable, 72, 05 uio, 8 variace, 56 Waserstei distace, weak covergece, 49 weak law of large umbers, 63 zero-oe law, 36, 90 egative biomial distributio, 8 ormal distributio, 29 Poisso distributio, 8, 73 Polya s distributio, 8 Portmateau Theorem, 02 power set, 7 probability, 3 probability measure, 4 probability space, 3, 7 product measure, 82 quatile fuctio, 44, 0 radom elemet, 4 radom variable, 4 radom vector, 4 sample space, 3 Scheffe s theorem, 00 sectio, 8 semi-algebra, 5 semi-rig, 5 sigma-field geerated by A, 6 simple radom variable, 53