The Lindeberg central limit theorem

Transcription

1 The Lindeberg central limit theorem Jordan Bell Department of Mathematics, University of Toronto May 29, 205 Convergence in distribution We denote by P d the collection of Borel probability measures on d. Unless we say otherwise, we use the narrow topology on P d : the coarsest topology such that for each f C b d, the map µ fdµ d is continuous P d C. Because d is a Polish space it follows that P d is a Polish space. In fact, its topology is induced by the Prokhorov metric. 2 2 Characteristic functions For µ P d, we define its characteristic function µ : d C by µu = e iu x dµx. d Theorem. If µ P has finite kth moment, k 0, then, writing φ = µ:. φ C k. 2. φ k v = i k xk e ivx dµx. 3. φ k is uniformly continuous. 4. φ k v x k dµx. Charalambos D. Aliprantis and Kim C. Border, Infinite Dimensional Analysis: A Hitchhiker s Guide, third ed., p. 55, Theorem 5.5; jordanbell/notes/narrow.pdf 2 Onno van Gaans, Probability measures on metric spaces, nl/~vangaans/jancol.pdf; Bert Fristedt and Lawrence Gray, A Modern Approach to Probability Theory, p. 365, Theorem 25.

2 Proof. For 0 l k, define f l : C by f l v = x l e ivx dµx. For h 0, xl e ivx eihx h xl x = x l+, so by the dominated convergence theorem we have for 0 l k, That is, f l v + h f l v lim = lim h 0 h h 0 = x l e ivx lim h 0 = ix l+ e ivx dµx. f l = if l+. x l e ivx eihx dµx h e ihx h dµx And, by the dominated convergence, for ɛ > 0 there is some δ > 0 such that if w < δ then x k e iwx dµx < ɛ, hence if v u < δ then f k v f k u = x k e iux e iv ux dµx x k e iv ux dµx < ɛ, showing that f k is uniformly continuous. As well, f k v x k dµx But φ = f 0, i.e. φ 0 = f 0, so φ = f 0 = if, φ 2 = if = i 2 f 2,, φ k = i k f k. 2

3 If φ C k, Taylor s theorem tells us that for each x, φx = = = k l=0 l=0 l=0 φ l 0 x x l + l! 0 φ l 0 x x l + l! 0 φ l 0 x l + k x, l! x t k φ k tdt k! x t k φ k t φ k 0dt k! and k x satisfies k x sup 0 u φ k ux φ k 0 Define θ k : C by θ k 0 = 0 and for x 0 x k k!. with which, for all x, θ k x = k! x k kx, φx = l=0 φ l 0 x l + l! k! θ kxx k. Because k is continuous on, θ k is continuous at each x 0. Moreover, θ k x sup φ k ux φ k 0, 0 u and as φ k is continuous it follows that θ k is continuous at 0. Thus θ k is continuous on. Lemma 2. If µ P have finite kth moment, k 0, and for 0 l k, M l = x l dµx, then there is a continuous function θ : C for which The function θ satisfies µx = θx l=0 i l M l x l + l! k! θxxk. sup µ k ux µ k 0. 0 u 3

4 Proof. From Theorem, µ C k and µ l 0 = i l x l dµx = i l M l. Thus from what we worked out above with Taylor s theorem, µx = l=0 i l M l x l + l! k! θ kxx k, for which θ k x sup µ k ux µ k 0. 0 u For a and σ > 0, let pt, a, σ 2 = σ 2π exp t a2 2σ 2, t. Let γ a,σ 2 be the measure on whose density with respect to Lebesgue measure is p, a, σ 2. We call γ a,σ 2 a Gaussian measure. We calculate that the first moment of γ a,σ 2 is a and that its second moment is σ 2. We also calculate that γ a,σ 2x = exp iax 2 σ2 x 2. Lévy s continuity theorem is the following. 3 Theorem 3 Lévy s continuity theorem. Let µ n be a sequence in P d.. If µ P d and µ n µ, then for each µ n converges to µ pointwise. 2. If there is some function φ : d C to which µ n converges pointwise and φ is continuous at 0, then there is some µ P d such that φ = µ and such that µ n µ. 3 The Lindeberg condition, the Lyapunov condition, the Feller condition, and asymptotic negligibility Let Ω, F, P be a probability and let X n, n, be independent L 2 random variables. We specify when we impose other hypotheses on them; in particular, 3 p. 9, Theorem 5. 4

5 we specfify if we suppose them to be identically distributed or to belong to L p for p > 2. For a random variable X, write Write σx = VarX = E X EX 2. σ n = σx n, and, using that the X n are independent, = σ j= X j = j= σ 2 j /2 and For n and ɛ > 0, define η n = EX n. L n ɛ = s 2 n = s 2 n EX j η j 2 X j η j ɛ j= j= x η j ɛ x η j 2 dx j P x. We say that the sequence X n satisfies the Lindeberg condition if for each ɛ > 0, lim L nɛ = 0. so For example, if the sequence X n is identically distributed, then s 2 n = nσ 2, L n ɛ = nσ 2 j= = σ 2 x η ɛn /2 σ x η 2 dx P x x η ɛn /2 σ x η 2 dx P. But if µ is a Borel probability measure on and f L µ and K n is a sequence of compact sets that exhaust, then 4 \K n f dµ 0, n. Hence L n ɛ 0 a, showing that X n satisfies the Lindeberg condition. 4 V. I. Bogachev, Measure Theory, volume I, p. 25, Proposition

6 We say that the sequence X n satisfies the Lyapunov condition if there is some δ > 0 such that the X n are L 2+δ and lim s 2+δ n E X j η j 2+δ = 0. j= In this case, for ɛ > 0, then x η ɛ implies x η 2+δ x η 2 ɛ δ and hence L n ɛ s 2 n x η j x ηj ɛsn 2+δ ɛ δ dx j P x j= = ɛs 2+δ n = ɛs 2+δ n ɛs 2+δ n 0. x η j 2+δ dx j P x x η j ɛ X j η j 2+δ dp X j η j ɛ E X j η j 2+δ j= j= j= This is true for each ɛ > 0, showing that if X n satisfies the Lyapunov condition then it satisfies the Lindeberg condition. For example, if X n are identically distributed and L 2+δ, then s 2+δ n E X j η j 2+δ = j= n δ/2 σ 2+δ E X j η j 2+δ 0, showing that X n satisfies the Lyapunov condition. Another example: Suppose that the sequence X n is bounded by M almost surely and that. X n M almost surely implies that η n = EX n E X n EM = M. Therefore X n η n X n + η n 2M almost surely. Let δ > 0. Then, as s 2 n = nσ 2, s 2+δ n j= E X j η j 2+δ s 2+δ n = 2Mδ s δ n 0, N E X j η j 2 2M δ j= showing that X n satisfies the Lyapunov condition. 6

7 We say that a sequence of random variables X n satisfies the Feller condition when lim max σ j = 0, j n where σ j = σx j = VarX j and = j= We prove that if a sequence satisfies the Lindeberg condition then it satisfies the Feller condition. 5 Lemma 4. If a sequence of random variables X n satisfies the Lindeberg condition, then it satisfies the Feller condition. Proof. Let ɛ > 0 For n and k n, we calculate σk 2 = x η k 2 dx k P x = x η k 2 dx k P x + x η k 2 dx k P x x η k <ɛ x η k ɛ ɛ 2 + x η j 2 dx j P x x η j ɛ Hence j= = ɛ 2 s 2 n + s 2 nl n ɛ. σk max k n σ 2 j /2 2 ɛ 2 + L n ɛ, and so, because the X n satisfy the Lindeberg condition, lim sup max k n This is true for all ɛ > 0, which yields lim max k n σk σk. 2 ɛ 2. 2 = 0, namely, that the X n satisfy the Feller condition. We do not use the following idea of an asymptotically negligible family of random variables elsewhere, and merely take this as an excsuse to write out 5 Heinz Bauer, Probability Theory, p. 235, Lemma

8 what it means. A family of random variables X, n, j k n, is called asymptotically negligible 6 if for each ɛ > 0, lim max P X ɛ = 0. j k n A sequence of random variables X n converging in probability to 0 is equivalent to it being asymptotically negligible, with k n = for each n. For example, suppose that X are L 2 random variables each with EX = 0 and that they satisfy lim For ɛ > 0, by Chebyshev s inequality, whence max VarX = 0. j k n P X ɛ ɛ 2 E X 2 = ɛ 2 VarX, lim max P X ɛ lim sup max j k n j k n ɛ 2 VarX = 0, and so the random variables X are asymptotically negligible. Another example: Suppose that random variables X are identically distributed, with µ = X P. For ɛ > 0, X P n ɛ = P X nɛ = µa n, where A n = {x : x nɛ}. As A n, lim µa n = 0. Hence the random variables X n are asymptotic negligible. The following is a statement about the characteristic functions of an asymptotically negligible family of random variables. 7 Lemma 5. Suppose that a family X, n, j k n, of random variables is asymptotically negligible, and write µ = X P and φ = µ. For each x, lim max φ x = 0. j k n Proof. For any real t, e it t. For x, ɛ > 0, n, and j k n, φ x = e ixy dµ y e ixy dµ y + e ixy dµ y y <ɛ y <ɛ y ɛ xy dµ y + 2dµ y y ɛ ɛ x + 2P X ɛ. 6 Heinz Bauer, Probability Theory, p. 225, Heinz Bauer, Probability Theory, p. 227, Lemma

9 Hence max φ x ɛ x + 2 max P X ɛ. j k n j k n Using that the family X is asymptotically negligible, lim sup But this is true for all ɛ > 0, so proving the claim. lim sup max φ x 2ɛ x. j k n max φ x = 0, j k n 4 The Lindeberg central limit theorem We now prove the Lindeberg central limit theorem. 8 Theorem 6 Lindeberg central limit theorem. If X n is a sequence of independent L 2 random variables that satisfy the Lindeberg condition, then S n P γ, where S n = X j η j = j= n j= X j EX j σx + + X n. Proof. The sequence Y n = X n EX n are independent L 2 random variables that satisfy the Lindeberg condition and σy n = σx n. Proving the claim for the sequence Y n will prove the claim for the sequence X n, and thus it suffices to prove the claim when EX n = 0, i.e. η n = 0. For n and j n, let µ = The first moment of µ is Xj xd Xj P and the second moment of µ is x 2 d Xj P x = Ω P and τ = σ j. X j x = dp = EX j = 0, Ω Xj 8 Heinz Bauer, Probability Theory, p. 235, Theorem dp = s 2 EXj 2 = σ2 j n s 2 = τ, 2 n 9

10 for which j= τ 2 = s 2 n σj 2 =. j= For µ P with first moment xdµx = 0 and second moment x2 dµx = σ 2 <, Lemma 2 tells us that with µx = M 0 + im x M 2 2 x2 + 2 θ 2xx 2 = σ2 2 x2 + 2 θxx2, But by Lemma, so θx θx sup µ ux µ 0. 0 u µ ux = y 2 e iuxy dµy, sup 0 u sup 0 u y 2 e iuxy + dµy y 2 e iuxy dµy. For 0 { u }, e iuxy uxy xy, so for x and ɛ > 0, with ɛ δ = min ɛ, x, when y < δ and 0 u we have e iuxy < ɛ. Thus θx sup y 2 e iuxy dµy + sup y 2 e iuxy dµy 0 u ɛ y <δ ɛσ y <δ y 2 dµy + 2 y 2 dµy. y 2 dµy 0 u { } ɛ Let x and ɛ > 0, and take δ = min ɛ, x. On the one hand, for n and j n, because the first moment of µ is 0 and its second moment is τ 2, µ x = τ 2 2 x2 + 2 θ xx 2, with, from the above, θ x ɛτ y 2 dµ y. On the other hand, the first moment of the Gaussian measure γ 0,τ 2 is 0 and its second moment is τ 2. Its characteristic function is γ 0,τ 2 x = exp τ 2 2 x2 = τ 2 2 x2 + 2 ψ xx 2, 0

11 with, from the above, ψ x ɛτ In particular, for all x, y 2 dγ 0,τ 2 x. µ x γ 0,τ 2 x = x2 2 θ x ψ x. For k and for a l, b l C, l k, k k a l b l = l= l= l= b b l a l b l a l+ a k. l= If further a l, b l, then k k a l b l l= a l b l. l= Because the X n are independent, the distribution of S n = j= X j is the convolution of the distributions of the summands: whose characteristic function is µ n, µ n,n, φ n = n µ, j= since the characteristic function of a convolution of measures is the product of the characteristic functions of the measures. Using n j= τ 2 = and, for x we have φ n x e x2 2 = n n µ x e 2 τ 2 x2 j= j= µ x e 2 τ 2 x2 = j= µ x γ 0,τ 2 x j= = x2 2 θ x ψ x. j=

12 { } ɛ Therefore, for x, ɛ > 0, and δ = min ɛ, x, φ n x e x2 2 x2 ɛτ y 2 dµ y + ɛτ y 2 dγ 2 0,τ 2 y j= n n =ɛx 2 + x 2 y 2 dµ y + x 2 y 2 dγ 0,τ 2 y. We calculate j= L n δ = s 2 n = s 2 n = = = j= y 2 dx j P y j= Xj 2 dp j= X j δ 2 Xj dp j= j= j= X j δ sn y 2 d Xj y 2 dµ y. P y Hence, the fact that the X n satisfy the Lindeberg condition yields lim sup φ n x e x2 2 ɛx 2 + x 2 lim sup y 2 dγ 0,τ 2 y. 2 Write We calculate j= y 2 dγ 0,τ 2 y = j= α n = max τ σ j = max. j n j n j= y 2 exp y2 τ 2π 2τ 2 dy = τ 2 u 2 dγ 0, u j= u δ/τ τ 2 u 2 dγ 0, u j= u δ/α n = u 2 dγ 0, u. u δ/α n 2

13 Because the sequence X n satisfies the Lindeberg condition, by Lemma 4 it satisfies the Feller condition, which means that α n 0 a. Because α n 0 a, δ/α n a, hence u 2 dγ 0, u 0 u δ/α n a. Thus we get y 2 dγ 0,τ 2 y 0 j= a. Using this with 2 yields This is true for all ɛ > 0, so lim sup φ n x e x2 2 ɛx 2. lim φ nx e x 2 = 0, namely, φ n the characteristic function of S n P converges pointwise to e x2 2. Moreover, e x2 2 is indeed continuous at 0, and e x2 2 = γ 0, x. Therefore, Lévy s continuity theorem Theorem 3 tells us that S n P convergearrowly to γ 0,, which is the claim. 2 3