The Central Limit Theorem

Transcription

1 The Central Limit Theorem (A rounding-corners overiew of the proof for a.s. convergence assuming i.i.d.r.v. with 2 moments in L 1, provided swaps of lim-ops are legitimate) If {X k } n k=1 are i.i.d., all X k f, E(X k ) = µ and finite Var(X k ) = σ 2, and n 1 Z n = X k nµ σ = X n µ n σ/ n f Z n, then, under certain regularity and decay conditions of the moments of f, f Zn (x) a.s. e x2 /2, n. The Central Limit Theorem 1 / 9

2 Sums of independent r.v. are convolutions The convolution of two densities f 1,2 is given by f 1 f 2 (x) = def f 1 (x t)f 2 (t) dt = f 2 f 1 (t). Let X 1 f 1 and X 2 f 2 be independent. Then X 1 + X 2 f 1 f 2 : Let Y = X 1 + X 2. Then its cdf F Y = P(X 1 + X 2 y) is given by F Y (y) = x 1 +x 2 y = f 1 (x 1 )f 2 (x 2 ) dx 1 dx 2 = f 1 (x 1 )F 2 (y x 1 ) dx 1 = f 1 F 2 (y). y x1 f 1 (x 1 ) dx 1 f 2 (x 2 ) dx 2 Now differentiate under the integral sign: f Y (y) = F Y (y) = f 1 f 2 (y). The Central Limit Theorem 2 / 9

3 The Fourier transform (CF) of a probability density The FT (the characteristic function, CF) ˆf (ω) = Ff (ω) of a density f : Ff (ω) = def f (x) e iωx dx F 1ˆf (x) = 1 def f (x) e iωx dω. Uniqueness: If f 1,2 L 1 and continuous, f 1 f 2 ˆf 1 ˆf 2. The Fourier transform is an automorphism on the Schwartz class S(). The FT maps convolutions products: ( f 1 f 2 )(w) = ˆf 1 (ω)ˆf 2 (ω) Pf : ( f 1 f 2 )(ω) = e ixω ( f 1 (x t)f 2 (t) dt) dx = f 2 (t) e itω ( f 1 (x t)e iω(x t) dx) dt = ˆf 1 (ω) f 2 (x) e ixω dx = ˆf 1 (ω)ˆf 2 (ω). The Central Limit Theorem 3 / 9

4 If X f : E(X ) = iˆf (0), E(X 2 ) = ˆf X (0) If f is a distribution and X f, then ˆf (0) = f (x) e 0 dx = E(1) = 1. The Fourier transform maps derivatives moments, and in particular, ( d ) kˆf (ω) = ( ix) k f (x) e iωx dx dω {ˆf (0) = ie(x ) ˆf (0) = E(X 2 ). Note that: If E(X ) = 0, then Var(X ) = E(X 2 ) = ˆf (0). Standard (µ, σ 2 ) = (0, 1) always achievable by X X µ σ If f 1,2 are distributions E(f 1 f 2 ) = ( f 1 f 2 )(0) = ˆf 1 (0)ˆf 2 (0) = 1. The Central Limit Theorem 4 / 9

5 Scaled r.v. Y = ax, density and its Fourier transform: The density f ax of a scaled r.v. Y = ax : If X f X and a > 0, then the scaled r.v. Y = ax 1 a f X ( a ). 1 ( y ) 1 = f X (x) dx = {y = ax} = a f X dy. a Now, the Fourier transform of the density of a scaled r.v: For a > 0, the Fourier transform ˆf ax (ω) = ˆf X (aω): ˆf ax (ω) = 1 f X ( x a a )e ixω dx = f X (t)e it(aω) dx = ˆf X (aω). The Central Limit Theorem 5 / 9

6 The Gaussian is an eigenfunction for the FT: Fg = g Pf. Let g denote the Gaussian ( -scaled pdf of the normal distribution) g(x) = e x2 /2. Its FT is given by ĝ(ω) = e x2 /2 ixω dx = e (iω)2 /2 where φ( + iω) = e (x+iω)2 /2 dx; φ( ) = e (x+iω)2 /2 dx = g(ω)φ( + iω), e x2 /2 dx =. Will use that the Gaussian g(z) is an entire function to show that φ( + iω) = = const, ω. The Central Limit Theorem 6 / 9

7 The Gaussian = Gaussian, Fg = g (cont.) If γ be the rectangular contour in the complex plane γ : ( + iω) ( + iω) ( ) cycle, then the circulation of g around γ is zero: 0 = g(z) dz = γ [g(x) g(x + iω)] dx + }{{} φ( ) φ( +iω) ω 0 [g( +iy) g( +iy)] dy. The integrals on the vertical segments vanish rapidly far out, ω 0 g(± + iy) dy ω e (2 ω 2 )/2 0,. Now pass to yield the result: 0 = lim g(z) dz = φ( ) φ( + iω) Fg(ω) = g(ω). γ The Central Limit Theorem 7 / 9

8 Last, recap of two additional results: esult 1: F 1 ĝ(ω) = g(x) = 1 e x2 /2, the density of N(0, 1). Just a rephrasing of the fact that ĝ = Fg = g.. [ esult 2: For any fixed a, lim 1 + a ] n n n + o( 1 n ) = e a. For any fixed a and all sufficiently small ε > 0, ln[1 + aε + o(ε)] = [ξ ξ2 2 + ξ3 3 + ] ξ=aε+o(ε) = aε + o(ε). Hence, for fixed a and large integers n, [ 1 + a n + o( 1 n ) ] n = e n ln[1+a/n+o(n 1 )] = e n[a/n+o(n 1 )] = e a e o(1). The Central Limit Theorem 8 / 9

9 The Central Limit Theorem Let {X k } n 1 be i.i.d., X k f, E(X k ) = 0, Var(X k ) = 1 = E(Xk 2 ). Then, if n Z n = S n n, S n = k=1 X k, then f Zn (x) e x2 /2, n. Pf. By esult 1, it suffices to show that the CF, Ff Zn (ω) = e ω2 /2. Now ˆf Zn (ω) = ( f 1 n S n )(ω) = ( f Sn ) ( ω n ) = F(f f ) ( ω n ) = [ˆf ( ω n ) ] n. Maclaurin-expand f Zn & use ˆf (0) = ie(x ) = 0, ˆf (0) = E(X 2 ) = 1: ˆf Zn (ω) = [ˆf (0) + ˆf (0) 1! ω + ˆf (0) n 2! ω 2 n + o( 1 n ) ] n = [ 1 ω2 2n + o( 1 n ) ] n. Finally, pass n and use esult 2 with a = ω 2 /2. Done! The Central Limit Theorem 9 / 9