1 Probability Generating Function

Transcription

1 Chater 5 Characteristic Fuctio ad Probablity Covergece Theorem Probability Geeratig Fuctio Defiitio. For a discrete radom variable X which ca oly achieve o-egative itegers, we defie the robability geeratig fuctio of X as g(s = E(s X = s j P (X = j, s [, ]. ( j=0 Theorem. Assume g(s is the geeratig fuctio of X, the (. P (X = k = k! g(k (0, k = 0,,...; (2. EX = g (; (3. If EX <, the var(x = g ( + g ( (g ( 2 ; (4. If X, X 2,..., X are ideedet, g i (s = Es X i is the geeratig fuctio of X i, the the geeratig fuctio of Y = X + X X is Proof: ( is rove by g Y (s = g (sg 2 (s...g (s, s [, ]. k! g(k (0 = k! g(k (s s=0 = k! ds k (sj P (X = j s=0 = P (X = j. (2 is rove by g ( = j=0 jp (X = j = EX. (3 is rove by g ( + g ( (g ( 2 = (4 is rove by j=0 d k j(j P (X = j + jp (X = j (EX 2 j=0 = E[X(X ] + EX (EX 2 = EX 2 (EX 2 = var(x. g Y (s = Es X +X X = Es X Es X 2...Es X = g (sg 2 (s...g (s. as radom variables s X,..., s X 2 are ideedet. The most imortat roerty is ( which tells us the geeratig fuctio ca determie robability distributio ad the iverse is also true.

2 2 Characteristic Fuctio I robability theory ad statistics, the characteristic fuctio of ay real-valued radom variable comletely defies its robability distributio. If a radom variable admits a robability desity fuctio, the the characteristic fuctio is the iverse Fourier trasform of the robability desity fuctio. Thus it rovides the basis of a alterative route to aalytical results comared with workig directly with robability desity fuctios or cumulative distributio fuctios. There are articularly simle results for the characteristic fuctios of distributios defied by the weighted sums of radom variables. Defiitio 2. If ξ, η are radom variables, i =, the we call Z = ξ + iη comlex radom variable. If Eξ, Eη exist, we defie that EZ = Eξ + ieη. I additio, we defie the characteristic fuctio of X as ϕ(t def = Ee itx = E cos(tx + ie si(tx Theorem 2. (Iverse Fuctio Assume ϕ(t is the character fuctio of X, F (x is the distributio fuctio of X. If F (x is cotiuous o (a, b, the We leave the roof i future. F (b F (a = 2π lim T T T e ita e itb ϕ(tdt. (2 it The character fuctio is useful for aalyzig the robability distributio, exectatio, variace of a radom variable because the maig betwee the characteristic ad distributio fuctio is a oe-to-oe maig. Theorem 2.2 Assume ϕ(t = Ee itx, the ( ϕ(0 =, ϕ(t, ϕ( t = ϕ(t. (2 ϕ(t is uiformly cotiuous o (,. (3 If EX k exists, the ϕ (k (t = i k E(X k e itx, ϕ (k (0 = i k E(X k ; (3 (4 No-egative: for ay comlex umber a, a 2,..., a, ϕ(t k t j a k ā j 0; k= (5 If ϕ i (t is the characteristic fuctio of X i, X, X 2,..., X are ideedet, the the character fuctio of Y = X + X X is ϕ Y (t = ϕ (tϕ 2 (t...ϕ (t. 2

3 Proof: We oly rove the case whe X has desity fuctio f(x. ( Eϕ(0 = E =. Ad, ϕ(t = E cos(tx + ie si(tx =, as E(X EX 2 = EX 2 (EX 2 0. = [(E cos(tx 2 + (E si(tx 2 ] /2 [(E cos(tx 2 + (E si(tx 2 ] /2 ϕ( t = cos(tx + ie si( tx = cos(tx + ie si( tx = ϕ(t. (2 For ay ϵ > 0, there always exists a M > 0 such that 2 f(xdx < ϵ 2 x >M as f(xdx =. Ad, for this M, there always exists a δ such that if x < δ, the e ixm < ϵ 2. Therefore, we have for ay s t δ, as e isx. ϕ(t ϕ(s = E(e itx e isx = (e itx e isx f(xdx x M + e itx e isx f(xdx e i(t sx f(xdx x >M ϵ 2 + ϵ 2 = ϵ (3 Sice E X k = xk f(xdx <, therefore, e i(t sx f(xdx ϕ (k (t = dk dt k e itx d k f(xdx = dt k eitx f(xdx = i k x k e itx f(xdx = i k E(X k e itx 3

4 (4 For ay comlex vector a = (a, a 2,..., a, we have ϕ(t k t j a k ā j = E e i(t k t j X a k ā j k= = E a k e it kx k= as if a j = b + ci, the ā j = b + c( i. k= ā j e ( it jx = E a k e it kx (5 We rove it by rovig that if X, X 2 are ideedet ad have desity fuctios f (x, f 2 (x 2, the ( E e it(x +X 2 = E(e itx E(e itx 2. We give two theorems about the characteristic fuctio. Theorem 2.3 (Bocher-Khichi Theorem Let ϕ(t be cotiuous, t R with ϕ(0 =. A sufficiet ad ecessary coditio that ϕ(t is a characteristic fuctio is that it is ositive semi-defiite, i.e. that for all real t,..., t ad all comlex λ,..., λ, =, 2,... ϕ(t i t j λ i λj 0. i, Theorem 2.4 (Polya s Theorem Let a cotiuous eve fuctio ϕ(t satisfy ϕ(t 0, ϕ(0 =, ϕ(t 0 as t ad let ϕ(t be covex o 0 t <. The ϕ(t is a characteristic fuctio. k= Now, we give the characteristic fuctio of ormal distributio. Examle 2. Assume X N(µ, σ 2, calculate the characteristic fuctio of X. Aswer: We start our calculatio from the simlest case. First, we assume X N(0, ad cosider i as a fixed umber, the ϕ(t = Ee itx = 2π = e t2 /2 2π e itx e x2 /2 dx e (x it2 /2 dx = e t2 /2. However, we kow that i is actually a secial umber such that we ca t cosider it as a ormal fixed value, like or 2. We kow that ϕ(t = 2π e itx e x2 /2 dx = 2π 4 2 cos(txe x2 /2 dx

5 as si(tx = si(tx. Ad, Therefore, ϕ (t = 2π = 2π x si(txe x2 /2 dx si(txde x2 /2 = si(txe x2 /2 2π = 2π = tϕ(t. t cos(txe x2 /2 dx t cos(txe x2 /2 dx tϕ(t + ϕ (t = 0 = (tϕ(t + ϕ (te t2 /2 = 0 = d dt [ϕ(t ex(t2 /2] = 0. So, ϕ(t ex(t 2 /2 = C ad C = ϕ(0 = such that ϕ(t = ex( t 2 /2. If Y N(µ, σ 2, the Y = µ + σx. We have Ee ity = Ee it(µ+σx = e itµ Ee i(tσx = ex(iµt σ 2 t 2 /2. Now, we give a imortat results about ormal distributio. Lemma 2. If X, X 2,..., X are ideedet, ad X j N(µ j, σj 2, the Y N µ j,. Proof: Sice σ 2 j Ee ity = Π ex(iµ j t σj 2 t 2 /2 = ex it µ j t2 2 therefore, we have the above coclusio. Now, we give the character fuctio of a radom vector. Assume X = (X, X 2,..., X is a radom vector, we defie the character fuctio of X as The, we have the followig theorem: σ j, ϕ(t = E ex(itx, t = (t, t 2,..., t R. Theorem 2.5 Assume ϕ(t is the characteristic fuctio of X, ϕ j (t is the characteristic fuctio of X j, the ( The maig betwee ϕ(t ad the distributio fuctio of X is oe-to-oe. (2 The sufficiet ad ecessary coditio for X, X 2,..., X are ideedet is ϕ(t = ϕ (t ϕ 2 (t 2...ϕ (t. 5

6 3 Zero-or-Oe Laws 3. Borel-Catelli Lemma For a series of evets {A j }, C = j= A j is a decreasig series, D = j= A j is a icreasig series. We defie lim A = C j = = = j= A j, ad lim A = D j = = = j= as the uer ad lower limitatios of {A j } resectively. ( We ca fid that if ω = j= A j, the ω C for all, so there are ifiity ofte A j cotais ω. (2 Sice if ω = j= A j, which meas there must exists a k such that if = k, ω j==k A j such that for j k, ω A j, therefore ω j=l A j for all l =, 2,... such that ω = j= A j. Therefore, we have lim A = = j= A j lim A = = j= A j A j (3 P (lim A = lim P j= A j, P (lim A = lim P j= A j. Lemma 3. (Borel-Catelli Lemma For a series of evets {A j }, ( If P (A j <, the P (A i.o. = 0; (2 If {A j } are ideedet, P (A j =, the P (A i.o. =, where {A i.o} = lim A = = j= A j. Proof: ( By we have P (lim A = lim P j= A j, P (A i.o lim P (A j = 0 j= 6

7 (2 Sice x e x, therefore m P = Π m j=p (Āj = Π m j=( P (A j Therefore, P j= A j j= Ā j = lim m P = lim m Π m j= ex( P (A j = ex( 0 as m. m j= A j m P j= A j 3.2 Covergece of Radom Variables m m P (A j j= m = lim P j= Ā j =.. We kow that the series = / diverges ad the series = ( (/ coverges. We ask the followig questio. What ca we say that the covergece or divergeces of a series = (ξ /, where ξ, ξ 2,... is a sequece of ideedet idetically distributed Beroulli radom variables with P (ξ = = P (ξ = = /2? I other words, what ca be said about the covergece of a series whose geeral term is / or /, where the sigs are chose i a radom maer, accordig to the sequece ξ, ξ 2,...? Let A = { ω : } ξ coverges = be the set of samle oits for which = ξ / coverges (to a fiite umber ad cosider the robability P (A of this set. It s a remarkable fact that we are able to say that the robability ca have oly two values, 0 or. 2. Let (Ω, F, P be a robability sace, ad let ξ, ξ 2,... be a sequece of radom variables. Let F = σ(ξ +, ξ +.2,... be the σ field geerated by ξ +, ξ +2,... ad write A = F. = Sice a itersectio of σ-algebra is agai a σ-algebra, A is a σ algebra. It is called a tail algebra (or termial or asymtotic algebra, because every evet A A is ideedet of the values of ξ,..., ξ for every fiite umber, ad it is determied oly by the behavior of the ifiity remote values of ξ, ξ 2,... 7

8 For examle, if ξ, ξ 2,... is ay sequece, the { } { } ξ A = ω : coverges ξ = ω : coverges Fk, such that A k F k = =k = A. Similarly, the followig evets are also tail evets: { } A 2 = coverges, = ξ A 3 = {ξ I for ifiitely may }, A 4 = {S / coverges}, A 5 = { lim ξ < }, { } ξ ξ A 6 = lim <, { } ξ ξ A 7 = lim < c, with I σ(r. O the other had, B = {ξ = 0 for all }, { } B 2 = lim(ξ ξ exists ad is less tha c are examles of evets that do ot belog to A. Theorem 3. (Kolmogorov s Zero-or-Oe Law Let ξ, ξ 2,... be a sequece of ideedet radom variables ad let A A. The P (A ca oly have oe of the values zero or oe. We will leave the roof i future. 4 Four Kids of Covergeces of Radom Variables 4. Defiitios of Covergeces Assume x, x 2,..., x are samles of radom variable X. I the study of statistics, we are always iterested i the roerties of f (x, x 2,..., x, esecially whe samle size icreases. Now, we discuss the roerty of radom variables CONVERGENCE. I the followig, we assume X, X 2,..., X are radom variables. 8

9 Defiitio 4. Assume the distributio fuctios of X ad X are F (x ad F (x. ( If lim F (x = F (x (4 whe F (x is cotiuous at oit x, the we say that X coverges to X i distributio ad write as X d X. (2 If for ay ϵ > 0, lim P ( X X ϵ = 0, (5 the we say that X coverges to X i robability ad write as X X. (3 If ( P lim X = X =, (6 the we say that X coverges to X almost surely ad write as X X a.s.. (4 If E X X α 0 α > 0, (7 the we say that X coverges to X i L α ad write as X L α X. 4.. Relatig almost sure covergece ad covergece i robability a.s. The followig lemma relates X X to su k X X 0. Lemma 4. X a.s. X iff ϵ > 0, lim P ( su X X > ϵ k = 0, which meas X Y = su X X 0. X a.s. k Theorem 4. (Cauchy Criterio For ay ϵ > 0, ( a.s. X X lim P su X +m X ϵ m = (8 Proositio 4. If for ϵ > 0, = P ( X X > ϵ < ({X, X 2,...} is said to coverge comletely to X, the X X a.s.. 9

10 Now, we discuss X µ a.s.. Actually, X µ a.s. meas that, ({ } ϵ, η > 0, 0 N : P su X k µ > ϵ k 0 < η, which is equivalet to for ay ϵ > 0, lim P k { X k µ > ϵ} = 0. We firstly give a lemma which ca be used to determie whether X µ a.s.. Lemma 4.2 If for all ϵ > 0, = P ( X µ > ϵ <, the X µ a.s.. Proof: By Borel-Catelli Lemma, we have ( P X µ > < P { X µ > k } k i.o. = 0 = for ay k N. Sice {X µ} k= { X µ > k }, P (X µ = 0 P (X µ = 0. Proositio 4.2 Let {X, X 2,...} be a sequece of ideedet radom variables, the X a.s. µ 4.2 Law of Large Numbers P ( X µ > ϵ <, ϵ > 0. = We will give the most imortat theory i the study of robability i the followig which is about covergece of a series of radom variables. Theorem 4.2 (Weak Law of Large Numbers Assume radom variables {X } are ucorrelated, cov(x i, X j = 0 (i j. If with C be a fixed umber, the µ j = EX j, var(x j C, j =, 2,..., (X j µ j 0. Secially, if {X j } has the same mea µ j = µ, the X j µ. 0

11 Proof: Deote S = X + X X. The for ay ϵ > 0, P (X j µ j > ϵ = P ( S ES > ϵ 2 ϵ 2 var(s = 2 ϵ 2 var(x j C 0,. ϵ2 Secially, whe µ j = µ, we have X j µ. Theorem 4.3 (Strog Law of Large NumbersAssume {X j } is a series of ideedet ad idetical distributed radom variables, µ = EX, the X j µ, a.s.. (9 Proof: We oly rove the case whe E(X µ 2 <. Without lose of geerality, we assume µ = 0 as X j µ, a.s. is equivalet to (X j µ 0, a.s.. Deote S = X + X X. Sice E(X 3 i X j = EX 3 i EX j = 0, E(X 2 i X k X l = 0, E(X i X j X k X l = 0. If i j k l, therefore ES 4 = E Xj X j X k j<k 2 = E + 4E(X j X k 2 X 2 j = [EX 4 + ( (EX 2 2 ] + 4E Xj 2 X 2 k j<k = EX 4 + 3( (EX C 0,

12 where C 0 is a fixed umber. Therefore, for ϵ = /8, we have ( P S ϵ = P ( S ϵ such that = = = = 4 ϵ 4 ES 4 2 C 0 <. 4 /2 [ ] I S ϵ <, a.s., = which meas for almost ay ω Ω, there exists a M such that if > M, S ϵ. The secod method of the roof. Deote S = k= X k, the ( S P µ > ϵ = such that we ca ot use Lemma 4.2 ad Chebyshev i- Because equality = var(x ϵ 2 E( S µ 2 ϵ 2 = var(x ϵ 2. P = ( S µ > ϵ = var(x ϵ 2 to coclude SSLN directly. Therefore, we rove SSLN by the followig stes. Case. X 0. Sice var(x = <, it follows from Lemma 4.2, we have 2 ϵ 2 S m m µ a.s. as m = 2. Let 2 m ( + 2, the as X 0, Hece µ 2 S 2 m 2 S m m S m m µ ( + 2 m a.s.. S (+ 2 ( + 2 µ Case 2. Geeral X, we write X k = X + k X k, the S = k= X+ k k= X k EX + EX = EX. 2

13 4.2. Differet sufficiet coditio for SSLN Theorem 4.4 (Strog Law of Large NumbersAssume {X j } is a series of ideedet ad idetical distributed radom variables, ad E X <, µ = EX, the X j µ, a.s.. (0 Lemma 4.3 (Kroecker s Lemma Let x, x 2,... be real umbers, let S = x + x x. If k= coverges, the x k k S 0 as. Proof: Let b r = x k k=r k, the b r 0 as r which meas for ay ϵ > 0, there exists a R such that if r > R the b r < ϵ. This imlies that b r = M b r + r= r= r=m+ b r M r= b r + M ϵ 0 as ad ϵ 0. which is equivalet to { x ( + x2 2 + x ( 2 x x ( x x } such that S + r=+ x r r 0 ad S 0. Theorem 4.5 (Kolmogorov Theorem Assume X are ideedet radom variables, ad have the same exectatio µ = EX j. If var(x j j 2 <, the is true. Proof: Sice var X j j X j µ, a.s.. = var(x j j 2 <, X j (ω j therefore for almost ay ω Ω we have Therefore (0 is true. We ca also rove this theorem by the similar stes i the roof of Theorem coverges, such that S(ω 0.

14 Theorem 4.6 Assume X, X 2,... are air-wise ucorrelated radom variables, S = i= X i, the exectatio of X exist for, the ( If i= var(x / 2 <, the as. (2 If i= var(x / 3/2 <, the as. (S ES / 0 (S ES / a.s Alicatio of SLLN Examle 4. Assume {X j } is a series of radom variables i robability sace (Ω, F, P. We use x j to deote the observatio of {X j }, which meas give a ω Ω, we have x j = X j (ω, j =, 2,..., the the samle series {x j } ca determie the distributio fuctio F (x of X j with robability. Aswer: For ay fixed umber x (,, we ca fid that the radom variables g(x j = I {Xj x} for j =, 2,..., are ideedet ad idetically distributed. Therefore, lim g(x j = Eg(X j = P (X x = F (x a.s.. Therefore, we have with robability. lim g(x j = Eg(X j = F (x 4.3 Relatioshi betwee Four Kids of Covergece It s iterestig to discuss the relatioshi betwee these four kids of covergece. We have the followig theorem: Theorem 4.7 ( If X a.s. X, the X X. (2 If X X, the X d X. 4

15 (3 If E X X 0, the X X. (4 If X d c with c be a costat, the X c. Proof: ( For ay ϵ > 0, evet = k= { X k X ϵ} a.s. haes meas there are ifiite evets { X k X ϵ} hae, therefore from X X, we have ( P { X k X ϵ} = 0 such that lim P ( X X ϵ lim P = k= ( ( { X X ϵ} = P k= as k= { X X ϵ} is a decreasig series. = k= { X X ϵ} = 0 (2 For ay cotiuous oit x of F (x, we deote x 0 = x δ ad x = x 0 + δ with δ = δ(x > 0. The by ad F (x F (x = P (X x F (x = P (X x, X > x + P (X x, X x F (x P ( X X > δ + F (x F (x P ( X X > δ + F (x F (x 0 F (x F (x = [ P (X > x] [ P (X > x] = P (X > x P (X > x = P (X > x, X x 0 + P (X > x, X > x 0 P (X > x P ( X X > δ + P (X > x 0 P (X > x = P ( X X > δ + F (x F (x 0 P ( X X > δ + F (x F (x 0 therefore such that F (x F (x P ( X X > δ + F (x F (x 0 lim F (x F (x = 0 5

16 as δ 0. (3 We rove (3 by as. (4 For ay ϵ > 0, P ( X X ϵ E X X ϵ 0 as sice X d c Couter Examles: Examles: (X If P ( X c ϵ = P (X c + ϵ + P (X c ϵ X, but X X a.s.. The for all ϵ > 0, = P (X < c + ϵ + P (X c ϵ = P (c + ϵ X < c + ϵ 0 X (ω =, X 2 (ω = I [0,/2] (ω, X 3 (ω = I [/2,] (ω, X 4 (ω = I [0,/4] (ω,..., X 7 (ω = I [3/4,] (ω X 8 (ω = I [0,/8] (ω,..., X 7 (ω = I [7/8,] (ω P ( X 0 > ϵ 0 as. However, X (ω 0 for all ω (0,. (2X d X, but X X. If X N(0, for all ad are ideedet, the X d X. However, we have X X N(0, 2 such that Therefore, X X. P ( X X > ϵ = > 0. (3 X X, but E X X 0. If P (X = = /, P (X = 0 =, the P ( X 0 > ϵ 0 as. However, E X 0 = for all. (4 X X a.s., but E X X 0. If P (X = 2 = / 2, P (X = 0 = 2, the = P ( X 0 > ϵ < which 6

17 meas X X a.s.. However, E X 0 = for all. (5 E X X 0 but X X a.s.. If P (X = = /, P (X = 0 =, the E X X 0. However, = P ( X 0 > ϵ = which meas X X a.s.. (6 E(X X 2 0 E X X 0. However, the iverse is ot true. By EXY EX 2 EY 2, we have E X X E(X X 2 0. However, if P (X = =, P (X = 0 =, the E X 0 0, but E X 0 2 =. 5 Cetral Limit Theorem Before talkig about cetral limit theorem (CLT, we give more discussios about covergece i distributio i the followig subsectio. 5. More discussios o covergece i distributio Theorem 5. The followig are equivalet defiitios of X coverges to X i distributio or say covergig weakly to X. (a lim F (x = F (x for all cotiuous oits x of f(x. (b E(h(X E(h(X for all bouded, cotiuous fuctio of h from R R. (c P (X A = P (X A wheever P (X A = 0, where A is the boudary of Borel set A, for examle A = (, b, the A = {b}, if A = [a, b], the A = {a, b}. Theorem 5.2 (Delta-Method Let X, X 2,... be I.I.D radom variables with mea µ ad variace σ 2. Let S = X + X X. Let g be a fuctio which is cotiuously differetiable at µ (g (µ is cotiuous at µ, the [ g ( S g(µ ] d N ( 0, [g (µ] 2 σ 2 ( 7

18 Proof: d Let Y = S µ, Y = d N(0, such that Y Y a.s.. The [ ( ] Y g + µ g(µ = [ g(µ + Y ( ] g Y (µ + o g(µ = Y g (ξ Y g (µ (a.s. ξ ( µ, µ + Y as. Therefore, [ ( ] Y d g g(µ Y g (µ. Theorem 5.3 (Cotiuity Let the distributio fuctio(d.f. F has characteristic fuctio(ch.f. ϕ. ( Let D.F. F has Ch.F. ϕ. The F d F iff ϕ (t ϕ(t for all t. (2 More geerally, if we kow that ϕ (t g(t for all t ad for some fuctio g cotiuous at 0, the g is the Ch.F. of some D.F. F ad F d F. 5.2 Comlex Number We say a + ib a + ib iff a a ad b b. We have the followig roerties for comlex umbers z, z 2,... Proerty : z z 2 = z z 2 ; Proerty 2: z z iff z z 0; Proerty 3: z + z 2 z + z 2. Defiitio 5. Let W = U + iv, the EW = EU + iev. Theorem 5.4 EW 2 E( W 2 EW E( W. We rove the above theorem by EW 2 = (EU 2 + (EV 2 EU 2 + EV 2 = E W 2, Jese s iequality ad f(w = W is a covex fuctio for a comlex umber W as λw + ( λw 2 λ w + ( λ w 2. 8

19 5.3 Cetral Limit Theorem We kow that by calculatig the characteristic fuctio of S = X + X X, we ca obtai the distributio of S. We kow that if X,..., X are I.I.D., the S B(, if X B(, S (λ if X (λ S Γ(, λ if X ξ(λ S N(µ, σ 2 if X N(µ, σ 2. We kow the distributio of S whe is a fiite umber, however what s the distributio of S as goes to ifiity is ukow. We have the followig theorem. Theorem 5.5 Assume the series of radom variables {X j } are I.I.D, ad have the commo exectatio µ ad σ 2, the by deotig S = X + X X we have ξ = S µ σ 2 = X + X X µ σ 2 d N(0, (2 Proof: We kow the maig betwee the characteristic ad distribute fuctio is oe to oe by Theorem 5.3, therefore we rove (2 by calculatig the limitatio of characteristic fuctio of ξ. We begi from the case that µ = 0, σ 2 = ad rove that X +X X use ϕ(t = Ee itx such that to deote the characteristic fuctio of X. The we have ϕ (0 = iex = 0, ϕ (o = i 2 EX 2 =, ϕ(t = ϕ(0 + ϕ (0t + 2 ϕ (0t 2 + o(t 2 = t2 2 + o(t2. Therefore, the characteristic fuctio of ξ is ( ϕ (t = E ex i t (X X [ ( = E ex i t ] X ( ] = [ t2 t o e t2 2,. d N(0,. We which is the characteristic fuctio of N(0,. For a geeral µ ad σ 2, we kow that the exectatio ad variace of Y j = (X j µ/σ N(0,. Therefore, we have ξ = S µ = X j µ = d Y j N(0, σ 2 σ The CLT is quite widely used i Statistics for testig. Please see the examles i otebook. 9

20 5.4 Lideberg-Feller Theorem I Theorem 5.5, we eed {X j }s are I.I.D. However, this coditio is actually strict i some cases. Therefore, we release this restrictio i the followig theorem. Assume {X j }s are ideedet, the distributio fuctio, exectatio ad variace of X j are F j (x = P (X j x, µ j = EX j, σ 2 j = var(x j. We defie B 2 = var(x + X X = σj 2. Theorem 5.6 Assume {X j }s are ideedet radom variables, ad {σ 2 j } satisfies lim B =, lim σ2 /B 2 0, the the sufficiet ad ecessary coditio for the CLT lim P (X j µ j x = Φ(x (3 B with ϕ(x be the distributio fuctio of stadard ormal, is the Lideberg-Feller Coditio is true, which is for ay ϵ > 0, lim B 2 E(X j µ j 2 I[ X j µ j ϵb ] = 0. (4 We rove it by rovig the characteristic fuctio of ξ coverges to the characteristic fuctio of the stadard ormal. We give two lemmas before givig the roof of this theorem. Lemma 5. If z i ad w i, the Proof: (e.g. m=3 z...z m w...w m m z i w i. i= z z 2 z 3 w w 2 w 3 = (z z 2 z 3 z z 2 w 3 + (z z 2 w 3 z w 2 w 3 + (z w 2 w 3 w w 2 w 3 Lemma 5.2 eit ( + it t2 2 t3 6, eit ( + it t2 2 t 2. 20

21 Proof: Sice e iu 2, t (e iu t du e iu du 0 0 e it t i 2t ( e it t i i 2t e iu ( + iu 2u t { e iu ( + iu } t du 2udu 0 0 e it (t + it2 t 2 i 2 ( eit ti t2 t 2 2 We have show (2. To show (, start with e iu ad roceed i a similar way. Now, we rove the Lideberg-Feller Theorem. Proof: Without lose of geerality, we assume µ j = EX j = 0. By ideedecy, Ee its /B = Π i=ee itx k/b [ ( = Π i= t2 X 2 ] 2 E k B 2 + r k The secod equality is because [ Ee itx k/b = E + itx k + (itx k 2 ( (itxk 2 ] B 2 B 2 + o B 2. By Lemma 5.2, we have ( t 3 Xk 3 r k E B 3 ϵt 3 E(X2 k B 2 I { X k ϵ B + t2 B 2 E } + t2 X 2 k ( B 2 I { } X k >ϵ B Xk 2 I{ } X k >ϵ B. 2

22 The, by Lemma 5., [ ( EeitS /B e t 2 2 = Π k= t2 X 2 ] 2 E k B 2 + r k Π k= e t2 2 E(X2 k /B2 ( t2 X 2 2 E k + r k e t2 2 E(X2 k /B2 k= B 2 ( t2 X 2 2 E k k= = A + B. By Lideberg-Feller coditio, we have B 2 e t2 2 E(X2 k /B2 + r k k= max k EX 2 k B 2 0. Let t2 2 E(X2 k /B2 = z, the z should be small tha /2 if is big eough ad t is ot too large. By e z z k + z k! z2 2 ( + z + z z 2 as + z + z = z k=2 2, we have A t4 4 [E(Xk 2 /B2 ] 2 k= t4 4 max k EX 2 k B 2 0 = 0 E(Xk 2 /B2 k= Also, sice max k EX 2 k B 2 0 Therefore, we have B ϵt 3 + o( 0 as ϵ 0. Ee its /B e t2 2 as. such that we have the CLT by the cotiuity Theorem Differet Versio of Lideberg-Coditio We ca fid that uder the Lideberg-Coditio, we kow that B max j { X j µ j } 0, 22

23 which ca be rove by followig stes: for ay ϵ > 0, ( P max B { X j µ j } ϵ j = P { X j µ j ϵb } = j P ({ X j µ j ϵb } EI[ X j µ j ϵb ] ϵ 2 B 2 0. ( (Xj µ j 2 E ϵ 2 B 2 I[ X j µ j ϵb ] E ( (X j µ j 2 I[{ X j µ j } ϵb ] Proositio 5. Assume radom variables {X j }s are ideedet. If there exists a series of real umbers {C } such that max j X j µ j C, a.s. for all, ad lim C /B = 0, the the CLT (3 is true. Proof: We oly eed to rove the Lideberg-Feller Coditio is true. We fid that for ay ϵ > 0, there exist a N such that if > N, the C B, we have for ay j, we have I[ X j µ j ϵb ] = 0 a.s., therefore the Lideberg-Feller Coditio is true. Proositio 5.2 Assume radom variables {X j }s are ideedet. If there exists a real umber δ > 0 such that lim B 2+δ E X j µ j 2+δ = 0, (5 the the CLT (3 is true. Proof: We rove it by rovig that the Lideberg-Feller coditio is true. For ay ϵ > 0, we have 23

24 lim B 2 lim E ( (X j µ j 2 I[ X j µ j ϵb ] B 2 lim ϵ δ B 2+δ 0. ( Xj µ j 2+δ E (ϵb δ I[ X j µ j ϵb ] E ( X j µ j 2+δ 24