6 Infinite random sequences

Transcription

1 Tel Aviv Uiversity, 2006 Probability theory 55 6 Ifiite radom sequeces 6a Itroductory remarks; almost certaity There are two mai reasos for eterig cotiuous probability: ifiitely high resolutio; edless coi tossig. Of course, both are theoretical idealizatios. 64 Ifiite resolutio was discussed i Sect. 1c. Edless coi tossig was discussed i 1f4 ad 2b5 2b7. Except for these digressios, Sectios 1 5 are directed towards ifiitely high resolutio rather tha edless coi tossig. 65 Now we tur to the latter ad its geeralizatios. Almost certaity was itroduced i Sect. 1c; recall the termiology: A is egligible, A occurs almost ever, whe PA = 0 ; almost surely, A does ot occur A is almost certai, A occurs almost always, whe PA = 1. almost surely, A occurs Discrete probability gives us oly trivial examples of almost certai evets. Cotiuous probability gives better examples: let X have a cotiuous distributio, ad x be a umber, 66 the X x almost surely. Much deeper examples arise from ifiite sequeces of evets or radom variables, as we ll see soo. Let a coi be tossed edlessly, givig idepedet idetically distributed radom variables X 1, X 2,... each takig o two equiprobable values, say, +1 ad 1 or 0 ad 1, if you like. What about lim X? 67 Probably you believe that the limit does ot exist. Why? Sice there is a subsequece of +1, ad aother sequece of 1. However, why they exist? What if X cease to chage after some? It seems ureasoable, but we eed a proof. Cosider a evet A = { m > X m = +1 } ; we wat to prove that PA = 0. Itroduce evets A 1 = { X 1 = +1, X 2 = +1, X 3 = +1,... }, A 2 = { X 2 = +1, X 3 = +1,... }, A 3 = { X 3 = +1,... }, 64 We ofte prefer to idealize a ukow or irrelevat high resolutio. Say, we prefer d dx six = cosx to six si x = cosx si x. Similarly, we ofte prefer to move to ifiity a ukow or irrelevat legth of a log fiite sequece. 65 However, a umber of geeral theorems are applicable to both cases. 66 No-radom, of course. 67 It is ot the limit of frequecy, just the limit of X itself. 68 If you believe that its probability teds to 0, read Sect. 1c oce agai! 69 A evet is a subset of our probability space Ω; strictly speakig, we should write A = { ω Ω : m > X m ω = +1 }, but probabilists usually omit ω.

2 Tel Aviv Uiversity, 2006 Probability theory 56 ad so o; A = { m X m = +1 }. We have PA 1 = 0 by the followig argumet: PA 1 P X 1 = +1,..., X = +1 1 = = for every = 1, 2,..., therefore PA 1 = 0. The same argumet 70 shows that PA 2 = 0, ad similarly PA = 0 for all. We have a icreasig sequece of evets, A 1 A 2... thik, why, ad A is their uio. We may say that A = lim A, accordig to the defiitio give after 2d9: 71 6a1 lim A = { A 1 A 2... if A 1 A 2..., A 1 A 2... if A 1 A 2... Recall that probability depeds cotiuously o a evet, i the followig sese: 6a2 P lim A = lim PA for every mootoe sequece of evets see 2d9. So, PA = P lim A = lim PA = lim 0 = 0. Almost surely, there is o limit lim X for the coi tossig sequece X. We may treat X as biary digits 72 of a radom poit ω of [0, 1] recall 2b5, ω = 0.X 1 X That is, we may take Ω = [0, 1] with Lebesgue measure as our probability space. Evets A become subsets of [0, 1]: A 1 = {1}, A 2 = { 1 2, 1}, A 3 = { 1 4, 1 2, 3 4, 1},... thik, why. Their limit is the set of all biary-ratioal poits of [0, 1], We have A = lim A = { k 2 : = 0, 1, 2,..., k = 0, 1,..., 2 }. PA = 0 ; P [0, 1] \ A = 1. Both A ad [0, 1] \ A are dese i [0, 1], but A is egligible, while [0, 1] \ A is ot Quite iformally we could write PA 1 = 1/2 = 0, PA 2 = 1/2 1 = 0, ad so o. 71 It is a prelimiary defiitio, applicable oly for mootoe sequeces. A geeral defiitio will be give later after 6b5. 72 Of course, ow X takes o two values 0 ad 1 rather tha ±1. 73 Well, A is egligible sice it is coutable ad Lebesgue measure is oatomic. Further we ll meet ucoutable egligible sets, too.

3 Tel Aviv Uiversity, 2006 Probability theory 57 Is there a empirical test for the statemet that ω / A almost surely? No. Ay physical radom choice of ω, beig of a fiite resolutio, does ot allow to decide, whether ω A or ot. Similarly, ay physical coi tossig process, beig of fiite legth, is ot eough for determiig lim X. I this sese, covergece of radom sequeces is a formal mathematical theory with o empirical basis. The, why do we lear the elegat but groudless 74 theory? For a simple reaso: it helps us to uderstad log fiite sequeces. 6a3 Exercise. Let X 1, X 2, : Ω R be idepedet idetically distributed radom variables. Ca it happe that X +? Hit: cosider the media Me = X 1/2; we have P X > Me 1/2. It follows that the evet A = { m > X m > Me} is of probability 0. 6a4 Exercise. Let X 1, X 2, : Ω R be idepedet idetically distributed radom variables havig expoetial distributio PX x = 1 e x for x > 0. What is the probability that X > 2 for all? Hit. 75 P X1 > 2 1, X2 > 2 2,... = P X1 > 2 1 P X2 > = = exp 1 2 exp = exp = e a5 Exercise. For the same X as before, what is the probability that X > 1 Hit. exp = e = for all? You see, the evet X 2 has a o-degeerate probability ; i e cotrast, the evet X 1 occurs almost surely. I order to distiguish betwee the two cases, we eed distiguish betwee coverget ad diverget series. Recall some relevat argumets: }{{ 3 2 } log... }{{} covergece divergece < ; 1 < ; = ; 1 log 2 < ; 1 log = ; a < 2 a 2 < for a 0 ; fa < a < wheever f0 = 0, f 0 > 0, ad a Though groudless empirically, it is still well-fouded mathematically. It is based o measure theory. Thus, it caot lead to a cotradictio provided, of course, that measure theory is cosistet. 75 expa is the same as e a.

4 Tel Aviv Uiversity, 2006 Probability theory 58 The case fa = l1 a = l 1 1 a is especially importat: for ay a [0, 1 1 a > 0 l1 a > 6a6 log 1 1 a < a <. 6b Borel-Catelli lemma Sequeces that do ot coverge are quite usual i probability theory. Havig o limit, such a sequece has its upper limit lim sup ad lower limit lim if. Give a 1, a 2, R, we defie 6b1 lim if lim sup a = a = sup a = a = if sup m if a m = lim ifa, a +1,...; m a m = lim supa, a +1,.... sup a limsup a limif a if a I geeral, 6b2 if a lim if a lim sup a sup a +. If lim if a = lim sup a the lim a exists ad is equal to both. Otherwise lima does ot exist. Similarly, give evets A 1, A 2, Ω, we defie 6b3 I other words, 76 6b4 lim if A = A = lim sup A = A = =1 m= =1 m= A m = lim m= A m = lim m= A m ; A m. ω lim if A m ω A m #{m : ω / A m } < ω A evetually ; ω lim sup A m ω A m #{m : ω A m } = ω A ifiitely ofte. 76 Here #{m :...} stads for the umber of such m.

5 Tel Aviv Uiversity, 2006 Probability theory 59 I geeral, 6b5 =1 A lim if A lim sup A A Ω. Now we are i positio to geeralize 6a1 for o-mootoe sequeces of evets. By defiitio, if lim if A = lim sup A the lim A exists ad is equal to both. Otherwise lima does ot exist. If lim A exists the Plim A = lim PA by the sadwich argumet compare it with 6a2. A geometric example. Cosider geometric figures of the followig form: =1 Ar, R, ϕ = r ϕ R Let ϕ = 1 which meas 1 radias 77, the vertices of A = Ar, R, ϕ are a o-periodic sequece dese i the R-circle: lim sup A poit of the small disk belogs to A for all. A poit of the aulus betwee the two circles belogs to A ifiitely ofte, but ot evetually recall 6b4. A poit outside of the large disk belogs to o oe of A. Thus, 78 A = lim if A = the r-disk, ad lim sup A = A = the R-disk. There is o lim A. If you wat all the six sets i 6b5 to differ, try A = Ar, R, ϕ with r r, R R, r < R. 6b6 Exercise. Cosider coi tossig X 1, X 2, : Ω {0, 1} ad let A = {X = 1} = {X 0}. Show that { } { } A = 1 X = 0 ; A = X > 0 ; =1 lim if A = lim sup A = =1 =1 lim if =1 { } 1 X < = { X 1 } ; =1 { =1 } X = = { X 0 }. Does lim A exist? What about probability of the differece lim sup A \ lim if A? 77 Recall that the whole circle cotais 2π 6.28 radias. 78 There are some uaces cocerig boudary poits; I just igore them.

6 Tel Aviv Uiversity, 2006 Probability theory 60 6b7 Probably you kow elemetary relatios for two idicators, 79 A = B C 1 A = mi 1 B,1 C, A = B C 1 A = max 1 B,1 C. Now we have similar relatios for ifiite sequeces of idicators: A = A 1 A = if 1 A ; 6b8 A = =1 =1 A 1 A = sup1 A ; A = lim if 1 A = lim if A ; A = lim sup A 1 A = lim sup1 A. The followig result, traditioally called the first Borel-Catelli lemma or the first part of Borel-Catelli lemma is i fact a importat theorem. 6b9 Theorem. For ay 80 evets A 1, A 2,... Proof. First, =1 which is the limit for k of Secod, PA < = P P m= A m lim sup A = 0. PA m, m= P A m A m+1 A m+k PAm + PA m PA m+k. P lim sup A = P = lim P = lim = m=1 m= m=1 lim m= A m lim PA m m= 1 m=1 1 m=1 A m = PA m = PA m = PA m lim PA m = Idicators are fuctios, so, it is meat that 1 B C ω = mi 1 B ω, 1 C ω for each ω Ω. 80 Not just idepedet! 81 Do you see, where the first part of the proof is used below? 82 You see, tails m= a ted to 0 whe for every coverget series =1 a. Thik, what happes for a diverget series.

7 Tel Aviv Uiversity, 2006 Probability theory 61 Aother proof. Itroduce idicators X = 1 A, the A = {X = 1} ad lim sup A = { X = }. We have 83 Markov iequality 84 gives E X X = EX EX = PA PA. P X X M PA PA M for every M 0,. The limit for gives P X M 1 PA. M Aother limit, for M, gives P X = = 0. What about the coverse, =1 PA < = P lim sup A = 0? Check it for a simple case: A 1 A 2... Here, lim sup A = lim A ad Plim sup A = lim PA. Is it true that PA < = lim PA = 0? Evidetly, ot! 6b10 =1 A1 {}}{ A {}} 2 { A {}} 3 { The case of idepedet A is more iterestig ad more complicated. 6b11 A 5 A 4 A 3 A 2 A X < almost sure, but E X = I fact, E X = PA by the mootoe covergece theorem, but we do ot eed it. 84 Recall it: PX M 1 M EX for X : Ω [0,, M 0,. Sketch of a proof: M 1 X M X; thus M PX M EX

8 Tel Aviv Uiversity, 2006 Probability theory 62 Cotiuig the process show o 6b11 edlessly we get for the idepedet sum the same expectatio as for the mootoe sum 6b10; both are = +. However, it 2 3 is far from beig evidet, whether the fuctio show o 6b11 is fiite almost everywhere, like 6b10, or ot. The followig result, well-kow as the secod Borel-Catelli lemma or the secod part of Borel-Catelli lemma aswers the questio: the tower 6b11 is ifiite almost everywhere! 6b12 Theorem. For ay idepedet evets A 1, A 2,... =1 PA = = P lim sup A = 1. Proof. Itroduce idicators X = 1 A, the A = {X = 1} ad lim sup A = { X = }. We have 85 E exp X X = E e X1... e X = = Ee X 1... Ee X = 1 1 e 1 PA, sice Ee X k = e0 PX k = 0 + e 1 PX k = 1 = 1 1 PA k + 1 e PA k. Thus, k=1 E exp X X 0 for, sice =1 1 1 e PA = recall 6a6. Markov iequality gives for every M 0, P exp X X e M E exp X X. e M It follows that 86 P X k M e M E exp X X. k=1 The limit for gives Aother limit, for M, gives P k=1 X k M = 0. P X < = Did you ote, where the idepedece is used? 86 You see, 1 X k 1 X k.

9 Tel Aviv Uiversity, 2006 Probability theory 63 So, for idepedet evets the problem is solved: 6b13 =1 =1 PA < = P lim sup A = 0, PA = = P lim sup A = 1. Note that itermediate values betwee 0 ad 1 are excluded. A corollary: for ay sequece X 1, X 2,... of i.i.d. 87 radom variables, 6b14 E X 1 < = X 0 almost surely ; E X 1 = = lim sup X = almost surely. A explaatio. First, for ay radom variable X, E X < P X > <. =1 Moreover, E X 1 P X > E X accordig to a sadwich argumet: 0 1 x Now, Borel-Catelli lemma gives 88 E X1 = X > ifiitely ofte. 6b15 Exercise. Complete the explaatio, prove 6b14. Hit. E X 1 < E cx 1 < for ay c 0,. The ormal distributio is especially importat. Let X 1, X 2,... be i.i.d. N0, 1 radom variables. The E X 1 <, therefore X / 0 almost surely. 89 Moreover, the desity f X x = cost exp x 2 /2 teds to 0 for x expoetially fast, which esures that x k f X x dx < for each k. Thus, for istace, E X 1 10 <. Applyig 6b14 to the sequece X1 10, X10 2,... we get X10 / 0, that is, X / 10 0 almost sure. It is much more tha X / 0. Still more, cosider E expcx1 2 ; it is fiite for c < 1/2 but ifiite for c 1/2 check it. Therefore, expx/2 2 > ifiitely ofte, that is, X > 2 l 87 i.i.d. = idepedet, idetically distributed. 88 You see, P X > = P X 1 >. 89 Do you thik that, say, X / l l also teds to 0, just because l l? Wait a little...

10 Tel Aviv Uiversity, 2006 Probability theory 64 ifiitely ofte, ad lim sup X / 2 l 1. O the other had, takig c a bit less tha 1/2 we get, say, X 2.02 l evetually, thus, lim sup X / 2 l It meas that 6b16 lim sup X 2 l = 1 almost sure for idepedet radom variables X 1, X 2... havig the ormal distributio with the mea 0 ad the variace 1. I fact, lim supx / 2 l = 1 ad lim if X / 2 l = 1 a.s. 6c Modes of covergece After all, does X / 2 l coverge to 0, or ot? It depeds... 6c1 Exercise. For every radom variable X : Ω R, E X = 0 P X = 0 = 1. Prove it. Hit: P X 0 = lim ε 0 P X ε ; also, P X ε E X /ε. 6c2 Exercise. Let X 1, X 2,... : Ω R be radom variables. The a if Ω is fiite the P X 0 = 1 = E X 0 ; b i geeral, it does ot hold. Prove it. Hit: a max ω X ω 0; b x X 1/ 1 ω What happes if Ω is coutable? What if Ω has both a discrete part ad a cotiuous part? 6c3 Exercise. Let X 1, X 2,... : Ω R be radom variables. The a if Ω is fiite or coutable the b i geeral, it does ot hold. Prove it. Hit: a Xω E X /P {ω} ; E X 0 = P X 0 = 1 ; X 1 X 2 X 3 X 4 X 5 X 6 X 7 X 8 X 9... b What happes if Ω has both a discrete part ad a cotiuous part? 0 1 For a sequece of umbers x 1, x 2,... R the coditio x 0 is uambiguous. I cotrast, for a sequece of radom variables we have several oequivalet iterpretatios of X 0, that is, several modes of covergece.

11 Tel Aviv Uiversity, 2006 Probability theory 65 6c4 Defiitio. Let X, X 1, X 2,... : Ω R be radom variables. a X X almost surely, if 90 P {ω Ω : X ω Xω 0} = 1 ; b X X i square mea, if E X 2 < ad E X X 2 0 ; c X X i absolute mea, if E X < ad E X X 0 ; d X X i probability, 91 if for every ε > 0 P X X > ε 0. 6c5 Exercise. Let X be idicators, X = 1 A, ad X = 0. Show that each oe of b, c, d is equivalet to PA 0, while a is ot. What happes for idepedet A? 6c6 Exercise. Let c, X = c 1 A, ad X = 0. Show that ad b c 2 PA 0, c c PA 0, d PA 0 irrespective of c, a Plim sup A = 0 irrespective of c. Show by examples that there are o two equivalet coditios amog a, b, c, d. 6c7 Exercise. b = c = d for ay X, X 1, X 2,... Prove it. Hit: E X X 2 E X X 2 = Var X X 0; also, P X X ε E X X /ε. 6c8 Lemma. a = d for ay X, X 1, X 2,... Proof. Almost surely X X 0, therefore, X X ε evetually. Itroduce evets A = { X X ε, X +1 X ε,... }, the A 1 A 2... ad Plim A = 1. It follows that lim PA = 1. However, A is icompatible with X X > ε; thus, P X X > ε 1 PA It ca be show that the set {ω Ω : X ω Xω 91 Aalysts say i measure. 0} is measurable.

12 Tel Aviv Uiversity, 2006 Probability theory 66 So, a b c d All the 4 modes a d are modes of covergece of radom variables, ot distributios. Say, for the coi tossig sequece X 1, X 2,... distributio fuctios F of X are all the same, F 1 = F 2 = = F, thus, F F trivially. However, X does ot coverge. 92 Do ot cofuse covergece of distributios ad covergece of radom variables! 6c9 Defiitio. a A sequece F 1, F 2,... of distributio fuctios coverges weakly to a distributio fuctio F, if F x Fx for every x such that F is cotiuous at x. b A sequece X 1, X 2,... of radom variables coverges i distributio to a radom variable X, if F X F X weakly. 93 Item b, covergece i distributio, is rather illogical; you see, covergece of distributios should ot be ascribed to radom variables. Say, for the coi tossig sequece X 1, X 2,... we may say that X X 1 i distributio, as well as X X 2 i distributio! Still, the termiology is widely used. Moreover, people say X F i distributio, havig i mid F X F weakly. For istace, a claim of the form X N0, 1 i distributio appears ofte i limit theorems. The simplest case of Defiitio { 6c9 is the degeerate case: each F is cocetrated at 1 for x a, a poit a, that is, F x = 0 for x < a. Accordigly, X = a almost sure. Here, X coverges i distributio, if ad{ oly if a coverges to a umber a, +, ad the 1 for x a, F F weakly, where Fx = 0 for x < a. Accordigly, X X i distributio, where X = a almost sure. Note that PX = a eed ot coverge to PX = a; this is why the covergece F F is called weak. Clearly, X X i distributio if ad oly if X X i distributio. The latter appears to be equivalet to X p X p for every p 0, 1 such that X is cotiuous at p. Do ot thik that X p± X p±. 6c10 Lemma. a If X X i probability the X X i distributio. b If X 0 i distributio the X 0 i probability. I give o proof. 6c11 Exercise. a If X X i absolute mea the EX EX. b The coverse is geerally wrog. Prove it. Hit: a X X X X X X ; take the expectatio. b It may happe that E X 1 X = 0 but E X X 0; try X 1 = X 2 = To ay limit, i ay mode. 93 Of course, F X x = PX x, ad F X x = PX x.

13 Tel Aviv Uiversity, 2006 Probability theory 67 Here are two famous theorems of measure theory formulated here for probability measures, i probabilistic laguage. For Ω = 0, 1 we may thik i terms of mes c12 Theorem Mootoe Covergece Theorem. Let X, X 1, X 2,... : Ω R be radom variables such that P X X = 1 ad E X 1 <. The 94 lim EX = EX, + ]. 6c13 Theorem Domiated Covergece Theorem. Let Y, X, X 1, X 2,... : Ω R be radom variables such that The 95 P X X = 1, P X Y = 1, ad E Y <. lim EX = EX. For Ω = 0, 1 these facts may be deduced from 5c13. Thik also about the bouded case, especially, idicators... 6c14 Exercise. If Y is o-itegrable the it may happe that P X = 0 = 1 but EX. Give a example. Hit: recall 6c6. 6c15 Exercise. If E X < the there exist bouded radom variables X 1, X 2,... such that E X X 0. Prove it. Recall also the momets as the derivatives of the MGF... 6d Laws of large umbers 6d1 Theorem Weak law of large umbers. Let X 1, X 2,... : Ω R be idepedet, idetically distributed radom variables, E X 1 <, µ = EX 1. The X X µ i absolute mea. Note that covergece i absolute mea implies covergece i probability ad i distributio. The square-itegrable case, E X 1 2 <, is easy: X1 + + X 6d2 Var = VarX 1 0, 94 Existece of the fiite or ifiite limit is evidet due to mootoicity. 95 Note that X Y, thus X must be itegrable.

14 Tel Aviv Uiversity, 2006 Probability theory 68 which gives covergece i square mea, the more so, i absolute mea. The geeral case follows via approximatio: 96 6d3 E X X X k = Y k + Z k, E Y k 2 <, E Z k ε ; µ E Y Y µ + E Z Z. }{{}}{{} EY µ ε ε 6d4 Theorem Strog law of large umbers. Let X 1, X 2,... : Ω R be idepedet, idetically distributed radom variables, E X 1 <, µ = EX 1. The X X µ almost surely, ad i absolute mea. Several proofs are well-kow; o oe is simple eough for beig reproduced here. The most atural proof for my opiio is give by theory of martigales. It combies a clever observatio that E X 1 S, S +1,... = 1 S here S = X X ad a deep theorem: For every itegrable radom variable X ad every radom variables Y 1, Y 2,... quite arbitrary, ot at all idepedet, radom variables E X Y, Y +1,... coverge almost surely, ad i absolute mea. It remais to ote that the limit is just µ by the weak law. Normal umbers ad sigular measures may be metioed here... Here is a sketch of a proof assumig that the momet geeratig fuctio 4f4 of X 1 is fiite i a eighborhood of 0. The case of bouded X 1 is thus covered. By 4f7, d dt MGF X1 t = µ, t=0 therefore for every ε > 0 MGF X1 t < e µ+εt provided that t > 0 is small eough. We apply Markov iequality 97 to e tx 1+ +X : X1 + + X P µ + ε = P e tx 1+ +X e tµ+ε EetX 1+ +X Ee tx 1 = = q, e tµ+ε e tµ+ε where q = MGF X 1 t < 1. By the first Borel-Catelli lemma 6b9, X 1+ +X < µ + ε for large e tµ+ε, almost surely. It holds for all ε > 0, thus lim sup X X µ a.s. Similarly, MGF X1 t < e µ εt for some t < 0, which leads to lim if µ. { 96 X k if X k M, Cosider Y k = where M is large eough. 0 otherwise, 97 It was also used i the secod proof of 6b9.

15 Tel Aviv Uiversity, 2006 Probability theory 69 6e Cetral limit theorem Here is probably the most famous theorem of the whole probability theory. 6e1 Theorem Cetral Limit Theorem. Let X 1, X 2,... : Ω R be idepedet, idetically distributed radom variables, EX 1 = µ, Var X 1 = σ 2 0,. The X X µ σ N0, 1 i distributio. 6e2 Here N0, 1 is the stadard ormal distributio. I other words, X1 + + X µ lim P σ z = Φz = 1 z e u2 /2 du 2π for all z R. Oe says that X X is asymptotically ormal for. We caot hope for covergece i probability, sice S m ad S are early idepedet for m 1. Several proofs of the cetral limit theorem are well-kow; o oe is simple. I give a sketch of oe proof i the form of several lemmas. Recall the Poisso distributio Pλ, 6e3 X Pλ P X = k = λk k! e λ for k = 0, 1, 2,... ; EX = λ, σ X = λ ; λ [0,. 6e4 Lemma. Pλ is asymptotically ormal for λ. That is, lim λ P X λ λ λ z = Φz, where X λ Pλ. Hit. Use the Stirlig formula k! 2πkk k e k for k. 6e5 Lemma. Let N λ Pλ be a radom variable idepedet of X 1, X 2,... Itroduce S = X X, S Nλ = X X Nλ. The E S N N µ σ S µ σ 2 0. Hit. O oe had, N / 1 i absolute mea by the weak law of large umbers. O the other had, E S m mµ σ S 2 µ σ m 1 ; ad E... = E E... N.

16 Tel Aviv Uiversity, 2006 Probability theory 70 6e6 Corollary. The followig two coditios are equivalet. S µ σ N0, 1 i distributio; S N N µ σ N0, 1 i distributio. Hit. If two radom variables are close i square mea or i absolute mea, or eve i probability the their distributios are close. 6e7 Lemma. The cetral limit theorem holds whe X 1 takes o a fiite umber of values oly. Hit. Let X 1 take o just two values x 1 ad x 2. We have S N = x 1 N +x 2 N where N is the umber of k {1,...,N } such that X k = x 1 ; similarly, N ad x 2. Due to remarkable properties of Poisso distributio, radom variables N ad N are idepedet ad have Poisso distributios, N P P X 1 = x 1 ad N P P X 1 = x 2. So, SN is the sum of two idepedet radom variables, each beig approximately ormal by 6e4. Fially, the geeral case of the cetral limit theorem follows from 6e7 via approximatio: 6e8 X k = Y k + Z k, Y k takes o a fiite umber of values, E Z k 2 εσ 2, EZ k = 0 ; X X µ σ = Y Y µ σ } {{ } N0,1 + Z Z σ. }{{} E... 2 ε