Characteristic functions and the central limit theorem

Transcription

1 Characteristic fuctios ad the cetral limit theorem Saifuddi Syed, # Uiversity of Waterloo Submitted to Dr. Kathry Hare for PMATH 800 April 9, 03 Abstract The cetral limit theorem is oe of the corerstoes of probability ad statistics. It allows us to determie how the sample mea deviates from the the true mea mea regardless of the uderlyig probability distributio. I this paper, we discuss how we ca aalyse the limitig behaviour of sequeces of radom variables via characteristic fuctios. The Lideberg-Lévy- Feller theorem is prove ad cosequeces are provided. We metio applicatios of the cetral limit theorem, icludig the delta method ad Stirlig s formula. Rates of covergece ad depedece of radom variables are also discussed. History/Importace The cetral limit theorem is cosidered by some to be the most importat theorem i probability theory ad statistics. It allows us to determie log term behaviour of radom variables, ad approximate complicated problems. The strog law of large umbers tells us that i the case of iid radom variables, the arithmetic mea coverges to the expectatio value a.s. The cetral limit theorem tell us the asymptotic distributio of the error betwee sample mea ad the expectatio value. It allows us the determie the rate at which strog law of large umbers holds. Theorem. Cetral limit theorem. Suppose X, X,... are iid radom variables such that EX = µ, VarX = σ <. Let S = i= X i, ad let N be a ormally distributed radom variable, with EN = 0, VarN =. The, S E S Var S = S µ σ This theorem is importat because of its simplicity. = N It implies that whe is large, approximately ormally distributed with mea µ ad variace σ. The cetral limit theorem was origially postulated by Abraham demoivre i 733 whe he used the ormal distributio to approximate the distributio of the umber of heads that appear after tossig a fair coi. It was largely forgotte util 8 whe Laplace expaded o DeMoivre s work to use the ormal distributio to approximate a biomial radom variable. It was fially proved by Aleksadr Lyapuov i 90 [?]. George Pólya coied the term cetral limit theorem, referrig to it as cetral due to its importace i probability theory [?]. I the comig sectios, we will itroduce characteristic fuctios, which will be hady tools whe provig the cetral limit theorem ad its geeralizatios. After provig the theorem, we will provide applicatios such as the delta method, ad Stirlig s approximatio. Fially some further topics such as depedecy of radom variables ad rate of covergece will be discussed. S is

2 Covergece i Distributio Throughout this paper, assume we are dealig with a uderlyig probability space, M, ad N will deote a ormally distributed radom variable, with EN = 0, VarN = Defiitio.. Let X be a radom variable. The P X is the probability distributio of X, ad F X is the distributio fuctio of X. Defiitio.. Let P, P, P,... be probability measures with distributio fuctios F, F, F,... respectively. P coverges to P weakly or i distributio if ad oly if for all bouded cotiuous real fuctios f, fdp fdp. We say F coverges to F weakly or i distributio if ad oly if P does. Weak covergece i probability distributio ad distributio fuctio is deoted by respectively P P, F F as Defiitio.3. Let X, X, X,... be radom variables. We say that X coverge to X weakly, or i distributio, if P X coverge weakly to P X. This will be deoted by X X as I will state some importat theorems regardig covergece i distributio that I will take advatage of throughout this paper. Their proofs will be omitted, as they are stadard results. Theorem.4. Let X, X, X,... be radom variables. If X P X, the X X. Theorem.5 Skorokod s Represetatio Theorem. Suppose P ad P are probability measures ad P P. The there exist radom variables X, X such that P = P X ad P = P X, ad X ω Xω for all ω. Theorem.6 Cotiuous mappig theorem. Let X, X, X,... be radom variables, g : R R be cotiuous. If X X the gx gx. Theorem.7 Slutskys Theorem. Let X, X, X,... ad Y, Y, Y,... be radom variables, a R. Suppose The a X + Y X + a as b X Y X a as 3 Characteristic fuctios X X ad Y P a as Before we ca prove the cetral limit theorem we first eed to build some machiery. To do this, we will trasform our radom variable from the space of measure fuctios to the space of cotiuous complex values fuctio via a Fourier trasform, show the claim holds i the fuctio space, ad the ivert back. I this sectio, the goal is to develop the properties that will allow us to achieve our goal. We will develop tools to allow us to show uiqueess ad to deal with sequeces of radom variables. So we begi by defiig the Fourier trasform of a measure.

3 Distributio Notatio Characteristic Fuctio Oe Poit δ a e ita Biomial Bi, p p + pe it Poisso Poism e meit Uiform Ua, b e itb e ita itb a Expoetial Expθ itθ Gamma Gamk, θ itθ k Normal Nµ, σ e itµ t σ Stadard Normal N0, e t Cator Distributio cos t 3 k Table 3.: Example of some commo characteristic fuctios [?] Defiitio 3.. Let X be a radom variable, we defie the characteristic fuctio of X by ϕ X : R C by ϕ X t = Ee itx = EcostX + iesitx Remark 3.. Aside from a egative sig i the expoetial or the π factor, the characteristic fuctio is the Fourier trasform of the probability measure. Table 3. gives examples of some commo characteristic fuctios. Note that these are stadard distributios oe would see i a elemetary probability class, so their defiitios are omitted. 3. Properties of Characteristic Fuctios We begi our aalysis of characteristic fuctio by givig some elemetary properties. Theorem 3.3. [?] Let X, X, X be radom variables ad a, b R. The, a ϕ X t exists, ad ϕ X t ϕ X 0 = b ϕ X t = ϕ X t = ϕ X t c ϕ ax+b t = e ibt ϕ X at d If X, X are idepedet the, ϕ X +X = ϕ X ϕ X e ϕ X t is uiformly cotiuous Proof. a ϕ X t E e itx = E = = Ee i0x = ϕ X 0. Hece e itx is itegrable ad ϕ exists. b Clear from defiitio of ϕ. c ϕ ax+b t = Ee itax+b = e itb Ee iatx = e itb ϕ X at. d Let Y i = costx i, Z i = sitx i, i =,. The we have {Y, Z } ad {Y, Z } are idepedet. ϕ X tϕ X t = E[Y ] + ie[z ]E[Y ] + ie[z ] = E[Y ]E[Y ] E[Z ]E[Z ] + ie[y ]E[Z ] + E[Z ]E[Y ] = E[Y Y Z Z ] + ie[y Z + Z Y ] = E[costX + tx ] + ie[sitx + tx ] = p X +X t 3

4 e ϕ X t + h ϕ X t = Ee it+hx e itx e itx E e ihx = E e ihx. Sice e ihx 0 as h 0, ad is domiated by, we ca apply domiated covergece theorem to get E e ihx 0 as h 0. So for all ɛ > 0, we ca show h to be sufficietly small so isure that ϕ X t + h ϕ X t < ɛ. Properties a,e are what make characteristic fuctios particularly ice. Characteristic fuctios allow us to view radom variables as bouded, cotiuous, complex valued fuctios. The fact that they always exist with o restrictios o their domai makes them them more appealig tha other similar trasformatios such as momet geeratig fuctios. Property b tells us that a characteristic fuctio is real valued if ad oly if X is symmetric about 0. I geeral, properties c,d give us tools to compute a liear combiatio of radom variables. I particular, if X, X,... are idepedet radom variables ad a k are real umbers, the ϕ a kx k = ϕ Xk a k t. Lemma 3.4. [?] For y R, N, we have eiy iy k { y k! mi y + },.! +! k=0 I will ot prove this sice it is a stadard result from complex aalysis ad is just a simple applicatio of Taylor s theorem. We should pay close attetio to the case where = ad y = tx, where X is a radom variable eitx + itx { } t X mi tx, tx3. 6 So after takig expectatios we have ϕ Xt + itex t EX E mi } { tx, tx This particular approximatio will be useful whe provig the cetral limit theorem. At the ed of the day, the characteristic fuctios do ot have a meaig o their ow; we use them as tools to deduce properties about X. This is possible sice the distributio of X is uiquely determied by its characteristic fuctio. Theorem 3.5 Iversio. If X is a radom variable ad a < b the P a < X < b + P X = a + P X = b T = lim T π T e itb e ita ϕtdt. 4 it 4

5 Proof. [?] We begi by fixig T. Sice we have by Taylor theorem for complex variables T e itb e ita T ϕtdt T it e itb e ita ϕt dt T it T e ita e itb a dt t T T T = T b a. tb a dt t The secod last iequality came from a applicatio of lemma 3.4 where = 0. Hece we ca apply Fubii s theorem. T I := lim T π T Sice ft := T 0 theorem. Usig 0 where six x six x e itb e ita it dx π 0 dx = π I = = T lim ϕtdt = lim T π = lim T π = lim T π = lim T π T T T T T T T 0 e itb e ita it e itx dp dt e itx a e itx b dp dt it e itx a e itx b dtdp it sitx a t sitx b dtdp. t six x dx π, we ca apply Lebesgue domiated covergece we get, T π 0 gxdp, sitx a t 0 if x < a if x = a gx = if a < x < b if x = b 0 if x > b I = P a < X < b + sitx b dtdp t P X = a + P X = b 5 Corollary 3.6. If X, Y are radom variables such that ϕ X = ϕ Y, the P X = P Y ad F X = F Y. Proof. We have that P X [a, b] = P Y [a, b] by the previous theorem, so the distributio of X, Y agree o all the itervals. By the uiqueess part of Carathéodory s extesio theorem, we have P X = P Y or F X = F Y. 5

6 Example 3.7. Suppose X,..., X are idetically distributed expoetial radom variables with parameter θ, so they have the followig distributio { e x θ if x 0 F X x = 0 if x < 0 The after referrig to Table 3. ad usig we have ϕ S = ϕ Xk = itθ = itθ This is the same characteristic fuctio as a gamma distributed radom variable with parameters θ ad. Therefore by the uiqueess of the characteristic fuctio, S has the Gamθ, distributio. 3. Cotiuity of Characteristic Fuctios Our goal at the ed of the day is to determie the distributio of S as approaches ifiity. So we eed a better way to deal with sequeces of radom variables. It is atural to ask, if we have a sequece of radom variables X, X,... such that their characteristic fuctio coverge, the do their distributios also coverge? The problem is that the limit of characteristic fuctios may ot be a characteristic fuctio. I will show that the desired result will follow if we have the added coditio that the limit of characteristic fuctios is cotiuous at 0. Theorem 3.8 Helly s Selectio Theorem. For every sequece of F of distributio fuctios, there exists a subsequece F k k ad a o-decreasig right cotiuous F such that lim k F k x = F x at cotiuity poits of F. This is a fairly stadard result from aalysis, ad the proof is omitted. It ca be show relatively easily with a applicatio of the diagoal method. The F resultig from the selectio theorem is ot ecessarily a distributio fuctio. For example, take F as the distributio fuctios of δ, the poit mass probability measure at. The { if x F x = 0 if x <. I this case F x = 0 for all x ad is clearly ot valid distributio fuctio. The issue i this example is that the probability masses are ot well behaved, i the sese there is o oe bouded iterval that cotais majority of the probability for each distributio. For our purposes, we eed a coditio that will esure we get a valid distributio. This leads to our ext defiitio. Defiitio 3.9. A sequece of distributios fuctios {F } is tight if ɛ > 0 there exist a, b R such that F a < ɛ, F b > ɛ. Equivaletly, a sequece of probability measures {P } is tight if ɛ > 0 there is a half ope iterval a, b] such that P a, b] > ɛ. A sequece of radom variables {X } is tight if {P X } is tight. The term tightess refers to the fact that the mass of a family of distributios does ot diverge. Example 3.0. Let {a } be a sequece of real umbers, ad let X are radom variables with P X = δ a. The X are tight if ad oly if {a } is bouded. 6

7 Theorem 3.. [?] Let {P } be probability measures, the {P } are tight if ad oly if for every subsequece {P k } k, there exists a sub-subsequece {P kj } j ad probability measure P, such that P kj P as j. Proof. Suppose {P } are tight, let F be the distributio fuctios of P. We ca apply Helly s theorem to the subsequece {F k } k, so there exist a sub-subsequece {F kj } j, ad rightcotiuous o-decreasig F such that lim j F kj x = F x at cotiuity poits of F. Let ɛ > 0, the there exists a, b such that F a > ɛ ad F b > ɛ for all. Thus we have F a < ɛ, ad F b > ɛ, so F is a valid probability. We deote the probability associated with F by P. Thus we have P P. If {P } are ot tight, the there is some ɛ > 0 such that for all a, b], P a, b] < ɛ for some. Lets pick a subsequece P k such that P k k, k] ɛ. Suppose some sub-subsequece {P kj } were to coverge weakly to some P, the we ca pick a, b such that P a, b] > ɛ, ad P {a} = P {b} = 0. The for a large eough j, a, b] kj, kj] so ɛ P kj kj, kj] P kj a, b] P a, b] < ɛ This is a cotradictio, hece we are doe. Corollary 3.. [?] If {P } is a tight sequece of probability measures, ad if each subsequece that coverges weakly at all coverges to the probability measure P, the P P. Proof. By theorem 3., each sub-subsequece {P kj } j coverges weakly to the P i the hypothesis. If P does ot weakly coverge to P, the there exists a x such that P {x} = 0 but P, x] does ot coverge to P, x]. The there is a ɛ > 0 such that P, x] P k, x] ɛ for ay subsequece. Thus o sub-subsequece P kj ca coverge to P, this is a cotadictio, hece P P. Theorem 3.3 Cotiuity Theorem. [?] Let X, X,... be radom variables. If ϕ X t gt at each t to some g cotiuous at 0, the there exists a X such that X X ad ϕ X = g. Proof. By Fubii s theorem, u u u [ ϕ X tdt = u = u u ] e itx dt siux ux X >/u X /u ux = P X [ X u ]. dp X dp X dp X dp X First ote that the first itegral is real. Sice g is cotiuous at 0 ad g0 =, for all ɛ > 0 there is a u such that u u u gtdt < ɛ/. Sice ϕ t by domiated covegece theorem, there is a 0 such that for all > 0, u u u ϕ X tdt < ɛ. Let α := /u, the P X [ X α] < ɛ, >

8 We ca we icrease α so that P X X > α < ɛ for all, sice there are oly may 0. So P X are tight. Let {P Xk } k be a subsequece such that P Xk P as k, for some probability measure P. By Skorokod s represetatio theorem, there exists a radom variable X such, X k X as k ad P = P X. Sice e itx is a bouded cotiuous fuctio, by defiitio of weak covergece of probability measures we get e itx dp Xk e itx dp X, as. This is equivalet to lim k ϕ Xk t = ϕ X t = gt. Therefore g is a valid characteristic fuctio for some radom variable Y. Sice g = ϕ X, by the corollary to the iversio formula we get P Y = P X = P. Therefore ay subsequece that coverges weakly at all, coverge to P. By corollary 3. we get P X P, or equivaletly, X X. 4 Lideberg-Lévy-Feller CLT The classic cetral limit theorem requires the radom variables to be idetically distributed. However, the coditio of idetically distributed ca be weakeed as log as the radom variables are ice eough, which we will goig ito more detail. Defiitio 4.. Let {r } be a sequece of atural umbers, for each N, let X,,..., X,r be radom variables. Such a collectio of radom variables is called a triagular array. Give a triagular array we defie S = r X,k. Note that every sequece of radom variables X ca be viewed as a triagular array, i the case where r = ad X,k = X k. Triagular arrays ca be useful as they give us a meas to describe how a sequece of radom variables could chage with. Theorem 4. Lideberg-Levy-Feller. Let {X,k },k be a triagular array ad for each, let X,,..., X,r be idepedet. Deote EX,k = µ,k, VarX,k = σ,k, VarS = s = r σ,k. Disregard the degeerate case where all the variaces are 0. Further suppose ɛ > 0 lim Lɛ := s r X k µ k I { Xk µ l >ɛs } 0, 7 the we have r s We will refer to 7 as the Lideberg coditio. X k µ k N 8 Sice we are tryig to determie the behaviour of how S / deviates from the mea, we ca assume WLOG that µ,k = 0. I the case µ,k 0 oe ca replace X,k with X,k µ,k ad the proof will remai the same. This more geeral theorem is much stroger, sice it implies the classical cetral limit theorem. 8

9 Corollary 4.3 Cetral Limit Theorem. If X, X, X,... are iid radom variables with EX = 0, VarX = σ < the S σ N. Proof. Let r =, ad X,k = X k. The the Lideberg coditio is reduced to the followig L ɛ = s EXk I {X k >ɛs } = σ EX I {X>ɛ σ 0, as Here is a quick example of the cetral limit theorem to demostrate its power. Example 4.4. Let X, X,... be iid Poiso radom variables with mea µ, the X µ N. This example shows that the ormal distributio ca be used to approximate eve discrete radom variables. 4. Proof of CLT Before we begi our proof, we will eed a lemma Lemma 4.5. Let w, z,..., w, z C where w k, z k k, the w k z k w k z k. 9 Proof. The claim is trivially true whe =. Suppose its true for = m the we have m+ m+ w k By iductio we are doe. m m z k = w m m+ z m+ w k + z m+ w k z k m m w m+ z m+ + w k z k m+ w k z k. We ca ow prove theorem 4.. The followig proof took the ideas preseted i [?] ad applied them to the triagular array settig i [?]. Proof. By the cotiuity theorem, it is eough to show that ϕ S/s coverges to ϕ N = e t / poit wise. This is the same as showig We begi by otig that ϕ S/s = ϕ S/s t e t / 0 as. r ϕ X,k t/s ad e t / = r { } σ,k t exp. s 9

10 by defiitio of s. Hece we eed to show that { } r r σ,k t ϕ X,k t/s exp 0 as. However, by a applicatio of the lemma 4.5 ad triagle iequality we have { } r r σ,k t ϕ X,k t/s exp s r { } ϕ X,k t/s exp σ,k t s r ϕ X,k t/s σ,k t s r { } + exp σ,k t s σ,k t s. So we are doe if we ca show that r ϕ X,k t/s σ,k t s 0 as, 0 r { } exp σ,k t s σ,k t s 0 as. Note that the is a special case of the 0, i the case where X,k are ormally distributed with mea 0 ad variace σ,k. So if we ca show the first claim, we ca apply the idetical reasoig s to get the secod oe. Let ɛ > 0, the we have r ϕ X,k t/s σ,k t s { } r t X,k E mi s, t 3 X,k 3 r = t 3 6s 3 t 3 6s 3 t 3 ɛ 6s 6s 3 t 3 X,k 3 E 6s 3 I { X,k ɛs } r r r k = t 3 ɛ 6 + t L ɛ s + r E X,k 3 I { X,k ɛs } + t L ɛ E ɛs X,k I { X,k ɛs } + t L ɛ E X,k + t L ɛ 0 E t X,k s I { X,k >ɛs }

11 By the Lideberg coditio we get L ɛ 0, therefore lim sup ϕ X k t/s σ k t t 3 ɛ 6 sice ɛ is arbitrary, we are doe. 4. Lyapuov Coditio The Lideberg although powerful ca be difficult to show. There is a alterative coditio oe ca show to get the same result. The Lyapuov coditio offers a sufficiet coditio that is easier to verify. Theorem 4.6 Lyapuov CLT. Let {X,k },k be a triagular array. Suppose there is some δ > 0 such that X,k +δ are itegrable for all N, k r. Further suppose that The S s N. lim s +δ r s Proof. It is eough to show that the Lideberg coditio holds. lim Lɛ = lim lim r s r s lim ɛ δ s +δ lim = 0. Thus by Lideberg-Lévy-Feller, S s N. ɛ δ s +δ E X,k +δ = 0. E X,k I { X,k >ɛs } X,k +δ E ɛs δ I { X,k >ɛs } r r E X,k +δ I { X,k >ɛs } E X,k +δ Example 4.7. Suppose that X, X,... are idepedet radom variables such that EX = 0 ad are uiformly bouded, as i there exists by some costat K such that X K, N. Also suppose that s = EX k as. The we have S / N. Proof. We will show this by applyig Lyapuov coditio for δ =. s 3 E X k 3 KE X k = K 0 as. s s 3 The Lyapuov coditio is ot as strog as the Lideberg coditio, or i some cases eve the classical oe. There exist radom variables where the variace is fiite, but higher-order momets are ot. The followig example demostrates this:

12 Example 4.8. Let X, X,... be iid radom variables with desity { c if x > fx = x 3 log x 0 otherwise where c is a ormalizig costat. It is trivial to verify that f is a valid desity, with EX = 0 ad EX = c dx <, but for all δ > 0, E X xlog x +δ =. So the Lyapuov cetral limit theorem is icoclusive. However, X satisfies the coditios of the classical cetral limit theorem, so S s N. 5 Applicatios 5. Delta-Method I practice the true stregth of the cetral limit theorem comes i statistics where the sequece of radom variables ofte represets observed data. It is fair to ask how the limitig behaviour chages if the data is trasformed by some g : R R. If g is ice eough we ca determie the limitig behaviour of g S. Let X, X,... be iid radom variables with EX = µ, VarX = σ. The delta method allows us to determie the behaviour of g S as. Whe g is cotiuous, by strog law of large umbers we get g S a.s. gµ. To determie the distributio of g S as we require a bit more structure o g. The followig result is a geeralizatio of the oe preseted i [?]. Theorem 5. Delta Method. Let X, X,... be iid radom variables with EX = µ, VarX = σ, g : R R, N is a radom variable with the stadard ormal distributio. If g has derivative up to order m, g m is cotiuous at µ ad g i µ = 0 for i m, the we have m g S gµ Proof. by Taylor s theorem, there exist radom variables D such that D S g m S g k gµ = k! µ = gm D m! gm σ m N m 3 m! µ S ad k S µ + gm D m S m! µ S µ m We ca rewrite the above equality by m S g gµ = gm D σ m m! S µ σ m 4 By strog law of large umbers we have S a.s. µ, ad sice g m is cotiuous at µ, we have gmd a.s. g m µ. Sice S µ σ N ad x m is cotiuous, by cotiuous mappig theorem m we have N m. Fially, by applicatio of Slutsky s theorem we get S µ σ m g S gµ gm µσ m N m as 5 m!

13 5. Stirlig s Formula Aother applicatio of the cetral limit theorem is that it allows us to approximate umerical quatities usig probabilistic methods. For example, we ca prove Stirlig s formula, which approximates the value of! for large. Theorem 5. Momet Covergece Theorem. Let {X } be a sequece of radom variables such that X X for some radom variable X. If lim sup X <, the lim E X r = E X r for 0 r. 6 The proof of the momet covergece theorem follows routie measure theory argumets ad is omitted. Suppose you have iid radom variables X with EX = 0, EX = σ. Sice X satisfy the coditios of the cetral limit theorem ad the momet covergece theorem we get lim E We ow prove the Stirlig s Formula. Theorem 5.3 Stirlig s Formula. S σ = E N = lim π e! π. 7 = 8 Proof. [?] Let {X } be idepedet, idetically distributed expoetially radom variables with θ =. We defie Y := X, S = X k, it is easily verified that EY = 0, EY =. It the follows that lim E S = 9 π From example 3.7, we kow that S has the Gam, distributio. distributio fuctio Therefore F S x = { x 0 E S =! t e t dt if x 0 0 if x < 0! 0 t t e t dt I other words, S has This is a easy itegral to compute, ad ca be doe by elemetary calculus methods. The result is E S = e 0! Combiig 9 ad 0 we get or equivaletly, e lim =! π, lim π e! = 3

14 6 Further topics Is it possible to weake or remove the coditio of idepedece i the cetral limit theorem? What is the the rate at which the sample mea of radom variables coverge to N? I this sectio I will provide a brief discussio of these complex topics. 6. Depedecy The classical cetral limit theorem assumes idetically distributed radom variables with fiite variace. The Lideberg-Lévy-Feller cetral limit theorem showed that we ca weake the coditio of idetically distributed radom variables so log as they satisfy the Lideberg coditio. It is atural to ask what happes if istead of weakeig the idetically distributed hypothesis, we weake depedece. Jaso showed that you caot eve weake the coditio for pairwise idepedet radom variables. There exist idetically distributed radom variables X, X,... with EX = 0, 0 < EX = σ < such that S σ V, as, where V is a o-ormal, o-degeerate distributio. [?] Oe ca show that if you have a sequece of radom variables X, X,... that are almost idepedet the the cetral limit theorem holds. We will defie what almost meas. Defiitio 6.. Let X, X,... be radom variable. We say {X } is α-mixig if there is a sequece of o-egative umbers α such that α 0 as, ad for all A σx,..., X k, B σx k+, X k++,..., we have P A B P AP B α. If the distributio of the radom vector X, X +,..., X +j does ot deped o, the the sequece is called statioary. Theorem 6.. [?] Suppose X, X,... is statioary ad α-mixig with α = O 5 ad EX = 0, EX <. The V ars σ = EX + EX E k, 3 where the series coverges absolutely. If σ > 0, the S σ N. The coditios of α = O 5 ad EX < are much stroger tha ecessary but are stated to simplify the proof [?]. Eve with these implicatios, the proof is rather log ad subtle. 6. Berry-Essée Theorem Suppose X, X,... are iid radom variables with EX = µ, VarX = σ. The strog law of large umbers tells us that S S µ µ, ad the cetral limit theorem tells us that σ N as. Therefore, we have that for large, S is approximately ormally distributed with E S = µ ad Var S σ. Thus the cetral limit theorem provides us with iformatio o the rate at which the strog law of large umbers holds. Aother cocer we have is: exactly how fast is the rate of covergece i distributio for the cetral limit theorem? Both Berry ad Essées aswered this questio idepedetly durig the World War II whe they came up with the followig result. 4

15 Theorem 6.3. [?] [?] Let X, X,... be iid radom variables such that EX = µ, VarX = σ, E X 3 = γ 3 <. The sup F S µ x F N x C γ3 x σ σ 3 4 C is a umerical costat. It has bee show that C This is remarkable sice it tells us that the distributio fuctios coverge at a rate of O γ3. The proof of this result is, agai, very log, ad is omitted. 7 Coclusio The classical cetral limit theorem tells use how the sample mea of iid radom variables X, X,... deviates from their expected value. I particular, the distributio of the sample mea approaches a ormal distributio. Our primary tool to show this were characteristic fuctios. Characteristic fuctios allow us to represet the distributios of radom variables as uiformly cotiuous bouded complex-valued fuctios. We also showed that the represetatio is uique, ad respect sequeces give the limit is cotiuous at the origi. We the itroduced ad proved Lideberg-Lévy-Feller theorem, which geeralizes the classical cetral limit theorem. It states that you ca still get covergece i distributio to the ormal distributio, eve whe the X are ot idetically distributed, as log as the Lideberg coditio is satisfied. After tha we state the Lyapuov coditio, a weaker coditio that esures the Lideberg coditio is satisfied. Fially we discussed applicatios, ad advaced topics related to depedece ad rate of covergece. 5