The Central Limit Theorem

Transcription

1 The Ceral Limi Theorem The ceral i heorem is oe of he mos impora heorems i probabiliy heory. While here a variey of forms of he ceral i heorem, he mos geeral form saes ha give a sufficiely large umber, ay liear combiaio of radom variables eds o become ormally disribued. We formulae here a simpler versio of his heorem which is sufficie for mos applicaios. his more commo versio, he heorem saes ha if x is he sample mea of i.i.d. ideically, idepedely, disribued) radom variables wih fiie mea ad fiie variace, he samplig disribuio of x is approximaely ormal as. A more precise saeme of he heorem follows wih a derivaio for he special case of i.i.d. radom variables. A brief review of mome geeraig fucios ad Taylor series expasios used i his derivaio are icluded as addedums a he ed of his ui. Proposiio: f X, X,... is a sequece of i.i.d. radom variables wih E[X i ] ad Var[X i ], 0 < <, he R S U V P X ) ) T / W Φ where Φ) is he c.d.f. cumulaive disribuio fucio) of N0,). Proof: The derivaio of he ceral i heorem makes use of a fudameal propery of probabiliy heory. f X, X,... is a sequece of radom variables havig m.g.f. mome geeraig fucio) Ψ X i ), i,,... ad if ΨX ) Ψ ) is he m.g.f. of a radom variable X * havig c.d.f. * x), he * x) x) where x) is he c.d.f. of X. words: f, i a sequece of radom variables, he mome geeraig fucio of he sequece as ges large has he same mome geeraig fucio as he disribuio of radom variable X *, he he c.d.f. of he sequece is * x). More succicly, he m.g.f. uiquely deermies he disribuio. Le Z X. The he m.g.f. of Z is give by / Referece: m.g.f.; addedum

2 X ) Ψ Z E e ) [ ] Xi ) E[ e ] X i ) E[ e ] X i ) E[ e ] ) Realiig ha X i, i,,... are i.i.d. radom variables, ) may be reduced o Ψ Z X i ) ) E[ e ] E[ e X ) X i ) E[ e ] H G K J ] 3) X ) The Taylor series expasio of e is give by e X ) X X o + ) + ) + ) 4) where o ) is he remaider such ha o ) 0. Simplifyig 4) gives Ψ Z X i ) ) E[ e ] E X [ ) X + + ) + o ) 5) where E[ - ] is he expeced value operaor. Recogiig ha expecaio is a liear operaor ad ha

3 E[] f x) E X [ X f x d )] ) ) x X ) f x) 0 E X X [ ) ] ) f x ) X ) f x) gives Ψ Z E X ) [ ) X + + ) + o ) o + + )) 6) ially, recogiig ha Z is a fucio of ad akig he i for large gives Z ) )) o Ψ ) e 7) bu his is jus he m.g.f. of N0,). The ieresed reader should refer o a mahemaical ex i probabiliy ad saisics for a more rigorous proof or for he geeral from where T X + X X Which is also ormally disribued as.

4 Mome Geeraig ucios Cosider a radom variable X wih disribuio fucio x). The m.g.f. mome geeraig fucio) of a disribuio is defied as X ) E[ e ] x R S T x e d x) x e p x ), if X discree x e f x), if X coiuous 8) We call ) he mome geeraig fucio, because all of he momes of X ca be obaied by successively differeiaig ). or example, d X ) d E [ e ] E d d L N M X e ) X E[ Xe ] O QP 9) evaluaig 9) a 0 gives Similarly, 0 X 0) E[ Xe ] E[ X] d X ) d E [ Xe ] E d Xe d L N M X E[ X e ] ad X O QP 0X 0) E[ X e ] E[ X ] ) 0) geeral he h derivaive of ), deoed by Ψ ) X ), evaluaed a 0 equals E[X ]. Tha is, Ψ ) X 0 ) EX [ ] )

5 Aoher impora propery of geeraig fucios is ha he mome geeraig fucio of a sum of idepede radom variables is jus he produc of he idividual mome geeraig fucios. Cosider idepede radom variables X ad Y wih mome geeraig fucios of ) ad Ψ ) Y respecively. Le Ψ X+ Y )deoe he mome geeraig fucio of X + Y. The Ψ E e X+ Y ) X+ Y) [ ] X Y E e e E e Ψ [ ] X [ ] [ X E e ) Ψ ) Y Y ] ) or a radom variable T X + X X, X i i.i.d., Ψ X+ X X ) T ) E[ e ] E e Ψ Xi [ ] Xi Ψ )) X ) 3) Taylor Series Expasio The Taylor series expasio for a fucio fx) is give by x a) f x) f a) + x a) f a) +! x a) f a) ! ) f a) + R 4) where R is a remaider of he coiuaio of he expasio. or example, le f x) e k x ) Leig a, he Taylor series expasio is he give by k f x) + k x ) + x o! ) + ) 5) where o ) deoes a remaider R such ha R 0.