ECON 430 Revsed Sept. 06 Lecture Notes to Rce Chapter 5 By H. Goldste. Chapter 5 gves a troducto to probablstc approxmato methods, but s suffcet for the eeds of a adequate study of ecoometrcs. The commo o-lear ature of ecoomc models ofte requres approxmato methods for a tractable emprcal aalyss. A excellet summary of asymptotc (approxmato) techques ca be foud chapter 4 W.H. Greee s book, Ecoometrc Aalyss, retce Hall (ay edto). Wth the tool kt that book you hadle a large umber of approxmato problems commo ecoometrcs. Ths course does ot go all the way to Greee s summary, but should represet a good bass. The step from ths course up to Greee s level should ot be very large. There are may probablstc covergece cocepts avalable, of whch two, covergece probablty ad covergece dstrbuto are dscussed or mpled Rce. ef. Covergece robablty. Let Y, Y,, Y, be a sequece of r.v. s. TheY coverges probablty to a costat, c, as (wrtte shortly 0, ( Y c ) 0 as. Or equvaletly: Y Y c or plmy c), f, for ay c f, for ay 0, ( Y c ) as. Example : If,, are d wth E ( ) ad Var( ), the (oe of the laws of large umbers prove by Chebyshev s equalty). Example 3 below shows that S ( ), ad also S S (prove by the cotuty propertes of lmts probablty descrbed below). Ths shows that ˆ, ˆ S, ad ˆ S are all cosstet estmators.
I geeral, suppose that s a ukow parameter a model ad ˆ a estmator for depedg o observatos. If ˆ, we say that ˆ s a cosstet estmator for. Ths s a rather weak property but s usually cosdered a mmum requremet for the behavor of a estmator whe the umber of observatos grows large. Eve f t s a weak property, t turs out to be a very useful, ad much used, cocept ecoometrc hadlg of approxmato problems. [Note o the law of large umbers. I the lectures we gave a smple proof of the law of large umbers based o Chebyshev s equalty. That proof assumes that the varace, var( ), exsts. It ca be prove, however, that ths assumpto s ot ecessary. Thus: If,, are d wth E ( ), the (wthout ay assumptos o the varace). Ths s a classcal result probablty theory. ]. Trval dstrbutos. It s sometmes coveet to terpret costats as specal r.v. s. Let a be ay costat (a real umber). We may terpret a as a radom varable by troducg the r.v.,, by ( a). Hece ca oly take oe value, a. The probablty mass fucto s the gve by p( a) ( a). By the defto of expectato ad varace (that does ot exclude ths specal case), we have (check formally!), E( ) a ad var( ) 0. The cdf of becomes () 0 for F( x) ( x) for x a x a (see fgure ) Fgure
3 We may call ths dstrbuto the trval dstrbuto at a. Note that Fx ( ) s cotuous everywhere except at x a. () The momet geeratg fucto (mgf) for s M () t e t ta ta (.e. M( t) Ee e ( a) e ). Let a, a,, a, be a sequece of costats covergg to a as (see appedx (A) for the cocept of a sequece). Ths meas (slghtly more precse tha preseted Sydsæter I): For ay fxed 0, there s a umber N such that a a for every N. From ths defto t follows that covergece of sequeces the usual sese ca be cosdered as a specal case of covergece probablty. ta (3) If a a, the a a (where the a s are terpreted as r.v. s) roof: Let 0 be arbtrarly small. We eed to show that ( a a ). But ths probablty must be ether 0 or accordg to f a a s false or true (sce a, a, ad are costats ad therefore fxed ad ot subject to radom varato). Hece, choosg N such that a a for all N, we have f a a s true, whch t s for all N ( a a ) 0 f a a s false Ths shows that ( a a ) sce the probablty s for all large eough. Q.E.. Q.E.. meas ed of proof. It s short for the lat expresso: quod erat demostradum.
4.3 The cotuty property of probablty lmts. (See appedx (A) for some useful facts about cotuous fuctos.) Theorem (4) Let, Y,,, be two sequeces of r.v. s such that Y c ad d. Let gx ( ) be cotuous at x = c ad h( x, y ) be cotuous at x = c ad y = d. The g( ) g( c) ad h(, Y ) h( c, d) (Ths s also true whe h has more tha two argumets.) [A proof for those terested (optoal readg) s gve appedx.] Example. h( x, y) Suppose that c. The also xy s cotuous (see appedx (A)), ad that (3). [ I.e. h(, Y ) Y h( c,) c c sce c. Here we use that Y because of c ad Y. ] Example 3. Suppose that,, S (.e. ( ) are d wth E ( ) S s cosstet for Reaso (short argumet): It follows from theorem that S ( ) ). ad var( ). The Explaato: By the law of large umbers (see further detals below), E( ), ad. The, sce h( x, y) x y s cotuous, we get from theorem that ( ) h, h( E( ), ).
5 Fally, multplyg by example, we get S. From ths we also obta that aga). [Some more detals: ut wth (whch coverges to as ) ad the same argumet as S S sce g( x) x s cotuous (theorem. Sce,, are d, the,, are d as well E( ) E( ). The, by the law of large umbers, ( ) E have to bother about the varace of. (Note that, because of the ote to example, we do ot.)] S Exercse. Show that the sample correlato, r SSY cov( Y, ) the populato correlato, corre( Y, ), based o a d radom var( )var( Y) Y s a cosstet estmator for sample of pars, (, Y ), (, Y ),,(, Y ) (meag that the pars are depedet ad have all the same jot dstrbuto). Ht: To prove the cosstecy of the sample covarace, wrte SY Y Y. Note that Y s a mea, where Y,,, are d rv s. Hece, coverges probablty to E( ) E( Y ) cov(, Y ). The use the cotuty of the fucto Y g ( x, y, z) x y z, ad fally that (,, ) z g x y z also s cotous. x y Note that r s ot ubased as a estmator of. O the other had, the fact that t s cosstet, justfes ts use for large. Smulato studes ad other vestgatos show addto that t behaves reasoably well eve smaller samples, ad s therefore the most commo estmator of.
6.4 Covergece dstrbuto I the troductory statstcs course, the followg verso of the cetral lmt theorem (CLT) s preseted: Let,, be d wth E( ) var( ) ad var ( ) (mplyg that E( ) ). The, for large ( 30 usually cosdered suffcet), we have approxmately ~ N(0,) ( ~ meas s dstrbuted as ) Ths statemet s somewhat u-precse. What we mea s that coverges dstrbuto to, where ~ N(0, ), as. (We wrte ths shortly, smply ). The formal mathematcal defto, gve Rce, s: ad, or ef. (Covergece dstrbuto) Let YY,, be a sequece of r.v.. s wth cdf s, F( y) ( Y y), ad Y a r.v. wth cdf F( y) ( Y y). We say that y where the lmt cdf, F( y ), s cotuous. (The, for large, Y ~ F( y ) ) approx. Y Y f F ( y) F( y) for every Ths meas: If the lmt cdf, F( y ), s cotuous at y a ad y b, the ( a Y b) F ( b) F ( a) F( b) F( a) ( a Y b) Hece, ( a Y b) ( a Y b) for large. The mportace of ths property follows from the fact that we qute ofte fd ourselves a stuato where Y (beg e.g. a complcated estmator) has a very complcated dstrbuto whle the lmt dstrbuto of Y s qute smple (ofte ormal). Hece, for large, we may be able to replace complcated probablty statemets about Y wth smple probablty statemets about Y. Note that, f the lmt dstrbuto s N(0,) (whch s ofte the case), the the lmt cdf (usually wrtte ( x) ( x), where ~ N (0,) ) s cotuous for all x.
7 Aother useful techcal commet s that covergece probablty ca be terpreted as a specal case of covergece dstrbuto by the followg lemma: (5) Y c s equvalet to Y Y where Y s the trval r.v. at c (.e. ( Y c) ) wth the trval cdf as (). (The last statemet we may smply wrte Y c.) [For those terested, a proof s wrtte out appedx.].5 etermato of lmt dstrbutos It turs out dffcult (usually) to use the defto of lmt dstrbuto drectly to derve a lmt dstrbuto. Therefore, a umber of techques ad tools have bee developed for ths purpose the lterature. Oe mportat tool s by meas of momet geeratg fuctos (mgf s) formulated as theorem A Rce, chapter 5, ad cted below theorem. (A eve more mportat tool s by meas of so-called characterstc fuctos, (see Rce at the ed of secto 4.5), whch requres complex aalyss ad s omtted here.) Theorem (Theorem A Rce, chapter 5) Let Y,,, be a sequece of r.v. s wth cdf s, Y ~ F ( y) ( Y y). ty Suppose that the mgf s, M () t Ee, exst for all. Let Y be a r.v. wth cdf, Note that f F( y ) s the cdf of a ormal dstrbuto (whch s most ofte the case), the F( y ) s cotuous for all y. So, that case, F ( y) F( y) for all y, ad ( a Y b) ( a Y b) for all a ad b whe s large. ty F( y ) ad mgf M () t Ee, ad assume that M ( t) M ( t) for all t a ope terval that cotas 0. The where F( y ) s cotuous). Y Y (.e. F ( y) F( y) for all y
8 Example 4 (example A Rce, secto 5.3) We smplfy the argumet Rce by usg l Hôptal s rule stead of hs seres argumet. Let ~ pos( ),,, where,, s a sequece of umbers (see appedx (A)) such that. The, sce s posso dstrbuted, we have E( ) var( ). We wll show that the stadardzed E( ) var( ) coverges dstrbuto to ~ N (0,), whch follows f we ca show that the mgf of coverges to the mgf of ~ N(0, ),.e. secto 4.5, example A): t M () t e. The mgf of s (see Rce, t ( e ) M () t e We have from before that, f ad Y are r.v. s such that Y a b, the mgf of Y s, at M ( t) e M ( bt). Hece Y M () t e M t e e t t ( t e ) l( ( )) t M t t e or ut x. Sce, we have x 0. From l Hôptal s rule we get t e xt te t t e t l( M ( t)) e lm lm x x x x xt xt tx xt x0 x0 x0 Sce x e s a cotuous fucto of x, (Ed of example.) t M () t e, mplyg ~ N(0,).
9 We wll ow repeat Rce s proof of the cetral lmt theorem (CLT) suppled wth some detals. Note that ths proof ca be take as optoal readg, whch meas that a proper uderstadg of the proof s ot requred for exam purposes. However, the result tself cludg the more practcal verso gve the corollary (6) below, must be uderstood properly. Theorem 3 (CLT, theorem B Rce, secto 5.3) Let,, be a sequece of d r.v. s wth E( ) 0 ad Let S (mplyg E( S ) 0 ad var( S) ). The var( ). S S S ~ N(0,) (or x ( x) for all x sce var( S ) ( x) ( x) s cotuous everywhere). [Note. The proof s oly gve here for the specal case that the mgf of, M ( t) E( e t j ), exsts a ope terval cotag 0, whch s ot always the case (see the ote to (A5) appedx ). The proof for the geeral case s almost detcal to the gve oe, but based stead o characterstc fuctos (defed by g( t) E( e t j ) where s the complex umber, ). Characterstc fuctos exst for every probablty dstrbuto. Such a proof, however, requres some kowledge of complex aalyss, ad s omtted here. ] j roof (optoal readg): Assume that the commo mgf of,,, M ( t) E( e t ), exsts a ope terval, ( ab,, ) where a0 b. The, accordg to (A7) appedx, Mt, () has cotuous dervatves of all orders ( ab., ) Sce,, are depedet ad detcally dstrbuted, the mgf of S s t t M S ( t) Ee Ee e e Ee Ee E e M ( t) t = t t t t uttg S, we obta the mgf, () t M t M Applyg Taylor s formula (see (A4) appedx ) to Mt, () we have
0 3 t t M ( t) M (0) tm '(0) M ''(0) M '''( c) where c s somewhere betwee 0 3! 0 ad t. We have M(0) E( e ), M'(0) E( ) 0, ad M''(0) E( ). Hece 3 t t M ( t) M '''( c) 6 Substtutg to M () t, we obta or 3 t t t M ( t) M M ''( c ) 6 0 ad t M ( t) R t. where 3 t R M '''( c ) 3 3 6, ad c les betwee 3 t We wll ow prove that R 0,.e. R M '''( c ) 0 3 6 t t Sce c les betwee 0 ad, ad 0, we must have that c 0. Therefore, M '''( c ) M '''(0) sce M'''( t ) s cotuous 0 (see (A7) appedx ). Hece, M'''( c ) s bouded, ad M '''( c ) / 0, whch proves that R 0. We fally get t a M ( t) R Thus, usg (A6) appedx, we get N (0,) M () t e where t a t t R, whch s the mgf of. roperty A Rce, secto 4.5, tells us that the mgf uquely determes the probablty dstrbuto. Hece, ~ N(0,). Q.E..
I practce the followg reformulato of the CLT s the most commo or practcal: Corollary (CLT) (6) If,, ~ d, wth E( ) ad var( ) ( ) ~ N(0, ), whch meas that for large., the ~ N(, ) approxmately roof. We show how the result follows from Rce s CLT theorem 3: ut Y. The, YY,, ~ d, E( Y ) 0 ad var( Y ). We ca the use theorem 3: Y ( ) ~ N(0, ) approx. ( ) ~ N(0, ) for large approx. ( ) ~ N(0, ) for large approx. ~ N(0, ) for large approx. ~ N(, ) for large. Q.E.. [Note that we the proof several tmes have used the well kow property of the ormal dstrbuto: If ~ N(, ), the a b N a b b ~ (, ) where a,b are costats.] The ext result that we preset, s a extremely useful result for statstcal practce: Theorem 4 (Slutsky s lemma) Let A, B, be r.v. s such that ad. The A I partcular, f 0, the A a (costat), B A B a b b (costat), A B b (because of (5) above). Note that ecoometrc lterature (see e.g. Greee s book, Ecoometrc Aalyss ), t s usually theorem o the cotuty property of plm that s referred to by Slutsky s theorem, whle, the statstcal lterature t s usually ths theorem 4 that s meat. It appears that Slutsky proved several smpler versos of both these two ad other lmt results a paper 95. I ths course we wll refer to theorem 4 as Slutsky s lemma (or theorem), sce ths result s oe of the most mportat results of the course, ad sce t takes a lttle bt of trag exercses to lear to use t properly.
The proof s a straghtforward, but somewhat legthy, -, argumet alog the les llustrated appedx, ad s omtted here. Here we llustrate the result by makg some argumets for cofdece tervals preseted the troductory statstcs course more precse. Example 5. (Cofdece tervals) (Note: It s recommeded that you study ths example thoroughly ad lear the argumet used. I partcular ote how Slutsky s lemma s used the argumet. The example also gves a example of why the cocept of cosstecy s useful.) Suppose,, are d, wth E( ) (ukow) ad var( ). We wat a cofdece terval (CI) wth degree of cofdece,, for the ukow. Eve f the commo dstrbuto, Fx, ( ) for the s, s ukow, the dstrbuto of s approxmately kow for large ( 30 usually cosdered suffcet) because of the CLT, whch we utlze as follows: For large, approx. ~ N(0, ). Hece, ( z z ) where z s the upper -pot N (0,). Mapulatg the probablty (do t!), we get (as the basc statstcs course) z z Thus, f s kow, the a approxmately CI for s gve by (7) z I practce s usually ukow, but accordg to Slutsky s lemma, ca be replaced by a cosstet estmator, as the followg argumet shows: ut U where ˆ S ( ) s cosstet for (see ˆ example 3). We the have
3 U ˆ ˆ ˆ Sce (see theorem ad example 3), ad ˆ lemma U ~ N(0, ). Hece, for large, Mapulatg ths (do t!), we get, we have from Slutsky s ( z U z ). ˆ ˆ z z whch gves the approxmate CI for : z the approxmato s usually satsfactory for 30. ˆ. Smulato studes show that Notce that the CI s the same as the CI (7) where we have replaced the ukow wth a cosstet estmator ˆ, ad that t s Slutsky s lemma that allows us to do that. We have a smlar state of affars for posso- ad bomal models: The posso case: Suppose that the umber,, of workg accdets durg t tme uts a large frm, s ~ pos( t ), where s the ukow expected (.e. log ru average) accdet rate per tme ut the frm. The E( ) t var( ), whch mples that ˆ s a ubased estmator of. Sce ˆ var( ) 0, t follows from t t t Chebyshev s equalty (check!) that ˆ s cosstet for as well as t (.e., ˆ ). From example 4 we get that t t t ˆ t ˆ t t ~ N(0, ) sce t as t. t t t Slutzky s lemma shows that we ca replace by ˆ the deomator of destroyg the approxmato substatally,.e., t wthout
4 ˆ Ut t t ~ N(0, ) sce as t, usg ˆ ˆ t ˆ that the fucto, g( x) x s cotuous x. We the get for large t ( the crtero t 0 s usually cosdered suffcet ), the followg approxmato ˆ ˆ ˆ ˆ z z t t ˆ the gves a approxmate CI for : ˆ z. t scuss the bomal case yourself. Appedx (mathematcal prerequstes for Rce, chapter 5) Frst some revew of sequeces ad cotuous fuctos: (A) Sequeces (see also Sydsæter I, secto 6.4 o sequeces ( tallfølger )) A example of a sequece s: (),,,,,, (or descrbed more shortly smply as,,, ) 3 4 3 whch s a sequece of umbers cotued deftely. It cossts of ftely may umbers, oe for each teger,. Abstractly we ca descrbe a sequece (of umbers), a, a, a3,, a, smply as a fucto, a, defed for each atural umber. Thus, the sequece () ca also be descrbed as a for,,3,. We see that ths partcular sequece coverges to 0 as creases,.e., lm a lm 0. However, geeral a sequece does ot have to coverge. For example, the sequece () does ot coverge (),,,,,( ), (or a ( ) for,,3, ) Most umber sequeces we meet ths course, however, coverge. For example
5 () 3 4,,,,, 3 (or a for,3,4, ) whch coverges to. (Note that as ). A famous sequece s the followg: (v) for,,3, whch coverges to e,78888 as (prove A6 below). We also talk about a sequece of radom varables,,, 3,,,, whch just meas that the r.v. s well defed for ay atural umber. Whe we say that the fte sequece of r.v. s,,, 3,, s a d sequece, we mea that all the (fte umber of) varables are observed uder detcal codtos (.e., they have the same dstrbuto) ad observed depedetly of each other. From ths sequece we ca defe other sequeces, for example the sequece of meas ( ):,, 3,, where, ( ). ( ) ad so o. (Note that we have here put a dex,, o the mea to uderle ts depedece o,.e., the umber of observatos used. Usually we drop that dex from the otato f the umber of observatos s fxed ad uderstood from the cotext.) (A) Cotuous fuctos (revew) efto (see Sydsæter I, secto 6.9): A fucto y f ( x) s cotuous x f for ay sequece, x, x, x 3, that belogs to the area of defto of f ad coverges to x, ( lm x x), the also f ( x ) f ( x) as 3. 3 Smlarly for several argumets f: z f ( x, y) s cotuous (, ) x, x, x y, y, y we have that f ( x, y) f ( x, y) as. ad xy f, for every sequece,
6 I ths course we ofte eed to check that a fucto s cotuous order to use varous results from the theory. There are some smple rules you should kow that makes t easy most cases to check that a fucto s cotuous smply by lookg at the expresso for the fucto: () All elemetary fuctos used ths course are cotuous. Those clude e.g.: lear fuctos, y ax b where a,b are costats, power fuctos, y x where s a teger, or y x, r x for ay real r whe 0 x x expoetals, y exp( x) e or y a for ay a 0, log fuctos, y log( x) whe x 0, the gamma fucto, y ( x) whe x 0. () If y f ( x) ad y g( x) are both cotuous, the all the followg fuctos are cotuous as well: (a) y c f ( x) where c s a costat, (b) y f ( x) g( x) (c) y f ( x) g( x) (d) y f ( x) g( x) whe gx ( ) 0 (e) y f ( g( x)) - a fucto of a fucto. () The rules uder () are stll vald f f ad g deped o more tha oe varable. For example, f f ( x, y ) ad g( x, y ) are both cotuous x ad y, the h( x, y) f ( x, y) g( x, y) s cotuous x ad y (whe g( x, y) 0 ) ad so o. Examples: Usg () ad ()(a ad b), we see that geerally, ay polyomal x. y x x 5 3 4 s cotuous, ad, more h( x, y) x xy s cotuous sce both of ()). x ad xy are cotuous (the last oe because The pdf of the N(, ) dstrbuto, cotuous sce: x s cotuous s just a costat), ( ) f ( x) e ( x ), we mmedately see s x s cotuous (usg () ad ()e ad otg that
7 ( x ) e s cotuous (usg () ad ()e), f( x) s cotuous (usg ()a). The followg results are much used probablty theory (a motvato ca be foud Sydsæter I, secto 7.6). (A3) a () For ay real a, e ca be expressed as a fte seres a a a a e a 0!!! () If c s a commo factor, t ca be take outsde the sum, a a a c c ce!! 0 0 [Note. The theory of fte seres s ot treated the mathematcs currculum, except geometrc seres, so we wll ot go to ths here. We oly meto that the precse mathematcal meag of the fte sum s as a lmt of a correspodg sequece of umbers (see A above), a a s a,!!,,3,. The t ca be show that a lm lm a a e s a!!, s well defed ad true for every a whch s the precse meag of the statemet () (we say that the seres s coverget f the correspodg sequece coverges). The oly addtoal result from the theory of fte seres we eed s the last statemet that a commo factor ca be take outsde the sum. Ths partcular seres s maly used to derve the mgf for a posso r.v. (see Example A Rce, secto 4.5): ~ pos( ) mples that the mgf s M ( t) E( e ) e t t ( e ) ] Much of approxmato theory mathematcs ad probablty theory s based o the famous Taylor s formula (see Sydsæter I, secto 7.6):
8 (A4) Let f( x ) be tmes dfferetable a terval that cotas 0 ad x. The, f( x ) ca be approxmated by a polyomal as follows x x x ( ) ( ) (0) '(0) ''(0) (0) ( ) f x f f f f R x!!! where the error term, ( ) x R x, s R x f c ( )! s a umber lyg somewhere betwee 0 ad x. ( ) ( ) ( ), where c (A5) [Note: I (A4) we say that f( x ) s expaded aroud x 0 we ca expad f( x ) aroud ay other value, x f ( x) f ( x ) ad defe g( h) f ( h) where hx. The g(0) f( ) ad of f( x ) aroud x ( ) ( ) g (0) f ( ) :. Applyg (A4) to ( ). From (A4) t follows that, where f s dfferetable: Wrte gh, we obta a expaso x ( x ) ( ) ( x ) ( ) f ( x) g( x ) f ( ) f '( ) f ( ) f ( c)!! ( )! where c s a umber lyg somewhere betwee ad x.] Example 6. Rce secto 4.6 gves examples of fdg approxmate expressos of expectatos ad varaces. Let be a r.v. wth E( ) ad var( ). Suppose we wat the expectato ad varace of a trasformed r.v., Y g( ). If g s complcated t s ofte hard to fd EY ( ) ad var(y) exactly. If gx ( ) s dfferetable aroud x, however, we ca easly obta approxmate values by usg Taylor expaso aroud. Igorg the error term, we have from (A5) wth =: g( ) g( ) g '( )( ) By takg expected value ad varace o both sdes, we get (ote that g( ) ad g '( ) are costats) E( g( )) g( ) ad var( g( )) [ g '( )]
9 By cludg a extra term the expaso, we may obta a (hopefully t depeds o the error term) better approxmato to the expectato: g ''( ) g( ) g( ) g '( )( ) ( ) gves E( g( )) g( ) g ''( ) (read example B Rce, sec. 4.6) Note that t s usually ot a good dea ths cotext to clude may terms the Taylor approxmato sce terms lke ( ) r for larger r are ofte statstcally qute ustable, whch may destroy the approxmato. (Ed of example.) From (A4) we ca ow derve the followg much used result (also used the proof of the CLT): (A6) If a,,, s a sequece of umbers(see Sydsæter I, secto 6.4) covergg to a umber, a (.e. a a), the a e a roof (optoal readg): Takg log o both sdes, the result follows f we ca show that a x a l + a (sce e s a cotuous fucto 4 ). ut x. The x a a. Applyg (A4) to the fucto, f ( x) l(+x), wth oly oe term plus error, we get f ( x) f (0) f '( c) x x, where c s betwee 0 ad x. Note that f (0) 0 c x a. Therefore, f ( x) l( x) a, usg that x a ad that c 0 c. The last statemet follows sce c always les betwee 0 ad a ad x 0 sce the sequece,,,, bouded (.e., there s a umber C such that a x (mplyg 0 c x ), a coverges to a, ad therefore must be C for all ). Q.E.. 4 By defto the cotuty of x lm x x), the also e x e meas that for ay sequece, 3 x e as. ut x l x, x, x, covergg to x, ( a.
0 Note: The rest of appedx ad s optoal readg. I order to make the proof of the CLT (theorem 3, page 8) completely rgorous we eed oe more mathematcal fact. (A7) If the mgf, M ( t) E( e t ) of a r.v.,, exsts for all t a ope terval cotag 0 (.e. for all t ( a, b) where a0 b), the the -th dervatve, ( M ) () t, exsts for all,, ths terval. Ths mples, partcular that ( M ) () t s cotuous ( ab, ) for all. [Note. Ths result s ot hard to prove, but requres results from more advaced tegrato theory, ad s therefore omtted here. Note also that (A7) shows that the assumpto that Mt () exsts a ope terval aroud 0, s a qute strog assumpto o the dstrbuto of. It mples that momets, r r ( r) E( ), of all orders r,, exst. Ths follows sce, E( ) M (0) the exsts for all r. The assumpto s vald for most of the commo dstrbutos met ths course, but there are otable exceptos. For example t s ot true for t-dstrbutos, sce, f s t-dstrbuted wth r degrees of freedom, the t ca be show that E( ) exsts oly for r. ] Appedx (some proofs) roof of (4) (optoal readg) We wll prove the h(x,y)-case. Try to wrte out a proof for the smpler g(x)-case yourself ( case you do t realze that the g-case follows drectly from the h-case). Suppose c ad Y d ad that h( x, y ) s cotuous for x c, y d. Choose a 0 arbtrarly small. We eed to prove that ( h(, Y ) g( c, d) ). Accordg to the meag of cotuty (see e.g. Sydsæter I, sec. 6.9), there s a 0 such that, wheever xc ad yd, the h( x, y) h( c, d). efe evets, A, B, C by A ( c ), B ( Y d ), ad C ( h(, Y ) h( c, d) ). We the have A B C whch mples that ( C ) ( A B ). (Note that f A, B are evets such that A B, or A B terpreted as sets, the ( A) ( B) ). Accordg to the defto of probablty lmt, A ( ) ad B ( ). Ths mples that ( A B ) sce
( A B ) ( A ) ( B ) ( A B ) as (Note that ( A B ) ( A ) mples that ( A B ) ). Hece, sce ( C ) ( A B ), also C ( ) as. Q.E.. roof of (5) (optoal readg) ) Suppose that Y c. We eed to prove that Y Y where ( Y c). Let the cdf of Y be F ( y ) ad the cdf of Y be F( y ),.e. the trval cdf at c (see.) 0 for y c F( y) ( Y y) Thus F( y ) s cotuous for all y `c. for y c Hece, accordg to the defto of covergece dstrbuto, we eed to show that F ( ) ( ) y F y for all y `c, or F ( y) 0 for y c ad F ( y) for y c. Aga we use that f A B, the ( A) ( B). Suppose y c (or yc 0). The the followg evets satsfy ( Y c y c) ( ( y c) Y c y c) ( c ( y c) Y c y c) ( c y Y y) ( Y y) Hece F ( y) ( Y y) ( Y c y c) sce Y have that F ( y). Now, suppose y c (.e. cy 0). We have c. Therefore, we must c y ( Y y) ( Y y) ( c Y c y) ( Y c c y) ( Y c ) c y Thus, F ( y) ( Y y) Y c 0, whch mples that F ( y) 0, ad we have prove that Y ) Y. Now, coversely, suppose that Y Y where ( Y c). The F ( ) ( ) y F y for all y `c. Let 0 be arbtrary small. We have ( Y c ) ( c Y c ) ( c Y c ) F ( c ) F ( c )
Sce F( y ) s cotuous for yc ad yc, the last expresso coverges to F( c ) F( c ) 0 as. Hece ( Y c ), ad we have prove that Y c. Q.E..