LECTURE 8: ASYMPTOTICS I We are iterested i the properties of estimators as. Cosider a sequece of radom variables {, X 1}. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1
Defiitio: (Weak covergece) A sequece of radom variables {X, 1}is said to coverge weakly to a costat c if lim P( X c > ε) 0 P for every give ε > 0. This is writte plim X c or X P C ad is also called covergece i probability. N. M. Kiefer, Corell Uiversity, Ecoomics 60
Defiitio: (Strog covergece) A sequece of radom variables is said to coverge strogly to a costat c if or P(lim X c) 1. lim P(sup x c > ε) N > N 0 Strog covergece is also called almost sure covergece or covergece with probability oe ad is writte X C w.p.1 or X C. N. M. Kiefer, Corell Uiversity, Ecoomics 60 3
LAWS OF LARGE NUMBERS: Let {X, 1}be observatios ad suppose we look at the sequece X i 1 X i /. whe does ξ where ξ is some parameter? Weak Law of Large Numbers: (WLLN) Let E(X i ) µ, V(X i ) σ, cov(x i X j ) 0. The - µ 0 i probability N. M. Kiefer, Corell Uiversity, Ecoomics 60 4
P Proof: Recall Chebyshev's iequality: ( X E(X) µ k ) σ / k µ ad V(X) where σ Proof of Chebyshev s iequality σ + ( x µ ) df µ λσ µ + λσ ( x µ ) ( x µ ) df df + µ + λσ µ λσ ( x µ ) df Put i the smallest value of x i the first ad last itegral, ad drop the middle to get:. σ λ σ P( x µ λσ ) N. M. Kiefer, Corell Uiversity, Ecoomics 60 5
Sice we are iterested i X N, ote that E(X N ) µ ad V (X N ) σ /. Cosequetly, lim P ( X µ > ε ) lim σ / ε 0. N. M. Kiefer, Corell Uiversity, Ecoomics 60 6
Notes: 1. E(Xi) µi is O.K. Cosider -1 X - µ with µ µ i.. V(Xi) σ i is O.K.. As log as lim σ i / 0, our proof applies. 3. Existece of σ ca be dropped if we assume idepedet ad idetically distributed observatios. (I this case, the proof is differet.) Strog Law of Large Numbers: (SLLN) Xi are idepedet with E(Xi) µ, V(X ) σ ad i i i σ / i <. The X µ 0 i almost surely (a.s.). N. M. Kiefer, Corell Uiversity, Ecoomics 60 7
Note: Agai, we ca drop assumptio o the existece of σ i if we assume idepedet ad idetically distributed observatios. Some properties of plim: 1. plim XY plim X plim Y. plim (X+Y) plim X + plim Y 3. Slutsky's theorem: If the fuctio g is cotiuous at plim X, the plim g(x) g(plim X). N. M. Kiefer, Corell Uiversity, Ecoomics 60 8
CENTRAL LIMIT THEOREM: (Asymptotic Normality) Defiitio: The momet geeratig fuctio is defied as mt () E(e tx ) tx e f( xdx ). The ame comes from the fact that r dm tx r r x efxdx () E(X ) whe r - dt evaluated at t 0. N. M. Kiefer, Corell Uiversity, Ecoomics 60 9
Note the followig series expasio: tx 1 mt ( ) E(e ) E(1 + Xt + ( Xt) +...)! 1 + α t + 1 α 1 t +... where α r EX r (For example: α µ α µ +, σ ). 1 1 Property 1: The momet geeratig fuctio of i 1 X i whe Xi are idepedet is the product of the momet geeratig fuctios of Xi. (Exercise: Prove this.) Property : Let X ad Y be radom variables with cotiuous desities f(x) ad g(y). If the momet geeratig fuctio of X is equal to that of Y i a iterval -h < t < h, the f g. N. M. Kiefer, Corell Uiversity, Ecoomics 60 10
Example: The momet geeratig fuctio for X ~ q(0,1) 1 1 x tx tx mt () E(e ) e e dx π 1 t e ( ) / 1 x t e dx? π Uses of Asymptotic Distributios: Suppose X - µ 0 i probability. (What ca be said about the distributio of X - µ?) I order to get distributio theory, we eed to orm the radom variable; we usually look at 1/ ( X - µ). N. M. Kiefer, Corell Uiversity, Ecoomics 60 11
Note that the radom variable sequece {1/(X - µ), 1}does ot coverge i probability. (why ot?) We might be able to make probability statemets like 1 / lim P ( ( X µ ) < z) F(z) for some distributio F. The we could use F as a approximate distributio for 1/ (X - µ). This, of course, implies a approximate distributio for X It is a little easier to work with Y 1/ (X - µ)/σ. N. M. Kiefer, Corell Uiversity, Ecoomics 60 1
Cetral Limit Theorem: (CLT) (Lidberg- Levy) The distributio of Y (as defied above) as is Proof: Let m z Φ(z) 1 x e π (stadard ormal) X i µ / dx () t be the momet geeratig fuctio of (Xi - µ). That is, m X i µ σ t () t 1 + + o(t ) where o(t ) is the remaider term such that o(t )/ t 0 as t 0. N. M. Kiefer, Corell Uiversity, Ecoomics 60 13
N. M. Kiefer, Corell Uiversity, Ecoomics 60 14 We kow that Y X X i i1 ( ) ( ). µ σ µ σ The momet geeratig fuctio of Y is m t t Y () m 1 + t + o t X i µ σ l m l 1 + t + o t l 1 + t Y () t
As, m Y (t) e t / which is the momet geeratig fuctio of a stadard ormal radom variable. Poit of the Cetral Limit Theorem: The distributio fuctio of X for large ca be approximated by that of a ormal with mea µ ad variace σ /. Qualificatio: We really should use characteristic fuctio C(t) Ee itx i the proof. N. M. Kiefer, Corell Uiversity, Ecoomics 60 15
Notes: 1. Idetical meas ad variaces ca be dropped straightforwardly. We eed some restrictios o the variace sequece though. I this case, we work with ( X µ ) i1 i i Y. 1/ ( ) i σ 1 i. Versios of the Cetral Limit Theorem with radom vectors are also available. 3. The basic requiremets is that each term i the sum should make a egligible cotributio. N. M. Kiefer, Corell Uiversity, Ecoomics 60 16
Examples: 1. Estimatio of mea µ from a sample of ormal radom variables: I this case, we estimate µ by X, ad the asymptotic approximatio for the distributio of X or (X µ) is exact.. Cosider 1/ ( β $ β) where β$ is the LS esimator. 1 / 1 ( $ 1/ β β) ( X X) X ε -1 1 / [X X / ] [ X ε / ] Where [ XX / ] is the sample secod momet matrix of the regressors. N. M. Kiefer, Corell Uiversity, Ecoomics 60 17
Uder the assumptio that regressors are well-behaved (i.e. cotributio of ay particular observatio to [X ε/] is egligible), we ca apply a Cetral Limit Theorem ad coclude that 1 / ( β $ β) [X X/] -1 1 / [ X ε/ ] D q (, 0 σ Q 1 ). N. M. Kiefer, Corell Uiversity, Ecoomics 60 18
Defiitio: (Covergece i distributio) : A sequece of radom variables {Z, 1}with distributio fuctios {F(z) P(Z z), 1} is said to coverge i distributio to a radom variable Z with distributio fuctio F(z) if ad oly if lim F(z) F(z) at all poits of cotiuity of F(z). D Notatio: Z Z. N. M. Kiefer, Corell Uiversity, Ecoomics 60 19
Some properties of covergece i probability (plim) ad covergece i distributio: 1. X ad Y are radom variable sequeces. If plim (X - Y) 0 ad Y D Y, the X D Y as well.. If Y D Y ad X c i probability (i.e. plim X c), the a. X + Y c + Y b. XY D D cy c. Y / X Y / c, c 0. D 3. If X D X ad g is a cotiuous fuctio, the g(x) D g(x). N. M. Kiefer, Corell Uiversity, Ecoomics 60 0