Asymptotic Theory Greene Ch. 4, App. D, Kennedy App. C, R scripts: module3s1a, module3s1b, module3s1c, module3s1d

Transcription

1 Asymptotic Theory Greee Ch. 4, App. D, Keey App. C, R scripts: moule3s, moule3sb, moule3sc, moule3s Overview of key cocepts Cosier rom vrible x tht itself is fuctio of other rom vribles smple size (e.g the smple me, or the smple vrice). Such rom vrible is clle sequece rom vrible, sice its outcome epes lso o. We re itereste i the properties of x s. There re two possible scerios: (i) x c coverge i probbility to some costt c, or (ii) coverge i istributio to other rom vrible x. Most estimtors use i prctice coverge i probbility. If x coverges to x, we cll the istributio of x the limitig istributio of x. However, for most estimtors of iterest we o't kow or cot erive this limitig istributio. This lso iclues estimtors tht coverge i probbility sigle spike over costt c is ot very excitig or useful istributio! I tht cse, we erive iste the limitig istributio of fuctio ivolvig x (i.e. populr oe is ( x c), where x is smple sttistic c = plim ( x ) ). This fuctio oes ot collpse to spike. This limitig istributio is exctly correct s. We use it to erive the pproximte istributio (or symptotic istributio) of x itself tht hols pproximtely for fiite. The from tht, we c get the symptotic expecttio (for exmple to ssess symptotic ubiseess or "cosistecy"), the symptotic vrice (for exmple to ssess symptotic efficiecy ). Filly, we c the use these properties to compre ifferet estimtors, to perform hypothesis tests. Covergece i Probbility A sequece rom vrible x coverges i probbility to costt c if ( ) lim prob x c > ε = 0 ε > 0 () I wors, s icreses it becomes icresigly ulikely for x to be fr from c ( ε here is just some rbitrry costt, sy 0.00, ot error term). If () hols, we sy plim x = c, or x is cosistet for c. Exmple: Cosier the followig biry mixture istributio: prob( x 0) = = prob x = = Clerly, s it is less less likely tht x =. Sice there re oly two possible outcomes for x, it must be tht plim x = 0. Plims re geerlly ifficult to erive irectly, but the followig sufficiet coitio helps i most prcticl cses: If x hs me µ vrice respectively we sy tht such tht the oriry limits of µ re c 0 x coverges i me squre error (or coverges i qurtic me ) to c

2 plim x = c So covergece i MSE implies covergece i probbility (but the reverse is ot ecessrily true). Exmple: For smple me of vlues rw from y istributio with me µ vrice, we hve: i x = x E ( x ) = µ V ( x ) = µ V ( x ) lim E x = lim = 0 So by covergece i MSE, the smple me x is cosistet estimtor of the popultio me. The smplig istributio of the smple me coverges to spike t µ s the smple size goes to ifiity -> see R script moule3s. This otio extes to the me of y fuctio of give rom vrible. Specificlly, for y E g x fuctio g ( x ), where x is some rom vrible (ot ecessrily sequece RV), if ( ) V ( g ( x )) re fiite costts, the plim g ( xi ) = E ( g ( x) ). Exmple: Cosier ormlly istribute RV x with me µ vrice. It is kow tht ( ) ( ( xi )) = ( µ + ) E exp x = exp µ +, V exp x = exp µ + exp µ + Thus: plim exp exp Note: The bove results re specific exmples of Lws of Lrge Numbers (LLNs). These lws mke sttemets bout the behvior of verges of smples whe smple sizes become very lrge. Slutsky Theorem For y cotiuous fuctio g ( x ) tht is itself ot fuctio of, we hve plim g ( x ) g ( plim x ) =. This is importt LLN-type result, which llows us to fi plims for highly olier x s. I fct, ofte times we c t eve erive lyticl results for the expecttio of such x. Thks to the Slutsky theorem, we c t lest sy somethig bout the cosistecy of x. Exmple: i - i. The: Let x = x s = x x ( plim x ) x plim x µ plim = = = s plim s plim s

3 3 Covergece i Distributio / Limitig Distributio Assume we rw smple of x i s from some istributio, compute the sequece RV x (me, mei, etc.). The we repet this process my, my times to get smplig istributio for x (just F x like we o i script moule3s). Assume this series of x s hs cumultive istributio fuctio, which is likely ukow. Cosier other (o-sequece) RV x with cf F ( x ). We sy tht x coverges i istributio (CID) to x if F ( x ) F ( x) F ( x ), lim = 0 t ll cotiuity poits of F x is the limitig istributio of x. Symboliclly: x x. The limitig me limitig vrice of x re the me vrice of the limitig istributio, ssumig these momets exist. Ofte times (but ot lwys..) the momets of the limitig istributio of x re the oriry limits of the momets of the fiite smple istributio of x. Exmple: Limitig istributio of t - (Greee p. 048) Cosier smple size from str orml istributio. The populr t-sttistic for test of the hypothesis tht the popultio me is zero is give by x t = where s = x - i x, s Where (-) re clle egrees of freeom. Clerly t is sequece RV s it epes o. I this cse we ctully kow the exct (fiite smple) istributio of t - : f t Γ( / ) ( ) / ( ) / t π / = + Γ () This istributio hs me of zero vrice of (-)/(-3). The cf is geericlly give by =. As goes to ifiity, t - coverges to the str orml, i.e. t t F t f x x 0,. Note tht lim 0 = 0, lim =, so here we hve cse where the momets of the limitig 3 istributio (i.e. the str orml) re ieticl to the limit of the momets of the fiite smplig istributio. See script moule3sb for grphicl illustrtio of this exmple. Greee p. 049 (theorem D. 6) shows some importt rules for limitig istributios. Here is perhps the most importt, sort of the log to the Slutsky Theorem for Covergece i Probbility: If x x g x is cotiuous fuctio the g ( x ) g ( x).

4 4 Exmple: We ow kow tht t 0,. How bout the limitig istributio of t? Bse o the rule bove the limitig istributio will be tht of the squre of str orml, which is the chi istributio with oe DOF, i.e. t chi. See R script moule3sb. Most estimtors use i prctice re sequece rom vribles tht coverge i probbility. This implies tht their limitig istributio collpses to spike, which cuses ilemm, sice we ee ozero symptotic vrice to erive key properties of the estimtor perform hypothesis tests. The wy rou this is to first perform stbilizig trsformtio (ST) of the estimtor to rom vrible with well efie limitig istributio. The most commo ST is s follows: ( ˆ ) = ( ˆ plimˆ ) ( ˆ = = ) ST θ z θ θ θ θ (3) This costruct usully coverges to well efie limitig esity, i.e. z f ( z) ˆθ from bove) with this property is clle root cosistet.. A estimtor (i.e. our To get from there to the pproximte limitig istributio or symptotic istributio of ˆθ itself we ee to pply the (or, better, oe of the my versios of the) Cetrl Limit Theorem (CLT, ccorig to Greee the sigle most importt theorem i ecoometrics..). For sclr rom vrible, the Liberg-Levy CLT sttes: If x, x,, x costitute rom smple from y pf with fiite me µ vrice x = x the x µ i x µ orml 0, orml 0,. Similr CLTs for sclrs vectors re give i Greee s Appeix D. (So remember the followig: Lws of Lrge Numbers pply to Covergece i Probbility, Cetrl Limit Theorems pply to Covergece i Distributio.) See R script moule3sc for exmples of stbilizig trsformtios. Asymptotic Distributios A symptotic istributio of some estimtor ˆθ is the istributio use to pproximte the true (ukow) fiite smple istributio. I ecoometrics, we geerlly erive symptotic istributios by first erivig the limitig istributio of ST( ˆθ ), the pplyig CLT. Specificlly:

5 5 Sclr cse: x µ orml If ( 0,) equivletly, x ~ (, / ) Vector cse: µ., the pproximtely or symptoticlly x orml ( µ, / ) If ˆθ is estimtor for prmeter vector θ, ( ˆ ) or, θ θ 0,V the ˆ ~ ( ) θ θ, V. The term V is clle the symptotic vrice-covrice mtrix (or, sloppily, symptotic V θ ˆ. If the bove hols we sy tht ˆθ is symptoticlly vrice ), which I will geericlly lbel s ormlly istribute or symptoticlly orml. Also, if the symptotic vrice of y other V θˆ by o-egtive efiite mtrix, cosistet, symptoticlly orml estimtor (cll it θ ɶ ) excees ˆθ is si to be symptoticlly efficiet. Exmple: We lrey kow tht the symptotic vrice of the MLE estimtor ˆθ is the iverse of the iformtio V θˆ = I θ. It is lso true tht ˆθ is cosistet (s we will show below) mtrix, i.e. ( ) symptoticlly orml. So together: θ θ, ˆ ~ I θ. It c be show tht o other cosistet, symptoticlly orml estimtor hs smller symptotic vrice, so ˆθ is lso symptoticlly efficiet. The smllest possible symptotic vrice is ofte referre to s the Crmer-Ro Bou (CRB). So we c sy tht the MLE estimtor chieves the CRB. (Note: A cosistet estimtor is ofte lso clle symptoticlly ubise ). Asymptotic Properties of the Lest Squres Estimtor Eve so we lrey kow the fiite smple properties of b, it s still importt to lso uerst its symptotic qulities (sice we ofte ee to compre it to other estimtors with ukow fiite properties). Cosistecy of b Assume ( X X) plim = Q, positive efiite mtrix. We c write b = β + X X Xε = β + X X Xε (4) Note tht the / terms ccel out, so mthemticlly the lst expressio is truly equivlet to the first seco. Now, usig the Slutsky Theorem: ( ( )) ( ) β ( ) plim b = β + plim X X plim Xε = + Q - plim Xε (5)

6 6 As show i etil i Greee, Ch. 4, the plim of the lst term is zero, thus plim b = β, i.e. b is cosistet for β. Asymptotic Normlity of b the Delt Metho To erive the full symptotic istributio of b we first perform stbilizig trsformtio... ( ) = b β X X Xε (6)... the pply multivrite versio of the CLT (etils see textbook) to erive ( ) - b ~ β, Q (7) I prctice the symptotic vrice is estimte s ( ) proceure we use for the fiite vrice. It c be show tht ( ) Q. - V ˆ b = s X X, which of course is the sme s X X is cosistet estimtor for We c lso ote tht b will be symptoticlly efficiet is we the ormlity ssumptio for the error term. This result flows form the fct tht uer orml errors b = β ˆ MLE, MLE estimtors re lwys symptoticlly efficiet (i.e. chieve the CRB). Estimtig the Vrice for fuctio of origil estimtes: The Delt Metho the Krisky-Robb proceure Ofte times our ultimte costruct of iterest is (potetilly olier) fuctio of ll or some of the elemets i β, i.e. f ( β ). Usig f ( b ) s the estimtor for f ( β ), we c use the followig symptotic result: ( ( ) ) f b ~ f β, Γ V b Γ where Γ = f ( β) β I prctice, the symptotic vrice is estimte by ˆ ( ) ˆ f V f b = CV C where C = b This erivtio is commoly referre to s the Delt Metho. It c be use for y symptoticlly ormlly istribute estimtor, ot just the OLS estimtor. (8) (9) The squre roots of the igol of V f ( b ) c the be iterprete s str errors for is usully the ultimte objective of pplyig the Delt Metho. ˆ f b. This A ltertive symptoticlly equivlet proceure to erive these str errors is vi simultio, e.g. s suggeste by Krisky Robb (986). The Krisky-Robb (KR) metho works s follows:

7 7. Tke r=..r ( lrge umber, sy 0,000) rws of coefficiet estimtes from their symptotic istributio, i.e. from b ˆ r ~ b, V ( b ).. For ech cse, compute f b. r 3. Exmie the empiricl vrice-covrice mtrix for these rws. Use the igol to costruct str errors. For empiricl exmple of the Delt metho & KR see R script moule3s. I tht exmple, we hve origil estimte the error str evitio ˆ, log with its estimte V ˆ (the lst row, lst colum elemet of the iverte egtive Hessi). Assume you're vrice iste itereste i estimte of the error vrice, ˆ. You c use the Delt Metho to obti str errors for this vrice: I this (sigle-vrible) cse we hve f ( ˆ ) = ˆ ˆ C = = ˆ ˆ Vˆ Vˆ Vˆ ( ˆ ) = ˆ * ( ˆ )* ˆ = 4( ˆ ) ( ˆ ) Alyticl Exmple for the Delt metho: β b β / ( β ) β3 β = β b b = f ( β) = ββ β b 3 3 ˆ ˆ ˆ ( ) f = = = 3 f b / b b3 Vˆ ˆ ˆ ˆ b 3 f bb ˆ 3 ˆ 3 ˆ 33 f f f b b b3 b f ( b ) f f b f C = = b = ( b ) b b b 0 b b b 3 Vˆ f b = CVˆ b C ( ) We woul usully let the computer solve the fil expressio. Note the imesios: If b is k by Vˆ f b = CV ˆ b C will be J by J. f(b) hs J elemets (i.e. is J by ), C will be J by k, Cvet: Keep i mi tht the Delt Metho KR proceure re symptotic cocepts these methos c be highly urelible i smll-smple cotext!