Chapter 8. Single-Equation GMM

Transcription

1 Chatr 8. Sigl-Equatio GMM Th LSE, th GLS stimator, th MLE, th IV stimator ad th 2SLS stimator ar all scial cass of th gralizd mthod of momts (GMM) stimator. This stimator is hitd i,.g., Sarga (958), Ammiya (974) ad Whit (982b), but a formal dvlomt is usually crditd to Has (982). I statistics, a rlatd stimator is th gralizd stimatig quatios (GEE) stimator of Liag ad Zgr (986). This chatr covrs th sigl-quatio gralizd mthod of momts (GMM). Rlatd matrials ca b foud i Chatr 3 of Hayashi (2), Chatr 6 of Camro ad Trivdi (25), Chatr 7 of Has (27), ad Chatr 8 of Wooldrig (2). For a ituitiv itroductio o GMM, s Alastair Hall (993) ad Wooldridg (2); for discussio o mirical alicatio issus, s Ogaki (993); for a mor comrhsiv tratmt of GMM, s Alastair Hall (25). GMM Estimator W cosidr th liar modl i this sctio. I a liar modl, E [g(w i ; )] = E z i y i x i = ; () whr g(; ) is a st of momt coditios, ad w i = (y i ; x i ; z i ). This is th istrumt xogity coditio E [z i u i ] = i th dogous liar rgrssio modl y i = x i + u i with E[x i u i ] 6=. D th saml aalog of () g () = X g i () = i= X z i y i x i = Z y Z X : i= Wh l > k, w caot solv g () = xactly as ituitivly show i Figur. Th ida of th GMM is to d a stimator which sts g () "clos" to zro. For som l l wight matrix W >, lt J () = g () W g (): (2) This is a o-gativ masur of th "lgth" of th vctor g () udr th ir roduct h; i W i Sctio 3 of Chatr 2. For xaml, if W = I l, th, J () = g () g () = kg ()k 2, igyu@hku.hk

2 Figur : g () = Ca Not Hold Exactly for Ay : k = ; l = 2 th squar of th Euclida lgth. Th GMM stimator miimizs J (). Not that if l = k, th g () =. Th GMM stimator rducs to th MoM stimator (th IV stimator) ad W is ot rquird. Th rst ordr coditios for th GMM stimator J b g ( )W b g ( ) b = 2 X Z W Z y Z X b ; so b GMM = X Z W Z X X Z W Z y : (3) Whil th stimator dds o W, th ddc is oly u to scal, for if W is rlacd by cw for som c >, b GMM dos ot chag. I Sctio 4 of Chatr 7, is idti d as ( A ) A = E [x i z i ] E[z iz i ] AE[z i z i ] E [z i x i ] E [xi z i ] E[z iz i ] AE[z i z i ] E[z i y i ], so thr, W is th saml aalog of E[z i z i ] AE[z i z i ]. Wh A = E[z i z i ], w obtai th 2SLS stimator, that is, W = (Z Z). From th FOCs of GMM stimatio, w ca s that although w caot mak g () = xactly, w could lt som of its liar combiatios, say B g (), b zro, whr B is a k l matrix. For a wight matrix W, B = X Z W. If W! W >, ad X Z! E [x i z i ] = G, B covrgs to B = G W. So b is as if d d by a MoM stimator such that Bg ( b ) =. 2

3 2 Distributio of th GMM Estimator Not that X Z W Z X! G WG ad X Z W Z u d! G WN (; ) ; whr = E z i z i u2 i = E [gi g i ] with g i = z i u i. So bgmm d! N (; V) ; whr V = G WG G WWG G WG : (4) I gral, GMM stimators ar asymtotically ormal with "sadwich form" asymtotic variacs. It is asy to chck this asymtotic distributio is th sam as th MoM stimator d d by Bg ( ) b =. Exrcis Suos th momt coditios ar E [g(w i ; )] = with g(w; ) = g (w) g 2 (w). St u J () as i (2) ad driv th asymtotic distributio of th corrsodig GMM stimator of. A atural qustio is what is th otimal wight matrix W that miimizs V. This turs out to b. Th roof is lft as a xrcis. This yilds th cit GMM stimator: b = X Z Z X X Z Z y; which has th asymtotic variac V = G G. This corrsods to th liar combiatio matrix B = G. W W = is usually ukow i ractic, but it ca b stimatd cosisttly. For ay! W, w still call b th cit GMM stimator, as it has th sam asymtotic distributio. Exrcis 2 I th liar modl stimatd by GMM with gral wight matrix W, th asymtotic variac of b GMM is V i (4). (i) Lt V b this matrix wh W =. Show that V = G G. (ii) W wat to show that for ay W, V V. To do this, start by dig matrics A ad B such that V = A A ad V = B B. (iii) Show that B A = B B ad thrfor B (A B) =. (iv) Us th xrssios V = A A, A = B + (A B), ad B (A B) = to show that V V. 3

4 Exrcis 3 Show that wh a w istrumtal variabl is addd i, th otimal asymtotic variac matrix V will ot icras. Discuss wh th two asymtotic variac matrics will b qual. (Hit: us th rsult i Exrcis 2.) I th homoskdastic cas, E u 2 i jz i = 2, th = E [z i z i ] 2 / E [z i z i ] suggstig th wight matrix W = (Z Z), which grats th 2SLS stimator. So th 2SLS stimator is th cit GMM stimator udr homoskdasticity. Tstig E u 2 i jz i = 2 ca b similarly coductd as i tstig homoskdasticity i liar rgrssio. Nvrthlss, w d to mak a additioal assumtio to simlify th asymtotic argumts, i.., Cov(x i ; u i jz i ) is a costat. Without this assumtio, th tsts for htroskdasticity ar mor comlicatd; s Wooldridg (99) for th dtails. Exrcis 4 Tak th sigl quatio y = X + u; E[ujZ] = : Assum E[u 2 i jz i] = 2. Show that if b is stimatd by GMM with wight matrix W = (Z Z), th whr G = E[z i x i ] ad M = E[z iz i ]. b d! N ; 2 G M G ; Th followig xaml illustrats why W =. Examl (Otimal Wight Matrix) Suos E[x i ] = E[y i ] = ad Cov(x i ; y i ) =. W try to d a cit GMM stimator for. First, sort out momt coditios E[g(w i ; )] =, whr w i = (x i ; y i ) : g(w i ; ) = x i y i Sic aars i both momt coditios, w ho to d a bttr stimator tha x or y which uss oly o momt coditio. Of cours, E[g(w i ; )] = uss xtra iformatio (x i ad y i hav a commo ma) about, ad ot robust to such iformatio. W ca us x or y to stimat, but a wightd avrag may b bttr. Suos b =!x + (!) y, ad x i ad y i ar ucorrlatd; th th asymtotic distributio of b is d (b )! N ;! 2 2 x + (!) 2 2 y ;! : whr 2 x = V ar(x) ad 2 y = V ar(y). Miimizig th asymtotic variac, w hav! = 2 y 2 x + 2 : y Wh z i = x i as i liar rgrssio, Cov(x i; u ijz i) =, so such a assumtio is ot rquird. 4

5 That is, th saml (of x ad y) with a largr variac is giv a smallr wight, ad th saml with a smallr variac is giv a largr wight. (Chck that = x+y 2, which corrsods to W = I 2 i (2), may hav a largr asymtotic variac tha x or y). Th asymtotic variac udr this otimal wight is 2 x 2 y 2 x+ 2 y mi 2 x; 2 y. From Exrcis ad 2, th otimal wight matrix W = E[g(w i ; )g(w i ; ) ] = E[(x i ) 2 ] E[(x i ) (y i )] E[(x i ) (y i )] E[(y i ) 2 ]! = 2 x 2 y! ; so J () = g () W g () = (x )2 2 x +! (y )2 2 ; y ad b =!x + (!) y is th sam as th wightd avrag abov. I ractic, 2 x ad 2 y ar ukow. I this siml xaml, thy ca b substitutd by thir saml aalog. Th xt sctio dals with th gral cas. Exrcis 5 I Exrcis 2 of Chatr 7, d th cit GMM stimator of basd o th momt coditio E[z i (y i x i )] =. Dos it di r from 2SLS ad/or OLS? 3 Estimatio of th Otimal Wight Matrix Giv ay wight matrix W >, th GMM stimator b GMM is cosistt yt i cit. For xaml, w ca st W = I l. I th liar modl, a bttr choic is W = (Z Z) which corrsods to th 2SLS stimator. Giv ay such st-st stimator, w ca d th rsiduals bu i = y i x b i GMM ad momt quatios bg i = z i bu i = g w i ; b GMM. Costruct g = g ( b GMM ) = bg i = bg i g ; X bg i ; i= ad d W = X i= bg i bg i! = X bg i bg i g g! : (5) i= Th W!, ad GMM usig W as th wight matrix is asymtotically cit. Exrcis 6 Tak th modl y i = x i +u i with E[z i u i ] =. Lt bu i = y i x i whr is cosistt for (.g., a GMM stimator with arbitrary wight matrix). D th stimat of th otimal 5

6 GMM wight matrix W =! X z i z ibu 2 i : i= Show that W! whr = E[z i z i u 2 i ]. A commo altrativ choic is to st W = X bg i bg i! ; (6) i= which uss th uctrd momt coditios. Sic E [g i ] =, ths two stimators ar asymtotically quivalt udr th hyothsis of corrct sci catio. Howvr, Alastair Hall (2) has show that th uctrd stimator is a oor choic. Wh costructig hyothsis tsts, udr th altrativ hyothsis th momt coditios ar violatd, i.. E [g i ] 6=, so th uctrd stimator will cotai a udsirabl bias trm ad th owr of th tst will b advrsly a ctd. A siml solutio is to us th ctrd momt coditios to costruct th wight matrix. Hr is a siml way to comut th cit GMM stimator for th liar modl. First, st W = (Z Z), stimat b usig this wight matrix, ad costruct th rsidual bu i = y i x i b. Th st bg i = z i bu i, ad lt bg b th associatd l matrix. Th th cit GMM stimator is b = X Z bg bg g g Z X X Z bg bg g g Z y. I most cass, wh w say "GMM" w actually ma " cit GMM". Thr is littl oit i usig a i cit GMM stimator wh th cit stimator is asy to comut. A stimator of th asymtotic variac of b ca b s from th abov formula. St bv = X Z bg bg g g Z X : Asymtotic stadard rrors ar giv by th squar roots of th diagoal lmts of b V=. Exrcis 7 Suos w wat to stimat 2 i Exrcis 4. (i) Show that P i= bu2 i whr bu i = y i x i. b (ii) I th structural modl show that Y = (y; X 2 ). l 2 ; b 2;2SLS y = X + X u; X 2 = X 2 + Z V; is cosistt, Y M Z Y ; b 2;2SLS is also a cosistt stimator of 2, whr Giv th cit stimator b, w ca cotiu to rstimat V by rlacig bg i by g w i ; b ad costruct a w stimator of. This is ratd util th stimator covrgs or ough itratios ar coductd. Th stimator gratd from this rocdur is calld th itrativ stimator. 6

7 4 Noliar GMM Suos th momt coditios ar E [g(w i ; )] = ; whr g(; ) 2 R l is a gral oliar fuctio of 2 R k, l k. Th GMM stimator b miimizs J () = g () W g (); whr g () = P i= g(w i; ) P i= g i(), ad W is a cosistt stimator of E [g i ( )g i ( ) ] which is th otimal wight matrix. For xaml, W = X bg i bg i i= with bg i = g i costructd usig a rlimiary cosistt stimator, rhas obtaid by rst sttig W = I. Sic th GMM stimator dds uo th rst-stag stimator, oft th wight matrix W is udatd, ad th b rcomutd. This stimator ca b itratd if dd. I th Tchical Adix A, w show b is CAN udr som rgularity coditios basd o Nwy ad McFadd (994). Mor sci cally, g g! b d! N ; G G N (; V) ; (7) whr G = i ( )=@. Th asymtotic covariac matrix of b ca b cosisttly stimatd by V b bg b, G b whr b = P i= g i (b )gi (b ) with gi () = g i() g (), ad G b = P i( b )=@. (*) For th MLE, th o-st stimator is cit. Similarly, th o-st GMM stimator is cit. Such a stimator is d d as b = G G G g ( ); whr = + O (= ), G g (), ad () = = P i= g(w i; )g(w i ; ). Exrcis 8 Th quatio of itrst is y i = m(x i ; ) + u i ; E[z i u i ] = : Th obsrvd data is (y i ; x i ; z i ), z i is l ad is k, l k. Show how to costruct a cit GMM stimator for. 7

8 5 Hyothsis Tstig This sctio summarizs th tsts i GMM. W rst discuss two sci catio tsts - th Hausma tst for th rsc of dogity ad th J tst for th validity of ovridtifyig rstrictios. Th J tst is also calld th Sarga-Has tst du to a scial cas stablishd by Sarga (958) ad th gral cas by Has (982). W th cosidr th thr asymtotically quivalt tsts i th GMM framwork - th Wald, Lagrag Multilir (or Rao s Scor), ad Liklihood Ratio tst. Th LR tst is also calld th distac tst or th Nwy-Wst tst du to Nwy ad Wst (987a). Ths tsts ar coutrarts of thos i th liklihood framwork (s Sctio 4 of Chatr 4). It should b mhasizd that a sci catio tst is a tst for th whol modl, ot oly for th rstrictios of itrst. Oly if it is guaratd that th rst of th modl is sci d corrctly, a rjctio of th ull is a sig of violatio of th itrstd rstrictios. Nvrthlss, a rjctio of th ull is tyically caus for cocr. 5. Tstig for Exogity: Th Hausma Tst (*) Th ull is E[xu] =, i.., x is xogous. If th ull is tru, th o istrumts ar dd. Suos th modl is homoskdastic udr th ull, that is, E[u 2 jx] = 2. Udr th ull, th LSE is cit, whil udr th altrativ it is icosistt. O th othr had, th 2SLS stimator is cosistt udr both th ull ad altrativ. Th Hausma tst xamis th ull by chckig for a statistically sigi cat di rc btw th OLS ad 2SLS stimat of. Thr ar various vrsios of this tst, thr of which ar T j = b2;2sls 2;OLS b bv b2;2sls j b 2;OLS ; whr bv = bv 2 = bv 3 = X 2 P Z2 X 2 b 2 X 2SLS 2X2 b 2 OLS ; X 2P Z2 X2 X 2X2 b 2 2SLS; X 2P Z2 X2 X 2X2 b 2 OLS; b 2 2SLS ad b 2 OLS ar stimats of 2 basd o th 2SLS ad OLS rsiduals, rsctivly, X2 = M X X 2, ad Z 2 = M X Z 2. T 2 was roosd by Wu (973; his T 3 statistic) ad by Hausma d (978); T 3 was roosd by Durbi (954). Udr th ull, T j! 2, whr =rak(v j ) ad V j =lim bvj. S also Smith (994) for svral asymtotically quivalt limitd iformatio tsts. 8

9 5.2 Tstig Ovridtifyig Rstrictios: Th J Tst Th hyothss ar vrsus H : 9 s.t. E[g(w i ; )] = (8) H : 8 2 B, E[g(w i ; )] 6=, whr B is th aramtr sac. Wh l = k, thr always xists a 2 B such that E[g(w i ; )] =. So oly if l > k, w d this tst - to tst whthr th ovridtifyig rstrictios ar valid. For xaml, tak th liar modl y i = x i + x 2i 2 + u i with E[x i u i ] = ad E[x 2i u i ] =. It is ossibl that 2 =, so that th liar quatio may b writt as y i = x i + u i. Howvr, it is ossibl that 2 6=, ad i this cas it would b imossibl to d a valu of so that E[x i (y i x i )] = ad E[x 2i (y i x i )] = hold simultaously. I this ss a xclusio rstrictio ( 2 = ) ca b s as a ovridtifyig rstrictio. Not that g ( ) b! E[gi ( )], ad thus g ( ) b ca b usd to assss whthr or ot th hyothsis that E[g i ( )] = is tru or ot. Th tst statistic is th critrio fuctio at th aramtr stimats J = J b = g ( ) b W g ( ) b = 2 g ( ) b bg bg g g g ( ); b whr W is d d i (5). Udr th hyothsis of corrct sci catio, J d! 2 l k : Th dgrs of frdom of th asymtotic distributio ar th umbr of ovr-idtifyig rstrictios. If th statistic J xcds th chi-squar critical valu, w ca rjct th modl. Exrcis 9 Tak th liar modl y i = x i + u i ; E[z i u i ] = ; ad cosidr th GMM stimator b of. Lt J = g b b g b dot th tst of ovridtifyig rstrictios. Show that J d! 2 l k by dmostratig ach of th followig: (i) Sic >, w ca writ = CC ad = C C. (ii) J = C g b C C b C g b. 9

10 (iii) C g b = D C g ( ) whr D = I l C Z X X Z b Z X X Z b C ; g ( ) = Z u: (iv) D!Il R(R R) R whr R = C E[z i x i ]: (v) =2 C g ( ) d! Z N(; Il ). (vi) J d! Z I l R(R R) R Z. (vii) Z I l R(R R) R Z 2 l k. (Hit: I l R(R R) R is a rojctio matrix.) A altrativ way to udrstad th J tst is to show that it is actually a F tst i th homoskdastic liar modl y i = x i + x 2i 2 + u i ; (9) E [z i u i ] =, E[u 2 i jz i ] = 2, whr z i = (x i ; z 2i ). Exogity of th istrumts mas that thy ar ucorrlatd with u i, which suggsts that th istrumts should b aroximatly ucorrlatd with bu i, whr bu i = y i x b i x b 2i 2 with b = b ; b 2 big th 2SLS stimator. So w xct i th rgrssio bu i = x i + z 2i 2 + v i ; () th stimat of ; 2 is clos to zro. Lt F dot th homoskdasticity-oly F statistic tstig 2 = ; th l 2 F covrgs to 2 l 2 k 2 = 2 l k. Exrcis I th homoskdastic liar modl (9), show that (i) J = R 2 u = bu P Z2 bu bu bu=, whr R2 u is th uctrd R 2 i th rgrssio (), ad Z 2 = M X Z 2 ; (ii) l 2 F has th sam asymtotic distributio as J. To furthr arciat th ida of th J tst, cosidr th liar modl (9) agai. Suos w hav o dogous variabl x 2i ad two istrumts z 2i, ad th w ca us ithr istrumt to stimat ; 2. If H holds, w xct that ths two istrumts will grat similar stimats. If th two stimats ar vry di rt, th w susct H fails. Th J tst imlicitly maks this comariso; s Nwy (985b) for such a Hausma tst itrrtatio of th J tst. 2 2 S also Agrist (99) for th dummy istrumt cas.

11 Th GMM ovr-idti catio tst is a vry usful by-roduct of th GMM mthodology, ad it is advisabl to rort th statistic J whvr GMM is usd. Wh ovr-idti d modls ar stimatd by GMM, it is customary to rort th J statistic as a gral tst of modl adquacy. (**) As rortd i th July 996 issu of th Joural of Busiss ad Ecoomic Statistics, th J tst tds to ovr-rjct i it samls. O th othr had, Tauch (986) rortd cass i which J () valuatd at th itrativ stimator ld to udrrjctio of th ovridtifyig rstrictios. Wh th J tst rjcts th ull, thr ar two rsoss, ithr scrig corrct momts basd o som momt slctio rocdur as i Sctio 8. or stimatig th missci d modl dirctly. W bri y discuss th scod rsos hr basd o Hall ad Iou (23). It ca b show that (i) th robability limit of th GMM stimator dds o th limit of th wightig matrix; (ii) th limitig distributio of th GMM stimator dds o th limitig distributio of th lmts of th wightig matrix (scially th rat of covrc to its limit); (iii) th itratd stimators ar ot asymtotically quivalt; (iv) th thr asymtotically quivalt tsts discussd i th xt subsctio ar ot asymtotically quivalt or asymtotically 2 -distributd udr th ull. Thy roos statistics for tstig hyothss about th sudo-aramtrs which hav limitig 2 distributios udr th ull. (**) 5.3 Thr Asymtotically Equivalt Tsts: Th Wald, LM ad Distac Tst Suos w wat to tst H : r() = vs H : r() 6= : (q) (q) W imos th sam rgularity coditios, Assumtio RLS., as i Sctio 4 of Chatr 5 o r(). Sci cally, w assum that r() is cotiuously di rtiabl at th tru valu ad r() has rak q. W dscribd bfor how to costruct stimats of th asymtotic covariac matrix of th GMM stimats. Ths may b usd to costruct Wald tsts of statistical hyothss. Sci cally, h i W = r b br V b R b r b ; whr b is th urstrictd stimator b = arg mij (); ad for a giv wight matrix W i Sctio 3, th GMM critrio fuctio J () = g () W g () ; ad b R b =@.

12 Th ricial advatag of th Wald tst is that it oly rquirs th ucostraid stimator to comut it. Its ricial disadvatag is that it is ot ivariat to raramtrizatio as discussd i Sctio of Chatr 5. Wh th hyothsis is o-liar, a bttr aroach is to dirctly us th GMM critrio fuctio. This is somtims calld th GMM Distac statistic, ad somtims calld a LR-lik statistic. Th ida was rst ut forward by Nwy ad Wst (987a). D th rstrictd stimator as = arg mi r()= J (): Th two miimizig critrio fuctios for b ad ar J ( b ) ad J ( ). Th GMM distac statistic is th di rc D = J ( ) J ( b ): As discussd bfor, if r is o-liar, th Wald statistic ca work quit oorly. I cotrast, currt vidc suggsts that th D statistic aars to hav quit good samlig rortis, ad is th rfrrd tst statistic; s Has (26) for a comariso of its highr-ordr rortis with th Wald statistic. Nwy ad Wst (987a) suggstd to us th sam wight matrix W for both ull ad altrativ, as this surs that D. This rasoig is ot comllig, howvr, ad som currt rsarch suggsts that this rstrictio is ot cssary for good rformac of th tst. This tst shars th usful fatur of LR tsts i that it is a atural by-roduct of th comutatio of altrativ modls. as Aothr tst is th Lagrag multilir (LM) or th scor tst. Its tst statistic is costructd LM = g W G V G W g ; whr V = G W G ; ad G W g is th rst-ordr drivativ of J () at ad lays th rol of th scor fuctio i th liklihood framwork. As th LM tst statistic i th liklihood framwork, w d oly calculat th rstrictd stimator, whil w d to calculat both b ad i th distac statistic. As i Sctio of Chatr 5, w ca also cosidr th miimum distac (or miimum chisquar) tst. D b EMD = arg mi b bv b ; r()= whr b is th otimal GMM stimator, ad V b is a stimator of its asymtotic covariac matrix. It ca b show that bemd = o (); s,.g., Proositio 2 of Nwy ad Wst (987a). Th miimum chi-squar statistic is MC = b EMD b bv b EMD b : 2

13 Lik th distac statistic, it rquirs two miimizatios to comut Th Triity i GMM As i th liklihood cas, w ca show that th Wald, LM ad distac tsts ar asymtotically quivalt. Actually, w ca show thy ar asymtotically quivalt v udr th local altrativs ad wh th momt coditios ar oliar i. Morovr, thy ar v asymtotically quivalt to th miimum chi-squar tst. This rsult is rigorously statd i Thorm 2 of Nwy ad Wst (987a). Proositio Udr som rgularity coditios, ad th local altrativs = + =2 b, W d! 2 q (); whr = b R (R VR) Rb. I additio, W D = o (), W LM = o (), ad W MC = o (). It should b mhasizd that th otimal wight matrix is usd i th costructio of D ; othrwis, D is ot asymtotically chi-squard ad is ot asymtotically quivalt to W. This is aralll to th rsult i th missci d liklihood cas. Also, th form of th LM statistic would b mor comlicatd, ad would i gral ivolv th Jacobia matrix R of th costraits. So it is strogly suggstd to us th otimal wight matrix i th hyothsis tstig of GMM. W ow cosidr som scial cass. Th followig roositio follows from Proositio, 3 ad 4 of Nwy ad Wst (987a). Proositio 2 (i) Wh th modl is just-idti d, LM = D. (ii) Wh g(w; ) = g (w) g 2 (w), D = LM = MC. (iii) Wh g(w; ) = g (w) g 2 (w) ad r() = R c, W = D = LM = MC. Rsult (i) ca b asily rovd. I th just-idti d cas, g b =, so D = J ( ) = g W g. O th othr had, giv G is ivrtibl, LM = g W G V G W g = g W G G W G G W g = g W g : Th quivalc i (ii) dos ot iclud W bcaus it ivolvs th Jacobia of th costraits wh r() is oliar. Th followig xrcis shows Rsult (iii) i th liar istrumtal variabls cas. Th miimum distac statistic J i Sctio of Chatr 5 is umrically quivalt to D ad LM v if th rstrictios ar oliar sic it ca b ut i cas (ii) of Proositio 2. 3

14 Exrcis Tak th liar modl y i = x i + u i ; E[z i u i ] = ; ad cosidr th urstrictd GMM stimator b ad rstrictd GMM stimator of udr th liar costraits R = c. D J () = g () b g () ; ad th b = arg mij () ad = arg mi J (). D th Lagragia R =c L(; ) = 2 J () + R c : (i) Show that = b b = X Z b Z X R R X Z b Z X R R b R X Z b Z X R R b c : c ; (ii) Driv th asymtotic distributio of udr th ull. (iii) Show that J = J b + b X Z b Z X b : (iv) Show that th distac statistic is qual to th Wald statistic. modl (**)Th umrical quivalc rsult suggsts a covit way to calculat D i th liar y i = x i + u i ; E[z i u i ] = ; D z i = z i bu i, ad Z = (z ; ; z ), y i = y i =bu i, y = (y ; ; y ), x i = x i =bu i ad X = (x ; ; x ). Th 2SLS stimator b for a rgrssio of y o X with istrumts Z is a cit stimator of, whr W i (6) is usd as th cit wight matrix (why?). D bu i = y i x b i, ad bu = (bu ; ; bu ). Th sum of squars of th rdictd valus of a rgrssio of bu i o z i is bs = bu Z (Z Z ) Z bu = (y X b ) ZW Z (y X b )= = J ( b ): 4

15 Substitut out th costraits R c =, ad rat th rocdur abov for th rstrictd stimator. 3 Lt S b th coutrart of S b ; th D = S b S. Th umrical quivalc rsult sms uzzlig giv th rakig W LR LM i th ormal rgrssio modl (s Sctio 5 of Chatr 4). Part of th xlaatio is that w assum that all statistics us th sam stimat V b of V. Also, our objctiv fuctio uss th FOCs rathr tha th log-liklihood itslf. (**) Summary of th Triity i GMM ad th M-stimatio (*) W grally cosidr th xtrmum stimator which is d d as b = arg max Q () s.t. 2 R k whr Q () is a gral critrio fuctio. To tst H : r() =, w somtims d th costraid stimator, dotd as, which solvs max Q () s.t. r() = Amog xtrmum stimators, th most oular cass ar M-stimators (.g., th MLE) ad GMM stimators. Thir critrio fuctios ar as follows:. M-stimators: Q () = 2. GMM: Q () = 2 g () P m (w i ; ), that is, th objctiv fuctio is a saml avrag. i= W (l) (ll) (l) g () Hr, w d Q () = 2 J () to giv a aalog of th avrag log-liklihood i th ML cas. Th followig otatio will b usd throughout this sctio: s(w i ; ) (k) g () (l) G () (lk) (ll) () = i; = X i= ; H(w i ; ) (kk) g(w i ; ); R() () =@ ; ) (w i ; ; G (lk) = E X i= i; m(w i ; ; g(w i ; )g(w i ; ) ; (ll) = E[g(w i ; )g(w i ; ) ]: 3 Wh th costraits ar oliar, w caot substitut out for th costraits ad us th 2SLS calculatio to obtai th rsiduals for th rstrictd stimats. 5

16 W summariz th asymtotic aroximatio of ths statistics ad tsts i th followig two tabls, which ar Tabl 7. ad 7.2 of Hayashi (2). Tabl rovids th corrsodc of Taylor xasio for th samlig rror btw M-stimators ad GMM stimators. b + o ) N(; ), Avar b = Trms for substitutio M-stimators GMM Q ( P m (w i ; ) 2 g () W g () P s (w i ; ) [G ( )] P W g (w i ; ) i= i= E [H(w i ; )] G WG E [s(w i ; )s(w i ; ) ] G WWG Tabl : Taylor Exasio for th Samlig Error i= Tabl 2 givs th comots of th thr asymtotically quivalt tst statistics i th ML ad GMM stimatio. Also, Figur 2 rovids a ituitiv xlaatio for ths thr tsts. W ca b itrrtd as twic th di rc i th critrio fuctio at th two stimats, usig a quadratic aroximatio to th critrio fuctio at ; b LM ca b itrrtd as twic th di rc i th critrio fuctio at th two stimats, usig a quadratic aroximatio to th critrio fuctio at ; ad D is rcisly twic th di rc i th critrio fuctio at th ucostraid ad costraid stimats. Not that th sam covariac matrix b is usd i Q ( ) b ad Q ( ). h Wald: r b R( ) b b R( ) b i r b LR: 2 Q ( ) b Q ( ) i 4 Trms for substitutio Coditioal ML E cit GMM P Q () log f(y i jx i ; ) 2 g () b g() b 2 Q ( ) b P s(w i ; )s(w b i ; ) b 5 G b b G; b G b G ( b ); b = b i= (lk) (ll) rlac b by i abov G G; G G ( ); = (lk) (ll) Tabl 2: Triity 4 Th sam wightig matrix is usd i Q ( b ) ad Q ( ) for GMM to guarat LR is gratr tha i it samls. Also, w ca s D = J ( ) J ( ). b 5 or lt I() = E [s(w i; )s(w i; ) ], ad b = I( ). b 6

17 Figur 2: Triity 5.4 Co dc Rgios By ivrtig th tst statistics, w ca costruct co dc rgios for. A straightforward choic of th tst statistic is th Wald statistic. Howvr, as mtiod abov, th distac statistic may rform bttr i som cass of hyothsis tstig. W xct th co dc rgio by rvrtig th distac statistic would ihrit its good rortis i tstig. Suos w wat to costruct co dc rgio for 2, whr = ( ; 2) 2 R k ad 2 2 R k 2 d 2 such that J ( 2 ) ; 2 J b 2 k 2 ; ; is a subvctor of. W d to whr ( 2 ) = arg mi J ( ; 2 ) for a giv 2, th df of th 2 limitig distributio is k 2 bcaus th df of J ( 2 ) ; 2 is l k ad th df of J b is l k so th di rc is (l k ) (l k) = k k = k 2. Of cours, w ca costruct co dc rgio for 2 by collctig 2 s such that J ( 2 ) ; 2 2 l k ; dirctly. Howvr, by obsrvig that J ( 2 ) ; 2 = i hj ( 2 ) ; 2 J b + J b, w ca coclud that this co dc rgio is basd o th joit tst of ovridti catio ad 2 = 2. If th modl is missci d so that th ovridtifyig coditios ar ivalid, this co dc rgio ca b ull. 7

18 6 Altrativ Ifrc Procdurs ad Extsios(*) Mot Carlo studis hav show that stimatd asymtotic stadard rrors of th cit two-st GMM stimator ca b svrly dowward biasd i small samls. A ky obsrvatio for th sourc of this bias is that th wight matrix usd i th calculatio of th cit two-st GMM stimator is basd o iitial cosistt aramtr stimats whos variatio is ot mbodid i th asymtotic covariac matrix stimatio. Widmijr (25) shows that wh th momt coditios usd ar liar i th aramtrs, th xtra variatio du to th rsc of ths stimatd aramtrs i th wight matrix accouts for much of th di rc btw th it saml ad th usual asymtotic variac of th two-st GMM stimator. To this roblm, thr ar a fw ractios i th litratur. First, oliar rocdurs, scially th gralizd mirical liklihood (GEL) stimatio, ar roosd. Ifrcs basd o ths oliar rocdurs ar mor accurat bcaus thy circumvt th stimatio of th otimal wight matrix. 6 Scod, liar rocdurs ar roosd to icororat th variatio i th rst-stag stimator xlicitly. Third, bootstra rocdurs ar ut forward to r th ifrcs basd o th two-st GMM stimator. Not also that for th asymtotic argumts i th rvious sctios to go through, w d som critical assumtios o th data gratig rocss. Rlaxig ths assumtios is th task of currt coomtric ractics. W will ovrviw som xtsios i th litratur. Ths xtsios ar itractd with ach othr ad also with th altrativ ifrc mthods mtiod abov. 6. R mts o Ifrc Thr is a imortat altrativ to th two-st GMM stimator. Sci cally, w ca lt th wight matrix b cosidrd as a fuctio of. Th critrio fuctio is th J () = g ()! X gi ()gi () g () ; i= whr g i () = g i () g () : Th b which miimizs this fuctio is calld th cotiuously-udatd stimator (CUE) of GMM, ad was itroducd by Has t al. (996). A advatag of this stimator rlativ to th two-st stimator is that it is ivariat to how th momt coditios ar scald v wh aramtr-ddt scal factors ar itroducd. This stimator aars to hav som bttr rortis (.g., smallr bias) tha traditioal GMM, but ca b umrically tricky to obtai i som cass (s Sctio 4 of Has t al. (996)). Doald ad Nwy (2) itrrt th CUE as a jackkif stimator to xlai why th CUE is lss biasd. Esstially, th CUE maks th stimator of Jacobia G asymtotically ucorrlatd with g b, which limiats a imortat sourc of ozro xctatios for th FOCs, ad hc of bias. 6 Actually, th bias is also smallr. 8

19 Exrcis 2 (i) Writ out th objctiv fuctio of th CUE i th liar homoskdastic dogous modl. (ii) Show that this CUE is quivalt to th LIML stimator. (iii) Show that if g i () is rlacd by g i () i J (), th th w objctiv fuctio J () = J ()=( + J ()). Th CUE is a scial cas of th GEL stimator of Smith (997); s Nwy ad Smith (24) ad Smith (2) for furthr discussios o th GEL stimator. To dscrib GEL lt (v) b a fuctio of a scalar v that is cocav o its domai, a o itrval V cotaiig. Lt b () = j g i () 2 V; i = ; ;. Th stimator is th solutio to a saddl oit roblm b GEL = arg mi su X 2 2 b () i= g i () = arg mi R(); () 2 whr dots th aramtr sac. Th mirical liklihood (EL) stimator of Ow (988, 99) (ad its GMM xtsio by Qi ad Lawlss (994) ad Imbs (997)) is a scial cas with (v) = log( v) ad V = ( ; ). Th xotial tiltig (ET) stimator of Kitamura ad Stutzr (997) ad Imbs, Sady ad Johso (998) is a scial cas with (v) = which has th comutatioal advatag, rlativ to EL, of havig a urstrictd domai, although th rstrictd domai of EL is usually ot a roblm i ractic. 7 v Th CUE is a scial cas with (v) = ( + v) 2 =2. Associatd with ach GEL stimator ar mirical robabilitis for obsrvatios. Sci cally, b i = ( b bg i )= X whr is th rst drivativ of, bg i = g i ( b ), ad b = arg max 2 b ( ) b j= ( b bg j ); X bg i =: (2) i= For EL ad ET, b i = b bg i ( b bgi ) ad P ; j= b bg j rsctivly. Ths mirical robabilitis b i sum to o by costructio, satisfy th saml momt coditio P i= b ibg i = wh th FOCs for b hold, ad ar ositiv wh b bg i is small uiformly i i. Aothr formulatio of ths stimators is through th miimum discracy (MD) stimator of Corcora (998). Th MD stimator is d d as s.t. = arg mi 2; X i g i () =, i= X h( i ) (3) i= X i = ; i= 7 Aothr GEL stimator is th miimum Hlligr distac stimator (MHDE) of Kitamura t al. (23). 9

20 whr = ( ; ; ). Wh h() = l(), l() (th Kullback-Liblr iformatio critrio), ad 2 w gt th EL, ET ad CUE, rsctivly. For ach MD stimator thr is a dual GEL stimator wh h() is a mmbr of th Crssi ad Rad (984) family of discracis i which h() = [( + )] () + =. 8 Th corrsodig (v) = ( + v) (+)= =( + ). Wh = ; ad, w gt th h fuctios for th EL, ET ad CUE, rsctivly. i () is roortioal to th Lagrag multilirs for th rst (momt) costrait i (3). Th maximizatio roblm for i (2) is cosidrably asir tha th MD roblm, havig much smallr dimsio ad big a siml cocav rogrammig roblm. For = ad, thr ar o xlicit solutios for b ad b. Wh =, xlicit formula for b ad b ar ossibl, but thy usually ivolv o or mor of th b i s big gativ ad so giv ris to roblms of itrrtatio. If w wat to tst H : r() =, w ca us th LR statistic h LR = 2 R i R b ; which follows 2 q udr th ull, whr R() is d d i (), ad = arg mi r()= R(): As a rsult, th ( ) co dc rgio for 2, a k 2 -subvctor of, is h 2 R ( 2 ) ; 2 i R b 2 k 2 ; ; whr ( 2 ) = arg mi R ( ; 2 ) for a giv 2. This Wilks s homo is ivariat to th choic of () (or quivaltly, h()), but scod-ordr rsults dd itimatly o th choic of. For xaml, th GEL is Bartltt corrctabl if ad oly if (v) = log( Th LR tst ca also b usd to tst ovridtifyig rstrictios. I th EL cas, th logliklihoods ar L costraitd = X l(b i ) = i= L ucostraitd = log ; log X i= v). log + b bg i ; so LR = 2 L ucostraitd L costraitd = 2 X i= log + b bg i : 8 For two discrt distributios with commo suort = ( ; ; ) ad q = (q ; ; q ), th Crssi-Rad P + owr-divrgc statistic is d d as I (; q) = i P (+) i q i. i= h(i) masurs th distac btw ad th mirical distributio uif. = d s th Kullback-Liblr distac btw ad uif ; = d s th Kullback-Liblr distac btw uif ad ; = =2 d s th Hlligr distac btw ad uif. i= 2

21 d Udr th ull (8), LR! 2 l k. Kitamura (2) shows that this mirical liklihood ratio tst of th ovridtifyig rstrictios satis s th otimal critrio of Ho dig (965). Th EL ovridti catio tst is similar to th GMM ovridti catio tst. Thy ar asymtotically rstordr quivalt, ad hav th sam itrrtatio. Th ovridti catio tst is a vry usful by-roduct of EL stimatio, ad it is advisabl to rort th statistic LR whvr EL is th stimatio mthod. Widmijr (25) rooss a it-saml corrctio for th variac of liar cit two-st GMM stimators. His corrctio xlicitly icororats th variatio i th rst-stag stimator. Dtails ar icludd i th Tchical Adix B. As to th bootstra ifrc, thr ar basically two mthods attributd to Hall ad Horowitz (996) ad Brow ad Nwy (22) rsctivly. Bod ad Widmijr (22) rort roblms with th bootstra rocdurs wh th wight matrix is a oor stimat of th covariac matrix of th momt coditios, which occurs for xaml wh thr ar a larg umbr of ovridtifyig rstrictios. 6.2 Extsios From th Tchical Adix A, th asymtotic aroximatio i (7) rquirs at last th followig six assumtios. W list ths assumtios ad th rlvat litratur to rlax thm. (i) w i, i = ; ;, is a radom saml. If w i, i = ; ;, ar tim sris w t, t = ; ; T, such that g(w t ; ) ar corrlatd, th th otimal = T E g T ( )g T ( ) X = E g(w t ; )g(w t v= v ; ) X v : v= A cosistt stimator of is oft calld th htroskdasticity ad autocorrlatio cosistt (HAC) stimator. Radig starts from Nwy ad Wst (987b, 994), Adrws (99) ad Adrws ad Moaha (992). (ii) g(w; ) is smooth i. Wh g is odi rtiabl ad/or discotiuous i (.g., th momt coditios i quatil rgrssio), th asymtotic argumts i Sctio 4 ad th usual calculatio algorithm for th GMM stimator may b roblmatic; s Paks ad Pollard (989), Adrws (997a) ad Chrozhukov ad Hog (23) for classical rfrcs. (iii) G is full colum rak. Wh G C =2, th istrumts ar wak, ad caot b cosisttly stimatd. Th 2SLS stimator is clos to th LSE so su rs a srious bias roblm. This strad of litratur starts from Nlso ad Startz (99a,b) ad Boud t al. (995). Classical rfrcs iclud Staigr ad Stock (997) ad Stock ad Wright (2) o stimatio ad Wag ad Zivot (998), Morira (23) ad Klibrg (22, 25) o ifrc. 2

22 (iv) l is xd. Wh l ca go to i ity, thr ar may momt coditios which will icras th bias of th GMM stimator ad dtriorats th stimatio of. Radig starts from Bkkr (994), Chao ad Swaso (25), Ha ad Phillis (26). (v) k is xd. Wh k ca go to i ity, thr ar oaramtric aramtrs i th momt coditios. For idti catio, w d i it momt coditios. Radig starts from Nwy ad Powll (23), Ai ad Ch (23), Darolls t al. (2). (vi) Thr ar oly momt qualitis. If thr ar momt iqualitis, ca oly b artially idti d. Radig starts from Chrozhukov t al. (27), Brstau ad Moliari (28), Paks t al. (24). 7 Coditioal Momt Rstrictios I may cass, th modl may imly coditioal momt rstrictios E[u (w; ) jx] = ; whr u (w; ) is som s fuctio of th obsrvatio ad th aramtrs. For xaml, i liar rgrssio, u (w; ) = y x, w = (y; x ), ad s = ; i a joit modl of coditioal ma ad variac,! y x u (w; ) = (y x ) 2 f(x) for a sci catio V ar(yjx) = f(x), so s = 2. Coditioal momt rstrictios imly i it ucoditioal momt coditios, sic for ay fuctio of x, say (x), E[(x)u i (w; )] =. So a atural qustio is which istrumts ar otimal, or what is th smiaramtric cicy boud for. Chambrlai (987) drivd this boud by aroximatig th CDF F (x) ad th coditioal CDF F (wjx) with multiomial distributios. It turs out that th otimal istrumts ar A(x) = G(x) (x) ; whr G(x) = (w; ) =@ x, ad (x) = E u (w; ) u (w; ) x. A(x) is similar to th otimal liar combiatio B i th ucoditioal momt cas, but ow w coditio vry radom variabl o x. Usig th otimal istrumts, th ucoditioal momt coditios ar E [A(x)u (w; )] = : Alyig th formula of th asymtotic variac for th MoM stimator, w hav th smiara- 22

23 mtric cicy boud for E A(x)@u (w; ) =@ E A(x)u (w; ) u (w; ) A(x) E A(x)@u (w; ) =@ = E G(x) (x) G(x) : I th liar rgrssio cas, G(x) = x, ad (x) = 2 (x), so th otimal istrumt is x= 2 (x), which corrsods to th gralizd last squars stimator, ad th smiaramtric cicy boud for is E xx = 2 (x). Th otimal istrumts ivolv th coditioal ma stimatio. This will us oaramtric stimatio tchiqus which ar ot covrd by this cours; s Nwy (99b) for such stimatios. I ractic, w may oly wat to slct a grou of istrumts that d ot b (asymtotically) otimal. But giv a i it list of ottial istrumts, which should b usd? This is sstially a modl slctio roblm, ad will b bri y discussd i Sctio 8.2. (*) Kitamura t al. (24) study th mirical liklihood-basd ifrc i coditioal momt rstrictios modls. Adrws ad Shi (23) study ifrc basd o coditioal momt iqualitis. 8 Momt Slctio (*) If th J tst rjcts th ull, w susct thr ar som momt coditios which ar ivalid. Thus, it may b usful to mloy a momt slctio rocdur to stimat which momts ar corrct ad which ar icorrct. O th othr had, as mtiod i Sctio 7, w may d to slct momts amog may valid os to imrov it-saml ifrc. W us Adrws (999) ad Doald ad Nwy (2) to xmlify ths two scarios. 8. Adrws (999) Adrws (999) dvlos aralll iformatio critria (IC) as i th last squars viromt; h labls ths critria by addig th r x GMM. H shows that th GMM-vrsio IC is idd aalogu of th usual IC. To s why, ot that for a momt slctio vctor c (which is a vctor of ad with idicatig that th corrsodig momt quatio is icludd), is th sam as (with a o () di rc) 9 J(c) = if (g () D c ) W (g () D c ) (4) 2;2R l k J (c) = if 2 (g c()) W (c) (g c ()) ; 9 Actually, by th rsults of Back ad Brow (993), th miimizrs i ths two miimizatio roblms ar xactly th sam. 23

24 whr D c sts th momt coditios with zros i c qual to ( 2 R l k corrsods to th just-idti d modl, or th smallst modl allowd), g c icluds th lmts of g with os i c, ad W ad W (c) ar costructd i th sam fashio as i Sctio 3. Th umbr of aramtrs with slctio vctor c is k + l jcj, whr jcj is th umbr of momt coditios slctd by c, ad l jcj is th umbr of xcludd momt coditios. Rwrit k + l jcj as l q c, whr q c = jcj k is th umbr of "ovr-idtifyig rstrictios". All aramtrs i J ar ( ; ) 2 R l. With th slctio aramtr c, th last q c aramtrs ar st as zro. Thus, di rt slctio vctors corrsod to th sttig of di rt aramtrs qual to zro i (4), just as di rt modls corrsod to th sttig of di rt aramtrs qual to zro i th liklihood viromt of Sctio 4 i Chatr 6. Actually, J(c) d (= J (c) + o ()! 2 qc udr "corrct" slctio) lays th rol of 2 [`(M 2 ) `(M )], ad q c lays th rol of k 2 thr. Th gral momt slctio critria (MSC) is sci d as MSC (c) = J (c) h(jcj) ; whr c 2 C is a momt slctio vctor, C is th slctio st which may ot iclud all ossibl slctio vctors, h() is strictly icrasig,! ad = o(). Tak h(x) = x th umbr of aramtrs; th GMM-BIC: = l ad MSC BIC; (c) = J (c) (jcj k) l ; GMM-AIC: = 2 ad MSC AIC; (c) = J (c) 2 (jcj k) ; k, whr k is GMM-HQIC: = Q l l for som Q > 2 ad MSC HQIC; (c) = J (c) Q (jcj k) l l : Th MSC stimator, bc MSC, is th miimizr of MSC (c). It is show that bc MSC is cosistt i th ss that bc MSC! c, whr c is th st of momts whos xctatio is zro for som 2 ad whos umbr is largst amog all such sts of momts. W assum c 2 C ad is uiqu; othrwis, bc MSC will covrg to a st that icluds all ossibl slctio vctors which maximiz th umbr of valid momts. bc MSC dtrmis wh thr ar o ovr-idti catio rstrictios. I GMM-AIC, = 2 9, so th GMM-AIC rocdur is ot cosistt. It has ositiv robability v asymtotically of slctig too fw momts. A siml mthod ca b usd to dtct whthr a MSC is rliabl. I cass whr a MSC rforms oorly, thr ar tyically two or mor slctio vctors that yild MSC valus clos to th miimum ad that yild aramtr stimats di rig oticably from ach othr. I cass whr a momt slctio rocdur rforms wll, th lattr tyically dos ot occur. Adrws also cosidrs two tstig rocdurs to slct corrct momt coditios: dowward tstig (DT) ad uward tstig (UT) rocdurs. Ths rocdurs ar similar to iformal mthods basd o th J tst oft mloyd by mirical rsarchrs to dtrmi which momts to us. I th DT rocdur, w start with c 2 C for which jcj is th largst, ad th tst with rogrssivly smallr jcj util th ull caot b rjctd. Lt b k DT b this valu of jcj; bc DT is th slctio vctor that miimizs J (c) ovr c 2 C with jcj = b k DT. If th critical valus usd i th 24

25 J tst divrg to i ity at a slowr rat tha, th bc DT is cosistt. Th UT rocdur is a covrs rocdur of th DT rocdur ad is cosistt. Mot Carlo rsults show that GMM-BIC, DT, ad UT rocdurs rform bst ad about qually wll, th GMM-HQIC rocdur is xt bst ad th GMM-AIC rocdur is worst ovrall. Adrws ad Lu (2) xtd th rsults to covr simultaous momt ad modl slctio; thy aly thir rocdurs to dyamic al modls. Hog t al. (23) us GEL-statistics to rovid a altrativ itrrtatio of Adrws MSCs. Hall t al. (27) roos a troybasd momt slctio rocdur. DiTraglia (23) xtds th focusd iformatio critrio (FIC) of Clasks ad Hjort (23) to FMSC ad shows that th us of a ivalid but highly rlvat istrumt ca substatially imrov ifrc i it samls. Car (29) studis th LASSO-ty GMM stimator ad Liao (23) xtds to th gral shrikag stimator. Kurstir ad Okui (2) xtd th modl avragig to th 2SLS stimatio. 8.2 Doald ad Nwy (2) Di rt from Adrws (999) who is sarchig for th largst st of valid istrumts, Doald ad Nwy (2) roos a siml mthod to choos amog valid istrumts, by miimizig aroximat MSE. Thir mhasis is o MSE aroximatio wh th istrumts may b wak ad th umbr of istrumts may b larg. Th modl is y i = z i + x 2i 2 + u i = x i + u i ; E[u i jz i ] = ;!! z i z i x i = = f(z i ) + v i = + E [x 2i jz i ] x 2i v 2i! ; whr z i ca b a fw cotiuous variabls, may dummy variabls or v b i it dimsioal, ad (u i ; v2i ) is homoskdastic (s Doald, Imbs ad Nwy (29) for xtsios). Lt K i = K (z i ) = ( K (z i ); ; KK (z i )) K b a vctor of istrumts, whr i icluds z i. Bcaus E[u i jz i ] =, E K i u i =. Di rt valus of K corrsod to di rt istrumt sts. Usually, w scify such that th arlist trms hav th biggst imact o th rducd form. Giv that K i K i dds o K, th istrumt sts d ot form a std squc. K caot b too larg to avoid too variabl stimator du to th slctio rocss. D K K K, = ; ; ad P K = K K K K, whr A dots a gralizd ivrs. y;x 2 ;Z ;X ar d d by stackig th corrsodig vctors. b is th miimum of (y X) P K (y X)=(y X) (y X), ad = (K l 2)=. Th stimators cosidrd A gralizd ivrs of A satis s AA A = A. 25

26 ar 2SLS: b = (X P K X) X P K y; LIML: b = (X P K X X b X) X P K y X b y ; B2SLS: b = (X P K X X X) X P K y X y ; whr B2SLS is a bias adjustd vrsio of 2SLS. Th istrumt slctio is basd o miimizig th aroximat MSE of a liar combiatio b b of th IV stimator, whr b is som vctor of stimatd liar combiatio co cits. Estimatig th MSE rquirs rlimiary stimats of som of th aramtrs of th modl ad a goodss of t critrio for stimatio of th ( rst stag) rducd form usig th istrumt K i. Th rlimiary stimator ca b ithr a IV stimator with oly as may istrumts as right-had sid variabls or a IV stimator whr istrumts ar chos to miimiz o of th rst stag goodss of t critria blow. Not that th rlimiary stimator dos ot dd o K. Giv som rlimiary stimator of, say, d b 2 u = u u=, b 2 = v v =, b u = v u=; whr u = y X, v = V H, V = (I P K )X, ad H = X P K X= for som xd K is a stimator of f f= with f = [f(z ); ; f(z )]. As to goodss of t critrio, cross-validatio ad Mallows (973) rducd form goodss of t critria ar cosidrd. Th cross-validatio critrio is Th Mallows critrio is br cv (K) = br m X i= (K) = bvk bvk bv i K 2 P K 2 : ii + b 2 2K ; whr bv K = b V K H b with b V K = (I P K )X, ad A ij dots th i; jth lmt of a matrix A. Giv ths raratios, th aroximat MSE of th stimators ar 2SLS: LIML: B2SLS: S b (K) = b 2 K 2 u S b (K) = b 2 u br (K) b S (K) = b 2 u + b2 u br (K) b 2 u b 2 u br (K) + b2 u b 2 u K ; K ; b 2 K ; whr th aroach to calculatig th aroximat MSE is similar to Nagar s (959). 2SLS icluds a bias trm b 2 u K2, whil LIML ad B2SLS iclud oly variac trms. Th rst variac trm b 2 b ur (K) is commo ad coms from aroximatig th rducd form f(z) by liar combiatios 26

27 of K (z). Th scod variac trm coms from th FOCs of th thr stimators: 2SLS: X P K (y X b ) = ; LIML: X bub P K (y X b ) = ; b = X bu=bu bu; B2SLS: X P K (y X b ) (K l 2)X bu = ; whr bu = y X b for ach stimator b, bub i LIML limiats a imortat sourc of 2SLS bias arisig from th corrlatio btw X ad u, ad th bias corrctio for B2SLS subtracts a stimat of th bias from 2SLS FOCs. Th rst variac trm dcrass with K whil th scod icrass with K. For ach stimator, th b K that miimizs th corrsodig b S (K) will rsult i b b that has rlativly small MSE asymtotically. For 2SLS, b K accouts for a trad-o btw bias ad variac, whil for LIML ad B2SLS, b K accouts for a trad-o oly btw variac trms. Wh dim(x 2 ) =, th valu of K that miimizs ths critria will ot dd o b. Doald ad Nwy show that thir mthod ca imrov th it saml rortis of th thr IV stimators i th ss that S ( b K) mi K S (K) whr S () is th tru domiat trm of th xact MSE. Thy also comar th aroximat MSE of ths stimators, ad d th LIML is bst. Thir rsults also aly to th choic of oliar fuctios to us i th cit smiaramtric istrumtal variabls stimator of Nwy (99b). I this cas istrumt choic is aalogous to! ; choosig th smoothig aramtr i smiaramtric stimatio. Exrcis 3 (Emirical) Cotiu th mirical xrcis i th last chatr. (d) R-stimat th modl by cit GMM. I suggst that you us th 2SLS stimats as th rst-st to gt th wight matrix, ad th calculat th GMM stimator from this wight matrix without furthr itratio. Rort th stimats ad stadard rrors. () Calculat ad rort th J statistic for ovridti catio. (f) Discuss your digs. Tchical Adix A: Asymtotics for th Noliar GMM Th followig two thorms show that th GMM stimator is CAN. Thorm Suos that w i, i = ; 2;, ar i.i.d., W! W, ad (i) W ad WE[g(w; )] = oly if = ; (ii) 2, which is comact; (iii) g(w; ) is cotiuous at ach 2 with robability o; (iv) E[su 2 kg(w; )k] <. Th b!. Proof. W rov this thorm similarly as i th roof of th cosistcy of th MLE. Lt Q () b g () W g (); w d to vrify th four coditios at th d of Sctio 3. of Chatr 27

28 4. Coditio (I): Lt R b such that R R = W. If 6=, th 6= Wg() = R Rg() imlis Rg() 6= ad hc Q() = (Rg()]) (Rg()]) < Q( ) = for 6=, whr g() = E[g(w; )]. Coditio (II) is (ii) of th thorm. Giv (ii), (iii) ad (iv), Micky s Thorm imlis g() is cotiuous ad su 2 kg () g()k!. Thus coditio (III) holds by Q() = g() Wg() cotiuous. By comact, g() is boudd o, ad by th triagl ad Cauchy-Schwartz iqualitis, jq () Q ()j [g () g()] W [g () g()] + g() W + W [g () g()] + g() (W W) g() kg () g()k 2 kw k + 2 kg()k kg () g()k kw k + kg()k 2 kw Wk ; so that su 2 jq () Q ()j!, ad coditio (IV) holds. Thorm 2 Suos that th coditios i th abov thorm hold, W! W, ad (i) 2itrior of ; (ii) g(w; ) is cotiuously di rtiabl h i a ighborhood of N of, with robability aroachig o; (iii) E[g(w; )] = ad E kg(w; )k 2i < ; (iv) E[su 2N kr g(w; )k] <, whr r g(w; g(w; ) ; (v) G WG is osigular for G = E [r g(w; )]. Th for = E [g(w; )g(w; ) ], b d! N (; V), whr V = (G WG) G WWG (G WG). Proof. By (i), (ii) ad (iii), th FOC 2G b W g b = is satis d with robability aroachig o, whr G () = r g (). Exadig g ( b ) aroud, multilyig through by, ad solvig givs b = G b W G G b W g ( ) ; whr is th ma valu. By (iv), G b! G ad G! G, so that by (v), G b W G G b W! G WG G W: Th coclusio th follows by Slutsky s thorm. Th comlicatd asymtotic variac formula simli s to G G wh W =. As show i Has (982), this valu for W is otimal i th ss that it miimizs th asymtotic variac matrix of th GMM stimator. V ca b cosisttly stimatd by its saml aalog, bv = bg W b G bg W b W b G bg W b G ; whr b G = G b, ad b = P i= g(w i; b )g(w i ; b ). To rov th cosistcy of b V, w rst rov th followig lmma, which is Lmma 4.3 of Nwy ad McFadd (994). Lmma If w i is i.i.d., a(w; ) is cotiuous at with robability o, ad thr is a ighbor- 28

29 hood N of such that E [su 2N ka(w; )k] <, th for ay b E [a (w; )] :!, P i= a w i ; b! Proof. By cosistcy of b thr is! such that b with robability aroachig o. Lt (w) = su k k ka (w; ) a (w; )k. By cotiuity of a (w; ) at, (w)! with robability o, whil by th domiac coditio, for larg ough (w) 2 su 2N ka(w; )k. Th by th domiatd covrgc thorm, E [ (w)]!, so by th Markov iquality, 2 P P i= (w i ) > " E [ (w)] ="! for all " >, givig P i= (w i )!. By th LLN, P i= a (w i; )! E [a (w; )]. Also, with robability aroachig o, X a w i ; b i= X a (w i; ) X i= i= a w i ; b a (w i ; ) X (w i ) i=! ; so th coclusio follows by th triagl iquality. Th coditios i this lmma ar wakr tha thos of Micky s thorm, bcaus th coclusio is simly uiform covrgc at th tru aramtr. I articular, th fuctio is oly rquird to b cotiuous at th tru aramtr. Thorm 3 If th assumtios i th last thorm ar satis d, ad for ighborhood N of, E[su 2N kg(w; )k 2 ] <, th V b! V. Proof. Alyig th abov lmma to a(w; ) = g(w; )g(w; ), w gt b!, ad alyig to a(w; ) = r g(w; ), w gt G b! G. Th coclusio follows from th CMT ad cotiuity of matrix ivrsio ad multilicatio. Tchical Adix B: Liar Procdur of Widmijr (25) Cosidr oly th liar-i-aramtr modl. Suos th st-o stimator is b, ad W dds o b through Th st-two stimator satis s b 2 = W b = G W X g b i g b i : i= b G G W b g ( ) = G W ( ) G G W ( ) g ( ) +D b ;W ( ) + o ; Domiatd covrgc thorm: If X! X ad for ay, jxj Y with E [Y ] <, th E[X ]! E[X]. 2 Markov s iquality: For ay ogativ radom variabl X ad a >, P (X > a) E [X] =a. 29

30 whr G () =@ dos ot dd o, ad th jth colum of D ;W ( ) is giv by D ;W ( )[; j] = G W ( ) G " D ;W ( ) b G W ( j G W ( ) G G W ( ) g ( ) + G W ( ) G G W ( j # W ( ) G W ( ) g ( ) : = O ( ), so takig accout of this trm will rsult i a mor accurat aroximatio of th variac of b 2 i it samls. Not that wh th modl is just idti d, th corrctio disaars. A st-o liar stimator satis s b = G W G G W g ( ) ad th it saml corrctd stimat of th variac of b 2 ca b obtaid as dv ar b2 = G W + D b2 ;W ( b ) b G + Db2 ;W ( b d ) V ar b D 2 b ;W ( b ) b G + G W G W b G D b2 ;W ( b ) whr th rst trm is th covtioal stimat of th asymtotic variac; th rst trm of D b2 ;W ( b ) G is zro sic W b G G W b g b2 = from th FOCs, so ad D b2 ;W ( b ) = G W b G G W dv ar b = G W G G W dv ar b2 will rovid a bttr it saml stimat of V ar saml variatio of b j W b g b2 b W b W G G W ; G : b2 by takig ito accout th it 3