Improved Jive Estimators for Overidenti ed Linear Models with and without Heteroskedasticity

Transcription

1 Improved Jive Estimators for Overidenti ed Linear Models with and withot Heteroskedasticity Daniel A. Ackerberg and Pal J. Deverex` Febrary 11, 2003 Abstract This paper examines and extends work on estimators for overidenti edlinear instrmentalvariables models. In the rst part of the paper, we introdce two simple new variants of the JIVE estimator proposed by Phillips and Hale (1977, Blomqist and Dahlberg (1994,1999, and Angrist, Imbens, and Kreger (1995,1999. We show that or new estimators are sperior to the existing JIVE estimator, signi cantly redcing its small sample bias in many sitations. Or second contribtion is to compare or new estimators to existing agar (1959 type estimators when there is heteroskedasticity in the model. We show that when there is heteroskedasticity, or estimators have sperior properties to both the agar estimator and the related B2SLS estimator sggested in Donald and ewey (2001. Or theoretical reslts are veri ed in a set of Monte-Carlo experiments, as well as in an application to the retrns of schooling sing the data of Angrist and Kreger (1991. `Dept. of Economics, UCLA, Los Angeles, CA Thanks to Jin Hahn for helpfl discssions. All errors are or own. 1

2 1 Introdction This paper examines and extends recent work on estimators for overidenti ed linear instrmental variables models. In this case, over tting in the rst stage can generate large small sample bias in the standard Two Stage Least Sqares (TSLS estimator. The paper is composed of two main parts. In the rst part we start with the Jackknife Instrmental Variables (JIVE estimator, proposed in Phillips and Hale (1977, Angrist, Imbens and Kreger (1995, 1999 and Blomqist and Dahlberg (1994, The JIVE estimator was developed to address this over tting problem. In short, the JIVE estimator forms the predicted vale for observation i by rnning a rst stage regresssion on all observations other than i. This eliminates the over tting of predicted vales in TSLS, and reslts in a estimator whose small sample bias does not depend on the degree of overidenti cation. On the other hand, the JIVE procedre does not completely remove small sample bias. The remaining small sample bias depends on the nmber of inclded exogenos variables in the second stage eqation. Given the nmber of covariates often sed in empirical analysis, this nmber can be qite large, e.g. in Angrist and Kreger s (1991 stdy of the retrns to edcation, there are p to 60 sch variables. We sggest two very simple bt signi cant improvements to the JIVE estimator that eliminate this bias term, redcing both its small sample bias and variability. We call these new estimators the Improved JIVE (IJIVE estimator and the Unbiased IJIVE (UIJIVE estimator. In the second part of the paper we compare or IJIVE and UIJIVE estimators to agar s (1959 bias corrected estimator for overidenti ed models. agar s estimator has recently been investigated by Hahn and Hasman (2002 and Donald and ewey (2001. Interestingly, we show the IJIVE and agar estimators are similar in spirit and have very similar properties nder the typical assmption in this literatre that the errors are homoskedastic. However, we show that the IJIVE estimator has sperior properties to agar s estimator when the residals are heteroskedastic. In particlar, the IJIVE estimator (as well as the UIJIVE estimator is grop-asymptotically consistent, while the agar estimator (as well as Donald and ewey s related B2SLS estimator is not. We provide two sets of Monte-Carlo experiments that demonstrate these reslts. In the rst set of experiments we show that or IJIVE and UIJIVE estimators clearly dominate the JIVE estimator, particlarly when there are many covariates in the system. In the second set of experiments, we compare or IJIVE and UIJIVE estimators to the agar and B2SLS estimators. As expected, we nd that the IJIVE and UIJIVE estimators are sperior in the presence of heteroskedasticity. 1 We conclde by applying or new estimators to the Angrist and Kreger (1991 edcation data, nding reasonably large differences between or estimators and the standard JIVE estimator. 2 The JIVE Estimator Consider the following simltaneos eqations model: Y Ç X `;` Å W < ` Å > i (1 X ` Ç Z `H ` Å W =` i 1 In Appendix 2, we relate the JIVE estimator to the Split Sample Instrmental Variable estimators of Angrist and Kreger (1995, and propose some improvements to these estimators that are similar to or IJIVE and UIJIVE estimator. 2

3 The endogenos variable Y is an by 1 vector, X ` is an by L 1 matrix of endogenos explanatory variables, W is an by L 2 matrix of exogenos variables, and Z ` is an by K 1 matrix of exogenos instrments that are exclded from the main eqation. ;` is L 1 by 1, < ` is L 2 by 1, H` is K 1 by L 1,and=` is L 2 by L 1. We assme K 1 o L 1. Let L Ç L 1 Å L 2 and K Ç K 1 Å L 2. The nmber of overidentifying restrictions in this model is K 1 L 1 Ç K L. Å Æ ;` De ne: the by L matrix X Ç [X ` W], the by K matrix Z Ç [Z ` W], ; Ç,and H Ç < Å ` H ` 0 K1 xl 2 Æ. We can now write or model as: =` I L2 Y Ç X; Å > X Ç Z H ote that in this formlation, ; is an L vector, H is a K by L matrix, > is an vector, is an by L matrix. We assme > are independent across i and mean independent of W and Z. We also assme initially that > are homoskedastic with L Å 1byL Å 1 variance matrix >@. This homoskedasticity assmption is relaxed in section 3. We denote the probability limits of Z Z and X X as z and x respectively. For discssion prposes we sometimes consider the special case where Z ` is a set of mtally exclsive and exhastive dmmy variables (and when W are spersets of Z `. We refer to this as the groping case, as Z ` de nes a set of grop dmmies that are to be sed as instrments. 2 In either case, the OLS estimator for ; is ; OLS Ç X X 1 X Y (2 ; OLS is clearly a biased and inconsistent estimator of ; if there is any endogeneity problem, i.e. if >@ has any non-zero covariances between > De ne P z Ç ZZ Z 1 Z. The 2SLS estimator is ; 2SLS Ç X P z X 1 X P z Y (3 While ; 2SLS is consistent as goes to in nity, it is now well known (see Sargan (1958, agar (1959, Phillips and Hale (1977, Bond, Jaeger, Baker (1995, Staiger and Stock (1997, and others that it has poor nite sample properties when there are many instrments Z ` relative to the dimension of X `. The basic intition is that with many instrments, the rst stage regression tends to over t and the predicted vales X P z still contain some of the endogeneity in X `. As sch, ; 2SLS is typically biased towards the inconsistent OLS estimator. A rst order 2 An example of this groping case is a synthetic cohort model estimated sing repeated cross-sectional data. In this set-p, the instrments are cohort by year indicators and the control variables are typically cohort indicators and year indicators. 3

4 approximation to this bias (to order 1 given in Angrist, Imbens, and Kreger (1995 is: K L 1H z H 1 J >@ (4 where J >@ is an L vector of the covariances between > andeachofthel elements 3 In addition to being biased in small samples, ; 2SLS is not grop asymptotically consistent in the sense of Angrist and Kreger (1995. This version of grop asymptotics (see also the asymptotics in Bekker (1994 involves a sampling scheme where, as increases, the nmber of instrments increase sch that K is constant. These new instrments are ncorrelated with X, i.e. asymptotically they have no explanatory power in the rst stage. Given these nsatisfying properties of ; 2SLS, a nmber of alternative approaches have been proposed in the literatre. These inclde the bias corrected estimator of agar (1959 (see also Hahn and Hasman (2002 and Donald and ewey (2001, the limited information maximm likelihood (LIML estimator (see e.g. Anderson and Rbin (1949, 1950, and a jacknife bias corrected 2SLS estimator (Hahn, Hasman, and Krsteiner (2002. We retrn to these other estimators later in the paper. For the moment, we focs on the jackknife IV (JIVE estimator as a soltion to the over tting problem. Phillips and Hale (1977, henceforth PH, Angrist, Imbens, and Kreger (1995, 1999, henceforth AIK, and Blomqist and Dahlberg s (1994,1999, henceforth BD JIVE estimator works as follows. Let Z i and X i denote matrices eqal to Z and X with the ith row removed. Consider the following estimate of H for observation i: Hi Ç Z i Z i 1 Z i X i (5 De nee X JIVE to be the x L dimensional matrix with ith row Z i Hi The JIVE estimator is 4 : ; JIVE Ç E X JIVE X E 1 X JIVE Y (6 ote the intition behind the JIVE estimator: In forming the predicted vale of X for observation i, one ses a H coef cient estimated on all observations other than i. This eliminates the over tting problems in the rst stage. PH and AIK show that the small sample bias (to order 1 of this estimator is approximately eqal to: L 1H z H 1 J >@ (7 and AIK also show that JIVE is grop asymptotically consistent. Comparing the respective bias formlas, we see that nlike ; 2SLS, the approximate bias of the JIVE estimator does not increase with the nmber of overidentifying restrictions. It does, however, increase in the nmber of explanatory variables in the second stage eqation. ote lastly that the JIVE estimator can be written in very compact form that doesn t reqire iterating over observations 3 ote that since W is in both Z and X, ThelastL 2 of s are identically zero, and ths the last L 2 of the L covariances in J >@ are identically zero. 4 Angrist, Imbens, and Kreger also sggest a alternative version of the JIVE estimator, with Hi Ç Z Z 1 Z i X i 1 As all evidence there sggests that these two JIVE estimators have similar properties, we focs on the version in the text (eqs.(5 and (6. 4

5 to compte. Following PH and de ning: P z Ç Z Z Z 1 Z D Pz Ç diagp z C JIVE Ç I D Pz 1 P z D Pz we can write the JIVE estimator as: ; JIVE Ç X C JIVE X1 X C JIVE Y (8 ote that D Pz is a diagonal matrix with diagonal elements eqal to the diagonal of P z. 2.1 Intition for the Small Sample Bias of JIVE We next investigate the sorce of the small sample bias of the JIVE estimator and se the reslts to constrct two simple variants of the estimator that shold perform better in small samples. Recall that the small sample approximate bias of the JIVE estimator is: L 1H z H 1 J >@ Ç L 1 L 2 1H z H 1 J >@ (9 We focs on the L 2 term in this bias formla. As L 1 is the nmber of inclded endogenos variables in the second stage eqation and L 2 is the nmber of inclded exogenos variables (inclding the constant term in the second stage eqation, in most applications the L 2 term will be the primary sorce of small small bias in the JIVE estimator. We now investigate where this L 2 term comes from, rst intitively and then formally. The JIVE estimator (at least implicitly involves rnning rst stage regressions: For each observation i,the 1 other observations are sed in the rst stage regression. These rst stage regressions are rn withot any cross regression restrictions so that the coef cients on all variables in the rst stage (both Z ` and W are free to vary across the regressions. The W variables are also inclded in the second stage regression. In contrast to the rst stage, where the coef cients on W differ across observations, in the second stage regression the coef cients on W are restricted to be the same across all observations. Ths, in some sense, the JIVE process implicitly creates additional instrmental variables by inclding interactions of W with observation indicators in the rst stage, bt exclding these interactions in the second stage. These extra instrments add variance to the predicted vales Xi E JIVE and end p adding bias to ; JIVE. The extent of this bias is increasing in the dimension of W,i.e. L 2. This intition is even clearer in the groping case, where grop dmmies are sed as instrments. Sppose all the observations in the data are in the same grop. It shold obviosly be fritless to se this grop dmmy as an instrment as it is colinear with the constant term. Indeed, ; 2SLS is jsti ably not identi ed. However, ; JIVE does in fact generate an estimate. Why is this? ote that in the groping case, Xi E JIVE is eqal to the mean of X j (for j /Ç i in observation i s grop, i.e. Xi E JIVE Ç 1 /Çij +G i X j (where Gi is the nmber of observations in i s Gi 13 j 5

6 grop. In the case where there is only one grop, the instrment Xi E JIVE will vary across i, simply becase each average has a different observation left ot. As a reslt, the set of instrmentse X JIVE will not be colinear with the constant term and the second stage can be estimated. Obviosly, since this variation is completely sprios, ; JIVE is a meaningless estimate. Investigating the sorce of this bias more formally, consider an alternative, partialled ot representation of the second stage of the JIVE estimator. Recalling that X Ç [X` K W], note that Xi E JIVE is composed of ` EXi F JIVE Wi JIVEL wheref Wi JIVE Ç W i since W is itself inclded as a rst stage regessor. An alternative way to write the coef cients on the endogenos variables in the second stage is: ;` E JIVE Ç X ` JIVE M 1 * X ` E X ` JIVE M *Y (10 where M * is the orthogonal projection matrix I W W W 1 W. M * simply partials ot W. In this formlation, we can interpret X E` JIVE M * as instrments for the endogenos X `. ow let s consider whether these instrments behave as they shold, i.e. whether they are ncorrelated with the and >. By the JIVE rst stage constrction, there is no over tting and XìJIVE E is ncorrelated i (and ths > i. However, in a nite sample,e XìJIVE will be correlated j (and ths > j for j /Ç i. This is becase the residals of the other observations j help determine H `i throgh the rst stage regression coef cients. As a reslt of this correlation, the projection ofe X `JIVE on the sbspace orthogonal to W,i.e. M *E X `JIVE ends p being correlated (in a nite sample (and >. oting thate X ` JIVE Ç X ` C JIVE, the extent of this correlation can be compted as 5 : d E X E` JIVE *>e M Ç E X ` C JIVE *>e M (11 Ç E d Z `H ` Å W =` C JIVE *>e M Ç C JIVE *>e M Ç J >@ TraceC JIVE M * ÇJ >@ L 2 It is this correlation between the instrments and errors that generates the L 2 term in the small sample bias of JIVE. 5 oting that Z contains W, the last line follows from TraceC JIVE M * Ç TraceC JIVE TraceC JIVE P * Ç TraceC JIVE P * Ç TraceP z D Pz I D Pz 1 P * Ç TraceP * P z D Pz I D Pz 1 Ç TraceP * I D Pz I D Pz 1 Ç TraceP * ÇL 2 6

7 2.2 The Improved JIVE (IJIVE Estimator The discssion in the last sbsection sggests some simple adjstments to the JIVE estimator to eliminate the L 2 term in its small sample bias. As noted above, there is an asymmetry in how the variables in W (inclding the constant term are treated in this estimator. Coef cients on W are allowed to vary across the observations in the different rst-stage regressions, bt held xed across observations in the second-stage regression. This observation sggests two soltions: 1 to restrict coef cients on W in both rst and second stages, and 2 to allow coef cients on W to be Àexible in both the rst and second stages. Soltion 1 is easy to apply to the JIVE estimator. One can simply partial ot the W (inclding the constant term from Y X and Z before implementing the JIVE estimator. 6 Since there are now no W variables in the model, there is no problem arising from the coef cients on these variables differing across sample splits in the rst stage. We denote the partialled ot JIVE estimator as the IJIVE (Improved JIVE estimator. More precisely, with HY H X and H Z now representing variables partialled ot with respect to W (e.g. H Z Ç M* Z, wehave where andi PZ ÇH ;` IJIVE Ç H X C IJIVEH X 1 H X C IJIVEH Y (12 Z H Z H Z 1H Z. In Appendix 1, we show that ;` IJIVE has a small sample bias of7 C IJIVE Ç I DH PZ 1 I PZ DH PZ (13 L 1 1H H Z H 1 J >@ (14 This bias compares very favorably to the JIVE bias proportional to L 1, wherel is the total nmber of explanatory variables (endogenos+exogenos+constant term in the second stage. 8 ote that for the median application, L 1 Ç 1, where L is at least 2 (constant term pls endogenos variable and often mch larger (e.g. in Angrist and Kreger s (1991 edcation data, L 1 Ç 1andL Ç 61 in speci cations with year and state controls. While small sample variability is hard to explicitly calclate, or monte carlo reslts sggest that ;` IJIVE has lower variance than ;` JIVE, making it even more attractive. ote also that in or sprios identi cationgropingexample with only 1 grop, the IJIVE estimator is appropriately not identi ed. Soltion 2 is actally impossible to implement with the JIVE estimator. With JIVE one cannot allow the coef cients on W in second stage regressions to be different for each sample split as there is only one observation in each sample split. However, it is implementable for more general stacked split sample approaches, that we 6 Interestingly, the recent literatre on the agar estimator (see below has typically worked nder the assmption that W s have been partialled ot. In contrast, in the JIVE literatre, this has not been done. PH, AIK, and BD all consider exogenos W s (at least a constant term bt do not partial these terms ot. 7 ote that becase of the partialling ot of W, thisbiasvectorisl 1 dimensional J >@ is now the vector of correlations between > and only the rst L 1 elements ote also that the rst L 1 by L 1 block of H Z H 1 is identical to H H Z H 1. This, combined with the fact that the last L 2 elements of J >@ in the JIVE bias formla are zero, implies that the rst L 1 elements of the H Z H1 J >@ component in the JIVE bias are eqal to the H H Z H 1 J >@ component in the IJIVE bias. 8 ote that we cold de ne an alternative form of asymptotics, where L 1 increases with. The IJIVE estimator wold be consistent in this form of asymtotics, while the JIVE estimator wold not be. 7

8 discss in Appendix 2. Before contining, we note one frther adaption of the JIVEestimator thatimprovesits nite sample properties. The partialling ot of W removed the L 2 term from the bias. We now address the L 1 1term.De ne C UIJIVE Ç I DH PZ Å ÃI 1 I where Ã Ç L 1Å1. In the appendix, we show that PZ DH PZ Å ÃI ;` UIJIVE Ç H X C UIJIVEH X 1 H X C UIJIVEH Y is approximately nbiased (to order 1. Intitively, the additional ÃI terms change the trace of C, sbtracting ot the bias in the IJIVE estimator 9. In or Monte Carlo reslts, we also note that ;` UIJIVE tends to have lower dispersion than does ;` IJIVE. 2.3 The IJIVE Estimator and agar s Estimator We now show how the IJIVE estimator relates to agar s (1959 bias-corrected estimator. The agar estimator can be written as: where and where D Ç K 1 K 1 andj MZ Ç I I PZ ;` H agar Ç X C agarh 1 X X H C agarh Y (15 C agar ÇI DJ MZ (16 PZ. Maniplating gives: 1 C agar Ç 1 E D I PZ E DI (17 wheree D Ç K1. ThisE D formlation of the agar estimator is convenient in that it allows easy comparison to the IJIVE estimator. ote the similarities between (13 and (17. Since the trace ofi PZ is eqal to K 1, the average vale of the diagonal elements ofi PZ is K 1. As sch, the agar and IJIVE estimators differ only in that in IJIVE the actal diagonal elements of I PZ are sbtracted from I PZ while in agar, the average vale of the diagonal elements of I PZ is sbtracted fromi PZ. Likewise, in the denominator, the IJIVE estimator sbtracts the actal vale of the diagonal of I PZ while agar sbtracts the average vale of the diagonal elements ofi PZ In the special case where the diagonal elements ofi PZ are constant, the IJIVE and agar estimator are identical. For example, in the groping case when there are eqal nmbers of observations in each grop, the two estimators are identical since each diagonal term of I PZ eqals K 1. While the bias of agar is well known, for completeness, Appendix 1 shows that nder 9 As we note below, this correction is similar to the correction sed by Donald and ewey (2001 on the agar estimator to get an approximately nbiased estimator. 8

9 homoskedasticity, the agar estimator has the same approximate small sample bias as IJIVE, i.e. L 1 1H H Z H 1 J >@ (18 Donald and ewey (2001 also sggest a variant of the agar estimator, which they term the B2SLS estimator. This estimator is identical to the agar estimator except that D is de ned as: D Ç K 1 L 1 1 K 1 Å L 1 Å 1 (19 K or alternatively,e D Ç 1 L 1 Å1. This adjstment is analagos to or adjstment from IJIVE to UIJIVE, and it redces the approximate small sample bias of the agar estimator to zero (see Appendix 1. 3 Heteroskedasticity While the IJIVE (UIJIVE and agar (B2SLS estimators have very similar properties nder homoskedasticity, they diverge nder heteroskedasticity. To or knowledge, the comparison of these estimators nder heteroskedasticity has not been made in the literatre. 10 The heteroskedasticity we consider is in the instrments Z, i.e. we allow arbitrary heteroskedasticity of > in Z. We do not allow heteroskedasticity in X. 11 Changing notation slightly, we now assme that the exogenos variables (W have already been partialled ot of the model so we have: Y Ç X; Å > (20 X Ç Z H (21 In this formlation, Y is an vector, X is an L 1 matrix, Z is an K 1 matrix, > is an vector, is an L 1 matrix. To keep notation simple, in this section we assme X is one dimensional, i.e. L 1 Ç 1. The reslts can easily be generalized. Small sample bias calclations like those above are dif clt nder heteroskedasticity. As a reslt, we trn to a different methodology for assessing bias. Following Hahn and Hasman (2002, we consider moving the expectation (conditional on Z of an estimator inside the ratio, i.e. EE ; Ç E dx C X 1 X C Y e s E d X C Y e E [X C X] This approximation also corresponds to the grop asymptotics approach previosly sed by Bekker (1994 and AIK. The basic idea of this approach is to let both and K 1 go to in nity (in a xed proportion. Ths, the approach is in some sense in between standard asymptotics and the small sample bias approximations of the prior 10 AIK s Monte-Carlo reslts brieày consider the JIVE estimator in a sitation with heteroskedasticity. However, they do not establish theoretical properties of the estimator nder heteroskedasticity, nor do they compare the JIVE estimator with agar type estimators. 11 Allowing heteroskedasticity in X wold open p an entire set of other isses addressed in the treatment effects literatre (e.g. Heckman and Robb (1985, Angrist and Imbens (1994. (22 9

10 sections. In contrast to standard asymptotics, terms of order K 1 are not dropped from the bias calclation -in contrast to the second order bias approximations, terms of order 1 are dropped. If this ratio of expectations is eqal to ;, we describe the estimator as grop asymptotically consistent. To start, note that X C agar Ç X P z DM z Ç Z H P z DM z Ç H Z C agar (23 and X C IJIVE Ç X P z D Pz I D Pz 1 (24 Ç Z H P z D Pz I D Pz 1 Ç Ç H Z I D Pz I D Pz 1 C IJIVE H Z C IJIVE Ths, with C representing either C agar or C IJIVE, the nmerator of (22 is: EX C Y Ç EH Z Z H; Å EH Å E@ C Z H; Å E@ Å E@ C > (25 Ç H Z Z H; Å E@ Å E@ C > (26 and the denominator is: EX C X Ç EH Z Z H Å EH Å E@ C Z H Å E@ (27 Ç H Z Z H Å E@ (28 since Z are mean independent. Therefore, the approximate expectation of the estimator is EE ; Ç ; Å E@ C > EH Z Z H Å E@ (29 and grop asymptotic consistency comes down to whether C > Ç 1 Etrace@ C > Ç 1 EtraceC >@ Ç 1 tracec E>@ Ç 0 (30 Under homoskedasticity, this trace is zero for both C IJIVE and C AGAR.ForC IJIVE, this is becase C IJIVE has a zero diagonal, and E>@ is a diagonal matrix de to the independence of observations. Ths, the diagonal of C E>@ is identically zero. For C AGAR, since the diagonal elements of E>@ all eqal J >@ : 1 tracec AGAR E>@ Ç 1 tracep z DM z E>@ (31 1 Ç J >@ vtracep K 1 z tracem z K 1 w 10

11 Ç Ç 0 1 J >@ w vk 1 K 1 K 1 K 1 Ths, nder homoskedasticity, both IJIVE and agar are grop asymptotically consistent. Things are different nder heteroskedasticity. While P >@ Ç E>@ is still diagonal, the elements on the diagonal are now generally neqal and fnctions of Z i. However, since C IJIVE has a zero diagonal, we still have In contrast 1 tracec IJIVE E>@ Ç 0 (32 1 tracec AGAR E>@ Ç 1 tracep z DM z P >@ (33 ; Ç 1 P zii K 1 M zii J > K i 1 v i ; Ç 1 i ; Ç 1 i P zii K 1 K 1 K 1 K 1 H Pzii J > i w J > i where J > i is the ith diagonal element of P >@, i.e. the covariance between > i i. P zii is the ith diagonal element of P z, Pzii H is P zii 3i 1 P zii, and the last line relies on the fact that 3i 1 P zii Ç TraceP z Ç K 1. This term is not generally zero, and is of order 1. The size of this term depends on how the covariance term is correlated with deviations from average P zii. Since both J > i and Pzii H are fnctions of Z i, we expect sch correlation generally. Ths, nlike the IJIVE estimator, the agar estimator is generally not grop asymptotically consistent with heteroskedasticity. 12 ote that the intition behind this reslt follows directly from the discssion in the prior section. Unlike IJIVE, which sbtracts off the exact diagonal elements of P z, the agar estimator sbtracts off the expectation of this diagonal. The problem for agar with heteroskedasticity is that the difference between the expectation and realized vale will generally be correlated with J > i. ote that in one special case, this bias term disappears when the diagonal of P z is constant (i.e. Pzii H Ç 0 1i. This special case occrs, for example, in the groping case when there are eqal nmbers of observations in each grop (in this case, P zii Ç K 1 1i. This reslt shold not be srprising, given that the IJIVE and agar estimators are identical in the groping case with eqal nmbers of observations. Lastly, consider or UIJIVE estimator and Donald and ewey s B2SLS estimator nder heteroskedasticity. 12 ote that there is an isse of what happens to this bias term as K 1 and go to in nity. This depends on what happens to P H zii as K 1, *. In some cases, P H zii is of constant order as K 1, *., e.g. in the groping case with neqal grop sizes. In other cases, P H zii 0atrate T K 1 as K 1, *., e.g. if Z is composed of i.i.d. vectors. Ths what happens to the bias term depends on what happens to the heteroskedasticity as K 1, *.. For example, if the amont of heteroskedasticity remains constant, this term wold either not disappear (in the groping case or disappear at rate T K 1 (when Z k s are i.i.d.. Even thogh the estimator is formally grop asymtotically consistent in the T K 1 case, this is a slower convergence rate than the other terms dropped in the expansion, and we might expect it to case signi cant small sample biases. This is veri ed in or Monte-Carlos. 11

12 For the B2SLS estimator, we get 1 tracec B2SLS E>@ Ç 1 tracep z K 1 L 1 Å 1 K 1 Å L 1 Å M zp >@ (34 1 ; Ç 1 P zii K 1 L 1 Å 1 K 1 Å L 1 Å M ziij > i 1 v v i ; Ç 1 i ; Ç 1 i K 1 Å L P zii K 1 L 1 Å 1 1 Å 1 K 1 Å L 1 Å 1 K 1 Å L 1 H L 1 Å 1 Å 1 Pzii Å K 1 Å L 1 Å 1 w J > i While the second term in this expression is of order 1 and disappears nder grop asymptotics, the rsttermis of order 1 and does not. This rst term is the analoge of the expression for agar (eqation (33 and will also case small sample bias. 1 Unlike C IJIVE, the diagonal of C UIJIVE is not identically zero. Ths, tracec UIJIVE E>@ /Ç 0. However, the UIJIVE estimator is still approximately nbiased as K and go to in nity with K constant. Examining this trace we have: 1 tracec UIJIVE E>@ Ç 1 tracep z D Pz Å L 1 Å 1 I I D P z Å L 1 Å 1 I 1 P >@ (35 ; Ç 1 L 1 Å1 i 1 Å L J > 1Å1 i P ; zii 1 L 1 Å 1 Ç J > Å L 1 Å 1 P i zii i To determine the order of these terms, expand the smmands arond the mean of P zii, K 1. This gives: 1 ; v tracec UIJIVE E>@ s 1 L 1 Å 1 L 1 Pzii w Å 1 Å L 1 Å 1 K 1 2 Å J > i (36 i Å L 1 Å 1 K 1 H These terms (as well as the omitted terms in the expansion are all of order 1. As sch, the UIJIVE estimator is approximately nbiased in the grop asymptotic sense. In smmary, we have shown that JIVE type estimators have sperior properties to agar type estimators nder heteroskedasticity. These sperior properties will be very evident in or Monte-Carlo experiments. Comparing the IJIVE to the UIJIVE estimator, both are approximately nbiased, bt the UIJIVE estimator does have extra second order terms. This might sggest that the IJIVE estimator wold be more robst than the UIJIVE estimator nder heteroskedasticity. However, note that these extra terms in the UIJIVE estimator are aimed at cancelling ot second order terms that were already dropped in the initial approximation (eq. (22. As sch, we do not take a theory based stand on which might be preferable. However, we do examine the differences in or Monte-Carlo experiments. w J > i 12

13 4 Monte Carlo Analysis We perform two general sets of Monte Carlo analyses. The rst set compares or IJIVE and UIJIVE estimators to the JIVE estimator. We focs in particlar on what happens as the nmber of exogenos explanatory variables W increases, as this is the primary difference between the properties of or estimators and the existing JIVE estimator. As expected the IJIVE and UIJIVE estimators perform considerably better than JIVE. The second set of reslts compares the IJIVE and UIJIVE estimators to existing small sample instrmental variables estimators, inclding agar and B2SLS, as well as LIML and the Jackknife 2SLS (J2SLS estimator sggested by Hahn, Hasman, and Krsteiner (2002 (HHK. We focs on the possibility of heteroskedasticity, to illminate the theoretical differences between IJIVE (UIJIVE and agar (B2SLS. Again as expected, we nd that IJIVE and UIJIVE perform very well in comparison to the other estimators (in particlar with respect to agar and B2SLS, and arge that in many sitations, these shold be the estimators of choice. 4.1 Base Models: We se two base speci cations for or Monte Carlo analysis. The rst is a standard case where instrments are continos. The second is a groping case where the instrments are a set of mtally exclsive and exhastive dmmy variables. Base Model 1 - Continos Or base continos model is identical to the base overidenti ed model sed in AIK (Model 2 except for the addition of W s other than the constant term. We assme: X i Ç H 0 Å H 1 Z i Å H 2 W i i Y i Ç ; 0 Å ; 1 X i Å ; 2 W i Å > i where X i is one dimensional, Z i is 20 dimensional, and W i is L 2 1 dimensional. Both the Z i s and the W i s are distribted i.i.d. normal with mean zero and variance one. ; 0 Ç H 0 Ç 0, ; 1 Ç 1, all the ; 2 s and H 2 s are set eqal to 1, and as in AIK, the rst element of H 1 Ç 03 while the rest of the elements are zero. With or base nmber of observations, Ç 100, this is clearly a case where one wold worry abot over tting in the rst stage. We assme initially that the errors are homoskedastic, i.e.: > i Z i r ( i Base Model 2 -Groping Or base groping model is as follows. Sppose the data can be divided into a set of mtally exclsive and exhastive grops indexed by g. In addition, sppose that the data can also be divided into mtally exclsive and exhastive cohorts indexed by c that are spersets of these grops: X i Ç E g Å E c i Y i Ç ; 1 X i Å F c Å > i ; i 13

14 ote that there are cohort effects in both eqations. We set ; 1 Ç 1 and draw the grop and cohort effects E g E c F c independently from 0 1 distribtions. The error components, > i i are all independently distribted, 0 1 and 0 2 respectively. Endogeneity comes from the fact that the residal in the second stage depends i - as a reslt, the degree of endogeneity in this model is determined by the variance i. 13 The 2SLS estimator of ; 1 is attained by sing grop indicators as instrments for X i and an over tting problem arises when there are many sch grops. ote that this is very similar to Angrist and Kreger s (1991 retrns to edcation speci cations, where grops are qarter of birth interacted with state of birth. We initially choose Ç 100 and x the nmber of grops at 20 (ths there are 5 observations in each grop. otes on Experiments In Tables 1-5 and 7 we perform Monte-Carlo replications. In Tables 6 and 8 we perform 2000 replications. In all tables we report qantiles (10%, 25%, 50%, 75%, 90% of the distribtion of the estimator of ; 1 arond the tre ; 1. The 50% qantile is ths the median bias of the estimator. We also report the median absolte error of the estimator. Mean biases and mean sqared errors of or estimators are a bit more problematic. This is becase JIVE and agar type estimators are known not to have second moments. This makes their means extremely sensitive to otliers and makes mean sqared errors meaningless. To address this isse we trimmed the distribtions of all the estimators (at the 5th and 95th percentiles and report mean bias and mean absolte error for these trimmed distribtions. We also want to address the appropriateness of these estimators for hypothesis testing. One way to examine this is to look at con dence interval coverage rates. If, e.g., a nominal 90% con dence interval covers the tre parameter signi cantly less than 90% of the time, hypothesis tests will be nreliable. In or tables, we compte the coverage rate of tre con dence intervals. To illstrate, sppose we are performing monte-carlo replications of a model. We take the distribtion of these estimates and create a con dence interval for each of the individal estimates sing that distribtion. DenoteE ;005,E ;095,E ;05 as the 5th, 95th and 50th percentiles of this point distribtion. For an individal estimatee ;r, coverage is obtained if: E ;r E ;095 E ;05 ;E ;r Å E ;05 E ;005 We then report the proportion of the estimates for which coverage occrs. Clearly these con dence intervals are not feasible in practice, since we do not know the tre data generating process. However, if the coverage rates of these con dence intervals are off, one wold also expect any feasible 13 This model is directly related to the measrement error models that often arise in sitations with groped data (see, e.g., Deaton (1985, Deverex (2003. The basic setp in these models is that there is a postlated relationship between the expected X in a grop and the expected Y in the grop. X is not endogenos per-se, bt becase of sampling variation, the mean X in a grop is not eqal to the expected X, casing a measrement error based endogeneity problem. The 2SLS estimator instrments X i with grop dmmies. Ths, in the second stage, Y i is regressed on the within-grop sample mean of X i To see how this maps into or model, note that the model in the text can be written as: X i Ç E g Å E c i Y i Ç ; 0 Å ; 1 E g Å E c Å F c Å >i Here, the expectation of X i (i.e. E g Å E ciswhataffectsy i. Becase the sample mean of X i is sed instead of the expectation of X i in the second stage, the 2SLS bias can be interpreted as a measrement error problem. 14

15 con dence intervals (e.g. throgh asymptotic approximation or bootstrapping to also be off. For or IJIVE and UIJIVE estimators, we also tried in a variety of ways to constrct feasible con dence intervals that one cold se in practice. The rst approach was to se an asymptotic approximation, ignoring variance in the predicted vales X C IJIVE. AIK sed this approximation and fond that is worked qite well for the JIVE estimator withot heteroskedasticity. Given that many of or experiments inclde heteroskedasticity, we sed a White adjsted formla, i.e. VarE X1 ; IJIVE ; d Ç X C IJIVE i i Ç1Ee 2 X C IJIVEeid X C IJIVEe i X C IJIVE X 1 d whereee i are predicted residals given the IJIVE (or UIJIVE estimate and X C IJIVEei is the ith row of X C IJIVE. Thogh these are not reported in the tables, this approximation generated qite good coverage rates in all or models Experiment 1 In or rst set of experiments, we simply examine the performance of the varios JIVE estimators as the dimension of the exogenos variables increases. Table 1 does this for or continos model. The rst panel contains reslts when dim(w=0, i.e. when the only non-exclded exogenos variable is the constant term. The large median and mean biases of the OLS and 2SLS estimates sggest that there is both a signi cant endogeneity problem and asigni cant over tting problem. Con dence interval coverage is way off for these estimators. With only one non-exclded exogenos variable, we wold expect the JIVE estimator to do qite well at redcing bias, and it does. However, even with only the constant term, the IJIVE and UIJIVE estimators appear to perform better. Both the means and the medians of IJIVE are abot 0.04 closer to the trth than the JIVE and the UIJIVE estimator is approximately nbiased. The distribtions of the IJIVE and UIJIVE estimators are also slightly tighter than the JIVE estimator. Also note that con dence interval coverage for the IJIVE and UIJIVE estimators (as well as the JIVE estimator are reasonable. Moving throgh the panels of Table 1 corresponds to adding more exogenos variables to the system. The basic trend is that the JIVE s performance qickly deteriorates while the IJIVE and UIJIVE estimators contine to perform well. By the last panel, where dim(w is eqal to 10, the JIVE estimator has considerable bias - median bias is and the mean bias is This bias is even larger than that of 2SLS. In addition, the variance of the JIVE estimator increases tremendosly. In contrast, the IJIVE and UIJIVE estimators contine to perform well, 14 For the for panels in Table 1, the coverage rates of 90% con dence intervals generated by these asymptotic standard errors for the IJIVE estimator are 0.886, 0.878, 0.854, and For the 13 panels in Tables 3 throgh 5 (the models with heteroskedasticity, the coverage rates are 0.886, 0.903, 0.908, 0.870, 0.889, 0.893, 0.889, 0.894, 0.878, 0.886, 0.889, 0.891, respectively. 15 We also considered two bootstrapping approaches to contrcting feasible con dence intervals. In limited experiments, these also appeared to work reasonably well. The rst again ignores variance in the predicted vales. We simply bootstrap the second stage of the IJIVE/UIJIVE estimator, resampling from the empirical joint distribtion (C IJIVE X X Y and constrcting standard bootstrap con dence intervals. The second attempts to consider the variance de to rst stage estimation. ote that we cannot bootstrap the entire estimation process, as resampling with replacement will reslt in repeated observations in each bootstrapped sample. These repeat observations regenerate the over tting problem that the estimators attempt to avoid. As an alternative we tried bootstrapping each stage individally. In other words, we draw a set of observations from the sample with replacement. Then, to rn the rststageforeachofthe observations, we resampled (with replacement 1 observations fromremaining 1 observations. 15

16 both in terms of bias and variance. It is interesting to compare the IJIVE and UIJIVE estimators. The UIJIVE tends to perform a bit better as measred by Median and Mean Absolte Error. On the other hand, con dence interval coverage is a bit worse. These differences are small compared to the differences between JIVE and these two estimators thogh. ote that these speci cations only have one endogenos explanatory variable. Given or theoretical reslts, one might expect the relative performance of the UIJIVE estimator to increase as the nmber of endogenos variables increases. Also note that the small biases apparent in IJIVE and UIJIVE when dimw =10 are coming from high order terms - when one increases to, e.g., 200 the biases qickly disappear. Table 2 performs a similar set of experiments for the groping model. In the groping case, it is natral to increase the nmber of explanatory variables by increasing the nmber of cohorts. In the baseline case, all grops are assmed to belong to one cohort and the only exogenos explanatory variable is the constant. Moving throgh the panels, we then consider the case of two cohorts (the exogenos explanatory variables are a constant and a cohort indicator, 5 cohorts (exogenos explanatory variables are a constant and 4 cohort indicators, and 10 cohorts (exogenos explanatory variables are a constant and 9 cohort indicators. The reslts are similar to those in the continos model. The bias and variability of JIVE increases rapidly as the nmber of cohorts increases. On the other hand, the bias of IJIVE and UIJIVE remains small as the nmber of cohorts is increased and the variance increases by abot the same magnitde as those of OLS and 2SLS. Again, IJIVE and UIJIVE appear mch preferable to JIVE. 4.3 Experiment 2 In or second set of experiments, we x dim(w Ç 0 and consider adding heteroskedasticity to the model. Or basic goal is to compare the performance of the IJIVE and UIJIVE estimators to the agar and B2SLS estimators. For comparision prposes, we also consider two other estimators, the LIML estimator and the Jackknife 2SLS estimator of HHK. The LIML estimator can be conveniently written as: ; LIML Ç X C LIML X1 X C LIML Y (38 where C LIML Ç I DM z (39 and D is the smallest characteristic root of b [YX] P z [YX] cb [YX] M z [YX] c1. The LIML estimator has known optimality properties nder correct speci cation, bt this reqires i.i.d. normality and linearity. The Jackknife 2SLS estimator sggested by HHK performs a jackknife bias correction on the 2SLS estimator. This involves estimating the 2SLS model Å 1 times, once on the fll sample, and once on each sbsample of 1. Assming that the bias is linear in 1, a linear combination of the fll sample estimator and the average of the 1 sample estimators prodces an nbiased estimate. This is: ; J2SLS Ç ; 2SLS ; 1 1 n ; 2SLSn 16

17 where ; 2SLSn is the 2SLS estimate on the dataset withot observation n HHK show that relative to agar (and implicitly JIVE type estimators, ; J2SLS has considerably less variance, more bias, and lower mean sqared error. The choice between these two types of estimators will typically come down to whether one is interested in minimizing mean sqared error, or minimizing bias. To add heteroskedasticity to or continos model, we do the following. Consider i and > i i Ç K 1 F 1 Å K 2 F 2 (40 > i Ç K 1 F 1 Å K 3 F 3 where F 1, F 2,andF 3 are independent standard normals. F 1 is the common component of the error terms, generating their correlation, while F 2 and F 3 are idiosyncratic components. The exact variance covariance matrix of eqation (37 can be generated by setting K 1 Ç 0447 and K 2 Ç K 3 Ç With this formlation, we can introdce heteroskedasticity into three different parts of the model - the common error term or either of the idiosyncratic error terms. To add heteroskedasticity, we rst form: h i Ç; k and standardize this measre by its expectation and standard deviation. Then, to generate heteroskedasticity, e.g. in F 1 we form F 1 Ç expah i ` M where M is a standard normal. We lastly normalize the distribtion of F 1 to have variance 1 to preserve the original variances of eqation (37. The parameter A measres both the extent and shape of the heteroskedasticity. A Ç 0 implies no heteroskedasticity, A 0 implies positive heteroskedasticity with respect to h i,anda 0implies negative heteroskedasticity. 16 Tables 3 throgh 5 present reslts from these heteroskedastic models. The 3 tables add heteroskedasticity to F 1, F 2,andF 3 respectively. The most interesting reslts are in Table 3, with heteroskedasticity in the common error component. In the rst panel, reslts for the model withot heteroskedasticity are presented. Of interest here is how agar, B2SLS, J2SLS, and LIML perform compared to IJIVE and UIJIVE. As expected withot heteroskedasticity, the agar and B2SLS estimators look almost identical to their IJIVE and UIJIVE conterparts. J2SLS, as expected, has considerably lower spread and lower mean and median absolte error, bt worse bias and con dence interval coverage. LIML performs qite well, with almost no bias and the lowest mean and median absolte errors. The next 4 panels in the table present varios heteroskedastic speci cations. For A Ç5, we see the performance of agar and B2SLS deteriorate badly, with median biases of and and mean biases of and The variances of these estimators also increase sbstantially. On the other hand, becase of this increased variance, the con dence interval coverage remains ne. In contrast, IJIVE and UIJIVE are essentially naffected. 16 The reason changing A changes the shape of the heteroskedasticity is becase of the exponential in the heteroskedasticity fnction. z ik z ik 17

18 LIML appears to be slightly affected by the misspeci cation de to heteroskedasticity. Thogh the biases are small, they are enogh for UIJIVE to approximately eqal LIML in mean and median absolte error. The bias of J2SLS actally decreases in this sitation, althogh its spread is now approximately eqal to that of UIJIVE. Jmping to the last panel, where A Ç 5, we again see the agar and B2SLS estimators performing extremely poorly. ot only is the bias extremely high, bt the spread of the estimator arond this biased mean is now extremely low, generating very poor con dence interval coverage. The J2SLS estimator in this speci cation is also very poor. Its bias properties are not mch better than 2SLS and its con dence interval coverage is less than 40%. In contrast, both the IJIVE and UIJIVE estimators perform very well, with virtally no bias, good con dence interval coverage, and the lowest median absolte error. While the LIML estimator still performs reasonably well, it appears a bit more affected by the misspeci cation than when A Ç5, and con dence interval coverage is below 80%. Tables 4 and 5 pt the heteroskedasticity into the rst stage and second stage components of the error term, F 2 and F 3, respectively. agar and B2SLS perform well in these sitations, sggesting that heteroskedasticity in the common component of the error term is the most serios problem. This coincides with or theoretical reslts -the bias term in (33 depends on heteroskedasticity in the covariance between > Perhaps the most interesting reslt on these tables is the fact that LIML appears to be slightly affected by heteroskedasticity in the F 3 component of the error term. 17 Table 6 tries to approximate or grop asymptotic argments by increasing both the nmber of observations and the nmber of instrments. Starting with the speci cation with heteroskedasticity in the common error term (Table 3, we increase to 500 and K to 100. Looking throgh the panels, a rst observation is that both the mean and median bias of the IJIVE and UIJIVE estimators shrink to almost ndetectable levels. In contrast, while the biases of agar and B2SLS decrease, they do not decrease by mch, and given the smaller variance de to larger, con dence interval coverage actally is worse than with Ç 100. Also note that the proportion by which IJIVE and UIJIVE beat agar and B2SLS in terms of mean and median absolte error increases moving from Ç 100 to Ç 500. We introdce heteroskedasticity in or groping model by simply allowing the variance i to differ across grops. This exercise introdces heteroskedasticity in both the rst and second stage eqations. Becase IJIVE and agar are identical (and both are grop asymptotically consistent when there are the same nmber of observations in each grop, for this experiment we assme there are 2 grops of 23 observations and 18 grops of 3 observations (in total there are still 20 grops and 100 observations. In the monte carlos, we allow the variance i to vary between 0 and 4. The reslts are in Tables 7 and 8. In these tables, J1 refers to the variance i for the 18 small grops and J2referstothevarianceof@ i for the 2 big grops. The rst panel of Table 7 shows reslts for the model withot heteroskedasticity. Again as expected, IJIVE is very similar to agar and UIJIVE is similar to B2SLS. J2SLS has more bias bt lower mean absolte error than the JIVE or agar type estimators. LIML performs very well on bias and has the lowest mean absolte error. The next 17 That these biases are relatively small in LIML may reslt from the fact that the coef cients on F2 and F3 in (40 are small relative to that on F1. In fact starting with Panel D on Table 5, if we mltiply the coef cient on F3 by 5, we get median biases of the 8 estimators respectively of (0.587, 0.281, 0.103, , 0.045, , 0.029, Clearly, LIML is not performing well in this sitation. 18

19 for panels of the table show the reslts with heteroskedastic errors. In panel B, the variance i is zero for the 18 small grops and 4 for the 2 large grops. In this sitation, agar and B2SLS perform poorly with median biases in excess of 0.6 and mean biases in excess of 0.8. The variances of these estimators are also large and their mean and median absolte errors are high. In contrast, the IJIVE and UIJIVE contine to have small biases and vales of mean and median absolte error that are very low. Interestingly, the performance of 2SLS and J2SLS is very similar to the IJIVE and UIJIVE estimators in this case. It appears that the bias indced by the heteroskedasticity may be offsetting the bias of 2SLS and J2SLS in the standard homoskedastic case. LIML is very biased in this heteroskedastic case with mean and median biases of abot ext, consider panel E where the variance i is 4 for the 18 small grops and zero for the 2 large grops. Once again, agar and B2SLS are seriosly biased. In addition their coverage rates are well shy of 0.9. In contrast, both JIVE estimators have small biases and good coverage rates. LIML and J2SLS are also very biased in this case and both also have poor coverage rates. In Table 8, we redo the exercise with 500 observations and 100 grops. As sggested by the grop asymptotics, the biases of IJIVE and UIJIVE become very small with this setp. In contrast, the biases in agar, B2SLS, LIML, and J2SLS remain large and their coverage rates are generally poor. 18 The mean and median absolte errors of the IJIVE and UIJIVE estimators are low for all the forms of heteroskedasticity considered and coverage rates remain extremely good. 5 Application to Retrns to Edcation In this section, we follow AIK by applying or estimators to the retrns to edcation sing the data and methodology of Angrist and Kreger (1991. In a paper that motivated mch of the recent literatre on overidenti ed models, Angrist and Kreger estimated the retrn to schooling sing qarter of birth as an instrment in a sample of 329,500 men born between (from the 1980 Censs. We estimate two of their speci cations. In the rst speci cation, there are 30 instrments created by interacting qarter and year of birth and the control variables are a set of year indicators (so K Ç 30 and L Ç 11. The second speci cation contains 180 instrments constrcted by adding interactions of 50 state and qarter of birth dmmies to the original 30 instrments. In this second speci cation, both state and year xed effects are inclded as controls (so K Ç 180 and L Ç 61. See Angristand Kreger (1991 for frther description of the data. In table 9, we report estimates for the two speci cations along with asymptotic standard errors 19.TheOLS, 2SLS, LIML, and JIVE coef cients and standard errors are exactly the same as those reported in AIK (1999. Comparing the JIVE estimates to IJIVE and UIJIVE, we see that, as expected, the IJIVE and UIJVE estimates are smaller and have lower standard errors. This is particlarly the case in the second speci cation, where L is 18 ote that while the biases of agar and B2SLS improve slightly moving from Ç 100 K 1 Ç 20 to Ç 500 K 1 Ç 100 in the continos model, they do not appear to improve at all in the groping model. This coincides with the remarks in footnote 11, as in or monte-carlo groping case, the magnitde of H P zii remains constant in K 1, while in or continos monte-carlo model (since Z k are i.i.d., the magnitde of H P zii decreases at rate T K 1 19 For or new JIVE estimators, we obtained asymptotic standard errors following the discssion in section 4.1 (thogh we did not inclde the White heteroskedasticity correction to make or reslts comparable to the reslts in AIK. 19