Testing for Weak Instruments in Linear IV Regression

Transcription

1 Testing for Weak Instruments in Linear IV Regression August 001 James H. Stock Kennedy School of Government, Harvard University and the National Bureau of Economic Research and Motohiro Yogo* Department of Economics, Harvard University ABSTRACT The quality of the asymptotic normal approximation to the distributions of instrumental variables estimators in the linear IV regression model depends on the extent to which the instruments are relevant. If the instruments are weak, so that the system is nearly unidentified for a given sample size, then the sampling distribution can be quite different from its Gaussian limit. This raises a practical problem: under what circumstances can an applied researcher be confident that identification is good enough, that is, that the instruments are not weak? This problem has been addressed previously (informally) when there is a single endogenous regressor. This paper considers the problem of two or more included endogenous regressors. The paper has two specific contributions. First, we characterize sets of weak instruments in terms of specific population measures on quality of inference based on two stage least squares; these sets in turn depend on eigenvalues of the concentration matrix. Second, we provide a statistical procedure to test for whether the instruments at hand are weak, where the probability of a false negative (concluding that instruments are not weak, when in fact they are) is controlled asymptotically. *Prepared for the Festschrift in honor of Thomas Rothenberg. This research was supported by NSF grant SBR

2 1. Introduction Textbook treatments of instrumental variables regression stress that for instruments to be valid they must be exogenous. In practice, however, it is equally important that the second condition for a valid instrument, instrument relevance, also holds, for if the instruments are only marginally relevant, or weak, then the conventional first order asymptotic limits can provide poor approximations to the sampling distributions of standard instrumental variables regression statistics. At a formal level, the strength of the instruments matters because the natural measure of this strength the so-called concentration parameter plays a role formally akin to the sample size in IV regression statistics. In his survey of approximations to the distributions of estimators and test statistics, Rothenberg (1984) makes this point to illustrate that asymptotic expansions need not always be performed in orders of the number of observations. He considered the single equation IV regression model, y = Yβ + U, (1.1) where y is a T 1 vector of observations on the dependent variable, Y is the T 1 included endogenous variables, and U is T 1 a vector of i.i.d. N(0,σ uu ) errors (his notation is different). The reduced form equation for Y is, Y = ZΠ + V, (1.) 1

3 where Z is a T K matrix of exogenous instrumental variables, Π is K 1 coefficient vector, and V is a vector of i.i.d. N(0,σ vv ) errors, where corr(u t, v t ) = ρ. The two stage least squares (TSLS) estimator of β is β ˆ TSLS = (Y P Z y)/ (Y P Z Y), where P Z = Z(Z Z) -1 Z, which Rothenberg (1984) expresses as, cµ( ˆ X + ( s/ µ ) β TSLS β) =, (1.3) 1 + ( Y / µ ) + S/ µ ) where µ = Π Z ZΠ/σ vv, c = σ vv /σ uu, X = Π Z U/(σ uu Π Z ZΠ) 1/, Y = Π Z U/(σ vv Π Z ZΠ) 1/, s = V P Z U/(σ uu σ vv ) 1/, and S = V P Z V/σ vv. Under the assumptions of fixed instruments and normal errors, X and Y are standard normal variables with correlation ρ, and s and S are elements of a matrix with a central Wishart distribution; in particular the distributions of X, Y, s, and S do not depend on the sample size. Thus the sample size enters the distribution of the TSLS estimator only through the concentration parameter µ. Moreover, the form of (1.3) makes it clear that µ can be thought of as an effective sample size, in sense that µ formally plays the role usually associated with T in (1.3). Rothenberg (1984) proceeds to describe (among other things) expansions of the distribution of the TSLS estimator in orders of µ, and he emphasizes that the quality of these approximations can be poor when µ is small. For this reason, an important practical concern is whether µ is so small that is, whether the instruments are sufficiently weak that inferences based on conventional TSLS estimates and their standard errors are potentially unreliable. But µ is an unknown

4 population parameter; how is a researcher to know in practice whether his or her instruments are sufficiently weak to jeopardize the validity of conventional TSLS inferences? In this paper, we develop a procedure for detecting whether instruments are weak. The matter of whether a set of instruments is weak cannot be resolved in the abstract, rather it depends on the inferential task to which the instruments are applied. We therefore offer two concrete definitions of a weak instrument, both based on the performance of the most common IV method, TSLS. The first is that an instrument is weak if the bias of the TSLS estimator, relative to the bias of OLS, could exceed a certain threshold b, for example 10%. The second is that an instrument is weak if the size of the Wald test based on the TSLS estimator exceeds the level of the test by a gap r, for example 10%. In this light, we do two things. First, we provide an asymptotic characterization of these two sets of weak instruments in terms of the minimum eigenvalue of the matrix version of µ/k. Next, we provide a test of the hypothesis that the instruments are weak that is, they fall in the set of weak instruments against the alternative that they are not. The test we propose is a simple one: compute the minimum eigenvalue of the matrix version of the F-statistic in the first stage of TSLS and compare it to a critical value, tabulated below. If the minimum eigenvalue is less than the critical value, conclude that the instruments are weak; otherwise, conclude they are not. 1 Both the characterization of the set of weak instruments and the distribution theory for the test under the weak instrument null require asymptotic approximations to various IV statistic distributions. This requires approximating distributions that are 3

5 highly accurate even when the concentration parameter is quite small, so small that the approximations discussed in Rothenberg (1984) are poor. We therefore work within the asymptotic framework developed by Staiger and Stock (1997), in which is held constant as the number of observations increases. This allows stochastic objects like X and Y in the representation (1.3) to have limiting normal distributions even if the errors are not exactly normal and the instruments are stochastic, so that (1.3); it also provides substantial simplifications in the expressions for the Wald statistic. The tabulated boundaries of the weak instrument sets and the associated critical values for the test behave quite differently for moderate to large values of K. We investigate these differences by performing Nagar-type asymptotic expansions of the TSLS estimator and its Wald statistic in orders of 1/ K under the assumptions that the information per instrument, µ/k, is fixed and K /T 0. This is similar in spirit to Bekker s (1994) approach; he obtained a first order distribution under the sequence K, T, K /T c, 0 < c < 1, and µ/t is fixed. In Bekkers (1994) expansion, the TSLS estimator is approximately normally distributed, although it is biased. Although Bekker s expansion might be a reasonable approximation when there are very many instruments, the assumption that K /T c > 0 raises questions about its suitability cases in which K is moderate and the sample size is large, for example, K is 15 and T = The rest of the paper is organized as follows. The IV regression model and the proposed test statistic are presented in Section. The set of weak instruments is characterized in Section 3. Section 4 presents the limiting asymptotic distribution of the test on the boundary of the (composite) null hypothesis, and argues that the test is 4

6 asymptotically unbiased. Section 5 examines the power of the test, and the asymptotic expansions are presented in Section 6. Section 7 concludes.. The IV Regression Model and Proposed Test Statistic.1. The IV Regression Model The Population Regression Model. The general instrumental variables regression model extends (1.1) and (1.) to have n included endogenous regressors and K 1 included exogenous regressors: y = Yβ + Xγ + U, (.1) Y = ZΠ + XΦ + V, (.) where Y is now a T n matrix of included endogenous variables, X is a T K 1 matrix of included exogenous variables (one column of which is 1 s if the regressions include an intercept), and Z is a T K matrix of excluded exogenous variables to be used as instruments; it is assumed throughout that K n. Let Z = [Z X] denote the matrix of all the exogenous variables. The conformable vectors β and γ and the matrices Π and Φ are unknown parameters. Throughout this paper, we shall focus on inference about β. Let X t = ( x1t " xk t), Z t = ( z1t zk t) 1 ", and Z t = (Z t X t ) denote the vectors of the t th observations on these variables. Also let Q and Σ denote the population second moment matrices, 5

7 u σ Σ E = = Σ ( u V ) t uu uv t t Vt ΣVu ΣVV E X Q XX QZX Xt Zt = = Q. (.3) Zt QXZ QZZ t and ( ) The essential idea of weak instruments is that Z is only weakly related to Y, given X. Following Staiger and Stock (1997), weak instrument asymptotics are developed by modeling Π as local to zero: Assumption L Π : Π = Π T = C/ T, where C is a fixed K n matrix. the moments: Also following Staiger and Stock (1997), we make the following assumption on Assumption M. The following limits hold jointly: (a) (T -1 U U, T -1 V U, T -1 V V) p (σ uu, Σ Vu, Σ VV ); (b) T -1 Z Z p Q, where Q has blocks denoted Q XZ, etc.; (c) (T -1/ X U, T -1/ Z U, T -1/ X V, T -1/ Z V) À (< Xu, < Zu, < XV, < ZV ), where < œ [< Xu, < Zu, vec(< XV ), vec(< ZV ) ] is distributed N(0,Σ Q). Assumption M can hold for time series or cross-sectional data. Part (c) assumes that the errors are homoskedastic. 6

8 The TSLS Estimator and Wald Statistic. We focus on estimation of β by two stage least squares. Let the superscript A denote the residuals from the projection on X; that is, Y A = M X Y, y A = M X y, and so forth, where M X = I X(X X) -1 X. In this notation, the OLS estimator of β is ˆβ = (Y A Y A ) -1 (Y A y). The TSLS estimator is, β ˆ TSLS = (Y A PZ A Y A ) -1 (Y A PZ A y). (.4) The Wald statistic testing the q linear restrictions that Rβ = r 0 is, W T = ˆ TSLS A A 1 1 TSLS ( Rβ r)'[ R( Y ' P AY ) R'] ( Rβ r) Z TSLS qσˆ uu ˆ (.5) where σ ˆ TSLS uu = U ˆ TSLS U ˆ TSLS /(T K 1 n), where U ˆ TSLS = y A Y A β ˆ TSLS. We shall focus on the case q = n... The Proposed Test Statistic The proposed test is based on the eigenvalue of the matrix version of the F- statistic from the first stage regression of TSLS, G T = Σ ˆ 1/ VV Y A PZ A Y A Σ ˆ 1/ VV /K, (.6) where Σ ˆVV = (Y M Z Y)/(T K 1 K ). 7

9 The test statistic is the minimum eigenvalue of G T : g min = mineval(g T ). (.7) 3. Characterization of the Set of Weak Instruments 3.1. Weak Instrument First Order Asymptotic Representations We start by summarizing relevant weak instrument asymptotic results from Staiger and Stock (1997). Notation and definitions. The following notation in effect is a transformation of variables and parameters that simplifies the asymptotic expressions. Let ρ = θ = Σ Σ, 1/ 1/ VV ' Vuσ uu Σ Σ = 1 VV Vu 1/ σ uu 1/ ΣVV ρ, λ = Ω 1/ C Σ 1/ VV, and 1 Ω = Q ZZ Q ZX Q Q XZ. XX Note that ρ ρ 1. Define the K 1 and K n random variables, z u = Ω -1/ 1 (< Zu Q ZX Q < Xu ) XX z V = Ω -1/ 1 (< ZV Q ZX Q < XV ) XX σ 1/ uu 1/ Σ VV 8

10 so that zu 1 ρ ' N(, 0 Σ Ι K ), where Σ = vec( ). zv ρ In Also let ν 1 = (λ + z V ) (λ + z V ), and (3.1) ν = (λ + z V ) z u. (3.) Weak Instrument Asymptotic Representations. In this notation, the probability limit of the OLS estimator is ˆβ p β + θ. (3.3) theorem 1): The TSLS estimator and G T have the following limits (Staiger and Stock (1997), ˆ TSLS β À 1/ σ uu 1/ Σ VV ν ν, (3.4) 1 1 G T À ν 1 /K (3.5) and, when q = n and the null hypothesis r = r 0 is true, 9

11 W T À ν ' ν ν n(1 ρ ' ν ν + ν ' ν ν ) œ W*, (3.6) where À denotes weak convergence. That is, (3.4) says that, under Assumptions L Π and M, as T, the distribution of β ˆ TSLS converges to the distribution of 1/ σ uu 1/ Σ VV ν ν First Characterization of the Set of Weak Instruments: TSLS Bias Our first characterizations of the set of weak instruments is based on the maximum bias of TSLS relative to OLS. The expectation of the weak instrument asymptotic representation of the TSLS estimator exists only if the degree of overidentification K n, so this is assumed whenever discussing bias of TSLS. We consider the relative bias measure B T, which is the ratio of the bias of TSLS to the bias of OLS, where the coefficient vector has been put into standardized units by 1/ rotating by Σ Y A Y A : 3 B T = ˆTSLS ( )' ( ˆTSLS Eβ β Σ A A Eβ β ) Y Y. (3.7) ( Eβˆ β )' Σ ( Eβˆ β ) A A Y Y If n = 1, then the scaling matrix in (3.7) drops out and the expression simplifies to B T = E β ˆ TSLS β / E ˆβ β. 10

12 Under weak instrument asymptotics, β ˆ TSLS has the limit in (3.4), and the bias measure has the asymptotic limit, B T ρ ' hh ' ρ ρ ' ρ œ B, (3.8) 1 where h = E[ ν (λ + z V ) z V ]. 1 The asymptotic bias measure B depends on ρ and λ, which are unknown, as well as K and n. Our approach is to consider instruments to be strong if they lead to reliable inferences for all possible degrees of simultaneity ρ; otherwise they are weak. Applied to the relative bias measure and assuming ρ ρ > 0, this leads us to consider the worst-case asymptotic bias, B max = (max ρ: ρ ρ > 0 B ) 1/ (3.9) This first characterization of the set of weak instruments is based on this worstcase bias. We define the set of weak instruments, based on relative bias, to consist of those instruments that have the potential of leading to asymptotic relative bias greater than some value b. In population, the strength of an instrument is determined by the parameters of the reduced form equation (.). Accordingly, let = {Π, Σ VV, Ω}. The relative bias definition of a weak instrument is, 11

13 / bias = {: B max b} (3.10) The asymptotic results make it possible to characterize the set / bias. Because h depends on λ but not ρ, by (3.8) we have that B max ρ ' hh ' ρ = (max ρ ρ ' ρ )1/ = [maxeval(h h)] 1/, (3.11) where maxeval(a) denotes the maximum eigenvalue of the matrix A. By applying the singular value decomposition to lamba, it is further possible to show that the maximum eigenvalue of h h, and thus B max, depends only on K, n, and the eigenvalues of λ λ/k. In Section 6, we provide an asymptotic expansion for the TSLS estimator and its Wald statistic when the number of instruments K increases slowly with the sample size and in which λ λ/k is fixed, so λ λ/k = Λ, a fixed matrix. Proposition in that section states that the leading term in the associated expansion for B max is nonincreasing in each of the eigenvalues of Λ. It follows that, to the first order of approximation in that expansion, the set / bias can be characterized by the minimum eigenvalue of λ λ/k for a given n and K. That is, B max = B max (mineval(λ λ/k );K,n), where this function is increasing in its argument and therefore invertible. This in turn implies that B max (mineval(λ λ/k );K,n) b is equivalent to mineval(λ λ/k ) A bias (b;k,n), where 1

14 A bias is the inverse function of B max with respect to its first argument. That is, / bias in (3.10) can be written as, / bias = {: mineval(λ λ/k ) A bias ( b;k,n)}. (3.1) Our formal justification for the simplification that / bias depends only on the smallest eigenvalue of λ λ/k, rather than on all its eigenvalues, rests on the expansion in Section 6. Numerical analysis for n = suggests, however, that B max is decreasing in all the eigenvalues of λ λ/k for all values of K. This numerical analysis suggests that the simplification in (3.1), looking only at the minimum eigenvalue, is valid for all K under the weak instrument asymptotics, even though we currently cannot provide a formal justification Second Characterization of the Set of Weak Instruments: TSLS Wald Test Size The second characterization of the set of weak instruments is based on the maximal size of the TSLS-based Wald test of all the elements of β. Throughout we focus only on the possibility that the Wald test rejects too often under the null, which is in fact the case with weak instruments. In parallel with the approach for the bias measure, we consider an instrument strong from the perspective of the Wald statistic if the size of the test is close to its level for all possible configurations of the IV regression model. The actual rejection rate R T under the null hypothesis is, 13

15 R T = Pr[W T > χ /n r = r 0 ], (3.13) n; α where χ is the α-level critical value of the chi-squared distribution with n degrees of n; α freedom and α is the nominal level of the test. Under the weak instrument asymptotics the distribution of W T has the limiting representation (3.6) under the null hypothesis. Thus, R T Pr[W* > χ /n] œ R, (3.14) n; α Inspection of (3.6) reveals that R depends on ρ and λ, as well as K, n, and α. Following our treatment of the bias, because ρ is unknown we consider the worst-case size, R max 1 ν ' ν1 ν = max ρ R = max ρ Pr[ > 1 (1 ρ ' ν ν + ν ' ν ν ) 1 1 χ ]. (3.15) n; α It is not necessary to exclude ρ ρ = 0 in the maximization in (3.15) because there is no singularity at ρ ρ = 0. The set of weak instruments, / size, based on the size of the Wald statistic, consists of instruments that can lead to a size of at least r: 14

16 / size = {: R max r} (3.16) For example, if α =.05 then a researcher might consider it acceptable if the worst case size is r =.10. The maximal bias measure R max depends on the eigenvalues of λ λ/k as well as n and K (the argument is the same as for the similar assertion for B max ). Thus, under the weak instrument asymptotics, the weak instrument set / size is fully characterized by ρ and the eigenvalues of λ λ/k. In the expansion of W* in Section 6, the leading term is nonincreasing in the eigenvalues of λ λ/k (Proposition 3). This implies that we can characterize / size in terms of the minimum eigenvalue of λ λ/k, as well as K and n. The argument leading to (3.1) therefore applies here and leads to the characterization, / size = {: mineval(λ λ/k ) A size ( r;k,n,α) }. (3.17) where A size ( r;k,n,α) is the inverse function of R max (mineval(λ λ/k ); K,n,α) with respect to its first argument. As is the case for / bias, the justification for the simplification (3.17) in which / size depends on only the smallest eigenvalue of λ λ/k formally is based on the 15

17 expansion in Section 6 for weak instruments and moderate K. However, numerical analysis of the n = case suggests that this property holds for small K as well Numerical Results The worst case bias and the maximal size distortion were computed by Monte Carlo simulation for a grid of minimal eigenvalue of λ λ/k from 0 to 50, for K = n +,,100 for the relative bias, and for K = n,, 100 for the size distortions. All results are based on 0,000 Monte Carlo draws. Computing the maximum bias entails first computing h defined following (3.8) by Monte Carlo simulation, given n, K, and the minimum eigenvalue; as discussed above, for these computations we set λ λ/k = A I n, where A is the minimum eigenvalue. The maximum relative bias is then [maxeval(h h)] 1/. Computing the maximum size distortion is more involved because there is no simple analytic solution to the maximum problem (3.15). Numerical analysis indicates that R is maximized when ρ ρ = 1, so the maximization for n = was done by transforming to polar coordinates and performing a grid search over the half unit circle (half because of symmetry in the expression for W*). As in the relative bias computations and as discussed above, in these computations we set λ λ/k = A I n, where A is the minimum eigenvalue. The relative bias is plotted for n =1 as a function of K and λ λ/k in Figure 1, and the maximal relative bias is plotted for n = as a function of K and the minimum eigenvalue of λ λ/k in Figure (when n = 1, the relative bias B does not depend on ρ, so B max = B). 5 Evidently, for K sufficiently large, the maximal relative bias depends only weakly on K. The maximal relative bias seems to decline as approximately the inverse 16

18 of the minimum eigenvalue. Also, the contours for n = 1 in Figure 1 and n = in Figure are quite similar. The worst case size, R max, is plotted in Figure 3 for n =1 and in Figure 4 for n = for a Wald test of level 5%. In contrast to the plots of the worst case bias, the worst case size strongly depends on K over the full range considered. The worst case size approaches 5% much more rapidly as a function of the minimum eigenvalue for small K than when K is large. Also, the surfaces for n = 1 and n = are qualitatively similar but quantitatively different; curiously, given a value of K and the minimum eigenvalue, the size distortion when n = 1 is worse than when n =. These surfaces can be used to compute boundaries of the weak instrument regions / bias and / size, where the boundaries are determined by the minimum eigenvalue of λ λ/k, as well as by n and K. These boundaries are plotted in Figure 5: the relative bias region boundary is plotted in the top two panels for selected values of the relative bias cutoff b, and the size region boundary is plotted in the bottom two panels for selected values of the size cutoff r. First consider the boundary of / bias. These are essentially flat in K for K sufficiently large; moreover, the boundaries for n = 1 and n = are numerically very similar, even for small K. The boundary of the relative bias region for b =.1 (10% bias) asymptotes to approximately 8 for both n = 1 and n =. In contrast, the boundary of / size depends strongly on K and n. The boundary is approximately linear in K for K sufficiently large, for all the size distortion cutoffs r considered. Note also that the cutoffs are numerically quite large when the degree of 17

19 overidentification is large. For example, if one is willing to tolerate a maximal size of 15%, so the size distortion is 10% for the 5% level test, then with 0 instruments the minimum eigenvalue boundary is approximately 5 for n = 1 and approximately 0 for n=. We return to some of these features of the weak instrument region boundary functions when in the discussion of the asymptotic expansion in Section Asymptotic Distribution of the Test Statistic This section provides the weak instrument asymptotic representation of the test statistic g min and a bound on its distribution. This bound provides conservative critical values for the test based on g min. Distribution of g min. Recall that the statistic g min is the minimum eigenvalue of G T, where G T is given by (.6). Under weak instrument asymptotics, K G T is asymptotically distributed as a noncentral Wishart with dimension n, degrees of freedom K, identity covariance matrix, and noncentrality matrix λ λ: G T À G* _ W n (K, I n, λ λ)/k. (4.1) The joint pdf for the n eigenvalues of a noncentral Wishart is known in the sense that there is an infinite series expansion for the pdf in terms of zonal polynomials (Muirhead [1978]). This joint pdf depends on all the eigenvalues of λ λ, as well as n and K. In principal the pdf for the minimum eigenvalue can be determined from this joint 18

20 pdf for all the eigenvalues. It appears that this pdf (the exact asymptotic pdf of g min ) depends on all the eigenvalues of λ λ. This exact asymptotic distribution of g min is not very useful for applications both because of the computational difficulties it poses and because of its dependence on all the eigenvalues of λ λ. This latter consideration is especially important because in practice these eigenvalues are unknown nuisance parameters, so any critical values based on these eigenvalues would produce an infeasible test. We circumvent these two problems by performing the test using conservative critical values. These conservative critical values are based on a noncentral chi-squared bounding distribution, given in the following proposition. Proposition 1. Let W be distributed W n (k, I n, A). Then Pr[mineval(W) x] Pr[ χ (mineval(a)) x], where k variable with noncentrality parameter a. χ k (a) denotes a noncentral chi-squared random Proof. Let α be the eigenvector of A corresponding to its minimum eigenvalue. Then α Wα is distributed χ k (mineval(a)) (Muirhead [198, Theorem ]). But α Wα mineval(w), and the result follows. that Applying the proposition, (4.1), and the continuous mapping theorem, we have 19

21 χ K (mineval( λ ' λ) / K ) Pr[g min x] Pr[mineval(G*) x] Pr x. (4.) K Conservative critical values for the test based on g min are obtained by the following procedure. First, select the desired minimal eigenvalue of λ λ/k. Next, obtain the desired percentile, say the 95% point, of the noncentral chi-squared distribution with noncentrality parameter equal to this selected minimum eigenvalue, and divide this percentile by K. Weak instruments test. This yields the following testing procedure to detect for weak instruments. To be concrete, this is stated for a test based on the bias measure. The null hypothesis is that the instruments are weak, and the alternative is that they are not: H 0 : / bias v. H 1 : ± / bias. (4.3) The test procedure is, Reject H 0 if g min χ K, δ ( A bias ( b;k,n)), (4.4) where χ K, δ ( A ) is the 100δ% critical value (100(1-δ)% percentile) of the noncentral chisquared distribution with K degrees of freedom and noncentrality parameter A. 0

22 The critical value chosen by the researcher depends on the attitude of the researcher towards the maximal bias. If the researcher is satisfied if the maximal relative bias is no more than 10%, then she would set b =.10. The results of Section 3 and the bound in Proposition 1 imply that, asymptotically, the test (4.4) has the desired asymptotic level: lim T Pr[g min χ K, δ ( A bias ( b;k,n)) / bias ] δ. (4.5) The procedure for testing whether the instruments are weak from the perspective of the size of the Wald statistic is the same, except that the critical value in (4.4) is obtained using the size-based boundary eigenvalue function, A size ( r;k,n,α) Critical Values. Given a minimum eigenvalue A, conservative critical values for the test are percentiles of the noncentral chi-squared distribution, χ K, δ ( A ). The minimum eigenvalue A is obtained from the boundary eigenvalue functions, numerical values of which were reported in Section 3.4. Critical values for the relative bias version of the test are tabulated in Tables 1,, and 3 for n = 1, and 3, respectively, for various K and relative bias tolerances b. Critical values for the size version of the test are reported in Tables 4 and 5. The critical values reflect the features of the boundary eigenvalue functions discussed in Section 3.4. For example, the critical values for the relative bias at first increase then, for moderate K, essentially do not depend on K. In contrast, the critical values for the size version increase approximately linearly with K. 1

23 The critical values are plotted in Figure 6. These critical value plots are qualitatively similar to the boundary eigenvalue plots in Figure 5, except of course the critical values exceed the boundary eigenvalues to take into account the sampling distribution of the test statistic. Comparison to the Staiger-Stock (1997) rule of thumb. Staiger and Stock (1997) suggested the rule of thumb that, in the n = 1 case, instruments be deemed weak if the first stage F is less than ten. They motivated this suggestion based on the relative bias of TSLS. Because the 5% critical value for the relative bias weak instrument test with b =.1 is approximately 11 for all values of K, the Staiger-Stock rule of thumb can be viewed as 5% test that the worst case relative bias is approximately 10% or less. This provides a formal, and not unreasonable, testing interpretation of the Staiger-Stock rule of thumb. The Staiger-Stock rule of thumb fares less well from the perspective of size distortion. When the number of instruments is one or two, the Staiger-Stock rule of thumb corresponds to a 5% level test that the maximum size is no more than 15% (the maximum TSLS size distortion is no more than 10%). However, when the number of instruments is moderate or large, the critical value is much larger and this rule of thumb does not provide substantial assurance that the size distortion is controlled. 5. Asymptotic Properties of the Test as a Decision Rule This section discusses the asymptotic properties of the weak instruments test as a function of the smallest eigenvalue of λ λ/k. This entails providing and studying the asymptotic rejection rate of the test as a function of the smallest eigenvalue of λ λ/k.

24 When this eigenvalue exceeds the cutoff population eigenvalue A bias ( b;k,n) for a given b, this asymptotic rejection rate is the asymptotic power function. The asymptotic distribution of g min depends on all the eigenvalues of λ λ/k. It is bounded above by (4.). We conjecture that this distribution is bounded below by the distribution of the minimum eigenvalue of a random matrix with the noncentral Wishart distribution W n (K, I n, mineval(λ λ)i n )/K. 6 We will use these two bounding distributions to bound the distribution of g min as a function of mineval(λ λ/k ). The resulting bounds on the asymptotic rejection rate of the test (4.4) (based on relative bias) are plotted in Figures 7 for b =.10 and n =. The value of the horizontal axis (the minimum eigenvalue) at which the upper rejection rate curve equals 5% is A bias (.10;K,). Evidently, as the minimum eigenvalue increases, the rejection rate increases. For large K, this increase is rapid, and the test effectively has unit power against values of the minimum eigenvalue not much larger than one. The bounding distributions give a reasonably tight range for the actual power function, which depends on all the eigenvalues of λ λ/k. The analogous bounds for the test based on the size of the Wald statistic are plotted in Figure 8. The curves have a similar shape, but they are centered at much larger values of the minimal eigenvalue for the reasons discussed in Sections 4 and 5. Otherwise, the qualitative conclusions for the bias-based test apply to the size-based test. Interpretation as a decision rule. The test in (4.4) can, of course, be interpreted as a decision rule: if g min is less than the critical value, conclude that the instruments are weak; if it exceeds the critical value, conclude that they are strong. 3

25 Under this interpretation, the asymptotic rejection rates plotted in Figures 7 and 8 bound the asymptotic probability of deciding that the instruments are strong. Evidently, for small values of mineval(λ λ/k ), the probability of correctly concluding that the instruments are weak is effectively one. Thus, if in fact one is confronted by instruments that are quite weak, then the probability of not recognizing this using the testing procedure (4.4) is vanishingly small. Similarly, if one has instruments for which the minimum eigenvalue of λ λ/k is substantially above the threshold for the weak instruments set, then the probability of correctly concluding that they are strong also is effectively one. The range of ambiguity of the decision procedure is given by the values of the minimum eigenvalue for which the asymptotic rejection rates effectively fall between zero and one. When K is small, this range is fairly large. For K large, this range of potential ambiguity of the decision rule is quite small. From a practical perspective, it matters whether the researcher is concerned about bias or size. Concern about relative bias leads to much smaller weak instrument sets than concern about size distortions. 6. Asymptotic Expansion for Many Weak Instruments This section reports the results of, and conclusions based on, Nagar-type asymptotic expansions of the TSLS estimator β ˆ TSLS and its Wald statistic W. These expansions have two purposes. First, they provides a formal justification for the characterizations of the sets / bias and / size solely in terms of the minimum eigenvalue of 4

26 λ λ/k ; this is summarized in Propositions and 3 below. Second, they explain the peculiar feature, noted in Section 4, that the boundary of the weak instrument region / bias is essentially constant for K is sufficiently large (say, K n 10), whereas the boundary of the region / size is essentially linear in K for K sufficiently large. The expansion here holds the noncentrality matrix, λ λ/k, constant. We let K and T but, in contrast to Bekker (1994), K /T 0. The expansion is carried out only to order o p (1/ K ), insufficient to calculate an Edgeworth approximation but sufficient to address the matters of the previous paragraph. The expansion is performed for the IV regression model (1.1) and (1.) with no included exogenous variables, fixed Z, and i.i.d. Gaussian errors, except that we allow for n included endogenous variables. Let λ = (Z Z/T) -1/ ( T Π T ) Σ, 1/ VV z = (Z Z/T) -1/ (Z V/ T ) Σ, V 1/ VV z = (Z Z/T) -1/ (Z U/ T ) σ, u 1/ uu ν 1 = ( λ + z ) ( λ V + z V ), and ν = ( λ + z ) V z u. These definitions parallel the corresponding definitions of λ, etc. in the weak instrument asymptotic representation in Section 3, except that these are defined in the original sample space. 5

27 With these definitions, we have the exact relations, ˆ TSLS β = β + 1/ σ uu 1/ Σ VV ν ν, (6.1) W = ν ν ν /(ns T ), (6.) 1 where the expression for W holds under the null hypothesis and s T = (U U/T)/σ uu [ z u λ + ρ Σ 1/ VV (V V/T) 1/ 1/ Σ VV + σ uu (U V/T) 1 Σ ] ν ν 1/ VV 1 ν ν [( λ λ )/T + ( λ z )/T + ( z λ V V )/T + 1/ Σ VV (V V/T) 1/ Σ VV ] ν ν. 1 1 We make the following assumption: Assumption E. K, T, K /T 0, and λ λ /K = Λ is fixed. Assumption E replaces assumption L Π. Under this assumption, Π is implicitly modeled as Π T = K / T C, that is, as being in a K / T neighborhood of zero. This captures the notion that the instruments are weak while permitting an analysis of the statistics of interest for the case that the number of instruments is moderate, relative to the sample size. Under Assumption E, the assumption of homoskedastic Gaussian errors, and the assumption that Z Z/T p Q ZZ, we find that, 6

28 z1+ z1' + z ν 1/K = Λ + I n + K + o p ( 1 K ), and (6.3) z1+ zρ + z3 + z4 ν /K = I n ρ + K + o p ( 1 K ) (6.4) where z 1, z, z 3, and z 4 are independent and vec( z 1 ) _ N(0,I n Λ), vec( z ) _ N(0,H), z 3 _ N(0, (1-ρ ρ)λ), and z 4 _ N(0, (1-ρ ρ)i n ), and H = Bias. Applying these results to the relative bias formula yields the leading term, B = ρ '( Λ + I) ρ ' ρ ρ + O p ( 1 K ). (6.5) This allows us to characterize the weak instrument set / bias to the order of approximation in (6.5). 7

29 Proposition. For any ρ such that ρ ρ > 0, the leading term in the expansion (6.5) for the relative bias B is nonincreasing in each eigenvalue of Λ. Moreover, up to this leading term in the expansion, B max = [ mineval(λ) + 1] -1. Proof. For any ρ, the leading term in (6.5) can be written as n ai ( λi + 1), i= 1 where λ i is the i th eigenvalue of Λ, a 1,, a n do not depend on the eigenvalues of Λ and n ai = 1. The result follows. i= 1 Size. The Wald statistic has the expansion, W/K = ( ν / K )'( ν / K ) ( ν / K ) n[1 ρ '( ν / K ) ( ν / K ) + ( ν / K )'( ν / K ) ( ν / K )] o p ( 1 K ). (6.6) where ν 1 and ν denote the terms up to order O p (1/ K ) in (6.3) and (6.4), respectively. The leading term in this expansion is, W/K = 1 ρ '( Λ + I) ρ ρ Λ + I ρ + ρ Λ + I ρ + O p( 1 n[1 '( ) '( ) ] 1 K ). (6.7) This allows us to characterize the weak instrument set / size to the order of approximation in (6.7): 8

30 Proposition 3. For any ρ, the leading term in the expansion for W/K in (6.7) is nonincreasing in each eigenvalue of Λ. Moreover, this leading term is maximized when ρ ρ = 1. Proof. Write (Λ + I) -1 = JHJ, where H is the diagonal matrix with diagonal elements 1/(1+λ i ), where λ i is the i th eigenvector of Λ and J is the matrix of the associated eigenvectors of Λ. Then the leading term in (6.7) can be written, h = a Ha/{n[a (I H) a + (1 a a)]}, where a = Hρ. Differentiating h with respect to λ j shows that h/ λ j 0, j = 1,, n, proving the first statement. Because a a 1, h/ a j 0, j = 1,, n, for any a, h can be increased by increasing a proportionately, or equivalently by increasing ρ proportionately until ρ ρ = 1. It follows that h is maximized when ρ ρ = 1. The leading terms in the expansions (6.5) and (6.7) explain the curious feature that the boundary of / bias is constant for large K while it essentially increases linearly in K for / size. First consider the expression (6.5) for B. The leading term is constant for a given ρ and Λ, and by Proposition, B max depends only on the minimum eigenvalue of Λ, up to the order of this leading term. But Λ, and thus B max, does not depend on K. Similarly, the minimum eigenvalue of Λ does not depend on n. Thus, for K sufficiently large, B max depends only on the minimum eigenvalue of Λ, not on K or n. 9

31 Next turn to the expression (6.7) for W/K. Calculations reveal that this leading term, maximized over ρ, depends on n although it does not depend on K. Because W is divided by K in this expression, (6.7) alternatively implies that W = ck + O p ( K ), where c depends on ρ, the eigenvalues of Λ, and n. This explains the effectively linear increase in the boundary of / size and its dependence on n. 7. Conclusions The procedure proposed here is simple to implement: it entails comparing the minimum eigenvalue of the first stage F-statistic matrix the multivariate generalization of the first stage F-statistic to a critical value. The critical value is determined by the number of instruments K, the number of included endogenous regressors n, and the researcher s willingness to tolerate relative bias or size distortions. The test statistic is the same whether one focuses on size or relative bias; all that differs is the critical value. Viewed as a test, the procedure has good power (especially so when the number of instruments is large). Viewed as a decision rule, the procedure discriminates between weak and strong instruments quite effectively, with the region of ambiguity decreasing with the number of instruments. When there is a single included endogenous variable, this procedure provides a refinement and improvement to Staiger and Stock s (1997) suggested rule of thumb that, in the n = 1 case, instruments be deemed weak if the first stage F is less than ten. The difference between that rule of thumb and the procedure of this paper is that, instead of comparing the first stage F to ten, it should be compared to the appropriate entry in Table 30

32 1 (bias) or 4 (size). Those critical values indicate that their rule of thumb can be interpreted as a 5% test of the hypothesis that the maximum relative bias is (approximately) 10%. However, their rule of thumb does not ensure that TSLS-based Wald statistics will have good size. The results in this paper have two loose ends. First, the characterization of the set of weak instruments is based on the premise that the maximum relative bias and maximum size distortion are nonincreasing in each eigenvalue of λ λ/k. This was justified for moderate K using the expansion in Section 6 and numerical analysis suggests it is true for all K, but this remains to be proven. Second, the lower bound of the power function in Section 6 is based on the similar assumption that the cdf of the minimum eigenvalue of a noncentral Wishart random variable is nondecreasing in each of the eigenvalues of its noncentrality matrix. This too appears to be true based on numerical analysis but we do not have a proof nor does this result seem to be available in the literature. Beyond this, several avenues of research remain open. First, the testing procedure described here focuses on TSLS. One extension of this research would be to consider other estimators and other Wald statistics, particularly the LIML Wald statistic. LIML and its Wald statistic have better-behaved weak instrument sampling distributions that TSLS and its Wald statistic, so LIML-based inference might produce tighter boundaries of the weak instrument region. Second, the analysis here is predicated upon homoskedasticity, and it remains to extend these tests to GMM estimation of the linear IV regression model under heteroskedasticity. 31

33 Third, the asymptotic expansion in Section 6 could be extended in several ways. The expansion was for the classical fixed instrument/normal error model. Relaxing this assumption, as Rothenberg (1983) did for the Π fixed case, would complicate the expressions only somewhat to order o p (1/ K ) and would generalize the result considerably. It also remains to explore the quality of the resulting distributional approximations to the finite sample distribution of the TSLS estimator. 3

34 Endnotes 1 Cragg and Donald (1993) propose a test of the null hypothesis that the coefficients of interest are underidentified, against the alternative that they are identified. Our purpose is quite different: just because the parameters are identified does not mean that an applied researcher can reliably use use the conventional first order asymptotic approximations to the limiting distributions of TSLS statistics. The definition of G T in (.6) is G T in Staiger and Stock (1997, eq. (3.4)), divided by K to put it in F-statistic form. 3 Note that Σ Y A Y A = Σ VV under Assumption L Π. 4 The assumption that B max ( A ;K,n) is nonincreasing in A, needed to invert the function, has the effect of potentially including in (3.1) some instruments that are not weak. Because the maximal bias depends on all the eigenvalues, the maximal bias when all the eigenvalues are equal to some value A 0 might be greater than the maximal bias when one eigenvalue is slightly less than A 0 but the others are very large. For this reason the set / bias is potentially conservative. This comment applies to the size-based set developed in Section 3.3 as well. 5 Figure 1 previously appeared in Staiger and Stock (1997) and is included here for completeness. 6 We have not located a formal statement of this in the literature but numerical investigations suggest that it is true. 33

35 References Bekker, P.A. (1994), Alternative Approximations to the Distributions of Instrumental Variables Estimators, Econometrica, 6, Cragg, J.G. and S.G. Donald (1993), Testing Identifiability and Specification in Instrumental Variabel Models, Econometric Theory, 9, 40. Muirhead, R.J. (1978), Latent Roots and Matrix Variates: A Review of Some Asymptotic Results, Annals of Statistics, 6, Muirhead, R.J. (198), Aspects of Multivariate Statistical Theory. New York: Wiley. Rothenberg, T.J. (1983), Asymptotic Properties of Some Estimators in Structural Models, in S. Karlin, T. Amemiya, and L. Goodman, eds., Studies in Econometrics, Time Series, and Multivariate Statistics, Academic Press. Rothenberg, T.J. (1984), Approximating the Distributions of Econometric Estimators and Test Statistics, Ch. 15 in Z. Griliches and M.D. Intriligator (eds.), Handbook of Econometrics. Amsterdam: North Holland, Staiger, D. and J.H. Stock (1997), Instrumental Variables Regression with Weak Instruments, Econometrica, 65,

36 Table 1: Critical values of mineval G T /K at the 5% significance level (n =1) Relative bias K Relative bias K

37 Table : Critical values of mineval G T /K at the 5% significance level (n =) Relative bias K Relative bias K

38 Table 3: Critical values of mineval G T /K at the 5% significance level (n =3) Relative bias Relative bias K K

39 Table 4: Critical values of mineval G T /K at the 5% significance level (n =1) Size distortion K