In all data-generating processes considered until now, we can point

Transcription

1 web Extension 4 Section 13.7 Local Average Treatment Effects In all data-generating processes considered until now, we can point to the effect of a change in any one explanator on the value of Y. In the model, Y i = b 0 + b 1 X i + e i, the effect on Y of a one-unit change in X is b 1. But such a characterization is not always appropriate. For example, when one more convict is released from jail through an early release program for prisoners, what effect does this have on crime in a community? Not all prisoner release programs are the same; some increase crime rates little, some increase crime rates a lot. It depends on whom the program releases. If the program releases wellbehaved prisoners who had no prior criminal record before this conviction, the effect might be small and, thus, the program might prove harmless. (For simplicity, we call these benign programs. ) On the other hand, if the program releases predatory career criminals, the effect might be large. (For simplicity, we call these predator release programs. ) Similarly, what effect does reducing by one the size of a particular teacher s third-grade class have on student achievement? The effect might be one thing if the teacher believes the change is permanent altering syllabi, changing class organization, and the like might be worthwhile for a sustained smaller class size. But the effect might be much less if the teacher believes the change is temporary a blip in a succession of otherwise large third-grade classes might EXT 4-7

2 EXT 4-8 Web Extension 4 spur little adjustment. Not all teachers respond the same way to class size reductions because not all teachers perceive those changes in the same way. In examples like these, characterizing the effect of a change in X (the number of prisoners released or the size of a third-grade class) on Y can be misleading. In this section, we explore the consequences of such varying effects of explanators for ordinary least squares (OLS) and instrumental variables (IV) estimation. Random Coefficients Models The prisoner release and class size examples are suggestive about how we might formalize the idea of varying effects of X on Y. Consider first the prisoner release programs. If each observation in our sample is a community drawn at random from all the communities with prisoner release programs, we sometimes get the benign programs and sometimes get the predator release programs. Which effect do we see on Y from releasing one more prisoner? The effect of one more prisoner released, on Y, the number of crimes in the community, is a random variable. The effect is large, with a probability equal to the probability of drawing a community with a predator release program; the effect is small with a probability equal to the probability of drawing a community with a benign program. Now consider programs that reduce class sizes. If each observation in our sample is a teacher drawn at random from a population of teachers who have varying expectations about a class size reduction s permanency, sometimes we draw a teacher who expects a temporary reduction and sometimes we draw a teacher who expects a permanent reduction. Which effect we see from reducing class size by one is, once again, a random variable. Algebraically, we write, Y i = b 0 + (b 1 + n i )X i + e i, where the coefficient on X i is the random variable (b 1 + n i ), in which b 1 is the mean effect of one more unit of X on Y in the population, and n i is the i-th observation s deviation of the coefficient on X from the mean coefficient. We call Equation a random coefficients model. We can rewrite this equation as Y i = b 0 + b 1 X i + n i X i + e i = b 0 + b 1 X i + m i, where m i = n i X i + e i. The last formula in Equation is quite similar to our standard regression formulation, but it has two distinct features. First, the coefficient on X is now the mean coefficient on X, rather than the coefficient. Second, the appearance of n i X i in m i suggests the disturbances are heteroskedastic. In the prisoner release program example, the variation in the coefficients stems from X measuring two different phenomena. In the class size example, the

3 Local Average Treatment Effects EXT 4-3 variation stems from differences among the teachers. In general, randomness of coefficients can stem from either X measuring multiple phenomena or from observed individuals varying in their responsiveness to a single phenomenon. Random Coefficients Models and OLS OLS has proven a workhorse estimator for us, often yielding best linear unbiased estimators (BLUE) or, at least unbiased or consistent estimates. What are the properties of OLS in this random coefficients model? Answering this question requires some information about the relationship between X i and n i. Suppose that each n i and e i are jointly independent of all of the observations on the explanator. (Recall that for independent random variables, E(WV) = E(W)E(V).) In this case, E(bN ) = ax iy i a x2 i = E ax i(b 1 x i + m i - m) a x i 2 = E b 1 + ax i(m i - m) a x i 2 = b 1 + E ax i(m i - m) = b 1 + a BE x i a x i 2 = b 1 + E a x i a x i 2(m i - m) a x i 2 E(m i - m)r = b = b 1, where m is the mean disturbance in the sample, which has expectation zero. If each e i and each coefficient on X, (b 1 + n i ), are jointly independent of all the observations on X, then OLS is an unbiased estimator of the average coefficient on X, b 1. Similar reasoning with plims would reveal that if the disturbances and coefficients are contemporaneously uncorrelated with one another and with the explanator, OLS is a consistent estimator of the mean coefficient on X in the population. Because our interest in many applications is with the average effect of changes in X across many instances, the finding that OLS can unbiasedly or consistently estimate such an average coefficient further evidences the rich applicability of OLS. Nonetheless, it is important to note that if X i is correlated with its coefficient, (b 1 + n i ), then OLS biasedly and inconsistently estimates b 1, the mean coefficient on X in the population. For example, if the size of prisoner release programs varied systematically between benign programs and predator release programs, or if permanent changes in class sizes were systematically larger (or smaller) than temporary changes, then OLS would not unbiasedly or consistently estimate b 1. Also, as in models with nonrandom coefficients, if the explanator is contemporaneously correlated with the e i, OLS yields biased and inconsistent estimates of b 1. All these results generalize to multiple regression models with random coefficients.

4 EXT 4-10 Web Extension 4 Random Coefficients Models and IV Estimation In the prisoner release program example, OLS would inconsistently estimate b 1 if the reason that states expanded their predator prisoner release programs were because current crimes threatened to soon overcrowd the prisons. In such a case, we would see larger programs because of larger e i, and X i and e i would be correlated. Similarly, if class sizes were reduced in anticipation of a class doing poorly, X i and e i would be correlated, and OLS would again inconsistently estimate b 1. Can instrumental variables consistently estimate b 1 in such cases? Yes, they can, but the instruments chosen must be contemporaneously uncorrelated with both the e i and the n i, so that the instruments would be contemporaneously uncorrelated with the m i in Equation How IV estimation might be consistent and OLS not consistent when estimating b 1 is most easily seen when there are two distinct groups, each with a different, constant coefficient on X, as might be the case in the prison release example. Let s call the benign release programs group 1 and call the predator release programs group 2. Assume the specific, nonrandom coefficients on X in the two programs are b 1 and b 2, respectively, and that the fixed difference between b 2 and b 1 is d 2. The mean coefficient b is then a probability weighted sum of and : b = b 1 (1 - E(D 2 )) + b 2 E(D 2 ), in which the random variable D 2i = 1, if the i-th observation drew the second group, and D 2i = 0, otherwise. Following the form of Equation 13.25, we could write the new model as b 1 b 2 Y i = b 0 + bx i + n i X i + e i = b 0 + bx i + m i, where b is the mean coefficient on X. If X i is contemporaneously correlated with e i or with n i, OLS yields a biased estimate of b, the coefficient on the released prisoners variable in Equation IV estimation, in contrast, can consistently estimate b in this case. As just noted, an instrument contemporaneously uncorrelated with both the e i and the n i would allow us to consistently estimate b. But choosing an instrument uncorrelated with both the e i and the n i is not the only interesting IV estimation strategy in random coefficients models. An Alternative Use of IV Estimation A different, and sometimes useful, result follows from choosing a different IV estimator, instead. We could rewrite Equation as Y i = b 0 + (b 1 + d 2 D 2i )X i + e i,

5 Local Average Treatment Effects EXT 4-5 or Y i = b 0 + b 1 X i + d 2 D 2i X i + e i = b 0 + b 1 X i + (d 2 D 2i X i + e i ), in which the random variable D 2i = 1 if the i-th observation drew the second group, and D 2i = 0 otherwise. This specification suggests that if we choose the right sort of instrumental variable, we could consistently estimate not b 1, but b 1. Suppose we choose an instrument that is contemporaneously uncorrelated with the e i but is correlated with the n i. For example, suppose our instrument is the size of a religious group that champions benign release programs. The size of the religious group in the i-th observed community is R i. The size of the religious group is a good instrument in that we assume that the size of the religious group is uncorrelated with the disturbances in the crime rate; that is, E(r i e i ) = 0, with lowercase letters denoting deviations from means, as usual. Furthermore, we assume R i is correlated with X i, but only because the size of the religious group in the community has an effect on the number of prisoners released in benign programs; that is, E(r i x i ƒ D = 0) Z 0, but, E(r i x i ƒ D = 1) = 0. Moreover, we assume that the religious group influences which kind of program is in place in a community; that is, E(r i n i ) Z 0. Is R i a valid instrument for estimating the mean coefficient on released prisoners, b? No, it is not. R i is correlated with both X i and n i. Looking at Equation 13.26, we see that this implies that R i is correlated with the disturbance m i because the disturbance contains (n i X i ). R i is not a valid instrument for estimating the mean coefficient on released prisoners. However, it is striking to see what using R i as an instrument obtains as an estimate. We learned in Section 13.1 that, with no intercept term, the first step in constructing an IV estimator is weighting observations by the instrument, here R i. In the present model, there is an intercept term, so, as in OLS, we weight by r i instead of by R i : r i Y i = b 0 r i + b 1 r i X i + r i (d 2 D 2i X i + e i ). (Notice that b 0 a r i = 0. It is this trait that warrants weighting by r i instead of by R i. The estimator of the coefficient on X is then uninfluenced by b 0.) Summing

6 EXT 4-12 Web Extension 4 both sides of this expression and dividing through by a r ix i leads to the IV estimator, b IV = ar iy i a r ix i. (Recall that if a r i = 0, then a r i i = a r i n i.) We also learned in Section 13.1 that the key determinant of IV s consistency is a zero correlation between the instrument, here R i, and the disturbance, which here is (d 2 D 2i X i + e i ). What is E[r i (d 2 D 2i X i + e i )] = E[r i d 2 D 2i x i ] + E[r i e i ]? By assumption, the second term on the right-hand side of Equation is zero the size of the religious community is uncorrelated with the disturbances in the crime rate. But what of the first term on the right-hand side? The expected value of r i d 2 D 2i X i has two parts, the contribution to that mean when D 2i is 0, and the contribution when D 2i is 1: E(r i d 2 D 2i X i ) = E(r i d 2 D 2i X i ƒ D 2i = 0)Pr(D 2i = 0) + E(r i d 2 D 2i X i ƒ D 2i = 1)Pr(D 2i = 1). The first expectation is zero because in it D 2i = 0. The second expectation is also zero, because in it D 2i = 1 and the E(r i x i ƒ D 2i = 1) = 0 by assumption, so E(r i d 2 D 2i X i ƒ D 2i = 1) = d 2 E(r i X i ƒ D 2i = 1) = 0. IV estimation using R i consistently estimates Equation IV estimation consistently estimates not the b of Equation 13.26, but the b 1 of Equation If our interest is in the b of Equation 13.26, R i is a poor instrument choice. If our interest is, instead, in the b 1 of Equation 13.27, R i is an excellent instrument choice. Variable coefficients models make the choice of instruments more complicated than is the case for ordinary regression models. Local Average Treatment Effects IV estimation using the size of the church group consistently estimates the coefficient on the number of prisoners released in a benign program. It tells us nothing about the mean coefficient on prisoner releases or about the coefficient on predator prisoner releases. The intuition for this result is that the co-movement between the instrument and the troublesome X is limited, or localized, to

7 Local Average Treatment Effects EXT 4-7 co-movement in benign programs, and consequently, the IV can only tell us about the effects of changing X in those programs. Because the effect of X is different in some unspecified way for predator release programs, knowing the effect in the benign programs does not tell us about the effect of X in the predator programs. If the benign prisoner release program coefficient were random, instead of fixed at b 1, we could write for communities with benign release programs, b 1 = b 1 + n 1i. IV estimation using R i as the instrument would consistently estimate the mean coefficient on X for the benign prisoner releases, b 1, if R were uncorrelated with n 1i. We call the mean coefficient on X (here, b 1 ) for a subset (here, benign prisoner release programs) of the total population (here, all prisoner release programs) a local average effect. When the explanatory variable with a random coefficient is a dummy variable that indicates participation in a program, such as Head Start, or is a measure of how much of a treatment has been given a subject, for example, how much of a new fertilizer was applied to a field, econometricians call the local average effect the local average treatment effect (LATE). 1 What a local average treatment effect measures is not always so tidy as in this example. The instrument R cleanly separated benign program communities from predator community programs. The number of prisoners released and R were correlated only in the benign program communities. Suppose instead we had used the American Civil Liberties Union (ACLU) membership in each community as our instrument, and suppose the ACLU was more, but not exclusively, inclined to press for releases in states with benign release programs. The average effect we would consistently estimate with this instrument would be some weighted average of the coefficients for the benign and predator programs, with benign programs weighted more heavily. We would need to know more about the correlations between ACLU membership and prisoner releases in the two types of communities to determine the weights in that average. Similarly, if R were correlated with n i, in the case in which benign program communities differ in their coefficients, the LATE estimated consistently by IV using R would be some other weighted average of the benign program community coefficients besides b 1. This prisoner release program example teaches an important lesson. When the coefficients on the explanators are random, choosing suitable instruments and interpreting what they consistently estimate is a subtle intellectual exercise. When we perform IV estimation in random coefficients models, we must carefully consider the nature of the correlation between our instrument and the troublesome explanator, so that we can correctly interpret the coefficient estimate that we obtain as (i) an average coefficient for the population, or (ii) a local average effect for a subset of the population, or (iii) some other weighted average of the coefficients in the population.

8 EXT 4-14 Web Extension 4 Concepts for Review Random coefficients Local average effect Local average treatment effect (LATE) Endnotes 1. Guido Imbens and Joshua Angrist, Identification and Estimation of Local Average Treatment Effects, Econometrica 62 (1994):