Notes on causal effects

Transcription

1 Notes on causal effects Johan A. Elkink March 4, Decomposing bias terms Deriving Eq in Morgan and Winship (2007: 46): { Y1i if T Potential outcome = i = 1 Y 0i if T i = 0 Using shortcut E 11 = E[Y 1i T i = 1], E 01 = E[Y 0i T i = 1], etcetera: E[δ] ={πe 11 +(1 π)e 10 } {πe 01 +(1 π)e 00 } =π{e 11 E 01 }+(1 π){e 10 E 00 } =πe 11 πe 01 +(1 π)e 10 (1 π)e 00 =E 11 (1 π)e 11 E 01 +(1 π)e 01 +(1 π)e 10 E 00 +πe 00 E 11 E 00 =E[δ]+(1 π)e 11 +E 01 (1 π)e 01 (1 π)e 10 πe 00 =E[δ]+(1 π)e 11 +E 01 (1 π)e 01 (1 π)e 10 E 00 +(1 π)e 00 =E[δ]+(E 01 E 00 )+(1 π){(e 11 E 01 ) (E 10 E 00 )} =E[δ]+(E 01 E 00 )+(1 π){e[δ T i = 1] E[δ T i = 0]}, where π is the proportion of the population where T = 1 and δ is the causal effect (or treatment effect). (E 11 E 00 ) is the observed difference in the data, or the naive estimator of δ. (E 01 E 00 ) can then be called the baseline bias (Morgan and Winship 2007: 46) or selection bias (Angrist and Pischke 2009: 14-15), and (1 π){e[δ T i = 1] E[δ T i = 0]} the differential treatment effect bias (Morgan and Winship 2007: 45). Note that Angrist and Pischke (2009: 14-15) derive a slightly different equation, focusing only on the average treatment effect on the treated (ATT) instead of the overall average treatment effect (ATE), which allows them to ignore the differential treatment effect. References include those from the accompanying slides. 1

2 2 Implicit weighting in matching and regression Regression In regression analysis, we estimate the causal effect using y = Xβ +δt+ε. (1) In the derivation of the OLS estimator, we saw that this can be seen as the solution of a set of simultaneous equations, i.e. if X includes t, X Xˆβ OLS = X y could be written as: n+ ˆβ 2 xi2 + + ˆβ k xik = y i xi2 + ˆβ 2 (xi2 ) ˆβ k xi2 x ik = x i2 y i xi3 + ˆβ 2 xi3 x i2 + + ˆβ k xi3 x ik = x i3 y i xik + ˆβ 2 xik x i2 + + ˆβ k (xik ) 2 = x ik y i. In a simple regression, with only one independent variable, e.g. y = α+δt+ε, this leads to ti y i n tȳ ˆδ = = cov(y i,t i ) t 2 i n t 2 var(t i ). From the Frish-Waugh-Lovell theorem (Frisch and Waugh 1933) we know that regressing Y onx 1 andx 2 gives thesame coefficient estimate forx 1 asregressing Y ontheresiduals of regressing X 1 on X 2. Therefore, in Eq. (1), ˆδ R = cov(y i, t i ) var( t i ), where t i is a residual from regression t i = x i β + t i. This is known as the regression anatomy formula (Angrist and Pischke 2009: 74). Assuming that the regression is saturated, i.e. we have a dummy variable for each possible value of x (Angrist 1998: 256, fn 2

3 11): y i = E[y i t i,x i ]+ε i saturated model = E[y 0i x i ]+E[y 1i y 0i x i ]t i +ε i assumption of unconfoundedness = E[y 0i x i ]+δ x t i +ε i cov(y i, t i ) = E[y i t i ] E[y i ]E[ t i ] definition of covariance = E[y i t i ] in OLS, E[ε] = 0 = E[y i (t i E[t i x i ])] definition of residuals = E[(E[y 0i x i ]+δ x t i +ε i )(t i E[t i x i ])] = E[(E[y 0i x i ]+δ x t i )(t i E[t i x i ])] assuming cov(ε i,t i ) = 0 = E[E[y 0i x i ](t i E[t i x i ])]+E[δ x t i (t i E[t i x i ])] = E[δ x t i (t i E[t i x i ])] assuming no baseline bias = E[δ x (t i E[t i x i ]) 2 ] T is binary var( t i ) = E[( t i E[ t i ]) 2 ] definition of variance = E[ t 2 i ] in OLS, E[ε] = 0 = E[(t i E[t i x i ]) 2 ] definition of residuals ˆδ R = cov(y i, t i ) var( t i ) = E[(t i E[t i x i ]) 2 δ x ] E[(t i E[t i x i ]) 2 ] = E[E[(t i E[t i x i ]) 2 x i ]δ x ]. E[E[(t i E[t i x i ]) 2 x i ]] Using the fact that the variance of T is the variance of a Bernoulli distribution and thus p(1 p): ˆδ R = xδ xp(t i = 1 X i = x)(1 P(T i = 1 X i = x))p(x i = x) x P(T i = 1 X i = x)(1 P(T i = 1 X i = x))p(x i = x). The regression estimator ˆδ R thus provides a weighted estimator of the average treatment effect δ, weighted by the variance of the treatment, conditional on X (Angrist 1998; Angrist and Pischke 2009; Morgan and Winship 2007). The variance of a binary variable such as T is p(1 p), thus in this particular case, π(x)(1 π(x)), with π(x) the propensity score. This variance is highest where π(x) is close to 0.5, thus for values of X where there are the same number of treated as untreated cases; the variance is lowest where π(x) is close to 0 or 1. If π(x) = 0 or π(x) = 1, the case gets a zero weight and is excluded from the estimation of ˆδ so where there is lack of overlap, no contribution is made to the estimation. Matching We can apply a similar derivation for the matching estimator, using Bayes formula: P(X i = x T i = 1) = P(T i = 1 X i = x)p(x i = x), P(T i = 1) 3

4 so that we get: ˆδ M ATT = x δ x P(X i = x T i = 1) = P(T i = 1 X i = x)p(x i = x) δ x P(T x i = 1) = xδ xp(t i = 1 X i = x)p(x i = x) x P(T i = 1 X i = x)p(x i = x), so while ˆδ R is weighted by the variance of the treatment, conditional on X, ˆδM ATT is weighted by the probability of treatment, conditional on X, or the propensity score π(x). The matching estimator for ATE, ˆδ ATE M can then be seen as the unweighted estimator. Conclusion A regression estimator is a weighted estimator of the average treatment effect, which in cases where the treatment effect differs strongly by different values of the control variable(s), leads to a biased result. Otherwise, matching is quite similar to a fully saturated regression. Often, we of course do not use a fully saturated specification in our regression, so that our controls become a lot weaker, and the difference with a matching estimator larger. References Achen, Christopher H Let s put garbage-can regressions and garbage-can probits where they belong. Conflict Management and Peace Science 22: Angrist, Johua D Estimating the labor market impact of voluntary military service using social security data on military applicants. Econometrica 66(2): Angrist, Joshua D. and Jörn-Steffen Pischke Mostly harmless econometrics: An empiricist s companion. Princeton: Princeton University Press. Frisch, Ragnar and Frederick V. Waugh Partial time regressions as compared with individual trends. Econometrica 1(4): Gelman, Andrew and Jennifer Hill Data analysis using regression and multilevel/hierarchical models. Analytical Methods for Social Research Cambridge: Cambridge University Press. Gerring, John Social science methodology: a critical framework. Cambridge University Press. Gerring, John Social science methodology: A unified framework. Cambridge: Cambridge University Press. Imai, Kosuke, Gary King and Elizabeth A. Stuart Misunderstandings between experimentalists and observationalists about causal inference. Journal of the Royal Statistical Society A 171:

5 Imbens, Guido W Nonparametric estimation of average treatment effects under exogeneity: A review. Review of Economics and Statistics 86(1):4 29. Kennedy, Peter A guide to econometrics. 6th ed. Malden, MA: Blackwell. Lee, David S Randomized experiments from non-random selection in U.S. House elections. Journal of Econometrics 142(2). Lee, Myoung Jae Micro-econometrics for policy, program, and treatment effects. Oxford: Oxford University Press. Morgan, Stephen L. and Christopher Winship Counterfactuals and causal inference. Methods and principles for social research. New York: Cambridge University Press. Morgan, Stephen L. and David J. Harding Matching estimators of causal effects: Prospects and pitfalls in theory and practice. Sociological Methods & Research 35(1):3 60. Pearl, Judea Causality: Models, reasoning, and inference. Cambridge: Cambridge University Press. Rubin, Donald B Which ifs have causal answers (Comment on Statistics and causal inference by Paul W. Holland). Journal of the American Statistical Association 81: Smith, Jeffrey A. and Petra E. Todd Does matching overcome Lalonde s critique of nonexperimental estimators? Journal of Econometrics 125: