Advanced Quantitative Methods: Causal inference

Transcription

1 Advanced Quantitative Methods: Johan A. Elkink University College Dublin 2 March 2017

2 1 2 3

3 1 2 3

4 Inference In regression analysis we look at the relationship between (a set of) independent variable(s) and a dependent variable. Statistical is concerned with the question how likely it is to observe this relationship given the null hypothesis of no relationship (frequentist) or how much we should update our beliefs concerning this relationship given our new evidence (Bayesian). A different question is whether or not we can deduce that the independent variable is a cause of the dependent one, that is, drawing a causal.

5 correlation causation

6 Many possible patterns

7 Many possible patterns

8 If healthier patients take a drug and sicker patients do not, we can find an association between drug and recovery even when the drug does not work. If sicker patients take a drug and healthier patients do not, we might not find an association between drug and recovery even when the drug works. association causation

9 Marijuana Imagine you have conducted a study on college students and have found an inverse correlation between marijuana smoking and grade point average that is, those who smoke tend to have lower GPAs than those who do not, and the more smoked, the lower the GPA.

10 Marijuana Possible conclusion: marijuana use must be affecting the mental abilities, and hence grades.

11 Marijuana But perhaps the frustration of poor grades leads to escapist behaviour, e.g., marijuana use.

12 Marijuana Or, a difficult emotional state, or a lot of stress, causes one to use marijuana, and also affects grades.

13 Marijuana Or, perhaps the group of people you studied just happened to have lower GPAs among the marijuana users, purely by chance.

14 Fundamental problem Imagine, there are two kinds of people, one group, T = 1, that has a college degree, and another group, T = 0, that does not. We want to measure where a college degree leads to a higher salary, Y. What we would like to know is the difference for any individual i whether they have a college degree or not: Y T i =1 i Y T i =0 i. However, for every individual i, we either observe Y T i =1 i, or we observe Y T i =0 i they either have the degree or they don t.

15 We wish... Respondent Degree Y T i =0 i effect 1 Yes Yes No No Yes Yes No Yes i Y T i =1

16 We wish... we have... Respondent Degree Y T i =0 i 1 Yes Yes No 90 4 No 87 5 Yes Yes No 92 8 Yes 109 i Y T i =1 effect

17 outcome = { Y1i if T i = 1 Y 0i if T i = 0 E.g., Y 1i is the salary of individual i had (s)he a college degree, irrespective of whether (s)he actually does. Y i = Y 0i + (Y 1i Y 0i )T i = Y 0i + δt i, where δ = Y 1i Y 0i is the causal effect. (Angrist and Pischke, 2009, 13 14)

18 Average treatment effect Because it is impossible to observe individual treatment effect, we usually turn to average treatment effect: E[δ] = E[Y 1i Y 0i ] = E[Y 1i ] E[Y 0i ], which we could naively estimate with ˆδ = E[Y 1i T i = 1] E[Y 0i T i = 0]. This assumes that E[Y 1i ] reflects the salary for people with a college degree, irrespective of whether they got one or not, and that E[Y 0i ] reflects the salary without a college degree, irrespective of whether they got one or not.

19 Counterfactual causality By making such assumptions by looking at the ATE we are making a counterfactual argument. We are making assumptions of what Y 1i would have been, had i had a college degree. This implies that we cannot measure a causal effect, only estimate it. To understand when the ATE assumptions are reasonable, we need to look at the effect of covariates other that relate to Y, which we will denote by X.

20 Treatment effect: abbreviations ATE Average Treatment Effect E[δ] = E[Y 1i Y 0i ] ATT ATE for the Treated E[Y 1i Y 0i T i = 1] ATC ATE for the Control (untreated) E[Y 1i Y 0i T i = 0] PATE Population ATE E[Y 1i Y 0i ] SATE Sample ATE E n [Y 1i Y 0i ] LATE Local ATE E[Y 1i Y 0i X i = x] CATE Conditional ATE E[Y 1i Y 0i X i = x] and analogously we have LATT, PATT, SATC, etceta. Note that ATE, ATT, ATC implicitly refer to population values. E n [ ] is the sample mean, i.e. E n [x] = x = 1 n n i=1, where the E n allows for the formulation of conditional and counterfactual means.

21 Bias in causal Using shorthand E 01 = E[Y 0i T i = 1], etc., and taking π as the population proportion that received the treatment, E[δ] = πe[δ T i = 1] + (1 π)e[δ T i = 0] = π(e 11 E 01 ) + (1 π)(e 10 E 00 )

22 Bias in causal Using shorthand E 01 = E[Y 0i T i = 1], etc., and taking π as the population proportion that received the treatment, E[δ] = πe[δ T i = 1] + (1 π)e[δ T i = 0] = π(e 11 E 01 ) + (1 π)(e 10 E 00 ) can be decomposed into (E 11 E 00 ) = E[δ] + (E 01 E 00 ) + (1 π){(e 11 E 01 ) (E 10 E 00 )}.

23 Bias in causal Using shorthand E 01 = E[Y 0i T i = 1], etc., and taking π as the population proportion that received the treatment, can be decomposed into E[δ] = πe[δ T i = 1] + (1 π)e[δ T i = 0] = π(e 11 E 01 ) + (1 π)(e 10 E 00 ) (E 11 E 00 ) = E[δ] + (E 01 E 00 ) + (1 π){(e 11 E 01 ) (E 10 E 00 )}. (E 11 E 00 ) observed difference in effect E[δ] average treatment effect (E 01 E 00 ) selection bias (1 π){(e 11 E 01 ) (E 10 E 00 )} differential treatment effect bias (Morgan and Winship, 2007)

24 SUTVA The stable unit treatment value assumption SUTVA is simply the a priori assumption that the value of Y for unit i when exposed to treatment t will be the same no matter what mechanism is used to assign treatment t to unit i and no matter what treatments the other units receive. (Rubin 1986, 961, as cited in Morgan and Winship (2007, 37))

25 Propensity score The propensity score is the conditional probability of receiving the treatment, given control in X: π(x) P(T = 1 X = x) = E[T X = x]. (Imbens, 2004, 6)

26 When studying effect of, say, T on Y, by examining the statistical association between the two, we need to ascertain that the observed effect is not caused by a third variable, say, X. We can say that T and Y are confounded when there is a third variable X that influences both T and Y ; such a variable is then called a confounder of T and Y. (Pearl, 2000, )

27 Another way of saying this is that if E(Y T, X ) E(Y T ) and E(T X ) E(T ), X is a confounder of the effect of T on Y. Note that confounding is a causal concept, not an associational one! X has to have a causal effect on T and X has to have a causal effect on Y for there to be an issue. (Pearl, 2000; Lee, 2005, 44)

28 1 2 3

29 X affects both T and Y = control (Lee, 2005, 43 48)

30 Do control This is the typical case of a confounding factor, and hence should be eliminated through controlling.

31 X affects both T and Y = control T affects Y, which in turn affects X = do not control (Lee, 2005, 43 48)

32 Don t control In this case, X is an effect of Y. By controlling for X, you can severily underestimate the effect of T on Y. Imagine that a college degree leads to a better income leads to a nicer car. Controlling for the price of the car in estimating the effect of having a college degree on income might cancel the effect.

33 X affects both T and Y = control T affects Y, which in turn affects X = do not control T affects X, which in turn affects Y = do not control... (Lee, 2005, 43 48)

34 Don t control To get the overall effect of T on Y, you want to include the effect through X. E.g. if you want to know the effect of changing the policy regarding smoking in pubs on the amount of smoking in general, you do not care through what mechanism this happened (through peer pressure, laziness, etc.), but only about the overall effect.

35 X affects both T and Y = control T affects Y, which in turn affects X = do not control T affects X, which in turn affects Y = do not control unless you explicitly want only the direct effect (Lee, 2005, 43 48)

36 Maybe control A scholarship for poorer students might help them to get a college degree, which in turn might help them to earn more money later in life. Having a scholarship on your CV, however, might also further your career, independent of the effect of having a college degree. To see the overall effect of the scholarship, don t control on having a college degree. To see the effect of having a scholarship, independent of the effect of getting a college degree, do control for college degree.

37 X affects both T and Y = control T affects Y, which in turn affects X = do not control T affects X, which in turn affects Y = do not control unless you explicitly want only the direct effect X affects Y, but not T, nor the effect of T on Y (Lee, 2005, 43 48)

38 Maybe control When X affects Y, but not T, there is no confounding issue and the estimates for the effect of T on Y should not be affected by inclusion of X. However, including X in the model can still help for efficiency. (Gelman and Hill, 2007, 177)

39 Kitchen sink A typical approach in the quantitative social sciences is to collect a number of different theories / hypotheses, add them all as to a regression, and see who wins. This is the kitchen sink approach (or garbage can approach). If anything, the above discussion should have made clear that to draw causal s, a clear distinction of treatment from covariates is crucial. In other words: focus your research! (Note that the garbage can phrase has also been used to argue against ignoring nonlinearities (Achen, 2005), as opposed to careless specification of the causal effect.)

40 Causal diagrams The preceding examples underline how it is important to always draw out the causal diagram and consider carefully how you select cases and select controls when making causal s.

41 X- and Y-centered research Another way of putting the issue is that the above is all about trying to study the effect of a cause (treatment), rather than the cause of an effect. The latter is perhaps ill-defined and runs into the infinite regress of causation (Gelman and Hill, 2007, 187). (See Gerring (2001, 2012) for an extensive discussion of Y -centered and X -centered research.)

42 1 2 3

43 The ideal experiment To avoid any effect of covariates the ideal is to randomly select participants for your research from the overal population (enables to the population) and to randomly assign the treatment to these participants (enables causal ).

44 Controls Randomized assignment Blocking Multiple regression (Imai, King and Stuart, 2008)

45 The ideal experiment To avoid any effect of covariates the ideal is to randomly select participants for your research from the overal population (enables to the population) and to randomly assign the treatment to these participants (enables causal ).

46 Randomized selection Random selection, or random sampling, enables s from the sample to the overall population. It does not affect whether someone is treated or not, so it does not affect the causal. It helps drawing s from sample average treatment effect (SATE) to population average treatment effect (PATE). (Imai, King and Stuart, 2008)

47 Randomized assignment When the treatment is entirely randomly assigned, any effect of covariates, observed or unobserved, on the treatment, is removed - the covariates are now ignorable. P(T X, ε) = P(T ) Hence, randomized assignment is useful for causal s, either to the overall population (random selection) or the sample (non-random selection).

48 Randomized assignment In this case, the estimate of the effect is simply the difference in means ȳ T i =1 i ȳ T i =0 i, or ˆβ in the simple regression Y i = α + βt i + ε i. Adding pre-treatment covariates can still increase the efficiency of this estimator, however.

49 Blocking Blocking is the random assignment of units to treatment and control groups within strata (blocks) that are defined by a set of observed pretreatment covariates. Blocking removes the effect of covariates for the covariates we use to block. So it does not assist in dealing with unobserved confounders. P(T X, ε) = P(T ε) (Imai, King and Stuart, 2008, 486)

50 Unconfoundedness The assumption of unconfoundedness states that, conditional on X, the treatment assignment is ignorable: (Y 0, Y 1 ) T X. Alternative names: ignorable treatment assignment, conditional independence assumption, selection on observables, exogeneity assumption. When only interested in ATT instead of ATE, unconfoundedness for controls suffices: Y 0 T X. (Imbens, 2004, 7 8)

51 Overlap The assumption of overlap states that we have both treated and untreated observations across the values of X : 0 < P(T = 1 X ) < 1. When only interested in ATT instead of ATE, weak overlap suffices: P(T = 1 X ) < 1. (Imbens, 2004, 7 8)

52 A causal effect can be estimated using regression y = Xβ + δt + ε, with ˆδ OLS the estimate of the causal effect, controlling for X.

53 Sampling strategy: match control cases to treatment cases strategy: nonparametric use of controls (Similar to blocking, but post-experiment.) (Imai, King and Stuart, 2008; Morgan and Harding, 2006, 5 6)

54 With matching, we match each observation where T i = 1 with a set of M corresponding observations where T j = 0. and vice versa for Ŷ 1i. { Yi if T i = 0 Ŷ 0i = j Y j if T i = 1 1 M Our simple matching estimator of ATE is then ˆδ ATE SM = 1 n i (Ŷ 1i Ŷ 0i ). (Imbens, 2004, 14)

55 An alternative formulation to the same effect is ˆδ ATE M = δ x P(X i = x) x ˆδ ATT M = δ x P(X i = x T i = 1), x where δ x = E n [y i t i = 1, x i = x] E n [y i t i = 0, x i = x]. Here ˆδ is a weighted mean of the means of substrata, weighted by the probability of being in that substratum. This is comparable to the previous, if each treated case is matched with all control cases in the same substratum. (Angrist and Pischke, 2009, 71 72)

56 : example Imagine we have a treatment variable T, a control variable X with two categories, and a dependent variable Y. We will require the estimated proportions (left) and mean of y (right) for each combination of T, X : ˆδ M ATE = T = 0 T = 1 T = 0 T = 1 X = X = δ x P(X i = x) =.44 (40 20) +.56 (120 80) = 31.2 ˆδ ATT M = δ x P(X i = x T i = 1) x = (40 20) + (120 80) = ˆδ OLS = 33.5 x

57 vs regression Note that the regression coefficient can also be obtained using: ˆδ OLS = x δ xp(t i = 1 X i = x)(1 P(T i = 1 X i = x))p(x i = x) x P(T i = 1 X i = x)(1 P(T i = 1 X i = x))p(x i = x) (40 20).44(1.44).44 + (120 80).56(1.56 = ) (1.44) (1.56 ).56 = 33.5

58 vs regression ˆδ OLS = x δ xp(t i = 1 X = x)(1 P(T i = 1 X = x))p(x = x) x P(T i = 1 X = x)(1 P(T i = 1 X = x))p(x = x) ˆδ ATT M = x δ xp(t i = 1 X = x)p(x = x) x P(T i = 1 X = x)p(x = x) ˆδ ATE M = δ x P(X = x), x so while ˆδ R is weighted by the variance of the treatment, conditional on X, ˆδ ATT M is weighted by the probability of treatment, conditional on X, and ˆδ ATE M can be seen as the unweighted estimator. (See handout for derivation and references.)

59 Distance metrics The term corresponding here is not specified and there are many options: Use Euclidean distances in X ;... weighted by the (diagonal of the) inverse of their covariances;... weighted by the coefficient on estimating the propensity score;... weighted by the coefficients in a linear regression explaining Y ; etc. (Imbens, 2004, 14 15)

60 Using propensity scores As alternative to using distances in X, one can use the propensity score: By weighting the sample on the reciprocal of π(x); Divide the sample in subsamples with approximately the same π(x) ( blocking ); By adding π(x) as a covariate in a regression model; By matching cases based on closeness in π(x). π(x) typically needs to be estimated, e.g. using a logit model. Propensity scores help with many-valued control and they address the issue of lack of overlap. (Imbens 2004, 16; Morgan and Harding 2006, 22 28

61 Exact matching Here all cases where T i = 1 are matched with M cases where T j = 0 for which X i = X j. E.g. one can randomly select one control for each treated from all cases where X i = X j (so M = 1). For the ATT we can then use ˆδ ATT EM = 1 (y i t i = 1) n 1 i j w ij (y j t j = 0), where n 1 is the number of treated cases and w ij is a weight calculated as 1/M ij for the matched control and 0 otherwise. (Note that this is a reformulation of the estimator we used in the above example.) (Morgan and Harding, 2006, 31)

62 Nearest neighbour matching Same as exact matching, except instead of looking for controls where X i = X j, we match with the nearest control on some dimension. The most common would be the propensity score, matching each treated case with one or more controls that are closest in π(x). The weights are then calculated as in exact matching. Interval matching is very similar, bundling cases within a particular range and then selecting controls from within the same group. (Morgan and Winship, 2007, )

63 Kernel matching Same as exact matching, but all treated are matched against all controls, with weights such that those closest to the treatment case are given the greatest weight: [ ] π(xj ) π(x G i ) a n w ij = [ j G π(xj ], ) π(x i ) a n where a n is a bandwidth parameter that scales the difference in the estimated propensity scores based on the sample size and G[ ] is a kernel function. (A kernel function needs to have a mean of zero and integrate to 1, for example, Smith and Todd (2005) use G(s) = (s2 1) 2 for s 1, G(s) = 0 otherwise.) (Morgan and Winship, 2007, 109)

64 : final points, blocking, matching all control for observed confounders. are similar, but differ in their weighting of cases. can be combined, using regression on matched data ( doubly robust ). Choice of matching algorithm significantly affects estimates extensive research is still ongoing into comparing these estimators. (Imbens, 2004; Morgan and Winship, 2007)

65 Sharp RD is applicable in situations where a continuous variable determines treatment, as in: { 1 if xi x T i = 0 0 if x i < x 0, with x 0 a known cutoff point. Note that this is the opposite of the overlap assumption in matching. (Angrist and Pischke, 2009, )

66 RD estimation Estimation of Sharp RD can be very straightforward, e.g. by running the regression y = Xβ + ρt + ε, where X contains the variable determining T. Nonlinear extensions are straightforward, e.g. X can include quadratic terms. (Angrist and Pischke, 2009, 253)

67 RD example: incumbent advantage (Lee, 2008, Fig. 4)

68 RD extensions: local regression A nonlinear relationship between X and Y can occassionally be mistaken for a regression.focusing on the data close to x 0, or weighting this data more heavily in the regression, can address this issue. Typically, nonparametric RD designs are used in this context. The risk is inefficiency and model sensitivity due to lack of data near the cutoff point. (Angrist and Pischke, 2009, 256)

69 Mistaken RD Nonlinearity mistaken for RD: (Angrist and Pischke, 2009, 254)

70 RD extensions: local regression A nonlinear relationship between X and Y can occassionally be mistaken for a regression.focusing on the data close to x 0, or weighting this data more heavily in the regression, can address this issue. Typically, nonparametric RD designs are used in this context. The risk is inefficiency and model sensitivity due to lack of data near the cutoff point. (Angrist and Pischke, 2009, 256)

71 RD extensions: fuzzy RD Fuzzy RD refers to a model where where g 1 (x i ) g 0 (x i ). { g1 (x P(T i = 1 x i ) = i ) if x i x 0 g 0 (x i ) if x i < x 0 This type of model can be estimated using a polynomial of X and/or an indicator for the cutoff point as instruments for T. (Angrist and Pischke, 2009, )

72 1 2 3

73 The situation where cor(x, ε) 0 is called endogeneity. has three main causes: Measurement error in X Simultaneity or reverse causation Omitted

74 The idea of instrumental variable (IV) estimation is to find a variable which is correlated with x, but not with ε, and that has no causal relationship with y other than through x, such that any effect it has on y must be attributed to x. y i = β 0 + β 1 x i + ε i x i = γ 0 + γ 1 z i + v i, assuming cov(x i, ε i ) 0, x is a cause of y, but z is not, and cov(z i, ε i x i ) = 0, then z is a good instrument for x.

75 : 2SLS A simple procedure to estimate IV is to use two-stage least (2SLS): Regress x on z plus all other independent ; Regress y on ˆx plus all other independent. The ˆβ on ˆx is the estimate for β.

76 : 2SLS ˆβ OLS IV = (ˆX ˆX) 1 ˆX y = (X Z(Z Z) 1 Z X) 1 X Z(Z Z) 1 Z y If X and Z have the same dimensions, this reduces to: ˆβ OLS IV = (Z X) 1 Z y

77 : 2SLS Because only the information in x which overlaps with z is used, standard errors will be larger. Estimation: V ( ˆβ IV OLS ) = ˆσ2 (ˆX ˆX) 1 = ˆσ 2 (X Z(Z Z) 1 Z X) 1 e e ˆσ 2 = n k = (y X ˆβ OLS IV ) (y X ˆβ OLS IV ) n k Note the use of X, not ˆX, to determine ˆσ 2.

78 Weak instruments x and z will correlate, but not perfectly. If they only weakly correlate, we call z a weak instrument. If instruments are weak: bias of IV is large (but IV is still consistent); variances are underestimated; estimation becomes unreliable; even very weak endogeneity in instrument can make IV worse than OLS.

79 H 0 : ˆβ OLS = ˆβ OLS IV H 1 : ˆβ OLS ˆβ OLS IV If H 0 is rejected, only IV estimator is consistent. H = ( ˆβ IV ˆβ OLS ) (V ( ˆβ IV ) V ( ˆβ OLS )) 1 ( ˆβ IV ˆβ OLS ) H χ 2 (rank(v ( ˆβ IV ) V ( ˆβ OLS ))

80 ATE vs LATE One concern with IV estimation is the distinction between average treatment effect (ATE) and local average treatment effect (LATE). We assume that the effect of x on y is constant across subgroups in the population. With IV, we also assume that this does not change when only looking at the variation in x explained by z. If for some groups in the population x depends on z, but for other groups it does not, then our estimate of β is only valid for the former group (i.e. LATE). If there is no such correlation, the estimate applies to the entire population (ATE). One solution to reduce the difference between ATE and LATE would be to add more instruments for the same x. More instruments increases the risks of introducing endogeneity in the instruments (Kennedy, 2008).

81 Achen, Christopher H Let s put garbage-can regressions and garbage-can probits where they belong. Conflict Management and Peace Science 22: Angrist, Joshua D. and Jörn-Steffen Pischke Mostly harmless econometrics: An empiricist s companion. Princeton: Princeton University Press. Gelman, Andrew and Jennifer Hill Data analysis using regression and multilevel/hierarchical models. Analytical Methods for Social Research Cambridge: Cambridge University Press. Gerring, John Social science methodology: A critical framework. Cambridge: Cambridge University Press. Gerring, John Social science methodology: A unified framework. Cambridge: Cambridge University Press. Imai, Kosuke, Gary King and Elizabeth A. Stuart Misunderstandings between experimentalists and observationalists about causal. Journal of the Royal Statistical Society A 171: Imbens, Guido W Nonparametric estimation of average treatment effects under exogeneity: A review. Review of Economics and Statistics 86(1):4 29. Kennedy, Peter A guide to econometrics. 6th ed. Malden, MA: Blackwell. Lee, David S Randomized experiments from non-random selection in U.S. House elections. Journal of Econometrics 142(2). Lee, Myoung Jae Micro-econometrics for policy, program, and treatment effects. Oxford: Oxford University Press. Morgan, Stephen L. and Christopher Winship Counterfactuals and causal. Methods and principles for social research. New York: Cambridge University Press. Morgan, Stephen L. and David J. Harding estimators of causal effects: Prospects and pitfalls in theory and practice. Sociological Methods & Research 35(1):3 60. Pearl, Judea Causality: Models, reasoning, and. Cambridge: Cambridge University Press. Rubin, Donald B Which ifs have causal answers (Comment on Statistics and causal by Paul W. Holland). Journal of the American Statistical Association 81: Smith, Jeffrey A. and Petra E. Todd Does matching overcome Lalonde s critique of nonexperimental estimators? Journal of Econometrics 125: