Regression Discontinuity Designs Kosuke Imai Harvard University STAT186/GOV2002 CAUSAL INFERENCE Fall 2018 Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 1 / 1
Observational Studies In many cases, we cannot randomize the treatment assignment ethical constraints logistical constraints But, some important questions demand empirical evidence even though we cannot conduct randomized experiments! Designing observational studies find a setting where credible causal inference is possible Key = Knowledge of treatment assignment mechanism Regression discontinuiety design (RD Design): 1 Sharp RD Design: treatment assignment is based on a deterministic rule 2 Fuzzy RD Design: encouragement to receive treatment is based on a deterministic rule Originates from a study of the effect of scholarships on students career plans (Thistlethwaite and Campbell. 1960. J. of Educ. Psychol) Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 2 / 1
Regression Discontinuity Design Idea: Find an arbitrary cutpoint c which determines the treatment assignment such that T i = 1{X i c} 828 QUARTERLY JOURNAL OF ECONOMICS Close elections as RD design (Lee et al. 2004. Q. J. Econ): FIGURE I Total Effect of Initial Win on Future ADA Scores: This figure plots ADA scores after the election at time t 1 against the Kosuke Imai (Harvard) Democrat vote share, time Regression t. Each Discontinuity circle the Design average ADA score Stat186/Gov2002 within 0.01 Fall 2018 3 / 1
Identification Estimand: E(Y i (1) Y i (0) X i = c) Assumption: E(Y i (t) X i = x) is continuous in x for t = 0, 1 deterministic rather than stochastic treatment assignment violation of the overlap assumption: 0 < Pr(T i X i = x) < 1 for all x RD design is all about extrapolation Regression modeling: E(Y i (1) X i = c) = lim E(Y i (1) X i = x) = lim E(Y i X i = x) x c x c E(Y i (0) X i = c) = lim E(Y i (0) X i = x) = lim E(Y i X i = x) x c x c Advantage: internal validity Disadvantage: external validity Make sure nothing else is going on at X i = c Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 4 / 1
Analysis Methods under the RD Design Simple linear regression within a window How should we choose a window in a principled manner? How should we relax the functional form assumption? higher-order polynomial regression not robust to outliers Local linear regression (same h for both sides): better behavior at the boundary than other nonparametric regressions (ˆα +, ˆβ + ) = argmin α,β (ˆα, ˆβ ) = argmin α,β n ( ) 1{X i > c}{y i α (X i c)β} 2 Xi c K h n ( ) 1{X i < c}{y i α (X i c)β} 2 Xi c K h i=1 i=1 Weighted regression with a kernel function of one s choice: uniform kernel: K (u) = 1 21{ u < 1} triangular kernel: K (u) = (1 u )1{ u < 1} Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 5 / 1
Optimal Bandwidth (Imbens and Kalyanaraman. 2012. Rev. Econ. Stud.) Choose the bandwidth by minimizing the MSE: MSE = E[{(ˆα + ˆα ) (α + α )} 2 X] = E{(ˆα + α + ) 2 X} + E{(ˆα α ) 2 X} 2 E(ˆα + α + X) E(ˆα α X) = (Bias + Bias ) 2 + Variance + + Variance Bias and variance of local linear regression estimator at the boundary: Bias = E( ˆm(0) X) m(0), Variance = V( ˆm(0) X) where m(x) = E(Y i X i = x), ˆm(x) = ˆα(x), and n ( ) (ˆα(x), ˆβ(x)) = argmin (Y i α β(x i x)) 2 Xi x K α,β h i=1 Refinements, e.g., bias correction (Calonico et al. 2014. Econometrica) Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 6 / 1
The As-if Random Assumption RD design does NOT require the local randomization or as-if random assumption within a window: {Y i (1), Y i (0)} T i c 0 X i c 1 The as-if random assumption implies zero slope of regression lines difference-in-means within the window The assumption may be violated regardless of the window size a Democratic experience advantage b Share of total spending by Democratic candidate Proportion candidates with advantage 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.05 0.00 0.05 Democratic margin Proportion of total spending 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.05 0.00 0.05 Democratic margin Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 7 / 1
Close Elections Controversy (de la Cuesta and Imai. 2016. Annu. Rev. Political Sci) Sorting? 1 Pre-election behavior or characteristics of candidates, e.g., resource advantages steep slope 2 Post-election advantages in vote tallying, e.g., election fraud sorting CQ rating Dem. donation % Dem. spending % % Urban Dem. # prev. terms Dem. share t 1 Dem. margin t 1 Dem. experience adv. Dem. win t 1 Dem. inc. in race Dem. sec. of state Open seat % Foreign born Dem. governor Dem.-held open seat Rep.-held open seat % Black % Voter turnout % Govt. worker Dem. pres. margin Rep. inc. in race Rep. experience adv. Partisan swing Inc.'s D1 nominate Rep. # prev. terms a Difference-in-means in window 1.5 1.0 0.5 0.0 0.5 1.0 1.5 Estimated discontinuity (standard deviation units for nonbinary measures) Dem. spending % Dem. donation % Dem. # prev. terms CQ rating % Urban Dem. share t 1 Dem. experience adv. Dem. win t 1 Dem. governor Open seat % Black Dem. margin t 1 Dem.-held open seat % Foreign born Dem. inc. in race Dem. sec. of state Rep.-held open seat % Voter turnout Dem. pres. margin Rep. inc. in race Inc.'s D1 nominate Partisan swing % Govt. worker Rep. experience adv. Rep. # prev. terms c Local linear regression 1.5 1.0 0.5 0.0 0.5 1.0 1.5 Estimated discontinuity (standard deviation units for nonbinary measures) Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 8 / 1
ARTICLE IN PRESS Density Test of Sorting (McCrary. 2008. J. Econom.) J. McCrary / Journal of Econometrics 142 (2008) 698 714 Frequency Count 150 120 90 60 30 0-1 -0.8-0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1 Democratic Margin 1.60 1.40 1.20 1.00 0.80 0.60 0.40 0.20 0.00 Density Estimate Democratic vote share relative to cutoff: popular elections to the House of Representatives, 1900 1990. 1 Create histogram with a selected bin size 2 Fit local linear regression to bin midpoints to smooth the histogram 3 Estimate the difference in the logged histogram height at the stimates threshold Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 9 / 1
Placebo Test 838 QUARTERLY JOURNAL OF ECONOMICS FIGURE V Specification Test: Similarity of Historical Voting Patterns between Bare Democrat and Republican Districts The panel plots one time lagged ADA scores against the Democrat vote share. Time t and t 1 refer to congressional sessions. Each point is the average lagged ADA score within intervals of 0.01 in Democrat vote share. The continuous line is from a fourth-order polynomial in vote share fitted separately for points above and below the 50 percent threshold. The dotted line is the 95 percent confidence interval. What is a good placebo? 1 expected not to have any effect 2 closely related to outcome of interest Lagged outcome future cannot affect past V.C. Sensitivity to Alternative Measures of Voting Records Interpretation: Our results failure so far to are reject basedthe on anull particular the voting null index, is correct the ADA score. In this section we investigate whether our results Kosuke Imai (Harvard) generalize to other voting Regression scores. Discontinuity We find Design that the findings Stat186/Gov2002 do not Fall 2018 10 / 1
Fuzzy RD Design (Hahn et al. 2001. Econometrica) Sharp regression discontinuity design: T i = 1{X i c} What happens if we have noncompliance? Forcing variable as an instrument: Z i = 1{X i c} Potential outcomes: T i (z) and Y i (z, t) Assumptions 1 Monotonicity: T i (1) T i (0) 2 Exclusion restriction: Y i (0, t) = Y i (1, t) 3 E(T i (z) X i = x) and E(Y i (z, T i (z)) X i = x) are continuous in x Estimand: Estimator: E(Y i (1, T i (1)) Y i (0, T i (0)) Complier, X i = c) lim x c E(Y i X i = x) lim x c E(Y i X i = x) lim x c E(T i X i = x) lim x c E(T i X i = x) Disadvantage: external validity Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 11 / 1
Class Size Effect (Angrist and Lavy. 1999. Q. J. Econ) Effect of class-size on student test scores Maimonides Rule: Maximum class size = 40 z f (z) = + 1 z 1 40 Class Size 10 15 20 25 30 35 40 Maimonides Rule Actual class size 0 50 100 150 200 Enrollment Count Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 12 / 1
Empirical Analysis Y i : class average verbal test score Window size: w Construction of forcing variable: 40 Z i if 40 w/2 Z i 40 + w/2 X i = 80 Z i if 80 w/2 Z i 80 + w/2.. Linear models (cluster standard errors by schools): T i = α 1 I{X i >= 0} + β 1 X i + γ 1 X i I{X i >= 0} + ɛ 1i Y i = α 2 I{X i >= 0} + β 2 X i + γ 2 X i I{X i >= 0} + ɛ 2i where ˆα 1 = 7.90 (s.e. = 1.90) and ˆα 2 = 0.056 (s.e. = 2.08) Two-stage least squares estimate: est. = 0.007 (s.e. = 0.261) Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 13 / 1
Summary Observational studies treatment assignment is not random Design observational studies for credible causal inference Sharp regression discontinuity designs: deterministic (rather than stochastic) treatment assignment rule continuity assumption no sorting does not require as-if random assumption limited external validity extrapolation required for generalization incumbency effects controversy (Eggers et al. 2015. Am. J. Political Sci) Importance of placebo tests Fuzzy regression discontinuity designs: noncompliance Suggested readings: ANGRIST AND PISCHKE, Chapter 6 IMBENS AND LEMIEUX. (2008). J. of Econom Kosuke Imai (Harvard) Regression Discontinuity Design Stat186/Gov2002 Fall 2018 14 / 1