Matching and Weighting Methods for Causal Inference

Transcription

1 Matching and Weighting Methods for ausal Inference Kosuke Imai Princeton University Methods Workshop, Duke University Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) / 57

2 References to Relevant Papers Matching as Nonparametric Preprocessing for Reducing Model Dependence in Parametric ausal Inference. Political Analysis (27) Misunderstandings among Experimentalists and Observationalists about ausal Inference. Journal of the Royal Statistical Society, Series A (28) he Essential Role of Pair Matching in luster-randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation. Statistical Science (29) ovariate Balancing Propensity Score. Working paper On the Use of Linear Fixed Effects Regression Models for ausal Inference. Working paper All papers are available at Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 2 / 57

3 Software Implementation ausal inference with regression: Zelig: Everyone s Statistical Software ausal inference with matching: MatchIt: Nonparametric Preprocessing for Parametric ausal Inference ausal inference with propensity score: BPS: ovariate Balancing Propensity Score ausal inference with fixed effects: wfe: Weighted Fixed Effects Regressions for ausal Inference All software is available at Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 3 / 57

4 Matching and Weighting What is matching? Grouping observations based on their observed characteristics pairing 2 subclassification 3 subsetting What is weighting? Replicating observations based on their observed characteristics All types of matching are special cases with discrete weights What matching and weighting methods can do: flexible and robust causal modeling under selection on observables What they cannot do: eliminate bias due to unobserved confounding Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 4 / 57

5 Matching for Randomized Experiments Matching can be used for randomized experiments too! Randomization of treatment unbiased estimates Improving efficiency reducing variance Why care about efficiency? You care about your results! Randomized matched-pair design Randomized block design Intuition: estimation uncertainty comes from pre-treatment differences between treatment and control groups Mantra (Box, Hunter, and Hunter): Block what you can and randomize what you cannot Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 5 / 57

6 luster Randomized Experiments Units: i =, 2,..., n j lusters of units: j =, 2,..., m reatment at cluster level: j {, } Outcome: Y ij = Y ij ( j ) Random assignment: (Y ij (), Y ij ()) j Estimands at unit level: SAE m j= n j n m j (Y ij () Y ij ()) j= i= PAE E(Y ij () Y ij ()) Random sampling of clusters and units Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 6 / 57

7 Merits and Limitations of REs Interference between units within a cluster is allowed Assumption: No interference between units of different clusters Often easier to implement: Mexican health insurance experiment Opportunity to estimate the spill-over effects D. W. Nickerson. Spill-over effect of get-out-the-vote canvassing within household (APSR, 28) Limitations: A large number of possible treatment assignments 2 Loss of statistical power Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 7 / 57

8 Design-Based Inference For simplicity, assume equal cluster size, i.e., n j = n for all j he difference-in-means estimator: ˆτ m j Y j m m j= m j= ( j )Y j where Y j n j i= Y ij/n j Easy to show E(ˆτ O) = SAE and thus E(ˆτ) = PAE Exact population variance: Var(ˆτ) = Var(Y j()) m Intracluster correlation coefficient ρ t : + Var(Y j()) m Var(Y j (t)) = σ2 t n { + (n )ρ t} σ 2 t Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 8 / 57

9 luster Standard Error luster robust sandwich variance estimator: Var((ˆα, m m m ˆβ) ) = X j ˆɛ jˆɛ j X j j= X j j= X j where in this case X j = [ j ] is an n j 2 matrix and ˆɛ j = (ˆɛ j,..., ˆɛ nj j) is a column vector of length n j Design-based evaluation (assume n j = n for all j): ( V(Yj ()) Finite Sample Bias = m 2 + V(Y ) j()) m 2 Bias vanishes asymptotically as m with n fixed j= X j Implication: cluster standard errors by the unit of treatment assignment Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 9 / 57 X j

10 Example: Seguro Popular de Salud (SPS) Evaluation of the Mexican universal health insurance program Aim: provide social protection in health to the 5 million uninsured Mexicans A key goal: reduce out-of-pocket health expenditures Sounds obvious but not easy to achieve in developing countries Individuals must affiliate in order to receive SPS services health clusters non-randomly chosen for evaluation Matched-pair design: based on population, socio-demographics, poverty, education, health infrastructure etc. reatment clusters : encouragement for people to affiliate Data: aggregate characteristics, surveys of 32, individuals Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) / 57

11 Matching and Blocking for Randomized Experiments Okay, but how should I match/block without the treatment group? Goal: match/block well on powerful predictors of outcome (prognostic factors) (oarsened) Exact matching Matching based on a similarity measure: Mahalanobis distance = (X i X j ) Σ (X i X j ) ould combine the two Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) / 57

12 Relative Efficiency of Matched-Pair Design (MPD) ompare with completely-randomized design Greater (positive) correlation within pair greater efficiency PAE: MPD is between.8 and 38.3 times more efficient! Relative Efficiency, PAE Relative Efficiency, UAE Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 2 / 57

13 hallenges of Observational Studies Randomized experiments vs. Observational studies radeoff between internal and external validity Endogeneity: selection bias Generalizability: sample selection, Hawthorne effects, realism Statistical methods cannot replace good research design Designing observational studies Natural experiments (haphazard treatment assignment) Examples: birthdays, weather, close elections, arbitrary administrative rules and boundaries Replicating randomized experiments Key Questions: Where are the counterfactuals coming from? 2 Is it a credible comparison? Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 3 / 57

14 Identification of the Average reatment Effect Assumption : Overlap (i.e., no extrapolation) < Pr( i = X i = x) < for any x X Assumption 2: Ignorability (exogeneity, unconfoundedness, no omitted variable, selection on observables, etc.) {Y i (), Y i ()} i X i = x for any x X onditional expectation function: µ(t, x) = E(Y i (t) i = t, X i = x) Regression-based estimator: ˆτ = n n {ˆµ(, X i ) ˆµ(, X i )} i= Delta method is pain, but simulation is easy via Zelig Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 4 / 57

15 Matching as Nonparametric Preprocessing READING: Ho et al. Political Analysis (27) Assume exogeneity holds: matching does NO solve endogeneity Need to model E(Y i i, X i ) Parametric regression functional-form/distributional assumptions = model dependence Non-parametric regression = curse of dimensionality Preprocess the data so that treatment and control groups are similar to each other w.r.t. the observed pre-treatment covariates Goal of matching: achieve balance = independence between and X Replicate randomized treatment w.r.t. observed covariates Reduced model dependence: minimal role of statistical modeling Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 5 / 57

16 Sensitivity Analysis onsider a simple pair-matching of treated and control units Assumption: treatment assignment is random Difference-in-means estimator Question: How large a departure from the key (untestable) assumption must occur for the conclusions to no longer hold? Rosenbaum s sensitivity analysis: for any pair j, Γ Pr( j = )/ Pr( j = ) Pr( 2j = )/ Pr( 2j = ) Γ Under ignorability, Γ = for all j How do the results change as you increase Γ? Limitations of sensitivity analysis FURHER READING: P. Rosenbaum. Observational Studies. Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 6 / 57

17 he Role of Propensity Score he probability of receiving the treatment: π(x i ) Pr( i = X i ) he balancing property (no assumption): i X i π(x i ) Exogeneity given the propensity score (under exogeneity given covariates): (Y i (), Y i ()) i π(x i ) Dimension reduction But, true propensity score is unknown: propensity score tautology (more later) Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 7 / 57

18 lassical Matching echniques Exact matching Mahalanobis distance matching: Propensity score matching (X i X j ) Σ (X i X j ) One-to-one, one-to-many, and subclassification Matching with caliper Which matching method to choose? Whatever gives you the best balance! Importance of substantive knowledge: propensity score matching with exact matching on key confounders FURHER READING: Rubin (26). Matched Sampling for ausal Effects (ambridge UP) Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 8 / 57

19 How to heck Balance Success of matching method depends on the resulting balance How should one assess the balance of matched data? Ideally, compare the joint distribution of all covariates for the matched treatment and control groups In practice, this is impossible when X is high-dimensional heck various lower-dimensional summaries; (standardized) mean difference, variance ratio, empirical DF, etc. Frequent use of balance test t test for difference in means for each variable of X other test statistics; e.g., χ 2, F, Kolmogorov-Smirnov tests statistically insignificant test statistics as a justification for the adequacy of the chosen matching method and/or a stopping rule for maximizing balance Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 9 / 57

20 An Illustration of Balance est Fallacy t statistic "Statistical insignificance" region Math test score QQ Plot Mean Deviation Difference in Means Number of ontrols Randomly Dropped Number of ontrols Randomly Dropped Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 2 / 57

21 Problems with Hypothesis ests as Stopping Rules Balance test is a function of both balance and statistical power he more observations dropped, the less power the tests have t-test is affected by factors other than balance, nm (X mt X mc ) s 2 mt r m + s2 mc r m X mt and X mc are the sample means s 2 mt and s 2 mc are the sample variances n m is the total number of remaining observations r m is the ratio of remaining treated units to the total number of remaining observations Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 2 / 57

22 Recent Advances in Matching Methods he main problem of matching: balance checking Skip balance checking all together Specify a balance metric and optimize it Optimal matching: minimize sum of distances Full matching: subclassification with variable strata size Genetic matching: maximize minimum p-value oarsened exact matching: exact match on binned covariates SVM subsetting: find the largest, balanced subset for general treatment regimes Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 22 / 57

23 Inverse Propensity Score Weighting Matching is inefficient because it throws away data Matching is a special case of weighting Weighting by inverse propensity score (Horvitz-hompson): n n i= ( i Y i ˆπ(X i ) ( ) i)y i ˆπ(X i ) Unstable when some weights are extremely small An improved weighting scheme: n i= { iy i /ˆπ(X i )} n i= { i/ˆπ(x i )} n i= {( i)y i /( ˆπ(X i ))} n i= {( i)/( ˆπ(X i ))} Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 23 / 57

24 Weighting Both Groups to Balance ovariates { Balancing condition: E i X i π(x i ) ( i )X i π(x i ) } = AE weighted ontrol units AE weighted reated units Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 24 / 57

25 Weighting ontrol Group to Balance ovariates { } Balancing condition: E i X i π(x i )( i )X i π(x i ) = A weighted ontrol units reated units Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 25 / 57

26 Efficient Doubly-Robust Estimators he estimator by Robins et al. : { n ˆτ DR ˆµ(, X i ) + n n n i= i= { n ˆµ(, X i ) + n n n i= i= } i (Y i ˆµ(, X i )) ˆπ(X i ) ( i )(Y i ˆµ(, X i )) ˆπ(X i ) onsistent if either the propensity score model or the outcome model is correct (Semiparametrically) Efficient FURHER READING: Lunceford and Davidian (24, Stat. in Med.) } Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 26 / 57

27 Marginal Structural Models for Longitudinal Data Units i =,..., N and time j =,..., J Eventual outcome Y i measured at time J reatment and covariate history: ij and X ij Quantity of interest: (marginal) AE = E{Y i ( t)} Sequential ignorability assumption: Y i (t) ij X ij, i,j Inverse-probability-of-treatment weight: w i = P( ij X ij ) = J j= P( ij i,j, X ij ) Stabilized weight: multiply w i by P( ij ) Analysis: weighted regression of Y i on ij FURHER READINGS: Robins et al. (2), Blackwell (23) Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 27 / 57

28 Propensity Score autology Propensity score is unknown Dimension reduction is purely theoretical: must model i given X i Diagnostics: covariate balance checking In practice, adhoc specification searches are conducted Model misspecification is always possible autology: propensity score works only when you get it right! In fact, estimated propensity score works even better than true propensity score when the model is correct heory (Rubin et al.): ellipsoidal covariate distributions = equal percent bias reduction Skewed covariates are common in applied settings Propensity score methods can be sensitive to misspecification Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 28 / 57

29 Kang and Schafer (27, Statistical Science) Simulation study: the deteriorating performance of propensity score weighting methods when the model is misspecified Setup: 4 covariates Xi : all are i.i.d. standard normal Outcome model: linear model Propensity score model: logistic model with linear predictors Misspecification induced by measurement error: X i = exp(xi/2) X i2 = Xi2/( + exp(xi) + ) X i3 = (XiX i3/25 +.6) 3 X i4 = (Xi + Xi4 + 2) 2 Weighting estimators to be evaluated: Horvitz-hompson 2 Inverse-probability weighting with normalized weights 3 Weighted least squares regression 4 Doubly-robust least squares regression Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 29 / 57

30 Weighting Estimators Do Great If the Model is orrect Bias RMSE Sample size Estimator GLM rue GLM rue () Both models correct H n = 2 IPW WLS DR H n = IPW WLS DR (2) Propensity score model correct H n = 2 IPW WLS DR H n = IPW WLS DR Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 3 / 57

31 Weighting Estimators Are Sensitive to Misspecification Bias RMSE Sample size Estimator GLM rue GLM rue (3) Outcome model correct H n = 2 IPW WLS DR H n = IPW WLS DR (4) Both models incorrect H n = 2 IPW WLS DR H n = IPW WLS DR Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 3 / 57

32 ovariate Balancing Propensity Score Recall the dual characteristics of propensity score onditional probability of treatment assignment 2 ovariate balancing score Implied moment conditions: Score equation: { i π β E (X i) ( i)π β (X } i) π β (X i ) π β (X i ) = 2 Balancing condition: { } i Xi E π β (X i ) ( i) X i π β (X i ) = where X i = f (X i ) is any vector-valued function Score condition is a particular covariate balancing condition! Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 32 / 57

33 Estimation and Inference Just-identified BPS: Find the values of model parameters that satisfy covariate balancing conditions in the sample Method of moments: # of parameters = # of balancing conditions Over-identified BPS: # of parameters < # of balancing conditions Generalized method of moments (GMM): ˆβ = argmin β Θ ḡ β (, X) Σ β ḡβ(, X) where ḡ β (, X) = N N i= i π β (X i ) π β (X i ) ( i )π β (X i ) π β (X i ) i Xi π β (X i ) ( i ) X i π β (X i ) and Σ β is the covariance of moment conditions Enables misspecification test Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 33 / 57

34 Revisiting Kang and Schafer (27) Bias RMSE Sample size Estimator GLM BPS BPS2 rue GLM BPS BPS2 rue () Both models correct H n = 2 IPW WLS DR H n = IPW WLS DR (2) Propensity score model correct H n = 2 IPW WLS DR H n = IPW WLS DR Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 34 / 57

35 BPS Makes Weighting Methods More Robust Bias RMSE Sample size Estimator GLM BPS BPS2 rue GLM BPS BPS2 rue (3) Outcome model correct H n = 2 IPW WLS DR H n = IPW WLS DR (4) Both models incorrect H n = 2 IPW WLS DR H n = IPW WLS DR Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 35 / 57

36 BPS Sacrifices Likelihood for Better Balance GLM BPS Log Likelihood GLM BPS ovariate Imbalance GLM BPS GLM BPS Log Likelihood (GLM BPS) Log Imbalance (GLM BPS) Likelihood Balance radeoff Log Likelihood (GLM BPS) Log Imbalance (GLM BPS) Neither Model Both Models Specified orrectly Specified orrectly Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 36 / 57

37 A lose Look at Fixed Effects Regression Fixed effects models are a primary workhorse for causal inference Used for stratified experimental and observational data Also used to adjust for unobservables in observational studies: Good instruments are hard to find..., so we d like to have other tools to deal with unobserved confounders. his chapter considers... strategies that use data with a time or cohort dimension to control for unobserved but fixed omitted variables (Angrist & Pischke, Mostly Harmless Econometrics) fixed effects regression can scarcely be faulted for being the bearer of bad tidings (Green et al., Dirty Pool) ommon claim: Fixed effects models are superior to matching estimators because the latter can only adjust for observables Question: What are the exact causal assumptions underlying fixed effects regression models? Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 37 / 57

38 Matching and Regression in ross-section Settings Units reatment status Outcome Y Y 2 Y 3 Y 4 Y 5 Estimating the Average reatment Effect (AE) via matching: Y 2 (Y 3 + Y 4 ) Y 2 2 (Y 3 + Y 4 ) 3 (Y + Y 2 + Y 5 ) Y 3 3 (Y + Y 2 + Y 5 ) Y 4 Y 5 2 (Y 3 + Y 4 ) Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 38 / 57

39 Matching Representation of Simple Regression ross-section simple linear regression model: Y i = α + βx i + ɛ i Binary treatment: X i {, } Equivalent matching estimator: where Ŷ i () = Ŷ i () = ˆβ = N N (Ŷi Ŷi()) () i= { Y i if X i = N N i = X i i = X i Y i if X i = { N N i = ( X i ) i = ( X i )Y i if X i = Y i if X i = reated units matched with the average of non-treated units Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 39 / 57

40 One-Way Fixed Effects Regression Simple (one-way) FE model: Y it = α i + βx it + ɛ it ommonly used by applied researchers: Stratified randomized experiments (Duflo et al. 27) Stratification and matching in observational studies Panel data, both experimental and observational ˆβ FE may be biased for the AE even if X it is exogenous within each unit It converges to the weighted average of conditional AEs: where σ 2 i ˆβ FE = t= (X it X i ) 2 / p E{AE i σi 2 } E(σi 2 ) How are counterfactual outcomes estimated under the FE model? Unit fixed effects = within-unit comparison Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 4 / 57

41 Mismatches in One-Way Fixed Effects Model ime periods Units : treated observations : control observations ircles: Proper matches riangles: Mismatches = attenuation bias Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 4 / 57

42 Matching Representation of Fixed Effects Regression Proposition Ŷ it (x) = K = ˆβ FE = K { N { N Y it t t Y it N X it i= t= N i= t= if X it = x if X it = x ) } (Ŷit () Ŷit(), for x =, ( X it ) + ( X it ) t t t t X it. K : average proportion of proper matches across all observations More mismatches = larger adjustment Adjustment is required except very special cases Fixes attenuation bias but this adjustment is not sufficient Fixed effects estimator is a special case of matching estimators Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 42 / 57

43 Unadjusted Matching Estimator ime periods Units onsistent if the treatment is exogenous within each unit Only equal to fixed effects estimator if heterogeneity in either treatment assignment or treatment effect is non-existent Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 43 / 57

44 Unadjusted Matching = Weighted FE Estimator Proposition 2 he unadjusted matching estimator where Ŷ it () = ˆβ M = N N i= Y it if X it = t = X it Y it t = X it if X it = (Ŷit Ŷit()) () t= and Ŷ it () = is equivalent to the weighted fixed effects model (ˆα M, ˆβ M ) = argmin W it (α,β) N i= W it (Y it α i βx it ) 2 t= t = X it if X it =, t = ( X it ) if X it =. t = ( X it )Y it t = ( X it ) if X it = Y it if X it = Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 44 / 57

45 Equal Weights reatment Weights Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 45 / 57

46 Different Weights reatment Weights Any within-unit matching estimator leads to weighted fixed effects regression with particular weights We derive regression weights given any matching estimator for various quantities (AE, A, etc.) Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 46 / 57

47 First Difference = Matching = Weighted One-Way FE Y it = β X it + ɛ it where Y it = Y it Y i,t, X it = X it X i,t reatment Weights Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 47 / 57

48 Mismatches in wo-way FE Model ime periods Y it = α i + γ t + βx it + ɛ it Units riangles: wo kinds of mismatches Same treatment status Neither same unit nor same time Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 48 / 57

49 Mismatches in Weighted wo-way FE Model Units ime periods Some mismatches can be eliminated You can NEVER eliminate them all Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 49 / 57

50 ross Section Analysis = Weighted ime FE Model Average Outcome control group treatment group counterfactual time t time t+ Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 5 / 57

51 First Difference = Weighted Unit FE Model Average Outcome control group treatment group counterfactual time t time t+ Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 5 / 57

52 What about Difference-in-Differences (DiD)? Average Outcome control group treatment group counterfactual time t time t+ Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 52 / 57

53 General DiD = Weighted wo-way (Unit and ime) FE 2 2: standard two-way fixed effects General setting: Multiple time periods, repeated treatments reatment Weights Weights can be negative = the method of moments estimator Fast computation is available Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 53 / 57

54 Effects of GA Membership on International rade ontroversy Rose (24): No effect of GA membership on trade omz et al. (27): Significant effect with non-member participants 2 he central role of fixed effects models: Rose (24): one-way (year) fixed effects for dyadic data omz et al. (27): two-way (year and dyad) fixed effects Rose (25): I follow the profession in placing most confidence in the fixed effects estimators; I have no clear ranking between country-specific and country pair-specific effects. omz et al. (27): We, too, prefer FE estimates over OLS on both theoretical and statistical ground Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 54 / 57

55 Data and Methods Data Data set from omz et al. (27) Effect of GA: countries, and 96,27 (dyad-year) observations Year fixed effects model: standard and weighted ln Y it = α t + βx it + δ Z it + ɛ it X it : Formal membership/participant () Both vs. One, (2) One vs. None, (3) Both vs. One/None Z it : 5 dyad-varying covariates (e.g., log product GDP) Year fixed effects: standard, weighted, and first difference wo-way fixed effects: standard and difference-in-differences Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 55 / 57

56 Empirical Results Year Fixed Effects Dyad Fixed Effects Year and Dyad Fixed Effects omparison Membership Standard Weighted Standard Weighted First Diff. Standard Diff.-in-Diff. Formal (N=96,27) (.3) (.3) (.25) (.23) (.54) (.28) (.33) White s p-value Both vs. Mix Participants (N=96,27) (.34) (.35) (.3) (.29) (.3) (.34) (.28) White s p-value Formal (N=75,84) (.3) (.3) (.25) (.23) (.55) (.28) (.33) White s p-value Both vs. One Participants (N=87,65) (.35) (.36) (.3) (.29) (.3) (.34) (.29) White s p-value Formal (N=9,72) (.53) (.56) (.4) (.4) (.67) (.43) (.378) White s p-value One vs. None Participants (N=7,298) (.72) (.79) (.62) (.58) (.63) (.66) (.85) White s p-value covariates dyad-varying covariates year-varying covariates no covariate able : Estimated Effects of GA on the Logarithm of Bilateral rade based on Various Fixed Effects Models: For models with either year or dyad fixed effects, the Weighted columns present the estimates based on the regression weights given in Proposition 2, which yield the estimated average treatment effects. Other weighted fixed effects estimators, i.e., first differences and difference-in-differences, Kosuke Imai (Princeton) are also presented. Matching Bothand vs. Weighting Mix ( Both Methods vs. One ) represents Duke (January the comparison 8 9, 23) between dyads 56 / 57of

57 oncluding Remarks Matching methods do: make causal assumptions transparent by identifying counterfactuals make regression models robust by reducing model dependence But they cannot solve endogeneity Only good research design can overcome endogeneity Recent advances in matching methods directly optimize balance the same idea applied to propensity score Weighting methods generalize matching methods Sensitive to propensity score model specification Robust estimation of propensity score model Next methodological challenges for causal inference: temporal and spatial dynamics, networks effects Kosuke Imai (Princeton) Matching and Weighting Methods Duke (January 8 9, 23) 57 / 57