Unpacking the Black-Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies

Similar documents
Unpacking the Black-Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies

Unpacking the Black-Box: Learning about Causal Mechanisms from Experimental and Observational Studies

Unpacking the Black-Box: Learning about Causal Mechanisms from Experimental and Observational Studies

Experimental Designs for Identifying Causal Mechanisms

Causal Mechanisms Short Course Part II:

Statistical Analysis of Causal Mechanisms

Statistical Analysis of Causal Mechanisms

Statistical Analysis of Causal Mechanisms

Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies

Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies

Causal Mediation Analysis in R. Quantitative Methodology and Causal Mechanisms

Statistical Analysis of Causal Mechanisms for Randomized Experiments

Introduction to Statistical Inference

Identification and Inference in Causal Mediation Analysis

Identification and Sensitivity Analysis for Multiple Causal Mechanisms: Revisiting Evidence from Framing Experiments

Identification, Inference, and Sensitivity Analysis for Causal Mediation Effects

When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data?

Experimental Designs for Identifying Causal Mechanisms

When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data?

Telescope Matching: A Flexible Approach to Estimating Direct Effects

When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Panel Data?

mediation: R Package for Causal Mediation Analysis

Discussion of Papers on the Extensions of Propensity Score

Use of Matching Methods for Causal Inference in Experimental and Observational Studies. This Talk Draws on the Following Papers:

Causal Inference Lecture Notes: Causal Inference with Repeated Measures in Observational Studies

R and Stata for Causal Mechanisms

Causal Interaction in Factorial Experiments: Application to Conjoint Analysis

Statistical Models for Causal Analysis

Gov 2002: 13. Dynamic Causal Inference

Covariate Balancing Propensity Score for General Treatment Regimes

Telescope Matching: A Flexible Approach to Estimating Direct Effects *

An Introduction to Causal Mediation Analysis. Xu Qin University of Chicago Presented at the Central Iowa R User Group Meetup Aug 10, 2016

Bayesian Causal Mediation Analysis for Group Randomized Designs with Homogeneous and Heterogeneous Effects: Simulation and Case Study

Use of Matching Methods for Causal Inference in Experimental and Observational Studies. This Talk Draws on the Following Papers:

Identification, Inference and Sensitivity Analysis for Causal Mediation Effects

Causal Interaction in Factorial Experiments: Application to Conjoint Analysis

Casual Mediation Analysis

Causal Directed Acyclic Graphs

On the Use of Linear Fixed Effects Regression Models for Causal Inference

Published online: 19 Jun 2015.

Statistical Analysis of Randomized Experiments with Nonignorable Missing Binary Outcomes

Combining Difference-in-difference and Matching for Panel Data Analysis

Causal mediation analysis: Definition of effects and common identification assumptions

ONLINE APPENDICES Emotions and the Micro-Foundations of Commitment Problems

Front-Door Adjustment

Causal Inference with General Treatment Regimes: Generalizing the Propensity Score

Gov 2002: 4. Observational Studies and Confounding

Identification and Estimation of Causal Mediation Effects with Treatment Noncompliance

Conceptual overview: Techniques for establishing causal pathways in programs and policies

Structural Nested Mean Models for Assessing Time-Varying Effect Moderation. Daniel Almirall

A Decision Theoretic Approach to Causality

Selection on Observables: Propensity Score Matching.

Causal Interaction in Factorial Experiments: Application to Conjoint Analysis

Causal Inference Lecture Notes: Selection Bias in Observational Studies

What s New in Econometrics. Lecture 1

Robust Estimation of Inverse Probability Weights for Marginal Structural Models

Regression Discontinuity Designs

Help! Statistics! Mediation Analysis

ESTIMATION OF TREATMENT EFFECTS VIA MATCHING

High Dimensional Propensity Score Estimation via Covariate Balancing

Structural Nested Mean Models for Assessing Time-Varying Effect Moderation. Daniel Almirall

Advanced Quantitative Research Methodology, Lecture Notes: Research Designs for Causal Inference 1

Flexible mediation analysis in the presence of non-linear relations: beyond the mediation formula.

Difference-in-Differences Methods

Causal Inference with a Continuous Treatment and Outcome: Alternative Estimators for Parametric Dose-Response Functions

The Supervised Learning Approach To Estimating Heterogeneous Causal Regime Effects

SC705: Advanced Statistics Instructor: Natasha Sarkisian Class notes: Introduction to Structural Equation Modeling (SEM)

arxiv: v3 [stat.me] 4 Apr 2018

Applied Health Economics (for B.Sc.)

Statistical Analysis of List Experiments

Discussion: Can We Get More Out of Experiments?

An Introduction to Causal Analysis on Observational Data using Propensity Scores

Assess Assumptions and Sensitivity Analysis. Fan Li March 26, 2014

ECO Class 6 Nonparametric Econometrics

PANEL ON MEDIATION AND OTHER MIRACLES OF THE 21 ST CENTURY

14.74 Lecture 10: The returns to human capital: education

Interaction Effects and Causal Heterogeneity

Logistic regression: Why we often can do what we think we can do. Maarten Buis 19 th UK Stata Users Group meeting, 10 Sept. 2015

Assessing In/Direct Effects: from Structural Equation Models to Causal Mediation Analysis

When Should We Use Fixed Effects Regression Models for Causal Inference with Longitudinal Data?

Measuring Social Influence Without Bias

Introduction to Causal Inference. Solutions to Quiz 4

Ratio of Mediator Probability Weighting for Estimating Natural Direct and Indirect Effects

mediation: R Package for Causal Mediation Analysis

Item Response Theory for Conjoint Survey Experiments

Combining multiple observational data sources to estimate causal eects

PSC 504: Differences-in-differeces estimators

Propensity Score Methods for Causal Inference

Lecture 2: Constant Treatment Strategies. Donglin Zeng, Department of Biostatistics, University of North Carolina

Guanglei Hong. University of Chicago. Presentation at the 2012 Atlantic Causal Inference Conference

Flexible Estimation of Treatment Effect Parameters

Causal Inference in Observational Studies with Non-Binary Treatments. David A. van Dyk

SIMULATION-BASED SENSITIVITY ANALYSIS FOR MATCHING ESTIMATORS

Causal Interaction in High Dimension

Causal Inference with Big Data Sets

Causal inference in multilevel data structures:

Marginal versus conditional effects: does it make a difference? Mireille Schnitzer, PhD Université de Montréal

Causality II: How does causal inference fit into public health and what it is the role of statistics?

Randomized Experiments

Michael Lechner Causal Analysis RDD 2014 page 1. Lecture 7. The Regression Discontinuity Design. RDD fuzzy and sharp

Transcription:

Unpacking the Black-Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies Kosuke Imai Princeton University February 23, 2012 Joint work with L. Keele (Penn State) D. Tingley (Harvard) T. Yamamoto (MIT) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 1 / 27

Quantitative Research and Causal Mechanisms Causal inference is a central goal of scientific research Scientists care about causal mechanisms, not just causal effects Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 2 / 27

Quantitative Research and Causal Mechanisms Causal inference is a central goal of scientific research Scientists care about causal mechanisms, not just causal effects Randomized experiments often only determine whether the treatment causes changes in the outcome Not how and why the treatment affects the outcome Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 2 / 27

Quantitative Research and Causal Mechanisms Causal inference is a central goal of scientific research Scientists care about causal mechanisms, not just causal effects Randomized experiments often only determine whether the treatment causes changes in the outcome Not how and why the treatment affects the outcome Common criticism of experiments and statistics: black box view of causality Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 2 / 27

Quantitative Research and Causal Mechanisms Causal inference is a central goal of scientific research Scientists care about causal mechanisms, not just causal effects Randomized experiments often only determine whether the treatment causes changes in the outcome Not how and why the treatment affects the outcome Common criticism of experiments and statistics: black box view of causality Qualitative research uses process tracing Question: How can quantitative research be used to identify causal mechanisms? Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 2 / 27

Overview of the Talk Goal: Convince you that statistics can be useful for learning about causal mechanisms Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 3 / 27

Overview of the Talk Goal: Convince you that statistics can be useful for learning about causal mechanisms Method: Causal Mediation Analysis Mediator, M Treatment, T Outcome, Y Direct and indirect effects; intermediate and intervening variables Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 3 / 27

Overview of the Talk Goal: Convince you that statistics can be useful for learning about causal mechanisms Method: Causal Mediation Analysis Mediator, M Treatment, T Outcome, Y Direct and indirect effects; intermediate and intervening variables New tools: framework, estimation algorithm, sensitivity analysis, research designs, easy-to-use software Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 3 / 27

Causal Mediation Analysis in American Politics The political psychology literature on media framing Nelson et al. (APSR, 1998) Popular in social psychology Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 4 / 27

Causal Mediation Analysis in Comparative Politics Resource curse thesis Authoritarian government civil war Natural resources Slow growth Causes of civil war: Fearon and Laitin (APSR, 2003) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 5 / 27

Causal Mediation Analysis in International Relations The literature on international regimes and institutions Krasner (International Organization, 1982) Power and interests are mediated by regimes Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 6 / 27

Current Practice in Political Science Regression: Y i = α + βt i + γm i + δx i + ɛ i Each coefficient is interpreted as a causal effect Sometimes, it s called marginal effect Idea: increase T i by one unit while holding M i and X i constant But, if you change T i, that may also change M i The Problem: Post-treatment bias Usual advice: only include causally prior (or pre-treatment) variables But, then you lose causal mechanisms! Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 7 / 27

Formal Statistical Framework of Causal Inference Units: i = 1,..., n Treatment : T i = 1 if treated, T i = 0 otherwise Pre-treatment covariates: X i Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 8 / 27

Formal Statistical Framework of Causal Inference Units: i = 1,..., n Treatment : T i = 1 if treated, T i = 0 otherwise Pre-treatment covariates: X i Potential outcomes: Y i (1) and Y i (0) Observed outcome: Y i = Y i (T i ) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 8 / 27

Formal Statistical Framework of Causal Inference Units: i = 1,..., n Treatment : T i = 1 if treated, T i = 0 otherwise Pre-treatment covariates: X i Potential outcomes: Y i (1) and Y i (0) Observed outcome: Y i = Y i (T i ) Voters Contact Turnout Age Party ID i T i Y i (1) Y i (0) X i X i 1 1 1? 20 D 2 0? 0 55 R...... n 1 0? 62 D Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 8 / 27

Formal Statistical Framework of Causal Inference Units: i = 1,..., n Treatment : T i = 1 if treated, T i = 0 otherwise Pre-treatment covariates: X i Potential outcomes: Y i (1) and Y i (0) Observed outcome: Y i = Y i (T i ) Voters Contact Turnout Age Party ID i T i Y i (1) Y i (0) X i X i 1 1 1? 20 D 2 0? 0 55 R...... n 1 0? 62 D Causal effect: Y i (1) Y i (0) Problem: only one potential outcome can be observed per unit Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 8 / 27

Potential Outcomes Framework for Mediation Binary treatment: T i Pre-treatment covariates: X i Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 9 / 27

Potential Outcomes Framework for Mediation Binary treatment: T i Pre-treatment covariates: X i Potential mediators: M i (t) Observed mediator: M i = M i (T i ) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 9 / 27

Potential Outcomes Framework for Mediation Binary treatment: T i Pre-treatment covariates: X i Potential mediators: M i (t) Observed mediator: M i = M i (T i ) Potential outcomes: Y i (t, m) Observed outcome: Y i = Y i (T i, M i (T i )) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 9 / 27

Potential Outcomes Framework for Mediation Binary treatment: T i Pre-treatment covariates: X i Potential mediators: M i (t) Observed mediator: M i = M i (T i ) Potential outcomes: Y i (t, m) Observed outcome: Y i = Y i (T i, M i (T i )) Again, only one potential outcome can be observed per unit Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 9 / 27

Causal Mediation Effects Total causal effect: τ i Y i (1, M i (1)) Y i (0, M i (0)) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 10 / 27

Causal Mediation Effects Total causal effect: τ i Y i (1, M i (1)) Y i (0, M i (0)) Causal mediation (Indirect) effects: δ i (t) Y i (t, M i (1)) Y i (t, M i (0)) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 10 / 27

Causal Mediation Effects Total causal effect: τ i Y i (1, M i (1)) Y i (0, M i (0)) Causal mediation (Indirect) effects: δ i (t) Y i (t, M i (1)) Y i (t, M i (0)) Causal effect of the treatment-induced change in M i on Y i Change the mediator from M i (0) to M i (1) while holding the treatment constant at t Represents the mechanism through M i Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 10 / 27

Total Effect = Indirect Effect + Direct Effect Direct effects: ζ i (t) Y i (1, M i (t)) Y i (0, M i (t)) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 11 / 27

Total Effect = Indirect Effect + Direct Effect Direct effects: ζ i (t) Y i (1, M i (t)) Y i (0, M i (t)) Causal effect of T i on Y i, holding mediator constant at its potential value that would be realized when T i = t Change the treatment from 0 to 1 while holding the mediator constant at M i (t) Represents all mechanisms other than through M i Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 11 / 27

Total Effect = Indirect Effect + Direct Effect Direct effects: ζ i (t) Y i (1, M i (t)) Y i (0, M i (t)) Causal effect of T i on Y i, holding mediator constant at its potential value that would be realized when T i = t Change the treatment from 0 to 1 while holding the mediator constant at M i (t) Represents all mechanisms other than through M i Total effect = mediation (indirect) effect + direct effect: τ i = δ i (t) + ζ i (1 t) = 1 2 {δ i(0) + δ i (1) + ζ i (0) + ζ i (1)} Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 11 / 27

What Does the Observed Data Tell Us? Quantity of Interest: Average causal mediation effects (ACME) δ(t) E(δ i (t)) = E{Y i (t, M i (1)) Y i (t, M i (0))} Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 12 / 27

What Does the Observed Data Tell Us? Quantity of Interest: Average causal mediation effects (ACME) δ(t) E(δ i (t)) = E{Y i (t, M i (1)) Y i (t, M i (0))} Average direct effects ( ζ(t)) are defined similarly Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 12 / 27

What Does the Observed Data Tell Us? Quantity of Interest: Average causal mediation effects (ACME) δ(t) E(δ i (t)) = E{Y i (t, M i (1)) Y i (t, M i (0))} Average direct effects ( ζ(t)) are defined similarly Y i (t, M i (t)) is observed but Y i (t, M i (t )) can never be observed We have an identification problem Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 12 / 27

What Does the Observed Data Tell Us? Quantity of Interest: Average causal mediation effects (ACME) δ(t) E(δ i (t)) = E{Y i (t, M i (1)) Y i (t, M i (0))} Average direct effects ( ζ(t)) are defined similarly Y i (t, M i (t)) is observed but Y i (t, M i (t )) can never be observed We have an identification problem = Need additional assumptions to make progress Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 12 / 27

Identification under Sequential Ignorability Proposed identification assumption: Sequential Ignorability (SI) {Y i (t, m), M i (t)} T i X i = x, (1) Y i (t, m) M i (t) T i = t, X i = x (2) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 13 / 27

Identification under Sequential Ignorability Proposed identification assumption: Sequential Ignorability (SI) {Y i (t, m), M i (t)} T i X i = x, (1) Y i (t, m) M i (t) T i = t, X i = x (2) (1) is guaranteed to hold in a standard experiment (2) does not hold unless X i includes all confounders Limitation: X i cannot include post-treatment confounders Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 13 / 27

Identification under Sequential Ignorability Proposed identification assumption: Sequential Ignorability (SI) {Y i (t, m), M i (t)} T i X i = x, (1) Y i (t, m) M i (t) T i = t, X i = x (2) (1) is guaranteed to hold in a standard experiment (2) does not hold unless X i includes all confounders Limitation: X i cannot include post-treatment confounders Under SI, ACME is nonparametrically identified: E(Y i M i, T i = t, X i ) {dp(m i T i = 1, X i ) dp(m i T i = 0, X i )} dp(x i ) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 13 / 27

Example: Anxiety, Group Cues and Immigration Brader, Valentino & Suhat (2008, AJPS) How and why do ethnic cues affect immigration attitudes? Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 14 / 27

Example: Anxiety, Group Cues and Immigration Brader, Valentino & Suhat (2008, AJPS) How and why do ethnic cues affect immigration attitudes? Theory: Anxiety transmits the effect of cues on attitudes Anxiety, M Media Cue, T Immigration Attitudes, Y Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 14 / 27

Example: Anxiety, Group Cues and Immigration Brader, Valentino & Suhat (2008, AJPS) How and why do ethnic cues affect immigration attitudes? Theory: Anxiety transmits the effect of cues on attitudes Anxiety, M Media Cue, T Immigration Attitudes, Y ACME = Average difference in immigration attitudes due to the change in anxiety induced by the media cue treatment Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 14 / 27

Example: Anxiety, Group Cues and Immigration Brader, Valentino & Suhat (2008, AJPS) How and why do ethnic cues affect immigration attitudes? Theory: Anxiety transmits the effect of cues on attitudes Anxiety, M Media Cue, T Immigration Attitudes, Y ACME = Average difference in immigration attitudes due to the change in anxiety induced by the media cue treatment Sequential ignorability = No unobserved covariate affecting both anxiety and immigration attitudes Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 14 / 27

Traditional Estimation Method Linear structural equation model (LSEM): M i = α 2 + β 2 T i + ξ 2 X i + ɛ i2, Y i = α 3 + β 3 T i + γm i + ξ 3 X i + ɛ i3. Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 15 / 27

Traditional Estimation Method Linear structural equation model (LSEM): M i = α 2 + β 2 T i + ξ2 X i + ɛ i2, Y i = α 3 + β 3 T i + γm i + ξ3 X i + ɛ i3. Fit two least squares regressions separately Use product of coefficients ( ˆβ 2ˆγ) to estimate ACME Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 15 / 27

Traditional Estimation Method Linear structural equation model (LSEM): M i = α 2 + β 2 T i + ξ2 X i + ɛ i2, Y i = α 3 + β 3 T i + γm i + ξ3 X i + ɛ i3. Fit two least squares regressions separately Use product of coefficients ( ˆβ 2ˆγ) to estimate ACME The method is valid under SI Can be extended to LSEM with interaction terms Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 15 / 27

Traditional Estimation Method Linear structural equation model (LSEM): M i = α 2 + β 2 T i + ξ2 X i + ɛ i2, Y i = α 3 + β 3 T i + γm i + ξ3 X i + ɛ i3. Fit two least squares regressions separately Use product of coefficients ( ˆβ 2ˆγ) to estimate ACME The method is valid under SI Can be extended to LSEM with interaction terms Problem: Only valid for the simplest LSEMs Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 15 / 27

Proposed General Estimation Algorithm 1 Model outcome and mediator Outcome model: p(y i T i, M i, X i ) Mediator model: p(m i T i, X i ) These models can be of any form (linear or nonlinear, semi- or nonparametric, with or without interactions) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 16 / 27

Proposed General Estimation Algorithm 1 Model outcome and mediator Outcome model: p(y i T i, M i, X i ) Mediator model: p(m i T i, X i ) These models can be of any form (linear or nonlinear, semi- or nonparametric, with or without interactions) 2 Predict mediator for both treatment values (M i (1), M i (0)) 3 Predict outcome by first setting T i = 1 and M i = M i (0), and then T i = 1 and M i = M i (1) 4 Compute the average difference between two outcomes to obtain a consistent estimate of ACME Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 16 / 27

Proposed General Estimation Algorithm 1 Model outcome and mediator Outcome model: p(y i T i, M i, X i ) Mediator model: p(m i T i, X i ) These models can be of any form (linear or nonlinear, semi- or nonparametric, with or without interactions) 2 Predict mediator for both treatment values (M i (1), M i (0)) 3 Predict outcome by first setting T i = 1 and M i = M i (0), and then T i = 1 and M i = M i (1) 4 Compute the average difference between two outcomes to obtain a consistent estimate of ACME 5 Monte Carlo or bootstrap to estimate uncertainty Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 16 / 27

Example: Estimation under Sequential Ignorability Original method: Product of coefficients with the Sobel test Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 17 / 27

Example: Estimation under Sequential Ignorability Original method: Product of coefficients with the Sobel test Valid only when both models are linear w/o T M interaction (which they are not) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 17 / 27

Example: Estimation under Sequential Ignorability Original method: Product of coefficients with the Sobel test Valid only when both models are linear w/o T M interaction (which they are not) Our method: Calculate ACME using our general algorithm Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 17 / 27

Example: Estimation under Sequential Ignorability Original method: Product of coefficients with the Sobel test Valid only when both models are linear w/o T M interaction (which they are not) Our method: Calculate ACME using our general algorithm Product of Average Causal Outcome variables Coefficients Mediation Effect (δ) Decrease Immigration.347.105 δ(1) [0.146, 0.548] [0.048, 0.170] Support English Only Laws.204.074 δ(1) [0.069, 0.339] [0.027, 0.132] Request Anti-Immigration Information.277.029 δ(1) [0.084, 0.469] [0.007, 0.063] Send Anti-Immigration Message.276.086 δ(1) [0.102, 0.450] [0.035, 0.144] Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 17 / 27

Need for Sensitivity Analysis Even in experiments, SI is required to identify mechanisms SI is often too strong and yet not testable Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 18 / 27

Need for Sensitivity Analysis Even in experiments, SI is required to identify mechanisms SI is often too strong and yet not testable Need to assess the robustness of findings via sensitivity analysis Question: How large a departure from the key assumption must occur for the conclusions to no longer hold? Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 18 / 27

Need for Sensitivity Analysis Even in experiments, SI is required to identify mechanisms SI is often too strong and yet not testable Need to assess the robustness of findings via sensitivity analysis Question: How large a departure from the key assumption must occur for the conclusions to no longer hold? Sensitivity analysis by assuming {Y i (t, m), M i (t)} T i X i = x but not Y i (t, m) M i (t) T i = t, X i = x Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 18 / 27

Need for Sensitivity Analysis Even in experiments, SI is required to identify mechanisms SI is often too strong and yet not testable Need to assess the robustness of findings via sensitivity analysis Question: How large a departure from the key assumption must occur for the conclusions to no longer hold? Sensitivity analysis by assuming {Y i (t, m), M i (t)} T i X i = x but not Y i (t, m) M i (t) T i = t, X i = x Possible existence of unobserved pre-treatment confounder Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 18 / 27

Parametric Sensitivity Analysis Sensitivity parameter: ρ Corr(ɛ i2, ɛ i3 ) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 19 / 27

Parametric Sensitivity Analysis Sensitivity parameter: ρ Corr(ɛ i2, ɛ i3 ) Sequential ignorability implies ρ = 0 Set ρ to different values and see how ACME changes Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 19 / 27

Parametric Sensitivity Analysis Sensitivity parameter: ρ Corr(ɛ i2, ɛ i3 ) Sequential ignorability implies ρ = 0 Set ρ to different values and see how ACME changes When do my results go away completely? Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 19 / 27

Parametric Sensitivity Analysis Sensitivity parameter: ρ Corr(ɛ i2, ɛ i3 ) Sequential ignorability implies ρ = 0 Set ρ to different values and see how ACME changes When do my results go away completely? δ(t) = 0 if and only if ρ = Corr(ɛ i1, ɛ i2 ) where Y i = α 1 + β 1 T i + ɛ i1 Easy to estimate from the regression of Y i on T i : Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 19 / 27

Parametric Sensitivity Analysis Sensitivity parameter: ρ Corr(ɛ i2, ɛ i3 ) Sequential ignorability implies ρ = 0 Set ρ to different values and see how ACME changes When do my results go away completely? δ(t) = 0 if and only if ρ = Corr(ɛ i1, ɛ i2 ) where Y i = α 1 + β 1 T i + ɛ i1 Easy to estimate from the regression of Y i on T i : Alternative interpretation based on R 2 : How big does the effects of unobserved confounders have to be in order for my results to go away? Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 19 / 27

Example: Sensitivity Analysis Average Mediation Effect: δ(1) 0.4 0.2 0.0 0.2 0.4 1.0 0.5 0.0 0.5 1.0 Sensitivity Parameter: ρ ACME > 0 as long as the error correlation is less than 0.39 (0.30 with 95% CI) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 20 / 27

Beyond Sequential Ignorability Without sequential ignorability, standard experimental design lacks identification power Even the sign of ACME is not identified Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 21 / 27

Beyond Sequential Ignorability Without sequential ignorability, standard experimental design lacks identification power Even the sign of ACME is not identified Need to develop alternative research design strategies for more credible inference New experimental designs: Possible when the mediator can be directly or indirectly manipulated Observational studies: use experimental designs as templates Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 21 / 27

Crossover Design Recall ACME can be identified if we observe Y i (t, M i (t)) Get M i (t), then switch T i to t while holding M i = M i (t) Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 22 / 27

Crossover Design Recall ACME can be identified if we observe Y i (t, M i (t)) Get M i (t), then switch T i to t while holding M i = M i (t) Crossover design: 1 Round 1: Conduct a standard experiment 2 Round 2: Change the treatment to the opposite status but fix the mediator to the value observed in the first round Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 22 / 27

Crossover Design Recall ACME can be identified if we observe Y i (t, M i (t)) Get M i (t), then switch T i to t while holding M i = M i (t) Crossover design: 1 Round 1: Conduct a standard experiment 2 Round 2: Change the treatment to the opposite status but fix the mediator to the value observed in the first round Very powerful identifies mediation effects for each subject Must assume no carryover effect: Round 1 doen t affect Round 2 Can be made plausible by design Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 22 / 27

Example: Labor Market Discrimination Experiment Bertrand & Mullainathan (2004, AER) Treatment: Black vs. White names on CVs Mediator: Perceived qualifications of applicants Outcome: Callback from employers Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 23 / 27

Example: Labor Market Discrimination Experiment Bertrand & Mullainathan (2004, AER) Treatment: Black vs. White names on CVs Mediator: Perceived qualifications of applicants Outcome: Callback from employers Quantity of interest: Direct effects of (perceived) race Would Jamal get a callback if his name were Greg but his qualifications stayed the same? Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 23 / 27

Example: Labor Market Discrimination Experiment Bertrand & Mullainathan (2004, AER) Treatment: Black vs. White names on CVs Mediator: Perceived qualifications of applicants Outcome: Callback from employers Quantity of interest: Direct effects of (perceived) race Would Jamal get a callback if his name were Greg but his qualifications stayed the same? Round 1: Send Jamal s actual CV and record the outcome Round 2: Send his CV as Greg and record the outcome Assumptions are plausible Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 23 / 27

Designing Observational Studies Key difference between experimental and observational studies: treatment assignment Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 24 / 27

Designing Observational Studies Key difference between experimental and observational studies: treatment assignment Sequential ignorability: 1 Ignorability of treatment given covariates 2 Ignorability of mediator given treatment and covariates Both (1) and (2) are suspect in observational studies Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 24 / 27

Designing Observational Studies Key difference between experimental and observational studies: treatment assignment Sequential ignorability: 1 Ignorability of treatment given covariates 2 Ignorability of mediator given treatment and covariates Both (1) and (2) are suspect in observational studies Statistical control: matching, propensity scores, etc. Search for quasi-randomized treatments: natural experiments Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 24 / 27

Designing Observational Studies Key difference between experimental and observational studies: treatment assignment Sequential ignorability: 1 Ignorability of treatment given covariates 2 Ignorability of mediator given treatment and covariates Both (1) and (2) are suspect in observational studies Statistical control: matching, propensity scores, etc. Search for quasi-randomized treatments: natural experiments How can we design observational studies? Experiments can serve as templates for observational studies Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 24 / 27

Example: Incumbency Advantage Estimation of incumbency advantages goes back to 1960s Why incumbency advantage? Scaring off quality challenger Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 25 / 27

Example: Incumbency Advantage Estimation of incumbency advantages goes back to 1960s Why incumbency advantage? Scaring off quality challenger Use of cross-over design (Levitt and Wolfram, LSQ) 1 1st Round: two non-incumbents in an open seat 2 2nd Round: same candidates with one being an incumbent Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 25 / 27

Example: Incumbency Advantage Estimation of incumbency advantages goes back to 1960s Why incumbency advantage? Scaring off quality challenger Use of cross-over design (Levitt and Wolfram, LSQ) 1 1st Round: two non-incumbents in an open seat 2 2nd Round: same candidates with one being an incumbent Assumption: challenger quality (mediator) stays the same Estimation of direct effect is possible Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 25 / 27

Concluding Remarks Quantitative analysis can be used to identify causal mechanisms! Estimate causal mediation effects rather than marginal effects Wide applications across social and natural science disciplines Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 26 / 27

Concluding Remarks Quantitative analysis can be used to identify causal mechanisms! Estimate causal mediation effects rather than marginal effects Wide applications across social and natural science disciplines Under standard research designs, sequential ignorability must hold for identification of causal mechanisms Under SI, a general, flexible estimation method is available SI can be probed via sensitivity analysis Easy-to-use software mediation is available in R and STATA Credible inference is possible under alternative research designs Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 26 / 27

Concluding Remarks Quantitative analysis can be used to identify causal mechanisms! Estimate causal mediation effects rather than marginal effects Wide applications across social and natural science disciplines Under standard research designs, sequential ignorability must hold for identification of causal mechanisms Under SI, a general, flexible estimation method is available SI can be probed via sensitivity analysis Easy-to-use software mediation is available in R and STATA Credible inference is possible under alternative research designs Ongoing research: multiple mediators, instrumental variables Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 26 / 27

The project website for papers and software: http://imai.princeton.edu/projects/mechanisms.html Email for comments and suggestions: kimai@princeton.edu Kosuke Imai (Princeton) Causal Mechanisms Dartmouth (February 23, 2012) 27 / 27