Causal Interaction in Factorial Experiments: Application to Conjoint Analysis

Size: px

Start display at page:

Download "Causal Interaction in Factorial Experiments: Application to Conjoint Analysis"

Scarlett Morris
5 years ago
Views:

1 Causal Interaction in Factorial Experiments: Application to Conjoint Analysis Naoki Egami Kosuke Imai Princeton University Talk at the University of Macau June 16, 2017 Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 1 / 23

2 Causal Heterogeneity and Interaction Effects Causal inference revolution in social sciences Randomized experiments: laboratory, field, and survey experiments Observational studies: natural experiments, research designs Many methods for estimating average treatment effect (ATE) Beyond ATE Causal heterogeneity 1 Moderation: How does the effect of a treatment vary across individuals? Interaction between the treatment variable and pre-treatment covariates 2 Causal interaction: What combination of treatments is efficacious? Interaction among multiple treatment variables 3 Individualized treatment regimes: What treatment combination is optimal for a given individual? Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 2 / 23

3 Factorial Experiments for Causal Interaction how many visits voter turnout by not necessary to to canvassers or olitical scientists, tiveness of varitra information. s less effective, ffective because y visits than by pliance rates for Causal interaction requires multiple treatments Vol. 99, No. 2 Randomized experiments with a factorial design Factor = categorical variable with discrete values or levels TABLE 1. Example: 2 2 The Original Experimental Design 3 4 design (Gerber and Green, 2000) Reported in Gerber and Green (2000) Mail None Once Twice 3 times Phone Visit Civic Neighbor/civic a Close No visit Civic Neighbor/civic a Close ILIZATION No phone Visit Civic 1, end Gerber and Neighbor bilization study. Close nd conducted an No visit Civic in randomly sere encouraged to Neighbor 10, y means of pers. Close They then ex- Note: The figures represent the number of registered voters in which strategies FactorialNew design Haven for iseach often treatment used assignment for audit combination. studies For and conjoint analysis example, 122 voters were assigned to receive three postcards, ion to Egami the voting and Imai (Princeton) a phone call, and a personal visitcausal with theinteraction civic duty message. Macau (June, 2017) 3 / 23

4 Conjoint Analysis Survey experiments with a factorial design Respondents evaluate several pairs of randomly selected profiles defined by multiple factors Social scientists use it to analyze multidimensional preferences Example: Immigration preference (Hopkins and Hainmueller 2014) representative sample of 1,407 American adults each respondent evaluates 5 pairs of immigrant profiles gender 2, education 7, origin 10, experience 4, plan 4, language 4, profession 11, application reason 3, prior trips 5 What combinations of immigrant characteristics do Americans prefer? High dimension: over 1 million treatment combinations Methodological challenges: Many interaction effects false positives, difficulty of interpretation Very few applied researchers study interaction Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 4 / 23

5 The Overview of the Talk 1 New causal estimand: Average Marginal Interaction Effect (AMIE) relative magnitude does not depend on baseline condition intuitive interpretation even for high dimension estimation using ANOVA with weighted zero-sum constraints regularization done directly on AMIEs 2 Comparison with the conventional interaction effect: lack of invariance to the choice of baseline condition difficulty of interpretation for higher-order interaction 3 Reanalysis of the conjoint analysis on ethnic voting in Africa Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 5 / 23

6 Factorial Experiments with Two Treatments Two factorial treatments (e.g., gender and race): A A = {a 0, a 1,..., a LA 1} B B = {b 0, b 1,..., b LB 1} Assumption: Full factorial design 1 Randomization of treatment assignment {Y (a l, b m )} al A,b m B {A, B} 2 Non-zero probability for all treatment combination Pr(A = a l, B = b m ) > 0 for all a l A and b m B Fractional factorial design not allowed 1 Use a small non-zero assignment probability 2 Focus on a subsample 3 Combine treatments Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 6 / 23

7 Main Causal Estimands in Factorial Experiments 1 Average Combination Effect (ACE): Average effect of treatment combination (A, B) = (a l, b m ) relative to the baseline condition (A, B) = (a 0, b 0 ) τ AB (a l, b m ; a 0, b 0 ) = E{Y (a l, b m ) Y (a 0, b 0 )} Effect of being Asian male 2 Average Marginal Effect (AME; Hainmueller et al. 2014; Dasgupta et al. 2015): Average effect of treatment A = a l relative to the baseline condition A = a 0 averaging over the other treatment B ψ A (a l, a 0 ) = E{Y (a l, B) Y (a 0, B)}dF (B) Effect of being male averaging over race Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 7 / 23

8 The New Causal Interaction Effect Average Marginal Interaction Effect (AMIE): π AB (a l, b m ; a 0, b 0 ) = τ AB (a l, b m ; a 0, b 0 ) ψ }{{} A (a l, a 0 ) ψ }{{} B (b m, b 0 ) }{{} ACE of (a l, b m) AME of a l AME of b m Interpretation: additional effect induced by A = a l and B = b m together beyond the separate effect of A = a l and that of B = b m Additional effect of being Asian male beyond the sum of separate effects for being male and being Asian Decomposition of ACE: τ AB = ψ A + ψ B + π AB Invariance: the relative magnitude of AMIE does not depend on the choice of baseline condition AMIEs depend on the distribution of treatment assignment: 1 specified by one s experimental design 2 motivated by a target population Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 8 / 23

9 The Conventional Causal Interaction Effect Average Interaction Effect (AIE): ξ AB (a l, b m ; a 0, b 0 ) = E{Y (a l, b m ) Y (a 0, b m ) Y (a l, b 0 ) + Y (a 0, b 0 )} Equal to linear regression coefficients Interactive effect interpretation (similar to AMIE): τ AB (a l, b m ; a 0, b 0 ) }{{} ACE of (a l, b m) Conditional effect interpretation: E{Y (a l, b 0 ) Y (a 0, b 0 )} E{Y (a 0, b m ) Y (a 0, b 0 )} }{{}}{{} Effect of A = a l when B = b 0 Effect of B = b m when A = a 0 E{Y (a l, b m ) Y (a 0, b m )} E{Y (a l, b 0 ) Y (a 0, b 0 )} = E{Y (a l, b m ) Y (a l, b 0 )} E{Y (a 0, b m ) Y (a 0, b 0 )} difference in effect of being male between Asian and White difference in effect of being Asian between male and female Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 9 / 23

10 Comparison between AMIE and AIE AIE is NOT invariant to baseline category: 1 cannot compare regression coefficients 2 zero interaction when a baseline category is involved ξ AB (a l, b 0 ; a 0, b 0 ) = ξ AB (a 0, b m ; a 0, b 0 ) = 0 for all l, m 3 cannot regularize regression coefficients AMIE and AIE are closely related: 1 Conditional effect as a function of AMIE E{Y i (a l, b 0 ) Y i (a 0, b 0 )} = ψ A (a l ; a 0 ) + π AB (a l, b 0 ; a 0, b 0 ) 2 AIE is a linear function of AMIEs ξ AB (a l, b m; a 0, b 0) = π AB (a l, b m; a 0, b 0) π AB (a l, b 0; a 0, b 0) π AB (a 0, b m; a 0, b 0) 3 AMIE is also a linear function of AIEs 4 No causal interaction zero AMIEs, zero AIEs Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 10 / 23

11 Higher-order Causal Interaction J factorial treatments with L j levels each: T = (T 1,..., T J ) Assumptions: 1 Full factorial design Y (t) T and Pr(T = t) > 0 for all t 2 Independent treatment assignment T j T j for all j Assumption 2 is not necessary for identification but considerably simplifies estimation We are interested in the K-way interaction where K J We extend all the results for the 2-way interaction to this general case Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 11 / 23

12 Higher-order Average Marginal Interaction Effect General definition: the difference between ACE and the sum of all lower-order AMIEs (first-order AMIE = AME) Example: 3-way AMIE, π 1:3 (t 1, t 2, t 3 ; t 01, t 02, t 03 ), equals τ 1:3 (t 1, t 2, t 3 ; t 01, t 02, t 03 ) }{{} ACE { π 1:2 (t 1, t 2 ; t 01, t 02 ) + π 2:3 (t 2, t 3 ; t 02, t 03 ) + π 1:3 (t 1, t 3 ; t 01, t 03 ) } }{{} sum of all 2-way AMIEs { ψ(t 1 ; t 01 ) + ψ(t 2 ; t 02 ) + ψ(t 3 ; t 03 ) } }{{} sum of AMEs Properties: 1 K-way ACE = the sum of all K-way and lower-order AMIEs 2 Invariance to the baseline condition Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 12 / 23

13 Difficulty of Higher-order AIEs Generalize the 2-way ATIE by marginalizing the other treatments T 1:2 ξ 1:2 (t 1, t 2 ; t 01, t 02 ) = E { Y (t 1, t 2, T 1:2 ) Y (t 01, t 2, T 1:2 ) Y (t 1, t 02, T 1:2 ) + Y (t 01, t 02, T 1:2 ) } df (T 1:2 ) In the literature, the 3-way ATIE is defined as ξ 1:3 (t 1, t 2, t 3 ; t 01, t 02, t 03 ) = ξ 1:2 (t 1, t 2 ; t 01, t 02 T 3 = t 3 ) ξ }{{} 1:2 (t 1, t 2 ; t 01, t 02 T 3 = t 03 ) }{{} 2-way AIE when T 3 = t 3 Higher-order ATIEs are similarly defined sequentially 2-way AIE when T 3 = t 03 This representation is based on the conditional effect interpretation Problem: conditional effect of conditional effects! Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 13 / 23

14 Nonparametric Estimation of AMIE 1 Difference-in-means estimator estimate ACE and AMEs using the difference-in-means estimators estimate AMIE as ˆπ AB = ˆτ AB ˆψ A ˆψ B higher-order AMIEs can be estimated sequentially uses the empirical treatment assignment distribution 2 ANOVA based estimator saturated ANOVA include all interactions up to the Jth order weighted zero-sum constraints: for all factors and levels, L A 1 l=0 L B 1 m=0 Pr(A i = a l )β A l = 0, Pr(B i = b m)β B m = 0, L A 1 l=0 L B 1 m=0 AMIEs are differences of coefficients: Pr(A i = a l )β AB lm = 0, Pr(B i = b m)β AB lm = 0, and so on E( ˆβ A l ˆβ A 0 ) = ψ A (a l ; a 0 ), E( ˆβ AB lm ˆβ AB 00 ) = π AB (a l, b m ; a 0, b 0 ) can use any marginal treatment assignment distribution of choice Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 14 / 23

15 Regularization via GASH-ANOVA Too many coefficients to be estimated over fitting, false positives, difficult interpretation Need for regularization by collapsing levels and selecting factors Grouping and Selection using Heredity in ANOVA (Post and Bondell): wll A max{φa (l, l )} + wmm B max{φb (m, m )} l,l m,m where φ A (l, l ) = βl A βa l }{{} L B 1 βlm AB βab l m }{{} m=0 AME AMIE The adaptive weight takes the following form: w A ll = [(L A + 1) L A max{ φ A (l, l )} ] 1 c }{{} cost parameter where φ A (l, l ) is AMEs and AMIEs estimated without regularization Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 15 / 23

16 Conjoint Analysis of Ethnic Voting in Africa Ethnic voting and accountability: Carlson (2015, World Politics) Do voters prefer candidates of same ethnicity regardless of their prior performance? Do ethnicity and performance interact? Conjoint analysis in Uganda: 547 voters from 32 villages Each voter evaluates 3 pairs of hypothetical candidates 5 factors: Coethnicity 2, Prior record 2, Prior office 4, Platform 3, Education 8 Prior record = No if Prior office = businessman combine these two factors into a single factor with 7 levels Collapse Education into 2 levels: relevant degrees (MA in business, law, economics, development) and other degrees Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 16 / 23

17 A Statistical Model of Preference Differentials ANOVA regression with one-way and two-way effects: Y i (T i ) = µ + L J j 1 β j l 1{T ij = l} + L j 1 j j j=1 l=0 l=0 L j 1 m=0 β jj lm 1{T ij = l, T ij = m} + ɛ i with appropriate weighted zero-sum constraints In conjoint analysis, we observe the sign of preference differentials Linear probability model of preference differential: Pr(Y i (T i ) > Y i (T i ) T i, T i ) L J j 1 = µ + β j l (1{T ij = l} 1{Tij = l}) + j j j=1 L j 1 l=0 l=0 L j 1 m=0 β jj lm (1{T ij = l, T ij = m} 1{Tij = l, T ij = m}) where µ = 0.5 if the position of profile does not matter We apply GASH-ANOVA to this model Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 17 / 23

18 Ranges of Estimated AMEs and AMIEs Selection Range prob. AME Record Coethnicity Platform Degree AMIE Coethnicity Record Record Platform Platform Coethnic Coethnicity Degree Platform Degree Record Degree Factor selection probability based on bootstrap Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 18 / 23

19 Close Look at the Estimated AMEs Selection Factor AME prob. Record Yes/Village Yes/District Yes/MP No/Village No/District No/MP { No/Businessman Platform { Jobs Clinic base base { Education Coethnicity Degree Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 19 / 23

20 Effect of Regularization on AMIEs Yes/Village Yes/District Yes/MP No/Village No/District No/MP No/Business Yes/Village Yes/District Yes/MP No/Village No/District No/MP No/Business Platform Coethnicity Yes/Village Yes/District Yes/MP No/Village No/District No/MP No/Business Record Yes/Village Yes/District Yes/MP No/Village No/District No/MP No/Business Record Without Regularization With Regularization Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 20 / 23

21 Decomposition and Conditional Effects Decomposition of ACE (Coethnicity Record interaction): τ(coethnic, No/Business; Non-coethnic, No/MP) }{{} 2.4 = ψ(coethnic; Non-coethnic) + ψ(no/business; No/MP) }{{}}{{} π(coethnic, No/Business; Non-coethnic, No/MP) }{{} 3.1 Conditional effects (Platform Record interaction): AMIE: π(education, No/MP}; {Job, No/MP}) = 2.3 Conditional effect of Education relative to Job for No/MP is approximately zero AME: ψ(education; Job) = 2.3 Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 21 / 23

22 Concluding Remarks Interaction effects play an essential role in causal heterogeneity 1 moderation 2 causal interaction Randomized experiments with a factorial design 1 useful for testing multiple treatments and their interactions 2 social science applications: audit studies, conjoint analysis 3 challenge: estimation and interpretation in high dimension Average Marginal Interaction Effect (AMIE) 1 invariant to baseline condition 2 straightforward interpretation even for high order interaction 3 enables effect decomposition 4 enables regularization through ANOVA Designing factorial experiments (work in progress) 1 select factors and levels via our method to reduce dimension 2 use unregularized ANOVA for the main study Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 22 / 23

23 References 1 Egami, Naoki and Kosuke Imai. Causal Interaction in Factorial Experiments: Application to Conjoint Analysis. Working paper available at 2 Egami, Naoki, Marc Ratkovic, and Kosuke Imai. FindIt: Finding Heterogeneous Treatment Effects. R package available at CRAN Send comments and suggestions to negami@princeton.edu or kimai@princeton.edu Egami and Imai (Princeton) Causal Interaction Macau (June, 2017) 23 / 23

Causal Interaction in Factorial Experiments: Application to Conjoint Analysis

Causal Interaction in Factorial Experiments: Application to Conjoint Analysis Naoki Egami Kosuke Imai Princeton University Talk at the Institute for Mathematics and Its Applications University of Minnesota