Heterogeneous Treatment Effect Analysis

Transcription

1 Heterogeneous Treatment Effect Analyss Ben Jann ETH Zurch In cooperaton wth Jenne E. Brand (UCLA) and Yu Xe (Unversty of Mchgan) German Stata Users Group Meetng Berln, June 25, 2010 Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

2 Introducton Methods for causal nference from observatonal data have receved much attenton n the last two decades or so, especally n econometrcs, but also n many other felds. Startng pont of ths lterature s the Rubn Causal Model (a.k.a. Potental Outcomes Model a.k.a Counterfactual Causalty). Assume a bnary treatment varable D and let Y 1 and Y 0 be the potental outcomes wth and wthout treatment, respectvely. The treatment effect for ndvdual s then smply the dfference between the potental outcomes, that s δ = Y 1 Y 0 The fundamental problem of causal nference, however, s that we can only observe Y 1 or Y 0. One of the potental outcomes must be counterfactual because what we observe s Y 1 f D = 1 Y = Y 0 f D = 0 Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

3 Introducton The dea of defnng causalty n terms of potental outcomes s not new: Thus, f a person eats of a partcular dsh, and des n consequence, that s, would not have ded f he had not eaten of t, people would be apt to say that eatng of that dsh was the cause of hs death. 1 John Stuart Mll ( ) 1 John Stuart Mll (2002). A System of Logc. Reprnted from the 1981 edton (frst publshed 1843). Honolulu, Hawa: Unversty Press of the Pacfc. P Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

4 Introducton A basc paradgm of the lterature based on the potental outcomes model s that there can be ndvdual heterogenety n treatment effects, whch stands n contrast to tradtonal regresson modelng assumng constant parameters. The vew that treatment effects can be heterogeneous led to new methods for causal nference and also to new uses and nterpretatons of exstng methods (e.g. LATE nterpretaton of IV estmators, revval of matchng and regresson dscontnuty desgns). Surprsngly, however, not much attenton s usually pad to the explct analyss of the heterogenety of treatment effects n appled studes. The basc quantty of nterest s the average treatment effect (ATE) ATE = E[δ ] = E[Y 1 Y 0 ] = E[Y 1 ] E[Y 0 ] or sometmes the average treatment effect on the treated (ATT = E[δ D = 1]) or the average treatment effect on the untreated (ATC = E[δ D = 0]). Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

5 Introducton Why should we care about analyzng heterogeneous treatment effects? The nave estmator of the average treatment effect based on observatonal data can be decomposed as NATE = E[Y 1 D = 1] E[Y 0 D = 0] = E[δ ] + E[Y 0 D = 1] E[Y 0 D = 0] } {{ } pre-treatment heterogenety bas + (1 E[D ]) (E[δ D = 1] E[δ D = 0]) } {{ } treatment-effect heterogenety bas The focus of most estmaton approaches s to elmnate the frst type of bas, but also the second type of bas mght threaten the valdty of causal nference. Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

6 Introducton For example, n the lterature on economc returns to hgher educaton varous theores have been proposed that mply heterogeneous effects dependng on the probablty to go to college. Human-captal theory n economcs predcts postve selecton nto treatment, because people choose to go to college based on the expected economc returns. Ths s a wdely accepted vew. More socologcally orented lterature suggests that college attendance s strongly nfluenced by socal orgn, whch leads to negatve selecton nto treatment under certan condtons. To evaluate these theores t s therefore crucal to analyze how treatment effects vary wth treatment probablty. Ultmately, beleves about the mechansms at play determne educatonal polcy. Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

7 Analyss of Heterogeneous Treatment Effects To support the analyss of treatment-effect heterogenety we developed a new tool called hte. The approach of hte s to assume, at least provsonally, condtonal unconfoundedness gven a set of covarates and use propensty score stratfcaton to estmate treatment effects at varous ponts over the range of the propensty score. These strata-specfc effects are then analyzed to determne whether there s a trend n treatment effects. Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

8 Algorthm The hte algorthm conssts of four basc steps. 1. Estmaton of the propensty score (.e. the condtonal probablty to receve treatment). hte uses probt or logt, but t s also possble to manually estmate the propensty score beforehand and then provde t to hte. 2. Constructon of balanced propensty score strata. hte calls the pscore command for ths purpose Estmaton of strata-specfc average treatment effects. In each stratum, a regresson model on treatment s estmated, optonally ncludng control varables to account for remanng covarate mbalance wthn strata. 4. Estmaton of the trend of treatment effects across propensty score strata. hte regresses the strata-specfc treatment effects on strata rank usng varance weghted least squares (vwls; wth the varance based on the standard errors of the strata specfc treatment effects). 1 Becker, S. O., A. Ichno (2002). Estmaton of average treatment effects based on propensty scores. The Stata Journal 2: Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

9 Example Syntax Example An applcaton of the procedure can be found, for example, n: Brand, J. E., Y. Xe (2010). Who Benefts Most From College? Evdence for Negatve Selecton n Heterogeneous Economc Returns to Hgher Educaton. Amercan Socologcal Revew 75: Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

10 Syntax Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

11 Example. hte hghschool chldcare /// > female peduclow peduchgh lnhhnc motherlfp mmgrant /// > sblngs1-sblngs3 cohort1991-cohort1995 east Number of obs = 594 hghschool Coef. Std. Err. z P> z [95% Conf. Interval] TE by strata Lnear trend _slope _cons TE = treatment effect Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

12 Example slope of lnear trend (s.e.) = (0.039) Treatment Effect Propensty Score Strata 95% CI TE wthn strata lnear trend Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

13 Work n Progress Plans for hte: Optonal nonparametrc estmaton of propensty score. Improve the balanced propensty score strata algorthm and provde better output (descrptve nformaton on strata, balancng tests, etc.). Requres a rewrte of pscore. Automate wthn strata covarate adjustment. Formal tests for treatment-effect heterogenety. Improve level-2 estmaton. hte2: fully nonparametrc approach Estmate observaton-specfc counterfactual outcomes. Use non-parametrc estmators to analyze the trend n treatment effects over propensty score or across the values of covarates. Example. Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

14 Example psmatch2 chldcare /// female peduclow peduchgh lnhhnc motherlfp mmgrant /// sblngs brthyr east, /// outcome(hghschool) kernel bw(0.025) ate gen double treatefct = cond(chldcare==0, /// _hghschool-hghschool, hghschool-_hghschool) twoway scatter treatefct _pscore f chldcare==0 & _support==1, /// jtter(2) msym(oh) /// scatter treatefct _pscore f chldcare==1 & _support==1, /// jtter(2) msym(oh) psty(p4) /// lpoly treatefct _pscore f _support==1, /// pstyle(p6) lw(*2) degree(1) /// ylne(0) xt(propensty Score) yt(treatment Effect) /// legend(order(1 "chldcare==0" 2 "chldcare==1") /// cols(1) poston(4)) Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

15 Example Treatment Effect Propensty Score chldcare==0 chldcare==1 Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16

16 Thank you for lstenng! Ben Jann (ETH Zurch) Heterogeneous Treatment Effect Analyss DSUG / 16