Potential Outcomes Model (POM)

Transcription

1 Potential Outcomes Model (POM) Relationship Between Counterfactual States Causality Empirical Strategies in Labor Economics, Angrist Krueger (1999): The most challenging empirical questions in economics involve what if statements about counterfactual outcomes. Interest in these questions is motivated by immediate policy concerns, theoretical considerations, problems facing individual decision makers. The causal relationships at the heart of many economic quesitons involve comparisons of counterfactual states of the world. Someone would like to know what outcomes would have been observed if a variable were manipulated or changed in some way. Differences in the observed outcomes for a state that occured the unobserved potential outcomes for a possible state that did not occur, which is known as the counterfactual, define the the causual effects of interest. Fundamental Problem of Causal Inference Consider a binary treatment for observation i, denoted T i. The potential outcomes for each i are: { y 0i if T i = 0 y i y 1i if T i = 1 We just assume that the treatment of i only affects outcomes of i; this is known as the stable unit treatment value assumption (SUTVA). We want to observe y 0i y 1i for all i. The treatment effect for i is y 1i y 0i With an i.i.d. sample from the population, this is a rom variable with a specific distribution. We might be interested in several features of this distribution. The average treatment effect (ATE) is the expected treatment effect for a romly drawn person from the distribution, i.e., AT E = E[y 1i y 0i ] The ATE is useful for treatment with broad applicability (e.g., nutrition, taxes, etc.), but it is not always relevant. This is because a large segment of the population may not be eligible for treatment. 1

2 For example, the average effect of job training is not very interesting when the population includes millionaires. What is the average treatment effect on the treated (ATET)? For those who received treatment, how did it affect their outcomes? AT ET = E[y 1i y 0i T i = 1] For what fraction of the population is the treatment effect greater than 0? Remember, individual treatment effects are not observable. Only one outcome out of y 0i y 1i is actually observed. We call the unobserved potential outcome the counterfactual. We can see that or y i = y 1i T i + y 0i (1 T i ) y i = y 0i + (y 1i y 0i )T i Our hope is that averages ( other statistics) can inform us about the distribution of treatment effects despite these issues. Example. Suppose we wish to examine the effect of a college education on the present-day value of lifetime earnings. Let T college education y PDV of lifetime earnings. Assume that individuals know their potential outcomes the cost of T is $260,000. y 0i y 1i i HS College Treatment Effect T i Observed Outcome Low Ability 1.7 (Observed) 1.9 (Not observed) High Ability 2.8 (Not observed) 3.6 (Observed) The observed difference across T = 0,1 is = 1.9 We can also calculate the average treatment effect the average treatment effect on the treated: AT E = (.2+.8)/2 =.5 AT ET =.8 Selection Bias in POM Let us consider the following switching model, in which different groups have different expectation functions. { E[y 0i ] + v 0i = µ 0 + v 0i if T i = 0 y i = E[y 1i ] + v 1i = µ 1 + v 1i if T i = 1 Page 2 of 8

3 where E[v 0i ] = 0 E[v 1i ] = 0, i.e., some benefit more than average others less. However, it is important to note that E[v 0i T i ] E[v 1i T i ] may not equal zero. So, we have y i = T i (µ 1 + v i1 ) + (1 T i )(µ 0 + v 0i ) = µ 0 + T i (µ 1 µ 0 + v 1i v 0i ) + v 0i where the second term is the effect of treatment on observation i. We can clearly see that AT E = E[y 1i y 0i ] = µ 1 µ 0 AT ET = E[y 1i y 0i T i = 1] = µ 1 µ 0 + E[v 1i v 0i T i ] Think about v 1i v 0i as individual specific gains from treatment. What do we do when comparing expected outcomes of treated controls? E[y i T i = 1] E[y i T i = 0] = E[µ 0 + µ 1 µ 0 + v 1i v 0i + v 0i T i = 1] E[µ 0 + v i0 T i = 0] = E[µ 1 + v 1i T i = 1] E[µ 0 + v 0i T i = 0] = µ 1 µ 0 + E[v 1i T i = 1] E[v 0i T i = 0] = AT E + selection bias The same thing arises with conditional expectations: E[y 1i x] = µ 1 (x) E[y 0i x] = µ 0 (x) Under What Conditions Can We Identify Meaningful Parameters? Pure Rom Assignment With pure rom assignment, we have y 0i,y 1i T i Page 3 of 8

4 which implies that the distribution of potential outcomes is independent of treatment status. Let F( ) denote the CDF, then F(y 0i T i = 0) = F(y 0i T i = 1) = F(y 0i ) F(y 1i T i = 0) = F(y 1i T i = 1) = F(y 1i ) So, knowing the treatment status does not provide any information about potential outcomes. In addition, there are implications for all moments of the distribution, i.e., E[y 0i T i = 0] = E[y 0i T i = 1] = E[y 0i ] E[y 1i T i = 0] = E[y 1i T i = 1] = E[y 1i ] This allows for the consideration of counterfactuals in the aggregate. Ideally, we would like to consider ÂT E = 1 N (y 1i y 0i ) Instead, we can estimate 1 N y 1i 1 T =1 N y 0i T =0 which converges in probability to E[y 1i T i = 1] E[y 0i T i = 0] = E[y 1i T i = 1] E[y 0i T i = 1] (by the assumption of rom assignmen = E[y 1i y 0i T i = 1] (ATET) = E[y 1i ] E[y 0i ] = E[y 1i y 0i ] (ATE) Therefore, with pure rom assignment, the difference in means identify the average treatment effect (ATE) the average treatment effect on the treated (ATET). Rom Assignment Conditional on Observables The other names for this condition are: ignorability of treatment, unconfoundedness, selection on observables. Also, how often this condition is likely to hold depends on the context. Now, with rom assignment conditional on observables, we have (y 1i,y 0i T i ) x i Page 4 of 8

5 So, the distribution of potential outcomes will be independent of the treatment conditional on x, which means that F(y 0i x i,t i = 0) = F(y 0i x i,t i = 1) = F(y 0i x i ) F(y 1i x i,t i = 0) = F(y 1i x i,t i = 1) = F(y 1i x i ) By implication, the same result holds true for expectations, i.e, E[y 1i x i,t i = 1] E[y 0i x i,t i = 0] = E[y 1i x] E[y 0i x] Taking expecation over x yields E[AT E(x)] = δ x P(x i = x) x = AT E where δ x = AT E(x) = E[y 1i y 0i x] = AT E(x) If we assume overlap, then we can estimate these terms. Assumption. (The Overlap Assumption): x : 0 < P(T i = 1 x) < 1 Incomplete overlap implies that δ x is undefined. The given assumption suggests an obvious approach to estimation: a. For all x, compare average outcomes across treatment controls, i.e., ȳ T =1,x ȳ T =0,x, to obtain ˆδ x b. Weight differences by a fraction of the population for each x, i.e., P(xi = x) or N x i =x N. It is important to note that this exact matching approach is non-parametric since we have not imposed any structure on the functional form of the conditional expectation function. Step 1 of the procedure above is equivalent to estimating a fully saturated regression model, one that allows expectations to differ across all T x combinations. E[y i x i,t i ] = β 1 x 1i + γ 1 x 1i T i + β 2 x 2i + γ 2 x 2i T i + = β x 1(x i = x) + γ x 1(x i = x)t i x x where x 1,...,x K are indicators reflecting all possible combinations of x. However, unlike a model that is just saturated-in-x i, step 2 has us weight differences by a fraction of the population for each x. Page 5 of 8

6 Note. x can represent a group of observations. Example. Suppose x represents years of schooling, then x 0 = 1(Years = 0) x 1 = 1(Years = 1) x 2 = 1(Years = 2). Now, suppose x signifies gender whether person i was a high school graduate. x 1 = 1(Male = 1)1(HSgrad = 1) x 2 = 1(Male = 1)1(HSgrad = 0) x 3 = 1(Male = 0)1(HSgrad = 1) x 4 = 1(Male = 0)1(HSgrad = 0) Finally, suppose x is only associated with gender. E[y i1 x,t i ] = γ 1 1(Male = 1) + γ 2 (Male = 1)T i + γ 3 (Male = 0) + γ 4 (Male = 0)T i = θ 1 + θ 2 Male i + θ 3 T i + θ 4 Male i T i Therefore, different parameterizaitons can estimate the same fully saturated model. Clearly, this exact matching approach controls for x. What about an OLS procedure that only controls fully for x? This type of model is said to be saturated-in-x i since it includes a parameter for every value of x i, i.e., y i = β x 1(x i = x) + δ R T i x This model is not fully saturated, however, because there is a single additive effect for T i with no T i x i interactions. While the OLS regression exact matching estimates are inherently different, they are still somewhat similar because regression, too, can be seen as a sort of matching estimator. The regression estim differs from the matching estims only in the weights used to combine the covariate-specific effects, δ x, into a single average effect. In particular, while matching uses the distribution of covariates among the treated to weight covariate-specific estimates into an estimate of the effect of treatment on the treated (or the average treatment effect), regression produces a variance-weighted average of these effects. Important to note, the treatment-on-the-treated estim puts the most weight on covariate cells containing those who are most likely to be treated. In contrast, OLS regression is efficient for the homoskedastic, constant effects linear model. This implies that regression puts the most weight on covarite cells where the conditional variance of treatment status is large. As a rule, treatment variance is maximized when P(T i = 1 x i = x) = 0.5, in other words, for cells where there are Page 6 of 8

7 equal numbers of treated control observations, not the x where most of the population lies. Consequently, OLS may not give us what we want if heterogeneity is present in treatment effects probability of treatment. What if x includes many variables, including ones that are continuous? Then, we have to make assumptions about the conditional expectation function. The linear regression model is one choice, i.e., E[y i x i,t i ] = x iβ + ψt i Keep in mind that this is a very restrictive model. We have a linear conditional expectation if observation i does not receive treatment µ 0 (X i ) = x iβ a linear conditional expectation if observation i does receive treatment µ 1 (X i ) = x iβ + ψ This represents a homogeneous additive treatment effect. Though, instead of a linear regression model, the true functional form may involve: non-linearities in x interactions between x heterogeneous treatment effects selection on observables Non-parametric estimation can be straight forward: ignore x without treated observation. It is tricky if x is continuous or has many variables/values (even more so because we don t know what range of x should be considered). What about just obtaining an average treatment effect on the treated (ATET)? Well, we can relax to the complete overlap assumption to: x : P(T = 1 x) < 1 Takeaways Though weaker than RA, the ignorability assumption is itself quite strong in most circumstances. Even if the assumption holds conditional on x i, it may not hold conditional on x i β. That being said, the assumption seems to have a better chance of holding when we have access to a richer set of observables, but x can still be too rich. Yet, everyone acknowledges the challenge of identifying meaningful parameters. There are two approaches we can employ, which are not mutually exclusive: Look for data where x is very rich Look for, or create, circumstances associated data where treatment is plausibly rom, or rom conditional on x. Page 7 of 8

8 Review Let us recall what we need to get the average treatment effect: Observe all variation related to treatment, T, potential outcomes. For all x, P(T = 1 x) is not equal to 0 or 1. We must know about the process of how observable variables influence outcomes, which is relevant for obtaining the correct function. We must know how treatment influences outcomes, which is also relevant for obtaining the correct functional form. Examples DiNardo Pischke (1997) This paper was a response to Krueger (1993), who used rich observational data to argue that computers increased wages. There was a concern with the original study. Computer users are fundamentally different from non-computer users (i.e., white collar vs. blue collar) even after controlling for observable characteristics The falsification tests asked: does the identication strategy yield correct estimates when the truth is known? Are there effects of variables that should not have effects? Are there effects on outcomes, or characteristics, that should not be affected? Are there effects on groups that are not treated? Are there effects on time periods that are not treated? The authors used data from Germany because it was comparable to data from the US, it offered information on other tools that the US data did not. The identification strategy was not convincing. Even with a rich set of observable characteristics, it was still difficult to obtain unbiased estimates with observational data. Freedman (1991) Snow s research on Cholera as a shining example of research using observational data. Snow s research is question driven. Is Cholera transmitted from person to person, in particular, through contact with human waste? Page 8 of 8