Impact Evaluation Technical Workshop:

Transcription

1 Impact Evaluation Technical Workshop: Asian Development Bank Sept 1 3, 2014 Manila, Philippines Session 19(b) Quantile Treatment Effects

2 I. Quantile Treatment Effects Most of the evaluation literature is concerned with mean treatment effects. However, it is often also of interest to know the distribution of treatment effects, for example, to assess how many individuals experience a benefit from the program and to what extent program effects vary across individuals. This section considers the class of quantile treatment effect (QTE) estimators, drawing heavily on a recent survey by Frohlich and Melly (2010) that reviews five types of QTE estimators (and their implementation in Stata): Conditional QTE when treatment is exogenous conditional on X Conditional QTE when treatment is endogenous and a binary IV Z is available Unconditional QTE with random treatment assignment (i.e. a randomized trial) Unconditional QTE when treatment is exogenous conditional on X Unconditional QTE when treatment is endogeneous. This lecture discusses only the first four types of QTE estimators (the last type has only recently been developed and is only beginning to be applied). 2

3 As in previous lectures, write the observed outcome as: Y (observed) = PY 1 + (1 P)Y 0 where P is an indicator for program participation. The treatment effect, Δ = Y 1 Y 0, is not directly observed for anyone, even in a randomized experiment. Also, in almost all cases the treatment effect will vary over the population, so that there is a distribution of treatment effects. Estimating the distribution of treatment effects requires additional assumptions (relative to estimating average treatment effects). To get an idea of what we mean by distribution of treatment effects, let s look at the following two figures. 3

4 Example: Linear Regression with Homoskedasticity Source: Deaton (1997, p.80) 4

5 The figure above shows the standard case of a simple linear regression with homoskedasticity (i.e. variance of y does not depend on the value of x). The regression line E[y x] = α + β x is the expected values of y conditional on x, and the three bellshaped curves illustrate the conditional densities of the errors given x (imagine they are rising perpendicularly from the page). The three straight lines are the quantile regression lines that connect the points on the 10 th, the 50 th and the 90 th percentiles of the distribution of y, for each given value of x. Note that when the distribution of errors (the deviations from the mean) is symmetric (as in this case), the conditional mean (or regression function) is simply the 50 th percentile line. So in the homoskedasticity case, the quantile regression lines are parallel (all quantile regression lines have the same slope β). 5

6 Example: Food Engle curve under heteroscedasticity The next figure shows an example with heteroskedastity (i.e. non constant variance), using data from rural Pakistan. It shows a food Engle curve ( y = budget share for food; x = log (per capita expenditure)) for 9,119 households interviewed in the Household Income and Expenditure Survey of Pakistan. Notice two characteristics of the distribution of income effects on food consumption. First, the slopes of the three quantile regression lines (the β s) differ, although they are all negative. Second, the 10 th and 90 th percentiles of the conditional distribution are much further apart among richer than poorer households. So, we know that while richer households devote much less of their budgets to food, they exhibit more dispersion of tastes. 6

7 Source: Deaton (1997, pp.81) 7

8 QTE Estimators Alternative QTE estimators make different types of identifying assumptions, and the conditional and unconditional versions focus on different parameters of interest. Conditional QTE estimators recover features of the distribution of the treatment effect conditional on X. Unconditional QTE estimators recover features of the distribution of the treatment impact without conditioning on X. To think about the difference, note that an individual can be high in the overall Δ distribution, while at the same time being low in the conditional distribution. (Such a person would have values of X that are associated with a large value of Δ, but relative to other people with those values of X would have a low Δ.) 8

9 Our definition of the conditional QTE estimators will assume a parametric model for outcomes. As discussed by Frohlich and Melly (2010), parametric conditional QTE estimators converge at the parametric ( n) rate, which is faster than the rate of convergence for nonparametric conditional QTE estimators. The parametric versions are more often used in applied work. In contrast, unconditional QTE estimators can be estimated at a any parametric assumptions. n rate without making 9

10 Conditional QTE when Treatment (Participation) is Exogenous Conditional QTE estimators estimate the distribution of treatment effects under an assumption that treatment (participation) is exogenous conditional on a set of observables X. (Recall that this selection on observables assumption is also often used by matching estimators and cross sectional regression estimators.) Assume that the model for Y can be written as a linear function of X and P, but now the coefficients on X and P are allowed to vary by quantile as in the usual quantile regression framework. In effect, the quantile refers to the distribution of the error term in standard linear regression. The maintained assumption on potential outcomes is: Y = X β τ + Pδ τ + ε τ where it is also assumed that the τth quantile of the ε τ distribution is zero: Quant τ [ε τ ] = 0 where Quant τ [ε τ ] is the value of ε τ for which (τ 100)% of the density of ε τ is below. 10

11 In this equation, δ τ represents the conditional QTE. The assumption that treatment (participation) is exogenous conditional on a set of observables is: ε τ (P, X) Under these assumptions, the conditional QTE can be estimated using the standard quantile regression estimator introduced in Koenker and Bassett (1978) and implemented in Stata (using either the qreg or the newer ivqte commands). 11

12 Conditional QTE when Treatment (Participation) is Endogenous Sometimes the selection on observables assumption is doubtful. If treatment assignment is endogenous, but a binary instrument Z is available, then with some additional assumptions it is possible to implement the IV quantile regression estimator of Abadie, Angrist and Imbens (2002). They define two potential treatment indicators, P 0 and P 1, which indicate the value that P would take (for a particular person i) when Z equals 0 or 1, respectively. Just as Y 0 and Y 1 exist for all people, so do P 0 and P 1. The estimator proposed by Abadie et al. (2002) requires four assumptions: (Y 0, Y 1, P 1, P 0 ) Z X 0 < Prob[Z = 1 X] < 1 E[P 0 X] E[P 1 X] Prob[P 1 P 0 X] = 1 12

13 As in the LATE IV estimator, the population can be divided into four types of people. Those for whom P 1 > P 0 are referred to as compliers, because they are induced by a change in the value of the instrument to obtain the treatment. People for whom P 1 = P 0 = 1 are always takers, and people with P 1 = P 0 = 0 are never takers. The last assumption implies there are no people who would have treatment status P = 1 when Z = 0 but would have treatment status P = 0 when Z = 1. In the language of Abadie, Angrist and Imbens (2002), there are no defiers. Assuming the same linear model for outcomes used above, Y = X β τ + Pδ τ + ε τ, the conditional QTE can be estimated by a weighted quantile regression: ( ˆβ IV, ˆ IV) = arg min β, n W i ρ τ (Y i X i β P i δ) i 1 P (1 Z where W i = 1 1 Prob[ Z 1 X ] i i ) i (1 Pi ) Zi Prob[ Z 1 X ] i and ρ τ (u) = u(τ 1[u < 0]) 13

14 For example, if τ = 0.5, then ρ τ (u) = 0.5u when u > 0 and = 0.5*( u) when u < 0. Thus ρ τ (u) = 0.5 u, so it symmetrically weights positive and negative residuals. This method requires an initial estimator of Prob[Z = 1 X]. However, as Abadie, Angrist and Imbens (2002) note, and as Frohlich and Melly (2010) discuss, the problem is not convex because some weights are negative and some positive. Thus Abadie et al. suggest the use of alternative weights: W i + = E[W i Y i, P i, X i ] The weights are unknown and must be estimated. Frohlich and Melly (2010) discuss the use of Stata to estimate the weights, using the ivqte command. 14

15 Unconditional QTE with Random Treatment (Participation) Assignment When treatment (program participation) is exogenous, for example under strict random assignment, it is possible to implement an unconditional QTE estimator. The unconditional QTE (which does not condition on covariates) is given by: Δ τ = Quant τ [Y 1 ] Quant τ [Y 0 ] where Quant τ [Y i ] is the τth quantile of Y i distribution (i = 0 or 1). For example, if τ = 0.75, Quant 0.75 [Y 1 ] is the value in the Y 1 distribution for which 25% of the values of (the distribution of) Y 1 are greater than that value, and 75% of the values of (the distribution of) Y 1 are smaller than that value. For this simple estimator to give quantiles of the treatment impact distribution, we are implicitly assuming that the ranking of the Y 1 distribution lines up with the ranking of the Y 0 distribution (this was also true of the parametric model given above). That is, someone at the τ th quantile of the Y 0 distribution is assumed to also be at the same quantile of the Y 1 distribution, which is a strong assumption that is commonly made because we cannot directly determine the joint density from the data. For an alternative way of drawing inferences about the treatment impact distribution that relaxes the strong assumption on invariance of the rankings, see Heckman, Clements and Smith (1997). 15

16 Although covariates are excluded in the definition of the unconditional QTE with exogenous treatment assignment, they are sometimes used in estimation to improve the efficiency of the estimator. That it, they are included in a first stage regression and then integrated out. This is similar to the use of covariates to improve efficiency in estimating mean treatment effects under randomized assignment. See Frohlich and Melly (2010) for related discussion. An advantage of using unconditional QTE estimators in comparison to conditional QTE estimators is that they are fully nonparametric but can be still be estimated at a n rate. The conditional QTE model without covariates is the same as the unconditional QTE model. The estimator can be implemented by directly calculating quantiles of the Y 1 and Y 0 distributions, or using the qreg or newer ivqte commands in Stata. 16

17 Unconditional QTE when Treatment (Participation) is Exogenous Conditional on X Firpo (2007), Frohlich (2007b) and Melly (2006) all considered estimation of unconditional QTE for the case when treatment is exogenous conditional on observables X (selection on observables). The estimator they study can be written as: ( ˆ, ˆ ) = arg min, = n i 1 W i F ρ τ (Y i α P i Δ) where W F i = 1 Prob[ P 1 X ] i P i 1 Prob[ P 1 X ] i 1 P Note that these weights are equal to the inverse of the probability of participating, or the propensity score. Thus implementation requires obtaining a preliminary estimate of the propensity score. i i i 17

18 Unconditional QTE when Treatment (Participation) is Endogeneous A final case considered in the literature, which we will not discuss further here is the case where the object of interest is the unconditional QTE and where treatment assignment is endogenous. For a discussion of this case, see Frolich and Melly (2008). Also, see Frolich and Melly (2010) for an extensive discussion of how to implement the various QTE estimators and assess the variability of the estimators using the ivqte command in Stata. 18

19 Example: Union Effects on Wages in South Africa (Schultz & Mwabu, 1998, IIR) One interesting application of quantile regressions is the Schultz and Mwabu (1998) study on how South African unions affect the distribution of wages. Labor unions are an important economic and political force in South Africa, and estimates indicate that union members grew from 400,000 in 1985 to 1,205,612 in 1993, the latter figure representing 37% of workers that year. The union share of the labor force is quite high for a country with a relatively low income level. Yet wage inequality is higher in South Africa than in almost any other country in the world. Schultz and Mwabu (1998) explore how South African unions affect the distribution of economic welfare. They use quantile regressions to see how wages differ depending on whether or not a worker is a union member. Using nationally representative data from 9,000 households collected in 1993, their quantile regressions (Table 3) show that, among African (black) workers, union membership increases wages by 145% (exp. (.895) 1.0) at the bottom 10th percentile of the wage distribution and by 11% (exp. (.107) 1.0) at the top 90th percentile. Among white workers, union membership raises wages by 21% at the 10th percentile but reduces them by 24% at the 90th percentile (Table 4). 19

20 20

21 21

22 Schultz and Mwabu (1998) further examine the possibility that the union effects presented in Tables 3 and 4 may have captured some inter industry wage effects (e.g., some industries pay higher wages, and workers cannot easily move across industries). Controlling for nine industry groups, their quantile regressions confirm that a substantial part of the effect of union membership can be explained by industry categories, which perhaps reflects the influence of Industrial Councils or the spillover of administrated wages by industry. The union log wage advantage, controlling for industry, ranges from.35 for the lowest decile to.01 for the top decile among African men, and from.14 to.25 for white men at the bottom and top deciles, respectively. 22

23 23

24 24