Inference on distributional and quantile treatment effects

Transcription

1 Inference on distributional and quantile treatment effects David M. Kaplan University of Missouri Matt Goldman UC San Diego NIU, 214 Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 1 / 4

2 Outline 1 Distributional inference 2 Quantile inference 3 Nonparametric Bayesian inference on distributions and quantiles 4 Conditional QTE inference 5 Conclusion Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 2 / 4

3 1 Distributional inference 2 Quantile inference 3 Nonparametric Bayesian inference on distributions and quantiles 4 Conditional QTE inference 5 Conclusion Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 3 / 4

4 Example: effect of gift wage treatment on productivity Gneezy and List (26) Control: come work for advertised wage Treatment: paid more than advertised Tasks: data entry in library; door-to-door fundraising Sample sizes: 1 and 9 (lib); 1 and 13 (fund) Possible questions: did the treatment affect productivity in any way? how/did it affect the mean? how/did it affect the median, upper quartile, etc.? where in the distribution was the treatment effect statistically significant? Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 4 / 4

5 Example: effect of gift wage treatment on productivity Did the treatment affect productivity in any way? Permutation or Cramér von Mises test: H F T ( ) = F C ( ), equality of treatment and control CDFs Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 5 / 4

6 Example: effect of gift wage treatment on productivity Did the treatment affect productivity in any way? Permutation or Cramér von Mises test: H F T ( ) = F C ( ), equality of treatment and control CDFs How/did it affect the mean? t-test (or related CI), H E FT (Y ) = E FC (Y ); here not good since small sample with non-normal distributions Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 5 / 4

7 Example: effect of gift wage treatment on productivity Did the treatment affect productivity in any way? Permutation or Cramér von Mises test: H F T ( ) = F C ( ), equality of treatment and control CDFs How/did it affect the mean? t-test (or related CI), H E FT (Y ) = E FC (Y ); here not good since small sample with non-normal distributions How/did it affect the median, upper quartile, etc.? H FT (.5) = FC (.5), or CI for FT (.5) FC (.5). Covered in next section. Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 5 / 4

8 Example: effect of gift wage treatment on productivity Did the treatment affect productivity in any way? Permutation or Cramér von Mises test: H F T ( ) = F C ( ), equality of treatment and control CDFs How/did it affect the mean? t-test (or related CI), H E FT (Y ) = E FC (Y ); here not good since small sample with non-normal distributions How/did it affect the median, upper quartile, etc.? H FT (.5) = FC (.5), or CI for FT (.5) FC (.5). Covered in next section. Where in the distribution was the treatment effect statistically significant? Test lots of quantiles at 5% level: will reject one 5% of time (multiple testing problem) Kolmogorov Smirnov (KS) test: much lower power in tails than center of distribution Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 5 / 4

9 Example: effect of gift wage treatment on productivity Q: which part(s) of distribution show treatment effect? Approach: test at lots of quantiles but control familywise error rate, FWER = P (rej. 1 test H ) Desired property: same pointwise type I error rates across distribution Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 6 / 4

10 Distributional inference: Kolmogorov Smirnov, one-sample Empirical distribution function (EDF): ˆF n (x) = n i=1 1{X i x}/n Idea: reject if F n ( ) is different enough from null F ( ) Specifically, look at single-biggest vertical discrepancy: D n sup x ˆFn (x) F (x) (this occurs at some X i ) Limit: nd n d supt [,1] B(t) for Brownian bridge B( ), since n( ˆF n ( ) F ( )) B(F ( )) But: Var(B(t)) = t(1 t), max at t = 1/2, zero at t {, 1} (hence low tail power) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 7 / 4

11 Distributional inference: Kolmogorov Smirnov, one-sample Empirical distribution function (EDF): ˆF n (x) = n i=1 1{X i x}/n Idea: reject if F n ( ) is different enough from null F ( ) Specifically, look at single-biggest vertical discrepancy: D n sup x ˆFn (x) F (x) (this occurs at some X i ) Limit: nd n d supt [,1] B(t) for Brownian bridge B( ), since n( ˆF n ( ) F ( )) B(F ( )) But: Var(B(t)) = t(1 t), max at t = 1/2, zero at t {, 1} (hence low tail power) Anderson and Darling (1952, p. 23): 1/ t(1 t) weight to give each point of the distribution F (x) equal weights i.e., equal asy. pointwise variance and type I error. But: argument only valid over fixed quantile range overweights tails (and great computational difficulty) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 7 / 4

12 Pointwise one-sample type I error rates, α =.1, n = 2 Pointwise type I error, n=2 Rejection probability Dirichlet KS weighted KS Order statistic Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 8 / 4

13 Pointwise one-sample type I error rates, α =.1, n = 1 Pointwise type I error, n=1 Rejection probability Dirichlet KS weighted KS Order statistic Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 8 / 4

14 Dirichlet? Probability integral transform: R. A. Fisher (1932), Karl Pearson (1933), Neyman (1937) Y i iid F, continuous F (Yi ) d = U i iid Uniform(, 1) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 9 / 4

15 Dirichlet? Probability integral transform: R. A. Fisher (1932), Karl Pearson (1933), Neyman (1937) Y i iid F, continuous F (Yi ) d = U i iid Uniform(, 1) Y n k : kth order statistic (kth-smallest value in sample); F (Y n k ) d = U n k β(k, n + 1 k) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 9 / 4

16 Dirichlet? Probability integral transform: R. A. Fisher (1932), Karl Pearson (1933), Neyman (1937) Y i iid F, continuous F (Yi ) d = U i iid Uniform(, 1) Y n k : kth order statistic (kth-smallest value in sample); F (Y n k ) d = U n k β(k, n + 1 k) Wilks (1962): {F (Y n 1 ),..., F (Y n n )} Dir (1,..., 1; 1) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 9 / 4

17 Dirichlet? Probability integral transform: R. A. Fisher (1932), Karl Pearson (1933), Neyman (1937) Y i iid F, continuous F (Yi ) d = U i iid Uniform(, 1) Y n k : kth order statistic (kth-smallest value in sample); F (Y n k ) d = U n k β(k, n + 1 k) Wilks (1962): {F (Y n 1 ),..., F (Y n n )} Dir (1,..., 1; 1) Recently: Buja and Rolke (26) and Aldor-Noiman et al. (213), but slow computation; Eiger et al. (214), one-sample one-sided p-values Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 9 / 4

18 Dirichlet? N(,1), n=21 Probability Y Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 1 / 4

19 Dirichlet? N(,1), n=21, k=4, 9% CI Probability Y Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 1 / 4

20 Dirichlet? N(,1), n=21, k=11, 9% CI Probability Y Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 1 / 4

21 Dirichlet? N(,1), n=21, k=17, 9% CI Probability Y Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 1 / 4

22 Dirichlet-based inference Beta: 1 α CI for F ( ) at each Y n k Dirichlet: which α makes 1 α uniform confidence band Hypothesis testing, incl. 2-sample/FOSD n tests controlling FWER Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 11 / 4

23 Dirichlet-based inference N(,1), n=21 Probability Y Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 12 / 4

24 Dirichlet-based inference 9% uniform confidence band F(Y) Y Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 12 / 4

25 Dirichlet-based inference Properties: nonparametric/distribution-free; exact in finite samples; identifies quantile range(s) with treatment effect; even pointwise type I error Obstacle: computation. Specifically: mapping of α α Solution: Goldman and Kaplan (214c), Evenly sensitive KS-type inference on distributions One-sample: α(α, n) is a function of n with four parameters that are functions of α. After weeks of simulations: very precisely determined (see next graph) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 13 / 4

26 Fitted (lines) vs. simulated α(α, n) n = 2 to 1 6 exp(exp(2.6)); α =.1 (top) to.9 Universally fitted and simulated α ~ (α, n) ln(α ~ ) ln[ln(n)] Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 14 / 4

27 Dirichlet-based inference Two-sample: needs more simulations since two sample sizes Similar relationship, but have not (yet) found closed-form dependence on sample size ratio Lookup table for commonly desired α and large range of n no just-in-time simulation required for both one- and two-sample, computation is immediate Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 15 / 4

28 Pointwise two-sample type I error rates, α =.1, n = 4 Pointwise type I error, nx=ny=4, Fx=Fy=Unif(,1) Rejection probability Dirichlet KS X (or Y) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 16 / 4

29 Pointwise two-sample type I error rates, α =.1, n = 8 Pointwise type I error, nx=ny=8, Fx=Fy=Unif(,1) Rejection probability Dirichlet KS X (or Y) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 16 / 4

30 Example: effect of gift wage treatment on productivity Gift wage: library task, period 1 Books entered Treatment Control Quantile Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 17 / 4

31 Example: effect of gift wage treatment on productivity Gift wage: library task, period 1 Gift wage: fundraising task, period 1 Books entered Treatment Control Quantile Dollars raised Treatment Control Quantile Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 17 / 4

32 Example: U.S. city size distribution (e.g., Eeckhout, 29) U.S. city size CDF ln(population) Lognormal fit Dirichlet 95% band Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 18 / 4

33 Example: U.S. city size distribution (e.g., Eeckhout, 29) CDF U.S. city size ln(population) Data (EDF) Lognormal fit Dirichlet 95% band KS 95% band Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 18 / 4

34 Example: U.S. city size distribution (e.g., Eeckhout, 29) U.S. city size CDF ln(population) Data (EDF) Lognormal fit Dirichlet 95% band KS 95% band Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 18 / 4

35 1 Distributional inference 2 Quantile inference 3 Nonparametric Bayesian inference on distributions and quantiles 4 Conditional QTE inference 5 Conclusion Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 19 / 4

36 Quantile: now look horizontally N(,1), n=21, k=4, 9% CI Probability Y Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 2 / 4

37 Quantile: now look horizontally N(,1), n=21, k=11, 9% CI Probability Y Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 2 / 4

38 Quantile: probability integral transform Want: 1-sided, 1 α CI for median, F 1 Y (.5) = Q Y (.5), from iid Y i Form: (, Y n k ] where Y n k is kth order statistic (kth smallest value) Transform: α = P (Y n k < FY 1(.5)) = P (F Y (Y n k ) <.5), F Y (Y i ) iid Unif(, 1), F Y (Y n k ) β(k, n + 1 k) Property: exact in finite samples. Unless: k not integer. Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 21 / 4

39 Quantile: fractional order statistics Beran and Hall (1993): use binomial (equivalent to beta for integer k), interpolate based on probability; O(n 1 ) coverage probability error ( CPE ; i.e., how different from 1 α) Hutson (1999): linear interpolation using w = k k ; no theoretical results on CPE Goldman and Kaplan (214a, Fractional order statistic approximation for nonparametric conditional quantile inference ): Hutson CPE is O(n 1 ) See also Stigler (1977) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 22 / 4

40 Quantile treatment effect (QTE) Independent samples of iid X i and Y i Object: Q X (p) Q Y (p), the p-qte (Doksum, 1974; Lehmann, 1974); always defined, not always causal Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 23 / 4

41 Quantile treatment effect (QTE) Independent samples of iid X i and Y i Object: Q X (p) Q Y (p), the p-qte (Doksum, 1974; Lehmann, 1974); always defined, not always causal Why: robust to heavy tails Why: heterogeneity does treatment affect most productive workers, or lowest scoring students, or poorest recipients... ; is wage gap bigger for low-paid workers; etc. Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 23 / 4

42 Quantile treatment effect (QTE) Independent samples of iid X i and Y i Object: Q X (p) Q Y (p), the p-qte (Doksum, 1974; Lehmann, 1974); always defined, not always causal Why: robust to heavy tails Why: heterogeneity does treatment affect most productive workers, or lowest scoring students, or poorest recipients... ; is wage gap bigger for low-paid workers; etc. Technical difficulty: can t directly use probability integral transform, so need 1st-order Taylor approx first nuisance parameter γ (PDF ratio) account for estimation error and conditioning on ˆγ... (GK214b, Nonparametric inference on conditional quantile treatment effects using L-statistics ) So... : O(n 2/3 ) CPE (still quite good, and more robust to ˆγ error than other methods) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 23 / 4

43 Interquantile range treatment effect Goldman and Kaplan (214b) also includes treatment effects on interquantile ranges E.g., IQRTE = [Q X (.75) Q X (.25)] [Q Y (.75) Q Y (.25)] E.g., how does wage inequality differ from 1 years ago? (Buchinsky, 1994; Angrist, Chernozhukov, Fernández-Val, 26) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 24 / 4

44 1 Distributional inference 2 Quantile inference 3 Nonparametric Bayesian inference on distributions and quantiles 4 Conditional QTE inference 5 Conclusion Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 25 / 4

45 Nonparametric Bayesian inference: distributions Bayesian inference with (improper) uninformative Dirichlet process prior leads to posterior Dirichlet distribution over X i ; mean is EDF Posterior distribution of F (X n k ) is β(k, n k) very similar to β(k, n + 1 k) Joint posterior over all {F (X n k )} n k=1 is similar except in tails; notably, P (F (X n n ) = 1) = 1: sample max is population max Histospline smoothing of posterior exactly matches Wilks Dirichlet result our Dirichlet-based frequentist confidence bands are also valid Bayesian credible sets (without the usual computational difficulty) See Ferguson (1973, Dirichlet process prior), Rubin (1981, Bayesian bootstrap), Banks (1988, histospline smoothing), Chamberlain and Imbens (23, Bayesian bootstrap examples) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 26 / 4

46 Nonparametric Bayesian inference: quantiles Again, very similar due to Dirichlet posterior Unsmoothed: only difference is n vs. n + 1 and no interpolation; asymptotically equivalent Histospline smoothing (Banks, 1988): appears to be equivalent to Beran and Hall (1993) uniformly smoothing posterior probability between order statistics is like linearly interpolating based on probabilities QTE? Maybe no exact equivalence, but again must be very similar (and certainly asy. equivalent) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 27 / 4

47 1 Distributional inference 2 Quantile inference 3 Nonparametric Bayesian inference on distributions and quantiles 4 Conditional QTE inference 5 Conclusion Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 28 / 4

48 Nonparametric conditional inference Idea: embed unconditional methods in conditional framework w/ kernel smoothing/nearest neighbor for continuous elements of X Observe iid (X i, Y i ); interest in Q Y X (p X = x ) Take obs with X i near x ; apply unconditional method to corresponding Y i Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 29 / 4

49 Nonparametric conditional inference Data with IDEAL 95% Confidence Intervals Y X Data Pointwise Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 29 / 4

50 Nonparametric conditional inference Idea: embed unconditional methods in conditional framework w/ kernel smoothing/nearest neighbor for continuous elements of X Observe iid (X i, Y i ); interest in Q Y X (p X = x ) Take obs with X i near x ; apply unconditional method to corresponding Y i See also Yanqin Fan and Liu (working paper, first-order accuracy but weaker assumptions); Shu Shen (working paper, distributional partial effects) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 29 / 4

51 Nonparametric conditional inference Question: how near x? (i.e., bandwidth) Question: resulting CPE? Answers: Goldman and Kaplan (214a,b) CPE-optimal bandwidth: not just rate, but optimal value CPE better than normality/bootstrap in most (but not all) cases; e.g., always better if only one continuous covariate (plus however many discrete) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 3 / 4

52 Example: Engel curves (Banks, Blundell, & Lewbel, 1997) budget share Joint: Food Food.5 quantile.75 quantile.9 quantile log expenditure Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 31 / 4

53 Example: Engel curves (Banks, Blundell, & Lewbel, 1997) budget share Joint: Fuel, light, and power Fuel.5 quantile.75 quantile.9 quantile log expenditure Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 31 / 4

54 Example: Engel curves (Banks, Blundell, & Lewbel, 1997) budget share Joint: Clothing/footwear Clothing.5 quantile.75 quantile.9 quantile log expenditure Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 31 / 4

55 Example: Engel curves (Banks, Blundell, & Lewbel, 1997) budget share Joint: Alcohol Alcohol.5 quantile.75 quantile.9 quantile log expenditure Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 31 / 4

56 Example: Engel curves (Deaton and Paxson, 1998) Food, {2 adults, kids} minus {1 adult, kids}, UK LCFS 2ac pointwise minus 1ac, 9% Pointwise CIs 9% Budget Share Difference ln(pce).5 quantile.75 quantile.9 quantile Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 32 / 4

57 Example: Engel curves (Deaton and Paxson, 1998) Food, {2A, 2K} minus {1A, 1K}, UK LCFS 2a2c pointwise minus 1a1c, 9% Pointwise CIs 9% Budget Share Difference ln(pce).5 quantile.75 quantile.9 quantile Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 32 / 4

58 Example: Engel curves (Deaton and Paxson, 1998) Food, {3A, K} minus {2A, K}, UK LCFS 3ac pointwise minus 2ac, 9% Pointwise CIs 9% Budget Share Difference ln(pce).5 quantile.75 quantile.9 quantile Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 32 / 4

59 Example: Engel curves (Deaton and Paxson, 1998) Food (.9.5)-IQRTE, {2A, K} minus {1A, K}, UK LCFS 2ac minus 1ac, (.9.5) Interquantile pointwise 9% CIs Range, Pointwise 9% Interquantile Range Difference (.9 minus.5) quantile difference ln(pce) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 32 / 4

60 Example: Engel curves (Deaton and Paxson, 1998) Food, (.9.5)-IQRTE, {2A, 2K} minus {1A, 1K}, UK LCFS 2a2c minus 1a1c, (.9.5) Interquantile pointwise 9% CIs Range, Pointwise 9% Interquantile Range Difference (.9 minus.5) quantile difference ln(pce) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 32 / 4

61 Example: Engel curves (Deaton and Paxson, 1998) Food, (.9.5)-IQRTE, {3A, K} minus {2A, K}, UK LCFS 3ac minus 2ac, (.9.5) Interquantile pointwise 9% CIs Range, Pointwise 9% Interquantile Range Difference (.9 minus.5) quantile difference ln(pce) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 32 / 4

62 Simulations: conditional x(1 x)sin 2π( ) (x ) QY X(p;x) X Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 33 / 4

63 Joint Hypotheses Y X + - no shift Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 34 / 4

64 Joint Hypotheses Y X + - no shift shift = +.1 Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 34 / 4

65 Joint Hypotheses Y no shift shift = shift = X Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 34 / 4

66 Simulations: runtime Table: Computation time in seconds, including bandwidth selection. sample size, n Method #x L-stat Local cubic rqss n/a n/a n/a L-stat Local cubic rqss n/a n/a n/a L-stat Local cubic rqss n/a n/a n/a Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 35 / 4

67 Simulations: conditional quantile Joint Power Curves p =.5, χ 2 3 conditional distribution Rejection Probability (%) α = 5% L stat rqss local cubic Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 36 / 4

68 Simulations: conditional quantile Joint Power Curves p =.25, normal conditional distribution Rejection Probability (%) α = 5% L stat rqss local cubic Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 36 / 4

69 Simulations: conditional QTE Compare to local cubic w/ analytic or bootstrap SE, 1 α =.95: x value U i Coverage Probability L-stat N(, 1) Normal N(, 1) Bootstrap N(, 1) Median Interval Length L-stat N(, 1) Normal N(, 1) Bootstrap N(, 1) Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 37 / 4

70 Simulations: conditional QTE Compare to local cubic w/ analytic or bootstrap SE, 1 α =.95: x value U i Coverage Probability L-stat t Normal t Bootstrap t Median Interval Length L-stat t Normal t Bootstrap t Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 37 / 4

71 Simulations: conditional QTE Compare to local cubic w/ analytic or bootstrap SE, 1 α =.95: x value U i Coverage Probability L-stat Cauchy Normal Cauchy Bootstrap Cauchy Median Interval Length L-stat Cauchy Normal Cauchy Bootstrap Cauchy Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 37 / 4

72 1 Distributional inference 2 Quantile inference 3 Nonparametric Bayesian inference on distributions and quantiles 4 Conditional QTE inference 5 Conclusion Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 38 / 4

73 Recap Distributional [treatment effect] inference: identify range(s) of distribution w/ significant effect exact finite-sample coverage/size constant pointwise type I error rate α Quantile [treatment effect] inference: accurate, robust, fast effect on quantile of outcome distribution, or on IQR Nonparametric Bayesian distributional and quantile inference: very similar credible interval to frequentist CI smoothing equivalent (GK, Beran & Hall) Conditional QTE inference: allows for more heterogeneity of effect (but: need more data) still more accurate than normality Also: quantile marginal effects (Sasaki; Kaplan), e.g. slope of quantile Engel curve (elasticity) Code, examples, and papers on my website Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 39 / 4

74 Inference on distributional and quantile treatment effects David M. Kaplan University of Missouri Matt Goldman UC San Diego NIU, 214 Dave Kaplan (Missouri), Matt Goldman (UCSD) Distributional and QTE inference 4 / 4