A Framework for Eliciting, Incorporating and Disciplining Identification Beliefs in Linear Models

Size: px

Start display at page:

Download "A Framework for Eliciting, Incorporating and Disciplining Identification Beliefs in Linear Models"

Bernard Emil Davidson
6 years ago
Views:

1 A Framework for Eliciting, Incorporating and Disciplining Identification Beliefs in Linear Models Francis J. DiTraglia 1 Camilo García-Jimeno 2 1 University of Pennsylvania 2 University of Pennsylvania and NBER October 13th, 2016

2 Motivation Workhorse Linear Model Estimate causal effect of endogenous treatment using OLS, IV Since 2000 in AER Alone > 30 papers whose main results are a comparison of OLS and IV Three Key Challenges Measurement Error, Endogenous Regressor, Invalid Instrument Identifying causal effect requires imposing beliefs. 2/ 24

3 Two Kinds of Beliefs Formal Used in Estimation E.g. assume instrument satisfies exclusion restriction Informal Not Used in Estimation E.g. sign of regressor endogeneity, presence of measurement error Problems with Informal Beliefs Used for interpretation, e.g. reconcile OLS and IV, but: 1. Leaving money on the table 2. Contradictory Beliefs 3/ 24

4 Judgement Under Uncertainty: Heuristics and Biases (Kahneman & Tversky; Science, 1974)... an internally consistent set of subjective probabilities can be incompatible with other beliefs held by the individual... For judged probabilities to be considered adequate, or rational, internal consistency is not enough. The judgements must be compatible with the entire web of beliefs held by the individual. 4/ 24

5 This Project Identified set with endog. treatment, invalid instrument & measurement error: 1. Express using intelligible quantities 2. Show how the three problems interact Combine with all researcher beliefs using Bayesian tools: 1. Sensitivity analysis 2. Ensure beliefs are mutually consistent 3. Learn something that is implied by your beliefs, given data Two Cases: 1. Continuous treatment, classical measurement error 2. Binary treatment & instrument, non-differential meas. error 5/ 24

6 Continuous T, Classical Measurement Error y = βt + u T = πz + v T = T + w Classical Measurement Error: w (T, z, v, u) Endogenous Regressor: Cov(T, u) 0 Endogenous Instrument: Cov(z, u) 0 6/ 24

7 The Colonial Origins of Comparative Development Acemoglu, Johnson & Robinson (2001, AER) y = log(gdpc), T = institutions, z = log(settler mortality) OLS: 0.52 (0.06) IV: 0.94 (0.16) Researcher Beliefs Settler mortality is exogenous, i.e. Cov(z, u) = 0 Cov(T, u) > 0 & classical measurement error in institutions [The IV estimate is] in fact larger than the OLS estimates reported in Table 2. This suggests that measurement error in the institutions variables that creates attenuation bias is likely to be more important than reverse causality and omitted variables biases. 7/ 24

8 Parameters: Interpretable, Scale-Free, Bounded Support Reduced Form (Identified) Correlations between observables: ρ Ty, ρ Tz, ρ zy Structural (Un-identified) 1. Regressor Endogeneity: ρ T u 2. Instrument Endogeneity: ρ zu 3. Classical Measurement Error: κ κ Var(T ) Var(T ) (0, 1] 8/ 24

9 Identified Set for (ρ T u, ρ uz, κ) ( ) ρt ρ uz = uρ Tz 1 ρ 2 T (ρ Ty ρ Tz κρ zy ) u κ κ ( ) κ ρ 2 Ty ( ] ρ 2 Ty + ρ 2 Tz 2ρ Ty ρ Tz ρ zy κ, 1 1 ρ 2 zy ρ T u ( 1, 1) Any two of (ρ T u, ρ uz, κ) determine the third, given data. κ (κ, 1] upper bound for extent of measurement error. Data alone restrict neither treatment endogeneity ρ T u nor β. One-sided bound for ρ uz, but never rules out ρ uz = 0. 9/ 24

10 Colonial Origins: Identified Set at MLE for ρ T u > 0 β ρ uz ρt u κ / 24

11 Transparent Parameterization Reduced Form Parameters ϕ = (ρ Ty, ρ Tz, ρ zy ) Structural Parameters θ = (ρ T u, ρ zu, κ) Bayesian Inference Inference for ϕ is standard. Identified set Θ(ϕ) for θ updated by data only through ϕ. Beliefs enter by placing conditional prior on Θ given ϕ. 11/ 24

12 Bayesian Inference Step I: Reduced Form Draw ϕ (j) from posterior for ϕ = (ρ Ty, ρ Tz, ρ zy ). Step II: Structural Option A Frequentist Friendly 1. Place sign/interval restrictions R on Θ(ϕ (j) ) 2. Bounds for θ = (ρ T u, ρ uz, κ) and β given ϕ (j) 3. Inference for identified set Option B Fully Bayesian 1. Uniform prior on Θ(ϕ (j) ) R 2. Draw θ (j) given ϕ (j) and calculate implied β (j) 3. Inference for parameters 12/ 24

13 Colonial Origins: Inference Under ρ T u [0, 0.9] Under κ < 0.6 Empty identified set with approximately 30% probability! Under κ > 0.6 Frequentist-Friendly Fully Bayesian P(Valid) β β ρuz β [ 0.71, 0.25] [0.70, 1.00] [ 0.81, 0.15] [0.00, 0.93] 13/ 24

14 Implications for β When ρ T u > 0 and κ (0.6, 1] Red = OLS Estimate, Blue = IV Estimate β 14/ 24

15 Binary Treatment and Instrument y = c+βt + u Endogenous Treatment δ T = E[u T = 1] E[u T = 0] Invalid Instrument δ z = E[u z = 1] E[u z = 0] Non-differential Measurement Error Observe T such that α 0 = P(T = 1 T = 0) α 1 = P(T = 0 T = 1) T (z, y) T 15/ 24

16 Bringing Education to Afghan Girls Burde & Linden (2013, AEJ Applied) RCT: Build schools in treatment villages, compare girls test scores. y Girl s test score (standardized) T Girl s true school attendance (unobserved) T Parent s report of child s school attendance z School built in village (randomized) x Child and household characteristics 16/ 24

17 Afghan Girls RCT ITT OLS IV (0.06) (0.06) (0.12) z = 0 z = 1 Beliefs Positive selection p p Invalid Instrument? Spillovers (+) T = 1 T = 0 N = N = 49 N = N = 248 School Substitution ( ) / 24

18 Identified Set for (α 0, α 1, δ T, δ z ) ( ) p1 p 0 δ z = B(α 0, α 1) + δ T 1 α 0 α 1 α 0 [0, ᾱ 0) α 1 [0, ᾱ 1) δ T (, ) Linear relationship between δ T and δ z given α 0, α 1 (α 0 + α 1 ) < 1 = α 0 < min k {p k }, α 1 < min k {1 p k } Tighter bounds for (α 0, α 1 ) from Var(u T, z) > 0 Data alone place no restrictions on δ T, δ z, or β. 18/ 24

19 Afghan Girls RCT: α , α min k p k = 0.15, min k (1 p k ) = f 10 ᾱ 0 1 α1 f 00 f 11 ᾱ f 01 0 ᾱ 0 0 ᾱ α 0 19/ 24

20 Afghan Girls RCT: Contours of Identified Set at MLE δz α 0 =.05, α 1 =.06 α 0 = α 1 = 0 α 0 =.10, α 1 = δ T 20/ 24

21 Yet Another Transparent Parameterization Reduced Form Parameters ϕ = (E[y T, z], Var[y T, z], P[T, z]) Structural Parameters θ = (α 0, α 1, δ T, δ z ) Inference Same basic idea as in classical measurement error case. 21/ 24

22 Afghan Girls RCT: Posterior Inference Imposing δ T [0, 1], p = p Frequentist-Friendly δ z δ z β β [ 0.11, 0.04] [0.65, 0.82] [ 0.39, 0.08] [0.96, 1.26] δ z Fully Bayesian [0.10, 0.71] [ 0.15, 0.96] β 22/ 24

23 Afghan Girls RCT: Posterior for β Red = OLS Estimate, Blue = IV Estimate β 23/ 24

24 Summary Causal effect in linear model with measurement error, endogenous treatment & invalid instrument. Two cases: continuous treatment with classical measurement error; binary treatment & instrument under non-diff. measurement error. Incorporate, discipline and interrogate researcher beliefs. Implicit researcher beliefs can be very informative in practice. Extensions and Related Work R and STATA packages to implement our methods Heterogeneous treatment effects What does a valid instrument yield in the binary-binary setting? 24/ 24

PIER Working Paper

PIER Working Paper Penn Institute for Economic Research Department of Economics University of Pennsylvania 3718 Locust Walk Philadelphia, PA 19104-697 pier@econ.upenn.edu http://economics.sas.upenn.edu/pier PIER Working