Inference for Average Treatment Effects

Transcription

1 Inference for Average Treatment Effects Kosuke Imai Harvard University STAT186/GOV2002 CAUSAL INFERENCE Fall 2018 Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

2 Social Pressure and Turnout (Gerber, et al Am. Political Sci. Rev.) August 2006 Primary Election in Michigan Statewide elections: Governor, US Senator 180,000 households Send postcards with different messages Randomly assign each household to a group (or treatment) 1 no message (control group) 2 civic duty message 3 you are being studied message (Hawthorne effect) 4 household social pressure message 5 neighborhood social pressure message Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

3 Neighborhood Social Pressure Message Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

4 You are being studied Message Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

5 Standard Empirical Analysis Groups Control Civic duty Hawthorne Self Neighbor Turnout rate 29.7% 31.5% 32.2% 34.5% 37.5% # of voters 191,243 38,218 38,204 38,218 38,201 Neighborhood social pressure vs. Control ˆτ = = ( ) 29.7 ( ) s.e. = %CI = [ , ] = [7.2, 8.4] This calculation ignores the fact that some households have multiple voters: we will discuss this issue later in the course How can we justify this standard difference-in-means analysis from the randomization perspective? Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

6 Theoretical Motivation Permutation inference has a difficult time handling heterogeneity Population inference is also difficult Can we address these limitations while keeping the design-based approach? inference about average treatment effects We will begin with randomization inference for sample survey: close connection to Neyman s randomization inference Causal inference as a missing data problem Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

7 Estimation of the Sample Average Treatment Effect Due to Neyman (1923) Neyman (translated to English) Stat. Sci. Difference-in-means estimator: ˆτ 1 n T i Y i 1 n 1 n 0 n (1 T i )Y i Define O {Y i (0), Y i (1)} n Unbiasedness (over repeated treatment assignments): E(ˆτ O) = 1 n E(T i O)Y i (1) 1 n 1 n 0 = 1 n n {1 E(T i O)}Y i (0) n (Y i (1) Y i (0)) = SATE Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

8 Randomization Inference for SATE Variance of ˆτ: where for t = 0, 1, V(ˆτ O) = 1 n ( n0 S1 2 n + n ) 1 S0 2 1 n + 2S 01, 0 S 2 t = S 01 = 1 n 1 1 n 1 n (Y i (t) Y (t)) 2 sample variance of Y i (t) n (Y i (0) Y (0))(Y i (1) Y (1)) sample covariance The variance is NOT identifiable Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

9 Details of Variance Derivation 1 Let X i = Y i (1) + n 1 Y i (0)/n 0 and D i = nt i /n 1 1, and write ( V(ˆτ O) = 1 n ) 2 n 2 E D i X i O 2 Show 3 Use ➊ and ➋ to show, E(D i O) = 0, E(D 2 i O) = n 0 n 1, n 0 E(D i D j O) = n 1 (n 1) V(ˆτ O) = n 0 n(n 1)n 1 n (X i X) 2 4 Substitute the potential outcome expressions for X i Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

10 Conservative Variance Estimator The usual variance estimator is conservative on average: ( ) V(ˆτ O) S2 1 + S2 0 ˆσ 1 2 = E + ˆσ2 0 O n 1 n 0 n 1 n 0 where ˆσ t = 1 n t 1 n 1{T i = t}(y i Y t ) 2 for t = 0, 1 Under the constant additive unit causal effect assumption, i.e., Y i (1) Y i (0) = c for all i, S 01 = 1 2 (S2 1 + S2 0 ) and V(ˆτ O) = S2 1 + S2 0 n 1 n 0 The optimal treatment assignment rule: n opt 1 = n 1 + S 0 /S 1, n opt 0 = n 1 + S 1 /S 0 Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

11 Bounds on the Variance Use of the Cauchy-Schwartz inequality: 1 Upper bound: sample correlation between Y i (1) and Y i (0) is 1 2 Lower bound: sample correlation between Y i (1) and Y i (0) is 1 n 0 n 1 n ( S1 S ) 2 0 V(ˆτ O) n ( 0n 1 S1 + S ) 2 0 n 1 n 0 n n 1 n 0 Constant additive unit causal effect sample correlation is 1 n 0 n 1 n ( S1 + S ) 2 0 = S2 1 + S2 0 n 1 n 0 n 1 n 0 Sharp bounds based on the entire marginal distributions application of Hoeffding s lemma (Aronow et al Ann. Stat.) Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

12 Inference for Population Average Treatment Effect Assumption: simple random sampling from an infinite population Unbiasedness (over repeated sampling): Variance: E{E(ˆτ O)} = E(SATE) = PATE V(ˆτ) = V(E(ˆτ O)) + E(V(ˆτ O)) = σ2 1 n 1 + σ2 0 n 0 where σ 2 t is the population variance of Y i (t) for t = 0, 1 Unbiased variance estimator: for t = 0, 1 V(ˆτ) = ˆσ2 1 n 1 + ˆσ2 0 n 0 where E( V(ˆτ)) = V(ˆτ) Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

13 Asymptotic Inference for PATE Hold k = n 1 /n constant Rewrite the difference-in-means estimator as ˆτ = 1 n ( Ti Y i (1) (1 T ) i)y i (0) n k 1 k }{{} i.i.d. with mean PATE & variance nv(ˆτ) Consistency via Law of large numbers: ˆτ p PATE Asymptotic normality via the Central Limit Theorem: ( ) d n(ˆτ PATE) N 0, σ2 1 k + σ2 0 1 k (1 α) 100% Confidence intervals: [ˆτ s.e. z α/2, ˆτ + s.e. z α/2 ] Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

14 Heated Exchange at the Royal Statistiacal Society Neyman et al. (1935) Suppl. of J. Royal Stat. Soc Neyman: So long as the average yields of any treatments are identical, the question as to whether these treatments affect separate yields on single plots seems to be uninteresting Fisher: It may be foolish, but that is what the z test was designed for, and the only purpose for which it has been used. Neyman: I am considering problems which are important from the point of view of agriculture. Fisher: It may be that the question which Dr. Neyman thinks should be answered is more important than the one I have proposed and attempted to answer. I suggest that before criticizing previous work it is always wise to give enough study to the subject to understand its purpose. Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15

15 Summary: Fisher vs. Neyman Like Fisher, Neyman proposed randomization-based inference Unlike Fisher, 1 estimands are average treatment effects 2 heterogenous treatment effects are allowed 3 population as well as sample inference is possible 4 asymptotic approximation is required for inference Suggested reading: IMBENS AND RUBIN, CHAPTER 6 Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall / 15