arxiv: v1 [math.st] 28 Feb 2017
|
|
- Ferdinand Stevens
- 5 years ago
- Views:
Transcription
1 Bridging Finite and Super Population Causal Inference arxiv: v1 [math.st] 28 Feb 2017 Peng Ding, Xinran Li, and Luke W. Miratrix Abstract There are two general views in causal analysis of experimental data: the super population view that the units are an independent sample from some hypothetical infinite populations, and the finite population view that the potential outcomes of the experimental units are fixed and the randomness comes solely from the physical randomization of the treatment assignment. These two views differs conceptually and mathematically, resulting in different sampling variances of the usual difference-in-means estimator of the average causal effect. Practically, however, these two views result in identical variance estimators. By recalling a variance decomposition and exploiting a completeness-type argument, we establish a connection between these two views in completely randomized experiments. This alternative formulation could serve as a template for bridging finite and super population causal inference in other scenarios. Keywords: Completeness; Finite population correction; Potential outcome; Simple random sample; Variance of individual causal effects 1. Introduction Neyman (1923, 1935) defined causal effects in terms of potential outcomes, and proposed an inferential framework viewing all potential outcomes of a finite population as fixed and the treatment assignment as the only source of randomness. This finite population view allows for easy interpretation free of any hypothetical data generating process of the outcomes, and is used in a variety of contexts (e.g., Kempthorne 1952; Hinkelmann and Kempthorne 2008; Copas 1973; Rosenbaum Department of Statistics, University of California, Berkeley, California USA. pengdingpku@berkeley.edu Department of Statistics, Harvard University, Cambridge, Massachusetts USA. xinranli@fas.harvard.edu Graduate School of Education and Department of Statistics, Harvard University, Cambridge, Massachusetts USA. luke_miratrix@gse.harvard.edu 1
2 2002; Imai 2008; Freedman 2008a,b; Rosenbaum 2010; Aronow and Middleton 2013; Aronow et al. 2014; Abadie et al. 2014; Miratrix et al. 2013; Ding 2016; Lin 2013; Middleton and Aronow 2015; Imbens and Rubin 2015; Chiba 2015; Rigdon and Hudgens 2015; Li and Ding 2016). This approach is considered desirable because, in particular, it does not assume the data is somehow a representative sample of some larger (usually infinite) population. Alternative approaches, also using the potential outcomes framework, assume that the potential outcomes are independent and identical draws from a hypothetical infinite population. Mathematical derivations under this approach are generally simpler, but the approach itself can be criticized because of this typically untenable sampling assumption. Furthermore, this approach appears to ignore the treatment assignment mechanism. Thatbeingsaid, it iswellknownthatthefinalvarianceformulaefromeitherapproachtendtobe quite similar. For example, while the variance of the difference-in-means estimator for a treatmentcontrol experiment under an infinite population model is different from the one under Neyman (1923) s finite population formulation, this difference is easily represented as a function of the variance of the individual causal effects. Furthermore, this difference term is unidentifiable and is often assumed away under a constant causal effect model (Neyman 1923, 1935; Hodges and Lehmann 1964; Rubin 1990; Reichardt and Gollob 1999; Freedman et al. 2007), or by appeals to the final estimators being conservative. Strictly speaking, the infinite population variance estimate gives a conservative (overly large) estimate of the finite population variance for the difference in means. As deriving infinite population variance expressions, relative to finite population variance expressions, tends to be more mathematically straightforward, we might naturally wonder if we could use infinite population expressions as conservative forms of finite population expressions more generally. In this work we show that in fact we can assume an infinite population model as an assumption of convenience, and derive formula from this perspective. This shows that we can thus consider the resulting formula as focused on the treatment assignment mechanism and not on a hypothetical sampling mechanism, i.e., we show variance derivations under the infinite population framework can be used as conservative estimators in a finite context. Mathematically, this result comes from a variance decomposition and a completeness-style argument that characterizes the connection and the difference between these two views. The vari- 2
3 ance decomposition we use has previously appeared in Imai (2008), Imbens and Rubin (2015), and Balzer et al. (2016). The completeness-style argument, which we believe is novel in this domain, then sharpens the variance decomposition by moving from an expression on an overall average relationship to one that holds for any specific sample. Our overall goal is simple: we wish to demonstrate that if one uses variance formula derived from assuming an infinite population sampling model, then the resulting inference one obtains will be correct with regards to the analogous samples-specific treatment effects (although it could be potentially conservative in that the standard errors may be overly large) regardless of the existence of any sampling mechanism. 2. Super Population, Finite Population, and Samples Assume that random variables (Y(1),Y(0)) represent the pair of potential outcomes of an infinite super population, from which we take an independent and identically distributed (IID) finite population of size n: S = {(Y i (1),Y i (0)) : i = 1,...,n}. We first discuss completely randomized experiments, and comment on other experiments in Section 4. For a completely randomized experiment, we randomly assign n 1 units to receive treatment, leaving the remaining n 0 = n n 1 units to receive control. Let Z = (Z 1,...,Z n ) {0,1} n be the treatment assignment vector, which takes a particular value (z 1,...,z n ) {0,1} n with probability n 1!n 0!/n!, for any (z 1,...,z n ) {0,1} n with n z i = n 1. The observed outcome for unit i is then Y obs i = Z i Y i (1)+(1 Z i )Y i (0). At the super population level, the average potential outcomes are E{Y(1)} and E{Y(0)}, and the average causal effect is τ = E{Y(1) Y(0)}. The population variances of the potential outcomes and individual causal effect are V 1 = Var{Y(1)}, V 0 = Var{Y(0)}, V τ = Var{Y(1) Y(0)}. At the finite population level, i.e., for a fixed sample S, the average potential outcomes and 3
4 average causal effect are Ȳ 1 = 1 n Y i (1), Ȳ 0 = 1 n Y i (0), τ S = Ȳ1 Ȳ0. The corresponding finite population variances of the potential outcomes and individual causal effects are S 2 1 = 1 n 1 {Y i (1) Ȳ1} 2, S0 2 = 1 {Y i (0) Ȳ0} 2, Sτ 2 n 1 = 1 (τ i τ) 2, n 1 where we use the divisor n 1 following the tradition of survey sampling (Cochran 1977). All these quantities are fully dependent on S. In classical casual inference (Neyman 1923), the potential outcomes of these n experimental units, S, are treated as fixed numbers. Equivalently, we can consider such causal inference to be conducted conditional on S (e.g., Copas 1973; Rosenbaum 2002; Imbens and Rubin 2015; Rigdon and Hudgens 2015; Chiba 2015). Regardless, we have two parameters the population average treatment effect τ, and the sample average treatment effect τ S. After collecting the data, we would want to draw inference about τ or τ S. Our primary statistics are the averages of the observed outcomes and the difference-in-means estimator: Ȳ obs 1 = 1 n 1 Z i Yi obs, Ȳ0 obs = 1 n 0 (1 Z i )Y obs i, τ = Ȳ obs 1 Ȳobs 0. We also observe the sample variances of the outcomes under treatment and control using s 2 1 = 1 n 1 1 Z i (Y obs i Ȳobs 1 ) 2, s 2 0 = 1 n 0 1 (1 Z i )(Yi obs Ȳobs 0 ) 2. We do not have the sample analogue of S 2 τ or V τ because Y i (1) and Y i (0) are never jointly observed for any unit i in the sample. We summarize the infinite population, finite population and sample quantities in Table 1. 4
5 Table 1: Means and variances at the super population, finite population and sample levels. means variances treatment control effect treatment control effect super population E{Y(1)} E{Y(0)} τ = E{Y(1) Y(0)} V 1 V 0 V τ finite population Ȳ 1 Ȳ 0 τ S = Ȳ1 Ȳ0 S1 2 S0 2 Sτ 2 sample Ȳ1 obs Ȳ0 obs τ = Ȳ 1 obs 0 s 2 1 s Deriving Complete Randomization Results with an Independent Sampling Model The three levels of quantities in Table 1 are connected via independent sampling and complete randomization. Neyman (1923), without reference to any infinite population and by using the assignment mechanism as the only source of randomness, represented the assignment mechanism via an urn model, and found Var( τ S) = S2 1 n 1 + S2 0 n 0 S2 τ n. (1) He then observed that the final term was unidentifiable but nonnegative, and thus if we dropped it we would obtain an upper bound of the estimator s uncertainty. We next derive this result by assuming a hypothetical sampling mechanism from some assumed infinite super-population model of convenience. This alternative derivation of the above result, which can be extended to other assignment mechanisms, shows how we can interpret formulae based on super-population derivations as conservative formulae for finite-sample inference Sampling and randomization To begin, note that IID sampling of S from the super population implies three things: first, the finite population average causal effect τ S satisfies E(τ S ) = τ, second Var(τ S ) = Var [ 1 n ] {Y i (1) Y i (0)} = V τ n, (2) 5
6 and third, the sample variances are unbiased for the true variances: E(S 2 1) = V 1, E(S 2 0) = V 0, E(S 2 τ) = V τ. (3) Conditional on S, randomization of the treatment Z is the only source of randomness. In a completely randomized experiment, the outcomes in the treatment group form a simple random sample of size n 1 from {Y i (1) : i = 1,...,n}, and the outcomes in the control group form a simple random sample of size n 0 from {Y i (0) : i = 1,...,n}. Therefore, classical survey sampling theory (Cochran 1977) for the sample mean and variance gives E(Ȳ obs 1 S) = Ȳ1, E(Ȳ obs 0 S) = Ȳ0, E( τ S) = τ S, E(s 2 1 S) = S2 1, E(s2 0 S) = S2 0. (4) We do not use the notation E S ( ), because, depending on the contexts, such notation could either indicate expectation conditional on S or expectation averaged over S. We therefore use conditional expectations and conditional variances explicitly. If we do not condition on S, then the independence induced by the assignment mechanism means the outcomes under treatment are IID samples of Y(1) and the outcomes under control are IID samples of Y(0), and furthermore these samples are independent of each other. This independence makes it straightforward to show that τ is unbiased for τ with super population variance Var( τ) = Var(Ȳ obs 1 )+Var(Ȳ obs 0 ) = V 1 n 1 + V 0 n 0. (5) This is the classic infinite population variance formula for the two sample difference-in-means statistic. We could use it to obtain standard errors by plugging in s 2 1 and s2 0 for the two variances. This derivation, from (2) (5), is motivated by the sampling assumption: the assignment mechanism makes the two samples independent, but it is the IID assumption and classic sampling theory that gives this result (along with asymptotical normality which allows for associated testing and confidence interval generation). 6
7 3.2. Connecting the finite and infinite population inference with a variance decomposition We will now extend the above to indirectly derive the result on the variance of τ in finite population inference without explicitly enumerating the potential outcomes. The variance decomposition formula implies Var( τ) = E{Var( τ S)}+Var{E( τ S)} (6) = E{Var( τ S)}+Var(τ S ) = E{Var( τ S)}+ V τ n, (7) which further implies that the finite population variance of τ satisfies (using (3)) E{Var( τ S)} = Var( τ) V τ n = V 1 + V 0 V { } τ S 2 n 1 n 0 n = E 1 + S2 0 S2 τ. (8) n 1 n 0 n Compare to the classic variance expression (1), which is this without the expectation. Here we have that on average our classic variance expression holds. Now, because this is true for any infinite population, as it is purely a consequence of the IID sampling mechanism and complete randomization, we can close the gap between (1) and (8). Informally speaking, because (8) holds as an average over many hypothetical super populations, it should also hold for any finite population at hand, and indeed it does, as we next show using a completeness concept from statistics (Lehmann and Romano 2008) A Completeness argument First, define f(s) S2 1 + S2 0 S2 τ n 1 n 0 n Var( τ S), (9) a function of a fixed finite sample S, as the difference of the hypothesized finite sample variance formula and the actual finite sample variance. Formula (8) shows E{f(S)} = 0. Now we are going to show the stronger result f(s) = 0. 7
8 For any given sample S, we have fixed sample quantities: U = ( Y 1 (1) Ȳ1,...,Y n (1) Ȳ1), and W = ( Y 1 (0) Ȳ0,...,Y n (0) Ȳ0). Some algebra gives τ τ S = 1 Z i U i 1 (1 Z i )W i. (10) n 1 n 0 According to (10), the finite population variance Var(ˆτ S) = E{(ˆτ τ S ) 2 S} must be a quadratic form of (U,W). Therefore, from (9), f(s) must be a quadratic form of (U,W), and therefore must have the following expression: f(s) = a ij U i U j + b ij W i W j + c ij U i W j. j=1 j=1 j=1 for some a ij, b ij and c ij. f(s) is the same regardless of the ordering of units, so by symmetry the constants a ij, b ij and c ij must be the same regardless of i and j. This gives a ii = a 11,a ij = a 12,b ii = b 11,b ij = b 12,c ii = c 11,c ij = c 12, for all i j. Thus f(s) =a 11 Uj 2 +a 12 U i U j +b 11 i j Because n U i = 0, we have W 2 i +b 12 W i W j +c 11 U i W i +c 12 U i W j. i j Ui 2 + ( ) 2 U i U j = U i = 0. Similarly, as n W i = 0, we have i j i j W 2 i + W i W j = 0, i j U i W i + ( ) U i W j = W i = 0. i j U i)( Use the above to replace our cross terms of i j with single index terms to write f(s) as f(s) =a Ui 2 +b Wi 2 +c U i W i, where a = a 11 a 12,b = b 11 b 12, and c = c 11 c 12. Because E{f(S)} = 0, we have ae(u 2 1 ) + 8
9 ae(w 2 1 )+ce(u 1W 1 ) = 0, where E(U 2 1) = n 1 n Var{Y(1)}, E(W2 1) = n 1 n Var{Y(0)}, E(U 1W 1 ) 2 = n 1 n Cov{Y(1),Y(0)}. Thus, avar{y(1)}+bvar{y(0)}+ccov{y(1),y(0)} = 0. (11) Because(11)holdsforanypopulationsregardlessofitsvaluesofVar{Y(1)},Var{Y(0)}andCov{Y(1),Y(0)}, it must be true that a = b = c = 0, which implies f(s) = Discussion Equation (8) relies on the assumption that the hypothetical infinite population exists, but Equation (1) does not. However, the completeness-style argument allowed us to make our sampling assumption only for convenience in order to prove (1) by, in effect, dropping the expectation on both sides of (8). Similar argument exists in the classical statistics literature; see Efron and Morris (1973) for the empirical Bayes view of Stein s estimator. While the final result is, of course, not new, we offer it as it gives an alternative derivation that does not rely on asymptotics such as a growing superpopulation or a focus on the properties of the treatment assignment mechanism. Using Freedman (2008a) s results, Aronow et al. (2014) considered a super population with N units, with the finite population being a simple random sample of size n. Letting N, we can obtain similar results. We go in the other direction: we use the variance decomposition (6) to derive the finite population variance from the super population one. This decomposition approach also holds for other types of experiments. First, for a stratified experiment, each stratum is essentially a completely randomized experiment. Apply the result to each stratum, and then average over all strata to obtain results for a stratified experiment. Second, because a matched-pair experiment is a special case of a stratified experiment with two units within each stratum, we can derive the Neyman-type variance (cf. Imai 2008; Imbens and Rubin 2015) directly from that of a stratified experiment. Third, a cluster-randomized experiment is a completely randomized experiment on the clusters. If the causal parameters can be expressed as 9
10 cluster-level outcomes, then the result can be straightforwardly applied (cf. Aronow and Middleton 2013; Middleton and Aronow 2015). Fourth, for general experimental designs, the variance decomposition in (6) still holds, and therefore we can modify the derivation of the finite population variance according to different forms of (4) and (5). In a completely randomized experiment, the finite population sampling variance of τ in (1) depends on three terms: the first two can be unbiasedly estimated by s 2 1 /n 1 and s 2 0 /n 0, but the third term Sτ 2/n is unidentifiable from the data. Assuming a constant causal effect model, S2 τ = 0 and the variance estimators under both finite and super population inference coincide. However, Robins (1988), Aronow et al. (2014) and Ding and Dasgupta (2016) demonstrated that the treatment variation term Sτ 2 has a sharp lower bound that may be larger than 0, which allows for more precise variance estimators under the finite population view. This demonstrates that we can indeed make better inference conditional on the sample we have. On the other hand, in this work, we showed that assuming an infinite population, while not necessarily giving the tightest variance expressions, nonetheless gives valid (conservative) variance expressions from a finite-population perspective. We offer this approach as a possible method of proof that could ease derivations for more complex designs. More broadly, it is a step towards establishing that infinite population derivations for randomized experiments can be generally thought of as pertaining to their finite population analogs. Also see Samii and Aronow (2012) and Lin (2013), who provide alternative discussions of super population regression-based variance estimators under the finite population framework. Our discussion is based on the frequentists repeated sampling evaluations of the difference-inmeans estimator for the average causal effect. In contrast, Fisher (1935) proposed the randomization test against the sharp null hypothesis that Y i (1) = Y i (0) for all units, which is numerically the same as the permutation test for exchangeable units sampled from an infinite population (Lehmann and Romano 2008). This connection becomes more apparent when only the ranks of the outcomes are used to construct the test statistics, as discussed extensively by Lehmann (1975). Acknowledgments We thank the Associate Editor, Dr. Peter Aronow, and three anonymous reviewers for helpful comments. 10
11 References Abadie, A., Athey, S., Imbens, G. W., and Wooldridge, J. M. (2014). Finite population causal standard errors. Technical report, National Bureau of Economic Research. Aronow, P. M., Green, D. P., andlee, D. K.K.(2014). Sharpboundsonthevarianceinrandomized experiments. The Annals of Statistics, 42: Aronow, P. M. and Middleton, J. A. (2013). A class of unbiased estimators of the average treatment effect in randomized experiments. Journal of Causal Inference, 1: Balzer, L. B., Petersen, M. L., and Laan, M. J. (2016). Targeted estimation and inference for the sample average treatment effect in trials with and without pair-matching. Statistics in Medicine, 35: Chiba, Y. (2015). Exact tests for the weak causal null hypothesis on a binary out come in randomized trials. Journal of Biometrics and Biostatistics, 6:doi: / Cochran, W. G. (1977). Sampling Techniques. John Wiley & Sons, 3rd edition. Copas, J. (1973). Randomization models for the matched and unmatched 2 2 tables. Biometrika, 60: Ding, P. (2016). A paradox from randomization-based causal inference (with discussion). Statistical Science, in press. Ding, P. and Dasgupta, T.(2016). A potential tale of two by two tables from completely randomized experiments. Journal of American Statistical Association, 111: Efron, B. and Morris, C. (1973). Stein s estimation rule and its competitors an empirical bayes approach. Journal of the American Statistical Association, 68: Fisher, R. A. (1935). The Design of Experiments. Edinburgh, London: Oliver and Boyd, 1st edition. Freedman, D., Pisani, R., and Purves, R. (2007). Statistics. W. W. Norton & Company, 4th edition. Freedman, D. A. (2008a). On regression adjustments in experiments with several treatments. The Annals of Applied Statistics, 2:
12 Freedman, D. A. (2008b). Randomization does not justify logistic regression. Statistical Science, 23: Hinkelmann, K. and Kempthorne, O. (2008). Design and Analysis of Experiments, Volume 1: Introduction to Experimental Design. New Jersey: John Wiley & Sons, Inc., 2nd edition. Hodges, J. L. J. and Lehmann, E. L. (1964). Basic Concepts of Probability and Statistics. San Francisco: Holden-Day. Imai, K. (2008). Variance identification and efficiency analysis in randomized experiments under the matched-pair design. Statistics in Medicine, 27: Imbens, G. W. and Rubin, D. B. (2015). Causal Inference for Statistics, Social and Biometrical Sciences: An Introduction. Cambridge: Cambridge University Press. Kempthorne, O. (1952). The Design and Analysis of Experiments. New York: John Wiley and Sons. Lehmann, E. L. (1975). Nonparametrics: Statistical Methods Based on Ranks. San Francisco: Holden-Day, 1st edition. Lehmann, E. L. and Romano, J. P. (2008). Testing Statistical Hypotheses. New York : Wiley, 3rd edition. Li, X. and Ding, P. (2016). Exact confidence intervals for the average causal effect on a binary outcome. Statistics in Medicine, 35: Lin, W. (2013). Agnostic notes on regression adjustments to experimental data: Reexamining Freedman s critique. The Annals of Applied Statistics, 7: Middleton, J. A. and Aronow, P. M. (2015). Unbiased estimation of the average treatment effect in cluster-randomized experiments. Statistics, Politics and Policy, 6: Miratrix, L. W., Sekhon, J. S., and Yu, B. (2013). Adjusting treatment effect estimates by poststratification in randomized experiments. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 75:
13 Neyman, J. (1923). On the application of probability theory to agricultural experiments. essay on principles (with discussion). section 9 (translated). reprinted ed. Statistical Science, 5: Neyman, J. (1935). Statistical problems in agricultural experimentation (with discussion). Supplement to the Journal of the Royal Statistical Society, 2: Reichardt, C. S. and Gollob, H. F. (1999). Justifying the use and increasing the power of a t test for a randomized experiment with a convenience sample. Psychological Methods, 4: Rigdon, J. and Hudgens, M. G. (2015). Randomization inference for treatment effects on a binary outcome. Statistics in Medicine, 34: Robins, J. M. (1988). Confidence intervals for causal parameters. Statistics in Medicine, 7: Rosenbaum, P. R. (2002). Observational Studies. New York: Springer, 2nd edition. Rosenbaum, P. R. (2010). Design of Observational Studies. New York: Springer. Rubin, D. B. (1990). Comment: Neyman (1923) and causal inference in experiments and observational studies. Statistical Science, 5: Samii, C. and Aronow, P. M. (2012). On equivalencies between design-based and regression-based variance estimators for randomized experiments. Statistics and Probability Letters, 82:
arxiv: v1 [stat.me] 3 Feb 2017
On randomization-based causal inference for matched-pair factorial designs arxiv:702.00888v [stat.me] 3 Feb 207 Jiannan Lu and Alex Deng Analysis and Experimentation, Microsoft Corporation August 3, 208
arxiv: v1 [stat.me] 23 May 2017
Model-free causal inference of binary experimental data Peng Ding and Luke W. Miratrix arxiv:1705.08526v1 [stat.me] 23 May 27 Abstract For binary experimental data, we discuss randomization-based inferential
Understanding Ding s Apparent Paradox
Submitted to Statistical Science Understanding Ding s Apparent Paradox Peter M. Aronow and Molly R. Offer-Westort Yale University 1. INTRODUCTION We are grateful for the opportunity to comment on A Paradox
arxiv: v4 [math.st] 23 Jun 2016
A Paradox From Randomization-Based Causal Inference Peng Ding arxiv:1402.0142v4 [math.st] 23 Jun 2016 Abstract Under the potential outcomes framework, causal effects are defined as comparisons between
A randomization-based perspective of analysis of variance: a test statistic robust to treatment effect heterogeneity
Biometrika, pp 1 25 C 2012 Biometrika Trust Printed in Great Britain A randomization-based perspective of analysis of variance: a test statistic robust to treatment effect heterogeneity One-way analysis
DEALING WITH MULTIVARIATE OUTCOMES IN STUDIES FOR CAUSAL EFFECTS
DEALING WITH MULTIVARIATE OUTCOMES IN STUDIES FOR CAUSAL EFFECTS Donald B. Rubin Harvard University 1 Oxford Street, 7th Floor Cambridge, MA 02138 USA Tel: 617-495-5496; Fax: 617-496-8057 email: rubin@stat.harvard.edu
Inference for Average Treatment Effects
Inference for Average Treatment Effects Kosuke Imai Harvard University STAT186/GOV2002 CAUSAL INFERENCE Fall 2018 Kosuke Imai (Harvard) Average Treatment Effects Stat186/Gov2002 Fall 2018 1 / 15 Social
Research Note: A more powerful test statistic for reasoning about interference between units
Research Note: A more powerful test statistic for reasoning about interference between units Jake Bowers Mark Fredrickson Peter M. Aronow August 26, 2015 Abstract Bowers, Fredrickson and Panagopoulos (2012)
September 25, Abstract
On improving Neymanian analysis for 2 K factorial designs with binary outcomes arxiv:1803.04503v1 [stat.me] 12 Mar 2018 Jiannan Lu 1 1 Analysis and Experimentation, Microsoft Corporation September 25,
Conditional randomization tests of causal effects with interference between units
Conditional randomization tests of causal effects with interference between units Guillaume Basse 1, Avi Feller 2, and Panos Toulis 3 arxiv:1709.08036v3 [stat.me] 24 Sep 2018 1 UC Berkeley, Dept. of Statistics
Causal Mechanisms Short Course Part II:
Causal Mechanisms Short Course Part II: Analyzing Mechanisms with Experimental and Observational Data Teppei Yamamoto Massachusetts Institute of Technology March 24, 2012 Frontiers in the Analysis of Causal
Adjusting treatment effect estimates by post-stratification in randomized experiments
J. R. Statist. Soc. B (2013) 75, Part 2, pp. 369 396 Adjusting treatment effect estimates by post-stratification in randomized experiments Luke W. Miratrix Harvard University, Cambridge, USA and Jasjeet
Some challenges and results for causal and statistical inference with social network data
slide 1 Some challenges and results for causal and statistical inference with social network data Elizabeth L. Ogburn Department of Biostatistics, Johns Hopkins University May 10, 2013 Network data evince
University of California, Berkeley
University of California, Berkeley U.C. Berkeley Division of Biostatistics Working Paper Series Year 2015 Paper 334 Targeted Estimation and Inference for the Sample Average Treatment Effect Laura B. Balzer
Causal inference from 2 K factorial designs using potential outcomes
Causal inference from 2 K factorial designs using potential outcomes Tirthankar Dasgupta Department of Statistics, Harvard University, Cambridge, MA, USA 02138. atesh S. Pillai Department of Statistics,
Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities
Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities Peter M. Aronow and Cyrus Samii Forthcoming at Survey Methodology Abstract We consider conservative variance
Decomposing Treatment Effect Variation
Decomposing Treatment Effect Variation Peng Ding UC Berkeley Avi Feller UC Berkeley July 3, 27 Luke Miratrix Harvard GSE arxiv:65.6566v2 [math.st] 28 Jul 27 Abstract Understanding and characterizing treatment
arxiv: v1 [stat.me] 8 Jun 2016
Principal Score Methods: Assumptions and Extensions Avi Feller UC Berkeley Fabrizia Mealli Università di Firenze Luke Miratrix Harvard GSE arxiv:1606.02682v1 [stat.me] 8 Jun 2016 June 9, 2016 Abstract
Applied Statistics Lecture Notes
Applied Statistics Lecture Notes Kosuke Imai Department of Politics Princeton University February 2, 2008 Making statistical inferences means to learn about what you do not observe, which is called parameters,
Stratified Randomized Experiments
Stratified Randomized Experiments Kosuke Imai Harvard University STAT186/GOV2002 CAUSAL INFERENCE Fall 2018 Kosuke Imai (Harvard) Stratified Randomized Experiments Stat186/Gov2002 Fall 2018 1 / 13 Blocking
arxiv: v2 [math.st] 18 Oct 2017
Inference on a ew Class of Sample Average Treatment Effects Jasjeet S. Sekhon UC Berkeley Yotam Shem-Tov UC Berkeley arxiv:708.0240v2 [math.st] 8 Oct 207 October 9, 207 Abstract We derive new variance
Estimation of the Conditional Variance in Paired Experiments
Estimation of the Conditional Variance in Paired Experiments Alberto Abadie & Guido W. Imbens Harvard University and BER June 008 Abstract In paired randomized experiments units are grouped in pairs, often
Causal Inference from Experimental Data
30th Fisher Memorial Lecture 10 November 2011 hypothetical approach counterfactual approach data Decision problem I have a headache. Should I take aspirin? Two possible treatments: t: take 2 aspirin c:
An Introduction to Causal Analysis on Observational Data using Propensity Scores
An Introduction to Causal Analysis on Observational Data using Propensity Scores Margie Rosenberg*, PhD, FSA Brian Hartman**, PhD, ASA Shannon Lane* *University of Wisconsin Madison **University of Connecticut
The Econometrics of Randomized Experiments
The Econometrics of Randomized Experiments Susan Athey Guido W. Imbens Current version June 2016 Keywords: Regression Analyses, Random Assignment, Randomized Experiments, Potential Outcomes, Causality
The propensity score with continuous treatments
7 The propensity score with continuous treatments Keisuke Hirano and Guido W. Imbens 1 7.1 Introduction Much of the work on propensity score analysis has focused on the case in which the treatment is binary.
Causal Inference Lecture Notes: Causal Inference with Repeated Measures in Observational Studies
Causal Inference Lecture Notes: Causal Inference with Repeated Measures in Observational Studies Kosuke Imai Department of Politics Princeton University November 13, 2013 So far, we have essentially assumed
Shu Yang and Jae Kwang Kim. Harvard University and Iowa State University
Statistica Sinica 27 (2017), 000-000 doi:https://doi.org/10.5705/ss.202016.0155 DISCUSSION: DISSECTING MULTIPLE IMPUTATION FROM A MULTI-PHASE INFERENCE PERSPECTIVE: WHAT HAPPENS WHEN GOD S, IMPUTER S AND
An Introduction to Causal Mediation Analysis. Xu Qin University of Chicago Presented at the Central Iowa R User Group Meetup Aug 10, 2016
An Introduction to Causal Mediation Analysis Xu Qin University of Chicago Presented at the Central Iowa R User Group Meetup Aug 10, 2016 1 Causality In the applications of statistics, many central questions
Ignoring the matching variables in cohort studies - when is it valid, and why?
Ignoring the matching variables in cohort studies - when is it valid, and why? Arvid Sjölander Abstract In observational studies of the effect of an exposure on an outcome, the exposure-outcome association
Selection on Observables: Propensity Score Matching.
Selection on Observables: Propensity Score Matching. Department of Economics and Management Irene Brunetti ireneb@ec.unipi.it 24/10/2017 I. Brunetti Labour Economics in an European Perspective 24/10/2017
Is My Matched Dataset As-If Randomized, More, Or Less? Unifying the Design and. Analysis of Observational Studies
Is My Matched Dataset As-If Randomized, More, Or Less? Unifying the Design and arxiv:1804.08760v1 [stat.me] 23 Apr 2018 Analysis of Observational Studies Zach Branson Department of Statistics, Harvard
Flexible Estimation of Treatment Effect Parameters
Flexible Estimation of Treatment Effect Parameters Thomas MaCurdy a and Xiaohong Chen b and Han Hong c Introduction Many empirical studies of program evaluations are complicated by the presence of both
Statistical Analysis of Randomized Experiments with Nonignorable Missing Binary Outcomes
Statistical Analysis of Randomized Experiments with Nonignorable Missing Binary Outcomes Kosuke Imai Department of Politics Princeton University July 31 2007 Kosuke Imai (Princeton University) Nonignorable
CompSci Understanding Data: Theory and Applications
CompSci 590.6 Understanding Data: Theory and Applications Lecture 17 Causality in Statistics Instructor: Sudeepa Roy Email: sudeepa@cs.duke.edu Fall 2015 1 Today s Reading Rubin Journal of the American
Regression with a Single Regressor: Hypothesis Tests and Confidence Intervals
Regression with a Single Regressor: Hypothesis Tests and Confidence Intervals (SW Chapter 5) Outline. The standard error of ˆ. Hypothesis tests concerning β 3. Confidence intervals for β 4. Regression
Analysis of variance
Analysis of variance Andrew Gelman March 4, 2006 Abstract Analysis of variance (ANOVA) is a statistical procedure for summarizing a classical linear model a decomposition of sum of squares into a component
Statistical Models for Causal Analysis
Statistical Models for Causal Analysis Teppei Yamamoto Keio University Introduction to Causal Inference Spring 2016 Three Modes of Statistical Inference 1. Descriptive Inference: summarizing and exploring
Sensitivity analysis and distributional assumptions
Sensitivity analysis and distributional assumptions Tyler J. VanderWeele Department of Health Studies, University of Chicago 5841 South Maryland Avenue, MC 2007, Chicago, IL 60637, USA vanderweele@uchicago.edu
arxiv: v3 [stat.me] 20 Feb 2016
Posterior Predictive p-values with Fisher Randomization Tests in Noncompliance Settings: arxiv:1511.00521v3 [stat.me] 20 Feb 2016 Test Statistics vs Discrepancy Variables Laura Forastiere 1, Fabrizia Mealli
A noninformative Bayesian approach to domain estimation
A noninformative Bayesian approach to domain estimation Glen Meeden School of Statistics University of Minnesota Minneapolis, MN 55455 glen@stat.umn.edu August 2002 Revised July 2003 To appear in Journal
Hypothesis Testing. Part I. James J. Heckman University of Chicago. Econ 312 This draft, April 20, 2006
Hypothesis Testing Part I James J. Heckman University of Chicago Econ 312 This draft, April 20, 2006 1 1 A Brief Review of Hypothesis Testing and Its Uses values and pure significance tests (R.A. Fisher)
arxiv: v1 [stat.me] 16 Jun 2016
Causal Inference in Rebuilding and Extending the Recondite Bridge between Finite Population Sampling and Experimental arxiv:1606.05279v1 [stat.me] 16 Jun 2016 Design Rahul Mukerjee *, Tirthankar Dasgupta
Causal modelling in Medical Research
Causal modelling in Medical Research Debashis Ghosh Department of Biostatistics and Informatics, Colorado School of Public Health Biostatistics Workshop Series Goals for today Introduction to Potential
Optimal Blocking by Minimizing the Maximum Within-Block Distance
Optimal Blocking by Minimizing the Maximum Within-Block Distance Michael J. Higgins Jasjeet Sekhon Princeton University University of California at Berkeley November 14, 2013 For the Kansas State University
A MODEL-BASED EVALUATION OF SEVERAL WELL-KNOWN VARIANCE ESTIMATORS FOR THE COMBINED RATIO ESTIMATOR
Statistica Sinica 8(1998), 1165-1173 A MODEL-BASED EVALUATION OF SEVERAL WELL-KNOWN VARIANCE ESTIMATORS FOR THE COMBINED RATIO ESTIMATOR Phillip S. Kott National Agricultural Statistics Service Abstract:
Package RI2by2. October 21, 2016
Package RI2by2 October 21, 2016 Type Package Title Randomization Inference for Treatment Effects on a Binary Outcome Version 1.3 Author Joseph Rigdon, Wen Wei Loh, Michael G. Hudgens Maintainer Wen Wei
MA 575 Linear Models: Cedric E. Ginestet, Boston University Non-parametric Inference, Polynomial Regression Week 9, Lecture 2
MA 575 Linear Models: Cedric E. Ginestet, Boston University Non-parametric Inference, Polynomial Regression Week 9, Lecture 2 1 Bootstrapped Bias and CIs Given a multiple regression model with mean and
Matching on the Estimated Propensity Score
atching on the Estimated Propensity Score Alberto Abadie Harvard University and BER Guido W. Imbens Harvard University and BER October 29 Abstract Propensity score matching estimators Rosenbaum and Rubin,
Characterizing Forecast Uncertainty Prediction Intervals. The estimated AR (and VAR) models generate point forecasts of y t+s, y ˆ
Characterizing Forecast Uncertainty Prediction Intervals The estimated AR (and VAR) models generate point forecasts of y t+s, y ˆ t + s, t. Under our assumptions the point forecasts are asymtotically unbiased
Carl N. Morris. University of Texas
EMPIRICAL BAYES: A FREQUENCY-BAYES COMPROMISE Carl N. Morris University of Texas Empirical Bayes research has expanded significantly since the ground-breaking paper (1956) of Herbert Robbins, and its province
Why experimenters should not randomize, and what they should do instead
Why experimenters should not randomize, and what they should do instead Maximilian Kasy Department of Economics, Harvard University Maximilian Kasy (Harvard) Experimental design 1 / 42 project STAR Introduction
Beyond the Sharp Null: Permutation Tests Actually Test Heterogeneous Effects
Beyond the Sharp Null: Permutation Tests Actually Test Heterogeneous Effects Devin Caughey Allan Dafoe Luke Miratrix July 20, 2016 Abstract Permutation and randomization tests, despite their appealing
What s New in Econometrics. Lecture 1
What s New in Econometrics Lecture 1 Estimation of Average Treatment Effects Under Unconfoundedness Guido Imbens NBER Summer Institute, 2007 Outline 1. Introduction 2. Potential Outcomes 3. Estimands and
Combining multiple observational data sources to estimate causal eects
Department of Statistics, North Carolina State University Combining multiple observational data sources to estimate causal eects Shu Yang* syang24@ncsuedu Joint work with Peng Ding UC Berkeley May 23,
Potential Outcomes and Causal Inference I
Potential Outcomes and Causal Inference I Jonathan Wand Polisci 350C Stanford University May 3, 2006 Example A: Get-out-the-Vote (GOTV) Question: Is it possible to increase the likelihood of an individuals
Section 3 : Permutation Inference
Section 3 : Permutation Inference Fall 2014 1/39 Introduction Throughout this slides we will focus only on randomized experiments, i.e the treatment is assigned at random We will follow the notation of
arxiv: v1 [stat.me] 15 May 2011
Working Paper Propensity Score Analysis with Matching Weights Liang Li, Ph.D. arxiv:1105.2917v1 [stat.me] 15 May 2011 Associate Staff of Biostatistics Department of Quantitative Health Sciences, Cleveland
Testing Statistical Hypotheses
E.L. Lehmann Joseph P. Romano Testing Statistical Hypotheses Third Edition 4y Springer Preface vii I Small-Sample Theory 1 1 The General Decision Problem 3 1.1 Statistical Inference and Statistical Decisions
Use of Matching Methods for Causal Inference in Experimental and Observational Studies. This Talk Draws on the Following Papers:
Use of Matching Methods for Causal Inference in Experimental and Observational Studies Kosuke Imai Department of Politics Princeton University April 27, 2007 Kosuke Imai (Princeton University) Matching
Randomization Inference for Treatment Effect Variation
Randomization Inference for Treatment Effect Variation Peng Ding, Avi Feller, and Luke Miratrix Harvard University, Cambridge, MA, USA. Summary. Applied researchers are increasingly interested in whether
arxiv: v1 [math.st] 7 Jan 2014
Three Occurrences of the Hyperbolic-Secant Distribution Peng Ding Department of Statistics, Harvard University, One Oxford Street, Cambridge 02138 MA Email: pengding@fas.harvard.edu arxiv:1401.1267v1 [math.st]
Clustering as a Design Problem
Clustering as a Design Problem Alberto Abadie, Susan Athey, Guido Imbens, & Jeffrey Wooldridge Harvard-MIT Econometrics Seminar Cambridge, February 4, 2016 Adjusting standard errors for clustering is common
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Panel Data?
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Panel Data? Kosuke Imai Department of Politics Center for Statistics and Machine Learning Princeton University Joint
Gov 2002: 4. Observational Studies and Confounding
Gov 2002: 4. Observational Studies and Confounding Matthew Blackwell September 10, 2015 Where are we? Where are we going? Last two weeks: randomized experiments. From here on: observational studies. What
g-priors for Linear Regression
Stat60: Bayesian Modeling and Inference Lecture Date: March 15, 010 g-priors for Linear Regression Lecturer: Michael I. Jordan Scribe: Andrew H. Chan 1 Linear regression and g-priors In the last lecture,
Revision list for Pearl s THE FOUNDATIONS OF CAUSAL INFERENCE
Revision list for Pearl s THE FOUNDATIONS OF CAUSAL INFERENCE insert p. 90: in graphical terms or plain causal language. The mediation problem of Section 6 illustrates how such symbiosis clarifies the
Identification Analysis for Randomized Experiments with Noncompliance and Truncation-by-Death
Identification Analysis for Randomized Experiments with Noncompliance and Truncation-by-Death Kosuke Imai First Draft: January 19, 2007 This Draft: August 24, 2007 Abstract Zhang and Rubin 2003) derives
Bayesian Inference for Sequential Treatments under Latent Sequential Ignorability. June 19, 2017
Bayesian Inference for Sequential Treatments under Latent Sequential Ignorability Alessandra Mattei, Federico Ricciardi and Fabrizia Mealli Department of Statistics, Computer Science, Applications, University
Discussion of Identifiability and Estimation of Causal Effects in Randomized. Trials with Noncompliance and Completely Non-ignorable Missing Data
Biometrics 000, 000 000 DOI: 000 000 0000 Discussion of Identifiability and Estimation of Causal Effects in Randomized Trials with Noncompliance and Completely Non-ignorable Missing Data Dylan S. Small
NONPARAMETRICS. Statistical Methods Based on Ranks E. L. LEHMANN HOLDEN-DAY, INC. McGRAW-HILL INTERNATIONAL BOOK COMPANY
NONPARAMETRICS Statistical Methods Based on Ranks E. L. LEHMANN University of California, Berkeley With the special assistance of H. J. M. D'ABRERA University of California, Berkeley HOLDEN-DAY, INC. San
The Essential Role of Pair Matching in. Cluster-Randomized Experiments. with Application to the Mexican Universal Health Insurance Evaluation
The Essential Role of Pair Matching in Cluster-Randomized Experiments, with Application to the Mexican Universal Health Insurance Evaluation Kosuke Imai Princeton University Gary King Clayton Nall Harvard
Prerequisite: STATS 7 or STATS 8 or AP90 or (STATS 120A and STATS 120B and STATS 120C). AP90 with a minimum score of 3
University of California, Irvine 2017-2018 1 Statistics (STATS) Courses STATS 5. Seminar in Data Science. 1 Unit. An introduction to the field of Data Science; intended for entering freshman and transfers.
Partially Identified Treatment Effects for Generalizability
Partially Identified Treatment Effects for Generalizability Wendy Chan Human Development and Quantitative Methods Division, Graduate School of Education University of Pennsylvania, Philadelphia, Pennsylvania
Asymptotic Normality under Two-Phase Sampling Designs
Asymptotic Normality under Two-Phase Sampling Designs Jiahua Chen and J. N. K. Rao University of Waterloo and University of Carleton Abstract Large sample properties of statistical inferences in the context
Bayesian inference for sample surveys. Roderick Little Module 2: Bayesian models for simple random samples
Bayesian inference for sample surveys Roderick Little Module : Bayesian models for simple random samples Superpopulation Modeling: Estimating parameters Various principles: least squares, method of moments,
A Theory of Statistical Inference for Matching Methods in Applied Causal Research
A Theory of Statistical Inference for Matching Methods in Applied Causal Research Stefano M. Iacus Gary King Giuseppe Porro April 16, 2015 Abstract Matching methods for causal inference have become a popular
Identi cation of Positive Treatment E ects in. Randomized Experiments with Non-Compliance
Identi cation of Positive Treatment E ects in Randomized Experiments with Non-Compliance Aleksey Tetenov y February 18, 2012 Abstract I derive sharp nonparametric lower bounds on some parameters of the
A comparison study of the nonparametric tests based on the empirical distributions
통계연구 (2015), 제 20 권제 3 호, 1-12 A comparison study of the nonparametric tests based on the empirical distributions Hyo-Il Park 1) Abstract In this study, we propose a nonparametric test based on the empirical
A Theory of Statistical Inference for Matching Methods in Causal Research
A Theory of Statistical Inference for Matching Methods in Causal Research Stefano M. Iacus Gary King Giuseppe Porro October 4, 2017 Abstract Researchers who generate data often optimize efficiency and
Econometrics I KS. Module 2: Multivariate Linear Regression. Alexander Ahammer. This version: April 16, 2018
Econometrics I KS Module 2: Multivariate Linear Regression Alexander Ahammer Department of Economics Johannes Kepler University of Linz This version: April 16, 2018 Alexander Ahammer (JKU) Module 2: Multivariate
Imbens/Wooldridge, IRP Lecture Notes 2, August 08 1
Imbens/Wooldridge, IRP Lecture Notes 2, August 08 IRP Lectures Madison, WI, August 2008 Lecture 2, Monday, Aug 4th, 0.00-.00am Estimation of Average Treatment Effects Under Unconfoundedness, Part II. Introduction
Introduction to Statistical Analysis
Introduction to Statistical Analysis Changyu Shen Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology Beth Israel Deaconess Medical Center Harvard Medical School Objectives Descriptive
NBER WORKING PAPER SERIES FINITE POPULATION CAUSAL STANDARD ERRORS. Alberto Abadie Susan Athey Guido W. Imbens Jeffrey M.
NBER WORKING PAPER SERIES FINITE POPULATION CAUSAL STANDARD ERRORS Alberto Abadie Susan Athey Guido W. Imbens Jeffrey. Wooldridge Working Paper 20325 http://www.nber.org/papers/w20325 NATIONAL BUREAU OF
arxiv: v4 [math.st] 20 Jun 2018
Submitted to the Annals of Applied Statistics ESTIMATING AVERAGE CAUSAL EFFECTS UNDER GENERAL INTERFERENCE, WITH APPLICATION TO A SOCIAL NETWORK EXPERIMENT arxiv:1305.6156v4 [math.st] 20 Jun 2018 By Peter
STATS 200: Introduction to Statistical Inference. Lecture 29: Course review
STATS 200: Introduction to Statistical Inference Lecture 29: Course review Course review We started in Lecture 1 with a fundamental assumption: Data is a realization of a random process. The goal throughout
Test Code: STA/STB (Short Answer Type) 2013 Junior Research Fellowship for Research Course in Statistics
Test Code: STA/STB (Short Answer Type) 2013 Junior Research Fellowship for Research Course in Statistics The candidates for the research course in Statistics will have to take two shortanswer type tests
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data?
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data? Kosuke Imai Department of Politics Center for Statistics and Machine Learning Princeton University
Review of probability and statistics 1 / 31
Review of probability and statistics 1 / 31 2 / 31 Why? This chapter follows Stock and Watson (all graphs are from Stock and Watson). You may as well refer to the appendix in Wooldridge or any other introduction
Data Mining Chapter 4: Data Analysis and Uncertainty Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University
Data Mining Chapter 4: Data Analysis and Uncertainty Fall 2011 Ming Li Department of Computer Science and Technology Nanjing University Why uncertainty? Why should data mining care about uncertainty? We
INFERENCE APPROACHES FOR INSTRUMENTAL VARIABLE QUANTILE REGRESSION. 1. Introduction
INFERENCE APPROACHES FOR INSTRUMENTAL VARIABLE QUANTILE REGRESSION VICTOR CHERNOZHUKOV CHRISTIAN HANSEN MICHAEL JANSSON Abstract. We consider asymptotic and finite-sample confidence bounds in instrumental
Simple Linear Regression
Simple Linear Regression Kosuke Imai Harvard University STAT186/GOV2002 CAUSAL INFERENCE Fall 2018 Kosuke Imai (Harvard) Simple Linear Regression Stat186/Gov2002 Fall 2018 1 / 18 Linear Regression and
Estimators for the binomial distribution that dominate the MLE in terms of Kullback Leibler risk
Ann Inst Stat Math (0) 64:359 37 DOI 0.007/s0463-00-036-3 Estimators for the binomial distribution that dominate the MLE in terms of Kullback Leibler risk Paul Vos Qiang Wu Received: 3 June 009 / Revised:
Supplemental Appendix to "Alternative Assumptions to Identify LATE in Fuzzy Regression Discontinuity Designs"
Supplemental Appendix to "Alternative Assumptions to Identify LATE in Fuzzy Regression Discontinuity Designs" Yingying Dong University of California Irvine February 2018 Abstract This document provides
University of Michigan School of Public Health
University of Michigan School of Public Health The University of Michigan Department of Biostatistics Working Paper Series Year 003 Paper Weighting Adustments for Unit Nonresponse with Multiple Outcome
Control Function Instrumental Variable Estimation of Nonlinear Causal Effect Models
Journal of Machine Learning Research 17 (2016) 1-35 Submitted 9/14; Revised 2/16; Published 2/16 Control Function Instrumental Variable Estimation of Nonlinear Causal Effect Models Zijian Guo Department
R-squared for Bayesian regression models
R-squared for Bayesian regression models Andrew Gelman Ben Goodrich Jonah Gabry Imad Ali 8 Nov 2017 Abstract The usual definition of R 2 (variance of the predicted values divided by the variance of the
Bootstrapping Sensitivity Analysis
Bootstrapping Sensitivity Analysis Qingyuan Zhao Department of Statistics, The Wharton School University of Pennsylvania May 23, 2018 @ ACIC Based on: Qingyuan Zhao, Dylan S. Small, and Bhaswar B. Bhattacharya.
Gov 2002: 3. Randomization Inference
Gov 2002: 3. Randomization Inference Matthew Blackwell September 10, 2015 Where are we? Where are we going? Last week: This week: What can we identify using randomization? Estimators were justified via
AFT Models and Empirical Likelihood
AFT Models and Empirical Likelihood Mai Zhou Department of Statistics, University of Kentucky Collaborators: Gang Li (UCLA); A. Bathke; M. Kim (Kentucky) Accelerated Failure Time (AFT) models: Y = log(t