Combining Experimental and Non-Experimental Design in Causal Inference
|
|
- Augustus Walton
- 5 years ago
- Views:
Transcription
1 Combining Experimental and Non-Experimental Design in Causal Inference Kari Lock Morgan Department of Statistics Penn State University Rao Prize Conference May 12 th, 2017
2 A Tribute to Don Design trumps analysis Motivated by a real study Experimental design & rerandomization Observational study & propensity scores Rubin causal model & potential outcomes Educational testing (AP scores) (Missing data) (Noncompliance)
3 Design trumps Analysis For Objective Causal Inference, Design trumps Analysis Rubin 2008 X = covariates, W = treatment, Y = outcome(s) Design W X Analysis Y W, X Balance covariates As much as possible should be done without observed outcomes!
4 Knowledge in Action Goal: estimate causal effect of Knowledge in Action (KIA) (a form of project-based learning) in AP classes on AP scores and other outcomes Part 1 ( Efficacy Study ): randomize schools to KIA or control; compare outcomes after 1 year Part 2 ( Maturation Study ): continue to follow schools another year (experimental & observational)
5 *In this talk I ll just focus on one district Districts (blocks) District 1 District 5 Schools (clusters) RANDOMIZATION Teachers Students OUTCOMES
6 Covariates Covariates available at randomization: School covariates (e.g. Title 1 status, type, etc.) Teacher covariates (e.g. years of experience) Previous student (class) covariates: Race/ethnicity Poverty status Parental education PSAT scores x 1 8 th grade standardized test scores Total number of students Number of students who took the AP exam If covariates are available, we should use them when we randomize! x 2 2 covariates used for randomization
7 Rerandomization Collect covariate data Specify criteria for acceptable balance (Re)randomize Randomize units units to to treatment groups Check balance x $,& x $,( < 0.05 and x -,& x -,( < 0.05 unacceptable acceptable Conduct experiment Analyze results
8 Covariate Balance: Empirical x 1 x 2 Pure Randomization Rerandomization PRIV = 98.4% PRIV = 97.4% x T x C x T x C Percent reduction in variance: PRIV = var x 6,& x 6,( var x 6,& x 6,( rerand. var x 6,& x 6,(
9 Covariate Balance: Theoretical Suppose x 6,& x 6,( ~ Normal for j 1 k x $ x - x A Rerandomize if x 6,& x 6,( a 6 for j 1 k Then the PRIV for x 6 is p ap = p ai = p DE = 1 2 G H I, G P I, J E I IKJL(N E ) P SQ T P R SQU JI E IKJL(N E ) P SQ T P R SQU, _ where γ(b, c) y \]$ e ]^dy. `
10 Outcome PRIV If rerandomization is equal percent variance reducing (EPVR), then PRIV for the outcome difference in means is PRIV g = R - PRIV i Here, R and PRIV i 98%, so PRIV g = 74% Precision increases by a factor of $ $]`.lm = 3.85 Equivalent to almost quadrupling n!!! (Effective sample size goes from 76 to 293!)
11 Correlational Structure x 2T x 2C x 2T x 2C x 1T x 1C x 1T x 1C
12 Affine Invariance Affine invariance: rerandomization stays the same for any affine transformation a + bx If rerandomization criterion is affinely invariant and x is ellipsoidally symmetric 1. Ε Xp & Xp _ rerand. = Ε Xp & Xp _ = 0 => Rerandomization leads to unbiased estimates for any linear function of x 2. cov Xp & Xp _ rerand. cov Xp & Xp _ Preserves the correlations of Xp & Xp _ Balance improvement equal for each x 6 (equal percent variance reducing) (Morgan and Rubin, Annals of Statistics, 2012)
13 Mahalanobis Mahalanobis: Xp & Xp _ cov x ]$ Xp & Xp _ x 1 x 2 Pure Randomization Rerandomization PRIV = 97.4% PRIV = 97.4% x T x C x T x C
14 Knowledge in Action Part 1 ( Efficacy Study ): randomize schools to KIA or control; compare outcomes after 1 year Part 2 ( Maturation Study ): continue to follow schools another year (experimental & observational)
15 Covariate data for schools not in RCT MATCHING Matched Sample: 2 years of KIA no KIA Covariate data for schools in RCT RANDOMIZE WAVE 1: WAVE 2: KIA KIA: 2 nd year KIA: 1 st year 2 years of KIA no KIA? 1 year of KIA no KIA 2 years of KIA 1 year of KIA
16 2 years of KIA no KIA? 2 years of KIA no KIA Non-experimental direct approach Matched Sample: WAVE 1: KIA KIA: 2 nd year WHICH IS BETTER??? WAVE 2: KIA: 1 st year 1 year of KIA no KIA 2 years of KIA 1 year of KIA Experimental indirect approach
17 Potential Outcomes & Estimands Y~ 6 (W 6, t)= potential outcome for school j under treatment W 6 in year t Causal effect: compare potential outcomes under different treatments τ $, Y 1, t Y 0, t = 6 $ Y~ 6 1,t n 6 $ Y~ 6 0,t n τ -]$, Y 2, t Y 1, t = 6 $ Y~ 6 2, t n 6 $ Y~ 6 1, t n τ -, Y 2, t Y 0, t = 6 $ Y~ 6 2, t n 6 $ Y~ 6 1, t *Note: difference in means presented for clarity; actual analysis to use HLM n
18 Estimators τ $,-`$l Q EˆP E g~ E ($,-`$l) Q EˆP E τ -]$,-`$ 6 $ I E -Y~ 6 (2,2018) 6 $ I E - Q EˆP ($] E )g~ E (`,-`$l) Q EˆP($] E ) 6 $ I E $Y~ 6 (1,2018) 6 $ I E $ τ -,-`$ 6 $ I E -Y~ 6 (2,2018) 6 $ I E - 6 $ I E $Y~ 6 (0,2018) 6 $ I E $
19 2 years of KIA no KIA? 2 years of KIA no KIA Non-experimental direct approach Matched Sample: WAVE 1: KIA KIA: 2 nd year WHICH IS BETTER??? WAVE 2: KIA: 1 st year 1 year of KIA no KIA 2 years of KIA 1 year of KIA Experimental indirect approach
20 Propensity Score Matching 1 if in Wave 1 of experiment W 6 = Š 0 if not in experiment Propensity score: e 6 = P W 6 = 1 x 6 ) Match each Wave 1 teacher with a control with a similar propensity score Criteria for success: Quality of observed covariate data can only balance observed data Good matches available adequate overlap between groups large enough pool of potential controls
21 Propensity Score Matching If we have good matches, we can balance observed covariates Key point: unless we have data on all relevant covariates (which we won t), there will still be bias (baseline differences) Usually hard to quantify this bias BUT we have a very rare feature!!
22 1 year of KIA no KIA 2 years of KIA no KIA Matched Sample: WAVE 1: WAVE 2: KIA KIA: 2 nd year KIA: 1 st year We can validate the nonexperimental approach by comparing 1 year impact estimates! 1 year of KIA no KIA
23 2 years of KIA no KIA? 1 year of KIA no KIA 2 years of KIA no KIA Non-experimental direct approach Matched Sample: WAVE 1: KIA KIA: 2 nd year WHICH IS BETTER??? WAVE 2: KIA: 1 st year 1 year of KIA no KIA 2 years of KIA 1 year of KIA Experimental indirect approach
24 Experimental Indirect Approach τ -]$,-`$ + τ $,-`$l = Y 2,2018 Y 1, Y 1,2017 Y 0,2017 Critical assumption: potential outcomes may depend on year, but treatment effects do not That is, Y 1,2017 Y 1,2018, but τ $,-`$l = τ $,-`$ τ $ This implies τ $ + τ -]$ = τ -
25 Unbiased Define τ - τ $+ τ -]$ Theorem: Assuming treatment effects do not vary by year, Ε τ - = τ -. Proof: Ε τ - = E τ $+ τ -]$ = τ $ + τ -]$ = τ -.
26 Variance var(τ -) = var(τ $+ τ -]$) = var τ $ +var τ -]$ + 2cov(τ $, τ -]$) Both estimates are comparisons of the same teachers; likely to be highly positively correlated More than double the variance of each individual estimate
27 Constant Treatment Effect? Suppose constant treatment effect, so Y~ 6 1, t = Y~ 6 0,t + τ $ and Y~ 6 2, t = Y~ 6 1,t + τ -]$ j. Then: o τ $ = τ $ + Y $ (0, 2017) Y - (0, 2017) o τ -]$ = τ -]$ + Y $ (0, 2018) Y - (0, 2018) Under additivity, and if we again assume differences in time cancel with comparisons within the same year, then τ $ and τ -]$ are perfectly correlated! var(τ -) = var τ $ +var τ -]$ + 2 var τ $ var τ -]$ If var τ $ var τ -]$, then var(τ -) 4var τ $
28 2 years of KIA no KIA? 1 year of KIA no KIA 2 years of KIA no KIA Non-experimental direct approach Matched Sample: WHICH IS BETTER??? WAVE 1: WAVE 2: KIA KIA: 2 nd year KIA: 1 st year BIAS- VARIANCE TRADEOFF! Complementary! 1 year of KIA no KIA 2 years of KIA 1 year of KIA Experimental indirect approach
29 Other Interesting Tidbits Student-level versus school level analysis Combined analyses? Student/parental consent => missing data Joiners Non-compliance Teachers switching schools/courses Anticipation bias and more!
30 Conclusion Rerandomization can improve experimental design Propensity score matching can improve observational studies Bias-variance tradeoff for 2 year impact Lots of fun statistics in rich applied problems!
31 Funded by George Lucas Educational Foundation Joint work with Anna Saavedra, Amie Rappaport, Ying Liu, and Juan Saavedra
Rerandomization to Balance Covariates
Rerandomization to Balance Covariates Kari Lock Morgan Department of Statistics Penn State University Joint work with Don Rubin University of Minnesota Biostatistics 4/27/16 The Gold Standard Randomized
More informationMatching. Quiz 2. Matching. Quiz 2. Exact Matching. Estimand 2/25/14
STA 320 Design and Analysis of Causal Studies Dr. Kari Lock Morgan and Dr. Fan Li Department of Statistical Science Duke University Frequency 0 2 4 6 8 Quiz 2 Histogram of Quiz2 10 12 14 16 18 20 Quiz2
More informationWeighting. Homework 2. Regression. Regression. Decisions Matching: Weighting (0) W i. (1) -å l i. )Y i. (1-W i 3/5/2014. (1) = Y i.
Weighting Unconfounded Homework 2 Describe imbalance direction matters STA 320 Design and Analysis of Causal Studies Dr. Kari Lock Morgan and Dr. Fan Li Department of Statistical Science Duke University
More informationAddressing Analysis Issues REGRESSION-DISCONTINUITY (RD) DESIGN
Addressing Analysis Issues REGRESSION-DISCONTINUITY (RD) DESIGN Overview Assumptions of RD Causal estimand of interest Discuss common analysis issues In the afternoon, you will have the opportunity to
More informationPropensity Score Methods for Causal Inference
John Pura BIOS790 October 2, 2015 Causal inference Philosophical problem, statistical solution Important in various disciplines (e.g. Koch s postulates, Bradford Hill criteria, Granger causality) Good
More informationCausal Hazard Ratio Estimation By Instrumental Variables or Principal Stratification. Todd MacKenzie, PhD
Causal Hazard Ratio Estimation By Instrumental Variables or Principal Stratification Todd MacKenzie, PhD Collaborators A. James O Malley Tor Tosteson Therese Stukel 2 Overview 1. Instrumental variable
More informationObservational Studies and Propensity Scores
Observational Studies and s STA 320 Design and Analysis of Causal Studies Dr. Kari Lock Morgan and Dr. Fan Li Department of Statistical Science Duke University Makeup Class Rather than making you come
More informationBalancing Covariates via Propensity Score Weighting
Balancing Covariates via Propensity Score Weighting Kari Lock Morgan Department of Statistics Penn State University klm47@psu.edu Stochastic Modeling and Computational Statistics Seminar October 17, 2014
More informationOverlap Propensity Score Weighting to Balance Covariates
Overlap Propensity Score Weighting to Balance Covariates Kari Lock Morgan Department of Statistics Penn State University klm47@psu.edu JSM 2016 Chicago, IL Joint work with Fan Li (Duke) and Alan Zaslavsky
More informationDependence. MFM Practitioner Module: Risk & Asset Allocation. John Dodson. September 11, Dependence. John Dodson. Outline.
MFM Practitioner Module: Risk & Asset Allocation September 11, 2013 Before we define dependence, it is useful to define Random variables X and Y are independent iff For all x, y. In particular, F (X,Y
More informationOptimal Blocking by Minimizing the Maximum Within-Block Distance
Optimal Blocking by Minimizing the Maximum Within-Block Distance Michael J. Higgins Jasjeet Sekhon Princeton University University of California at Berkeley November 14, 2013 For the Kansas State University
More informationAssessing Studies Based on Multiple Regression
Assessing Studies Based on Multiple Regression Outline 1. Internal and External Validity 2. Threats to Internal Validity a. Omitted variable bias b. Functional form misspecification c. Errors-in-variables
More informationTechnical Track Session I: Causal Inference
Impact Evaluation Technical Track Session I: Causal Inference Human Development Human Network Development Network Middle East and North Africa Region World Bank Institute Spanish Impact Evaluation Fund
More informationIntroduction to Econometrics. Assessing Studies Based on Multiple Regression
Introduction to Econometrics The statistical analysis of economic (and related) data STATS301 Assessing Studies Based on Multiple Regression Titulaire: Christopher Bruffaerts Assistant: Lorenzo Ricci 1
More informationBalancing Covariates via Propensity Score Weighting: The Overlap Weights
Balancing Covariates via Propensity Score Weighting: The Overlap Weights Kari Lock Morgan Department of Statistics Penn State University klm47@psu.edu PSU Methodology Center Brown Bag April 6th, 2017 Joint
More informationChapter 7: Sampling Distributions
Chapter 7: Sampling Distributions Section 7.1 What is a Sampling Distribution? The Practice of Statistics, 4 th edition For AP* STARNES, YATES, MOORE Chapter 7 Sampling Distributions 7.1 What is a Sampling
More informationMATCHING FOR EE AND DR IMPACTS
MATCHING FOR EE AND DR IMPACTS Seth Wayland, Opinion Dynamics August 12, 2015 A Proposal Always use matching Non-parametric preprocessing to reduce model dependence Decrease bias and variance Better understand
More informationDependence. Practitioner Course: Portfolio Optimization. John Dodson. September 10, Dependence. John Dodson. Outline.
Practitioner Course: Portfolio Optimization September 10, 2008 Before we define dependence, it is useful to define Random variables X and Y are independent iff For all x, y. In particular, F (X,Y ) (x,
More informationarxiv: v1 [stat.me] 6 Nov 2015
Improving Covariate Balance in K Factorial Designs via Rerandomization Zach Branson, Tirthankar Dasgupta, and Donald B. Rubin Harvard University, Cambridge, USA Abstract arxiv:5.0973v [stat.me] 6 Nov 05
More informationAn Introduction to Causal Analysis on Observational Data using Propensity Scores
An Introduction to Causal Analysis on Observational Data using Propensity Scores Margie Rosenberg*, PhD, FSA Brian Hartman**, PhD, ASA Shannon Lane* *University of Wisconsin Madison **University of Connecticut
More informationComments on Best Quasi- Experimental Practice
Comments on Best Quasi- Experimental Practice Larry V. Hedges Northwestern University Presented at the Future of Implementation Evaluation, National Science Foundation, Arlington, VA, October 28, 2013
More informationWhat s New in Econometrics. Lecture 1
What s New in Econometrics Lecture 1 Estimation of Average Treatment Effects Under Unconfoundedness Guido Imbens NBER Summer Institute, 2007 Outline 1. Introduction 2. Potential Outcomes 3. Estimands and
More informationUnless provided with information to the contrary, assume for each question below that the Classical Linear Model assumptions hold.
Economics 345: Applied Econometrics Section A01 University of Victoria Midterm Examination #2 Version 1 SOLUTIONS Spring 2015 Instructor: Martin Farnham Unless provided with information to the contrary,
More informationImpact Evaluation of Mindspark Centres
Impact Evaluation of Mindspark Centres March 27th, 2014 Executive Summary About Educational Initiatives and Mindspark Educational Initiatives (EI) is a prominent education organization in India with the
More informationConceptual overview: Techniques for establishing causal pathways in programs and policies
Conceptual overview: Techniques for establishing causal pathways in programs and policies Antonio A. Morgan-Lopez, Ph.D. OPRE/ACF Meeting on Unpacking the Black Box of Programs and Policies 4 September
More informationII. MATCHMAKER, MATCHMAKER
II. MATCHMAKER, MATCHMAKER Josh Angrist MIT 14.387 Fall 2014 Agenda Matching. What could be simpler? We look for causal effects by comparing treatment and control within subgroups where everything... or
More informationChapter 7: Sampling Distributions
Chapter 7: Sampling Distributions Section 7.1 What is a Sampling Distribution? The Practice of Statistics, 4 th edition For AP* STARNES, YATES, MOORE Chapter 7 Sampling Distributions 7.1 What is a Sampling
More informationGov 2002: 5. Matching
Gov 2002: 5. Matching Matthew Blackwell October 1, 2015 Where are we? Where are we going? Discussed randomized experiments, started talking about observational data. Last week: no unmeasured confounders
More informationIntroduction to Econometrics. Review of Probability & Statistics
1 Introduction to Econometrics Review of Probability & Statistics Peerapat Wongchaiwat, Ph.D. wongchaiwat@hotmail.com Introduction 2 What is Econometrics? Econometrics consists of the application of mathematical
More informationTechnical and Practical Considerations in applying Value Added Models to estimate teacher effects
Technical and Practical Considerations in applying Value Added Models to estimate teacher effects Pete Goldschmidt, Ph.D. Educational Psychology and Counseling, Development Learning and Instruction Purpose
More informationSelection on Observables: Propensity Score Matching.
Selection on Observables: Propensity Score Matching. Department of Economics and Management Irene Brunetti ireneb@ec.unipi.it 24/10/2017 I. Brunetti Labour Economics in an European Perspective 24/10/2017
More informationPropensity Score Matching
Methods James H. Steiger Department of Psychology and Human Development Vanderbilt University Regression Modeling, 2009 Methods 1 Introduction 2 3 4 Introduction Why Match? 5 Definition Methods and In
More informationPrinciples Underlying Evaluation Estimators
The Principles Underlying Evaluation Estimators James J. University of Chicago Econ 350, Winter 2019 The Basic Principles Underlying the Identification of the Main Econometric Evaluation Estimators Two
More informationAssessing the Precision of Multisite Trials for Estimating the Parameters Of Cross-site Distributions of Program Effects. Howard S.
Assessing the Precision of Multisite Trials for Estimating the Parameters Of Cross-site Distributions of Program Effects Howard S. Bloom MDRC Jessaca Spybrook Western Michigan University May 10, 016 Manuscript
More informationEstimating and Using Propensity Score in Presence of Missing Background Data. An Application to Assess the Impact of Childbearing on Wellbeing
Estimating and Using Propensity Score in Presence of Missing Background Data. An Application to Assess the Impact of Childbearing on Wellbeing Alessandra Mattei Dipartimento di Statistica G. Parenti Università
More informationPSC 504: Instrumental Variables
PSC 504: Instrumental Variables Matthew Blackwell 3/28/2013 Instrumental Variables and Structural Equation Modeling Setup e basic idea behind instrumental variables is that we have a treatment with unmeasured
More informationVariable selection and machine learning methods in causal inference
Variable selection and machine learning methods in causal inference Debashis Ghosh Department of Biostatistics and Informatics Colorado School of Public Health Joint work with Yeying Zhu, University of
More informationTechnical Track Session I:
Impact Evaluation Technical Track Session I: Click to edit Master title style Causal Inference Damien de Walque Amman, Jordan March 8-12, 2009 Click to edit Master subtitle style Human Development Human
More informationEconometrics with Observational Data. Introduction and Identification Todd Wagner February 1, 2017
Econometrics with Observational Data Introduction and Identification Todd Wagner February 1, 2017 Goals for Course To enable researchers to conduct careful quantitative analyses with existing VA (and non-va)
More informationChapter 9: Assessing Studies Based on Multiple Regression. Copyright 2011 Pearson Addison-Wesley. All rights reserved.
Chapter 9: Assessing Studies Based on Multiple Regression 1-1 9-1 Outline 1. Internal and External Validity 2. Threats to Internal Validity a) Omitted variable bias b) Functional form misspecification
More informationAn Introduction to Path Analysis
An Introduction to Path Analysis PRE 905: Multivariate Analysis Lecture 10: April 15, 2014 PRE 905: Lecture 10 Path Analysis Today s Lecture Path analysis starting with multivariate regression then arriving
More informationWhen Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data?
When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data? Kosuke Imai Department of Politics Center for Statistics and Machine Learning Princeton University
More informationCollege Station Elem. School School Report Card Frasier Pike, PO Bx 670 College Station, AR
College Station Elem. School School Report Card 2014 2015 4710 Frasier Pike, PO Bx 670 College Station, AR 72053 501 490 5750 SCHOOL CHARACTERISTICS Principal Superintendent STUDENT DEMOGRAPHICS Emma Watson
More informationESTIMATION OF TREATMENT EFFECTS VIA MATCHING
ESTIMATION OF TREATMENT EFFECTS VIA MATCHING AAEC 56 INSTRUCTOR: KLAUS MOELTNER Textbooks: R scripts: Wooldridge (00), Ch.; Greene (0), Ch.9; Angrist and Pischke (00), Ch. 3 mod5s3 General Approach The
More informationJob Training Partnership Act (JTPA)
Causal inference Part I.b: randomized experiments, matching and regression (this lecture starts with other slides on randomized experiments) Frank Venmans Example of a randomized experiment: Job Training
More informationCan a Pseudo Panel be a Substitute for a Genuine Panel?
Can a Pseudo Panel be a Substitute for a Genuine Panel? Min Hee Seo Washington University in St. Louis minheeseo@wustl.edu February 16th 1 / 20 Outline Motivation: gauging mechanism of changes Introduce
More informationPropensity Score Matching and Analysis TEXAS EVALUATION NETWORK INSTITUTE AUSTIN, TX NOVEMBER 9, 2018
Propensity Score Matching and Analysis TEXAS EVALUATION NETWORK INSTITUTE AUSTIN, TX NOVEMBER 9, 2018 Schedule and outline 1:00 Introduction and overview 1:15 Quasi-experimental vs. experimental designs
More informationComparing Change Scores with Lagged Dependent Variables in Models of the Effects of Parents Actions to Modify Children's Problem Behavior
Comparing Change Scores with Lagged Dependent Variables in Models of the Effects of Parents Actions to Modify Children's Problem Behavior David R. Johnson Department of Sociology and Haskell Sie Department
More informationData Analysis and Statistical Methods Statistics 651
Data Analysis and Statistical Methods Statistics 651 http://www.stat.tamu.edu/~suhasini/teaching.html Boxplots and standard deviations Suhasini Subba Rao Review of previous lecture In the previous lecture
More informationCausal Inference with a Continuous Treatment and Outcome: Alternative Estimators for Parametric Dose-Response Functions
Causal Inference with a Continuous Treatment and Outcome: Alternative Estimators for Parametric Dose-Response Functions Joe Schafer Office of the Associate Director for Research and Methodology U.S. Census
More informationMultiple Regression. Midterm results: AVG = 26.5 (88%) A = 27+ B = C =
Economics 130 Lecture 6 Midterm Review Next Steps for the Class Multiple Regression Review & Issues Model Specification Issues Launching the Projects!!!!! Midterm results: AVG = 26.5 (88%) A = 27+ B =
More informationFinal Exam - Solutions
Ecn 102 - Analysis of Economic Data University of California - Davis March 17, 2010 Instructor: John Parman Final Exam - Solutions You have until 12:30pm to complete this exam. Please remember to put your
More informationThe Simple Linear Regression Model
The Simple Linear Regression Model Lesson 3 Ryan Safner 1 1 Department of Economics Hood College ECON 480 - Econometrics Fall 2017 Ryan Safner (Hood College) ECON 480 - Lesson 3 Fall 2017 1 / 77 Bivariate
More informationAn Introduction to Mplus and Path Analysis
An Introduction to Mplus and Path Analysis PSYC 943: Fundamentals of Multivariate Modeling Lecture 10: October 30, 2013 PSYC 943: Lecture 10 Today s Lecture Path analysis starting with multivariate regression
More informationRandom Intercept Models
Random Intercept Models Edps/Psych/Soc 589 Carolyn J. Anderson Department of Educational Psychology c Board of Trustees, University of Illinois Spring 2019 Outline A very simple case of a random intercept
More informationThe t-test: A z-score for a sample mean tells us where in the distribution the particular mean lies
The t-test: So Far: Sampling distribution benefit is that even if the original population is not normal, a sampling distribution based on this population will be normal (for sample size > 30). Benefit
More informationPropensity Score Analysis with Hierarchical Data
Propensity Score Analysis with Hierarchical Data Fan Li Alan Zaslavsky Mary Beth Landrum Department of Health Care Policy Harvard Medical School May 19, 2008 Introduction Population-based observational
More informationControlling for latent confounding by confirmatory factor analysis (CFA) Blinded Blinded
Controlling for latent confounding by confirmatory factor analysis (CFA) Blinded Blinded 1 Background Latent confounder is common in social and behavioral science in which most of cases the selection mechanism
More information2.1 Linear regression with matrices
21 Linear regression with matrices The values of the independent variables are united into the matrix X (design matrix), the values of the outcome and the coefficient are represented by the vectors Y and
More informationEstimation of the Conditional Variance in Paired Experiments
Estimation of the Conditional Variance in Paired Experiments Alberto Abadie & Guido W. Imbens Harvard University and BER June 008 Abstract In paired randomized experiments units are grouped in pairs, often
More informationOF CAUSAL INFERENCE THE MATHEMATICS IN STATISTICS. Department of Computer Science. Judea Pearl UCLA
THE MATHEMATICS OF CAUSAL INFERENCE IN STATISTICS Judea earl Department of Computer Science UCLA OUTLINE Statistical vs. Causal Modeling: distinction and mental barriers N-R vs. structural model: strengths
More informationCorrelation: Relationships between Variables
Correlation Correlation: Relationships between Variables So far, nearly all of our discussion of inferential statistics has focused on testing for differences between group means However, researchers are
More informationHarvard University. Rigorous Research in Engineering Education
Statistical Inference Kari Lock Harvard University Department of Statistics Rigorous Research in Engineering Education 12/3/09 Statistical Inference You have a sample and want to use the data collected
More informationCausal Inference with General Treatment Regimes: Generalizing the Propensity Score
Causal Inference with General Treatment Regimes: Generalizing the Propensity Score David van Dyk Department of Statistics, University of California, Irvine vandyk@stat.harvard.edu Joint work with Kosuke
More informationFlexible Estimation of Treatment Effect Parameters
Flexible Estimation of Treatment Effect Parameters Thomas MaCurdy a and Xiaohong Chen b and Han Hong c Introduction Many empirical studies of program evaluations are complicated by the presence of both
More informationDifference-in-Differences Methods
Difference-in-Differences Methods Teppei Yamamoto Keio University Introduction to Causal Inference Spring 2016 1 Introduction: A Motivating Example 2 Identification 3 Estimation and Inference 4 Diagnostics
More informationGov 2002: 4. Observational Studies and Confounding
Gov 2002: 4. Observational Studies and Confounding Matthew Blackwell September 10, 2015 Where are we? Where are we going? Last two weeks: randomized experiments. From here on: observational studies. What
More informationTHE DESIGN (VERSUS THE ANALYSIS) OF EVALUATIONS FROM OBSERVATIONAL STUDIES: PARALLELS WITH THE DESIGN OF RANDOMIZED EXPERIMENTS DONALD B.
THE DESIGN (VERSUS THE ANALYSIS) OF EVALUATIONS FROM OBSERVATIONAL STUDIES: PARALLELS WITH THE DESIGN OF RANDOMIZED EXPERIMENTS DONALD B. RUBIN My perspective on inference for causal effects: In randomized
More informationMatching with Multiple Control Groups, and Adjusting for Group Differences
Matching with Multiple Control Groups, and Adjusting for Group Differences Donald B. Rubin, Harvard University Elizabeth A. Stuart, Mathematica Policy Research, Inc. estuart@mathematica-mpr.com KEY WORDS:
More informationAGEC 661 Note Fourteen
AGEC 661 Note Fourteen Ximing Wu 1 Selection bias 1.1 Heckman s two-step model Consider the model in Heckman (1979) Y i = X iβ + ε i, D i = I {Z iγ + η i > 0}. For a random sample from the population,
More informationImpact Evaluation of Rural Road Projects. Dominique van de Walle World Bank
Impact Evaluation of Rural Road Projects Dominique van de Walle World Bank Introduction General consensus that roads are good for development & living standards A sizeable share of development aid and
More informationImplementing Matching Estimators for. Average Treatment Effects in STATA
Implementing Matching Estimators for Average Treatment Effects in STATA Guido W. Imbens - Harvard University West Coast Stata Users Group meeting, Los Angeles October 26th, 2007 General Motivation Estimation
More informationPath Analysis. PRE 906: Structural Equation Modeling Lecture #5 February 18, PRE 906, SEM: Lecture 5 - Path Analysis
Path Analysis PRE 906: Structural Equation Modeling Lecture #5 February 18, 2015 PRE 906, SEM: Lecture 5 - Path Analysis Key Questions for Today s Lecture What distinguishes path models from multivariate
More informationFrom Causality, Second edition, Contents
From Causality, Second edition, 2009. Preface to the First Edition Preface to the Second Edition page xv xix 1 Introduction to Probabilities, Graphs, and Causal Models 1 1.1 Introduction to Probability
More informationModeling Mediation: Causes, Markers, and Mechanisms
Modeling Mediation: Causes, Markers, and Mechanisms Stephen W. Raudenbush University of Chicago Address at the Society for Resesarch on Educational Effectiveness,Washington, DC, March 3, 2011. Many thanks
More informationSEQUENTIAL MULTIPLE ASSIGNMENT RANDOMIZATION TRIALS WITH ENRICHMENT (SMARTER) DESIGN
SEQUENTIAL MULTIPLE ASSIGNMENT RANDOMIZATION TRIALS WITH ENRICHMENT (SMARTER) DESIGN Ying Liu Division of Biostatistics, Medical College of Wisconsin Yuanjia Wang Department of Biostatistics & Psychiatry,
More informationMatching for Causal Inference Without Balance Checking
Matching for ausal Inference Without Balance hecking Gary King Institute for Quantitative Social Science Harvard University joint work with Stefano M. Iacus (Univ. of Milan) and Giuseppe Porro (Univ. of
More informationCausal Inference with Big Data Sets
Causal Inference with Big Data Sets Marcelo Coca Perraillon University of Colorado AMC November 2016 1 / 1 Outlone Outline Big data Causal inference in economics and statistics Regression discontinuity
More informationAdvanced Statistical Methods for Observational Studies L E C T U R E 0 1
Advanced Statistical Methods for Observational Studies L E C T U R E 0 1 introduction this class Website Expectations Questions observational studies The world of observational studies is kind of hard
More informationSelection on Correlated Characters (notes only)
Selection on Correlated Characters (notes only) The breeder s equation is best suited for plant and animal breeding where specific traits can be selected. In natural populations selection is rarely directed
More informationDynamics in Social Networks and Causality
Web Science & Technologies University of Koblenz Landau, Germany Dynamics in Social Networks and Causality JProf. Dr. University Koblenz Landau GESIS Leibniz Institute for the Social Sciences Last Time:
More informationECON 3150/4150, Spring term Lecture 7
ECON 3150/4150, Spring term 2014. Lecture 7 The multivariate regression model (I) Ragnar Nymoen University of Oslo 4 February 2014 1 / 23 References to Lecture 7 and 8 SW Ch. 6 BN Kap 7.1-7.8 2 / 23 Omitted
More information14.32 Final : Spring 2001
14.32 Final : Spring 2001 Please read the entire exam before you begin. You have 3 hours. No books or notes should be used. Calculators are allowed. There are 105 points. Good luck! A. True/False/Sometimes
More informationCato Elementary School School Report Card Jacksonville Cato Road North Little Rock, AR
Cato Elementary School School Report Card 2014 2015 9906 Jacksonville Cato Road North Little Rock, AR 72120 501 833 1160 SCHOOL CHARACTERISTICS Principal Superintendent STUDENT DEMOGRAPHICS Shyrel Lee
More informationLandmark Elementary School School Report Card Arch Street Pike Little Rock, AR
Landmark Elementary School School Report Card 2014 2015 16712 Arch Street Pike Little Rock, AR 72206 501 888 8790 SCHOOL CHARACTERISTICS Principal Superintendent STUDENT DEMOGRAPHICS Pam McCurry Jerry
More informationPsychometric Issues in Formative Assessment: Measuring Student Learning Throughout the Academic Year Using Interim Assessments
Psychometric Issues in Formative Assessment: Measuring Student Learning Throughout the Academic Year Using Interim Assessments Jonathan Templin The University of Georgia Neal Kingston and Wenhao Wang University
More informationThe Balance-Sample Size Frontier in Matching Methods for Causal Inference: Supplementary Appendix
The Balance-Sample Size Frontier in Matching Methods for Causal Inference: Supplementary Appendix Gary King Christopher Lucas Richard Nielsen March 22, 2016 Abstract This is a supplementary appendix to
More informationAdvanced Statistical Methods for Observational Studies L E C T U R E 0 1
Advanced Statistical Methods for Observational Studies L E C T U R E 0 1 introduction this class Website Expectations Questions observational studies The world of observational studies is kind of hard
More informationExploring Cultural Differences with Structural Equation Modelling
Exploring Cultural Differences with Structural Equation Modelling Wynne W. Chin University of Calgary and City University of Hong Kong 1996 IS Cross Cultural Workshop slide 1 The objectives for this presentation
More informationTargeted Maximum Likelihood Estimation in Safety Analysis
Targeted Maximum Likelihood Estimation in Safety Analysis Sam Lendle 1 Bruce Fireman 2 Mark van der Laan 1 1 UC Berkeley 2 Kaiser Permanente ISPE Advanced Topics Session, Barcelona, August 2012 1 / 35
More informationIntroduction to causal identification. Nidhiya Menon IGC Summer School, New Delhi, July 2015
Introduction to causal identification Nidhiya Menon IGC Summer School, New Delhi, July 2015 Outline 1. Micro-empirical methods 2. Rubin causal model 3. More on Instrumental Variables (IV) Estimating causal
More informationPrimal-dual Covariate Balance and Minimal Double Robustness via Entropy Balancing
Primal-dual Covariate Balance and Minimal Double Robustness via (Joint work with Daniel Percival) Department of Statistics, Stanford University JSM, August 9, 2015 Outline 1 2 3 1/18 Setting Rubin s causal
More informationECNS 561 Multiple Regression Analysis
ECNS 561 Multiple Regression Analysis Model with Two Independent Variables Consider the following model Crime i = β 0 + β 1 Educ i + β 2 [what else would we like to control for?] + ε i Here, we are taking
More informationModeling Log Data from an Intelligent Tutor Experiment
Modeling Log Data from an Intelligent Tutor Experiment Adam Sales 1 joint work with John Pane & Asa Wilks College of Education University of Texas, Austin RAND Corporation Pittsburgh, PA & Santa Monica,
More informationFor more information about how to cite these materials visit
Author(s): Kerby Shedden, Ph.D., 2010 License: Unless otherwise noted, this material is made available under the terms of the Creative Commons Attribution Share Alike 3.0 License: http://creativecommons.org/licenses/by-sa/3.0/
More informationRegression and correlation. Correlation & Regression, I. Regression & correlation. Regression vs. correlation. Involve bivariate, paired data, X & Y
Regression and correlation Correlation & Regression, I 9.07 4/1/004 Involve bivariate, paired data, X & Y Height & weight measured for the same individual IQ & exam scores for each individual Height of
More informationTopic 4: Model Specifications
Topic 4: Model Specifications Advanced Econometrics (I) Dong Chen School of Economics, Peking University 1 Functional Forms 1.1 Redefining Variables Change the unit of measurement of the variables will
More informationAP Statistics Unit 6 Note Packet Linear Regression. Scatterplots and Correlation
Scatterplots and Correlation Name Hr A scatterplot shows the relationship between two quantitative variables measured on the same individuals. variable (y) measures an outcome of a study variable (x) may
More informationSummary and discussion of The central role of the propensity score in observational studies for causal effects
Summary and discussion of The central role of the propensity score in observational studies for causal effects Statistics Journal Club, 36-825 Jessica Chemali and Michael Vespe 1 Summary 1.1 Background
More informationOUTLINE CAUSAL INFERENCE: LOGICAL FOUNDATION AND NEW RESULTS. Judea Pearl University of California Los Angeles (www.cs.ucla.
OUTLINE CAUSAL INFERENCE: LOGICAL FOUNDATION AND NEW RESULTS Judea Pearl University of California Los Angeles (www.cs.ucla.edu/~judea/) Statistical vs. Causal vs. Counterfactual inference: syntax and semantics
More information