Propensity Score Weighting with Multilevel Data

Size: px
Start display at page:

Download "Propensity Score Weighting with Multilevel Data"

Transcription

1 Propensity Score Weighting with Multilevel Data Fan Li Department of Statistical Science Duke University October 25, 2012 Joint work with Alan Zaslavsky and Mary Beth Landrum

2 Introduction In comparative effectiveness studies, the goal is usually to estimate the causal (treatment) effect. In noncausal descriptive studies, a common goal is to conduct an unconfounded comparison of two populations. Population-based observational studies are increasingly important sources both types of studies. Proper adjustment for differences between treatment groups is crucial to both descriptive and causal analyses. Regression has long been the standard method. Propensity score (Rosenbaum and Rubin, 1983) is a robust alternative to regression adjustment, applicable to both causal and descriptive studies.

3 Multilevel data Propensity score has been developed and applied in cross-sectional settings. Data in medical care and health policy research are often multilevel. Subjects are grouped in natural clusters, e.g., geographical area, hospitals, health service provider, etc. Significant within- and between-cluster variations. Ignoring cluster structure often leads to invalid inference. Inaccurate standard errors. Cluster-level effects could be confounded with individual-level effects. Hierarchical regression models provide a unified framework to study multilevel data.

4 Propensity score methods for multilevel data Propensity score methods for multilevel data have been less explored. Arpino and Mealli (2011) investigated propensity score matching in multilevel settings. We focus on propensity score weighting strategies: 1. Investigate the performance of different modeling and weighting strategies in the presence of model misspecification due to cluster-level confounders 2. Clarify the differences and connections between causal and unconfounded descriptive comparisons.

5 Notations Setting: (1) two-level structure, (2) treatment assigned at individual level. Cluster: h = 1,..., H. Cluster sample size: k = 1,..., n h ; total sample size: n = h n h. Treatment" (individual-level): Z hk, 0 control, 1 treatment. Covariates: X hk = (U hk, V h ), U hk individual-level; V h cluster-level. Outcome: Y hk.

6 Controlled descriptive comparisons Assignment": a nonmanipulable state defining membership in one of two groups. Objective: an unconfounded comparison of the observed outcomes between the groups Estimand: average controlled difference (ACD) - the difference in the means of Y in two groups with balanced covariate distributions. π ACD = E X [E(Y X, Z = 1) E(Y X, Z = 0)]. Examples: comparing outcomes among populations of different races; or of patients treated in two different years.

7 Causal comparisons Assignment: a potentially manipulable intervention. Objective: causal effect - comparison of the potential outcomes under treatment versus control in a common set of units Assuming SUTVA, each unit has two potential outcomes: Y hk (0), Y hk (1). Estimand: average treatment effect (ATE) π ATE = E[Y (1) Y (0)], Examples: evaluating the treatment effect of a drug, therapy or policy for a given population. Under the assumption of nonconfoundedness", (Y (0), Y (1)) Z X, we have π ATE = π ACD.

8 Propensity score Propensity score: e(x) = Pr(Z = 1 X). Balancing property - balancing propensity score also balances the covariates of different groups. X Z e(x). Using propensity score - two-step procedure: Step 1: estimate the propensity score, e.g., by logistic regression. Step 2: estimate the treatment effect by incorporating (matching, weighting, stratification, etc.) the estimated propensity score. Propensity score is a less parametric alternative to regression adjustment.

9 Propensity score weighting Foundation of propensity score weighting: [ ] ZY (1 Z )Y E = π ACD = π ATE = π, e(x) 1 e(x) Inverse-probability (Horvitz-Thompson) weight: { 1 e(x w hk = hk ), for Z hk = e(x hk ), for Z hk = 0. HT weighting balances (in expectation) the weighted distribution of covariates in the two groups.

10 Step 1: Propensity score models Marginal model-ignoring multilevel structure: logit(e hk ) = δ 0 + X hk α. Fixed effects model - adding cluster-specific main effect δ h : logit(e hk ) = δ h + U hk α. Key: the cluster membership is a nominal covariate. δ h absorbs the effects of both observed and unobserved cluster-level covariates V h. Estimates a balancing score without knowledge of Vh, but might lead to larger variance than the propensity score estimated under a correct model with fully observed V h. The Neyman-Scott problem: inconsistent estimates of δ h, α given large number of small clusters.

11 Step 1: Propensity score models Random effects model: logit(e hk ) = δ h + X hk α, with δ h N(δ 0, σ 2 δ ). Due to the shrinkage of random effects, V h need to be included. Pros: borrowing information across clusters, works better with many small clusters. Cons: produce a biased estimate if δ h are correlated with the covariates. Does not guarantee balance within each cluster. Random slopes can be incorporated.

12 Step 2: Estimators for the ACD or the ATE Two types of propensity-score-weighted estimators: 1. Nonparametric: applying the weights directly to the observed outcomes. 2. Parametric: applying the weights to the fitted outcomes from a parametric model (doubly-robust estimators).

13 Nonparametric Estimators Marginal estimator - ignore clustering: ˆπ ma = Z hk =1 Y hk w hk w 1 Z hk =0 Y hk w hk w 0, where w hk is the HT weight using the estimated propensity score, and w z = h,k:z hk =z w hk for z = 0, 1. Cluster-weighted estimator: (1) obtain cluster-specific ATE; and (2) average over clusters: ˆπ cl = h w hˆπ h. where ˆπ h = zhk =1 k h Y hk w hk w h1 h w h where w hz = z hk =z k h w hk for z = 0, 1. zhk =0 k h Y hk w hk, w h0

14 Parametric doubly-robust (DR) estimators DR estimator (Robins and colleagues): replace the observed outcome but fitted outcomes from a parametric model. ˆπ dr = Î hk /n, h,k where [ Z hk Y hk Î hk = ê hk (Z hk ê hk )Ŷ 1 hk ê hk ] [ (1 Z hk )Y hk 1 ê hk + (Z hk ê hk )Ŷ 0 hk 1 ê hk Ŷ z : the fitted outcome from a parametric model in group z. Double-robustness: in large samples, ˆπ dr is consistent if either the p.s. model or the outcome model is correct, but not necessarily both. SE in these estimators can be obtained via the delta method. ],

15 DR estimators: outcome model Marginal model - ignore clustering: Y hk = η 0 + Z hk γ + X hk β + ɛ hk, Fixed effects model - adjust for cluster-level main effects η h and covariates: Y hk = η h + Z hk γ + U hk β + ɛ hk, Random effects model - assume cluster-specific effects η h N(0, σ 2 η) Y hk = η h + Z hk γ + X hk β + ɛ hk. Different DR estimators: combinations of different choices of propensity score model and outcome model.

16 Large sample properties We investigate the large-sample bias of the nonparametric estimators in the presence of intra-cluster influence, in the setting with no observed covariates. n hz : number of subjects in Z group in cluster h. n z = h n hz, n h = n h1 + n h0, n = n 1 + n 0. Assume a Bernoulli assignment mechanism with varying rates by cluster: Z hk Bernoulli(p h ). (1) Assume an outcome model with cluster-specific random intercepts η h and constant treatment effect π, Y hk = η h +Z hk π+τd h +ɛ hk, η h N(η 0, σ 2 η), ɛ hk N(0, σ 2 ɛ ) (2) where d h = n h1 /n h is the cluster-specific proportion treated.

17 Nonparametric marginal estimator with marginal p.s. Violation to unconfoundness and/or SUTVA. Under marginal propensity score model, ê hk = n 1 /n, for all units. Then for the nonparametric marginal estimator: ˆπ ma ma = π + τ V V 0 + h η h ( n h1 n 1 n z hk =1 h0 ɛ hk ) + ( n 0 n 1 h,k z hk =0 h,k ɛ hk n 0 ), where V = n h (dh 2 (n 1/n) 2 ): weighted sample variance of {d h }; V0 = (n 1 n 0 )/n 2 : maximum of V, attainable if each cluster is assigned to either all treatment or all control (treatment assigned at cluster level). Expectation of the last two terms are asymptotically 0.

18 Nonparametric marginal estimator with marginal p.s. Therefore, under the generating model (1) and (2), the large sample bias of ˆπ ma ma is τv/v 0. Interpreting the result: 1. τ measures the variation in the outcome generating mechanism between clusters; 2. V/V 0 measures the variation in the treatment assignment mechanism between clusters. Both are ignored in ˆπ ma ma, introducing bias. We also analyze the non-parametric estimators that accounting for clustering in at least one stage - all lead to consistent estimates. Conclusion: with violations to unconfoundedness due to unobserved cluster effects, (1) ignoring clustering in both stages of p.s. weighting leads to biased estimates, (2) but accounting for clustering in at least one stage leads to consistent estimates.

19 Simulation Design The asymptotics do not apply to large number of small clusters". The Neyman-Scott problem: MLEs for δ h and α are inconsistent due to the growing number of parameters (clusters). The primary role of p.s. is to balance covariates - even inconsistent p.s. estimate may still work well, as long as it balances covariates. Simulated under balanced designs: (i) Small number of large clusters: H = 30, n h = 400, (ii) Large number of median clusters: H = 200, n h = 20, (iii) Large number of small clusters: H = 400, n h = 10. Additional goal: investigate the impact of an unmeasured cluster-level confounder in more realistic settings with observed covariates.

20 Simulation Design Two individual-level covariates: U 1 N(1, 1); U 2 Bernoulli(0.4). One cluster-level covariate V : (i) uncorrelated with U: V N(1, 2); (ii) correlated with the p.s. model: V h = 1 + 2δ h + u, u N(0, 0.5). True treatment assignment: logit Pr(Z hk = 1 X hk ) = δ h + X hk α, True parameters: δ h N(0, 1) and α = ( 1,.5, α 3 ). True (potential) outcome model - random effects model with interaction (η h N(0, 1), γ h N(0, 1), ɛ hk N(0, σ 2 y)): Y hk = η h + X hk β + Z hk (V h κ + γ h ) + ɛ hk, True parameters: κ = 2 and β = (1,.5, β 3 ).

21 Simulation Design Under each scenario, 500 replicates are generated. For each simulation, 1. Calculate the true value of π by h,k [Y hk(1) Y hk (0)]/n from the simulated units. 2. Estimate propensity score using the three models with U 1, U 2, omitting the cluster-level V. 3. Fit the three outcome models with U 1, U 2, omitting the cluster-level V. 4. Obtain the nonparametric estimates and the DR estimates for π from the above. For comparison, estimates from true models (benchmark) with U 1, U 2, V are also calculated. Random effects models are fitted via the lmer function in R package lme4 (penalized likelihood estimation).

22 Simulation Results nonparametric doubly-robust marginal cluster 1 benchmark marginal fixed random (H, n h ) bias mse bias mse bias mse bias mse bias mse bias mse BE (30,400) MA FE RE BE (200,20) MA FE RE BE (400,10) MA FE RE

23 Simulation Summaries 1. Estimators that ignore clustering in both propensity score and outcome models have much larger bias and RMSE than all the others. 2. The choice of the outcome model appears to have a larger impact on the results than the choice of the p.s. model. 3. Among the DR estimators, the ones with the benchmark outcome model and the ones with the random effects outcome model perform the best. 4. Given the same propensity score model, the nonparametric cluster-weighted estimator generally gives smaller bias than the DR estimator with fixed effects outcome model 5. When the number of cluster increases and the size of each cluster reduces, bias and RMSE increases considerably in all the estimators. The largest increase is observed in the nonparametric marginal estimators.

24 Application: racial disparity in health care Disparity: racial differences in care attributed to operations of health care system. Breast cancer screening data in different health insurance plans are collected from the Centers for Medicare and Medicaid Services (CMS). Focus on comparison between whites and blacks. Focus on the plans with at least 25 whites and 25 blacks: 64 plans with a total sample size of Subsample 3000 subjects from large (>3000) clusters to restrict impact of extremely large clusters, resulting sample size

25 Racial disparity data Treatment Z hk : black race (1=black, 0=white). Outcome Y hk : receive screening for breast cancer or not. Goal: investigate racial disparity in breast cancer screening - unconfounded descriptive comparison. Cluster level covariates V h : geographical code, non/for-profit status, practice model. Individual level covariates X hk : age category, eligibility for medicaid, poor neighborhood. Analysis: fit the three p.s. and the three outcome models, obtain estimates.

26 Balance Check: different propensity score models Figure : Histogram of cluster-specific weighted proportions of black enrollees. unweighted marginal p.s. model Frequency Frequency cluster specific proportion of blacks cluster specific proportion of blacks fixed effects p.s. model random effects p.s. model Frequency Frequency cluster specific proportion of blacks cluster specific proportion of blacks All p.s. models suggest: living in a poor neighborhood, being eligible for Medicaid and enrollment in a for-profit insurance plan are significantly associated with black race.

27 Results weighted doubly-robust marginal clustered marginal fixed random marginal (.79) (.83) (.85) (.41) (.43) fixed (.92) (.81) (.82) (.42) (.41) random (.91) (.82) (.44) (.39) (.39) Table : Average controlled difference in percentage of the proportion of getting breast cancer screening between blacks and whites. All estimators show the rate of receipt breast cancer screening is significantly lower among blacks than among whites with similar characteristics. Ignoring clustering in both stages doubled the estimates from analyses that account for clustering in at least one stage. Between-cluster variation is large. DR estimates have smaller SE.

28 Conclusion Propensity score is a powerful tool to balance covariates, for both causal and descriptive comparisons. We introduce and compare several propensity score weighting methods for multilevel data. Using analytical derivations and simulations, we show that 1. Ignoring the multilevel structure in both stages of p.s. weighting generally leads to severe bias for ATE/ACD. 2. Exploiting the multilevel structure, either parametrically or nonparametrically, in at least one stage can greatly reduce the bias. In complex observational data, correctly specifying the outcome model may be challenging. Propensity score methods are more robust. Interference between units in a cluster may exist (SUTVA does not hold), further research is needed.

Propensity Score Analysis with Hierarchical Data

Propensity Score Analysis with Hierarchical Data Propensity Score Analysis with Hierarchical Data Fan Li Alan Zaslavsky Mary Beth Landrum Department of Health Care Policy Harvard Medical School May 19, 2008 Introduction Population-based observational

More information

Balancing Covariates via Propensity Score Weighting

Balancing Covariates via Propensity Score Weighting Balancing Covariates via Propensity Score Weighting Kari Lock Morgan Department of Statistics Penn State University klm47@psu.edu Stochastic Modeling and Computational Statistics Seminar October 17, 2014

More information

Balancing Covariates via Propensity Score Weighting: The Overlap Weights

Balancing Covariates via Propensity Score Weighting: The Overlap Weights Balancing Covariates via Propensity Score Weighting: The Overlap Weights Kari Lock Morgan Department of Statistics Penn State University klm47@psu.edu PSU Methodology Center Brown Bag April 6th, 2017 Joint

More information

Weighting. Homework 2. Regression. Regression. Decisions Matching: Weighting (0) W i. (1) -å l i. )Y i. (1-W i 3/5/2014. (1) = Y i.

Weighting. Homework 2. Regression. Regression. Decisions Matching: Weighting (0) W i. (1) -å l i. )Y i. (1-W i 3/5/2014. (1) = Y i. Weighting Unconfounded Homework 2 Describe imbalance direction matters STA 320 Design and Analysis of Causal Studies Dr. Kari Lock Morgan and Dr. Fan Li Department of Statistical Science Duke University

More information

Assess Assumptions and Sensitivity Analysis. Fan Li March 26, 2014

Assess Assumptions and Sensitivity Analysis. Fan Li March 26, 2014 Assess Assumptions and Sensitivity Analysis Fan Li March 26, 2014 Two Key Assumptions 1. Overlap: 0

More information

Propensity Score Methods for Causal Inference

Propensity Score Methods for Causal Inference John Pura BIOS790 October 2, 2015 Causal inference Philosophical problem, statistical solution Important in various disciplines (e.g. Koch s postulates, Bradford Hill criteria, Granger causality) Good

More information

What s New in Econometrics. Lecture 1

What s New in Econometrics. Lecture 1 What s New in Econometrics Lecture 1 Estimation of Average Treatment Effects Under Unconfoundedness Guido Imbens NBER Summer Institute, 2007 Outline 1. Introduction 2. Potential Outcomes 3. Estimands and

More information

Selection on Observables: Propensity Score Matching.

Selection on Observables: Propensity Score Matching. Selection on Observables: Propensity Score Matching. Department of Economics and Management Irene Brunetti ireneb@ec.unipi.it 24/10/2017 I. Brunetti Labour Economics in an European Perspective 24/10/2017

More information

Causal Inference Basics

Causal Inference Basics Causal Inference Basics Sam Lendle October 09, 2013 Observed data, question, counterfactuals Observed data: n i.i.d copies of baseline covariates W, treatment A {0, 1}, and outcome Y. O i = (W i, A i,

More information

Combining multiple observational data sources to estimate causal eects

Combining multiple observational data sources to estimate causal eects Department of Statistics, North Carolina State University Combining multiple observational data sources to estimate causal eects Shu Yang* syang24@ncsuedu Joint work with Peng Ding UC Berkeley May 23,

More information

Overlap Propensity Score Weighting to Balance Covariates

Overlap Propensity Score Weighting to Balance Covariates Overlap Propensity Score Weighting to Balance Covariates Kari Lock Morgan Department of Statistics Penn State University klm47@psu.edu JSM 2016 Chicago, IL Joint work with Fan Li (Duke) and Alan Zaslavsky

More information

Covariate Balancing Propensity Score for General Treatment Regimes

Covariate Balancing Propensity Score for General Treatment Regimes Covariate Balancing Propensity Score for General Treatment Regimes Kosuke Imai Princeton University October 14, 2014 Talk at the Department of Psychiatry, Columbia University Joint work with Christian

More information

Balancing Covariates via Propensity Score Weighting

Balancing Covariates via Propensity Score Weighting Balancing Covariates via Propensity Score Weighting Fan Li Kari Lock Morgan Alan M. Zaslavsky 1 ABSTRACT Covariate balance is crucial for unconfounded descriptive or causal comparisons. However, lack of

More information

Data Integration for Big Data Analysis for finite population inference

Data Integration for Big Data Analysis for finite population inference for Big Data Analysis for finite population inference Jae-kwang Kim ISU January 23, 2018 1 / 36 What is big data? 2 / 36 Data do not speak for themselves Knowledge Reproducibility Information Intepretation

More information

Causal Inference Lecture Notes: Causal Inference with Repeated Measures in Observational Studies

Causal Inference Lecture Notes: Causal Inference with Repeated Measures in Observational Studies Causal Inference Lecture Notes: Causal Inference with Repeated Measures in Observational Studies Kosuke Imai Department of Politics Princeton University November 13, 2013 So far, we have essentially assumed

More information

multilevel modeling: concepts, applications and interpretations

multilevel modeling: concepts, applications and interpretations multilevel modeling: concepts, applications and interpretations lynne c. messer 27 october 2010 warning social and reproductive / perinatal epidemiologist concepts why context matters multilevel models

More information

Double Robustness. Bang and Robins (2005) Kang and Schafer (2007)

Double Robustness. Bang and Robins (2005) Kang and Schafer (2007) Double Robustness Bang and Robins (2005) Kang and Schafer (2007) Set-Up Assume throughout that treatment assignment is ignorable given covariates (similar to assumption that data are missing at random

More information

Causal Inference with Big Data Sets

Causal Inference with Big Data Sets Causal Inference with Big Data Sets Marcelo Coca Perraillon University of Colorado AMC November 2016 1 / 1 Outlone Outline Big data Causal inference in economics and statistics Regression discontinuity

More information

ESTIMATING AVERAGE TREATMENT EFFECTS: REGRESSION DISCONTINUITY DESIGNS Jeff Wooldridge Michigan State University BGSE/IZA Course in Microeconometrics

ESTIMATING AVERAGE TREATMENT EFFECTS: REGRESSION DISCONTINUITY DESIGNS Jeff Wooldridge Michigan State University BGSE/IZA Course in Microeconometrics ESTIMATING AVERAGE TREATMENT EFFECTS: REGRESSION DISCONTINUITY DESIGNS Jeff Wooldridge Michigan State University BGSE/IZA Course in Microeconometrics July 2009 1. Introduction 2. The Sharp RD Design 3.

More information

Bayesian regression tree models for causal inference: regularization, confounding and heterogeneity

Bayesian regression tree models for causal inference: regularization, confounding and heterogeneity Bayesian regression tree models for causal inference: regularization, confounding and heterogeneity P. Richard Hahn, Jared Murray, and Carlos Carvalho June 22, 2017 The problem setting We want to estimate

More information

Causal inference in multilevel data structures:

Causal inference in multilevel data structures: Causal inference in multilevel data structures: Discussion of papers by Li and Imai Jennifer Hill May 19 th, 2008 Li paper Strengths Area that needs attention! With regard to propensity score strategies

More information

Causal Inference with a Continuous Treatment and Outcome: Alternative Estimators for Parametric Dose-Response Functions

Causal Inference with a Continuous Treatment and Outcome: Alternative Estimators for Parametric Dose-Response Functions Causal Inference with a Continuous Treatment and Outcome: Alternative Estimators for Parametric Dose-Response Functions Joe Schafer Office of the Associate Director for Research and Methodology U.S. Census

More information

The Role of the Propensity Score in Observational Studies with Complex Data Structures

The Role of the Propensity Score in Observational Studies with Complex Data Structures The Role of the Propensity Score in Observational Studies with Complex Data Structures Fabrizia Mealli mealli@disia.unifi.it Department of Statistics, Computer Science, Applications University of Florence

More information

arxiv: v1 [stat.me] 15 May 2011

arxiv: v1 [stat.me] 15 May 2011 Working Paper Propensity Score Analysis with Matching Weights Liang Li, Ph.D. arxiv:1105.2917v1 [stat.me] 15 May 2011 Associate Staff of Biostatistics Department of Quantitative Health Sciences, Cleveland

More information

When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Panel Data?

When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Panel Data? When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Panel Data? Kosuke Imai Department of Politics Center for Statistics and Machine Learning Princeton University Joint

More information

Flexible Estimation of Treatment Effect Parameters

Flexible Estimation of Treatment Effect Parameters Flexible Estimation of Treatment Effect Parameters Thomas MaCurdy a and Xiaohong Chen b and Han Hong c Introduction Many empirical studies of program evaluations are complicated by the presence of both

More information

Gov 2002: 4. Observational Studies and Confounding

Gov 2002: 4. Observational Studies and Confounding Gov 2002: 4. Observational Studies and Confounding Matthew Blackwell September 10, 2015 Where are we? Where are we going? Last two weeks: randomized experiments. From here on: observational studies. What

More information

Imbens/Wooldridge, IRP Lecture Notes 2, August 08 1

Imbens/Wooldridge, IRP Lecture Notes 2, August 08 1 Imbens/Wooldridge, IRP Lecture Notes 2, August 08 IRP Lectures Madison, WI, August 2008 Lecture 2, Monday, Aug 4th, 0.00-.00am Estimation of Average Treatment Effects Under Unconfoundedness, Part II. Introduction

More information

Weighting Methods. Harvard University STAT186/GOV2002 CAUSAL INFERENCE. Fall Kosuke Imai

Weighting Methods. Harvard University STAT186/GOV2002 CAUSAL INFERENCE. Fall Kosuke Imai Weighting Methods Kosuke Imai Harvard University STAT186/GOV2002 CAUSAL INFERENCE Fall 2018 Kosuke Imai (Harvard) Weighting Methods Stat186/Gov2002 Fall 2018 1 / 13 Motivation Matching methods for improving

More information

Matching. Quiz 2. Matching. Quiz 2. Exact Matching. Estimand 2/25/14

Matching. Quiz 2. Matching. Quiz 2. Exact Matching. Estimand 2/25/14 STA 320 Design and Analysis of Causal Studies Dr. Kari Lock Morgan and Dr. Fan Li Department of Statistical Science Duke University Frequency 0 2 4 6 8 Quiz 2 Histogram of Quiz2 10 12 14 16 18 20 Quiz2

More information

Primal-dual Covariate Balance and Minimal Double Robustness via Entropy Balancing

Primal-dual Covariate Balance and Minimal Double Robustness via Entropy Balancing Primal-dual Covariate Balance and Minimal Double Robustness via (Joint work with Daniel Percival) Department of Statistics, Stanford University JSM, August 9, 2015 Outline 1 2 3 1/18 Setting Rubin s causal

More information

Vector-Based Kernel Weighting: A Simple Estimator for Improving Precision and Bias of Average Treatment Effects in Multiple Treatment Settings

Vector-Based Kernel Weighting: A Simple Estimator for Improving Precision and Bias of Average Treatment Effects in Multiple Treatment Settings Vector-Based Kernel Weighting: A Simple Estimator for Improving Precision and Bias of Average Treatment Effects in Multiple Treatment Settings Jessica Lum, MA 1 Steven Pizer, PhD 1, 2 Melissa Garrido,

More information

Empirical Likelihood Methods for Two-sample Problems with Data Missing-by-Design

Empirical Likelihood Methods for Two-sample Problems with Data Missing-by-Design 1 / 32 Empirical Likelihood Methods for Two-sample Problems with Data Missing-by-Design Changbao Wu Department of Statistics and Actuarial Science University of Waterloo (Joint work with Min Chen and Mary

More information

Estimating the Marginal Odds Ratio in Observational Studies

Estimating the Marginal Odds Ratio in Observational Studies Estimating the Marginal Odds Ratio in Observational Studies Travis Loux Christiana Drake Department of Statistics University of California, Davis June 20, 2011 Outline The Counterfactual Model Odds Ratios

More information

Quantitative Economics for the Evaluation of the European Policy

Quantitative Economics for the Evaluation of the European Policy Quantitative Economics for the Evaluation of the European Policy Dipartimento di Economia e Management Irene Brunetti Davide Fiaschi Angela Parenti 1 25th of September, 2017 1 ireneb@ec.unipi.it, davide.fiaschi@unipi.it,

More information

An Introduction to Causal Analysis on Observational Data using Propensity Scores

An Introduction to Causal Analysis on Observational Data using Propensity Scores An Introduction to Causal Analysis on Observational Data using Propensity Scores Margie Rosenberg*, PhD, FSA Brian Hartman**, PhD, ASA Shannon Lane* *University of Wisconsin Madison **University of Connecticut

More information

Multi-level Models: Idea

Multi-level Models: Idea Review of 140.656 Review Introduction to multi-level models The two-stage normal-normal model Two-stage linear models with random effects Three-stage linear models Two-stage logistic regression with random

More information

Bayesian causal forests: dealing with regularization induced confounding and shrinking towards homogeneous effects

Bayesian causal forests: dealing with regularization induced confounding and shrinking towards homogeneous effects Bayesian causal forests: dealing with regularization induced confounding and shrinking towards homogeneous effects P. Richard Hahn, Jared Murray, and Carlos Carvalho July 29, 2018 Regularization induced

More information

Causal Hazard Ratio Estimation By Instrumental Variables or Principal Stratification. Todd MacKenzie, PhD

Causal Hazard Ratio Estimation By Instrumental Variables or Principal Stratification. Todd MacKenzie, PhD Causal Hazard Ratio Estimation By Instrumental Variables or Principal Stratification Todd MacKenzie, PhD Collaborators A. James O Malley Tor Tosteson Therese Stukel 2 Overview 1. Instrumental variable

More information

Structural Nested Mean Models for Assessing Time-Varying Effect Moderation. Daniel Almirall

Structural Nested Mean Models for Assessing Time-Varying Effect Moderation. Daniel Almirall 1 Structural Nested Mean Models for Assessing Time-Varying Effect Moderation Daniel Almirall Center for Health Services Research, Durham VAMC & Dept. of Biostatistics, Duke University Medical Joint work

More information

Propensity Score Methods, Models and Adjustment

Propensity Score Methods, Models and Adjustment Propensity Score Methods, Models and Adjustment Dr David A. Stephens Department of Mathematics & Statistics McGill University Montreal, QC, Canada. d.stephens@math.mcgill.ca www.math.mcgill.ca/dstephens/siscr2016/

More information

DATA-ADAPTIVE VARIABLE SELECTION FOR

DATA-ADAPTIVE VARIABLE SELECTION FOR DATA-ADAPTIVE VARIABLE SELECTION FOR CAUSAL INFERENCE Group Health Research Institute Department of Biostatistics, University of Washington shortreed.s@ghc.org joint work with Ashkan Ertefaie Department

More information

Deductive Derivation and Computerization of Semiparametric Efficient Estimation

Deductive Derivation and Computerization of Semiparametric Efficient Estimation Deductive Derivation and Computerization of Semiparametric Efficient Estimation Constantine Frangakis, Tianchen Qian, Zhenke Wu, and Ivan Diaz Department of Biostatistics Johns Hopkins Bloomberg School

More information

High Dimensional Propensity Score Estimation via Covariate Balancing

High Dimensional Propensity Score Estimation via Covariate Balancing High Dimensional Propensity Score Estimation via Covariate Balancing Kosuke Imai Princeton University Talk at Columbia University May 13, 2017 Joint work with Yang Ning and Sida Peng Kosuke Imai (Princeton)

More information

Targeted Maximum Likelihood Estimation in Safety Analysis

Targeted Maximum Likelihood Estimation in Safety Analysis Targeted Maximum Likelihood Estimation in Safety Analysis Sam Lendle 1 Bruce Fireman 2 Mark van der Laan 1 1 UC Berkeley 2 Kaiser Permanente ISPE Advanced Topics Session, Barcelona, August 2012 1 / 35

More information

Combining Difference-in-difference and Matching for Panel Data Analysis

Combining Difference-in-difference and Matching for Panel Data Analysis Combining Difference-in-difference and Matching for Panel Data Analysis Weihua An Departments of Sociology and Statistics Indiana University July 28, 2016 1 / 15 Research Interests Network Analysis Social

More information

University of California, Berkeley

University of California, Berkeley University of California, Berkeley U.C. Berkeley Division of Biostatistics Working Paper Series Year 2015 Paper 334 Targeted Estimation and Inference for the Sample Average Treatment Effect Laura B. Balzer

More information

Estimating and Using Propensity Score in Presence of Missing Background Data. An Application to Assess the Impact of Childbearing on Wellbeing

Estimating and Using Propensity Score in Presence of Missing Background Data. An Application to Assess the Impact of Childbearing on Wellbeing Estimating and Using Propensity Score in Presence of Missing Background Data. An Application to Assess the Impact of Childbearing on Wellbeing Alessandra Mattei Dipartimento di Statistica G. Parenti Università

More information

Extending causal inferences from a randomized trial to a target population

Extending causal inferences from a randomized trial to a target population Extending causal inferences from a randomized trial to a target population Issa Dahabreh Center for Evidence Synthesis in Health, Brown University issa dahabreh@brown.edu January 16, 2019 Issa Dahabreh

More information

The Impact of Measurement Error on Propensity Score Analysis: An Empirical Investigation of Fallible Covariates

The Impact of Measurement Error on Propensity Score Analysis: An Empirical Investigation of Fallible Covariates The Impact of Measurement Error on Propensity Score Analysis: An Empirical Investigation of Fallible Covariates Eun Sook Kim, Patricia Rodríguez de Gil, Jeffrey D. Kromrey, Rheta E. Lanehart, Aarti Bellara,

More information

THE DESIGN (VERSUS THE ANALYSIS) OF EVALUATIONS FROM OBSERVATIONAL STUDIES: PARALLELS WITH THE DESIGN OF RANDOMIZED EXPERIMENTS DONALD B.

THE DESIGN (VERSUS THE ANALYSIS) OF EVALUATIONS FROM OBSERVATIONAL STUDIES: PARALLELS WITH THE DESIGN OF RANDOMIZED EXPERIMENTS DONALD B. THE DESIGN (VERSUS THE ANALYSIS) OF EVALUATIONS FROM OBSERVATIONAL STUDIES: PARALLELS WITH THE DESIGN OF RANDOMIZED EXPERIMENTS DONALD B. RUBIN My perspective on inference for causal effects: In randomized

More information

Constrained Maximum Likelihood Estimation for Model Calibration Using Summary-level Information from External Big Data Sources

Constrained Maximum Likelihood Estimation for Model Calibration Using Summary-level Information from External Big Data Sources Constrained Maximum Likelihood Estimation for Model Calibration Using Summary-level Information from External Big Data Sources Yi-Hau Chen Institute of Statistical Science, Academia Sinica Joint with Nilanjan

More information

Propensity Score Methods for Estimating Causal Effects from Complex Survey Data

Propensity Score Methods for Estimating Causal Effects from Complex Survey Data Propensity Score Methods for Estimating Causal Effects from Complex Survey Data Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School

More information

MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS

MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS MULTIPLE REGRESSION AND ISSUES IN REGRESSION ANALYSIS Page 1 MSR = Mean Regression Sum of Squares MSE = Mean Squared Error RSS = Regression Sum of Squares SSE = Sum of Squared Errors/Residuals α = Level

More information

CompSci Understanding Data: Theory and Applications

CompSci Understanding Data: Theory and Applications CompSci 590.6 Understanding Data: Theory and Applications Lecture 17 Causality in Statistics Instructor: Sudeepa Roy Email: sudeepa@cs.duke.edu Fall 2015 1 Today s Reading Rubin Journal of the American

More information

ANALYTIC COMPARISON. Pearl and Rubin CAUSAL FRAMEWORKS

ANALYTIC COMPARISON. Pearl and Rubin CAUSAL FRAMEWORKS ANALYTIC COMPARISON of Pearl and Rubin CAUSAL FRAMEWORKS Content Page Part I. General Considerations Chapter 1. What is the question? 16 Introduction 16 1. Randomization 17 1.1 An Example of Randomization

More information

Causal Inference Lecture Notes: Selection Bias in Observational Studies

Causal Inference Lecture Notes: Selection Bias in Observational Studies Causal Inference Lecture Notes: Selection Bias in Observational Studies Kosuke Imai Department of Politics Princeton University April 7, 2008 So far, we have studied how to analyze randomized experiments.

More information

Correlated and Interacting Predictor Omission for Linear and Logistic Regression Models

Correlated and Interacting Predictor Omission for Linear and Logistic Regression Models Clemson University TigerPrints All Dissertations Dissertations 8-207 Correlated and Interacting Predictor Omission for Linear and Logistic Regression Models Emily Nystrom Clemson University, emily.m.nystrom@gmail.com

More information

New Developments in Econometrics Lecture 11: Difference-in-Differences Estimation

New Developments in Econometrics Lecture 11: Difference-in-Differences Estimation New Developments in Econometrics Lecture 11: Difference-in-Differences Estimation Jeff Wooldridge Cemmap Lectures, UCL, June 2009 1. The Basic Methodology 2. How Should We View Uncertainty in DD Settings?

More information

The problem of causality in microeconometrics.

The problem of causality in microeconometrics. The problem of causality in microeconometrics. Andrea Ichino University of Bologna and Cepr June 11, 2007 Contents 1 The Problem of Causality 1 1.1 A formal framework to think about causality....................................

More information

An Introduction to Causal Mediation Analysis. Xu Qin University of Chicago Presented at the Central Iowa R User Group Meetup Aug 10, 2016

An Introduction to Causal Mediation Analysis. Xu Qin University of Chicago Presented at the Central Iowa R User Group Meetup Aug 10, 2016 An Introduction to Causal Mediation Analysis Xu Qin University of Chicago Presented at the Central Iowa R User Group Meetup Aug 10, 2016 1 Causality In the applications of statistics, many central questions

More information

Causal Mechanisms Short Course Part II:

Causal Mechanisms Short Course Part II: Causal Mechanisms Short Course Part II: Analyzing Mechanisms with Experimental and Observational Data Teppei Yamamoto Massachusetts Institute of Technology March 24, 2012 Frontiers in the Analysis of Causal

More information

Use of Matching Methods for Causal Inference in Experimental and Observational Studies. This Talk Draws on the Following Papers:

Use of Matching Methods for Causal Inference in Experimental and Observational Studies. This Talk Draws on the Following Papers: Use of Matching Methods for Causal Inference in Experimental and Observational Studies Kosuke Imai Department of Politics Princeton University April 13, 2009 Kosuke Imai (Princeton University) Matching

More information

PROPENSITY SCORE MATCHING. Walter Leite

PROPENSITY SCORE MATCHING. Walter Leite PROPENSITY SCORE MATCHING Walter Leite 1 EXAMPLE Question: Does having a job that provides or subsidizes child care increate the length that working mothers breastfeed their children? Treatment: Working

More information

Bootstrapping Sensitivity Analysis

Bootstrapping Sensitivity Analysis Bootstrapping Sensitivity Analysis Qingyuan Zhao Department of Statistics, The Wharton School University of Pennsylvania May 23, 2018 @ ACIC Based on: Qingyuan Zhao, Dylan S. Small, and Bhaswar B. Bhattacharya.

More information

Comment: Understanding OR, PS and DR

Comment: Understanding OR, PS and DR Statistical Science 2007, Vol. 22, No. 4, 560 568 DOI: 10.1214/07-STS227A Main article DOI: 10.1214/07-STS227 c Institute of Mathematical Statistics, 2007 Comment: Understanding OR, PS and DR Zhiqiang

More information

A Sampling of IMPACT Research:

A Sampling of IMPACT Research: A Sampling of IMPACT Research: Methods for Analysis with Dropout and Identifying Optimal Treatment Regimes Marie Davidian Department of Statistics North Carolina State University http://www.stat.ncsu.edu/

More information

Guanglei Hong. University of Chicago. Presentation at the 2012 Atlantic Causal Inference Conference

Guanglei Hong. University of Chicago. Presentation at the 2012 Atlantic Causal Inference Conference Guanglei Hong University of Chicago Presentation at the 2012 Atlantic Causal Inference Conference 2 Philosophical Question: To be or not to be if treated ; or if untreated Statistics appreciates uncertainties

More information

Telescope Matching: A Flexible Approach to Estimating Direct Effects *

Telescope Matching: A Flexible Approach to Estimating Direct Effects * Telescope Matching: A Flexible Approach to Estimating Direct Effects * Matthew Blackwell Anton Strezhnev August 4, 2018 Abstract Estimating the direct effect of a treatment fixing the value of a consequence

More information

Estimating Causal Effects from Observational Data with the CAUSALTRT Procedure

Estimating Causal Effects from Observational Data with the CAUSALTRT Procedure Paper SAS374-2017 Estimating Causal Effects from Observational Data with the CAUSALTRT Procedure Michael Lamm and Yiu-Fai Yung, SAS Institute Inc. ABSTRACT Randomized control trials have long been considered

More information

Gov 2002: 5. Matching

Gov 2002: 5. Matching Gov 2002: 5. Matching Matthew Blackwell October 1, 2015 Where are we? Where are we going? Discussed randomized experiments, started talking about observational data. Last week: no unmeasured confounders

More information

University of California, Berkeley

University of California, Berkeley University of California, Berkeley U.C. Berkeley Division of Biostatistics Working Paper Series Year 2011 Paper 288 Targeted Maximum Likelihood Estimation of Natural Direct Effect Wenjing Zheng Mark J.

More information

Robustness to Parametric Assumptions in Missing Data Models

Robustness to Parametric Assumptions in Missing Data Models Robustness to Parametric Assumptions in Missing Data Models Bryan Graham NYU Keisuke Hirano University of Arizona April 2011 Motivation Motivation We consider the classic missing data problem. In practice

More information

Implementing Matching Estimators for. Average Treatment Effects in STATA

Implementing Matching Estimators for. Average Treatment Effects in STATA Implementing Matching Estimators for Average Treatment Effects in STATA Guido W. Imbens - Harvard University West Coast Stata Users Group meeting, Los Angeles October 26th, 2007 General Motivation Estimation

More information

Specification Errors, Measurement Errors, Confounding

Specification Errors, Measurement Errors, Confounding Specification Errors, Measurement Errors, Confounding Kerby Shedden Department of Statistics, University of Michigan October 10, 2018 1 / 32 An unobserved covariate Suppose we have a data generating model

More information

Covariate selection and propensity score specification in causal inference

Covariate selection and propensity score specification in causal inference Covariate selection and propensity score specification in causal inference Ingeborg Waernbaum Doctoral Dissertation Department of Statistics Umeå University SE-901 87 Umeå, Sweden Copyright c 2008 by Ingeborg

More information

Econometric Analysis of Cross Section and Panel Data

Econometric Analysis of Cross Section and Panel Data Econometric Analysis of Cross Section and Panel Data Jeffrey M. Wooldridge / The MIT Press Cambridge, Massachusetts London, England Contents Preface Acknowledgments xvii xxiii I INTRODUCTION AND BACKGROUND

More information

Investigating mediation when counterfactuals are not metaphysical: Does sunlight exposure mediate the effect of eye-glasses on cataracts?

Investigating mediation when counterfactuals are not metaphysical: Does sunlight exposure mediate the effect of eye-glasses on cataracts? Investigating mediation when counterfactuals are not metaphysical: Does sunlight exposure mediate the effect of eye-glasses on cataracts? Brian Egleston Fox Chase Cancer Center Collaborators: Daniel Scharfstein,

More information

A comparison of weighted estimators for the population mean. Ye Yang Weighting in surveys group

A comparison of weighted estimators for the population mean. Ye Yang Weighting in surveys group A comparison of weighted estimators for the population mean Ye Yang Weighting in surveys group Motivation Survey sample in which auxiliary variables are known for the population and an outcome variable

More information

Variable selection and machine learning methods in causal inference

Variable selection and machine learning methods in causal inference Variable selection and machine learning methods in causal inference Debashis Ghosh Department of Biostatistics and Informatics Colorado School of Public Health Joint work with Yeying Zhu, University of

More information

Subgroup Balancing Propensity Score

Subgroup Balancing Propensity Score Subgroup Balancing Propensity Score Jing Dong Industrial and Commercial Bank of China Ltd. Junni L. Zhang Guanghua School of Management and Center for Statistical Science, Peking University Fan Li arxiv:1707.05835v1

More information

arxiv: v1 [stat.me] 8 Jun 2016

arxiv: v1 [stat.me] 8 Jun 2016 Principal Score Methods: Assumptions and Extensions Avi Feller UC Berkeley Fabrizia Mealli Università di Firenze Luke Miratrix Harvard GSE arxiv:1606.02682v1 [stat.me] 8 Jun 2016 June 9, 2016 Abstract

More information

The propensity score with continuous treatments

The propensity score with continuous treatments 7 The propensity score with continuous treatments Keisuke Hirano and Guido W. Imbens 1 7.1 Introduction Much of the work on propensity score analysis has focused on the case in which the treatment is binary.

More information

For more information about how to cite these materials visit

For more information about how to cite these materials visit Author(s): Kerby Shedden, Ph.D., 2010 License: Unless otherwise noted, this material is made available under the terms of the Creative Commons Attribution Share Alike 3.0 License: http://creativecommons.org/licenses/by-sa/3.0/

More information

Marginal versus conditional effects: does it make a difference? Mireille Schnitzer, PhD Université de Montréal

Marginal versus conditional effects: does it make a difference? Mireille Schnitzer, PhD Université de Montréal Marginal versus conditional effects: does it make a difference? Mireille Schnitzer, PhD Université de Montréal Overview In observational and experimental studies, the goal may be to estimate the effect

More information

NBER WORKING PAPER SERIES USE OF PROPENSITY SCORES IN NON-LINEAR RESPONSE MODELS: THE CASE FOR HEALTH CARE EXPENDITURES

NBER WORKING PAPER SERIES USE OF PROPENSITY SCORES IN NON-LINEAR RESPONSE MODELS: THE CASE FOR HEALTH CARE EXPENDITURES NBER WORKING PAPER SERIES USE OF PROPENSITY SCORES IN NON-LINEAR RESPONSE MODELS: THE CASE FOR HEALTH CARE EXPENDITURES Anirban Basu Daniel Polsky Willard G. Manning Working Paper 14086 http://www.nber.org/papers/w14086

More information

The consequences of misspecifying the random effects distribution when fitting generalized linear mixed models

The consequences of misspecifying the random effects distribution when fitting generalized linear mixed models The consequences of misspecifying the random effects distribution when fitting generalized linear mixed models John M. Neuhaus Charles E. McCulloch Division of Biostatistics University of California, San

More information

Imbens/Wooldridge, Lecture Notes 1, Summer 07 1

Imbens/Wooldridge, Lecture Notes 1, Summer 07 1 Imbens/Wooldridge, Lecture Notes 1, Summer 07 1 What s New in Econometrics NBER, Summer 2007 Lecture 1, Monday, July 30th, 9.00-10.30am Estimation of Average Treatment Effects Under Unconfoundedness 1.

More information

Gov 2002: 13. Dynamic Causal Inference

Gov 2002: 13. Dynamic Causal Inference Gov 2002: 13. Dynamic Causal Inference Matthew Blackwell December 19, 2015 1 / 33 1. Time-varying treatments 2. Marginal structural models 2 / 33 1/ Time-varying treatments 3 / 33 Time-varying treatments

More information

Structural Nested Mean Models for Assessing Time-Varying Effect Moderation. Daniel Almirall

Structural Nested Mean Models for Assessing Time-Varying Effect Moderation. Daniel Almirall 1 Structural Nested Mean Models for Assessing Time-Varying Effect Moderation Daniel Almirall Center for Health Services Research, Durham VAMC & Duke University Medical, Dept. of Biostatistics Joint work

More information

Introduction to Statistical Analysis

Introduction to Statistical Analysis Introduction to Statistical Analysis Changyu Shen Richard A. and Susan F. Smith Center for Outcomes Research in Cardiology Beth Israel Deaconess Medical Center Harvard Medical School Objectives Descriptive

More information

Bayesian Inference for Sequential Treatments under Latent Sequential Ignorability. June 19, 2017

Bayesian Inference for Sequential Treatments under Latent Sequential Ignorability. June 19, 2017 Bayesian Inference for Sequential Treatments under Latent Sequential Ignorability Alessandra Mattei, Federico Ricciardi and Fabrizia Mealli Department of Statistics, Computer Science, Applications, University

More information

When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data?

When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data? When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data? Kosuke Imai Department of Politics Center for Statistics and Machine Learning Princeton University

More information

Econometrics with Observational Data. Introduction and Identification Todd Wagner February 1, 2017

Econometrics with Observational Data. Introduction and Identification Todd Wagner February 1, 2017 Econometrics with Observational Data Introduction and Identification Todd Wagner February 1, 2017 Goals for Course To enable researchers to conduct careful quantitative analyses with existing VA (and non-va)

More information

Propensity Score for Causal Inference of Multiple and Multivalued Treatments

Propensity Score for Causal Inference of Multiple and Multivalued Treatments Virginia Commonwealth University VCU Scholars Compass Theses and Dissertations Graduate School 2016 Propensity Score for Causal Inference of Multiple and Multivalued Treatments Zirui Gu Virginia Commonwealth

More information

An Efficient Estimation Method for Longitudinal Surveys with Monotone Missing Data

An Efficient Estimation Method for Longitudinal Surveys with Monotone Missing Data An Efficient Estimation Method for Longitudinal Surveys with Monotone Missing Data Jae-Kwang Kim 1 Iowa State University June 28, 2012 1 Joint work with Dr. Ming Zhou (when he was a PhD student at ISU)

More information

Causal Inference with General Treatment Regimes: Generalizing the Propensity Score

Causal Inference with General Treatment Regimes: Generalizing the Propensity Score Causal Inference with General Treatment Regimes: Generalizing the Propensity Score David van Dyk Department of Statistics, University of California, Irvine vandyk@stat.harvard.edu Joint work with Kosuke

More information

Causal Inference with Measurement Error

Causal Inference with Measurement Error Causal Inference with Measurement Error by Di Shu A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Doctor of Philosophy in Statistics Waterloo,

More information

Marginal, crude and conditional odds ratios

Marginal, crude and conditional odds ratios Marginal, crude and conditional odds ratios Denitions and estimation Travis Loux Gradute student, UC Davis Department of Statistics March 31, 2010 Parameter Denitions When measuring the eect of a binary

More information

When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data?

When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data? When Should We Use Linear Fixed Effects Regression Models for Causal Inference with Longitudinal Data? Kosuke Imai Princeton University Asian Political Methodology Conference University of Sydney Joint

More information