Advanced Statistical Methods for Observational Studies L E C T U R E 0 1

Transcription

1 Advanced Statistical Methods for Observational Studies L E C T U R E 0 1

2 introduction

3 this class Website Expectations Questions

4 observational studies The world of observational studies is kind of hard to get into because it grew up in several distinct, but overlapping, disciplines: Epidemiology Demography Economics (econometrics) Political Science Sociology Biostatistics Statistics Psychology (psychometrics) Computer Science

5 a small bit about me I do causal inference: Observational studies of: NICU, surgical interventions, and smoking cessation. Randomized studies: six interventions here at Stanford to improve educational outcomes, two large trials of a sexual assault prevention program.

6 an aside You can call me Mike If you want to use my last name, Baiocchi, totally feel free to if you say it this way I ll definitely know you re talking to me: bye-oh-key

7 potential outcomes framework Design of Observational Studies: section 2.2

8 causal inference Our goal is to figure out what the change in the outcome will be for a person if we change from the control to the treatment: Y i t = 1 Y i t = 0 = Δ i

9 causal inference Y i t = 1 Y i t = 0 Fundamental problem of causality: We cannot observe both Y i t = 1 and Y i t = 0 at the same time.

10 notation DOS notation r C = response when the control is applied r T = response when the treatment is applied r Ti = response for observational unit i when the treatment is applied. Z i = assignment to treatment for unit i. R i = Z i r Ti 1 Z i r Ci = the observed response for unit i.

11 a table person i r_{c_i} r_{t_i} z_i r_i Delta_i

12 a table person i r_{c_i} r_{t_i} z_i r_i Delta_i Sometimes called: The Science Table

13 potential outcomes framework The potential outcomes framework is meant to clarify the challenge: We only get to observe half of the data we actually want. How do we get at the unobserved half?

14 study design vs. inference Don Rubin: For objective causal inference, design trumps analysis

15 study design vs. inference 90% of statistics classes are about inference Why? It s useful, getting you those confidence intervals and p-values. The math is pretty cool. It feels hard. Because many of us don t really know much about the real world

16 design R A N D O M I Z A T I O N A N D S A M P L I N G

17 where does the data come from? We design trials. Assign groups that are similar at baseline Examine counterfactuals We also design sampling schemes. Representative groups Understand population from subsets of those populations Both use elements of control and randomness

18 an example: randomization Want to study a pill. Design the study Uniform randomization Matched pairs randomization Crossover design Cluster-randomized Inference t-test Matched-pairs t-test Repeated measures model Generalized linear mixed model But maybe all of those could be GLMM.

19 an example: sampling Want to study an election. Design the study Simple random sample Stratified sampling Snowball sampling Inference t-test Inverse probability weighting Generalized linear mixed model But maybe all of those could be GLMM.

20 different beliefs about where data come from RCT and sampling Structural equation modeling y i = β 0 + β 1 x 1,i + + β p x p,i + ε i If you want to be disabused of SEM spend some time reading

21 where data come from If you d like to be abused by SEM please see

22 inference

23 picking inference Inference requires assumptions Linear regression: Linearity and additivity Independent errors Homoskedastiticity Normality of errors Permutation test: Known assignment mechanism to T or C Fancier methods tend to have more assumptions and thus leave you open to more lines of attack. These attacks can be obviated by careful preparation during the design phase.

24 picking inference Use the simplest method that gets the job done. If you want to accomplish more, collect more data or do additional analyses. ( If have to use something more complicated than a t-test then someone messed up ) The fewer assumptions there are, the easier it will be to perform a sensitivity analysis build an argument to beat back the haters.

25 picking inference Another option: Proof by intimidation This paper presents a breakthrough in rhetorical logic, a promising field of science, of great values to those writing research proposals. It provides new, and utterly convincing tools for closing embarrassing gaps in your reasoning, without having to resort to brute-force methods such as actually thinking about the problem in the first place. The Craske-Trump Theorem Conjecture will allow researchers in any field to use the technique of Proof by Intimidation fully. - Michael Wilkinson (Annals of Improbable Research 2000)

26 observational study design N E O N A T A L I N T E N S I V E C A R E U N I T S

27 Application: Regionalization Hospitals vary in their ability to care for premature infants. The American Academy of Pediatrics recognizes levels: 1, 2, 3A, 3B, 3C, 3D and Regional Centers. Regionalization of care refers to a policy that suggests or requires that high-risk mothers deliver at hospitals with greater levels of capabilities.

28 Outcome Outcome

29 Outcome Outcome

30 The data Every baby delivered in a 10+ year period California Pennsylvania Missouri Mothers information ICD9 codes Delivery Post-delivery complications Some pre-delivery Some SES information Zip code of residence Birth/death certificates Census information PA and MO have zip code level CA will have block group Pre-delivery Severity?

31 H H

32 H H

33 naïve model for observational studies Design of Observational Studies: section

34 the variation comes from somewhere The model starts with thinking about the assignment mechanism: π i = Pr(Z i = 1 r Ti, r Ci, x i, u i ) where π i will likely vary from person to person, x i are the observed covariates and u i are the unobserved covariates. Consider the vector of all treatment assignments, Z. This can be written as: Pr Z = z r T, r C, x, u = π 1 z 1 1 π 1 1 z 1 π n z n 1 π n 1 z n

35 motivation for matching Imagine we can find two subjects, k and l, such that Z k + Z l = 1 but π k = π l Recall the model π i = Pr(Z i = 1 r Ti, r Ci, x i, u i ) we can only match on a subset of these. If we were in the RCT framework then we could create these trivially by doing a matched pairs randomization and assigning k and l to the same pair. We re building a story about the randomization.

36 H H Excess Travel Time

37 H H Excess Travel Time

38 H H Excess Travel Time

39 H H McClellan, McNeil & Newhouse; "Does more intensive treatment of acute myocardial infarction reduce mortality? JAMA. 272(11): , September 1994

40 motivation for matching Recall the model π i = Pr(Z i = 1 r Ti, r Ci, x i, u i ) we can only match on a subset of these. r Ti, r Ci are whether baby lives or dies. Z i is whether baby is delivered at low or high level NIC. x i are the 40+ covariates of mom and baby, as well as where they live. u i are all the unmeasured features that influenced the baby, mom and the process of how they ended up delivering at the hospital they did.

41 H H

42 H H

43 motivation: natural experiments Perhaps the π i behave in ways that will allow us to draw inferences in ways that are similar to experiments. Loosely speaking, this will tend to happen when the assignment mechanism is haphazard in its assignment not using prognostically relevant covariates to sort the units into levels of the treatment. What happens if we can find matches such that Z k + Z l = 1 and π k = π l?

44 motivation: natural experiments What happens if we can find matches such that π k = π l? Pr Z k = z k, Z l = z l r Tk, r Ck, x k, u k, r Tl, r Cl, x l, u l = π k z k 1 π k 1 z k π l z l 1 πl 1 z l = π k z k +z l 1 πk 1 z k +(1 z l ) What if we design our study such that Z l + Z k = 1? Pr Z k = 1, Z l = 0, Z l + Z k = 1

45 motivation: natural experiments What happens if we can find matches such that π k = π l? Pr Z k = z k, Z l = z l r Tk, r Ck, x k, u k, r Tl, r Cl, x l, u l = π k z k +z l 1 πk 1 z k +(1 z l ) What if we design our study such that Z l + Z k = 1? Pr Z k = 1, Z l = 0, Z l + Z k = 1 = Pr Z k = 1, Z l = 0 Pr Z k = 1, Z l = 0 +Pr Z k = 0, Z l = 1 = π k πk 1 1 +(1 0) Pr Z k = 1, Z l = 0 +Pr Z k = 0, Z l = 1

46 motivation: natural experiments What if we design our study such that Z l + Z k = 1? Pr Z k = 1, Z l = 0, Z l + Z k = 1 = Pr Z k = 1, Z l = 0 Pr Z k = 1, Z l = 0 +Pr Z k = 0, Z l = 1 = π k πk 1 1 +(1 0) Pr Z k = 1, Z l = 0 +Pr Z k = 0, Z l = 1 = π k πk 1 1 +(1 0) π k π k 1 1 +(1 0) +π k π k 1 0 +(1 1) = 1 2 IF we can do this then we get to use the same tools developed for RCTs!

47 the naïve model Those that look alike (in our data set) are alike: π i = Pr Z i = 1 r Ti, r Ci, x i, u i = Pr Z i = 1 x i and with 0 < π i < 1 for all i = 1, 2,, n Pr Z = z r T, r C, x, u = n i=1 π i z i 1 πi 1 z i We could make this model true by randomly assigning, potentially using biased coins based on the observed covariates.

48 at the core: the propensity score This quantity has a name: e(x i ) = Pr Z i = 1 x i often, e(x i ) is referred to as the propensity score. It s the probability of assignment to treatment. It links the features of an individual unit to its assignment to the treatment. It s a description of the assignment mechanism. It s not magic. But it does have several cool features that are not immediately obvious: Propensity: facilitates randomization (Fisher) Score: short for balancing score which facilitates balance (Mill)

49 wishes If the ideal match and the naïve model were true then we could just match on the observed covariates and analyze data using traditional techniques for RCTs. Unfortunately, it s extraordinarily unlikely that strongly ignorable treatment assignment actually holds. It can be more plausible or less plausible given different situations, but it can never be proven.

50 takeaways

51 takeaways The potential outcomes framework helps organize our thinking on counterfactuals Design comes in two flavors (actually, three but the third one is not very healthy) In prospective studies design is an obvious consideration and one that MUST be passed through in order to obtain data In retrospective studies, design is a less obvious consideration but one that MUST be passed through unfortunately without much attention paid

52 fin. C H E C K O U T T H E W E B S I T E.