Advanced Statistical Methods for Observational Studies L E C T U R E 0 1

Transcription

1 Advanced Statistical Methods for Observational Studies L E C T U R E 0 1

2 introduction

3 this class Website Expectations Questions

4 observational studies The world of observational studies is kind of hard to get into because it grew up in several distinct, but overlapping, disciplines: Epidemiology Demography Economics (econometrics) Political Science Sociology Biostatistics Statistics Psychology (psychometrics) Computer Science

5 an aside You can call me Mike If you want to use my last name, Baiocchi, totally feel free to if you say it this way I ll definitely know you re talking to me: bye-oh-key

6 potential outcomes framework Design of Observational Studies: section 2.2

7 causal inference Our goal is to figure out what the change in the outcome will be for a person if we change from the control to the treatment: Y i t = 1 Y i t = 0 = Δ i

8 causal inference Y i t = 1 Y i t = 0 Fundamental problem of causality: We cannot observe both Y i t = 1 and Y i t = 0 at the same time.

9 notation DOS notation r C = response when the control is applied r T = response when the treatment is applied r Ti = response for observational unit i when the treatment is applied. Z i = assignment to treatment for unit i. R i = Z i r Ti 1 Z i r Ci = the observed response for unit i.

10 a table person i r_{c_i} r_{t_i} z_i r_i Delta_i

11 a table person i r_{c_i} r_{t_i} z_i r_i Delta_i

12 potential outcomes framework The potential outcomes framework is meant to clarify the challenge: We only get to observe half of the data we actually want. How do we get at the unobserved half?

13 study design vs. inference Don Rubin: For objective causal inference, design trumps analysis

14 study design vs. inference 90% of statistics classes are about inference Why? It s useful, getting you those confidence intervals and p-values. The math is pretty cool. It feels hard. Because many of us don t really know much about the real world

15 design R A N D O M I Z A T I O N A N D S A M P L I N G

16 where does the data come from? We design trials. Assign groups that are similar at baseline Examine counterfactuals We also design sampling schemes. Representative groups Understand population from subsets of those populations Both use elements of control and randomness

17 an example: randomization Want to study a pill. Design the study Uniform randomization Matched pairs randomization Crossover design Cluster-randomized Inference t-test Matched-pairs t-test Repeated measures model Generalized linear mixed model But maybe all of those could be GLMM.

18 an example: sampling Want to study an election. Design the study Simple random sample Stratified sampling Snowball sampling Inference t-test Inverse probability weighting Generalized linear mixed model But maybe all of those could be GLMM.

19 different beliefs about where data come from RCT and sampling Structural equation modeling y i = β 0 + β 1 x 1,i + + β p x p,i + ε i If you want to be disabused of SEM spend some time reading

20 where data come from If you d like to be abused by SEM please see

21 inference

22 picking inference Inference requires assumptions Linear regression: Linearity and additivity Independent errors Homoskedastiticity Normality of errors Permutation test: Known assignment mechanism to T or C Fancier methods tend to have more assumptions and thus leave you open to more lines of attack. These attacks can be obviated by careful preparation during the design phase.

23 picking inference Use the simplest method that gets the job done. If you want to accomplish more, collect more data or do additional analyses. ( If have to use something more complicated than a t-test then someone messed up ) The fewer assumptions there are, the easier it will be to perform a sensitivity analysis build an argument to beat back the haters.

24 picking inference Another option: Proof by intimidation This paper presents a breakthrough in rhetorical logic, a promising field of science, of great values to those writing research proposals. It provides new, and utterly convincing tools for closing embarrassing gaps in your reasoning, without having to resort to brute-force methods such as actually thinking about the problem in the first place. The Craske-Trump Theorem Conjecture will allow researchers in any field to use the technique of Proof by Intimidation fully. - Michael Wilkinson (Annals of Improbable Research 2000)

25 prospective study design R C T A N D S A M P L I N G

26 prospective study design A lot of the foundations have been worked out: Experimental design Sampling But, obviously, there are a lot of cool developments still going on: Experimental design: adaptive trials, point-of-care randomization, Sampling: active learning, explore-exploit learning

27 observational (and retrospective) design This seems weird Usually a data set is in front of you, so you just analyze it It takes some thought to see this Let s do an example.

28 observational study design N E O N A T A L I N T E N S I V E C A R E U N I T S

29 Application: Regionalization Hospitals vary in their ability to care for premature infants. The American Academy of Pediatrics recognizes levels: 1, 2, 3A, 3B, 3C, 3D and Regional Centers. Regionalization of care refers to a policy that suggests or requires that high-risk mothers deliver at hospitals with greater levels of capabilities.

30 1% 2%

31 Outcome Outcome

32 Outcome Outcome

33 The data Every baby delivered in a 10+ year period California Pennsylvania Missouri Mothers information ICD9 codes Delivery Post-delivery complications Some pre-delivery Some SES information Zip code of residence Birth/death certificates Census information PA and MO have zip code level CA will have block group Pre-delivery Severity?

34 Summary of Problem Want to quantify effect of level of NICU on rate of death Observational data Sorting bias Some sorting variables are unobserved

35 H H

36 H H

37 H H

38 linking design to inference

39 the fundamental form For RCTs, the fundamental form of inference is based on permutation tests (a.k.a. randomization tests) For sampling, the fundamental form of inference is bootstrap (debatable) Everything else is necessary concessions to the particularities of a given problem Connect the structure of the data to your form of inference

40 Design of Observational Studies: section 2.3 Fisher s sharp null

41 the meaning of no effect There are at least two definitions of no effect floating around In this class, we ll almost exclusively use what is referred to as Fisher s Sharp Null: H 0 : r Ci = r Ti for unit of observation i, the response under control is identical to the response if the unit were to have received treatment. In most introductory statistics classes H 0 : μ C = μ T the population C has the same mean as the population T.

42 a few words about no effect Fisher s Sharp Null is quite sharp, meaning that it points down to the individual level. This grew out of the RCT framework. Unsurprisingly, the no difference in population means came out of the sampling framework. There was no obvious emphasis on the individual level in sampling. If Fisher s Sharp Null is true then the no difference in population means is true. The converse does not hold.

43 Design of Observational Studies: section 2.3 the permutation test

44 intuition If we assume H 0 : r Ci = r Ti then we get a very powerful way of dealing with the fundamental problem of causality

45 intuition person i r_{c_i} r_{t_i} z_i r_i

46 intuition 1.29 Repeat these flips over and over again. Build up the null distribution. person i r_{c_i} r_{t_i} z_i r_i

47 takeaways

48 takeaways The potential outcomes framework helps organize our thinking on counterfactuals Design comes in two flavors (actually, three but the third one is not very healthy) In prospective studies design is an obvious consideration and one that MUST be passed through in order to obtain data In retrospective studies, design is a less obvious consideration but one that MUST be passed through unfortunately without much attention paid Fisher s Sharp Null and control of the assignment process leads to a very elegant and robust method for inference

49 fin. C H E C K O U T T H E W E B S I T E.