PROVING CAUSALITY IN SOCIAL SCIENCE: A POTENTIAL APPLICATION OF OLOGS

Transcription

1 PROVING CAUSALITY IN SOCIAL SCIENCE: A POTENTIAL APPLICATION OF OLOGS By Noam Agrist 1

2 THE GOALS OF SOCIAL SCIENCE Explai the world aroud us. What is really happeig ad why. Example: do Kidles boost test scores? Image courtesy of Dekuwa o Flickr. Available CC BY-NC-SA. 2

3 THE GOALS OF SOCIAL SCIENCE Did the Kidle itervetio work? 7 Average for studets with Kidles Before After 3

4 WHAT S WRONG WITH THIS? Some (ot all) Key Biases: (1) Self Selectio Bias (2) Omitted Variable Bias (3) Attritio Bias (4) Couterfactual Kidle Program Rich Test Scores 4

5 CONVENTIONAL METHODS OF ADDRESSING BIASES Add a cotrol group addresses couterfactual bias Kidle Cotrol Before After 5

6 MATHEMATICAL-BASED METHODS OF ADDRESSING BIASES Ecoometrics A) Regressios B) Cotrollig C) Istrumetal variables D) Radomized trials E) Other methods Woodridge, Jeffrey M. Itroductory Ecoometrics: A Moder Approach. Cegage Learig, Cegage Learig. All rights reserved. This cotet is excluded from our Creative Commos licese. For more iformatio, see 6

7 SIMPLE LINEAR REGRESSION: OVERVIEW Y i X i i where (x i x) (y i y) y x (x i x) 2 Slope Residual Itercept 7

8 SIMPLE LINEAR REGRESSION: DERIVATION The goal is to miimize the sum of square residuals i order to fid the lie of best fit: mi Q(, ), where 2 Q(, ) i (y x i ) 2 1) 2) a b (y i x i ) 2 (y i x i ) 2 8

9 SIMPLE LINEAR REGRESSION: DERIVATION 1) a (y i x i ) 2 2(y i x i )( 1) 0 2 (y i x i ) 0 y i x i 0 2) 2(y i x i )( x i ) 0 2 (y i x i x i x 2 i ) 0 2 x i y i x i x i 0 y x 9

10 SIMPLE LINEAR REGRESSION: DERIVATION Plug i alpha from equatio (1) ito equatio (2): 1) y x 2) 2 x i y i x i x i 0 2 x i y i (y x) x i x i 0 x i y i yx x i 2 x 2 10

11 SIMPLE LINEAR REGRESSION Y i X i i (x i x) (y i y) y x (x i x) 2 Y X Itercept Slope 11

12 SIMPLE LINEAR REGRESSION: AN EXAMPLE Does a kidle club (as described before) boost test scores? Let s fid out! Y i X i i where Y 2 testscores X i participatigikc Depedet Variable Idepedet Variable 12

13 READING PROGRAM: TEST SCORES Where: diff is the differece i test score over 8 weeks ad Kc is a dummy variable that equals 1 id a studet participated i the club ad 0 if they did t Thus, we have: Result: Participatio i the Kidle Club results i a icrease of.2134 o stadardized test scores relative to all studets i the school (everyoe else icreased aroud.22 aturally) 13

14 ECONOMETRICS: CONTROLLING I expad the simple liear regressio to iclude more idepedet (or predictor) variables: Y i i,1 X i,1 i,2 X i,2 i,3 X i,3 i, X i, i Multiple regressio allows me to cotrol for certai characteristics (i.e. I ca determie relatioships holdig/give certai variables costat). This takes ito accout covariace amog variables. Ituitio: coditioal probabilities 14

15 READING PROGRAM: CONTROLLING I cotrol for (1) icome status ad (2) grade level: Result: our estimate for impact of kidle club participatio o test score icrease relative to the whole school goes up by.04 Explaatio 1: free luch, harder to improve, so had more kids with free luch such that whe we cotrol, we have a higher impact Explaatio 2: higher grade level, less room for improvemet sice higher baselie so we had more kids at a higher grade level i group 15

16 DID IT WORK!? More issues Omitted Variable Bias (ca t cotrol for everythig) factors ot icluded i regressio which impact idepedet ad depedet variable Selectio Bias Attritio bias 16

17 ECONOMETRICS: INSTRUMENTAL VARIABLES Examples of Z i Causal Outcome Birth Date Geder Twis Returs of a extra year of schoolig Title IX affect o labor market outcomes Family size effect o schoolig 17

18 ECONOMETRICS: INSTRUMENTAL VARIABLES Usig a radom variable to istrumet for causality such that Z has o correlatio with Y outcome variable, but is highly correlated with X such that you ca attribute a causal impact of X o Y Y i X i i Y i Z i i Where Cov(Z i, i ) 0 Cov(Z i, X i ) 0 18 ad

19 ECONOMETRICS: RANDOMIZED TRIALS Program desig: radomly assig treatmet ad cotrol group (like cliical trials i medicie) elimiates motivatio/demographics biases i itervetio I this case Z i ca act as a istrumetal variable sice our treatmet dummy variable is determied by radom lottery Y i Z i i Now, β has a casual iterpretatio, ot just correlatio 19

20 USING RANDOM ASSIGNMENT FOR IV APPROACH Add a radom cotrol group addresses couterfactual bias, omitted variable bias, selfselectio Kidle Cotrol Before After 20

21 READING PROGRAM: RANDOMIZED TRIAL I regress the dummy treatmet variable X (1 if radomly selected ito the KC, 0 if radomly ot selected) o the differece i test scores after 8 weeks Result: our estimate for impact of kidle club o test score icrease is.26 of a readig level (causal sice relative to radom cotrol group) Note 1: this is a rigorous result. Also, otice that the regressio with cotrols yields a result closest to the cotrolled regressio Note 2: it is critical to check for statistical sigificace Note 3: measurig itetio to treat effect, so uderestimate of impact 21

22 IS CATEGORY THEORY USEFUL FOR SOCIAL SCIENTISTS? 22

23 SO WHERE DOES CATEGORY THEORY COME IN? Defiig ad determiig Omitted Variable Bias through some comprehesive olog Safe Neighborhood Rich Kidle Program Test Scores TestSctores i Kidle i,1 X i,1 Rich i,2 X i,2 SafeNeighborhood i,3 X i,3 i 23

24 ONE POTENTIAL OLOG: KINDLES AND TEST SCORES WHICH DOESN T WORK BUT IS USEFUL NONETHELESS (NOT FUNCTIONS FROM SETS TO SETS) A safe eighborhood lives i A perso from a wealthy family has access to Test preparatio has Space to study effectively has access to Grade Level icreases A Kidle eables studets to read Books through exposure to readig icreases F&P readig scores Through more study time icreases 24

25 A REAL OLOG f A pair (u,v) where u ad v are distict studets g A pair of scores (s1, s2) for oe test A pair of scores (s1, s2 ) for oe test The differece A umber The differece f: seds v to a bad school without a Kidle; sed u to a good school without a Kidle. g: sed v to a bad school with a Kidle; sed u to a good school with a Kidle. Note: choose a global variable which captures effects from other variables 25

26 WHAT CAN I CONCLUDE FROM THIS OLOG? Creatig a olog helps the social scietist thik through the various processes ad factors which might affect our outcomes of iterest There are multiple sources of omitted variable bias The process of creatig a olog helps a scietist determie a comprehesive system which ca icludes as may factors as the social scietist deems relevat A wealthy family captures may of the omitted variables, see by the coectig arrows thus cotrollig for havig a wealthy family should yield estimates close to those causal estimated usig a radomized cotrolled trial 26

27 READING PROGRAM: CONTROLLING I cotrol for (1) icome status ad (2) grade level: 27

28 INSTRUMENTAL VARIABLES APPLICATION USING OLOGGY-LIKE STUFF (ANOTHER BROKEN BUT USEFUL OLOG) Z A perso i a wealthy family Twis icrease ca afford goig to Birth cotrol A type of school yields Family Size X through dividig up resources per child affects educatio level of a child Y 28

29 CONCLUSION If we desig ologs before our aalysis phase we ca make sure that: (1) we come up with credible istrumetal variables (2) whe we cotrol for all relevat variables that might have otherwise bee omitted ad determie which variables ca proxy for others This is importat because: Radomized trials are expesive ad we ofte resort to cotrollig as a alterative optio to determie causal relatioships I the absece of radomized trials, we also eed good istrumets to determie causal relatioships 29

30 QUESTIONS/COMMENTS Image by MIT OpeCourseWare. 30

31 MIT OpeCourseWare 18.S996 Category Theory for Scietist Sprig 2013 For iformatio about citig these materials or our Terms of Use, visit: