I. Big Idea of Statistical Mediation PIER Summer 2017 Mediation Basics H. Seltman June 8, 2017 We find the treatment X changes outcome Y. Now we want to know how that happens. E.g., is the effect of X working through M, such that X -> M -> Y? Treatment of infection example Aside: In this context, interaction is called moderation. We could picture an arrow from X to Y and an arrow from moderator L to the arrow between X and Y (or an arrow from L to Y with an arrow from X to the arrow between L and Y, since interaction is agnostic as to the roles of the interacting IVs). II. Journal Articles for Mediation in Education Research a. Word Generation Randomized Trial: Discussion Mediates the Impact of Program Treatment on Academic Word Learning by Joshua F. Lawrence, et al. (American Educational Research Journal, 52(4): 750 786, 2015, DOI: 10.3102/0002831215579485) http://journals.sagepub.com/doi/full/10.3102/0002831215579485 b. Causal Mediation in Educational Research: An Illustration Using International Assessment Data by Daniel H Caro (Journal of Research on Educational Effectiveness, 8:4, 577-597, 2015, DOI: 10.1080/19345747.2015.1086913) http://dx.doi.org/10.1080/19345747.2015.1086913 1
III. An Introduction to Causality a. Many earlier texts including our textbook are murky on the definitions. b. Modern causal research is based on the idea of the counterfactual. c. For now we only need to understand the idea that, theoretically, if we consider, say, two treatment values (C for control, and A for active), then the outcome for subject i is either Y ic or Y ia. We observe one of these, and the other is counterfactual. d. The causal effect of treatment in the population is defined as mean(y.a) mean(y.c), where. Indicates all subjects. If we randomize treatment, then the corresponding sample means for the observed data are a good estimate of the population mean. e. With this definition, the causal effect of treatment matches the intuitive meaning of caused in a clear, non-ambiguous way. 2
IV. The Baron and Kenny Mediation approach a. This is a highly intuitive approach that is used less often recently. It only requires three separate ordinary regressions. b. Consider an experiment with randomly assigned treatment X and subsequent outcome Y as depicted at top. If we regress Y on X (or perform ANOVA with contrasts), we estimate the causal effect of X on Y as c (b X in regression 1). Assuming a two level treatment or a continuous treatment with a linear relationship to Y, we estimate that the mean change in Y when X goes up by one unit is c. If c is statistically significant, we make our first necessary statement: X causes Y (as a shortcut for changes in X cause changes in the mean of Y). What issues would you consider at this point? c. Now we consider meditation by trying to determine if X is affecting Y solely or partly by changing M, which then changes Y. We do a new experiment where X is assigned, M is measured sometime later, and Y is measured at an even later time. d. We can regress M on X, and call the coefficient for X as a (b X in regression 2). If a is statistically significant, we get our second necessary statement: X affects M. Note that because X is randomly assigned we can say that X causes M. 3
e. Finally we regress Y on X and M. Let s call the coefficient for M as b (b M in regression 3), and the coefficient for X as c-prime (b X in regression 3). i. If X works solely or partially through M (i.e., M is a mediator of the effect of X on Y), then the coefficient for M will be statistically significant, and we can say that M affects Y. ii. If X works solely through M, then we would expect c-prime to be zero, and its p-value will be not statistically significant (>0.05). This is because the p-value in a regression always tests the additional effect of any variable beyond the effects of all other variables in the model. We say that X has no direct effect on Y. (This is part of the original Baron and Kenny mediation formulation.) iii. The mediated effect of X on Y is the product a b. This makes sense: If X goes up by 1, then M goes up by a. Since Y goes up by b for each 1 unit increase in M, if M goes up by a, then Y goes up by a b. iv. If X has both direct effects on Y and indirect (unmediated) effects on Y, then cprime will be non-zero and will be a measure of the direct effect of X on Y. This is called partial mediation. v. Relationship of parameters in the two models: With complete mediation, we expect that a b = c. With partial mediation, we expect that a b + c = c. f. Taken together we get a logical approach to the concept of a mediated causal effect. i. If X does not affect Y, then there is no effect to study (nothing to be mediated). ii. If X does not affect M, then M can t be a mediator of X s effect on Y iii. If M does not affect Y, then M can t be a mediator of X s effect on Y iv. If X does affect Y, then M is not solely mediating the effect of X on Y. g. We can gain intuition by playing with http://shiny.stat.cmu.edu:3838/hseltman/xmy/. h. The Baron and Kenney approach has several problems, and is going out of favor. One is that it relies heavily on the retain vs. reject p-value results of several related tests. This is inefficient and more sensitive to type-1 and type-2 errors than other approaches. Another problem is excess sensitivity to model assumptions. A final problem with the original approach is that it focuses on complete mediation, when partial mediation is also common. 4
i. Here is the frequency of Baron and Kenney in Google books: j. In Google scholar, Baron Kenney has about 110,000 hits. Many of the more recent ones are reconsidering or beyond. Statistical mediation has over 700,000 hits. k. Bootstrap approaches to estimation of a, b, a b and c-prime are the modern standard. l. Some complications: i. Non-randomized treatment ii. Multiple mediators iii. Mediation with moderation iv. Mediation in longitudinal studies v. Mediation in hierarchical studies 5