A re-appraisal of fixed effect(s) meta-analysis

Transcription

1 A re-appraisal of fixed effect(s) meta-analysis Ken Rice, Julian Higgins & Thomas Lumley Universities of Washington, Bristol & Auckland

2 tl;dr Fixed-effectS meta-analysis answers a sensible question regardless of heterogeneity Other questions can also be sensible Fixed-effectS methods extend to useful measures of heterogeneity and meta-regression, small-sample corrections and Bayesian inference Knowing what question you re answering helps determine whether it s sensible has these slides and more. 1

3 Generic example Meta-analyzing trials to estimate some overall effect; Zinc Placebo Zinc Placebo Study N Mean SD N Mean SD better better Mean Difference, 95% CI Douglas [ 0.45, 9.25 ] Petrus [ 1.57, 0.17 ] Prasad [ 4.57, 2.63 ] Prasad [ 3.76, 2.48 ] Turner 2000b [ 1.88, 0.56 ] Turner 2000c [ 1.00, 1.70 ] Fixed effects, precision weighted average (PWA) estimator Random effects, DerSimonian Laird estimator 2.04 [ 2.45, 1.64 ] 1.21 [ 2.69, 0.28 ] Generic Q: Which average? Why? Mean Difference (days) * from Zinc for the Common Cold (2011) Cochrane review of zinc acetate lozenges for reducing duration of cold symptoms (days) 2

4 Fixed effect (singular)... based on the assumption that the results of each trial represents a statistical fluctuation around some common effect Steve Goodman Controlled Clinical Trials, 1989 In the fixed effect model for k studies we assume ˆβ i N(β i, σi 2 ), 1 i k, by the CLT, where β i = β 0, 1 i k and noise in σ i is negligible. Obvious (and optimal) estimate is the inverse variance-weighted or precision-weighted average: ˆβ F = k i=1 1 σ 2 i 1 σ 2 i ˆβ i, with Var[ ˆβ F ] = 1 1 σ 2 i. 3

5 Fixed effects (plural) But assuming β i exactly homogeneous is silly in (most) practice, as effects are not identical Environments & adherence differ (and much else) In my applied work, genetic ancestry differs But but but note that if ˆβ i N(β i, σi 2 ), 1 i k, by the CLT, and noise in σ i is negligible, then can still define ˆβ F = k i=1 1 σ 2 i 1 σ 2 i ˆβ i, with Var[ ˆβ F ] = 1 1 σ 2 i. The fixed effects estimate provides valid statistical inference on an average of the β i, regardless of their homogeneity/heterogeneity 4

6 Fixed effects: what average? First, consider possible data from three studies; Outcomes, Y Zn Pbo Study #1 Study #2 Study #3 Sample size n 1 Sample size n 2 Sample size n 3 Information n 1 /σ 1 2 Information n 2 /σ 2 2 Information n 3 /σ 3 2 β^1 Zn Pbo Study #1 Study #2 Study #3 Sample size n 1 Sample size n 2 Sample size n 3 Information n 1 /σ 1 2 Information n 2 /σ 2 2 Information n 3 /σ 3 2 β^2 Zn Pbo Study #1 Study #2 Study #3 Sample size n 1 Sample size n 2 Sample size n 3 Information n 1 /σ 1 2 Information n 2 /σ 2 2 Information n 3 /σ 3 2 β^3 Each n i = 200 here. We assume all σ 2 i known, for simplicity. 5

7 Fixed effects: what average? Population parameters those 3 studies are estimating; Density of outcomes, Y β 1 β 2 β 3 Zn Pbo Zn Pbo Zn Pbo Pop'n #1 Pop'n #2 Pop'n #3 Information φ 1 Information φ 2 Information φ 3 Parameters are differences in means (β i ) and information per observation (φ i ). 6

8 Fixed effects: what average? One overall population we might learn about; Density of outcomes, Y β combine Zn Combining #1 and #2 and #3 Pbo β combine is the mean difference (zinc vs placebo) with each subpopulation represented equally, i.e. weighted 1/1/1. 7

9 Fixed effects: what average? Another overall population we might learn about; Density of outcomes, Y β 1 β 2 β 3 Zn Pbo Zn Pbo Zn Pbo Proportion η 1 Proportion η 2 Proportion η 3 Information φ 1 Information φ 2 Information φ 3 Weights here are 2/7/1, not 1/1/1 as before. 8

10 Fixed effects: what average? Another overall population we might learn about; Density of outcomes, Y β combine Zn Pbo Combining #1 and #2 and #3, in proportions η 1,η 2,η 3 Still an average effect, but closer to β 2 than before. 8

11 Fixed effects: what average? And another; (obviously, there are unlimited possibilities) Density of outcomes, Y β 1 β 2 β 3 Zn Pbo Zn Pbo Zn Pbo Proportion η 1 Proportion η 2 Proportion η 3 Information φ 1 Information φ 2 Information φ 3 Weights here are 7/1/2. 9

12 Weights here are 7/1/2 smaller average effect, closer to β 1 10 Fixed effects: what average? And another; (obviously, there are unlimited possibilities) Density of outcomes, Y β combine Zn Pbo Combining #1 and #2 and #3, in proportions η 1,η 2,η 3

13 Fixed effects: general case Upweighting studies which are larger and more informative about their corresponding β i, we can estimate population parameter β F = η i φ i β i η i φ i = 1 σi 2 β i, 1 σi 2 by ˆβ F = 1 σ 2 i 1 σ 2 i ˆβ i, with Var[ ˆβ F ] = 1 1 σ 2 i. ˆβ F is the precision-weighted average, a.k.a. inverse-variance weighted average a.k.a. fixed effects estimator note the plural! ˆβ F is consistent for average effect β F under regime where all n i in fixed proportion Homogeneity, or tests for heterogeneity are not required to use ˆβ F and its inference 11

14 Fixed effects: general case Homogeneity, or tests for heterogeneity are not required to use ˆβ F and its inference Users who have only seen the fixed effect (singular) motivation tend to view it as the only reason for ever using ˆβ F. This is specious... If I see you buy eggs, should I think the only reason is that you re making an omelet? 12

15 Fixed effects: general case The basic ideas here are not new: Same average-effect argument already supports e.g. the Mantel-Haenszel estimate Fixed effects arguments presented by e.g. Peto (1987), Fleiss (1993) and Hedges (various, e.g. Handbook of Research Synthesis), all noting the validity of β F and inference using ˆβ F under heterogeneity Also Can still motivate ˆβ F when σ i are estimated, though Var[ ˆβ F ] requires more care (Domínguez-Islas & Rice 2018) Can use them in Bayesian work, with exchangeable priors (Domínguez-Islas & Rice, under review) much less sensitive than default methods 13

16 But what about heterogeneity? We all know the flaw of averages ; Average effect β F answers one question This does not mean other questions aren t interesting! 14

17 But what about heterogeneity? A weighted variance of effects: ζ 2 = 1 η i φ i And an empirical estimate of it: k i=1 η i φ i (β i β F ) 2. ˆζ 2 = σ 2 i (ˆβ i ˆβ F ) 2 (k 1) σ 2 i = Q (k 1) σ 2 i where Q is Cochran s Q and I 2 = 1 (k 1)/Q (truncated at zero) are standard statistics for assessing homogeneity. (Weighted) standard deviation ζ measure on the β scale is easier to interpret than Q or I 2 Inference on ζ far more stable than mean of (hypothetical) random effects distributions 15

18 But what about heterogeneity? Meta-regression essentially weighted linear regression of the ˆβ i on known study-specific covariates x i also tells us about differences from zero, beyond the overall effect ˆβ F. Using extensions of the arguments for ˆβ F, the standard linear meta-regression slope estimate can be written ˆβ MR = w i (x i ˆx F ) 2 ˆβ i ˆβ F x i ˆx F ki =1 w i (x i ˆx F ) 2, where ˆx F = w i x i ki =1 w i and w i = 1 σ 2 i, which with no further assumptions estimates β MR = η i φ i (x i x F ) 2 β i β x i x F ki =1 η i φ i (x i x)2, where x F = η i φ i x i ki =1 η i φ i. Var[ ˆβ MR ] also available ANOVA/ANCOVA breakdowns of total signal:noise available to accompany ζ 2 and ˆβ MR analysis 16

19 Are you going to stop now? Summary, under standard conditions; Assumptions: Name: Common effect Fixed effects All β i = β 0 β i unrestricted Plausible? Rarely Often! ˆβ F estimates: Single β 0 Sensible average, β F Valid estimate? Yes Yes StdErr[ ˆβ F ] valid? Yes Yes Estimate heterogeneity? Makes no sense Yes, via ζ 2, Q, I 2 Meta regression? Makes no sense Yes, via ˆβ MR *... if we can ignore uncertainty about the σ 2 i 17

20 Acknowledgements Thanks to: Clara Thomas Julian Dominguez- Lumley Higgins Islas Reminder: for slides & more. 18