Bayesian Nonparametric Meta-Analysis Model George Karabatsos University of Illinois-Chicago (UIC) Collaborators: Elizabeth Talbott, UIC. Stephen Walker, UT-Austin. August 9, 5, 4:5-4:45pm JSM 5 Meeting, Seattle Session on Multivariate Meta-Analysis 4: PM - 5:5 PM, CC-3 Organizer: Simina M. Boca, Georgetown University Chair: Valerie Langberg, Brown University Supported by NSF Research Grant SES-5637.
I. Aims of meta analysis Outline A. Meta analysis data framework. Examples of effect sizes. B. Aims of meta analysis. (Main aim: infer overall effect size from a universe of studies) C. Publication bias assessment. D. Conventional normal models for meta-analysis. E. Potential issues: Normality assumptions about errors (and random effects), when not supported by the effect size data, can cause misleading meta-analytic conclusions. II. Proposed solution: A Bayesian nonparametric (BNP) meta-analysis model, which allows the entire effect-size distribution (density) to vary flexibly as a function of covariates. A covariate-dependent infinite mixture model. III. Illustration of the BNP model on real meta-analytic data involving behavioral genetics research of antisocial behavior. IV. Conclusions. Free menu-based software for BNP analysis.
Meta Analysis Data Framework Data: D n = {y i,, x i } i=:n i n' ( n) studies provide data on n study reports y i of a common effect size variable (Y). Each effect size report y i has sample size n i, sampling variance, i and covariates x i = (, x i,, x ip ) describing study characteristics. 3
Meta Analysis Data Framework Examples of effect sizes: Effect-size Description Effect-size (y i ) Variance ( i ) Unbiased standardized mean difference, two independent groups (Hedges, 98). i i n i i n i i n i n i c n i n i n i n i y i n i n i c ; c 3 4 n i n i Fisher z transformation of the correlation i. log i i n i 3 Log odds ratio for two binary (-) variables. log n i/n i n i /n i n i n i n i n i More examples: E.g., see Cooper, Hedges, Valentine (9). 4
Aims of Meta Analysis Given a set of meta analytic data, D n = {y i,, x i } i=:n, infer the overall effect size, after accounting for the covariates x i and the observation weights /. Basic (regression) Parameters: = (,,, p ) T. Mean overall effect size: i i Publication bias (e.g., file drawer effect) may affect meta-analytic conclusions. Publication bias may be assessed from the data by / including ˆ as one of the p covariates in x. This provides regression analysis for the funnel plot (Egger et al., 997; Thompson & Sharp, 999). Significant regression slope coefficient for the covariate suggests a presence of publication bias. 5 ˆ /
Conventional Meta Analytic Models General f y i x i, i ; n yi x i i t i, i, i,,n; Normal x Model: i x i p x ip ;,, n n n, I n M n ; t n,, t,,t. Meta-analysis model: (,, p ) =. Meta-regression-analysis model: (,, p ) can be non-zero. Fixed effects model: = = =. -level random effects model: = =. 3-level random effects model: =. Model with correl. between study reports (Stevens-Ta.9). Each model can be fit by restricted ML (Harville,977) or by Bayesian MCMC methods (Spiegelhalter et.al. 9). 6
Conventional Meta Analytic Models General f y i x i, i ; n yi x i i t i, i, i,,n; Normal x Model: i x i p x ip ;,, n n n, I n M n ; t n,, t,,t. However, the normality assumptions (above) may not be exactly true for real meta-analytic data. This may cast doubt on the adequacy of the meta-analysis. Also, the normal model focuses inferences on how the mean effect size changes as a function of x, instead of how additional features of the effect size distribution (e.g., variance, quantiles, entire distribution/density, etc.) changes as a function of x. 7
Proposed BNP Meta-Analytic Model f y i x i, i ; n y i x i, i dgx j n y i j x i, i j x i,, i,,n, j x i, j x i / j x i / j n,, j,,, n,v k, k n, v k v k.5 k.5 k, k,,p ga a /,a / un,b, n p, 5 I p ga,. 8
Density f(y x) BNP Meta-Analytic Model (behavior) = / = / = = Mixture weight j.5.5.5.5 - Index j - Index j - Index j - Index j........ - y - y - y - y 9
Proposed BNP Meta-Analytic Model f y i x i, i ; n y i x i, i dgx n y i j x i, i j x i,, i,,n, j j x i, j x i / j x i / j n,, j,,, n,v k, k n, v k v k.5 k.5 k, k,,p ga a /,a / un,b, n p, 5 I p ga,. The posterior distribution of the model can be estimated by standard MCMC Gibbs sampling methods. Slice sampler for. See Karabatsos & Walker (, Appendix, Elec. J Stat.).
MZ-DZ Twin Comparison Sample Probability Density BNP Meta-Analysis Data Illustration 7 6 5 4 3 Heritability (Effect Size) Data 6 3 3 9 9 4 4 4 9 9 9 8 8 7 7 6 9 7 4 9 8 3 7 7 8 6 6 5 5 4 3 3 3 7 8 89 5 5 4.5 Heritability and Variance (+) 8 6 3 5 3 5 9 Heritability Distribution Over Studies 3.5 3.5.5.5 Mean=.5 Med=.5 Var= Skew=-. Kurt=8.3.5 Heritability Falconer (& Mackay 996) (antisocial) heritability: Sampling Variance: 4 MZ /n MZ DZ /n DZ h MZ DZ
BNP Meta-Analysis Data Illustration 4 covariates in data: Publication year; Square root heritability variance SE(ES) to assess for publication bias; Indicators (-) of female status versus male; Ten indicators of antisocial behavior ratings done by mother, father, teacher, self, independent observer, and ratings done on conduct disorder, aggression, delinquency, and externalizing antisocial behavior; Indicator of whether weighted aver of heritability measures was taken within study over different groups of raters who rated the same twins; Mean of the study subjects in months; Indicators of hi-majority ( 6%) white twins in study, zygosity obtained by questionnaire or through DNA samples, study inclusion of low socioeconomic (SES) status subjects versus mid-to-high SES subjects, missing SES information, representative sample, longitudinal sample; Latitude and longitude of study.
BNP Meta-Analysis Data Illustration Model D m Model D m BNP-ss. 6 DL-x 5. 5 DL- 4. 8 3L-x 5. 5 L-, by MZ-DZ 4. 8 L-, by Study 5. 8 3L- 4. 8 FE- 5. 9 DL-ss 5. 4 L-ss, by Study 6. L-ss, by MZ-DZ 5. 4 FE-ss 6. 3L-ss 5. 4 L-x, by Study 6. L-x, by MZ-DZ 5. 5 FE-x 6. Model comparisons: D(m) is posterior predictive mean-square error criterion (Laud & Ibrahim, 995). 3
Heritability Heritability BNP Meta-Analysis Data Illustration.8.8 Mom.8 Dad.6.6.6.4.4.4.....4.6.8 SE(ES) 5 5 5 5.8 Teacher.8 Self.8 Observer.6.6.6.4.4.4... 5 5 5 5 5 5 Posterior predictive median and IQR of heritability, as a function of rater type and child. Overall mean heritability ES estimate (of β₀) was.5. 4
Heritability Heritability BNP Meta-Analysis Data Illustration.8.8 Mom.8 Dad.6.6.6.4.4.4.....4.6.8 SE(ES) 5 5 5 5.8 Teacher.8 Self.8 Observer.6.6.6.4.4.4... 5 5 5 5 5 5 The SE(ES) covariate was not significant according to the spike-and-slab prior indicators. Thus, no significant publication bias in data. 5
Conclusions We proposed a useful and flexible BNP model for meta-analysis. Illustrated the model on real data. Better predictive utility vs. normal meta-analytic models. This leads to more reliable inferences. Provides richer meta-analysis compared to normal models. E.g., provides quantile regression of effect size. Article on BNP meta-analytic model: Karabatsos, G., Talbott, E., & Walker, S.G. (4). A Bayesian nonparametric meta-analysis model. Research Synthesis Methods, 6(), 8-44. 6
Conclusions Free User-friendly menu-driven software for BNP model (user point and click): http://www.uic.edu/~georgek/homep/bayessoftware.html Paper/software user s manual: http://arxiv.org/abs/56.5435 Software currently offers 83 Bayesian mixture models, including normal mixture models, several versions of the BNP infinite-mixture model of this talk, as well as other BNP infinite-mixture models, with mixture distribution assigned a prior defined by either the Dirichlet process (Ferguson 973); Pitman-Yor (997) process, the normalized stable process (Kingman, 975); geometric weights process (Fuentes-Garcia, Mena, Walker ); or the normalized inverse-gaussian process (Lijoi et al. 5). 7
8