A comparison of fully Bayesian and two-stage imputation strategies for missing covariate data

Transcription

1 A comparison of fully Bayesian and two-stage imputation strategies for missing covariate data Alexina Mason, Sylvia Richardson and Nicky Best Department of Epidemiology and Biostatistics, Imperial College London, UK 5th Annual Bayesian Biostatistics Conference January 23-25,

2 Outline Motivation Simulations general setup non-hierarchical linear hierarchical linear v-shaped informative missingness Application

3 Missing covariate problems The common problem of how to analyse datasets with incomplete covariates can arise directly indirectly - missing covariate problems in disguise Motivating example: reframes problem of unmeasured confounding as a non standard missing data problem the primary data source misses important confounders information on these unmeasured confounders is available from a supplementary data source the two data sources are matched Combining datasets in this way can lead to extreme amounts of missing data

4 Motivating example: water disinfection by-products and risk of low birth weight Objective: estimate the association between trihalomethane concentrations and the risk of full term low birth weight (<2.5kg) Primary data: 8969 birth records between 2000 and 2001 from the Hospital Episode Statistics (HES) database linked to estimated trihalomethane water concentrations data on mother s age, baby gender and an index of deprivation but no data on maternal smoking and ethnicity Supplementary data: survey information from the Millennium Cohort Study (MCS) contains detailed information on smoking and ethnicity 824 cohort births matched to primary data Over 90% of smoking and ethnicity missing

5 Imputation One generally recommended approach to analysing data with incomplete covariates is 1. create multiple completed datasets by imputing the missing values (requires an imputation model) 2. analyse the completed datasets (requires an analysis model) However, this approach can take different forms, e.g. one-stage strategy: fit imputation and analysis models simultaneously two-stage strategy: create imputations first, then carry out analysis Motivating question: how should we advise a practitioner?

6 Multiple Imputation (MI) spectrum feedback from analysis model to imputation model cut averages over small number of draws series of univariate conditional distributions Fully Bayesian Model (FBM) Bayesian MI (feedforward only model) Standard MI with joint multivariate distribution Standard MI with chained equations (MICE) Increasing approximation to FBM FBM and MICE can be considered the extremes of the spectrum

7 FBM - pros Pros and cons of FBM and MICE theoretically sound coherent model estimation uncertainty fully propagated can add missingness model to explore informative missingness

8 FBM - pros Pros and cons of FBM and MICE theoretically sound coherent model estimation uncertainty fully propagated can add missingness model to explore informative missingness FBM - cons implementation can be challenging

9 FBM - pros Pros and cons of FBM and MICE theoretically sound coherent model estimation uncertainty fully propagated can add missingness model to explore informative missingness FBM - cons implementation can be challenging MICE - pros range of readily available packages speed

10 FBM - pros Pros and cons of FBM and MICE theoretically sound coherent model estimation uncertainty fully propagated can add missingness model to explore informative missingness FBM - cons implementation can be challenging MICE - pros range of readily available packages speed MICE - cons conditional distributions may not correspond to a joint distribution difficult to explore informative missingness

11 FBM - pros Pros and cons of FBM and MICE theoretically sound coherent model estimation uncertainty fully propagated can add missingness model to explore informative missingness FBM - cons implementation can be challenging MICE - pros range of readily available packages speed MICE - cons conditional distributions may not correspond to a joint distribution difficult to explore informative missingness But how do FBM and MICE actually perform in practice? We investigate with some simulations

12 General setup of simulations Generate 1000 simulated data sets with 2 correlated explanatory variables, x and u response, y, dependent on x and u missingness imposed on u dependent on y Each simulated dataset analysed by a series of models Performance of models assessed for coefficient for u, β u, (true value=-2) coefficient for x, β x, (true value=1) We report average estimate (average of the posterior means) bias (average estimate - true value) coverage rate (proportion of times true value is contained in the 95% interval) interval width (average width of 95% interval)

13 Simulation setup: model descriptions We run 5 types of models GOLD: correct analysis model run on complete datasets EXU: excludes u from analysis model CC: complete case analysis FBM: Fully Bayesian Model (analysis and imputation models) MICE: uses 20 imputations GOLD provides performance targets GOLD, EXU, CC, FBM all fitted using WinBUGS software MICE fitted using functions from mice package in R software FBM has several variants dependent on scenario All models have the correct analysis model

14 Non-hierarchical linear simulation Data generated all variables continuous no hierarchical structure 1000 individuals 90% missingness Note: with single covariate with missing values, the approximation of using chained equations disappears Results EXU: extreme bias and 0 coverage (β x only) CC: serious bias and very low coverage FBM and MICE: both correct most of the bias and achieve nominal coverage

15 Non-hierarchical linear simulation Data generated all variables continuous no hierarchical structure 1000 individuals 90% missingness Note: with single covariate with missing values, the approximation of using chained equations disappears Results EXU: extreme bias and 0 coverage (β x only) CC: serious bias and very low coverage FBM and MICE: both correct most of the bias and achieve nominal coverage Even with extreme levels of missingness, FBM and MICE have similar performance with non-complex data

16 Hierarchical linear simulation - description Data generated with hierarchical structure (individuals within clusters) 10 clusters, each with 100 individuals 50% missingness FBM models: 3 variants with different imputation models for u no hierarchical structure (no HS) random intercepts (HS: ri) random intercepts + random slopes on x (HS: ri+rs) MICE: no hierarchical structure in imputation model in theory could run variants with hierarchical structure but implementation difficulties

17 Hierarchical linear simulation - β u results average coverage interval bias estimate rate width GOLD CC FBM (no HS) FBM (HS: ri) FBM (HS: ri+rs) MICE (no HS)

18 Hierarchical linear simulation - β u results average coverage interval bias estimate rate width GOLD CC FBM (no HS) FBM (HS: ri) FBM (HS: ri+rs) MICE (no HS) If hierarchical structure ignored in imputation model FBM - slight bias and poor coverage

19 Hierarchical linear simulation - β u results average coverage interval bias estimate rate width GOLD CC FBM (no HS) FBM (HS: ri) FBM (HS: ri+rs) MICE (no HS) If hierarchical structure ignored in imputation model FBM - slight bias and poor coverage MICE - much worse (no feedback from structure in analysis model)

20 Hierarchical linear simulation - β u results average coverage interval bias estimate rate width GOLD CC FBM (no HS) FBM (HS: ri) FBM (HS: ri+rs) MICE (no HS) If hierarchical structure incorporated in imputation model bias corrected nominal coverage rate achieved

21 Hierarchical linear simulation - β x results average coverage interval bias estimate rate width GOLD EXU CC FBM (no HS) FBM (HS: ri) FBM (HS: ri+rs) MICE (no HS) Pattern of bias and coverage results similar to β u

22 V-shaped informative missingness - description Data generated with no hierarchical structure 100 individuals missingness imposed on u depends on y and u 50% missingness FBM models: 4 variants MAR: no model of covariate missingness MNAR: assumes linear shape (linear) MNAR: allows v-shape (v-shape) MNAR: allows v-shape + priors inform signs of slopes (v-shape+) MICE: MAR, no model of covariate missingness most implementations do not readily extend to MNAR ad hoc sensitivity analysis to MNAR possible by inflating or deflating imputations (van Buuren and Groothuis-Oudshoorn, 2011)

23 V-shaped informative missingness - β u results average coverage interval bias estimate rate width GOLD CC MAR: FBM MNAR: FBM (linear) MNAR: FBM (vshape) MNAR: FBM (vshape+) MAR: MICE

24 V-shaped informative missingness - β u results average coverage interval bias estimate rate width GOLD CC MAR: FBM MNAR: FBM (linear) MNAR: FBM (vshape) MNAR: FBM (vshape+) MAR: MICE MAR results in bias and slightly reduced coverage

25 V-shaped informative missingness - β u results average coverage interval bias estimate rate width GOLD CC MAR: FBM MNAR: FBM (linear) MNAR: FBM (vshape) MNAR: FBM (vshape+) MAR: MICE MAR results in bias and slightly reduced coverage improvements if allow MNAR, even if wrong form

26 V-shaped informative missingness - β u results average coverage interval bias estimate rate width GOLD CC MAR: FBM MNAR: FBM (linear) MNAR: FBM (vshape) MNAR: FBM (vshape+) MAR: MICE MAR results in bias and slightly reduced coverage improvements if allow MNAR, even if wrong form further improvements from correct form

27 V-shaped informative missingness - β u results average coverage interval bias estimate rate width GOLD CC MAR: FBM MNAR: FBM (linear) MNAR: FBM (vshape) MNAR: FBM (vshape+) MAR: MICE MAR results in bias and slightly reduced coverage improvements if allow MNAR, even if wrong form further improvements from correct form and even better with informative priors

28 V-shaped informative missingness - β x results average coverage interval bias estimate rate width GOLD EXU CC MAR: FBM MNAR: FBM (linear) MNAR: FBM (vshape) MNAR: FBM (vshape+) MAR: MICE

29 V-shaped informative missingness - β x results average coverage interval bias estimate rate width GOLD EXU CC MAR: FBM MNAR: FBM (linear) MNAR: FBM (vshape) MNAR: FBM (vshape+) MAR: MICE MAR results in modest bias (FBM and MICE)

30 V-shaped informative missingness - β x results average coverage interval bias estimate rate width GOLD EXU CC MAR: FBM MNAR: FBM (linear) MNAR: FBM (vshape) MNAR: FBM (vshape+) MAR: MICE MAR results in modest bias (FBM and MICE) wrong MNAR (linear) slightly worse than MAR

31 V-shaped informative missingness - β x results average coverage interval bias estimate rate width GOLD EXU CC MAR: FBM MNAR: FBM (linear) MNAR: FBM (vshape) MNAR: FBM (vshape+) MAR: MICE MAR results in modest bias (FBM and MICE) wrong MNAR (linear) slightly worse than MAR little gain in correct MNAR over MAR

32 hierarchical structure Summary of simulation results β u and β x - clear benefits from incorporating structure in FBM imputation model (unable to assess MICE) informative missingness β u - benefits from correct MNAR β x - no clear benefits from allowing MNAR FBM CIM AM MoCM AM = Analysis Model CIM = Covariate Imputation Model MoCM = Model of Covariate Missingness With FBM for informative missingness, 3 linked models fitted simultaneously

33 Application results CC (MCS only) FBM Bayesian MI Standard MI MICE odds ratio for smoking during pregnancy (u) EXU (HES only) CC (MCS only) FBM Bayesian MI Standard MI MICE odds ratio for Trihalomethanes > 60µg/L (x)

34 Practical advice and future work Future work: extend simulations to multiple covariates non-linear (glm) analysis models

35 Practical advice and future work Future work: extend simulations to multiple covariates non-linear (glm) analysis models Practical advice simple complex small dataset and few covariates with missingness FBM MICE FBM large dataset and/or many covariates with missingness MICE

36 Direction for missing data research feedback from analysis model to imputation model cut averages over small number of draws series of univariate conditional distributions Fully Bayesian Model (FBM) Bayesian MI (feedforward only model) Standard MI with joint multivariate distribution Standard MI with chained equations (MICE) Increasing approximation to FBM Where should our starting point be? FBM: improve computational efficiency, more case studies MICE: robust in simple setup, but more work needed for realistic situations

37 Further Information and Acknowledgements See BIAS web site ( Funding by ESRC: the BIAS project (PI N Best), based at Imperial College, London, is a node of the Economic and Social Research Council s National Centre for Research Methods (NCRM) Daniels, M. J. and Hogan, J. W. (2008). Missing Data In Longitudinal Studies: Strategies for Bayesian Modeling and Sensitivity Analysis. Chapman & Hall. Mason, A., Richardson, S., Plewis, I., and Best, N. (2012). Strategy for modelling non-random missing data mechanisms in observational studies using Bayesian methods. Journal of Official Statistics, to appear. Mason, A. J. (2009). Bayesian methods for modelling non-random missing data mechanisms in longitudinal studies. PhD thesis, Imperial College London, available at van Buuren, S. and Groothuis-Oudshoorn, K. (2011). mice: Multiple Imputation by Chained Equations in R. Journal of Statistical Software, 45, (3), 1 67.

38 Comparison of FBM and MICE FBM MICE Imputation Model Analysis Model Imputation Model Analysis Model 1 stage procedure fit Imputation and Analysis Models simultaneously imputation model uses joint distribution of all missing variables uses full posterior distribution of missing values 2 stage procedure 1. fit Imputation Model 2. fit Analysis Model imputation model based on a set of univariate conditional distributions uses small number of draws of missing values from their predictive distribution

39 Non-hierarchical linear simulation - results average coverage interval bias estimate rate width β x GOLD β x EXU β x CC β x FBM β x MICE β u GOLD β u CC β u FBM β u MICE

40 Hierarchical linear simulation - equations Generate full data set as follows: x c 0 u c MVN 0 α c 1 ( ) (( xi xc MVN u i u c, ), y i N(α c + x i 2u i, 1) ( )) c indicates cluster level data; i indicates individual level data Impose missingness such that u i is missing with probability p i logit(p i ) = y i

41 V-shaped informative missingness - equations Generate full data set as follows: ( ) (( x 0 MVN u 0 y N(1 + x 2u, 4 2 ) ) ( 1 0.5, )) Impose missingness such that u is missing with probability p logit(p) = u + 0.5y